Gracefully shutdown UNIX-socket server on NodeJS running under Forever

forever, node.js, unix-socket

Solution

As other people have mentioned, you cannot do anything in response to SIGKILL, which is in general why `forever` (and everybody else) should not be using SIGKILL except in extreme circumstances. So the best you can do is clean up in another process.

I suggest you clean up on start. When you get `EADDRINUSE`, try to connect to the socket. If the socket connection succeeds, another server is running and so this instance should exit. If the connection fails then it is safe to unlink the socket file and create a new one.

var fs = require('fs');
var net = require('net');
var server = net.createServer(function(c) { //'connection' listener
    console.log('server connected');
    c.on('end', function() {
        console.log('server disconnected');
    });
    c.write('hello\r\n');
    c.pipe(c);
});

server.on('error', function (e) {
    if (e.code == 'EADDRINUSE') {
        var clientSocket = new net.Socket();
        clientSocket.on('error', function(e) { // handle error trying to talk to server
            if (e.code == 'ECONNREFUSED') {  // No other server listening
                fs.unlinkSync('/tmp/app-monitor.sock');
                server.listen('/tmp/app-monitor.sock', function() { //'listening' listener
                    console.log('server recovered');
                });
            }
        });
        clientSocket.connect({path: '/tmp/app-monitor.sock'}, function() { 
            console.log('Server running, giving up...');
            process.exit();
        });
    }
});

server.listen('/tmp/app-monitor.sock', function() { //'listening' listener
    console.log('server bound');
});

Problem

I have an NodeJS application which sets up a UNIX-socket to expose some interprocess communication channel (some kind of monitoring stuff). UNIX-socket file is placed in `os.tmpdir()` folder (i.e. `/tmp/app-monitor.sock`). ``` var net = require('net'); var server = net.createServer(...); server.listen('/tmp/app-monitor.sock', ...); ``` I use a signal handling (SIGINT, SITERM, etc...) to gracefully shutdown my server and remove a socket file. ``` function shutdown() { server.close(); // socket file is automatically removed here process.exit(); } process.on('SIGINT', shutdown); // and so on ``` My application is running with `forever start ...` to monitor it's lifecycle. I have a problem with `forever restartall` command. When forever doing `restartall` it's using a `SIGKILL` to terminate all child processes. `SIGKILL` can't be handled by a process so my app dies without any shutdown procedures. The problem is a socket file which is not removed when `SIGKILL` is used. After the child process is restarted, new server can't be started cause' a `listen` call will cause a `EADDRINUSE` error. I can't remove a existing socket file during an app startup cause' I don't know if it a real working socket or some traces of a previous unclean shutdown. So, the question is... What is the better way to handle such situation (SIGKILL and UNIX-socket server)?

Original source