Catching the TERM signal in Gevent based applications

daemon, gevent, python, signals

Solution

After I did some searching I found the solution. As mentioned here, Gevent's monkey patching does not patch Python's built-in `signal.signal` function. In order for my solution to work, the main greenlet must call `gevent.signal` instead of `signal.signal` in order to set the handler. Also, `gevent.signal` expects handler functions which accept no arguments, unlike `signal.signal` which expects handler functions which accept 2 arguments. Since I don't care about these arguments anyway, I changed my term handler to look like this:

def _term_handler(*_):
    raise TermSignal()

So that the same term handle can be used in Gevent application and in regular Python applications. Although the same handler can be used in both cases, it needs to be set by the appropriate function (`signal.signal` or `gevent.signal`)

Problem

I wrote a simple daemon in Python which is based on Gevent. The daemon needs to do some cleanup before it exists, so it is necessary that it can handle the TERM signal, preform its cleanup and exit gracefully. In a single threaded daemon which is not based on Gevent, I used the Python's signal module to set a signal handler for the TERM signal. The signal handler throws an user defined exception which is called `TermSignal`. The main thread of the daemon can just catch the TermSignal exception, do its cleanup and exit. This solution did not work as expected when I tried to implement it in a daemon based on Gevent. The daemon has a main greenlet that calls `joinall` on the workers greenlets. the `joinall` call is wrapped in try/except block which catches the `KeyboardInterrupt`, which allows the daemon to preform its cleanup when it runs without daemonizing. However, when I implemented the solution above and sent a TERM signal to the process, I could see in the console window that one of the worker greenlets raised the `TermSignal` exception instead of the main greenlet. This uncaught exception did not proagate to the main greenlet, despite the face that I called `joinall` with `raise_error` parameter set to `True`. The result was that one of the worker greenlets crashed due to the exception, but the daemon did not exit at all.

Original source