Signal handlers and logging in Python

问题:

Documentation for logging module says that

If you are implementing asynchronous signal handlers using the signal module, you may not be able to use logging from within such handlers. This is because lock implementations in the threading module are not always re-entrant, and so cannot be invoked from such signal handlers.

This suggests that one should not make logging calls from the code invoked by the signal handler directly or indirectly. If you do once in a while program will be left is a state when only kill -9 helps.

Important question for me now is following. Can this locking problem also happen when other threads call logging methods at the time when main thread is processing a signal?

回答1:

Signal handlers need special handling in UNIX programming at all. Only a defined list of POSIX C functions are declared as reentrant and can be called within a POSIX signal handler. IEEE Std 1003.1 lists 118 reentrant UNIX functions you find at https://www.opengroup.org/ (login required).

But Python signals are asynchronous: The signal module have an clarification:

Although Python signal handlers are called asynchronously as far as the Python user is concerned, they can only occur between the “atomic” instructions of the Python interpreter. This means that signals arriving during long calculations implemented purely in C (such as regular expression matches on large bodies of text) may be delayed for an arbitrary amount of time.

In this case, signals in Python are postponed until the GIL is free.

Back to your question. No, as long you use re-entrant functions within signal processing function. If logging is used in threads only, there will be no problems.

回答2:

The GIL (Global Interpreter Lock) prevents any Python code from running at the same time, so technically the main thread can't process a signal while other threads are running. It may "appear" that way but there is one big mutex that only allows one python thread to run at a time.

The best I can guess, the problem with signals and threading is that a signal is normally caused by an interrupt which can happen at anytime. So I would imagine Python stops what it's doing and calls the handler. At this point a lock may already have been acquired and so if logging tries to lock again, you get a deadlock. In some implementations this works OK because the mutex can be locked multiple times (re-entrant) but others there is only one lock.

Hopefully someone else can back this up.