I'm running 64-bit Python 2.7.3 on Win7 64-bit. I can reliably crash the Python interpreter by doing this:
>>> from scipy import stats
>>> import time
>>> time.sleep(3)
and pressing Control-C during the sleep. A KeyboardInterrupt is not raised; the interpreter crashes. The following is printed:
forrtl: error (200): program aborting due to control-C event
Image PC Routine Line Source
libifcoremd.dll 00000000045031F8 Unknown Unknown Unknown
libifcoremd.dll 00000000044FC789 Unknown Unknown Unknown
libifcoremd.dll 00000000044E8583 Unknown Unknown Unknown
libifcoremd.dll 000000000445725D Unknown Unknown Unknown
libifcoremd.dll 00000000044672A6 Unknown Unknown Unknown
kernel32.dll 0000000077B74AF3 Unknown Unknown Unknown
kernel32.dll 0000000077B3F56D Unknown Unknown Unknown
ntdll.dll 0000000077C73281 Unknown Unknown Unknown
This makes it impossible to interrupt long-running scipy calculations.
Googling for "forrtl" and the like, I see suggestions that this kind of problem is due to use of a Fortran library that overrides Ctrl-C handling. I don't see a bug on the Scipy trackerbut given that Scipy is a library for use with Python, I would consider this a bug. It breaks Python's handling of Ctrl-C. Is there any workaround for this?
Edit: Following @cgohlke's suggestion I tried to add my own handler after importing scipy. This question about a related issue shows that adding a signal handler doesn't work. I tried using the Windows API SetConsoleCtrlHandler function via pywin32:
from scipy import stats
import win32api
def doSaneThing(sig, func=None):
print "Here I am"
raise KeyboardInterrupt
win32api.SetConsoleCtrlHandler(doSaneThing, 1)
After this, hitting Ctrl-C prints "Here I am", but Python still crashes with the forrtl error. Sometimes I also get a message saying "ConsoleCtrlHandler function failed", which quickly disappears.
If I run this in IPython, I can see a normal Python KeyboardInterrupt traceback before the forrtl error. I also see a normal Python traceback followed by the forrtl error if I raise some other error instead of KeyboardInterrupt (e.g., ValueError):
ValueError Traceback (most recent call last)
<ipython-input-1-08defde66fcb> in doSaneThing(sig, func)
3 def doSaneThing(sig, func=None):
4 print "Here I am"
----> 5 raise ValueError
6 win32api.SetConsoleCtrlHandler(doSaneThing, 1)
ValueError:
forrtl: error (200): program aborting due to control-C event
[etc.]
It seems that whatever the underlying handler is doing, it's not just trapping Ctrl-C directly, but is reacting to the error condition (ValueError) and crashing itself. Is there any way to eliminate this?
Here's a variation on your posted solution that may work. Maybe there's a better way to solve this problem -- or maybe even avoid it all together by setting an environment variable that tells the DLL to skip installing a handler. Hopefully this helps until you find a better way.
Both the
time
module (lines 868-876) and_multiprocessing
module (lines 312-321) callSetConsoleCtrlHandler
. In the case of thetime
module, its console control handler sets a Windows event,hInterruptEvent
. For the main thread,time.sleep
waits on this event viaWaitForSingleObject(hInterruptEvent, ul_millis)
, whereul_millis
is the number of milliseconds to sleep unless interrupted by Ctrl+C. Since the handler that you've installed returnsTrue
, thetime
module's handler never gets called to sethInterruptEvent
, which meanssleep
cannot be interrupted.I tried using
imp.init_builtin('time')
to reinitialize thetime
module, but apparentlySetConsoleCtrlHandler
ignores the 2nd call. It seems the handler has to be removed and then reinserted. Unfortunately, thetime
module doesn't export a function for that. So, as a kludge, just make sure you import thetime
module after you install your handler. Since importingscipy
also importstime
, you need to pre-load libifcoremd.dll usingctypes
to get the handlers in the right order. Finally, add a call tothread.interrupt_main
to make sure Python'sSIGINT
handler gets called[1].For example:
[1]
interrupt_main
callsPyErr_SetInterrupt
. This tripsHandlers[SIGINT]
and callsPy_AddPendingCall
to addchecksignals_witharg
. In turn this callsPyErr_CheckSignals
. SinceHandlers[SIGINT]
is tripped, this callsHandlers[SIGINT].func
. Finally, iffunc
issignal.default_int_handler
, you'll get aKeyboardInterrupt
exception.I have been able to get a half-workaround by doing this:
Returning true in the handler stops the chain of handlers so that the meddling Fortran handler is no longer called. However, this workaround is only partial, for two reasons:
time.sleep(3)
and hit Ctrl-C, the sleep is immediately aborted and I get a KeyboardInterrupt. With the above workaround, the sleep is not aborted, and control returns to the prompt only after the sleep time is up.Nonetheless, this is still better than crashing the whole session. To me this raises the question of why SciPy (and any other Python libraries that rely on these Intel libraries) don't do this themselves.
I'm leaving this answer unaccepted in the hope that someone can provide a real solution or workaround. By "real" I mean that pressing Ctrl-C during a long-running SciPy calculation should work just like it does when SciPy is not loaded. (Note that this doesn't mean it has to work immediately. Non-SciPy calculations like plain Python
sum(xrange(100000000))
may not immediately abort on Ctrl-C, but at least when they do, they raise a KeyboardInterrupt.)Try
Setting the environment variable
FOR_DISABLE_CONSOLE_CTRL_HANDLER
to1
seems to fix the issue.EDIT: While Ctrl+C doesn't crash python anymore, it also fails to stop the current calculation.
Here's code to patch the dll to remove the call that installs the Ctrl-C handler:
EDIT: Here's how I added a patch for the x64. Run python.exe in the debugger, and set a breakpoint for
SetConsoleCtrlHandler
until you get to the call you want to patch out:We'll patch out the
lea
instruction with a relativejmp
(which is0xeb
followed by the number of bytes to jump)I don't know how the .dll file is mapped in this process, so I'll just search for
0d 00 ef ff ff
in the file with a hex editor. It is a unique hit, so we can calculate the location in the .dll to patch.Ok, I've patched the dll at
0x3fdd9
. Let's see what it looks like now:So now were are
jmp
ing over pushing the arguments on the stack and the function call. So its Ctrl-C handler will not be installed.Workaround: patch
SetControlCtrlHandler