Logging uncaught exceptions in Python

Question

How do you cause uncaught exceptions to output via the logging module rather than to stderr?

I realize the best way to do this would be:

try:
    raise Exception, 'Throwing a boring exception'
except Exception, e:
    logging.exception(e)

But my situation is such that it would be really nice if logging.exception(...) were invoked automatically whenever an exception isn't caught.

Answer 1

Why not:

import sys
import logging
import traceback

def log_except_hook(*exc_info):
    text = "".join(traceback.format_exception(*exc_info))
    logging.error("Unhandled exception: %s", text)

sys.excepthook = log_except_hook

None()

Here is the output with sys.excepthook as seen above:

$ python tb.py
ERROR:root:Unhandled exception: Traceback (most recent call last):
  File "tb.py", line 11, in <module>
    None()
TypeError: 'NoneType' object is not callable

Here is the output with the sys.excepthook commented out:

$ python tb.py
Traceback (most recent call last):
  File "tb.py", line 11, in <module>
    None()
TypeError: 'NoneType' object is not callable

The only difference is that the former has ERROR:root:Unhandled exception: at the beginning of the first line.

Answer 2

As Ned pointed out, sys.excepthook is invoked every time an exception is raised and uncaught. The practical implication of this is that in your code you can override the default behavior of sys.excepthook to do whatever you want (including using logging.exception).

As a straw man example:

>>> import sys
>>> def foo(exctype, value, tb):
...     print 'My Error Information'
...     print 'Type:', exctype
...     print 'Value:', value
...     print 'Traceback:', tb
... 

Override sys.excepthook:

>>> sys.excepthook = foo

Commit obvious syntax error (leave out the colon) and get back custom error information:

>>> def bar(a, b)
My Error Information
Type: <type 'exceptions.SyntaxError'>
Value: invalid syntax (<stdin>, line 1)
Traceback: None

For more information about sys.excepthook: http://docs.python.org/library/sys.html#sys.excepthook

Answer 3

To build on Jacinda's answer, but using a logger object:

def catchException(logger, typ, value, traceback):
    logger.critical("My Error Information")
    logger.critical("Type: %s" % typ)
    logger.critical("Value: %s" % value)
    logger.critical("Traceback: %s" % traceback)

# Use a partially applied function
func = lambda typ, value, traceback: catchException(logger, typ, value, traceback)
sys.excepthook = func

Answer 4

The method sys.excepthook will be invoked if an exception is uncaught: http://docs.python.org/library/sys.html#sys.excepthook

When an exception is raised and uncaught, the interpreter calls sys.excepthook with three arguments, the exception class, exception instance, and a traceback object. In an interactive session this happens just before control is returned to the prompt; in a Python program this happens just before the program exits. The handling of such top-level exceptions can be customized by assigning another three-argument function to sys.excepthook.

Answer 5

Wrap your app entry call in a try...except block so you'll be able to catch and log (and perhaps re-raise) all uncaught exceptions. E.g. instead of:

if __name__ == '__main__':
    main()

Do this:

if __name__ == '__main__':
    try:
        main()
    except Exception as e:
        logger.exception(e)
        raise

Answer 6

Although @gnu_lorien's answer gave me good starting point, my program crashes on first exception.

I came with a customised (and/or) improved solution, which silently logs Exceptions of functions that are decorated with @handle_error.

import logging

__author__ = 'ahmed'
logging.basicConfig(filename='error.log', level=logging.DEBUG)


def handle_exception(exc_type, exc_value, exc_traceback):
    import sys
    if issubclass(exc_type, KeyboardInterrupt):
        sys.__excepthook__(exc_type, exc_value, exc_traceback)
        return
    logging.critical(exc_value.message, exc_info=(exc_type, exc_value, exc_traceback))


def handle_error(func):
    import sys

    def __inner(*args, **kwargs):
        try:
            return func(*args, **kwargs)
        except Exception, e:
            exc_type, exc_value, exc_tb = sys.exc_info()
            handle_exception(exc_type, exc_value, exc_tb)
        finally:
            print(e.message)
    return __inner


@handle_error
def main():
    raise RuntimeError("RuntimeError")


if __name__ == "__main__":
    for _ in xrange(1, 20):
        main()