How to print the full traceback without halting th

Question

I'm writing a program that parses 10 websites, locates data files, saves the files, and then parses them to make data that can be readily used in the NumPy library. There are tons of errors this file encounters through bad links, poorly formed XML, missing entries, and other things I've yet to categorize. I initially made this program to handle errors like this:

try:
    do_stuff()
except:
    pass

But now I want to log errors:

try:
    do_stuff()
except Exception, err:
    print Exception, err

Note this is printing to a log file for later review. This usually prints very useless data. What I want is to print the exact same lines printed when the error triggers without the try-except intercepting the exception, but I don't want it to halt my program since it is nested in a series of for loops that I would like to see to completion.

Answer 1

First, don't use prints for logging, there is stable, proven and well-thought stdlib's module to do that: logging. You definitely should use it instead.

Second, don't be tempted to do mess with unrelated tools when there is native and simple approach. Here is it:

log = logging.getLogger(__name__)

try:
    call_code_that_fails()
except MyError:
    log.exception('Any extra info you want to see in your logs')

That's it. You are done now.

Explanation for anyone who is interested in how things go under the hood

What log.exception doing is actually just call to log.error (that is, log event with level ERROR) and print traceback then.

Why it is better?

Well, here is some considerations:

• it is just right;
• it is straightforward;
• it is simple.

Why nobody shouldn't use treceback nor call logger with exc_info=True nor get him/her hands dirty with sys.exc_info?

Well, because! They are all exist for different purposes. For example, traceback.print_exc's output is a little bit different from tracebacks produced by interpreter itself. If you going to use it, you will confuse anyone who will be banging him head with your logs.

Passing exc_info=True to log calls is just inappropriate. But, it is useful when you catching recoverable errors and want to log them (using, e.g INFO level) with tracebacks as well, because log.exception produces logs of only one level - ERROR.

And you definitely should avoid messing with sys.exc_info as much as you can. It is just not public interface, it is internal one - you can use it if you definitely know what you are doing. It is not intended just for printing exceptions.

Answer 2

You want the traceback module. It will let you print stack dumps like Python normally does. In particular, the print_last function will print the last exception and a stack trace.

Answer 3

In addition to @Aaron Hall's answer, if you are logging, but don't want to use logging.exception() (since it logs at the ERROR level), you can use a lower level and pass exc_info=True. e.g.

try:
    do_something_that_might_error()
except Exception:
    logger.info('General exception noted.', exc_info=True)

Answer 4

How to print the full traceback without halting the program?

When you don't want to halt your program on an error, you need to handle that error with a try/except:

try:
    do_something_that_might_error()
except Exception as error:
    handle_the_error(error)

To extract the full traceback, we'll use the traceback module from the standard library:

import traceback

And to create a decently complicated stacktrace to demonstrate that we get the full stacktrace:

def raise_error():
    raise RuntimeError('something bad happened!')

def do_something_that_might_error():
    raise_error()

Printing

To print the full traceback, use the traceback.print_exc method:

try:
    do_something_that_might_error()
except Exception as error:
    traceback.print_exc()

Which prints:

Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
  File "<stdin>", line 2, in do_something_that_might_error
  File "<stdin>", line 2, in raise_error
RuntimeError: something bad happened!

Better than printing, logging:

However, a best practice is to have a logger set up for your module. It will know the name of the module and be able to change levels (among other attributes, such as handlers)

import logging
logging.basicConfig(level=logging.DEBUG)
logger = logging.getLogger(__name__)

In which case, you'll want the logger.exception function instead:

try:
    do_something_that_might_error()
except Exception as error:
    logger.exception(error)

Which logs:

ERROR:__main__:something bad happened!
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
  File "<stdin>", line 2, in do_something_that_might_error
  File "<stdin>", line 2, in raise_error
RuntimeError: something bad happened!

Or perhaps you just want the string, in which case, you'll want the traceback.format_exc function instead:

try:
    do_something_that_might_error()
except Exception as error:
    logger.debug(traceback.format_exc())

Which logs:

DEBUG:__main__:Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
  File "<stdin>", line 2, in do_something_that_might_error
  File "<stdin>", line 2, in raise_error
RuntimeError: something bad happened!

Conclusion

And for all three options, we see we get the same output as when we have an error:

>>> do_something_that_might_error()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 2, in do_something_that_might_error
  File "<stdin>", line 2, in raise_error
RuntimeError: something bad happened!