Stacktrace for UserWarning

I see a warning like this in my logs:

py.warnings.__init__: WARNING .../bs4/__init__.py:219: UserWarning: "foo" 
  looks like a filename, not markup. You should probably open this file 
  and pass the filehandle into Beautiful Soup

This message does not help very much.

I would like to see the stacktrace where this happens.

Please don't look into the content of this warning. This question is not about Beautiful Soup :-)

An easy solution would be to modify the third party code (bs4/__init__.py at line 219) and add something like this:

import traceback
logger.warn('Exc at ...\n%s' % ''.join(traceback.format_stack()))

But I would like to avoid this. Reasons:

This is a warning from a production system. I don't want to change the source.
The next time a warning like this occurs, I would like to see the stacktrace immediately

Is there a flag or setting for python which I can change, to see not only one line, but the while stacktrace? I need the upper frames to debug this.

In this environment Python 2.7 gets used.

回答1:

You would need to do the following:

Create if USER_SITE does not exists: issue python -c "import site; site._script()", see USER_SITE variable contents

Place a file usercustomize.py in that directory with the following code:

import traceback
import warnings


_old_warn = warnings.warn
def warn(*args, **kwargs):
    tb = traceback.extract_stack()
    _old_warn(*args, **kwargs)
    print("".join(traceback.format_list(tb)[:-1]))
warnings.warn = warn

Credits to this answer for the code.

Run the code as usual. My test code:

import warnings

def f():
    warnings.warn("foz")

f()

Before the procedure:

$ python test_warn.py
test_warn.py:4: UserWarning: foz
  warnings.warn("foz")

After:

$ python test_warn.py
<USER_SITE_REDACTED>/usercustomize.py:6: UserWarning: foz
  _old_warn(*args, **kwargs)
  File "test_warn.py", line 6, in <module>
    f()
  File "test_warn.py", line 4, in f
    warnings.warn("foz")

回答2:

If I want to find the root of a warning I generally just promote Warnings to Exceptions.

In your case you could simply use warnings.simplefilter or warnings.filterwarnings.

For example:

import warnings

def func():
    warnings.warn('abc', UserWarning)
    
def func2():
    func()
    
# Here I promote all UserWarnings to exceptions, but you could also use "warnings.filterwarnings"
# to only promote warnings from a specified module or matching a specified message.
# You may need to check which is most useful/appropriate for you.
warnings.simplefilter("error", UserWarning)   # you might need to reset this later :)
func2()

which gives a complete traceback:

---------------------------------------------------------------------------
UserWarning                               Traceback (most recent call last)
<ipython-input-11-be791e1071e7> in <module>()
      8 
      9 warnings.simplefilter("error", UserWarning)
---> 10 func2()

<ipython-input-11-be791e1071e7> in func2()
      5 
      6 def func2():
----> 7     func()
      8 
      9 warnings.simplefilter("error", UserWarning)

<ipython-input-11-be791e1071e7> in func()
      2 
      3 def func():
----> 4     warnings.warn('abc', UserWarning)
      5 
      6 def func2():

UserWarning: abc

And if you want to debug this you could easily hook Pythons pdbs on the last encountered exception:

import pdb

pdb.pm()

resulting in:

> <ipython-input-11-be791e1071e7>(4)func()
-> warnings.warn('abc', UserWarning)
(Pdb) _________________

This starts a post-mortem analysis of the last encountered exception. Which should enable you to dig through the frames and inspect the variables, etc.

You also asked about a flag, and indeed there is a flag that enables "warning handling", the -W flag. It's very much like the warnings.filterwarnings function. For convenience I just copied the documentation of the -W flag here:

Warning control. Python’s warning machinery by default prints warning messages to sys.stderr. A typical warning message has the following form:
file:line: category: message
By default, each warning is printed once for each source line where it occurs. This option controls how often warnings are printed.

Multiple -W options may be given; when a warning matches more than one option, the action for the last matching option is performed. Invalid -W options are ignored (though, a warning message is printed about invalid options when the first warning is issued).

Starting from Python 2.7, DeprecationWarning and its descendants are ignored by default. The -Wd option can be used to re-enable them.

Warnings can also be controlled from within a Python program using the warnings module.

The simplest form of argument is one of the following action strings (or a unique abbreviation) by themselves:

ignore

Ignore all warnings.

default

Explicitly request the default behavior (printing each warning once per source line).

all

Print a warning each time it occurs (this may generate many messages if a warning is triggered repeatedly for the same source line, such as inside a loop).

module

Print each warning only the first time it occurs in each module.

once

Print each warning only the first time it occurs in the program.

error:

Raise an exception instead of printing a warning message.

The full form of argument is:
action:message:category:module:line
Here, action is as explained above but only applies to messages that match the remaining fields. Empty fields match all values; trailing empty fields may be omitted. The message field matches the start of the warning message printed; this match is case-insensitive. The category field matches the warning category. This must be a class name; the match tests whether the actual warning category of the message is a subclass of the specified warning category. The full class name must be given. The module field matches the (fully-qualified) module name; this match is case-sensitive. The line field matches the line number, where zero matches all line numbers and is thus equivalent to an omitted line number.