Test exception chaining and traceback output with

Question

How to test a 'multi traceback' using doctest? It seems that using several ELLIPSIS and <BLANKLINE> won't do the trick:

def myfunc():
    """

    >>> myfunc()
    Traceback (most recent call last):
     ...
    ValueError: this is
    <BLANKLINE>
    The above exception was the direct cause of the following exception:
    <BLANKLINE>
    Traceback (most recent call last):
     ...
    TypeError: it

    """
    try:
        raise ValueError('this is')
    except ValueError as err:
        raise TypeError('it') from err


import doctest
doctest.testmod(optionflags=doctest.REPORT_NDIFF|doctest.ELLIPSIS)

Result:

"test.py" 23L, 490C written
**********************************************************************
File "test.py", line 4, in __main__.myfunc
Failed example:
    myfunc()
Differences (ndiff with -expected +actual):
      Traceback (most recent call last):
    -  ...
    +   File "test.py", line 17, in myfunc
    +     raise ValueError('this is')
      ValueError: this is
      <BLANKLINE>
      The above exception was the direct cause of the following exception:
      <BLANKLINE>
      Traceback (most recent call last):
    -  ...
    +   File "/usr/lib/python3.7/doctest.py", line 1329, in __run
    +     compileflags, 1), test.globs)
    +   File "<doctest __main__.myfunc[0]>", line 1, in <module>
    +     myfunc()
    +   File "test.py", line 19, in myfunc
    +     raise TypeError('it') from err
      TypeError: it
**********************************************************************
1 items had failures:
   1 of   1 in __main__.myfunc
***Test Failed*** 1 failures.

But if I squash all, it will pass:

>>> myfunc()
Traceback (most recent call last):
 ...
TypeError: it

Answer 1

I'm afraid that is not possible to check "multi tracebacks" in that way.

The problem is that doctest ignores everything except the exception class and its message.

In your example, it will be just:

TypeError: it

If you are interested in how this works, check the doctest.py and search for

exc_msg = traceback.format_exception_only(*exception[:2])[-1]

That "exc_msg" will only contain the raised exception's details:

TypeError: it

Alternatives

If it is possible, you could change your test to not raise any exception but print the wanted message.

Another possibility could be use another "doctest engine" like byexample. It works in the same way that doctest does but it's more flexible (quick overview here).

If you have a lot of tests, you may want to try its compatibility mode with doctest to avoid rewriting everything.

For your example, this should be:

"""                                                                                                                            
>>> from your_module import myfunc                                                                                             
"""                                                                                                                            

def myfunc():                                                                                                                  
    """                                                                                                                        

    >>> myfunc()                                                                                                               
    Traceback (most recent call last):                                                                                         
     ...                                                                                                                       
    ValueError: this is                                                                                                        
    <BLANKLINE>                                                                                                                
    The above exception was the direct cause of the following exception:                                                       
    <BLANKLINE>                                                                                                                
    Traceback (most recent call last):                                                                                         
     ...                                                                                                                       
    TypeError: it                                                                                                              

    """                                                                                                                        
    try:                                                                                                                       
        raise ValueError('this is')                                                                                            
    except ValueError as err:                                                                                                  
        raise TypeError('it') from err

And to run it you do from the shell:

byexample -l python -o '+py-doctest -py-pretty-print +ELLIPSIS' your_module.py

Disclaimer: I'm the author of byexample. I'm a really fan of doctest but I know that has its limitations and checking exceptions is one of them (specially if you work in a dual Python 2.x / 3.x project).

For that reason I created byexample: It is really useful to me and I really hope that it would be useful to others.

Ask me any question that you have, here or in github