Getting Python's unittest results in a tearDow

Question

Is it possible to get the results of a test (i.e. whether all assertions have passed) in a tearDown() method? I'm running Selenium scripts, and I'd like to do some reporting from inside tearDown(), however I don't know if this is possible.

Answer 1

If you are using Python2 you can use the method _resultForDoCleanups. This method return a TextTestResult object:

<unittest.runner.TextTestResult run=1 errors=0 failures=0>

You can use this object to check the result of your tests:

def tearDown(self):
    if self._resultForDoCleanups.failures:
        ...
    elif self._resultForDoCleanups.errors:
        ...
    else:
        #Success

If you are using Python3 you can use _outcomeForDoCleanups:

def tearDown(self):
    if not self._outcomeForDoCleanups.success:
        ...

Answer 2

Name of current test can be retrieved with unittest.TestCase.id() method. So in tearDown you can check self.id().

Example shows how to:

• find if current test has error or failure in errors or failures list
• print test id with PASS or FAIL or EXCEPTION

Tested example here works with @scoffey 's nice example.

def tearDown(self):
    result = "PASS"
    #### find and show result for current test
    # I did not find any nicer/neater way of comparing self.id() with test id stored in errors or failures lists :-7
    id = str(self.id()).split('.')[-1]
    # id() e.g. tup[0]:<__main__.MyTest testMethod=test_onePlusNoneIsNone>
    #           str(tup[0]):"test_onePlusOneEqualsThree (__main__.MyTest)"
    #           str(self.id()) = __main__.MyTest.test_onePlusNoneIsNone
    for tup in self.currentResult.failures:
        if str(tup[0]).startswith(id):
            print ' test %s failure:%s' % (self.id(), tup[1])
            ## DO TEST FAIL ACTION HERE
            result = "FAIL"
    for tup in self.currentResult.errors:
        if str(tup[0]).startswith(id):
            print ' test %s error:%s' % (self.id(), tup[1])
            ## DO TEST EXCEPTION ACTION HERE
            result = "EXCEPTION"

    print "Test:%s Result:%s" % (self.id(), result)

example of result:

python run_scripts/tut2.py 2>&1 
E test __main__.MyTest.test_onePlusNoneIsNone error:Traceback (most recent call last):
  File "run_scripts/tut2.py", line 80, in test_onePlusNoneIsNone
    self.assertTrue(1 + None is None) # raises TypeError
TypeError: unsupported operand type(s) for +: 'int' and 'NoneType'

Test:__main__.MyTest.test_onePlusNoneIsNone Result:EXCEPTION
F test __main__.MyTest.test_onePlusOneEqualsThree failure:Traceback (most recent call last):
  File "run_scripts/tut2.py", line 77, in test_onePlusOneEqualsThree
    self.assertTrue(1 + 1 == 3) # fails
AssertionError: False is not true

Test:__main__.MyTest.test_onePlusOneEqualsThree Result:FAIL
Test:__main__.MyTest.test_onePlusOneEqualsTwo Result:PASS
.
======================================================================
ERROR: test_onePlusNoneIsNone (__main__.MyTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "run_scripts/tut2.py", line 80, in test_onePlusNoneIsNone
    self.assertTrue(1 + None is None) # raises TypeError
TypeError: unsupported operand type(s) for +: 'int' and 'NoneType'

======================================================================
FAIL: test_onePlusOneEqualsThree (__main__.MyTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "run_scripts/tut2.py", line 77, in test_onePlusOneEqualsThree
     self.assertTrue(1 + 1 == 3) # fails
AssertionError: False is not true

----------------------------------------------------------------------
Ran 3 tests in 0.001s

FAILED (failures=1, errors=1)

Answer 3

Here's a solution for those of us who are uncomfortable using solutions that rely on unittest internals:

First, we create a decorator that will set a flag on the TestCase instance to determine whether or not the test case failed or passed:

import unittest
import functools

def _tag_error(func):
    """Decorates a unittest test function to add failure information to the TestCase."""

    @functools.wraps(func)
    def decorator(self, *args, **kwargs):
        """Add failure information to `self` when `func` raises an exception."""
        self.test_failed = False
        try:
            func(self, *args, **kwargs)
        except unittest.SkipTest:
            raise
        except Exception:  # pylint: disable=broad-except
            self.test_failed = True
            raise  # re-raise the error with the original traceback.

    return decorator

This decorator is actually pretty simple. It relies on the fact that unittest detects failed tests via Exceptions. As far as I'm aware, the only special exception that needs to be handled is unittest.SkipTest (which does not indicate a test failure). All other exceptions indicate test failures so we mark them as such when they bubble up to us.

We can now use this decorator directly:

class MyTest(unittest.TestCase):
    test_failed = False

    def tearDown(self):
        super(MyTest, self).tearDown()
        print(self.test_failed)

    @_tag_error
    def test_something(self):
        self.fail('Bummer')

---

It's going to get really annoying writing this decorator all the time. Is there a way we can simplify? Yes there is!^* We can write a metaclass to handle applying the decorator for us:

class _TestFailedMeta(type):
    """Metaclass to decorate test methods to append error information to the TestCase instance."""
    def __new__(cls, name, bases, dct):
        for name, prop in dct.items():
            # assume that TestLoader.testMethodPrefix hasn't been messed with -- otherwise, we're hosed.
            if name.startswith('test') and callable(prop):
                dct[name] = _tag_error(prop)

        return super(_TestFailedMeta, cls).__new__(cls, name, bases, dct)

Now we apply this to our base TestCase subclass and we're all set:

import six  # For python2.x/3.x compatibility

class BaseTestCase(six.with_metaclass(_TestFailedMeta, unittest.TestCase)):
    """Base class for all our other tests.

    We don't really need this, but it demonstrates that the
    metaclass gets applied to all subclasses too.
    """


class MyTest(BaseTestCase):

    def tearDown(self):
        super(MyTest, self).tearDown()
        print(self.test_failed)

    def test_something(self):
        self.fail('Bummer')

There are likely a number of cases that this doesn't handle properly. For example, it does not correctly detect failed subtests or expected failures. I'd be interested in other failure modes of this, so if you find a case that I'm not handling properly, let me know in the comments and I'll look into it.

---

^*If there wasn't an easier way, I wouldn't have made _tag_error a private function ;-)

Answer 4

Following on from amatellanes' answer, if you're on Python3.4, you can't use _outcomeForDoCleanups. Here's what I managed to hack together:

def _test_has_failed(self):
    for method, error in self._outcome.errors:
        if error:
            return True
    return False

yucky, but it seems to work.

Answer 5

Tested for python 3.7 - sample code for getting info of failing assertion but can give idea how to deal with errors:

def tearDown(self):
    if self._outcome.errors[1][1] and hasattr(self._outcome.errors[1][1][1], 'actual'):
        print(self._testMethodName)
        print(self._outcome.errors[1][1][1].actual)
        print(self._outcome.errors[1][1][1].expected)