Python Unittests continue after failure [duplicate

Question

This question already has an answer here:

• Continuing in Python's unittest when an assertion fails 10 answers

What's the best way of continuing tests after failure in unittest?

#!/usr/env/bin python2.7

import unittest
from fib import fib


class FibTests(unittest.TestCase):

    def test_0(self):
        self.assertEqual(fib(0), 0)
        self.assertEqual(fib(1), 1)
        self.assertEqual(fib(2), 1)
        self.assertEqual(fib(5), 5)
        self.assertEqual(fib(10), 55)

    def test_1(self):
        self.assertEqual(fib(0), 1)

    def test_2(self):
        self.assertEqual(fib(1), 0)
        self.assertEqual(fib(5), 0)

if __name__ == '__main__':
    suite = unittest.TestLoader().loadTestsFromTestCase(FibTests)
    result = unittest.TextTestRunner(verbosity=2).run(suite)

Looking at test_2, it'll only tell me there's 1 failure instead of 2.

Additionally, how can I collect the results at the end stating:

test_0 -- 0 failures
test_1 -- 1 failures
test_2 -- 2 failures

Motivation:

I'm trying to create a testing game. People submit tests and if they fail others' programs they get points; each test failure is one point. What's the easiest way of providing this type of capability?

Answer 1

You have to separate each test case into different method. To make it easier to create new test / save typing time, you can auto-generate test case based on input like this

#!/usr/env/bin python2.7

import unittest
from fib import fib


class FibTests(unittest.TestCase):
    pass

def test_gen(expected, actual):
    def test_method(self):
        return self.assertEqual(expected, actual)
    return test_method

if __name__ == '__main__':
    cases = ((fib(1), 1), (fib(2), 1), (fib(5), 5))
    for index, case in enumerate(cases):
        test_name = 'test_{0}'.format(index)
        test = test_gen(case[1], case[0])
        setattr(FibTests, test_name, test)
    suite = unittest.TestLoader().loadTestsFromTestCase(FibTests)
    result = unittest.TextTestRunner(verbosity=2).run(suite)