unittest and metaclass: automatic test_* method ge

Question

As I create tests for a framework, I start noticing the following pattern:

class SomeTestCase(unittest.TestCase):

    def test_feat_true(self):
        _test_feat(self, True)

    def test_feat_false(self):
        _test_feat(self, False)

    def _test_feat(self, arg):
        pass    # test logic goes here

So I want to programmatically create test_feat_* methods for these type of test classes with a metaclass. In other words, for each private method with signature _test_{featname}(self, arg), I want two top-level, discoverable methods with the signatures test_{featname}_true(self) and test_{featname}_false(self) to be created.

I came up with something like:

#!/usr/bin/env python

import unittest


class TestMaker(type):

    def __new__(cls, name, bases, attrs):
        callables = dict([
            (meth_name, meth) for (meth_name, meth) in attrs.items() if
            meth_name.startswith('_test')
        ])

        for meth_name, meth in callables.items():
            assert callable(meth)
            _, _, testname = meth_name.partition('_test')

            # inject methods: test{testname}_{[false,true]}(self)
            for suffix, arg in (('false', False), ('true', True)):
                testable_name = 'test{0}{1}'.format(testname, suffix)
                attrs[testable_name] = lambda self: meth(self, arg)

        return type.__new__(cls, name, bases, attrs)


class TestCase(unittest.TestCase):

    __metaclass__ = TestMaker

    def _test_this(self, arg):
        print 'this: ' + str(arg)

    def _test_that(self, arg):
        print 'that: ' + str(arg)


if __name__ == '__main__':
    unittest.main()

I expect some output like:

this: False
this: True
that: False
that: True

But what I got is:

$ ./test_meta.py
that: True
.that: True
.that: True
.that: True
.
----------------------------------------------------------------------
Ran 4 tests in 0.000s

OK

It looks like there are some closure rules that I am missing. How do I get around this? Is there a better approach?

Thanks,

edit: Fixed. See: the snippet.

Answer 1

Rather than going the metaclass route I would look into using nose test generators for this sort of thing:

http://somethingaboutorange.com/mrl/projects/nose/1.0.0/writing_tests.html#test-generators

The downside of test generators is that they are a nose specific feature, so you need to introduce a dependency outside the stdlib. The upside is that I think they are easier to write and understand.

Answer 2

Indeed, it is a closure issue:

Change

attrs[testable_name] = lambda self: meth(self, arg)

to

attrs[testable_name] = lambda self,meth=meth,arg=arg: meth(self, arg)

By using a default value, arg inside the lambda is bound to the default value arg set during each iteration of the loop. Without the default value, arg takes on the last value of arg after all the iterations of the loop have completed. (And the same goes for meth).