Python unittest: how to run only part of a test fi

Question

I have a test file that contains tests taking quite a lot of time (they send calculations to a cluster and wait for the result). All of these are in specific TestCase class.

Since they take time and furthermore are not likely to break, I'd want to be able to choose whether this subset of tests does or doesn't run (the best way would be with a command-line argument, ie "./tests.py --offline" or something like that), so I could run most of the tests often and quickly and the whole set once in a while, when I have time.

For now, I just use unittest.main() to start the tests.

Thanks.

Answer 1

To run only a single specific test you can use:

$ python -m unittest test_module.TestClass.test_method

More information here

Answer 2

Actually, one can pass the names of the test case as sys.argv and only those cases will be tested.

For instance, suppose you have

class TestAccount(unittest.TestCase):
    ...

class TestCustomer(unittest.TestCase):
    ...

class TestShipping(unittest.TestCase):
    ...

account = TestAccount
customer = TestCustomer
shipping = TestShipping

You can call

python test.py account

to have only account tests, or even

$ python test.py account customer

to have both cases tested

Answer 3

I tried @slott's answer:

if __name__ == "__main__":
    suite = eval(sys.argv[1])  # Be careful with this line!
    unittest.TextTestRunner().run(suite)

But that gave me the following error:

Traceback (most recent call last):
  File "functional_tests.py", line 178, in <module>
    unittest.TextTestRunner().run(suite)
  File "/usr/lib/python2.7/unittest/runner.py", line 151, in run
    test(result)
  File "/usr/lib/python2.7/unittest/case.py", line 188, in __init__
    testMethod = getattr(self, methodName)
TypeError: getattr(): attribute name must be string

The following worked for me:

if __name__ == "__main__":
    test_class = eval(sys.argv[1])
    suite = unittest.TestLoader().loadTestsFromTestCase(test_class)
    unittest.TextTestRunner().run(suite)

Answer 4

I'm doing this using a simple skipIf:

import os

SLOW_TESTS = int(os.getenv('SLOW_TESTS', '0'))

@unittest.skipIf(not SLOW_TESTS, "slow")
class CheckMyFeature(unittest.TestCase):
    def runTest(self):
        …

This way I need only decorate an already existing test case with this single line (no need to create test suites or similar, just that one os.getenv() call line in the beginning of my unit test file), and as a default this test gets skipped.

If I want to execute it despite being slow, I just call my script like this:

SLOW_TESTS=1 python -m unittest …

Answer 5

You have basically two ways to do it:

1. Define your own suite of tests for the class
2. Create mock classes of the cluster connection that will return actual data.

I am a strong proponent of he second approach; a unit test should test only a very unit of code, and not complex systems (like databases or clusters). But I understand that it is not always possible; sometimes, creating mock ups is simply too expensive, or the goal of the test is really in the complex system.

Back to option (1), you can proceed in this way:

suite = unittest.TestSuite()
suite.addTest(MyUnitTestClass('quickRunningTest'))
suite.addTest(MyUnitTestClass('otherTest'))

and then passing the suite to the test runner:

unittest.TextTestRunner().run(suite)

More information on the python documentation: http://docs.python.org/library/unittest.html#testsuite-objects

Answer 6

Or you can make use of the unittest.SkipTest() function. Example, add a skipOrRunTest method to your test class like this:

def skipOrRunTest(self,testType):
    #testsToRun = 'ALL'
    #testsToRun = 'testType1, testType2, testType3, testType4,...etc'
    #testsToRun = 'testType1'
    #testsToRun = 'testType2'
    #testsToRun = 'testType3'
    testsToRun = 'testType4'              
    if ((testsToRun == 'ALL') or (testType in testsToRun)):
        return True 
    else:
        print "SKIPPED TEST because:\n\t testSuite '" + testType  + "' NOT IN testsToRun['" + testsToRun + "']" 
        self.skipTest("skipppy!!!")

Then add a call to this skipOrRunTest method to the very beginning of each of your unit tests like this:

def testType4(self):
    self.skipOrRunTest('testType4')