Unit testing Anti-patterns catalogue

Question

anti-pattern : there must be at least two key elements present to formally distinguish an actual anti-pattern from a simple bad habit, bad practice, or bad idea:

• Some repeated pattern of action, process or structure that initially appears to be beneficial, but ultimately produces more bad consequences than beneficial results, and
• A refactored solution that is clearly documented, proven in actual practice and repeatable.

Vote for the TDD anti-pattern that you have seen "in the wild" one time too many.
The blog post by James Carr and Related discussion on testdrivendevelopment yahoogroup

If you've found an 'unnamed' one.. post 'em too. One post per anti-pattern please to make the votes count for something.

My vested interest is to find the top-n subset so that I can discuss 'em in a lunchbox meet in the near future.

Answer 1

The Cuckoo -- Frank Carver
A unit test which sits in a test case with several others, and enjoys the same (potentially lengthy) setup process as the other tests in the test case, but then discards some or all of the artifacts from the setup and creates its own.
Advanced Symptom of : Inappropriately Shared Fixture

Answer 2

The Environmental Vandal

A 'unit' test which for various 'requirements' starts spilling out into its environment, using and setting environment variables / ports. Running two of these tests simultaneously will cause 'unavailable port' exceptions etc.

These tests will be intermittent, and leave developers saying things like 'just run it again'.

One solution Ive seen is to randomly select a port number to use. This reduces the possibility of a conflict, but clearly doesnt solve the problem. So if you can, always mock the code so that it doesn't actually allocate the unsharable resource.

Answer 3

Chain Gang

A couple of tests that must run in a certain order, i.e. one test changes the global state of the system (global variables, data in the database) and the next test(s) depends on it.

You often see this in database tests. Instead of doing a rollback in teardown(), tests commit their changes to the database. Another common cause is that changes to the global state aren't wrapped in try/finally blocks which clean up should the test fail.

Answer 4

The Flickering Test (Source : Romilly Cocking)

A test which just occasionally fails, not at specific times, and is generally due to race conditions within the test. Typically occurs when testing something that is asynchronous, such as JMS.

Possibly a super set to the 'Wait and See' anti-pattern and 'The Sleeper' anti-pattern.

The build failed, oh well, just run the build again. -- Anonymous Developer

Answer 5

Second Class Citizens - test code isn't as well refactored as production code, containing a lot of duplicated code, making it hard to maintain tests.

Answer 6

The Giant

A unit test that, although it is validly testing the object under test, can span thousands of lines and contain many many test cases. This can be an indicator that the system under tests is a God Object (James Carr's post).

A sure sign for this one is a test that spans more than a a few lines of code. Often, the test is so complicated that it starts to contain bugs of its own or flaky behavior.