What Makes a Good Unit Test? [closed]

Question

I'm sure most of you are writing lots of automated tests and that you also have run into some common pitfalls when unit testing.

My question is do you follow any rules of conduct for writing tests in order to avoid problems in the future? To be more specific: What are the properties of good unit tests or how do you write your tests?

Language agnostic suggestions are encouraged.

Answer 1

I covered these principles a while back in This MSDN Magazine article which I think is important for any developer to read.

The way I define "good" unit tests, is if they posses the following three properties:

• They are readable (naming, asserts, variables, length, complexity..)
• They are Maintainable (no logic, not over specified, state-based, refactored..)
• They are trust-worthy (test the right thing, isolated, not integration tests..)

Answer 2

Tests should be isolated. One test should not depend on another. Even further, a test should not rely on external systems. In other words, test your code, not the code your code depends on.You can test those interactions as part of your integration or functional tests.

Answer 3

Think about the 2 types of testing and treat them differently - functional testing and performance testing.

Use different inputs and metrics for each. You may need to use different software for each type of test.

Answer 4

Some properties of great unit tests:

• When a test fails, it should be immediately obvious where the problem lies. If you have to use the debugger to track down the problem, then your tests aren't granular enough. Having exactly one assertion per test helps here.

• When you refactor, no tests should fail.

• Tests should run so fast that you never hesitate to run them.

• All tests should pass always; no non-deterministic results.

• Unit tests should be well-factored, just like your production code.

@Alotor: If you're suggesting that a library should only have unit tests at its external API, I disagree. I want unit tests for each class, including classes that I don't expose to external callers. (However, if I feel the need to write tests for private methods, then I need to refactor.)

---

EDIT: There was a comment about duplication caused by "one assertion per test". Specifically, if you have some code to set up a scenario, and then want to make multiple assertions about it, but only have one assertion per test, you might duplication the setup across multiple tests.

I don't take that approach. Instead, I use test fixtures per scenario. Here's a rough example:

[TestFixture]
public class StackTests
{
    [TestFixture]
    public class EmptyTests
    {
        Stack<int> _stack;

        [TestSetup]
        public void TestSetup()
        {
            _stack = new Stack<int>();
        }

        [TestMethod]
        [ExpectedException (typeof(Exception))]
        public void PopFails()
        {
            _stack.Pop();
        }

        [TestMethod]
        public void IsEmpty()
        {
            Assert(_stack.IsEmpty());
        }
    }

    [TestFixture]
    public class PushedOneTests
    {
        Stack<int> _stack;

        [TestSetup]
        public void TestSetup()
        {
            _stack = new Stack<int>();
            _stack.Push(7);
        }

        // Tests for one item on the stack...
    }
}

Answer 5

• Unit Testing just tests the external API of your Unit, you shouldn't test internal behaviour.
• Each test of a TestCase should test one (and only one) method inside this API.
  • Aditional Test Cases should be included for failure cases.
• Test the coverage of your tests: Once a unit it's tested, the 100% of the lines inside this unit should had been executed.

Answer 6

Good tests need to be maintainable.

I haven't quite figured out how to do this for complex environments.

All the textbooks start to come unglued as your code base starts reaching into the hundreds of 1000's or millions of lines of code.

• Team interactions explode
• number of test cases explode
• interactions between components explodes.
• time to build all the unittests becomes a significant part of the build time
• an API change can ripple to hundreds of test cases. Even though the production code change was easy.
• the number of events required to sequence processes into the right state increases which in turn increases test execution time.

Good architecture can control some of interaction explosion, but inevitably as systems become more complex the automated testing system grows with it.

This is where you start having to deal with trade-offs:

• only test external API otherwise refactoring internals results in significant test case rework.
• setup and teardown of each test gets more complicated as an encapsulated subsystem retains more state.
• nightly compilation and automated test execution grows to hours.
• increased compilation and execution times means designers don't or won't run all the tests
• to reduce test execution times you consider sequencing tests to take reduce set up and teardown

You also need to decide:

where do you store test cases in your code base?

• how do you document your test cases?
• can test fixtures be re-used to save test case maintenance?
• what happens when a nightly test case execution fails? Who does the triage?
• How do you maintain the mock objects? If you have 20 modules all using their own flavor of a mock logging API, changing the API ripples quickly. Not only do the test cases change but the 20 mock objects change. Those 20 modules were written over several years by many different teams. Its a classic re-use problem.
• individuals and their teams understand the value of automated tests they just don't like how the other team is doing it. :-)

I could go on forever, but my point is that:

Tests need to be maintainable.