Is there a way to unit test against side effects?

Question

Any code can provide side effects. Most of the time, side effects can be a sign of bad design and/or need of refactorisation, but when unit testing I find it hard to test against. Consider the following example:

[Test]
public void TrimAll_Removes_All_Spaces()
{
    // Arrange
    var testSubject = "A    string  with     lots   of     space";
    var expectedResult = "Astringwithlotsofspace";

    // Act
    var result = testSubject.TrimAll();

    // Assert
    Assert.AreEqual(expectedResult, result);
}

that tests the following extension:

public static string TrimAll(this string str)
{
    PokeAround();

    return str.Replace(" ", "");
}

The test will pass, but there is no guard agains side effects. The effects of the call to PokeAround will go completely unnoticed.

Given that you don't know what PokeAround is - it could be anything! - how do you write a test that guards against it? Is it at all possible?

Clarification: There have been a couple of comments about the PokeAround as completely unknown being a very unlikely scenario, since we have the source when we write the test. The reason I asked this question, though, was to find a way to guard against side effects added later on. That is, when I write the test, I might have the exension method look like this:

public static string TrimAll(this string str)
{
    return str.Replace(" ", "");
}

The test passes, all is good. Then, a month later when I'm on vacation, a colleague add's the PokeAround call. I want the test I already wrote to fail because he did.

Answer 1

Given that you don't know what PokeAround is - it could be anything! - how do you write a test that guards against it? Is it at all possible?

This question is specious. The situation is unlikely to occur in the real world.

1. You always know what PokeAround is. It's unit testing. You have the source.

   If -- through some organizational evil -- you are prohibited from reading the source, you have an organizational problem, not a technical problem.

   If you don't know what PokeAround is, you have people who are being specifically evil and preventing success. They need new jobs. Or you do.

2. You must use Mocks for this PokeAround so you can observe the side-effects.

"guard against side effects added later on."

This is not an example of a mysterious piece of code. You still know what PokeAround is. You always know what PokeAround is.

This is why we do Regression Testing. http://en.wikipedia.org/wiki/Regression_testing

It's still unit testing.

• You still test PokeAround with a stand-alone unit test.

• And you test things that use PokeAround with a mock of PokeAround.

Answer 2

I don't program in this language, but the following would be one way to test whether the original string was modified:

[Test]
public void TrimAll_Removes_All_Spaces()
{
    // Arrange
    var testSubject = "A    string  with     lots   of     space";
    var testSubjectCopy = "A    string  with     lots   of     space";
    var expectedResult = "Astringwithlotsofspace";

    // Act
    var result = testSubject.TrimAll();

    // Assert
    Assert.AreEqual(expectedResult, result);

    // Check for side effects
    Assert.AreEqual(testSubjectCopy, testSubject);
}

Answer 3

Keep in mind that unit tests are only one tool in an arsenal of verifications and checks, some of which may catch your colleague's "PokeAround":

1. Unit tests
2. Code Inspections/Reviews
3. Design by Contract
4. Assertions
5. Static analysis tools like FindBugs and the Checker Framework (@NonNull, @Pure, and @ReadOnly all rock!)

others?

The test passes, all is good. Then, a month later when I'm on vacation, a colleague add's the PokeAround call. I want the test I already wrote to fail because he did.

What makes you think that your colleague wouldn't change the test as well?

Answer 4

You could look at it another way, that by writing a unit test for this code you are forcing yourself to acknowledge a problem in the code (side effects). You could then re-write / restructure the code in such a way that it is testable, possibly by moving PokeAround into it's own function, or just relocating it from the code you are trying to test, if it needs more input/state to be testable.