Cobertura coverage and the assert keyword

问题:

My line coverage for unit tests measured by Cobertura is suffering, because I have assert statements which are not covered in tests. Should I be testing assertions, and is there any way to get Cobertura to ignore them so they do not affect my test coverage?

回答1:

The line coverage of your Java assert statements should be simply covered by running your test suite with assertions enabled, i.e., giving -ea as argument to the jvm. If you do this, you'll see that cobertura easily reports 100% line coverage if the rest of your lines are covered as well.

Nevertheless, the assert lines will still be colored red, suggesting insufficient coverage. This is because your assertions are usually always true, so you never hit the false branch.

Thus, branch coverage in Cobertura is messed up by using assert, since assert lines will have 50% branch coverage, rendering the overall branch coverage percentage hard to interpret (or useless).

Clover has a nice feature to ignore assert when computing coverage. I haven't seen this feature in any open source Java coverage tool.

If you use assertions in a design-by-contract style, there is no need to add tests that make your Java assert statements fail. As a matter of fact, for many assertions (e.g., invariants, post-conditions) you cannot even create objects that would make them fail, so it is impossible to write such tests. What you can do, though, is use the invariants/postconditions to derive test cases exercising their boundaries -- see Robert Binder's invariant boundaries pattern. But this won't make your assertions fail.

Only if you have a very tricky pre-condition for a given method, you may want to consider writing a test aimed at making the pre-condition fail. But then re-thinking your pre-condition may be a better idea.

回答2:

There's a feature request (1959691) already logged for Cobertura to add the ability to ignore assert lines; feel free to watch it in case they implement and add a comment to the page if you think it's a good idea. It's a shame none of the opensource coverage tools support this yet, as I definitely think assert statements are a good idea, but testing them usually isn't practical/possible.

回答3:

Code coverage is a tool which allow you to improve your testing, it is not a some kind of proof for the validity of your tests. You get the value from it by aiming to having 100% code coverage, looking at what is not covered, and reflecting on how to improve your tests.

Finding that there are uncovered assert statements is just an example where you should call: "ok, no need to cover this". A similar examples for this is with tracing macros and debug code which adds hidden "if statements" which are never covered.

I guess what what you don't like about your answer is that you like to have a minimal code-coverage requirement for your code; I faced the same issue with some of my projects. If one must not lose the coverage data, one solution is to have "coverage build" where the problematic statements (assert, trace, etc.) are replaced with empty ones (say through macro or linker magic). But I believe this it is usually a better trade-off just to accept the non-perfect coverage.

Good luck.

回答4:

Presumably you're using assertions to verify pre-conditions or some other set of conditions. Consider whether your situation is similar to Argument Exceptions should be Unit Tested.

In any case, I expect you're ultimately trying to determine if you should test the negative-path branches of these statements.

Yes, by all means, test those. Your assertions are themselves logic about your assumptions, and it's a Good Thing to test that your guard statements prevent/protect the scenarios you think they do.