How can I have Jenkins fail a build only when the

Question

We've customized a product which includes their own phpunit test suite. In Jenkins, I have two jobs setup: the first runs our own test suite that covers our customizations, and the second job runs the existing core unit tests.

The core unit tests were not designed to be run on a customized version, so failures are expected. Out of the ~5000 tests, 81 fail. What I'd like to setup in Jenkins, is have the build marked as a failure only if the number of failed tests changes from the previous build.

I've looked at the Performance plugin but the documentation seems sparse and I'm trying to find something that matches our use case.

Any suggestions?

Answer 1

You should have a look at the plugin https://wiki.jenkins-ci.org/display/JENKINS/xUnit+Plugin

It handle a threasolding mechanism (I specified this requirement for the xunit plugin when my team developed it )

expect this helps.. But you want to associates the failure to a change .... Hum maybe more complex .. have to ask .. if such thing should be developped.