I am using JUnit 4.10 for running test suites, and I have implemented a "retry failed test" rule following Matthew Farwell's awesome notes in the How to Re-run failed JUnit tests immediately? post. I created a class "RetryTestRule" with the following code:
public class RetryTestRule implements TestRule {
private final int retryCount;
public RetryTestRule(int retryCount) {
this.retryCount = retryCount;
}
@Override
public Statement apply(Statement base, Description description) {
return statement(base, description);
}
private Statement statement(final Statement base, final Description description) {
return new Statement() {
@Override
public void evaluate() throws Throwable {
Throwable caughtThrowable = null;
// retry logic
for (int i = 0; i < retryCount; i++) {
try {
base.evaluate();
return;
} catch (Throwable t) {
caughtThrowable = t;
System.err.println(description.getDisplayName() + ": run " + (i + 1) + " failed");
}
}
System.err.println(description.getDisplayName() + ": Giving up after " + retryCount
+ " failures");
throw caughtThrowable;
}
};
}
}
When using this as a rule inside a test case it works perfectly, but it seems not optimal to use the @Rule notation in every test case of a suite instead of a single notation in the Suite definition, so after checking a bit I tried the new @ClassRule notation in my Suite class:
@RunWith(Suite.class)
@SuiteClasses({
UserRegistrationTest.class,
WebLoginTest.class
})
public class UserSuite {
@ClassRule
public static RetryTestRule retry = new RetryTestRule(2);
}
Problem is this does not work as expected: failed tests are not being retried. Does anyone have tried this and knows a solution? Help is much appreciated!