continue running cucumber steps after a failure

Question

Is there any way to continue executing Cucumber Steps even when one of the steps fails. In my current setup when a step fails , cucumber skips remaining steps....I wonder if there is some way to twick cucumber runner setup..

I could comment out failing steps but its not practical when you dont know which step will fail...If i could continue with remaining step i would know complete set of failing Tests in one shot....rather than going in cycle over cycle...

Environment: Cucumber JVM , R , Java , Ibatis , Spring Framework, Maven

Answer 1

One way would be to catch all the assertion errors and decide in the last step whether to fail or pass the test case. In this case, you can tailor it, say, check at any step to see if there is more than n errors and fail the test, if so.

Here's what I have done:

1. initialize a StringBuffer for Errors in your @Before for the test cases
2. catch the Assertion Errors and add to the StringBuffer, so that, they do not get thrown and terminate the test case.

3. Check the StringBuffer to determine whether to fail the test case.

   StringBuffer verificationErrors;
   
   // Initialize your error SringBuffer here
   
   @Before
   public void initialize() {
       verificationErrors = new StringBuffer();
   }
   // The following is one of the steps in the test case where I need to assert     something
   
   @When("^the value is (\\d+)$")
   public void the_result_should_be(int arg1) {
       try  {
           assertEquals(arg1, 0);
       }
       catch(AssertionError ae) {
           verificationErrors.append("Value is incorrect- "+ae.getMessage());  
       }
   

Check the StringBuffer in @After or in the last step of test case to determine if you can pass it or fail it, as follows:

if (!(verificationErrors.size()==0)) {
   fail(verificationErrors.toString());
}

The only issue would be that, in the report, all the steps would look green but the test case looks failed. Then you might have to look through the Errors String to know which step(s) failed. You could add extra information to the String whenever there is an Assertion Error to help you locate the step easily.

Answer 2

It is not a good idea to continue executing steps after a step failure because a step failure can leave the World with an invariant violation. A better strategy is to increase the granularity of your scenarios. Instead of writing a single scenario with several "Then" statements, use a list of examples to separately test each postconditions. Sometimes a scenario outline and list of examples can consolidate similar stories. https://docs.cucumber.io/gherkin/reference/#scenario-outline

There is some discussion about adding a feature to tag certain steps to continue after failure. https://github.com/cucumber/cucumber/issues/79