maven-failsafe-plugin Failures and BUILD SUCCESS?

Question

I want to use maven-failsafe-plugin to run some integration tests. If any test fails, I want Maven to fail the build and not BUILD SUCCESS.

Tests run: 103, Failures: 1, Errors: 0, Skipped: 26
[INFO] BUILD SUCCESS*

how can I configure it, that build not success is?

My failsafe plugin is configured as:

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-failsafe-plugin</artifactId>
    <version>${failsafe.version}</version>
    <configuration>
        <systemProperties>
            <CI_INTEGRATION_OVERRIDE_PATH>${basedir}/..</CI_INTEGRATION_OVERRIDE_PATH>
        </systemProperties>
        <includes>
            <include>**/integration/**/*.java</include>
        </includes>
        <excludes>
            <exclude>**/integration/**/*TestSuite.java</exclude>
        </excludes>
    </configuration>
    <executions>
        <execution>
            <id>integration-test</id>
            <goals>
                <goal>integration-test</goal>
                <goal>verify</goal>
            </goals>
        </execution>
    </executions>
</plugin>

Answer 1

solution.

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-surefire-plugin</artifactId>
  <configuration>
    <skip>true</skip>
  </configuration>
  <executions>
    <execution>
      <id>unit-test</id>
      <phase>test</phase>
      <goals>
        <goal>test</goal>
      </goals>
      <configuration>
        <skip>false</skip>
        <excludes>
          <exclude>**/*IntegrationTest.java</exclude>
        </excludes>
      </configuration>
      </execution>
      <execution>
        <id>integration-test</id>
        <phase>integration-test</phase>
        <goals>
          <goal>test</goal>
        </goals>
        <configuration>
          <skip>false</skip>
          <enableAssertions>false</enableAssertions>
          <includes>
            <include>**/*IntegrationTest.java</include>
          </includes>
          <systemPropertyVariables>
            <integration>${integration}</integration>
          </systemPropertyVariables>
        </configuration>
      </execution>
    </executions>
</plugin>

Answer 2

Since you are running mvn clean install both the integration-test and verify phases should be executing. According to the failsafe plugin docs the failsafe:integration-test and failsafe:verify goals are bound to those phases, so I don't believe the extra call to failsafe:integration-test is required.

That said however, I'm not sure I trust the failsafe plugin documentation. I answered a similar question for someone earlier this year. It turned out he had to explicitly bind each goal to the correct phase and then failsafe worked as expected. Might be worth a shot.

Answer 3

As Andrew pointed out, the correct solution is to use failsafe as intended. The integration-test goal is specifically designed not to fail the build. If you want to fail the build, call mvn verify or mvn failsafe:verify

For the verify goal to work, it needs to read the results of the integration test which by default are written to ${project.build.directory}/failsafe-reports/failsafe-summary.xml so make sure that is getting generated.

Also, you have to make sure you bind your maven-failsafe-plugin configuration to both the integration-test goal and the verify goal in the executions part.

Failure to add either of those will lead to maven succeeding the build instead of failing it when integration tests fail.