I am trying to incorporate a maven plugin cucumber-jvm-parallel-plugin into my Cucumber-JVM code and am running into some problems... I think I have configured my pom.xml
properly, but my cucumber features are still getting ran one by one instead of in parallel.
How do I configure my pom.xml to make my cucumber features run in parallel? I have some feature files that block execution and I don't want to have to wait for them to finish in order to start testing the other features.
I followed two tutorials, and can't find where I am going wrong, or if this is expected:
- https://opencredo.com/test-automation-concepts-parallel-test-execution/
- http://automationrhapsody.com/running-cucumber-tests-in-parallel/
Structure
Here's my program's structure:
├── features/
│ └── api/
│ ├── 006-Email.feature
│ └── 999-Login.feature
├── results/
│ └── api-json.json
└── src/
└── java/
├── pom.xml
├── src/
│ └── test/
│ └── java/
│ └── com/
│ └── mycompany/
│ └── commonapps/
│ └── queuemanager/
│ ├── Globals.java
│ ├── GmailHelper.java
│ ├── HttpDeleteWithBody.java
│ ├── JsonHelper.java
│ ├── RequestHelper.java
│ ├── RunApiTest.java
│ ├── ThreadedSteps.java
│ └── UserHelper.java
└── steps/
└── com/
└── mycompany/
└── commonapps/
└── queuemanager/
├── GivenSteps.java
├── Setup.java
├── ThenSteps.java
└── WhenSteps.java
Maven Pom Logic
The way I configured my pom is as follows:
I copy
../../features/api/
to my build path using Maven Resources Plugin<build> <resources> <resource> <directory>../../features/api/</directory> </resource> </resources> </build>
I tell cucumber-jvm-parallel-plugin that my
features
are now located attarget/classes/active/
, because I think../../features/api/
was copied totarget/
, sotarget/active/
should contain my.feature
files.. I think.<plugin> <groupId>com.github.temyers</groupId> <artifactId>cucumber-jvm-parallel-plugin</artifactId> ... <configuration> ... <featuresDirectory>${project.build.directory}/classes/active/</featuresDirectory> ... </configuration> </plugin>
Then, after I run
maven test
, all.feature
files are then compiled intotarget/test-classes/Parallel**IT.class
. One per feature file.So I tell maven-surefire-plugin to run all
<include>**/Parallel*IT.class</include>
files in parallel, which are the feature files created by cucumber-jvm-parallel-plugin.<plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-surefire-plugin</artifactId> <executions> <execution> <id>acceptance-test</id> <phase>integration-test</phase> <goals> <goal>test</goal> </goals> <configuration> <forkCount>5</forkCount> <reuseForks>true</reuseForks> <includes> <include>**/Parallel*IT.class</include> </includes> </configuration> </execution> </executions> </plugin>
Full pom.xml
And my pom.xml is:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.mycompany.commonapps</groupId>
<artifactId>e2e-server</artifactId>
<version>0.0.1-SNAPSHOT</version>
<dependencies>
...
</dependencies>
<build>
<!-- COPY RESOURCES TO TARGET/ -->
<resources>
<!-- copies features to target: "${project.build.directory}/classes/active/" -->
<resource>
<directory>../../features/api/</directory>
</resource>
</resources>
<!-- RUN EACH CUCUMBER FEATURE FILE AS A FORK -->
<plugins>
<plugin>
<groupId>com.github.temyers</groupId>
<artifactId>cucumber-jvm-parallel-plugin</artifactId>
<version>1.2.1</version>
<executions>
<execution>
<id>generateRunners</id>
<phase>generate-test-sources</phase>
<goals>
<goal>generateRunners</goal>
</goals>
<configuration>
<glue>com.mycompany.commonapps.queuemanager</glue>
<outputDirectory>${project.build.directory}/generated-test-sources</outputDirectory>
<featuresDirectory>${project.build.directory}/classes/active/</featuresDirectory>
<cucumberOutputDir>${project.build.directory}</cucumberOutputDir>
<format>json</format>
<!--<strict>true</strict>
<monochrome>true</monochrome>
<useTestNG>false</useTestNG>
<namingScheme>simple</namingScheme>-->
<parallelScheme>FEATURE</parallelScheme>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<!-- RUN ALL CLASSES CREATED BY cucumber-jvm-parallel-plugin IN PARALLEL -->
<executions>
<execution>
<id>acceptance-test</id>
<phase>integration-test</phase>
<goals>
<goal>test</goal>
</goals>
<configuration>
<forkCount>5</forkCount>
<reuseForks>true</reuseForks>
<includes>
<include>**/Parallel*IT.class</include>
</includes>
</configuration>
</execution>
</executions>
</plugin>
<!-- ADD ../../STEPS/ TO THE SOURCE -->
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>build-helper-maven-plugin</artifactId>
<version>1.7</version>
<executions>
<execution>
<id>add-source</id>
<phase>generate-sources</phase>
<goals>
<goal>add-source</goal>
</goals>
<configuration>
<sources>
<source>steps/</source>
<source>src/test/java/</source>
</sources>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>
Cucumber Runner
And my cucumber running class looks like this:
@RunWith(Cucumber.class)
@CucumberOptions(plugin={"pretty","json:../../results/api-json.json"}, glue={"com.mycompany.commonapps.queuemanager"}, features={"../../features/api/active"})
public class RunApiTest {
}
Output
When I run mvn test
, I get the following:
...
[INFO] ------------------------------------------------------------------------
[INFO] Building e2e Server Tests 0.0.1-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[WARNING] The artifact org.apache.commons:commons-io:jar:1.3.2 has been relocated to commons-io:commons-io:jar:1.3.2
[INFO]
[INFO] --- build-helper-maven-plugin:1.7:add-source (add-source) @ e2e--server ---
[INFO] Source directory: /media/ifc-dev-1/ESD-ISO/e2e-/src/java/steps added.
[INFO] Source directory: /media/ifc-dev-1/ESD-ISO/e2e-/src/java/src/test/java added.
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ e2e--server ---
[WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent!
[INFO] Copying 7 resources
[INFO] Copying 2 resources
[INFO] Copying 7 resources
[INFO]
[INFO] --- maven-compiler-plugin:3.2:compile (default-compile) @ e2e--server ---
[INFO] Changes detected - recompiling the module!
[WARNING] File encoding has not been set, using platform encoding UTF-8, i.e. build is platform dependent!
[INFO] Compiling 12 source files to /media/ifc-dev-1/ESD-ISO/e2e-/src/java/target/classes
[INFO] /media/ifc-dev-1/ESD-ISO/e2e-/src/java/src/test/java/com/mycompany/commonapps/queuemanager/GmailHelper.java: /media/ifc-dev-1/ESD-ISO/e2e-/src/java/src/test/java/com/mycompany/commonapps/queuemanager/GmailHelper.java uses unchecked or unsafe operations.
[INFO] /media/ifc-dev-1/ESD-ISO/e2e-/src/java/src/test/java/com/mycompany/commonapps/queuemanager/GmailHelper.java: Recompile with -Xlint:unchecked for details.
[INFO]
[INFO] --- cucumber-jvm-parallel-plugin:1.2.1:generateRunners (generateRunners) @ e2e--server ---
[INFO] Adding /media/ifc-dev-1/ESD-ISO/e2e-/src/java/target/generated-test-sources to test-compile source root
[INFO]
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ e2e--server ---
[WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent!
[INFO] skip non existing resourceDirectory /media/ifc-dev-1/ESD-ISO/e2e-/src/java/src/test/resources
[INFO]
[INFO] --- maven-compiler-plugin:3.2:testCompile (default-testCompile) @ e2e--server ---
[INFO] Changes detected - recompiling the module!
[WARNING] File encoding has not been set, using platform encoding UTF-8, i.e. build is platform dependent!
[INFO] Compiling 10 source files to /media/ifc-dev-1/ESD-ISO/e2e-/src/java/target/test-classes
[INFO] /media/ifc-dev-1/ESD-ISO/e2e-/src/java/src/test/java/com/mycompany/commonapps/queuemanager/GmailHelper.java: /media/ifc-dev-1/ESD-ISO/e2e-/src/java/src/test/java/com/mycompany/commonapps/queuemanager/GmailHelper.java uses unchecked or unsafe operations.
[INFO] /media/ifc-dev-1/ESD-ISO/e2e-/src/java/src/test/java/com/mycompany/commonapps/queuemanager/GmailHelper.java: Recompile with -Xlint:unchecked for details.
[INFO]
[INFO] --- maven-surefire-plugin:2.17:test (default-test) @ e2e--server ---
[INFO] Surefire report directory: /media/ifc-dev-1/ESD-ISO/e2e-/src/java/target/surefire-reports
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running com.mycompany.commonapps.queuemanager.RunApiTest
....
*Tests run one by one properly*
...
Tests run: 31, Failures: 2, Errors: 0, Skipped: 0
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 02:40 min
[INFO] Finished at: 2016-11-10T15:49:11-08:00
[INFO] Final Memory: 20M/261M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.17:test (default-test) on project e2e--server: There are test failures.
[ERROR]
[ERROR] Please refer to /media/ifc-dev-1/ESD-ISO/e2e-/src/java/target/surefire-reports for the individual test results.
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
Does anyone see what I did wrong?
More Debugging Info
- cucumber-jvm-parallel-plugin's
outputDirectory
is getting created correctly and contains a bunch of Parallel**IT.class files - cucumber-jvm-parallel-plugin's
featuresDirectory
is getting accessed correctly. - cucumber-jvm-parallel-plugin's
cucumberOutputDir
is NOT getting created at all after a build.
I finally got it to work! Here's what I had to do to make my tests run in parallel:
I move some fields in the
maven-surefire-plugin
execution
tag to theconfiguration
tag. I moved this section:maven-surefire-plugin
now looks like this:Since
cucumber-jvm-parallel-plugin
is now running the show, I had to remove my cucumber runner class. I commented it out like this:And that's it. Since
Parallel**IT.class
where getting created, it wasmaven-surefire-plugin
that was not working right.