Integration Test Coverage in SonarQube from JaCoCo

2020-06-01 08:00发布

We have a multi-module multi-language maven java project with coverage analysis with jacoco. The main part of the modules is backend (Java code) with a REST API and our webapp module contains the frontend (AngularJS) and the Integration-Tests in java. Our Jacoco-IT.exec file contains around 100Kb of data so we guess some Coverage data for the Integration tests could get collected. Nevertheless we cant see any coverage in SonarQube (using Version 5.0 and 4.5)

We run build the Project and run the integration tests with

mvn clean install

and analyse the data with

mvn sonar:sonar

The Project is configured as multi language, but we just analyse the java code yet (we used to have a monolanguage configuration but there was no difference in the results related to the coverage data)

Our parent POM:

<properties>
    <hibernate.version>4.2.2.Final</hibernate.version>
    <spring.version>3.2.3.RELEASE</spring.version>
    <resteasy.version>3.0.4.Final</resteasy.version>
    <cucumber.version>1.1.3</cucumber.version>
    <selenium-java.version>2.33.0</selenium-java.version>
    <!-- Sonar -->
    <sonar.java.coveragePlugin>jacoco</sonar.java.coveragePlugin>
    <sonar.dynamicAnalysis>reuseReports</sonar.dynamicAnalysis>
    <sonar.jacoco.itReportPath>${project.basedir}/../target/jacoco-it.exec</sonar.jacoco.itReportPath>
    <jacoco.dataFile>${project.basedir}/../target/jacoco-it.exec</jacoco.dataFile>
</properties>
<build>
   <plugins>
       <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-compiler-plugin</artifactId>
            <configuration>
                <source>1.7</source>
                <target>1.7</target>
                <debug>true</debug>
            </configuration>
        </plugin>
        <plugin>
            <groupId>org.jacoco</groupId>
            <artifactId>jacoco-maven-plugin</artifactId>

            <executions>
                <execution>
                    <id>default-prepare-agent</id>
                    <goals>
                        <goal>prepare-agent</goal>
                    </goals>
                </execution>
                <execution>
                    <id>default-prepare-agent-integration</id>
                    <goals>
                        <goal>prepare-agent-integration</goal>
                    </goals>
                </execution>
                <execution>
                    <id>default-report</id>
                    <goals>
                        <goal>report</goal>
                    </goals>
                </execution>
                <execution>
                    <id>default-report-integration</id>
                    <goals>
                        <goal>report-integration</goal>
                    </goals>
                </execution>
            </executions>
        </plugin>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>2.16</version>
        </plugin>
    </plugins>
    <pluginManagement>
            <plugin>
                <groupId>org.jacoco</groupId>
                <artifactId>jacoco-maven-plugin</artifactId>
                <version>0.7.2.201409121644</version>
            </plugin>
    </pluginMangement>
</build>

Our POM from the webapp with the Integration Tests:

<properties>
    <sonar.sources>src/main</sonar.sources>
</properties>
<build>
    <finalName>web</finalName>
    <plugins>
        <plugin>
        <groupId>org.mortbay.jetty</groupId>
        <artifactId>jetty-maven-plugin</artifactId>
        <version>8.1.7.v20120910</version>
        <configuration>...</configuration>
        <executions>...</executions>
    </plugin>
        <plugin>
            <groupId>org.jacoco</groupId>
            <artifactId>jacoco-maven-plugin</artifactId>
            <configuration>
                <!-- The destination file for the code coverage report has to be set to the same value
                in the parent pom and in each module pom. Then JaCoCo will add up information in
                the same report, so that, it will give the cross-module code coverage. -->
                <destFile>${project.basedir}/../target/jacoco-it.exec</destFile>
            </configuration>
        </plugin>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-failsafe-plugin</artifactId>
            <version>2.16</version>
            <executions>
                <execution>
                    <id>default</id>
                    <goals>
                        <goal>integration-test</goal>
                        <goal>verify</goal>
                    </goals>
                </execution>
            </executions>
        </plugin>
    </plugins>
</build>

As you can see in our parent POM we configured the itReportPath to be in the same directory for all Modules. So when Sonar analyses all modules it reads always the same coverage data and should align them with the debug information from the binaries of the current module. During the analyse phase of maven (mvn sonar:sonar) we get some messages where we are not sure if they are a problem:

for all java modules we get the folloing lines:

    [INFO] [17:30:42.992] Sensor SurefireSensor...
    [INFO] [17:30:42.992] parsing \git\airport-core\rest\target\surefire-reports
    [INFO] [17:30:43.009] Sensor SurefireSensor done: 17 ms
    [INFO] [17:30:43.009] Sensor JaCoCoItSensor...
    [INFO] [17:30:43.010] Analysing \git\airport-core\rest\..\target\jacoco-it.exec
    [INFO] [17:30:43.018] No information about coverage per test.
    [INFO] [17:30:43.018] Sensor JaCoCoItSensor done: 9 ms
    [INFO] [17:30:43.018] Sensor JaCoCoOverallSensor...
    [INFO] [17:30:43.027] Analysing \git\airport-core\rest\target\sonar\jacoco-overall.exec
    [INFO] [17:30:43.046] No information about coverage per test.
    [INFO] [17:30:43.046] Sensor JaCoCoOverallSensor done: 28 ms

Is "No information about coverage per test." a Problem? Ok, we dont know which test caused the coverage, but we should get a different value in SonarQube than 0%

and for our webapp module, which does not contain any java sources/classes except for the integration test sources itself, we get the following lines:

    [INFO] [17:30:48.370] Sensor JaCoCoItSensor...
    [INFO] [17:30:48.370] No JaCoCo analysis of project coverage can be done since there is no class files.
    [INFO] [17:30:48.370] Sensor JaCoCoItSensor done: 0 ms

This shouldn't be a problem, as there are no classes in this module. Is this observation correct?

To the Main part our configruation bases on the official SonarCube demo configuration (https://github.com/SonarSource/sonar-examples/tree/master/projects/languages/java/code-coverage/combined%20ut-it/combined-ut-it-multimodule-maven-jacoco), we just added the "prepare-agent-integration" goal. And of course changed it to a multi-language Project.

We found a lot of depricated information on the web (argLine settings and so on) which didn't change anything for us.

Links to the Documentation and Tutorials we used:

5条回答
看我几分像从前
2楼-- · 2020-06-01 08:20

Im using a similar setup and everything works perfectly with the following configuration:

However the maven-surefire-plugin was causing troubles initially. Thats why I had to add the @{argLine} in the argLine tag, since the jacoco-maven-plugin:prepare-agent goal is executed before the maven-surefire-plugin. (see https://stackoverflow.com/a/25774988)

Hope I could help

<plugins>
    <plugin>
        <groupId>org.sonarsource.scanner.maven</groupId>
        <artifactId>sonar-maven-plugin</artifactId>
        <version>3.4.0.905</version>
    </plugin>
    <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-surefire-plugin</artifactId>

        <executions>
            <execution>
                <phase>test</phase>
            </execution>
        </executions>
        <configuration>
            <!-- the @{argLine} is needed for proper jacoco execution -->
            <argLine>@{argLine} -Xmx256m</argLine>
        </configuration>
    </plugin>

    <plugin>
        <groupId>org.jacoco</groupId>
        <artifactId>jacoco-maven-plugin</artifactId>
        <version>0.7.9</version>
        <executions>
            <execution>
                <id>jacoco-initialize</id>
                <goals>
                    <goal>prepare-agent</goal>
                </goals>
            </execution>
            <!-- generate fancy html report -->
            <execution>
                <id>jacoco-site</id>
                <phase>package</phase>
                <goals>
                    <goal>report</goal>
                </goals>
            </execution>
        </executions>
    </plugin>
</plugins>
查看更多
Evening l夕情丶
3楼-- · 2020-06-01 08:20

Considering other answers I want do add one thing, which is important to show test coverage on Sonar:

  1. you have to build project before expecting coverage to be shown: mvn clean install
  2. you have to build without -DskipTests
  3. you have to keep right configurations for plugin which run tests (surefire, failsafe, ... plugin) - coverage will be shown considering only those tests which are included in plugin configurations by <includes> or similar configurations.
查看更多
家丑人穷心不美
4楼-- · 2020-06-01 08:28

I use JUnit and in my case the issue was because of having TestNG dependency in my pom.xml in addition to JUnit. After removing this unnecessary dependency, everything started to work as expected.

查看更多
Ridiculous、
5楼-- · 2020-06-01 08:37

After fighting very long time with Jacoco and Maven only this way it is working for me:

  1. Create profile in pom.xml

    <profile>
      <id>coverage-per-test</id>
      <build>
        <plugins>
          <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>2.19.1</version>
            <configuration>
             <!-- disable parallel execution so that JaCoCo listener can properly work -->
              <parallel>none</parallel>
              <perCoreThreadCount>false</perCoreThreadCount>
              <forkCount>1</forkCount>
              <properties>
                <property>
                  <name>listener</name>
                  <value>org.sonar.java.jacoco.JUnitListener</value>
                </property>
              </properties>
            </configuration>
          </plugin>
        </plugins>
      </build>
    
      <dependencies>
        <dependency>
          <groupId>org.sonarsource.java</groupId>
          <artifactId>sonar-jacoco-listeners</artifactId>
          <version>3.10</version>
          <scope>test</scope>
        </dependency>
      </dependencies>
    </profile>
    
  2. Update maven surefire plugin to latest version

  3. Execute commands:

    mvn clean org.jacoco:jacoco-maven-plugin:prepare-agent install -Pcoverage-per-test -Dmaven.test.failure.ignore=true
    
    mvn sonar:sonar
    

Similar as describe in github sonar example

查看更多
Root(大扎)
6楼-- · 2020-06-01 08:44

I had the same experience when I was configuring JaCoCo for Jersey. I ended up applying the bisection method in order to find out which modules are causing the code coverage for the whole project to drop to 0%.

If I remember it correctly, the biggest surprise was maven-shade-plugin which somehow corrupted the association between the collected coverage and the location of classes which caused Sonar to show 0% code coverage. I ended up disabling its execution by binding it to phase none in order to fix it (see this commit).

As an example, this is how to extend a pom.xml of a sub-module in order to have working sonar (assuming, sonar is executed with sonar maven profile).

<profile>
    <id>sonar</id>
    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <configuration>
                    <!-- disable parallel execution so that JaCoCo listener can properly work -->
                    <parallel>none</parallel>
                    <perCoreThreadCount>false</perCoreThreadCount>
                    <forkCount>1</forkCount>
                </configuration>
            </plugin>
            <plugin>
                <artifactId>maven-shade-plugin</artifactId>
                <executions>
                    <execution>
                        <id>shade-archive</id>
                        <phase>none</phase>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>
    <dependencies>
        <dependency>
            <groupId>org.ow2.asm</groupId>
            <artifactId>asm-debug-all</artifactId>
            <optional>false</optional>
        </dependency>
    </dependencies>
</profile>
查看更多
登录 后发表回答