Fact - End-to-end tests are critical if you want to make sure your software works as it should. To be 100% sure that you covered every (or almost every) possible branch in your business code, it is worth to check what code has been invoked after your E2E suite finished successfully. The solution I am going to present may be much easier, if you are able to stand up your application locally. Sometimes though, it is not the case.
In this scenario, we are going to start two instances of some web application at the remote servers, with code invocation recording turned on. Next, we are going to run integration test pointing to them one after another with different parameters, to test different paths. After the tests are done, we are going to pull the data from these servers to localhost, merge it and transform it to HTML report.
We are going to test this simple REST controller:
@RestController public class ExampleController { @GetMapping(path = "/test") public int exampleMethod(@RequestParam int parameter) { if(parameter > 10) { return invokeThisBranch(parameter); } else { return invokeAnotherBranch(parameter); } } private int invokeThisBranch(int parameter) { System.out.println("Taking this branch.."); if(parameter < 10) { System.out.println("This cannot be tested.."); } return 3; } private int invokeAnotherBranch(int parameter) { System.out.println("Taking another branch.."); return 3; } }
To make things simple we will run two instances of the same application on our local machines (on different ports).
Before we do that we need to download jacoco agent from https://mvnrepository.com/artifact/org.jacoco/org.jacoco.agent. Remember that the version must match the version of your jacoco plugin in maven. This agent will be attached to JVM and record the code coverage. The problem is that you need to push it to your remote server manually (if you find a way to automate this clean way let me know :)).
If you are using Inttellij, add this to VM options in your Run Configuration (or add this to startup script on remote server):
-javaagent:<path-to-agent>/jacocoagent.jar=port=36320,destfile=jacoco-it.exec,output=tcpserver
By specyfying output as a tcpserver and providing a port, you are enabling further connection from maven plugin, which will download data file (jacoco-it.exec).
Our test suite contains one test, which will cause different methods to be invoked depending of the parameter value.
@RunWith(SpringRunner.class) public class TestCoverageExampleApplicationTests { private TestRestTemplate testRestTemplate = new TestRestTemplate(); @Test public void anotherBranchTest() { //given int appPort = Integer.parseInt(System.getProperty("app.port")); int parameter = Integer.parseInt(System.getProperty("parameter")); //when Integer result = testRestTemplate.getForObject( "http://localhost:" + appPort + "/test?parameter=" + parameter, Integer.class); //then assertThat(result).isEqualTo(3); } }
We want to make sure that even though the code coverage was spread among instances, we will merge it correctly to be able to see the whole picture. Let’s run the test - diferent paths for different instance:
mvn test -Dapp.port=8080 -Dparameter=5 mvn test -Dapp.port=8081 -Dparameter=15
Before we generate coverage report, we need to configure jacoco plugins in pom.xml file. Full version can be found at my github account:
<plugins> <plugin> <groupId>org.jacoco</groupId> <artifactId>jacoco-maven-plugin</artifactId> <version>${jacoco.version}</version> <executions> <execution> <id>pull-test-data</id> <phase>post-integration-test</phase> <goals> <goal>dump</goal> </goals> <configuration> <destFile>${project.build.directory}/jacoco-it-${app.host}:${app.port}.exec</destFile> <address>${app.host}</address> <port>${app.port}</port> <reset>false</reset> <skip>${skip.dump}</skip> </configuration> </execution> <execution> <id>merge-test-data</id> <goals> <goal>merge</goal> </goals> <configuration> <destFile>target/jacoco-it.exec</destFile> <skip>${skip.dump}</skip> <fileSets> <fileSet implementation="org.apache.maven.shared.model.fileset.FileSet"> <directory>target</directory> <includes> <include>*it*.exec</include> </includes> </fileSet> </fileSets> </configuration> </execution> </executions> <configuration> <append>true</append> </configuration> </plugin> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-antrun-plugin</artifactId> <version>${antrun-plugin.version}</version> <executions> <execution> <id>generate-report</id> <phase>post-integration-test</phase> <goals> <goal>run</goal> </goals> <configuration> <skip>${skip.int.tests.report}</skip> <target> <taskdef name="report" classname="org.jacoco.ant.ReportTask"> <classpath path="${basedir}/target/jacoco-jars/org.jacoco.ant.jar"/> </taskdef> <mkdir dir="${basedir}/target/coverage-report"/> <report> <executiondata> <fileset dir="${basedir}"> <include name="target/jacoco-it*.exec"/> </fileset> </executiondata> <structure name="jacoco-multi Coverage Project"> <group name="jacoco-multi"> <classfiles> <fileset dir="target/classes"/> </classfiles> <sourcefiles encoding="UTF-8"> <fileset dir="src/main/java"/> </sourcefiles> </group> </structure> <html destdir="${basedir}/target/coverage-report/html"/> <xml destfile="${basedir}/target/coverage-report/coverage-report.xml"/> <csv destfile="${basedir}/target/coverage-report/coverage-report.csv"/> </report> </target> </configuration> </execution> </executions> <dependencies> <dependency> <groupId>org.jacoco</groupId> <artifactId>org.jacoco.ant</artifactId> <version>${jacoco.ant.version}</version> </dependency> </dependencies> </plugin> </plugins>
Let’s create automated script for coverage report generation:
mvn jacoco:dump@pull-test-data -Dapp.host=localhost -Dapp.port=36320 -Dskip.dump=false mvn jacoco:dump@pull-test-data -Dapp.host=localhost -Dapp.port=36321 -Dskip.dump=false mvn jacoco:merge@merge-test-data -Dskip.dump=false mvn antrun:run@generate-report -Dskip.int.tests.report=false
Script will pull the data files from two remote servers (localhost in our case), merge them into one file, and then generate report from it.
Final report is a joy to analyze:
Our class:
As you can see even though different code was invoked on different instances, final report contains both paths.
Some real world problems you may encounter:
- If you use byte code manipulators like lombok or aspectj, jacoco won’t be able to find a source code that match the invoked lines, you can use auto-value or immutables instead of lombok for some use cases and spring-aop instead of aspectj
- If you write your tests in Spock and you want to upload your jacoco-it.exec files to Sonar to show the code coverage there, you have to make sure groovy’s expressive method names will be correctly transformed in failsafe report - you need to add org.sonar.java.jacoco.JUnitListener as a listener
Full code can be found on my github account.
Add a comment