Verifying End-to-end Test Code Coverage using Jacoco Agent

Fact - End-to-end tests are critical if you want to make sure your software works as it should. To be 100% sure that you covered every (or almost every) possible branch in your business code, it is worth to check what code has been invoked after your E2E suite finished successfully. The solution I am going to present may be much easier, if you are able to stand up your application locally. Sometimes though, it is not the case.

In this scenario, we are going to start two instances of some web application at the remote servers, with code invocation recording turned on. Next, we are going to run integration test pointing to them one after another with different parameters, to test different paths. After the tests are done, we are going to pull the data from these servers to localhost, merge it and transform it to HTML report.

We are going to test this simple REST controller:

@RestController
public class ExampleController {

    @GetMapping(path = "/test")
    public int exampleMethod(@RequestParam int parameter) {
        if(parameter > 10) {
            return invokeThisBranch(parameter);
        } else {
            return invokeAnotherBranch(parameter);
        }
    }

    private int  invokeThisBranch(int parameter) {
        System.out.println("Taking this branch..");
        if(parameter < 10) {
            System.out.println("This cannot be tested..");
        }
        return 3;
    }

    private int invokeAnotherBranch(int parameter) {
        System.out.println("Taking another branch..");
        return 3;
    }
}

To make things simple we will run two instances of the same application on our local machines (on different ports).

Before we do that we need to download jacoco agent from https://mvnrepository.com/artifact/org.jacoco/org.jacoco.agent. Remember that the version must match the version of your jacoco plugin in maven. This agent will be attached to JVM and record the code coverage. The problem is that you need to push it to your remote server manually (if you find a way to automate this clean way let me know :)).

If you are using Inttellij, add this to VM options in your Run Configuration (or add this to startup script on remote server):

-javaagent:<path-to-agent>/jacocoagent.jar=port=36320,destfile=jacoco-it.exec,output=tcpserver

By specyfying output as a tcpserver and providing a port, you are enabling further connection from maven plugin, which will download data file (jacoco-it.exec).

Our test suite contains one test, which will cause different methods to be invoked depending of the parameter value.

@RunWith(SpringRunner.class)
public class TestCoverageExampleApplicationTests {

    private TestRestTemplate testRestTemplate = new TestRestTemplate();

    @Test
    public void anotherBranchTest() {
        //given
        int appPort =  Integer.parseInt(System.getProperty("app.port"));
        int parameter = Integer.parseInt(System.getProperty("parameter"));

        //when
        Integer result = testRestTemplate.getForObject(
                "http://localhost:" + appPort + "/test?parameter=" + parameter, Integer.class);

        //then
        assertThat(result).isEqualTo(3);
    }

}

We want to make sure that even though the code coverage was spread among instances, we will merge it correctly to be able to see the whole picture. Let’s run the test - diferent paths for different instance:

mvn test -Dapp.port=8080 -Dparameter=5
mvn test -Dapp.port=8081 -Dparameter=15

Before we generate coverage report, we need to configure jacoco plugins in pom.xml file. Full version can be found at my github account:

<plugins>
    <plugin>
        <groupId>org.jacoco</groupId>
        <artifactId>jacoco-maven-plugin</artifactId>
        <version>${jacoco.version}</version>
        <executions>
            <execution>
                <id>pull-test-data</id>
                <phase>post-integration-test</phase>
                <goals>
                    <goal>dump</goal>
                </goals>
                <configuration>
                    <destFile>${project.build.directory}/jacoco-it-${app.host}:${app.port}.exec</destFile>
                    <address>${app.host}</address>
                    <port>${app.port}</port>
                    <reset>false</reset>
                    <skip>${skip.dump}</skip>
                </configuration>
            </execution>
            <execution>
                <id>merge-test-data</id>
                <goals>
                    <goal>merge</goal>
                </goals>
                <configuration>
                    <destFile>target/jacoco-it.exec</destFile>
                    <skip>${skip.dump}</skip>
                    <fileSets>
                        <fileSet implementation="org.apache.maven.shared.model.fileset.FileSet">
                            <directory>target</directory>
                            <includes>
                                <include>*it*.exec</include>
                            </includes>
                        </fileSet>
                    </fileSets>
                </configuration>
            </execution>
        </executions>
        <configuration>
            <append>true</append>
        </configuration>
    </plugin>
    <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-antrun-plugin</artifactId>
        <version>${antrun-plugin.version}</version>
        <executions>
            <execution>
                <id>generate-report</id>
                <phase>post-integration-test</phase>
                <goals>
                    <goal>run</goal>
                </goals>
                <configuration>
                    <skip>${skip.int.tests.report}</skip>
                    <target>
                        <taskdef name="report" classname="org.jacoco.ant.ReportTask">
                            <classpath path="${basedir}/target/jacoco-jars/org.jacoco.ant.jar"/>
                        </taskdef>
                        <mkdir dir="${basedir}/target/coverage-report"/>
                        <report>
                            <executiondata>
                                <fileset dir="${basedir}">
                                    <include name="target/jacoco-it*.exec"/>
                                </fileset>
                            </executiondata>
                            <structure name="jacoco-multi Coverage Project">
                                <group name="jacoco-multi">
                                    <classfiles>
                                        <fileset dir="target/classes"/>
                                    </classfiles>
                                    <sourcefiles encoding="UTF-8">
                                        <fileset dir="src/main/java"/>
                                    </sourcefiles>
                                </group>
                            </structure>
                            <html destdir="${basedir}/target/coverage-report/html"/>
                            <xml destfile="${basedir}/target/coverage-report/coverage-report.xml"/>
                            <csv destfile="${basedir}/target/coverage-report/coverage-report.csv"/>
                        </report>
                    </target>
                </configuration>
            </execution>
        </executions>
        <dependencies>
            <dependency>
                <groupId>org.jacoco</groupId>
                <artifactId>org.jacoco.ant</artifactId>
                <version>${jacoco.ant.version}</version>
            </dependency>
        </dependencies>
    </plugin>
</plugins>

Let’s create automated script for coverage report generation:

mvn jacoco:dump@pull-test-data -Dapp.host=localhost -Dapp.port=36320 -Dskip.dump=false
mvn jacoco:dump@pull-test-data -Dapp.host=localhost -Dapp.port=36321 -Dskip.dump=false
mvn jacoco:merge@merge-test-data -Dskip.dump=false
mvn antrun:run@generate-report -Dskip.int.tests.report=false

Script will pull the data files from two remote servers (localhost in our case), merge them into one file, and then generate report from it.

Final report is a joy to analyze:

Our class:

As you can see even though different code was invoked on different instances, final report contains both paths.

Some real world problems you may encounter:

If you use byte code manipulators like lombok or aspectj, jacoco won’t be able to find a source code that match the invoked lines, you can use auto-value or immutables instead of lombok for some use cases and spring-aop instead of aspectj
If you write your tests in Spock and you want to upload your jacoco-it.exec files to Sonar to show the code coverage there, you have to make sure groovy’s expressive method names will be correctly transformed in failsafe report - you need to add org.sonar.java.jacoco.JUnitListener as a listener

Full code can be found on my github account.

Posted 9th November 2017 by Anonymous

Labels: jacoco java testing

Add a comment

Jul

Lifting functions to work with monads in Java

Functional Programming in Java

Stream and Optional classes - added to Java 8 - allow you to have some fun with functional programming. The problem is Java still misses quite a lot to be taken as a serious FP language. Lambda notation and two monads (Optional and Stream) are just the tip of the iceberg. This leads to arising of libraries like vavr or functionaljava - both deriving from purely functional language Haskell.

One of the first things you need to get rid of when trying to be more functional, is the attempt to unwrap the monad too early. It usually involves using methods like Optional.get() or Stream.collect() where there is yet no need. Sometimes though, Java doesn't help with that, so let me give you some custom code for the rescue. This article will bring you closer to the concept of method lifting.

Calculations on Optionals

Suppose we have some nice API we would like to use to calculate numbers:

    public interface Math {

        int multiply(int a, int b);

        double divide(int a, int b);

        ..
    }

We would like to use it to do some calculations on numbers wrapped with Optional:

    public interface NumberProvider {

        Optional<Integer> getNumber();

    }

Let's say we want to write a method, which returns the result of the division of two numbers wrapped with Optional, or empty Optional if any one of them is empty (we skip the 0 divisor case here). It may look something like this:

public Optional<Double> divideFirstTwo(NumberProvider numberProvider, Math math) {
        Optional<Integer> first = numberProvider.getNumber();
        Optional<Integer> second = numberProvider.getNumber();
        if(first.isPresent() && second.isPresent()) {
            double result = math.divide(first.get(), second.get());
            return Optional.of(result);
        } else {
            return Optional.empty();
        }
    }

That's rather nasty. It involves a lot of code which the only purpose is to wrap and unwrap the Optional. Let's try to make it more functional:

  public Optional<Double> divideFirstTwo(NumberProvider numberProvider, Math math) {
        return numberProvider.getNumber()
                .flatMap(first -> numberProvider.getNumber()
                                         .map(second -> math.divide(first, second)));
    }

That's much better. It turns out that invoking flatMap on the first monad and map on the second one inside the lambda, can be extracted even further to the method called lift:

public interface Optionals {

    static <R, T, Z> BiFunction<Optional<T>, Optional<R>, Optional<Z>> lift(BiFunction<? super T, ? super R, ? extends Z> function) {
        return (left, right) -> left.flatMap(leftVal -> right.map(rightVal -> function.apply(leftVal, rightVal)));
    }
}

Lift is able to promote any function, which takes two arguments, to the function with the arguments and the result type wrapped with Optional. It actually adds Optional behavior to the function in such a way that if one of the arguments is empty, then the result will also be empty. If JDK extracted flatMap and map methods to some common interface, for example Monad, then we could write one lift function for every instance of Java monad (Stream, Optional, custom classes). Unfortunately, we need to do this copy pasting for every instance. The final code for divideFirstTwo becomes:

import static com.ps.functional.monad.optional.Optionals.lift;
...

    public Optional<Double> divideFirstTwo(NumberProvider numberProvider, Math math) {
        return lift(math::divide).apply(numberProvider.getNumber(), numberProvider.getNumber());
    }

Summary

I hope this article encouraged you to play with functional style in Java. JDK needs to be greatly improved for the language to be called functional in the future. Unfortunately, Java 9 doesn't promise any major improvements besides just a few additional methods. Source code can be found here.

Posted 20th July 2017 by Anonymous

Labels: functional java optional

Add a comment

Jul

Composing functions with java.util.function.Function.andThen() trick

In this article, I am going to present you a simple trick that will make using java.util.function.Function.andThen() more useful.

As an example I will use ExternalSystemGateway class, which job is to call external system along with serializing/mapping the messages:

class Request {...}

class Response {...}

class ExternalSystemRequest {...}

class ExternalSystemResponse {...}

public class ExternalSystemGateway {

    public Response invoke(Request request) {
        ExternalSystemRequest externalSystemRequest = mapRequest(request);
        String requestString = marshallRequest(externalSystemRequest);
        String responseString = sendAndReceive(requestString);
        ExternalSystemResponse externalSystemResponse = unmarshallResponse(responseString);
        Response response = mapResponse(externalSystemResponse);
        return response;
    }

    private ExternalSystemRequest mapRequest(Request request) {
        // ..
    }

    private String marshallRequest(ExternalSystemRequest externalSystemRequest) {
        // ..
    }

    private String sendAndReceive(String requestString) {
        //..
    }

    private ExternalSystemResponse unmarshallResponse(String responseString) {
        //..
    }

    private Response mapResponse(ExternalSystemResponse externalSystemResponse) {
        //..
    }

}

You can see that every line of the invoke method does some kind of action, which transforms some input type to another output type. Let's try to get rid of the variables, which are declared only to be used in the next line:

    public Response invoke(Request request) {
        return mapResponse(
                unmarshallResponse(
                        sendAndReceive(
                                marshallRequest(
                                        mapRequest(request)))));
    }

Variables are gone but this code looks weird. What is more you need to read it backwards to understand the flow. Let's try to compose invocations using java.util.function.Function.andThen():

public Response invoke(Request request) {
        return ((Function<Request, ExternalSystemRequest>) (this::mapRequest))
                .andThen(this::marshallRequest)
                .andThen(this::sendAndReceive)
                .andThen(this::unmarshallResponse)
                .andThen(this::mapResponse)
                .apply(request);

Better, code reads nicer now, but can we get rid of the (Function<Request, ExternalSystemRequest>) cast at the beginning? Typing just this::mapRequest is not representing java.util.function.Function type so the trick is to achieve this using this simple interface:

public interface FunctionUtils {

    static <T, R> Function<T, R> function(Function<T, R> function) {
        return function;
    }
}

Our invoke method now becomes:

import static com.ps.functional.util.FunctionUtils.function;
...

    public Response invoke(Request request) {
        return function(this::mapRequest)
                .andThen(this::marshallRequest)
                .andThen(this::sendAndReceive)
                .andThen(this::unmarshallResponse)
                .andThen(this::mapResponse)
                .apply(request);
    }

Method function is just casting any one argument method reference to a java.util.function.Function type. You can create the same method for two argument functions:

1
2
3

static <T, U, R> BiFunction<T, U, R> biFunction(BiFunction<T, U, R> biFunction) {
        return biFunction;
    }

You can now easily compose functions in a readable way, without a need to do any casting. Source code for FunctionUtils is available here.

Posted 10th July 2017 by Anonymous

Labels: functional java

View comments

Jul

Exception-free code using functional approach

Checked exceptions & Java 8

Defining custom exceptions (both checked and unchecked) is a common approach to handling errors in Java applications. It usually leads to creating a new class for every different type of error, marking methods with throws keyword or wrapping code with try-catch blocks. This can lead to the code which is hard to read since every block adds another level of complexity.

Lambdas in Java 8 started the boom for functional approach in writing the code. Developers are now more familiar with these concepts and seeing -> in the code has became daily thing. Unfortunately, functional programming doesn't play well with Java exceptions - especially checked ones.

The main drawback is that method cannot be matched to functional interfaces from java.util.function if it throws a checked exception. This makes developers create custom functional interfaces with throws clauses, duplicating the code from JDK or write nasty try-catch block in lambdas. You can use RuntimeException type to model your custom exceptions but then the information about possible failure is lost and handling it is not required by compiler anymore.

Result monad

Solution suggested by myself uses custom Result class which is similar to the Optional monad. Optional is a box for objects that may exist - Optional keeps a reference to that object - or not - Optional keeps null. Result on the other hand can be a Success - code was executed correctly and the value is ready - or Failure - some error occurred. Here is the fragment of Result interface:

public interface Result<T> {

    boolean isSuccess();

    T getResult();

    ErrorType getError();

    String getOptionalErrorMessage();

    ...
}

Success implementation:

public final class Success<T> implements Result<T> {

    private final T result;

    ...

    public boolean isSuccess() {
        return true;
    }

    @Override
    public ErrorType getError() {
        throw new NoSuchElementException("There is no error in Success");
    }

    @Override
    public String getOptionalErrorMessage() {
        throw new NoSuchElementException("There is no optional error message in Success");
    }

    @Override
    public T getResult() {
        return result;
    }
}

Failure implementation:

public final class Failure<T> implements Result<T> {

    private final ErrorType error;

    private final String optionalErrorMessage;

        ...

    @Override
    public boolean isSuccess() {
        return false;
    }

    @Override
    public ErrorType getError() {
        return error;
    }

    @Override
    public String getOptionalErrorMessage() {
        return optionalErrorMessage;
    }

    @Override
    public T getResult() {
        throw new NoSuchElementException("There is no result is Failure");
    }
}

ErrorType is a type which represents predefined error condition. It sometimes comes with the optional error message. We could actually store Exception type instead of ErrorType but custom class gives more flexibility. Similarly to Optional, Result has get* methods which allow to access the internal state. They shouldn't be used without isSuccess method because they could result in throwing NoSuchElementException. They are helpful for unit tests, when you just want the value inside to assert on it.

Nothing fun so far. Let's have a look at the rest of the Result interface:

public interface Result<T> {

     ...

    default <R> Result<R> map(Function<? super T, ? extends R> mapper) {
        return isSuccess() ?
                Success.of(mapper.apply(getResult())):
                (Failure<R>) this;
    }

    default <R> Result<R> flatMap(Function<? super T, Result<R>> mapper) {
        return isSuccess() ?
                mapper.apply(getResult())
                (Failure<R>) this;
    }

    default <R> R fold(Function<? super T, ? extends R> successFunction, Function<Failure<R>, ? extends R> failureFunction) {
        return isSuccess() ?
                successFunction.apply(getResult()) :
                failureFunction.apply((Failure<R>) this);
    }
}

More about these methods later. Now, let's put it to use. When using libraries that declare checked exceptions, we would like to map every successful method call to Success and every cached exception to Failure. So the code like this:

    public String getNameById(int id) throws DatabaseAccessException {
        try {
            return dbClient.getNameById(id);
        } catch (DatabaseAccessException e) {
            throw new CustomException(e);
        }
    }

will become:

    public Result<String> getNameById(int id) {
        try {
            return Success.of(dbClient.getNameById(id));
        } catch (DatabaseAccessException e) {
            return Failure.of(PREDEFINED_DB_ERROR, e.getMessage());
        }
    }

(There are existing solutions like scala's/vavyr's Try, which skip the try - catch block totally and operate on Exception type. I prefer to use my custom ErrorType but you are free to use different approach. By using ErrorType you will also save some time by not wrapping the exception in custom one, which uses costly Throwable.fillInStackTrace - article.)

Result usage in business code

Now suppose we would like to use it in our business code. Here is the interface from persistence layer interface modeled with the usage of Result:

public interface DbConnector {

    Result<String> getNameById(int id);

    Result<Integer> getAgeByName(String name);

}

Suppose we want to use it in our business code:

class BusinessClass {

    private final DbConnector dbConnector;

    ...

    public Result<Boolean> isAdult(int userId) {
        return dbConnector.getNameById(userId)
                .flatMap(userName -> dbConnector.getAgeByName(userName)) // or simpler: .flatMap(dbConnector::getAgeByName)
                .map(userAge -> userAge >= 18);
    }

}

Method isAdult pulls the user's name from database and then uses it to fetch the age of that person (weird API but that's just an example :)). Finally it maps the age of that person to the boolean which indicates whether this person is an adult or not. What is important here, is that there are two calls which can result in Failure, sequenced using flatMap. The final invocation of map cannot fail because the predicate 'userAge >=18' will not be applied if the result from flatMap is Failure. The same way, function passed to flatMap will not be applied if getNameById returns a Failure - this Failure will be passed down the invocation stream to the caller of isAduld. Both map and flatMap will only invoke the passed method, if the Result is Success. This way we can model the business flow, but what if we want to unwrap the value inside, for example to return it to the client?:

    Result<Boolean> isAduld = isAduld(userId);
    String response = isAduld.fold(
            userIsAdult -> "adult: " + userIsAdult,
            failure -> "error occurred: " + failure.getError().show());

Fold will reduce the Result to the single value using provided functions - first if it's Success or second if it's Failure. Remember not to use just get method instead.

Summing up

map - applies function to value inside if it's Success, otherwise does nothing

flatMap - applies function which returns Result to value inside if it's Success, otherwise does nothing

fold - applies success function to value inside if it's Success, otherwise applies failure function to the ErrorType - reduces Result to singe value

That's it. Source code can be found here: https://github.com/pszeliga/functional-java. Other methods from Result will be discussed in future posts.

Posted 5th July 2017 by Anonymous

Labels: exception functional java result

Stream[Article]

Verifying End-to-end Test Code Coverage using Jacoco Agent

Add a comment

Lifting functions to work with monads in Java

Functional Programming in Java

Calculations on Optionals

Summary

Add a comment

Composing functions with java.util.function.Function.andThen() trick

View comments

Exception-free code using functional approach

Checked exceptions & Java 8

Result monad

Result usage in business code

Summing up

View comments