Mar

4

Paxos protocol simulation in Akka

I have recently started implementing different distributed system protocols to get some understanding how they work. I think that using Akka Actors to simulate hosts is a good choice because they are easy to set up. What is more you can kill actors on demand to test some failure scenarios.

First protocol I implemented is Paxos - probably the most important consensus protocol, which guarantees storng consistecy, full transactions, read/write failover and prevents from data loss. This comes with the cost of higher latency and lower throughput. In terms of CAP theorem, Paxos is used to make your system CP. I used this sweet article as a reference.

Implementation can be found on my github repository. Go to the Test.scala and run some simulations.

Posted 4th March 2018 by Anonymous

Labels: akka distributed paxos scala

0
Add a comment
Feb

20

Don't sleep when you test!
When I started working as a Java Developer, me and my teammate got a first task to repair all broken tests (great task for new starters!) in some old project. Replacing some old configuration and upgrading a few libraries helped making the tests status green but there was another problem. End-to-end test suite took a few hours to pass making it impossible to get the feedback for a new change quickly. Sometimes build run for a few hours and hanged so E2E tetst became excluded from the feedback loop at the end.

Most of the tests from that suite did the same thing:
- Invoke a few commands on the mainframe through HTTP interface
- Sleep for 30 seconds (probably just to be extra safe) for each command and check if the result was persisted to database by another system.
It turned out, that first try to reduce the sleep time to 15 seconds by my teammate, didn’t make any tests fail, so that was a nice first try. But what was the optimal wait time for each test?

After some time, we tried another approach. If the flow has a asynchronous nature then why not testing it asynchronously.

I introduced this simple class (or at least class which does the same):
public class Await {

private static final BiPredicate<Long, Long> notTimedOut =

(start, timeoutMillis) -> System.currentTimeMillis() - start < timeoutMillis;

public static void await(Supplier<AssertionResult> assertionVerifier, long timeoutMillis) {

final long startTime = System.currentTimeMillis();

Stream.generate(assertionVerifier)

.takeWhile(result -> notTimedOut.test(startTime, timeoutMillis) || fail(result))

.filter(AssertionResult::isOK)

.findFirst();

}

public static boolean fail(AssertionResult result) {

throw new RuntimeException("Assertion not passed after timeout, problem: " + result.optionalMessage);

}

}
Method await checks some assertion in a loop until either it won’t throw AssertionError or the timeout is passed. This assertion is usually some state we are waiting for that should be ready in the nearest future. If it is not ready after the timeout, the last message from the AssertionError will be thrown wrapped as a RuntimeException. This greatly reduces test time because if the result is ready earlier, we don’t have to spend unnecessary time in the sleep method.

To represent the assertion result we can use classes (minimal implementation):
enum Status { OK, ERROR }

class AssertionResult {

public static final AssertionResult OK = new AssertionResult(Status.OK, "");

public final Status status;

public final String optionalMessage;

private AssertionResult(Status status, String optionalMessage) {

this.status = status;

this.optionalMessage = optionalMessage;

}

public static AssertionResult error(String optionalMessage) {

return new AssertionResult(Status.ERROR, optionalMessage);

}

public boolean isOK() {

return status == Status.OK;

}

}
We also need to prepare our AssertionResult suppliers - wrappers which fetch the data and invoke assertions on the result. Usually it is possible to define generic supplier if the way we fetch the data is similar. For example we can write this builder method:
public static Supplier<AssertionResult> dbDataAssertion(Consumer<String> userAssertion, long userId) {

return () -> {

String usernameFromDB = db.getUserName(userId); //or any other more generic version

try {

userAssertion.accept(usernameFromDB);

return AssertionResult.OK;

} catch (AssertionError assertionError) {

return AssertionResult.error(assertionError.getMessage());

}

};

}
We are ready to replace Thread.sleep() with call to Await in our tests:
...

long timeoutMillis = 1000L;

long userId = 1234L;

...

//then

Await.await(

dbDataAssertion(userName -> assertEquals(4, userName.length()), userId),

timeoutMillis);
Conclusion

Don’t overuse Thread.sleep() in your test code. According to this Google research it is one of the main reasons of flaky tests. Asynchronous approach may complicate your tests but the time savings can be tremendous. If you are looking for some existing implementations there is a nice class in Spock which does the same.
Posted 20th February 2018 by Anonymous

Labels: java testing

0
Add a comment
Feb

10

5 Less Known Cool Intellij Features

In this article I am going to share some cool features I stumbled upon while coding with Intellij. These are not the most popular/productivity improving ones - for these you should watch this video.

1. Set debugger breakpoint by pattern

Problem: You want to check if the debugger steps into some code during the run. Usually you suspect where the flow will end up but sometimes finding the exact line could be hard.

Solution: Breakpoints in Debugger pane -> Click (+) -> Java Method Breakpoint -> Fill class/method pattern.

For example this setting will stop the debugger anytime the code enters any method from any class from com.fasterxml.jackson package.

2. Attaching custom log file to be followed after Spring Boot app startup.

Problem: You have your app log configured to write to the file after the startup. However, this makes it harder to read that log because now you have to open this file somehow instead of just checking the console output.

Solution: Run/Debug Configuration -> Your Spring Boot app -> Logs -> Click (+) -> Fill alias and Log File Location. This will open additional pane with file content after you run the configuration that is easy to work with.

3. Invoke any code one the screen when in debug mode

Problem: You want to check the result of invoking some code on the screen but you are to lazy to paste it to Evaluate Expression console.

Solution: When the program is paused in debug mode, hit the alt key and click any fragment you want to see the result.

4. Compare with clipboard

Problem: You would like to compare some text the same way Intellij’s VCS diff shows changes in files.

Solution: Copy text to the clipboard -> Select any other text -> Right Click -> Click Compare With Clipboard.

I think it is worth to have a key assigned to this one.

5. Shelve changes

Problem: I used to use Git stash command externally to save some changes I didn’t want to commit yet. Is there a way to do this from Intellij?

Solution: VCS -> Local Changes -> Select the files you want to stash -> Right Click -> Shelve changes.

You can move them back from the shelf by right clicking on the selection and choosing Unshelve.

Conclusion

I hope you have learned at least one thing that might be helpful at you work. Please share other cool but less know Intellij features in the comment section.

Posted 10th February 2018 by Anonymous

Labels: intellij

0
Add a comment

Simple Map Reduce concept implementation

I have recently pushed very simple Map Reduce concept implementation on my Github account (click). My idea was to focus on the concept and mock the rest. You can follow the code to understand how it works but I will enumerate most implementation details:

Text files are represented as Strings and stored in memory
Master is responsible for splitting the data and scheduling the map/reduce tasks to workers
Workers are represented as threads, they are run using CompletableFuture api
When map task is finished, the combine function will be run on the result
When Master will notice that all map tasks have finished, it will take all resulted distinct keys and pass them as an argument to reduce tasks
When reduce tasks are finished, result is printed and the executor is closed

It is easy to define new user programs, check example package.

In memory data:

"file1.txt" -> "This is the first file \ncontent. "
"file2.txt" -> "And this is the second file content. "
"file3.txt" -> "More text in \nthird file"
"file4.txt" -> "And some random text here"
"file5.txt" -> "Why not \none more"
"file6.txt" -> "Lululu tengo manzana"

Distributed Grep (search for "And"):

New map task for key: file5.txt
New map task for key: file4.txt
New map task for key: file3.txt
New map task for key: file6.txt
New map task for key: file1.txt
New map task for key: file2.txt
Combine task for: file2.txt
Combine task for: file4.txt
Writing: IntermediateResult{key=file2.txt, value=And this is the second file content. }
Writing: IntermediateResult{key=file4.txt, value=And some random text here}
New reduce task for: file4.txt
New reduce task for: file2.txt

Results: 
file4.txt - And some random text here
file2.txt - And this is the second file content.

Word count:

New map task for key: file5.txt
New map task for key: file4.txt
New map task for key: file3.txt
New map task for key: file6.txt
New map task for key: file1.txt
New map task for key: file2.txt
Combine task for: not
Combine task for: here
Combine task for: the
Combine task for: manzana
Combine task for: the
Combine task for: More
Writing: IntermediateResult{key=More, value=1}
Writing: IntermediateResult{key=the, value=1}
Writing: IntermediateResult{key=here, value=1}
Writing: IntermediateResult{key=the, value=1}
Writing: IntermediateResult{key=not, value=1}
Writing: IntermediateResult{key=manzana, value=1}
Combine task for: file
Combine task for: random
Writing: IntermediateResult{key=file, value=1}
Writing: IntermediateResult{key=random, value=1}
Combine task for: file
Combine task for: tengo
Combine task for: file
Combine task for: more
Writing: IntermediateResult{key=tengo, value=1}
Writing: IntermediateResult{key=file, value=1}
Combine task for: some
Combine task for: third
Writing: IntermediateResult{key=some, value=1}
Combine task for: This
Combine task for: Lululu
Writing: IntermediateResult{key=Lululu, value=1}
Writing: IntermediateResult{key=more, value=1}
Writing: IntermediateResult{key=file, value=1}
Combine task for: one
Writing: IntermediateResult{key=one, value=1}
Writing: IntermediateResult{key=This, value=1}
Combine task for: And
Writing: IntermediateResult{key=And, value=1}
Writing: IntermediateResult{key=third, value=1}
Combine task for: in
Combine task for: text
Writing: IntermediateResult{key=in, value=1}
Combine task for: content.
Combine task for: Why
Writing: IntermediateResult{key=content., value=1}
Writing: IntermediateResult{key=Why, value=1}
Combine task for: And
Writing: IntermediateResult{key=And, value=1}
Combine task for: this
Combine task for: is
Combine task for: text
Writing: IntermediateResult{key=text, value=1}
Writing: IntermediateResult{key=text, value=1}
Writing: IntermediateResult{key=is, value=1}
Writing: IntermediateResult{key=this, value=1}
Combine task for: first
Combine task for: content.
Writing: IntermediateResult{key=first, value=1}
Writing: IntermediateResult{key=content., value=1}
Combine task for: is
Writing: IntermediateResult{key=is, value=1}
Combine task for: second
Writing: IntermediateResult{key=second, value=1}
New reduce task for: not
New reduce task for: more
New reduce task for: one
New reduce task for: the
New reduce task for: file
New reduce task for: This
New reduce task for: content.
New reduce task for: first
New reduce task for: Lululu
New reduce task for: More
New reduce task for: in
New reduce task for: text
New reduce task for: Why
New reduce task for: random
New reduce task for: here
New reduce task for: third
New reduce task for: tengo
New reduce task for: manzana
New reduce task for: is
New reduce task for: second
New reduce task for: this
New reduce task for: And
New reduce task for: some

Results: 
not - 1
more - 1
one - 1
Why - 1
the - 2
file - 3
This - 1
content. - 2
is - 2
first - 1
manzana - 1
tengo - 1
Lululu - 1
More - 1
third - 1
in - 1
text - 2
here - 1
random - 1
some - 1
And - 2
this - 1
second - 1

Posted 2nd February 2018 by Anonymous

Labels: java mapreduce

Add a comment

No more comments

Jan

29

Kotlin null safety for Optional experienced
In this article I will try to map methods of Java’s Optional to Kotlin’ssimilar scattered language features and built-in functions. The code in the examples is written in Kotlin, because the language has all the JDK classes available.

Representation

Let’s start with the representation. The Opitonal usage, requires creating a new object for the wrapper every time some value is wrapped or transformed to another type, with exclusion of when the Optional is empty (singleton empty Optional is used). In Kotlin there is no additional overhead, lanuage uses plain old null. The code below shows both approaches:
val kotlinNullable: String? = "Kotlin way, this can be null"

val javaNullable: Optional<String> = Optional.ofNullable("Java way, can be null as well”)
Map using method of inner value’s type

To transform the value inside Optional using the inner value’s method we can apply a method reference to map. To do the same in Kotlin, we can use safe call operator .? as demonstrated below:
val lengthKotlin: Int? = kotlinNullable?.length

val lengthJava: Optional<Int> = javaNullable.map(String::length)
Map using external mapper

If the transformation cannot be performed by simple method call then Optional’smap method is happy to take lambda as well. Kotlin provides built-in method let, which we can invoke on any object. In our case we also need to use safe call operator to skip the calculation for null values:
val lengthKotlin: Int? = kotlinNullable?.let { external.computeLength(it) }

val lengthJava: Optional<Int> = javaNullable.map { external.computeLength(it) }
Filter

In Optional, filter allows to remove the value inside, if the provided predicate test returns false. Kotlin offers two built-in functions with this behaviour - takeIf and takeUntil. First is simialar to the Optional’sfilter, second one drops the value if the predicate returns true - opposite to takeIf. Example code:
val kotlinFiltered: String? = kotlinNullable.takeIf(predicate)

val kotlinFiltered2: String? = kotlinNullable.takeUnless(predicate)

val javaFiltered: Optional<String> = javaNullable.filter(predicate)
FlatMap

There is no built-in Kotlin function with the flatMap behaviour because it’s actually not neccassary. Nullability is not represented by a type and simple mapping turns out to be working fine.

IsPresent

In Kotlin this check can be performed by a simple null comparision:
val kotlinIsNull: Boolean = kotlinNullable != null

val javaIsNull: Boolean = javaNullable.isPresent()
It is worth to know, that after we do this, compiler is sure that the value is present and allows us to drop all required null checks in the further code (String? becomes String).
if (kotlinNullable != null) { // String?

val notNullable: String = kotlinNullable // String from this line

}
Get

Both Optional and Kotin approaches discourage users from getting the inside value with the straight call, because it may cause NPE. Kotlin introduces rude !! operator, which you should use only as a last resort:
val notSafeGetKotlin: String = kotlinNullable!!

val notSafeGetJava: String = javaNullable.get()
OrElse/OrElseThrow

Kotlin introduces elvis operator ?: which allows to set the default value in case of null or throw an excepcion:
val kotlinValue: String = kotlinNullable?: ""

val javaValue: String = javaNullable.orElse("")

val kotlinExc: String = kotlinNullable ?: throw RuntimeException()

val javaExc: String = javaNullable.orElseThrow { RuntimeException() }
Conclusion

I hope reading this article will help you laverage your Optional's experience to quickly learn Kotlin'snull safety features. I think it is worth to give Kotlin a try if only to expand your programming horizonts.
Posted 29th January 2018 by Anonymous

Labels: java kotlin optional

0
Add a comment
Jan

5

Understanding flatMap
Have you ever scrolled someone’s code and bumped into this weird method called flatMap, not knowing what it actually does from the context? Or maybe you compared it with method map but didn’t really see much difference? If that is the case then this article is for you.

flatMap is extremly usfull when you try to do proper functional programming. In Java it usually means using Streams and Optionals - concepts introduced in version 8. These two types can be thought of as some kind of wrapper over a type, which adds some extra behaviour - Stream<T> wrapps type T allowing to store any number of elements of type T inside, whereas Optional<T> wrapps some type T to implicitly say that the element of that type may or may not be present inside. Both of them share methods map and flatMap. Before we move on, I want to make sure you understand what the map method is doing. It basically allows you to apply the method to the element inside the wrapper and possibly change it’s type:

Function<String, Long> toLong = Long::parseLong; // function which maps String to Long

Optional<String> someString = Optional.of("12L");

Optional<Long> someLong = someString.map(toLong); //aplying the function to the possible String inside

Stream<String> someStrings = Stream.of("10L", "11L");

Stream<Long> someLongs = someStrings.map(toLong); //applying the function to all Strings inside

After applying the function toLong to our wrappers, their inner types changed to the second type of the toLong signature (the result type).

Let’s examine the function Long::parseLong. If we call it using a string that is not actually a valid long it will throw NumberFormatException. But what if Java designers decide to implement it so it returns Optional<Long> instead of just Long and removed the exception? Our code for Optional part would look like that:

Function<String, Optional<Long>> toLongOpt = Long::parseLongOpt;//method I made up

Optional<String> someString = Optional.of("12L");

Optional<Optional<Long>> someLong = someString.map(toLongOpt); //:<

Wow, that is nasty! When we appliednew method to our wrapper, the inner type was changed from String to Optional<Long> (the result type of toLongOpt which we applied). We don’t really need to have a double Optional because just one is perfectly fine. Now, to get the value we need to extract it twice, not mentioning how it would look like when we want to map it again without unwrapping… To restore it to the single type we would need to write a method like this one:

public static <T> Optional<T> flatten(Optional<Optional<T>> optional) {

return optional.orElse(Optional.empty());

}

This method will flatten our Optional<Optional<T>> to Optional<T> without changing the inner value. The code will look like this:

Function<String, Optional<Long>> toLongOpt = Long::parseLongOpt;

Optional<String> someString = Optional.of("12L");

Optional<Long> someLong = flatten(someString.map(toLongOpt));

This is exactly what method flatMap is doing. It first applies the function returning another Optional to the inside object (if present) and then flattens the result before returning it, so you don’t have to do it yourself. This is how we can use it:

Function<String, Optional<Long>> toLongOpt = Long::parseLongOpt;

Optional<String> someString = Optional.of("12L");

Optional<Long> someLong = someString.flatMap(toLongOpt);

For Stream, we can use it in the situation when the function we want to map our elements with, returns the Stream. (example signature: Function<String, Stream<Long>>).

That’s it. Just remember that flatMap = map + flatten.

If you want to dive into the topic read about the Functor and Monad concepts and the relationship between them.
Posted 5th January 2018 by Anonymous

Labels: functional functor java monad optional stream

0
Add a comment

Nov

Verifying End-to-end Test Code Coverage using Jacoco Agent

Fact - End-to-end tests are critical if you want to make sure your software works as it should. To be 100% sure that you covered every (or almost every) possible branch in your business code, it is worth to check what code has been invoked after your E2E suite finished successfully. The solution I am going to present may be much easier, if you are able to stand up your application locally. Sometimes though, it is not the case.

In this scenario, we are going to start two instances of some web application at the remote servers, with code invocation recording turned on. Next, we are going to run integration test pointing to them one after another with different parameters, to test different paths. After the tests are done, we are going to pull the data from these servers to localhost, merge it and transform it to HTML report.

We are going to test this simple REST controller:

@RestController
public class ExampleController {

    @GetMapping(path = "/test")
    public int exampleMethod(@RequestParam int parameter) {
        if(parameter > 10) {
            return invokeThisBranch(parameter);
        } else {
            return invokeAnotherBranch(parameter);
        }
    }

    private int  invokeThisBranch(int parameter) {
        System.out.println("Taking this branch..");
        if(parameter < 10) {
            System.out.println("This cannot be tested..");
        }
        return 3;
    }

    private int invokeAnotherBranch(int parameter) {
        System.out.println("Taking another branch..");
        return 3;
    }
}

To make things simple we will run two instances of the same application on our local machines (on different ports).

Before we do that we need to download jacoco agent from https://mvnrepository.com/artifact/org.jacoco/org.jacoco.agent. Remember that the version must match the version of your jacoco plugin in maven. This agent will be attached to JVM and record the code coverage. The problem is that you need to push it to your remote server manually (if you find a way to automate this clean way let me know :)).

If you are using Inttellij, add this to VM options in your Run Configuration (or add this to startup script on remote server):

-javaagent:<path-to-agent>/jacocoagent.jar=port=36320,destfile=jacoco-it.exec,output=tcpserver

By specyfying output as a tcpserver and providing a port, you are enabling further connection from maven plugin, which will download data file (jacoco-it.exec).

Our test suite contains one test, which will cause different methods to be invoked depending of the parameter value.

@RunWith(SpringRunner.class)
public class TestCoverageExampleApplicationTests {

    private TestRestTemplate testRestTemplate = new TestRestTemplate();

    @Test
    public void anotherBranchTest() {
        //given
        int appPort =  Integer.parseInt(System.getProperty("app.port"));
        int parameter = Integer.parseInt(System.getProperty("parameter"));

        //when
        Integer result = testRestTemplate.getForObject(
                "http://localhost:" + appPort + "/test?parameter=" + parameter, Integer.class);

        //then
        assertThat(result).isEqualTo(3);
    }

}

We want to make sure that even though the code coverage was spread among instances, we will merge it correctly to be able to see the whole picture. Let’s run the test - diferent paths for different instance:

mvn test -Dapp.port=8080 -Dparameter=5
mvn test -Dapp.port=8081 -Dparameter=15

Before we generate coverage report, we need to configure jacoco plugins in pom.xml file. Full version can be found at my github account:

<plugins>
    <plugin>
        <groupId>org.jacoco</groupId>
        <artifactId>jacoco-maven-plugin</artifactId>
        <version>${jacoco.version}</version>
        <executions>
            <execution>
                <id>pull-test-data</id>
                <phase>post-integration-test</phase>
                <goals>
                    <goal>dump</goal>
                </goals>
                <configuration>
                    <destFile>${project.build.directory}/jacoco-it-${app.host}:${app.port}.exec</destFile>
                    <address>${app.host}</address>
                    <port>${app.port}</port>
                    <reset>false</reset>
                    <skip>${skip.dump}</skip>
                </configuration>
            </execution>
            <execution>
                <id>merge-test-data</id>
                <goals>
                    <goal>merge</goal>
                </goals>
                <configuration>
                    <destFile>target/jacoco-it.exec</destFile>
                    <skip>${skip.dump}</skip>
                    <fileSets>
                        <fileSet implementation="org.apache.maven.shared.model.fileset.FileSet">
                            <directory>target</directory>
                            <includes>
                                <include>*it*.exec</include>
                            </includes>
                        </fileSet>
                    </fileSets>
                </configuration>
            </execution>
        </executions>
        <configuration>
            <append>true</append>
        </configuration>
    </plugin>
    <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-antrun-plugin</artifactId>
        <version>${antrun-plugin.version}</version>
        <executions>
            <execution>
                <id>generate-report</id>
                <phase>post-integration-test</phase>
                <goals>
                    <goal>run</goal>
                </goals>
                <configuration>
                    <skip>${skip.int.tests.report}</skip>
                    <target>
                        <taskdef name="report" classname="org.jacoco.ant.ReportTask">
                            <classpath path="${basedir}/target/jacoco-jars/org.jacoco.ant.jar"/>
                        </taskdef>
                        <mkdir dir="${basedir}/target/coverage-report"/>
                        <report>
                            <executiondata>
                                <fileset dir="${basedir}">
                                    <include name="target/jacoco-it*.exec"/>
                                </fileset>
                            </executiondata>
                            <structure name="jacoco-multi Coverage Project">
                                <group name="jacoco-multi">
                                    <classfiles>
                                        <fileset dir="target/classes"/>
                                    </classfiles>
                                    <sourcefiles encoding="UTF-8">
                                        <fileset dir="src/main/java"/>
                                    </sourcefiles>
                                </group>
                            </structure>
                            <html destdir="${basedir}/target/coverage-report/html"/>
                            <xml destfile="${basedir}/target/coverage-report/coverage-report.xml"/>
                            <csv destfile="${basedir}/target/coverage-report/coverage-report.csv"/>
                        </report>
                    </target>
                </configuration>
            </execution>
        </executions>
        <dependencies>
            <dependency>
                <groupId>org.jacoco</groupId>
                <artifactId>org.jacoco.ant</artifactId>
                <version>${jacoco.ant.version}</version>
            </dependency>
        </dependencies>
    </plugin>
</plugins>

Let’s create automated script for coverage report generation:

mvn jacoco:dump@pull-test-data -Dapp.host=localhost -Dapp.port=36320 -Dskip.dump=false
mvn jacoco:dump@pull-test-data -Dapp.host=localhost -Dapp.port=36321 -Dskip.dump=false
mvn jacoco:merge@merge-test-data -Dskip.dump=false
mvn antrun:run@generate-report -Dskip.int.tests.report=false

Script will pull the data files from two remote servers (localhost in our case), merge them into one file, and then generate report from it.

Final report is a joy to analyze:

Our class:

As you can see even though different code was invoked on different instances, final report contains both paths.

Some real world problems you may encounter:

If you use byte code manipulators like lombok or aspectj, jacoco won’t be able to find a source code that match the invoked lines, you can use auto-value or immutables instead of lombok for some use cases and spring-aop instead of aspectj
If you write your tests in Spock and you want to upload your jacoco-it.exec files to Sonar to show the code coverage there, you have to make sure groovy’s expressive method names will be correctly transformed in failsafe report - you need to add org.sonar.java.jacoco.JUnitListener as a listener

Full code can be found on my github account.

Posted 9th November 2017 by Anonymous

Labels: jacoco java testing

Add a comment

Jul

Lifting functions to work with monads in Java

Functional Programming in Java

Stream and Optional classes - added to Java 8 - allow you to have some fun with functional programming. The problem is Java still misses quite a lot to be taken as a serious FP language. Lambda notation and two monads (Optional and Stream) are just the tip of the iceberg. This leads to arising of libraries like vavr or functionaljava - both deriving from purely functional language Haskell.

One of the first things you need to get rid of when trying to be more functional, is the attempt to unwrap the monad too early. It usually involves using methods like Optional.get() or Stream.collect() where there is yet no need. Sometimes though, Java doesn't help with that, so let me give you some custom code for the rescue. This article will bring you closer to the concept of method lifting.

Calculations on Optionals

Suppose we have some nice API we would like to use to calculate numbers:

    public interface Math {

        int multiply(int a, int b);

        double divide(int a, int b);

        ..
    }

We would like to use it to do some calculations on numbers wrapped with Optional:

    public interface NumberProvider {

        Optional<Integer> getNumber();

    }

Let's say we want to write a method, which returns the result of the division of two numbers wrapped with Optional, or empty Optional if any one of them is empty (we skip the 0 divisor case here). It may look something like this:

public Optional<Double> divideFirstTwo(NumberProvider numberProvider, Math math) {
        Optional<Integer> first = numberProvider.getNumber();
        Optional<Integer> second = numberProvider.getNumber();
        if(first.isPresent() && second.isPresent()) {
            double result = math.divide(first.get(), second.get());
            return Optional.of(result);
        } else {
            return Optional.empty();
        }
    }

That's rather nasty. It involves a lot of code which the only purpose is to wrap and unwrap the Optional. Let's try to make it more functional:

  public Optional<Double> divideFirstTwo(NumberProvider numberProvider, Math math) {
        return numberProvider.getNumber()
                .flatMap(first -> numberProvider.getNumber()
                                         .map(second -> math.divide(first, second)));
    }

That's much better. It turns out that invoking flatMap on the first monad and map on the second one inside the lambda, can be extracted even further to the method called lift:

public interface Optionals {

    static <R, T, Z> BiFunction<Optional<T>, Optional<R>, Optional<Z>> lift(BiFunction<? super T, ? super R, ? extends Z> function) {
        return (left, right) -> left.flatMap(leftVal -> right.map(rightVal -> function.apply(leftVal, rightVal)));
    }
}

Lift is able to promote any function, which takes two arguments, to the function with the arguments and the result type wrapped with Optional. It actually adds Optional behavior to the function in such a way that if one of the arguments is empty, then the result will also be empty. If JDK extracted flatMap and map methods to some common interface, for example Monad, then we could write one lift function for every instance of Java monad (Stream, Optional, custom classes). Unfortunately, we need to do this copy pasting for every instance. The final code for divideFirstTwo becomes:

import static com.ps.functional.monad.optional.Optionals.lift;
...

    public Optional<Double> divideFirstTwo(NumberProvider numberProvider, Math math) {
        return lift(math::divide).apply(numberProvider.getNumber(), numberProvider.getNumber());
    }

Summary

I hope this article encouraged you to play with functional style in Java. JDK needs to be greatly improved for the language to be called functional in the future. Unfortunately, Java 9 doesn't promise any major improvements besides just a few additional methods. Source code can be found here.

Posted 20th July 2017 by Anonymous

Labels: functional java optional

Add a comment

Jul

Composing functions with java.util.function.Function.andThen() trick

In this article, I am going to present you a simple trick that will make using java.util.function.Function.andThen() more useful.

As an example I will use ExternalSystemGateway class, which job is to call external system along with serializing/mapping the messages:

class Request {...}

class Response {...}

class ExternalSystemRequest {...}

class ExternalSystemResponse {...}

public class ExternalSystemGateway {

    public Response invoke(Request request) {
        ExternalSystemRequest externalSystemRequest = mapRequest(request);
        String requestString = marshallRequest(externalSystemRequest);
        String responseString = sendAndReceive(requestString);
        ExternalSystemResponse externalSystemResponse = unmarshallResponse(responseString);
        Response response = mapResponse(externalSystemResponse);
        return response;
    }

    private ExternalSystemRequest mapRequest(Request request) {
        // ..
    }

    private String marshallRequest(ExternalSystemRequest externalSystemRequest) {
        // ..
    }

    private String sendAndReceive(String requestString) {
        //..
    }

    private ExternalSystemResponse unmarshallResponse(String responseString) {
        //..
    }

    private Response mapResponse(ExternalSystemResponse externalSystemResponse) {
        //..
    }

}

You can see that every line of the invoke method does some kind of action, which transforms some input type to another output type. Let's try to get rid of the variables, which are declared only to be used in the next line:

    public Response invoke(Request request) {
        return mapResponse(
                unmarshallResponse(
                        sendAndReceive(
                                marshallRequest(
                                        mapRequest(request)))));
    }

Variables are gone but this code looks weird. What is more you need to read it backwards to understand the flow. Let's try to compose invocations using java.util.function.Function.andThen():

public Response invoke(Request request) {
        return ((Function<Request, ExternalSystemRequest>) (this::mapRequest))
                .andThen(this::marshallRequest)
                .andThen(this::sendAndReceive)
                .andThen(this::unmarshallResponse)
                .andThen(this::mapResponse)
                .apply(request);

Better, code reads nicer now, but can we get rid of the (Function<Request, ExternalSystemRequest>) cast at the beginning? Typing just this::mapRequest is not representing java.util.function.Function type so the trick is to achieve this using this simple interface:

public interface FunctionUtils {

    static <T, R> Function<T, R> function(Function<T, R> function) {
        return function;
    }
}

Our invoke method now becomes:

import static com.ps.functional.util.FunctionUtils.function;
...

    public Response invoke(Request request) {
        return function(this::mapRequest)
                .andThen(this::marshallRequest)
                .andThen(this::sendAndReceive)
                .andThen(this::unmarshallResponse)
                .andThen(this::mapResponse)
                .apply(request);
    }

Method function is just casting any one argument method reference to a java.util.function.Function type. You can create the same method for two argument functions:

1
2
3

static <T, U, R> BiFunction<T, U, R> biFunction(BiFunction<T, U, R> biFunction) {
        return biFunction;
    }

You can now easily compose functions in a readable way, without a need to do any casting. Source code for FunctionUtils is available here.

Posted 10th July 2017 by Anonymous

Labels: functional java

View comments

Jul

Exception-free code using functional approach

Checked exceptions & Java 8

Defining custom exceptions (both checked and unchecked) is a common approach to handling errors in Java applications. It usually leads to creating a new class for every different type of error, marking methods with throws keyword or wrapping code with try-catch blocks. This can lead to the code which is hard to read since every block adds another level of complexity.

Lambdas in Java 8 started the boom for functional approach in writing the code. Developers are now more familiar with these concepts and seeing -> in the code has became daily thing. Unfortunately, functional programming doesn't play well with Java exceptions - especially checked ones.

The main drawback is that method cannot be matched to functional interfaces from java.util.function if it throws a checked exception. This makes developers create custom functional interfaces with throws clauses, duplicating the code from JDK or write nasty try-catch block in lambdas. You can use RuntimeException type to model your custom exceptions but then the information about possible failure is lost and handling it is not required by compiler anymore.

Result monad

Solution suggested by myself uses custom Result class which is similar to the Optional monad. Optional is a box for objects that may exist - Optional keeps a reference to that object - or not - Optional keeps null. Result on the other hand can be a Success - code was executed correctly and the value is ready - or Failure - some error occurred. Here is the fragment of Result interface:

public interface Result<T> {

    boolean isSuccess();

    T getResult();

    ErrorType getError();

    String getOptionalErrorMessage();

    ...
}

Success implementation:

public final class Success<T> implements Result<T> {

    private final T result;

    ...

    public boolean isSuccess() {
        return true;
    }

    @Override
    public ErrorType getError() {
        throw new NoSuchElementException("There is no error in Success");
    }

    @Override
    public String getOptionalErrorMessage() {
        throw new NoSuchElementException("There is no optional error message in Success");
    }

    @Override
    public T getResult() {
        return result;
    }
}

Failure implementation:

public final class Failure<T> implements Result<T> {

    private final ErrorType error;

    private final String optionalErrorMessage;

        ...

    @Override
    public boolean isSuccess() {
        return false;
    }

    @Override
    public ErrorType getError() {
        return error;
    }

    @Override
    public String getOptionalErrorMessage() {
        return optionalErrorMessage;
    }

    @Override
    public T getResult() {
        throw new NoSuchElementException("There is no result is Failure");
    }
}

ErrorType is a type which represents predefined error condition. It sometimes comes with the optional error message. We could actually store Exception type instead of ErrorType but custom class gives more flexibility. Similarly to Optional, Result has get* methods which allow to access the internal state. They shouldn't be used without isSuccess method because they could result in throwing NoSuchElementException. They are helpful for unit tests, when you just want the value inside to assert on it.

Nothing fun so far. Let's have a look at the rest of the Result interface:

public interface Result<T> {

     ...

    default <R> Result<R> map(Function<? super T, ? extends R> mapper) {
        return isSuccess() ?
                Success.of(mapper.apply(getResult())):
                (Failure<R>) this;
    }

    default <R> Result<R> flatMap(Function<? super T, Result<R>> mapper) {
        return isSuccess() ?
                mapper.apply(getResult())
                (Failure<R>) this;
    }

    default <R> R fold(Function<? super T, ? extends R> successFunction, Function<Failure<R>, ? extends R> failureFunction) {
        return isSuccess() ?
                successFunction.apply(getResult()) :
                failureFunction.apply((Failure<R>) this);
    }
}

More about these methods later. Now, let's put it to use. When using libraries that declare checked exceptions, we would like to map every successful method call to Success and every cached exception to Failure. So the code like this:

    public String getNameById(int id) throws DatabaseAccessException {
        try {
            return dbClient.getNameById(id);
        } catch (DatabaseAccessException e) {
            throw new CustomException(e);
        }
    }

will become:

    public Result<String> getNameById(int id) {
        try {
            return Success.of(dbClient.getNameById(id));
        } catch (DatabaseAccessException e) {
            return Failure.of(PREDEFINED_DB_ERROR, e.getMessage());
        }
    }

(There are existing solutions like scala's/vavyr's Try, which skip the try - catch block totally and operate on Exception type. I prefer to use my custom ErrorType but you are free to use different approach. By using ErrorType you will also save some time by not wrapping the exception in custom one, which uses costly Throwable.fillInStackTrace - article.)

Result usage in business code

Now suppose we would like to use it in our business code. Here is the interface from persistence layer interface modeled with the usage of Result:

public interface DbConnector {

    Result<String> getNameById(int id);

    Result<Integer> getAgeByName(String name);

}

Suppose we want to use it in our business code:

class BusinessClass {

    private final DbConnector dbConnector;

    ...

    public Result<Boolean> isAdult(int userId) {
        return dbConnector.getNameById(userId)
                .flatMap(userName -> dbConnector.getAgeByName(userName)) // or simpler: .flatMap(dbConnector::getAgeByName)
                .map(userAge -> userAge >= 18);
    }

}

Method isAdult pulls the user's name from database and then uses it to fetch the age of that person (weird API but that's just an example :)). Finally it maps the age of that person to the boolean which indicates whether this person is an adult or not. What is important here, is that there are two calls which can result in Failure, sequenced using flatMap. The final invocation of map cannot fail because the predicate 'userAge >=18' will not be applied if the result from flatMap is Failure. The same way, function passed to flatMap will not be applied if getNameById returns a Failure - this Failure will be passed down the invocation stream to the caller of isAduld. Both map and flatMap will only invoke the passed method, if the Result is Success. This way we can model the business flow, but what if we want to unwrap the value inside, for example to return it to the client?:

    Result<Boolean> isAduld = isAduld(userId);
    String response = isAduld.fold(
            userIsAdult -> "adult: " + userIsAdult,
            failure -> "error occurred: " + failure.getError().show());

Fold will reduce the Result to the single value using provided functions - first if it's Success or second if it's Failure. Remember not to use just get method instead.

Summing up

map - applies function to value inside if it's Success, otherwise does nothing

flatMap - applies function which returns Result to value inside if it's Success, otherwise does nothing

fold - applies success function to value inside if it's Success, otherwise applies failure function to the ErrorType - reduces Result to singe value

That's it. Source code can be found here: https://github.com/pszeliga/functional-java. Other methods from Result will be discussed in future posts.

Posted 5th July 2017 by Anonymous

Labels: exception functional java result

Add a comment

Stream[Article]

Add a comment

Conclusion

Add a comment

1. Set debugger breakpoint by pattern

2. Attaching custom log file to be followed after Spring Boot app startup.

3. Invoke any code one the screen when in debug mode

4. Compare with clipboard

5. Shelve changes

Conclusion

Add a comment

Add a comment

Representation

Map using method of inner value’s type

Map using external mapper

Filter

FlatMap

IsPresent

Get

OrElse/OrElseThrow

Conclusion

Add a comment

Add a comment

Add a comment

Functional Programming in Java

Calculations on Optionals

Summary

Add a comment

View comments

Checked exceptions & Java 8

Result monad

Result usage in business code

Summing up

View comments