1. When I started working as a Java Developer, me and my teammate got a first task to repair all broken tests (great task for new starters!) in some old project. Replacing some old configuration and upgrading a few libraries helped making the tests status green but there was another problem. End-to-end test suite took a few hours to pass making it impossible to get the feedback for a new change quickly. Sometimes build run for a few hours and hanged so E2E tetst became excluded from the feedback loop at the end. 
    Most of the tests from that suite did the same thing: 
    • Invoke a few commands on the mainframe through HTTP interface
    • Sleep for 30 seconds (probably just to be extra safe) for each command and check if the result was persisted to database by another system. 
    It turned out, that first try to reduce the sleep time to 15 seconds by my teammate, didn’t make any tests fail, so that was a nice first try. But what was the optimal wait time for each test?

    After some time, we tried another approach. If the flow has a asynchronous nature then why not testing it asynchronously. 
    I introduced this simple class (or at least class which does the same): 
    public class Await {
        private static final BiPredicate<Long, Long> notTimedOut =
                (start, timeoutMillis) -> System.currentTimeMillis() - start < timeoutMillis;
        public static void await(Supplier<AssertionResult> assertionVerifier, long timeoutMillis) {
            final long startTime = System.currentTimeMillis();
            Stream.generate(assertionVerifier)
                    .takeWhile(result -> notTimedOut.test(startTime, timeoutMillis) || fail(result))
                    .filter(AssertionResult::isOK)
                    .findFirst();
        }
        public static boolean fail(AssertionResult result) {
            throw new RuntimeException("Assertion not passed after timeout, problem: " + result.optionalMessage);
        }
    }

    Method await checks some assertion in a loop until either it won’t throw AssertionError or the timeout is passed. This assertion is usually some state we are waiting for that should be ready in the nearest future. If it is not ready after the timeout, the last message from the AssertionError will be thrown wrapped as a RuntimeException. This greatly reduces test time because if the result is ready earlier, we don’t have to spend unnecessary time in the sleep method.

    To represent the assertion result we can use classes (minimal implementation):
    enum Status { OK, ERROR }
    class AssertionResult {
        public static final AssertionResult OK = new AssertionResult(Status.OK, "");
        public final Status status;
        public final String optionalMessage;
        private AssertionResult(Status status, String optionalMessage) {
            this.status = status;
            this.optionalMessage = optionalMessage;
        }
        public static AssertionResult error(String optionalMessage) {
            return new AssertionResult(Status.ERROR, optionalMessage);
        }
        public boolean isOK() {
            return status == Status.OK;
        }
    }

    We also need to prepare our AssertionResult suppliers - wrappers which fetch the data and invoke assertions on the result. Usually it is possible to define generic supplier if the way we fetch the data is similar. For example we can write this builder method:
    public static Supplier<AssertionResult> dbDataAssertion(Consumer<String> userAssertion, long userId) {
        return () -> {
            String usernameFromDB = db.getUserName(userId); //or any other more generic version
            try {
                userAssertion.accept(usernameFromDB);
                return AssertionResult.OK;
            } catch (AssertionError assertionError) {
                return AssertionResult.error(assertionError.getMessage());
            }
        };
    }

    We are ready to replace Thread.sleep() with call to Await in our tests:
    ...
    long timeoutMillis = 1000L;
    long userId = 1234L;
    ...
    //then
    Await.await(
            dbDataAssertion(userName -> assertEquals(4, userName.length()), userId),
            timeoutMillis);

    Conclusion


    Don’t overuse Thread.sleep() in your test code. According to this Google research it is one of the main reasons of flaky tests. Asynchronous approach may complicate your tests but the time savings can be tremendous.  If you are looking for some existing implementations there is a nice class in Spock which does the same.
    Posted by
    Labels: java testing
    0

    Add a comment

  2. In this article I am going to share some cool features I stumbled upon while coding with Intellij. These are not the most popular/productivity improving ones - for these you should watch this video. 

    1. Set debugger breakpoint by pattern

    Problem: You want to check if the debugger steps into some code during the run. Usually you suspect where the flow will end up but sometimes finding the exact line could be hard.
    Solution: Breakpoints in Debugger pane -> Click (+) -> Java Method Breakpoint -> Fill class/method pattern. 
    For example this setting will stop the debugger anytime the code enters any method from any class from com.fasterxml.jackson package.
    Image title

    2. Attaching custom log file to be followed after Spring Boot app startup.

    Problem: You have your app log configured to write to the file after the startup. However, this makes it harder to read that log because now you have to open this file somehow instead of just checking the console output. 
    Solution: Run/Debug Configuration -> Your Spring Boot app -> Logs -> Click (+) -> Fill alias and Log File Location. This will open additional pane with file content after you run the configuration that is easy to work with. 
    Image title

    3. Invoke any code one the screen when in debug mode

    Problem: You want to check the result of invoking some code on the screen but you are to lazy to paste it to Evaluate Expression console.
    Solution: When the program is paused in debug mode, hit the alt key and click any fragment you want to see the result.
    Image title

    4. Compare with clipboard

    Problem: You would like to compare some text the same way Intellij’s VCS diff shows changes in files.
    Solution: Copy text to the clipboard -> Select any other text -> Right Click -> Click Compare With Clipboard.
    I think it is worth to have a key assigned to this one.
    Image title

    5. Shelve changes

    Problem: I used to use Git stash command externally to save some changes I didn’t want to commit yet. Is there a way to do this from Intellij? 
    Solution: VCS -> Local Changes -> Select the files you want to stash -> Right Click -> Shelve changes.
    You can move them back from the shelf by right clicking on the selection and choosing Unshelve.
    Image title

    Conclusion

    I hope you have learned at least one thing that might be helpful at you work. Please share other cool but less know Intellij features in the comment section. 
    Posted by
    Labels: intellij
    0

    Add a comment

  3. I have recently pushed very simple Map Reduce concept implementation on my Github account (click). My idea was to focus on the concept and mock the rest. You can follow the code to understand how it works but I will enumerate most implementation details:
    • Text files are represented as Strings and stored in memory
    • Master is responsible for splitting the data and scheduling the map/reduce tasks to workers
    • Workers are represented as threads, they are run using CompletableFuture api
    • When map task is finished, the combine function will be run on the result
    • When Master will notice that all map tasks have finished, it will take all resulted distinct keys and pass them as an argument to reduce tasks
    • When reduce tasks are finished, result is printed and the executor is closed
    It is easy to define new user programs, check example package.

    In memory data:

    "file1.txt" -> "This is the first file \ncontent. "
"file2.txt" -> "And this is the second file content. "
"file3.txt" -> "More text in \nthird file"
"file4.txt" -> "And some random text here"
"file5.txt" -> "Why not \none more"
"file6.txt" -> "Lululu tengo manzana"


    Distributed Grep (search for "And"):

    New map task for key: file5.txt
New map task for key: file4.txt
New map task for key: file3.txt
New map task for key: file6.txt
New map task for key: file1.txt
New map task for key: file2.txt
Combine task for: file2.txt
Combine task for: file4.txt
Writing: IntermediateResult{key=file2.txt, value=And this is the second file content. }
Writing: IntermediateResult{key=file4.txt, value=And some random text here}
New reduce task for: file4.txt
New reduce task for: file2.txt

Results: 
file4.txt - And some random text here
file2.txt - And this is the second file content.

    Word count:

    New map task for key: file5.txt
New map task for key: file4.txt
New map task for key: file3.txt
New map task for key: file6.txt
New map task for key: file1.txt
New map task for key: file2.txt
Combine task for: not
Combine task for: here
Combine task for: the
Combine task for: manzana
Combine task for: the
Combine task for: More
Writing: IntermediateResult{key=More, value=1}
Writing: IntermediateResult{key=the, value=1}
Writing: IntermediateResult{key=here, value=1}
Writing: IntermediateResult{key=the, value=1}
Writing: IntermediateResult{key=not, value=1}
Writing: IntermediateResult{key=manzana, value=1}
Combine task for: file
Combine task for: random
Writing: IntermediateResult{key=file, value=1}
Writing: IntermediateResult{key=random, value=1}
Combine task for: file
Combine task for: tengo
Combine task for: file
Combine task for: more
Writing: IntermediateResult{key=tengo, value=1}
Writing: IntermediateResult{key=file, value=1}
Combine task for: some
Combine task for: third
Writing: IntermediateResult{key=some, value=1}
Combine task for: This
Combine task for: Lululu
Writing: IntermediateResult{key=Lululu, value=1}
Writing: IntermediateResult{key=more, value=1}
Writing: IntermediateResult{key=file, value=1}
Combine task for: one
Writing: IntermediateResult{key=one, value=1}
Writing: IntermediateResult{key=This, value=1}
Combine task for: And
Writing: IntermediateResult{key=And, value=1}
Writing: IntermediateResult{key=third, value=1}
Combine task for: in
Combine task for: text
Writing: IntermediateResult{key=in, value=1}
Combine task for: content.
Combine task for: Why
Writing: IntermediateResult{key=content., value=1}
Writing: IntermediateResult{key=Why, value=1}
Combine task for: And
Writing: IntermediateResult{key=And, value=1}
Combine task for: this
Combine task for: is
Combine task for: text
Writing: IntermediateResult{key=text, value=1}
Writing: IntermediateResult{key=text, value=1}
Writing: IntermediateResult{key=is, value=1}
Writing: IntermediateResult{key=this, value=1}
Combine task for: first
Combine task for: content.
Writing: IntermediateResult{key=first, value=1}
Writing: IntermediateResult{key=content., value=1}
Combine task for: is
Writing: IntermediateResult{key=is, value=1}
Combine task for: second
Writing: IntermediateResult{key=second, value=1}
New reduce task for: not
New reduce task for: more
New reduce task for: one
New reduce task for: the
New reduce task for: file
New reduce task for: This
New reduce task for: content.
New reduce task for: first
New reduce task for: Lululu
New reduce task for: More
New reduce task for: in
New reduce task for: text
New reduce task for: Why
New reduce task for: random
New reduce task for: here
New reduce task for: third
New reduce task for: tengo
New reduce task for: manzana
New reduce task for: is
New reduce task for: second
New reduce task for: this
New reduce task for: And
New reduce task for: some

Results: 
not - 1
more - 1
one - 1
Why - 1
the - 2
file - 3
This - 1
content. - 2
is - 2
first - 1
manzana - 1
tengo - 1
Lululu - 1
More - 1
third - 1
in - 1
text - 2
here - 1
random - 1
some - 1
And - 2
this - 1
second - 1

    Posted by
    Labels: java mapreduce
    0

    Add a comment

Popular Posts
Popular Posts
About Me
About Me
Labels
Labels
Blog Archive
Blog Archive
Loading
Dynamic Views theme. Powered by Blogger. Report Abuse.