vmlens Blog http://vmlens.com/blog-2/ multithreaded programming with java en-US Wed, 20 May 2020 19:45:51 GMT 2020-05-20T19:45:51Z en-US How to define thread-safety in Java? http://vmlens.com/articles/cp/thread_safety_definition Thread-safety is typically defined as works correctly even when used by multiple threads. But all those definitions have the disadvantage that they do not tell us how. They do not tell us how the methods behave when called from multiple threads and they do not tell us how to use the methods in a thread-safe way. So I rather define thread-safety as: A class is thread-safe if all its public methods are either atomic or quiescent. <p>Thread-safety is typically defined as works correctly even when used by multiple threads.</p> <p>In <a href="https://jcip.net/">Java Concurrency in Practice</a>, for example, thread-safety is defined as:</p> <blockquote> <p>A class is thread-safe if it behaves correctly when accessed from multiple threads, regardless of the scheduling or interleaving of the execution of those threads by the runtime environment, and with no additional synchronization or other coordination on the part of the calling code.</p> </blockquote> <p>and in <a href="https://www.amazon.com/Effective-Java-Joshua-Bloch/dp/0134685997/">Effective Java</a>:</p> <blockquote> <p>Unconditionally thread-safe: Instances of this class are mutable, but the class has sufficient internal synchronization that its instances can be used concurrently without the need for any external synchronization. Examples include Random and ConcurrentHashMap.</p> </blockquote> <p>But all those definitions have the disadvantage that they do not tell us how. They do not tell us how the methods behave when called from multiple threads and they do not tell us how to use the methods in a thread-safe way.</p> <p>The latter point is especially problematic because of classes like <a href="https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/atomic/LongAccumulator.html"><code>java.util.concurrent.atomic.LongAccumulator</code></a>.</p> <p>So I rather follow the <a href="https://www.elsevier.com/books/the-art-of-multiprocessor-programming-revised-reprint/herlihy/978-0-12-397337-5">Art of Multiprocessor Programming</a>. by using a correctness property to define thread-safety. And since there are two correctness properties used in Java, e.g. linearizable respectively atomic methods and quiescent methods, we can define thread-safety as:</p> <p>A class is thread-safe if all its public methods are either atomic or quiescent.</p> <h1><a href="#when-is-a-method-atomic" id="when-is-a-method-atomic"></a>When is a method atomic?</h1> <p>A method is atomic when the method call appears to take effect instantaneously, at some time between the start and the end of the method call. So other threads either see the state before or after the method call but no intermediate state. Threads reading the state see the most recently written state. Or more formally, reading method calls create a happens-before relation to previous writing method calls.</p> <p>Atomic methods lead to a correctness property called linearizability. Maurice Herlihy and Jeannette Wing introduced the notion of linearizability in 1990 in the paper <a href="https://cs.brown.edu/~mph/HerlihyW90/p463-herlihy.pdf">Linearizability: a correctness condition for concurrent objects</a></p> <h1><a href="#types-of-atomic-classes" id="types-of-atomic-classes"></a>Types of atomic classes</h1> <h2><a href="#stateless" id="stateless"></a>Stateless</h2> <p>Classes without state are automatically atomic. Since they have no state, different threads can never see any intermediate state. Examples for stateless classes include <a href="https://docs.oracle.com/javase/8/docs/api/java/lang/Math.html"><code>java.lang.Math</code></a> or <a href="https://docs.oracle.com/javase/8/docs/api/java/util/Arrays.html"><code>java.util.Arrays</code></a>. A more complicated example is the class <a href="https://github.com/google/gson"><code>com.google.gson.Gson</code></a>, <a href="https://vmlens.com/articles/cp/how_does_gson_achieve_thread_safety/">see this blog post</a>.</p> <h2><a href="#immutable" id="immutable"></a>Immutable</h2> <p>Immutable classes are similar to classes without state automatically atomic. Since their state does not change, different threads can never see an inconsistent intermediate state. Examples of immutable classes are <a href="https://docs.oracle.com/javase/8/docs/api/java/lang/String.html"><code>java.lang.String</code></a> or <a href="https://docs.oracle.com/javase/8/docs/api/java/lang/Integer.html"><code>java.lang.Integer</code></a>.</p> <h2><a href="#using-locks" id="using-locks"></a>Using locks</h2> <p>The next technique to make methods is to use locks. You can either use the Java intrinsic locks, e.g. synchronized blocks or the locks from the package <a href="https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/locks/package-summary.html"><code>java.util.concurrent.locks</code></a>.</p> <p>The easiest way is to use a single monitor and surround each public method of the class with a synchronized block using this monitor. The methods synchronizedSet, synchronizedList and so on from the class <a href="https://docs.oracle.com/javase/8/docs/api/java/util/Collection.html"><code>java.util.Collections</code></a>, for example, use this technique to make the methods of an underlying collection atomic.</p> <p>The following shows part of the source code of the class SynchronizedList. This class is used by the method synchronizedList to create a class with atomic methods:</p> <p><pre class="brush: java">static class SynchronizedList&lt;E&gt; extends SynchronizedCollection&lt;E&gt; implements List&lt;E&gt; { final List&lt;E&gt; list; public E get(int index) { synchronized (mutex) {return list.get(index);} } public E set(int index, E element) { synchronized (mutex) {return list.set(index, element);} } // other methods omitted } </pre></p> <p>Besides those intrinsic locks, Java provides <a href="https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/locks/ReentrantLock.html">reentrant exclusive</a>, <a href="https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/locks/ReentrantReadWriteLock.html">read-write</a>, and <a href="https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/locks/StampedLock.html">stamped locks</a> in the package <code>java.util.concurrent.locks</code>.</p> <h2><a href="#lock-free" id="lock-free"></a>Lock-free</h2> <p>Current processors provide machine code instructions to atomically compare and swap or fetch and add a variable. Those instructions are used by the JVM to implement locks. And you can use those instructions directly through the classes in the package <a href="https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/atomic/package-summary.html"><code>java.util.concurrent.atomic</code></a> and starting with JDK 9 through the class <a href="https://docs.oracle.com/javase/9/docs/api/java/lang/invoke/VarHandle.html"><code> java.lang.invoke.VarHandle</code></a> .</p> <p>The class <a href="https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/ConcurrentSkipListMap.html"><code>java.util.concurrent.ConcurrentSkipListMap</code></a> is an example for a class using those operations to implement atomic methods. And the class <a href="https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/ConcurrentHashMap.html"><code>java.util.concurrent.ConcurrentHashMap</code></a> is an example of the combination of those machine code instructions with locks.</p> <h1><a href="#other-meanings-of-atomic" id="other-meanings-of-atomic"></a>Other meanings of atomic</h1> <p>Atomic is an overloaded term. It is used to describe database transactions through the ACID property, e.g. Atomicity Consistency Isolation Durability. And it is used to describe the effect of machine code instructions. The meanings are similar but with subtle differences. In our context an atomic method means the following:</p> <ol> <li>all or nothing: other threads either see the state before or after the method call but no intermediate state</li> <li>visibility: Threads reading the state see the most recently written state. Or more formally, reading method calls create a happens-before relation to previous writing method calls.</li> </ol> <h1><a href="#when-is-a-method-quiescent" id="when-is-a-method-quiescent"></a>When is a method quiescent?</h1> <p>The second type of method, quiescent methods, is typically used to collect statistics. Suppose you want to track how often a specific method was called. You have multiple threads incrementing a counter every time they execute a specific method. And when all threads are stopped you collect the result.</p> <p>This is how a quiescent method works.</p> <p>When a quiescent method gets called at a time of quiescent, e.g. no other method calls are pending, it sees the result of all previous method calls.</p> <p>The classes <a href="https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/atomic/DoubleAccumulator.html"><code>java.util.concurrent.atomic.DoubleAccumulator</code></a>, <a href="https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/atomic/DoubleAdder.html"><code>java.util.concurrent.atomic.DoubleAdder</code></a>, <a href="https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/atomic/LongAccumulator.html"><code>java.util.concurrent.atomic.LongAccumulator</code></a> and <a href="https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/atomic/LongAdder.html"><code>java.util.concurrent.atomic.LongAdder</code></a> are examples of classes with quiescent methods.</p> <p>Quiescent consistency was first introduced implicitly in 1994 by James Aspnes, Maurice Herlihy, and Nir Shavit in the paper <a href="https://dl.acm.org/doi/10.1145/185675.185815">Counting networks</a>.</p> <h1><a href="#how-do-those-methods-behave-and-how-to-use-them-correctly" id="how-do-those-methods-behave-and-how-to-use-them-correctly"></a>How do those methods behave and how to use them correctly?</h1> <p>In the beginning, I complained that the typical definition of thread-safety does not tell us how thread-safe classes behave and how we can use them. So what does our new definition tells us about how our class works when called from multiple threads?</p> <p>Both types, atomic and quiescent allows us to map the concurrent method calls to an equivalent sequential one. And instead of reasoning about the concurrent method calls, we can reason about the equivalent sequential method flow.</p> <p>Atomic methods always map to a sequential method flow. But the usage of atomic methods still can lead to a race condition. Typically errors happen when we call multiple non-commutative atomic methods from the same thread. The following method leads to a race condition since the method get and put of <code>java.util.concurrent.ConcurrentHashMap</code> are not commutative:</p> <p><pre class="brush: java">public void update(ConcurrentHashMap&lt;Integer, Integer&gt; map) { Integer result = map.get(1); if(result == null) { map.put(1, 1); } else { map.put(1, result + 1); } }</pre></p> <p>To implement an update without race condition you should use the method compute:</p> <p><pre class="brush: java">public void update(ConcurrentHashMap&lt;Integer,Integer&gt; map) { map.compute(1, (key, value) -&gt; { if (value == null) { return 1; } return value + 1; }); }</pre></p> <p>Quiescent methods on the other side only map to a sequential flow when they are called at a time of quiescent, e.g. no other method calls are pending. So a typical scenario for quiescent methods looks like this:</p> <p><pre class="brush: java">LongAdder longAdder = new LongAdder(); ExecutorService service = Executors.newCachedThreadPool(); service.submit( () -&gt; { longAdder.increment(); } ); service.submit( () -&gt; { longAdder.increment(); } ); service.shutdown(); service.awaitTermination(10, TimeUnit.SECONDS); longAdder.longValue();</pre></p> <h1><a href="#summary" id="summary"></a>Summary</h1> <p>A class is thread-safe if all its public methods are either atomic or quiescent.</p> <p>A method is atomic when the method call appears to take effect instantaneously, at some time between the start and the end of the method call. So other threads either see the state before or after the method call but no intermediate state. Threads reading the state see the most recently written state. Or more formally, reading method calls create a happens-before relation to previous writing method calls.</p> <p>A method is quiescent when it sees the result of all previous method calls at a time of quiescent, e.g. no other method calls are pending.</p> Tue, 19 May 2020 22:00:00 GMT http://vmlens.com/articles/cp/thread_safety_definition 2020-05-19T22:00:00Z The Java Memory Model enables testing of multithreaded Java http://vmlens.com/articles/cp/java_memory_model_enables_tests Testing multithreading Java seams impossible. Bugs depend on the specific timing and sometimes even on a specific processor type or JVM. But Java has a specification that enables us to test multithreaded software: The Java memory model. <p>Testing multithreading Java seams impossible. Bugs depend on the specific timing and sometimes even on a specific processor type or JVM. But Java has a specification that enables us to test multithreaded software: The Java memory model.</p> <p>The Java memory model enables us to execute all thread interleavings of a multithreaded program as long as the program is data race free. And it defines rules to automatically check if a program contains a data race. So we can use a two-step approach for testing: In the first step, we execute all thread interleavings. In the second step, we check if the test contained data races. I have implemented those two steps in a tool called <a href="https://vmlens.com">vmlens</a>.</p> <h1><a href="#what-is-the-java-memory-model" id="what-is-the-java-memory-model"></a>What is the Java memory model?</h1> <p>The Java memory model is a specification to define what happens when multiple threads read and write the same memory location As Hans Boehm, one of the authors of the Java memory model writes:</p> <blockquote> <p><a href="https://cacm.acm.org/magazines/2010/8/96610-memory-models-a-case-for-rethinking-parallel-languages-and-hardware/fulltext">We have been repeatedly surprised at how difficult it is to formalize the seemingly simple and fundamental property of “what value a read should return in a multithreaded program.”</a></p> </blockquote> <p>The Java memory model answers this question in the following way:</p> <blockquote> <p><a href="https://docs.oracle.com/javase/specs/jls/se8/html/jls-17.html#jls-17.4.3">If a program has no data races, then all executions of the program will appear to be sequentially consistent.</a></p> </blockquote> <p>Sequential consistency means that a run of a multi-threaded program is one specific interleaving of the source code statements of the different threads. So to execute a sequential consistent program we can use the following algorithm: Select one thread and execute the current statement of this thread. Repeat this till all threads are terminated.</p> <p>But Java programs are only sequential consistent when they are data race free. A program contains a data race when it contains a read and a write or two writes to the same memory location which are not ordered by the so-called happens-before order. Synchronization actions like the read and write from a volatile field generate an order between multiple threads, the happens-before order. For example the write to a volatile variable happens-before all subsequent volatile reads from this variable. And if all memory accesses can be ordered through this happens-before relation our program is data race free.</p> <p>In the following example, the read and write to the field i are ordered through a happens-before relation. In this interleaving the write to the volatile variable v in thread A comes before the read from this variable in thread B. So the write to the normal field i in the thread A can be ordered with the reading from this field in thread B. And so the following interleaving is data race free:</p> <p><table > <tr> <th>Operation</th> <th>Thread</th> </tr> <tr> <td>write normal field i </td> <td class="text-center" >A</td> </tr> <tr> <td>write volatile field v </td> <td class="text-center" >A</td> </tr> <tr> <td>read volatile field v </td> <td class="text-center" >B</td> </tr> <tr> <td>read normal field i </td> <td class="text-center" >B</td> </tr> </table></p> <p>Compare this to the following interleaving of the same program. Here the read from the field i in thread B can not be ordered with the write to this field in thread A. And so the following interleaving contains a data race.</p> <p><table > <tr> <th>Operation</th> <th>Thread</th> </tr> <tr> <td>read volatile field v</td> <td class="text-center" >B</td> </tr> <tr> <td>read normal field i</td> <td class="text-center" >B</td> </tr> <tr> <td>write normal field i</td> <td class="text-center" >A</td> </tr> <tr> <td>write volatile field v</td> <td class="text-center" >A</td> </tr> </table></p> <p>The weaker form of sequential consistency of the Java memory model has the advantage that it allows the JVM to apply optimizations of the generated machine code. And it makes caching in the processor possible.</p> <p><a href="https://mechanical-sympathy.blogspot.com/2011/07/memory-barriersfences.html">This blog post from Martin Thompson</a> shows how the caching in the CPU works and how caching is related to the Java memory model. And <a href="https://shipilev.net/blog/2014/jmm-pragmatics/">this post from Aleksey Shipilev</a> gives a comprehensive overview of the Java memory model.</p> <p>And this weaker form of sequential consistency means that we only need to consider synchronization actions when we want to test all thread interleavings.</p> <h1><a href="#how-does-the-java-memory-model-enable-the-testing-of-multithreaded-java" id="how-does-the-java-memory-model-enable-the-testing-of-multithreaded-java"></a>How does the Java memory model enable the testing of multithreaded Java?</h1> <p>When testing we only need to consider thread interleavings which lead to a different happen-before order. So we calculate all orders the synchronization actions of the test run can create. And for each order, we execute one thread interleaving which leads to this order.</p> <p>After we executed all those thread interleavings we need to check that the test is data race free. Therefore we check for each multi-threaded memory access if this access can be ordered according to the happens-before order.</p> <p>Testing multi-threaded software by executing all thread interleavings is not new. The tool <a href="https://concuerror.com/">Concuerror</a> implements this approach for the language Erlang, a programming language without shared memory. Finding data races at runtime is also not new. A prominent example is <a href="https://github.com/google/sanitizers/wiki/ThreadSanitizerCppManual">ThreadSanitizer which detects data races in C++ programs</a> and <a href="https://blog.golang.org/race-detector">golang</a> and <a href="https://openjdk.java.net/jeps/8208520">probably at sometime Java</a>.</p> <p>New is the combination of those two techniques in one tool to systematically test multithreaded Java programs.</p> <h1><a href="#a-multithreaded-unit-test" id="a-multithreaded-unit-test"></a>A multithreaded unit test</h1> <p>The following JUnit test implements the examples from above in a real test. One thread reads the variable i and the volatile variable v. The other thread writes to those two variables. To test all thread interleavings we put the complete test in a while loop iterating over all thread interleavings using the class <a href="https://vmlens.com/apidocs/annotation/com/vmlens/api/AllInterleavings.html"><code>com.vmlens.api.AllInterleavings</code></a>.</p> <p>You can download the source code of the example from <a href="https://github.com/vmlens/examples">Github here</a>.</p> <p><pre class="brush: java">import org.junit.Test; import com.vmlens.api.AllInterleavings; public class TestReadWrite { volatile int v = 0; int i = 0; @Test public void testUpdate() throws InterruptedException { try (AllInterleavings allInterleavings = new AllInterleavings(TestReadWrite.class.getName());) { while (allInterleavings.hasNext()) { Thread first = new Thread(() -&gt; { i = 5; v = 7; }); Thread second = new Thread(() -&gt; { int x = v; int y = i; }); first.start(); second.start(); first.join(); second.join(); } } } }</pre></p> <p>vmlens traces all multi-threaded memory accesses and synchronization actions as java agent using byte code transformation. Using this trace vmlens calculates all potential thread interleavings and checks for data races. Running the above test, vmlens reports the following data race.</p> <p><img style="width: 100%; height: auto;" src="../../../img/articles/cp/readWriteRace.png" /></p> <h1><a href="#performance-of-the-multithreaded-unit-tests" id="performance-of-the-multithreaded-unit-tests"></a>Performance of the multithreaded unit tests</h1> <p>Using this two-step approach it is possible to test even complicated data structures like <a href="https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/ConcurrentHashMap.html"><code>java.util.concurrent.ConcurrentHashMap</code></a>. For example, testing put using two threads takes 353 iterations and less than 3 seconds on my Intel i5 3,40 GHz 4 core CPU, see the test <a href="https://github.com/vmlens/examples/blob/master/src/test/java/com/vmlens/examples/javaMemoryModel/TestConcurrentHashMapWithoutAtomicTwoPuts.java"><code>com.vmlens.examples.javaMemoryModel.TestConcurrentHashMapWithoutAtomicTwoPuts</code></a>. Since we always run one thread at a time the performance depends on the CPU clock speed and not on the available cores.</p> <p>But typically we do not write new concurrent data structures. We use existing data structures. Here we can use the fact that most of the methods of concurrent data structures are atomic. And for two atomic methods a and b we only need to test two combinations. The combination a before b and the combination b before a. So we can drastically reduce the iteration count when testing the usage of atomic methods.</p> <h1><a href="#summary" id="summary"></a>Summary</h1> <p>The Java memory model guarantees that all data race free programs are sequentially consistent. This allows us to test multithreaded programs in a two-step approach. In the first step, we execute all thread interleavings and in the second step, we check for data races. This allows us to test multithreaded Java in a systematic, reproducible way.</p> Tue, 05 May 2020 22:00:00 GMT http://vmlens.com/articles/cp/java_memory_model_enables_tests 2020-05-05T22:00:00Z Lambdas for concurrent maps http://vmlens.com/articles/cp/lambdas_for_concurrent_collections The package java.util.concurrent contains two concurrent maps, the class ConcurrentHashMap, and the class ConcurrentSkipListMap. Both classes are thread-safe and high performant. But using them is error-prone because of read modify write race conditions. Here come lambda expressions into the play. <p>The package java.util.concurrent contains two concurrent maps, the class ConcurrentHashMap, and the class ConcurrentSkipListMap. Both classes are thread-safe and high performant. But using them is error-prone because of read modify write race conditions. Here come lambda expressions into the play. Lambda expressions help us to avoid these race conditions elegantly.</p> <p>Let us see how.</p> <h1><a href="#read-modify-write-race-condition" id="read-modify-write-race-condition"></a>Read modify write race condition</h1> <p>The race condition happens when we read an element from the map, modify this element and write the element back into the map. Like in the following example:</p> <p><pre class="brush: java">import com.vmlens.api.AllInterleavings; public class TestUpdateWrong { public void update(ConcurrentHashMap&lt;Integer, Integer&gt; map) { Integer result = map.get(1); if (result == null) { map.put(1, 1); } else { map.put(1, result + 1); } } @Test public void testUpdate() throws InterruptedException { try (AllInterleavings allInterleavings = new AllInterleavings(&quot;TestUpdateWrong&quot;);) { while (allInterleavings.hasNext()) { final ConcurrentHashMap&lt;Integer, Integer&gt; map = new ConcurrentHashMap&lt;Integer, Integer&gt;(); Thread first = new Thread(() -&gt; { update(map); }); Thread second = new Thread(() -&gt; { update(map); }); first.start(); second.start(); first.join(); second.join(); assertEquals(2, map.get(1).intValue()); } } } }</pre></p> <p>You can download the source code of the example from <a href="https://github.com/vmlens/examples">Github here</a>.</p> <p>Here we implement a per-key counter. In the update method, we initialize the count to 1 if no mapping exists otherwise we increment the count by one. To reproduce the race condition we update the map from two different threads. After both threads are stopped we check if the value is indeed two.</p> <p>To test all thread interleavings we put the complete test in a while loop iterating over all thread interleavings using the class <a href="https://vmlens.com/apidocs/annotation/com/vmlens/api/AllInterleavings.html">AllInterleavings</a> from <a href="https://vmlens.com">vmlens</a>, line 15. Running the test we see the following error:</p> <p><pre class="brush: java">java.lang.AssertionError: expected:&lt;2&gt; but was:&lt;1&gt;</pre></p> <p>To see why the result is one and not two as expected we can look at the report vmlens generated:</p> <p><img style="width: 100%; height: auto;" src="../../../img/articles/cp/update_wrong.png" /></p> <p>The problem is that the update method is not atomic. Both threads can read the same. This lets the last thread override the result of the first thread.</p> <h1><a href="#avoiding-read-modify-write-race-condition-with-lambda-expressions" id="avoiding-read-modify-write-race-condition-with-lambda-expressions"></a>Avoiding read modify write race condition with lambda expressions</h1> <p>To avoid this race condition we need a way to execute all three operations, the read, the modification and the write in one atomic method call. The method compute does exactly this using a lambda expression:</p> <p><pre class="brush: java">public void update( ConcurrentHashMap&lt;Integer,Integer&gt; map ) { map.compute(1, (key, value) -&gt; { if (value == null) { return 1; } return value + 1; }); }</pre></p> <p>Now the read modify write operations happen in one atomic method and the race disappears.</p> <h1><a href="#lambdas-should-be-pure" id="lambdas-should-be-pure"></a>Lambdas should be pure</h1> <p>What properties should a lambda function have?</p> <p>The lambda expressions in the ConcurrentHashMap get executed under a synchronized block on a bin entry of the hash map. Therefore you must not call another writing operation of this ConcurrentHashMap instance. Otherwise, this can lead to deadlocks, as shown in this <a href="../concurrentHashMap_one_method/">blog post</a>.</p> <p>When we use a ConcurrentSkipListMap on the other side our lambda expressions might be called multiple times, since this map uses an optimistic concurrency scheme. To not depend on implementation details the general advice is to make lambda expression pure. This means they should have no side effects and simply calculate a new immutable value from a given immutable value.</p> <h1><a href="#conclusion" id="conclusion"></a>Conclusion</h1> <p>Lambda expressions help to avoid read modify write race conditions elegantly. When using lambda expressions the elements in the collection should be immutable. And the lambda function itself should be pure.</p> Wed, 15 Apr 2020 22:00:00 GMT http://vmlens.com/articles/cp/lambdas_for_concurrent_collections 2020-04-15T22:00:00Z Gson an example for a stateless thread-safe utility class http://vmlens.com/articles/cp/how_does_gson_achieve_thread_safety Stateless classes are perfect for multi-threaded programming. They can be called from multiple threads without ever worrying about synchronization of their state since there is none. But how can I write a performant, thread-safe stateless service? <p>Stateless classes are perfect for multi-threaded programming. They can be called from multiple threads without ever worrying about synchronization of their state since there is none. But how can I write a performant, thread-safe stateless class?</p> <p><a href="https://github.com/google/gson">Gson</a>, an open-source library to convert Java Objects into JSON and back from google, is a good example of such a class. Gson uses the following three techniques to implement a performant, thread-safe stateless class:</p> <ol> <li>Stack confinement or create a new instance for every method call</li> <li>Thread-safe caching to reduce the creation time</li> <li>Configuration of the instance at construction time</li> </ol> <p>Let us look at those three techniques in more detail.</p> <h1><a href="#stack-confinement-or-create-a-new-instance-for-every-method-call" id="stack-confinement-or-create-a-new-instance-for-every-method-call"></a>Stack confinement or create a new instance for every method call</h1> <p>Stack confinement is described in the book <a href="https://jcip.net/">Java Concurrency in Practice, by Brian Goetz et al.</a>:</p> <blockquote> <p>If data is only accessed from a single thread, no synchronization is needed. This technique, thread confinement, is one of the simplest ways to achieve thread safety. [...] Stack confinement is a special case of thread confinement in which an object can only be reached through local variables.</p> </blockquote> <p>So the idea is to only use local variables to achieve thread-safety. To do this we create a new instance at the beginning of our method and store this instance in a local variable.</p> <p>The following shows how this is done in Gson:</p> <p><pre class="brush: java">public void toJson(Object src, Type typeOfSrc, Appendable writer) throws JsonIOException { try { JsonWriter jsonWriter = newJsonWriter( Streams.writerForAppendable(writer)); toJson(src, typeOfSrc, jsonWriter); } catch (IOException e) { throw new JsonIOException(e); } }</pre></p> <p>We create a new instance of JsonWriter in the method newJsonWriter for every method call. This instance is only reachable through local variables. So our data is stack confined and therefore thread-safe.</p> <h1><a href="#thread-safe-caching-using-concurrenthashmap" id="thread-safe-caching-using-concurrenthashmap"></a>Thread-safe caching using ConcurrentHashMap</h1> <p>Now since we create a new instance for each method call, the instance creation time becomes performance-critical. To improve this time we use thread-safe caching.</p> <p>We need to things for this: First a thread-safe hash map and second thread-safe cached elements. For the hash map, we use the class java.util.concurrent.ConcurrentHashMap. This class provides a high performant thread-safe hash map. For the elements, we either use immutable classes or we need to synchronize their methods.</p> <p>In Gson the type adapters are cached in a ConcurrentHashMap:</p> <p><pre class="brush: java">private final Map&lt;TypeToken&lt;?&gt;, TypeAdapter&lt;?&gt;&gt; typeTokenCache = new ConcurrentHashMap&lt;TypeToken&lt;?&gt;, TypeAdapter&lt;?&gt;&gt;(); </pre></p> <p>Gson uses both immutable elements like for example <a href="https://github.com/google/gson/blob/master/gson/src/main/java/com/google/gson/internal/bind/ArrayTypeAdapter.java">com.google.gson.internal.bind.ArrayTypeAdapter</a> and synchronized elements like in <a href="https://github.com/google/gson/blob/master/gson/src/main/java/com/google/gson/internal/bind/DateTypeAdapter.java">com.google.gson.internal.bind.DateTypeAdapter</a>.</p> <h1><a href="#configure-the-instance-at-creation-time" id="configure-the-instance-at-creation-time"></a>Configure the instance at creation time</h1> <p>Gson has multiple configuration options. To make the configuration thread-safe it can only be configured at construction time. And to make it easy to use only the default configuration can be constructed by the constructor. For all other configurations, Gson uses the builder pattern. The builder pattern <a href="https://en.wikipedia.org/wiki/Builder_pattern">encapsulates the creation of a complex object in a separate Builder object.</a></p> <p>The following example shows how the GsonBuilder can be used to create a Gson instance with a specific configuration:</p> <p><pre class="brush: java"> Gson gson = new GsonBuilder() .registerTypeAdapter(Id.class, new IdTypeAdapter()) .enableComplexMapKeySerialization() .serializeNulls() .create();</pre></p> <h1><a href="#conclusion" id="conclusion"></a>Conclusion</h1> <p>For me, Gson is a good example of how to implement a stateless thread-safe utility class. Gson shows how to implement a thread-safe class by creating a new instance at each method call. Thereby using stack confinement to achieve thread-safety. It shows how to use caching to reduce the object creation time. And it shows how to configure the instance only at construction time using the builder pattern.</p> Fri, 10 Apr 2020 22:00:00 GMT http://vmlens.com/articles/cp/how_does_gson_achieve_thread_safety 2020-04-10T22:00:00Z ConcurrentHashMap: Call only one method per key http://vmlens.com/articles/cp/concurrentHashMap_one_method Each method of ConcurrentHashMap is thread-safe. But calling multiple methods from ConcurrentHashMap for the same key leads to race conditions. And calling the same method from ConcurrentHashMap recursively for different keys leads to deadlocks. <p>Each method of ConcurrentHashMap is thread-safe. But calling multiple methods from ConcurrentHashMap for the same key leads to race conditions. And calling the same method from ConcurrentHashMap recursively for different keys leads to deadlocks.</p> <p>Let us look at an example to see why this happens:</p> <h1><a href="#calling-multiple-methods" id="calling-multiple-methods"></a>Calling multiple methods</h1> <p>In the following test, I use two methods from ConcurrentHashMap for the same key 1. The method update, line 3 till 10, first gets the value from the ConcurrentHashMap using the method get. Than update increments the value and put it back using the method put, line 6 and 8:</p> <p><pre class="brush: java">import com.vmlens.api.AllInterleavings; public class TestUpdateWrong { public void update(ConcurrentHashMap&lt;Integer, Integer&gt; map) { Integer result = map.get(1); if (result == null) { map.put(1, 1); } else { map.put(1, result + 1); } } @Test public void testUpdate() throws InterruptedException { try (AllInterleavings allInterleavings = new AllInterleavings(&quot;TestUpdateWrong&quot;);) { while (allInterleavings.hasNext()) { final ConcurrentHashMap&lt;Integer, Integer&gt; map = new ConcurrentHashMap&lt;Integer, Integer&gt;(); Thread first = new Thread(() -&gt; { update(map); }); Thread second = new Thread(() -&gt; { update(map); }); first.start(); second.start(); first.join(); second.join(); assertEquals(2, map.get(1).intValue()); } } } }</pre></p> <p>You can download the source code of all examples from <a href="https://github.com/vmlens/examples">Github here</a>.</p> <p>To test what happens I use two threads, line 18 and 21. I start those two threads, line 25 and 25. And then wait till both are ended using thread join, line 26 and 27. After both threads are stopped I check if the value is indeed two, line 28.</p> <p>To test all thread interleavings we put the complete test in a while loop iterating over all thread interleavings using the class AllInterleavings from <a href="https://vmlens.com">vmlens</a>, line 15. Running the test I see the following error:</p> <p><pre class="brush: java">java.lang.AssertionError: expected:&lt;2&gt; but was:&lt;1&gt;</pre></p> <p>To see why the result is one, not two as expected we can look at the report vmlens generated:</p> <p><img style="width: 100%; height: auto;" src="../../../img/articles/cp/update_wrong.png" /></p> <p>So the problem is that first both threads call get and after that both threads call put. So both threads see an empty value and update the value to one. Which leads to a result of one and not as expected two. The trick to solving this race condition is to use only one method instead of two methods to update the value. Using the method compute we can do this. So the correct version looks like this:</p> <p><pre class="brush: java">public void update( ConcurrentHashMap&lt;Integer,Integer&gt; map ) { map.compute(1, (key, value) -&gt; { if (value == null) { return 1; } return value + 1; }); }</pre></p> <h1><a href="#calling-the-same-method-recursively" id="calling-the-same-method-recursively"></a>Calling the same method recursively</h1> <p>Now let us look at an example for calling the same method from ConcurrentHashMap recursively:</p> <p><pre class="brush: java">public class TestUpdateRecursive { private final ConcurrentHashMap&lt;Integer, Integer&gt; map = new ConcurrentHashMap&lt;Integer, Integer&gt;(); public TestUpdateRecursive() { map.put(1, 1); map.put(2, 2); } public void update12() { map.compute(1, (key,value) -&gt; { map.compute(2, ( k , v ) -&gt; { return 2; } ); return 2; }); } public void update21() { map.compute(2, (key,value) -&gt; { map.compute(1, ( k , v ) -&gt; { return 2; } ); return 2; }); } @Test public void testUpdate() throws InterruptedException { Thread first = new Thread(() -&gt; { update12(); }); Thread second = new Thread(() -&gt; { update21(); }); first.start(); second.start(); first.join(); second.join(); } } </pre></p> <p>Here we call the compute method inside of the compute method for different keys. Once for the key one than two and once for the key two than one. If we run the test we see the following deadlock:</p> <p><img style="width: 100%; height: auto;" src="../../../img/articles/cp/deadlockConcurrentHashMap.png" /></p> <p>To understand why this deadlock happens, we have to look at the internals of the ConcurrentHashMap. ConcurrentHashMap uses an array to store the mapping between the keys and the values. Every time we update such a mapping of the ConcurrentHashMap locks the array element in which the mapping is stored. So in our test, the call to compute for the key one locked the array element for the key one. And then we try to lock the array element for the key two. But this key is already locked by the other thread who called compute for key two and tries to lock the array element for the key one. A deadlock.</p> <p>Note that only updates need a lock to an array element. Methods which reads only, like for example get, do not use locks. So it is no problem to use a get method inside a compute call.</p> <h1><a href="#conclusion" id="conclusion"></a>Conclusion</h1> <p>Using the ConcurrentHashMap in a thread-safe way is easy. Select the one method which fits your need. And use it exactly once per key</p> Thu, 16 Jan 2020 23:00:00 GMT http://vmlens.com/articles/cp/concurrentHashMap_one_method 2020-01-16T23:00:00Z How to write thread-safe yet scalable classes? http://vmlens.com/articles/cp/thread_safe_yet_scalable When writing thread-safe classes the main issue is to separate the data into multiple independent parts. And to choose the right size for those parts. If the part is too small our class is not thread-safe. If the part is too large the class is not scalable. <p>When writing thread-safe classes the main issue is to separate the data into multiple independent parts. And to choose the right size for those parts. If the part is too small our class is not thread-safe. If the part is too large the class is not scalable.</p> <p>Let us look at an example which illustrates that point:</p> <h1><a href="#an-example" id="an-example"></a>An example</h1> <p>Suppose we want to track how many people live in a city. We want to support two methods, one to get the current count of people living in a city and one to move a person from one city to another. So we have the following interface:</p> <p><pre class="brush: java">public interface CityToCount { static final String[] ALL_CITIES = new String[] { &quot;Springfield&quot; , &quot;South Park&quot; }; static final int POPULATION_COUNT = 1000000; void move( String from, String to ); int count(String name); }</pre></p> <p>You can download the source code of all examples from <a href="https://github.com/vmlens/tutorial-copy-on-write">GitHub here</a>.</p> <p>Since we want to use this interface from multiple threads in parallel we have to options to implement this interface. Either use the class java.util.concurrent.ConcurrentHashMap or uses the class java.util.HashMap and a single lock. Here is the implementation using the class java.util.concurrent.ConcurrentHashMap:</p> <p><pre class="brush: java">public class CityToCountUsingConcurrentHashMap implements CityToCount { private ConcurrentHashMap&lt;String, Integer&gt; map = new ConcurrentHashMap&lt;String, Integer&gt;(); public CityToCountUsingConcurrentHashMap() { for (String city : ALL_CITIES) { map.put(city, POPULATION_COUNT); } } public void move(String from, String to) { map.compute(from, (key, value) -&gt; { if (value == null) { return POPULATION_COUNT - 1; } return value - 1; }); map.compute(to, (key, value) -&gt; { if (value == null) { return POPULATION_COUNT + 1; } return value + 1; }); } public int count(String name) { return map.get(name); } } </pre></p> <p>The method move uses the thread-safe method compute to decrement the count in the source city. Than compute is used to increment the count in the target city. The count method uses the thread-safe method get.</p> <p>And here is the implementation using the class java.util.HashMap:</p> <p><pre class="brush: java">public class CityToCountUsingSynchronizedHashMap implements CityToCount { private HashMap&lt;String, Integer&gt; map = new HashMap&lt;String, Integer&gt;(); private Object lock = new Object(); public CityToCountUsingSynchronizedHashMap() { for (String city : ALL_CITIES) { map.put(city, POPULATION_COUNT); } } public void move(String from, String to) { synchronized (lock) { map.compute(from, (key, value) -&gt; { if (value == null) { return POPULATION_COUNT - 1; } return value - 1; }); map.compute(to, (key, value) -&gt; { if (value == null) { return POPULATION_COUNT + 1; } return value + 1; }); } } public int count(String name) { synchronized (lock) { return map.get(name); } } }</pre></p> <p>The method move also uses the method compute to increment and decrement the count in the source and target city. Only this time, since the compute method is not thread-safe, both methods are surrounded by a synchronized block. The count method uses the get method again surrounded by a synchronized block.</p> <p>Both solutions are thread-safe.</p> <p>But in the solution using ConcurrentHashMap multiple cities can be updated from different threads in parallel. And in the solution using a HashMap, since the lock is around the complete HashMap, only one thread can update the HashMap at a given time. So the solution using ConcurrentHashMap should be better scalable. Let us see.</p> <h1><a href="#too-large-means-not-scalable" id="too-large-means-not-scalable"></a>Too large means not scalable</h1> <p>To compare the scalability of the two implementations I use the following benchmark:</p> <p><pre class="brush: java">import org.openjdk.jmh.annotations.Benchmark; import org.openjdk.jmh.annotations.State; import org.openjdk.jmh.annotations.Scope; @State(Scope.Benchmark) public class CityToCountBenchmark { public CityToCount cityToCountUsingSynchronizedHashMap = new CityToCountUsingSynchronizedHashMap(); public CityToCount cityToCountUsingConcurrentHashMap = new CityToCountUsingConcurrentHashMap(); @Benchmark public void synchronizedHashMap() { String name = Thread.currentThread().getName(); cityToCountUsingSynchronizedHashMap.move(name, name + &quot;2&quot;); } @Benchmark public void concurrentHashMap() { String name = Thread.currentThread().getName(); cityToCountUsingConcurrentHashMap.move(name, name + &quot;2&quot;); } }</pre></p> <p>The benchmark uses <a href="http://openjdk.java.net/projects/code-tools/jmh/">jmh</a>, an OpenJDK framework for micro-benchmarks. In the benchmark, I move people from one city to another. Each worker thread updates different cities. The name of the source city is simply the thread id and the target city the thread id plus two. I ran the benchmark on an Intel i5 4 core CPU with these results:</p> <p><img style="width: 100%; height: auto;" src="../../../img/articles/cp/timesThreadSafeYetScalable.png" /></p> <p>As we see the solution using ConcurrentHashMap scales better. Starting with two thread it performs better than the solution using a single lock.</p> <h1><a href="#too-small-means-not-thread-safe" id="too-small-means-not-thread-safe"></a>Too small means not thread-safe</h1> <p>Now I want an additional method to get the complete count overall cities. Here is this method for the implementation using the class ConcurrentHashMap:</p> <p><pre class="brush: java">public int completeCount() { int completeCount = 0; for (Integer value : map.values()) { completeCount += value; } return completeCount; }</pre></p> <p>To see if this solution is thread-safe I use the following test:</p> <p><pre class="brush: java">import com.vmlens.api.AllInterleavings; public class TestCompleteCountConcurrentHashMap { @Test public void test() throws InterruptedException { try (AllInterleavings allInterleavings = new AllInterleavings(&quot;TestCompleteCountConcurrentHashMap&quot;);) { while (allInterleavings.hasNext()) { CityToCount cityToCount = new CityToCountUsingConcurrentHashMap(); Thread first = new Thread(() -&gt; { cityToCount.move(&quot;Springfield&quot;, &quot;South Park&quot;); }); first.start(); assertEquals(2 * CityToCount.POPULATION_COUNT, cityToCount.completeCount()); first.join(); } } } }</pre></p> <p>I need two threads to test if the method completeCount is thread-safe. In one thread, I move one person from Springfield to South Park. In the other thread I get the completeCount and check if the result equals the expected result.</p> <p>To test all thread interleavings we put the complete test in a while loop iterating over all thread interleavings using the class AllInterleavings from <a href="https://vmlens.com">vmlens</a>, line 7. Running the test I see the following error:</p> <p><pre class="brush: java">expected:&lt;2000000&gt; but was:&lt;1999999&gt;</pre></p> <p>The vmlens report shows what went wrong:</p> <p><img style="width: 100%; height: auto;" src="../../../img/articles/cp/flowThreadSafeYetScalable.png" /></p> <p>As we see the problem is that the calculation of the complete count is done while the other thread still moves a person from Springfield to South Park. The decrement for Springfield was already executed but not the increment for South Park.</p> <p>By allowing the parallel update of different cities the combination between completeCount and move leads to the wrong results. If we have methods which operate over all cities we need to lock all cities during this method. So to support such a method we need to use the second solution using a single lock. For this solution we can implement a thread-safe countComplete method as shown below:</p> <p><pre class="brush: java">public int completeCount() { synchronized (lock) { int completeCount = 0; for (Integer value : map.values()) { completeCount += value; } return completeCount; } }</pre></p> <h1><a href="#conclusion" id="conclusion"></a>Conclusion</h1> <p>The example surely does not reflect the complexity of your data structure. But what is true in the example is also true in the real world. There is no way to update multiple dependent fields in a thread-safe way except to update them one thread after the other. So the only way to achieve scalability and thread safety is to find independent parts in your data. And then update them in parallel from multiple threads.</p> Thu, 16 Jan 2020 23:00:00 GMT http://vmlens.com/articles/cp/thread_safe_yet_scalable 2020-01-16T23:00:00Z Why are there so many concurrent queues implementations in Java? http://vmlens.com/articles/cp/why_so_many_queues There exist multiple queue implementations in Java. Six alone implementations in the package java.util.concurrent. Why are there so many implementations? For a data structure that is implemented in the single-threaded case as a linked list? <p>There exist multiple queue implementations in Java. Six alone implementations in the package java.util.concurrent. Why are there so many implementations? For a data structure that is implemented in the single-threaded case as a linked list?</p> <p>Concurrent queues let two threads communicate asynchronously. And since this communication is often performance-critical we have multiple implementations optimized for a specific communication pattern. Let us look at those communication patterns in more detail and see which queue is optimized for which pattern.</p> <h1><a href="#unbounded" id="unbounded"></a>Unbounded</h1> <p>The queue implementation for the first communication pattern, the unbounded multi -producer multi-consumer pattern, is similar to the single-threaded linked list implementation. It is implemented through the class java.util.concurrent.ConcurrentLinkedQueue The implementation is based on an algorithm from Michael and Scott from 1996 described in the paper <a href="https://www.cs.rochester.edu/~scott/papers/1996_PODC_queues.pdf">Simple, Fast, and Practical Non-Blocking and Blocking Concurrent Queue Algorithms</a>.</p> <p>It uses a linked list data structure and compare and swap operations to update this data structure. I am not aware of any other implementation for an unbounded queue in Java.</p> <p>Here is an example for its usage offering and polling a message from the queue:</p> <p><pre class="brush: java">Queue&lt;String&gt; queue = new ConcurrentLinkedQueue&lt;String&gt;(); queue.offer(&quot;hello world&quot;); String message = queue.poll(); System.out.println(message); </pre></p> <p>The problem of the unbounded queue is that if the producer side is faster than the consumer the queue grows without limit. Or till the Java Heap is exhausted and we receive an OutOfMemory Exception. This leads us to the second type of concurrent queue, the bounded concurrent queue.</p> <h1><a href="#bounded" id="bounded"></a>Bounded</h1> <p>The package java.util.concurrent contains two implementations for a bounded multi-consumer, multi-producer queue, the class java.util.concurrent.ArrayBlockingQueue and the class java.util.concurrent.LinkedBlockingQueue. The class LinkedBlockingQueue uses a linked list data-structure similar to the class ConcurrentLinkedQueue. But instead of using compare and swap operations it uses locks. The class ArrayBlockingQueue on the other side uses an array to store the messages. And locks to update the array</p> <p>Here is an example for its usage offering and polling a message from the queue:</p> <p><pre class="brush: java">Queue&lt;String&gt; queue = new ArrayBlockingQueue&lt;String&gt;(10); queue.offer(&quot;hello world&quot;); String message = queue.poll(); System.out.println(message); </pre></p> <p>Both queues implement the same interface so you can easily switch between the implementation to see which of the queue performs better in your application.</p> <h1><a href="#extra-bit-of-performance" id="extra-bit-of-performance"></a>Extra bit of performance</h1> <p>The open-source library <a href="https://github.com/JCTools/JCTools">JCTools</a> also contains a bounded multi-consumer, multi-producer queue, the class org.jctools.queues.MpmcArrayQueue. This queue is based on an array similar to the class java.util.concurrent.ArrayBlockingQueue. But instead of using locks it uses compare and swap operations. The class MpmcArrayQueue implements the Queue interface so you can switch implementations to see if this implementation improves the performance in your application.</p> <h1><a href="#single-consumer-and-single-producer-queues" id="single-consumer-and-single-producer-queues"></a>Single consumer and single producer queues.</h1> <p>When only one thread writes to the queue we can avoid the expensive lock or compare and swap operations for writing. The same is true for reading. This is used by JCTools. JCTools provides an implementation for each combination.</p> <p>Here is an example for creating a multi-producer, single-consumer queue using the JCTools factory:</p> <p><pre class="brush: java">Queue&lt;String&gt; queue = QueueFactory.newQueue( ConcurrentQueueSpec.createBoundedMpsc(10));</pre></p> <p>Again all those queue implementations implement the Queue interface. So you can switch implementations to see if those queues improve the performance in your application.</p> <h1><a href="#avoiding-memory-creation" id="avoiding-memory-creation"></a>Avoiding memory creation</h1> <p>If we use a bounded queue backed by an array we can reuse the events after they were consumed. This idea is implemented by the <a href="https://lmax-exchange.github.io/disruptor/">LMAX Disruptor</a>. Here you use an array pre-initialized with events. Instead of creating a new event you use one of the already created events. The LMAX Disruptor does not implement the Queue interface so switching implementations is harder. But if you need to avoid garbage collections it is worth a look.</p> <h1><a href="#still-more-queues" id="still-more-queues"></a>Still more queues</h1> <p>So far we have seen three queue implementations from the package java.util.concurrent. Here are the three still missing implementations:</p> <dl> <dt><strong>java.util.concurrent.LinkedTransferQueue</strong></dt> <dd>LinkedTransferQueue lets the producer optionally wait till a consumer has consumed its element. [<q>A TransferQueue may be useful for example in message passing applications in which producers sometimes (using method transfer(E)) await receipt of elements by consumers invoking take or poll, while at other times enqueue elements (via method put) without waiting for receipt. </q>](https://docs.oracle.com/javase/7/docs/api/java/util/concurrent/TransferQueue.html)</dd> <dt><strong>java.util.concurrent.SynchronousQueue</strong></dt> <dd>A queue where every producer waits until its element was consumed by a corresponding consumer.</dd> <dt><strong>java.util.concurrent.DelayQueue</strong></dt> <dd>A queue whose element can only be consumed when the elements are expired.</dd> </dl> <h1><a href="#conclusion" id="conclusion"></a>Conclusion</h1> <p>So why are there so many concurrent queues implementations in Java? First Java is a pretty mature programming language. And second, there are many thread to thread communication patterns. And to achieve the maximum performance you need a data structure optimized for this special communication pattern.</p> Thu, 09 Jan 2020 23:00:00 GMT http://vmlens.com/articles/cp/why_so_many_queues 2020-01-09T23:00:00Z Java Concurrency: AtomicReference http://vmlens.com/articles/cp/atomic_reference java.util.concurrent.atomic.AtomicReference is a class designed to update variables in a thread-safe way. Why do we need the class AtomicReference? Why can we not simply use a volatile variable? And how to use it correctly? <p>java.util.concurrent.atomic.AtomicReference is a class designed to update variables in a thread-safe way. Why do we need the class AtomicReference? Why can we not simply use a volatile variable? And how to use it correctly?</p> <h1><a href="#why-atomicreference" id="why-atomicreference"></a>Why AtomicReference</h1> <p>For the tool I am writing I need to detect if an object was called from multiple threads. I use the following immutable class for this:</p> <p><pre class="brush: java">public class State { private final Thread thread; private final boolean accessedByMultipleThreads; public State(Thread thread, boolean accessedByMultipleThreads) { super(); this.thread = thread; this.accessedByMultipleThreads = accessedByMultipleThreads; } public State() { super(); this.thread = null; this.accessedByMultipleThreads = false; } public State update() { if(accessedByMultipleThreads) { return this; } if( thread == null ) { return new State(Thread.currentThread() , accessedByMultipleThreads); } if(thread != Thread.currentThread()) { return new State(null,true); } return this; } public boolean isAccessedByMultipleThreads() { return accessedByMultipleThreads; } }</pre></p> <p>You can download the source code of all examples from <a href="https://github.com/vmlens/tutorial-atomic-reference">GitHub here</a>.</p> <p>I store the first thread accessing an object in the variable thread, line 2. When another thread accesses the object I set the variable accessedByMultipleThreads to true and the variable thread to null, line 23. When the variable accessedByMultipleThreads is true I do not change the state, line 15 till 17.</p> <p>I use this class in every object to detect if it was accessed by multiple threads. The following example uses the state in the class UpdateStateNotThreadSafe:</p> <p><pre class="brush: java">public class UpdateStateNotThreadSafe { private volatile State state = new State(); public void update() { state = state.update(); } public State getState() { return state; } }</pre></p> <p>I store the state in the volatile variable state, line 2. I need the volatile keyword to make sure that the threads always see the current values, as explained in <a href="http://vmlens.com/articles/cp/why_volatile/">greater detail here</a>.</p> <p>To check if using a volatile variable is thread-safe I use the following test:</p> <p><pre class="brush: java">import com.vmlens.api.AllInterleavings; public class TestNotThreadSafe { @Test public void test() throws InterruptedException { try (AllInterleavings allInterleavings = new AllInterleavings(&quot;TestNotThreadSafe&quot;);) { while (allInterleavings.hasNext()) { final UpdateStateNotThreadSafe object = new UpdateStateNotThreadSafe(); Thread first = new Thread( () -&gt; { object.update(); } ) ; Thread second = new Thread( () -&gt; { object.update(); } ) ; first.start(); second.start(); first.join(); second.join(); assertTrue( object.getState().isAccessedByMultipleThreads() ); } } } }</pre></p> <p>I need two threads to test if using a volatile variable is thread-safe, created in line 9 and 10. I start those two threads, line 11 and 12. And then wait till both are ended using thread join, line 13 and 14. After both threads are stopped I check if the flag accessedByMultipleThreads is true, line 15.</p> <p>To test all thread interleavings we put the complete test in a while loop iterating over all thread interleavings using the class AllInterleavings from <a href="https://vmlens.com">vmlens</a>, line 7. Running the test I see the following error:</p> <p><pre class="brush: java">java.lang.AssertionError: at org.junit.Assert.fail(Assert.java:91) at org.junit.Assert.assertTrue(Assert.java:43) at org.junit.Assert.assertTrue(Assert.java:54)</pre></p> <p>The vmlens report shows what went wrong:</p> <p><img style="width: 100%; height: auto;" src="../../../img/articles/cp/incorectUpdate.png" /></p> <p>The problem is that for a specific thread interleaving both threads first read the state. So one thread overwrites the result of the other thread.</p> <h1><a href="#how-to-use-atomicreference" id="how-to-use-atomicreference"></a>How to use AtomicReference?</h1> <p>To solve this race condition I use the compareAndSet method from AtomicReference.</p> <p>The compareAndSet method takes two parameters, the expected current value, and the new value. The method atomically checks if the current value equals the expected value. If yes the method updates the value to the new value and return true. If not the method leaves the current value unchanged and returns false.</p> <p>The idea to use this method is to let compareAndSet check if the current value was changed by another thread while we calculated the new value. If not we can safely update the current value. Otherwise, we need to recalculate the new value with the changed current value.</p> <p>The following shows how to use the compareAndSet method to atomically update the state:</p> <p><pre class="brush: java">public class UpdateStateWithCompareAndSet { private final AtomicReference&lt;State&gt; state = new AtomicReference&lt;State&gt;(new State()); public void update() { State current = state.get(); State newValue = current.update(); while( ! state.compareAndSet( current , newValue ) ) { current = state.get(); newValue = current.update(); } } public State getState() { return state.get(); } }</pre></p> <p>I now use an AtomicReference for the state, line 2. To update the state I first need to get the current value, line 5. Then I calculate the new value, line 6 and try to update the AtomicReference using compareAndSet, line 7. If the update succeeds I am done. If not I need to get the current value again, line 8 and recalculate the new value, line 9. Then I can try again to update the AtomicReference using compareAndSet. I need a while loop since the compareAndSet might fail multiple times.</p> <p>As Grzegorz Borczuch pointed out in a comment to this article there is since JDK 1.8 an easier to use method in AtomicReference which achieves the same result: updateAndGet. This method internally uses compareAndSet using a while loop to update the AtomicReference.</p> <h1><a href="#conclusion" id="conclusion"></a>Conclusion</h1> <p>Using volatile variables lead to race conditions since for specific thread interleavings a thread overwrites the computation of the other threads. By using the compareAndSet method from the class AtomicReference we can circumvent this race condition. We atomically check if the current value is still the same as when we started the computation. If yes we can safely update the current value. Otherwise, we need to recalculate the new value with the changed current value.</p> Mon, 06 Jan 2020 23:00:00 GMT http://vmlens.com/articles/cp/atomic_reference 2020-01-06T23:00:00Z How to test if a class is thread-safe in Java? http://vmlens.com/articles/ct/unit_test_tutorial Tests for thread safety differ from typical single-threaded tests. To test if a method is thread-safe we need to call the method in parallel from multiple threads. We need to do this for all potential thread interleavings. And afterward, we need to check if the result is correct. <p>Tests for thread safety differ from typical single-threaded tests. To test if a method is thread-safe we need to call the method in parallel from multiple threads. We need to do this for all potential thread interleavings. And afterward, we need to check if the result is correct.</p> <p>Those three requirements for our test lead to a special type of tests for thread safety which differ from typical single-threaded tests. Since we want to test all thread interleavings our test must be repeatable and run automatically. And since the methods run in parallel the potential result is a combination of different outcomes.</p> <p>Let us look at an example to see how this looks in practice.</p> <h1><a href="#a-test-for-thread-safety" id="a-test-for-thread-safety"></a>A test for thread safety</h1> <p>Suppose we want to test if the following class representing an Address is thread-safe. It offers one method to update the street and city, the method update and one method to read the complete Address, the method toString:</p> <p><pre class="brush: java">public class MutableAddress { private volatile String street; private volatile String city; private volatile String phoneNumber; public MutableAddress(String street, String city, String phoneNumber) { this.street = street; this.city = city; this.phoneNumber = phoneNumber; } public void update(String street ,String city ) { this.street = street; this.city = city; } public String toString() { return &quot;street=&quot; + street + &quot;,city=&quot; + city + &quot;, phoneNumber=&quot; + phoneNumber; } }</pre></p> <p>I use volatile fields, line 2 till 4, to make sure that the threads always see the current values, as explained in <a href="https://vmlens.com/articles/cp/why_volatile/">greater detail here</a>. You can download the source code of all examples from <a href="https://github.com/vmlens/tutorial-copy-on-write">GitHub here</a>.</p> <p>Now let us first see if the combination of toString and update is thread-safe. Here is the test:</p> <p><pre class="brush: java">import com.vmlens.api.AllInterleavings; public class TestToStringAndUpdate { @Test public void testMutableAddress() throws InterruptedException { try (AllInterleavings allInterleavings = new AllInterleavings(&quot;TestToStringAndUpdate_Not_Thread_Safe&quot;);) { while (allInterleavings.hasNext()) { MutableAddress address = new MutableAddress(&quot;E. Bonanza St.&quot;, &quot;South Park&quot;, &quot;456 77 99&quot;); String readAddress = null; Thread first = new Thread(() -&gt; { address.update(&quot;Evergreen Terrace&quot;, &quot;Springfield&quot;); }); first.start(); readAddress = address.toString(); first.join(); assertTrue(&quot;readAddress:&quot; + readAddress,readAddress.equals( &quot;street=E. Bonanza St.,city=South Park,phoneNumber=456 77 99&quot;) || readAddress.equals( &quot;street=Evergreen Terrace,city=Springfield,phoneNumber=456 77 99&quot;)); } } } }</pre></p> <p>The test executes the two methods in parallel from two threads. To test all thread interleavings we put the complete test in a while loop iterating over all thread interleavings using the class AllInterleavings from <a href="https://vmlens.com">vmlens</a>, line 7. To see if the class is thread-safe we compare the result against the to potential outcomes, the value before the update and after the update, lines 17 till 20.</p> <p>Running the test leads to the following error:</p> <p><pre class="brush: java">java.lang.AssertionError: readAddress:street=Evergreen Terrace ,city=South Park,phoneNumber=456 77 99 at com.vmlens.tutorialCopyOnWrite.TestToStringAndUpdate. testMutableAddress(TestToStringAndUpdate.java:22)</pre></p> <p>To see what went wrong we look at the report vmlens generated.</p> <p><img style="width: 100%; height: auto;" src="../../../img/articles/ct/unit_tests.png" /></p> <p>The problem is that for one thread interleaving the thread with Thread id 30 first updates the street name and then the main thread, thread id 1, reads the street and city name. So the main thread reads a partial updated address which leads to the error.</p> <p>To make the address class thread-safe we copy the address value every time we update the address. Here is a thread-safe implementation using this technique. It consists of two classes, an immutable value, and a mutable container.</p> <p>First the immutable value class:</p> <p><pre class="brush: java">public class AddressValue { private final String street; private final String city; private final String phoneNumber; public AddressValue(String street, String city, String phoneNumber) { super(); this.street = street; this.city = city; this.phoneNumber = phoneNumber; } public String getStreet() { return street; } public String getCity() { return city; } public String getPhoneNumber() { return phoneNumber; } }</pre></p> <p>Second the mutable container class:</p> <p><pre class="brush: java">public class AddressUsingCopyOnWrite { private volatile AddressValue addressValue; private final Object LOCK = new Object(); public AddressUsingCopyOnWrite(String street, String city, String phone) { this.addressValue = new AddressValue( street, city, phone); } public void update(String street ,String city ) { synchronized(LOCK){ addressValue = new AddressValue( street, city, addressValue.getPhoneNumber() ); } } public String toString() { AddressValue local = addressValue; return &quot;street=&quot; + local.getStreet() + &quot;,city=&quot; + local.getCity() + &quot;,phoneNumber=&quot; + local.getPhoneNumber(); } }</pre></p> <p>The class AddressUsingCopyOnWrite creates a new address value every time it updates the variable addressValue. This makes sure that we always read a consistent address, either the value before or after the update.</p> <p>If we run the test with those two classes, the test succeeds.</p> <h1><a href="#what-do-we-need-to-test" id="what-do-we-need-to-test"></a>What do we need to test?</h1> <p>So far we tested the combination of toString and update for thread safety. To test if a class is thread-safe we need to test all combinations of modifying methods and all combinations of read-only methods together with modifying methods. So for our example class, we need to test the following two combinations:</p> <ol> <li>update and update</li> <li>toString and update</li> </ol> <p>Since the combinations of read-only methods are automatically thread-safe we do not need to test the combination of the method toString with itself.</p> <h1><a href="#data-races" id="data-races"></a>Data Races</h1> <p>So far we used volatile fields to avoid data races. Let us see what happens when we use normal fields instead. So in our thread-safe class AddressUsingCopyOnWrite we remove the volatile modifier and re-run our test. Now, vmlens reports a data race in the file target/interleave/issues.html</p> <p>A data race is an access to a field where a thread might read a stale value. If the thread indeed reads a stale value depends on external factors like which optimizations the compiler is using or on which hardware architecture the JVM is running and on which cores the threads are running. To make it possible to always detect such a data race independent of those external factors, vmlens searches for data races in the execution trace of the test run. And if vmlens have found one as in the example it reports them in the issue report.</p> <h1><a href="#summary" id="summary"></a>Summary</h1> <p>Tests for thread safety differ from typical single-threaded tests. To test if the combination of two methods, a and b, is thread-safe call them from two different threads. Put the complete test in a while loop iterating over all thread interleavings with the help from the class AllInterleavings from <a href="https://vmlens.com">vmlens</a>. Test if the result is either a after b or b after a. And to test if a class is a thread-safe test all combinations of modifying methods and all combinations of read-only methods together with modifying methods.</p> Thu, 12 Dec 2019 23:00:00 GMT http://vmlens.com/articles/ct/unit_test_tutorial 2019-12-12T23:00:00Z What does thread safety mean in Java? http://vmlens.com/articles/cp/java_thread_safety Thread safety in java means that the methods of a class are either atomic or quiescent. So what does atomic and what does quiescent mean? And why are there no other types of thread-safe methods in java? <p>Thread safety in java means that the methods of a class are either atomic or quiescent. So what does atomic and what does quiescent mean? And why are there no other types of thread-safe methods in java?</p> <h1><a href="#meaning-of-atomic" id="meaning-of-atomic"></a>Meaning of Atomic</h1> <p>A method is atomic when the method call appears to take effect instantaneously. So other threads either see the state before or after the method call but no intermediate state. Let us look at a non-atomic method to see how an atomic method makes a class thread-safe. You can download the source code of all examples from <a href="https://github.com/vmlens/tutorial-thread-safe">GitHub here</a>.</p> <p><pre class="brush: java">public class UniqueIdNotAtomic { private volatile long counter = 0; public long nextId() { return counter++; } }</pre></p> <p>The class UniqueIdNotAtomic creates unique ids by using the volatile variable counter. I use a volatile field, line 2, to make sure that the threads always see the current values, as explained in <a href="https://vmlens.com/articles/cp/why_volatile/">greater detail here</a>. To see if this class is thread-safe we use the following test:</p> <p><pre class="brush: java">public class TestUniqueIdNotAtomic { private final UniqueIdNotAtomic uniqueId = new UniqueIdNotAtomic(); private long firstId; private long secondId; private void updateFirstId() { firstId = uniqueId.nextId(); } private void updateSecondId() { secondId = uniqueId.nextId(); } @Test public void testUniqueId() throws InterruptedException { try (AllInterleavings allInterleavings = new AllInterleavings(&quot;TestUniqueIdNotAtomic&quot;);) { while(allInterleavings.hasNext()) { Thread first = new Thread( () -&gt; { updateFirstId(); } ) ; Thread second = new Thread( () -&gt; { updateSecondId(); } ) ; first.start(); second.start(); first.join(); second.join(); assertTrue( firstId != secondId ); } } } }</pre></p> <p>To test if the counter is thread-safe we need two threads, created in lines 16 and 17. We start those two threads, lines 18 and 19. And then wait till both are ended using thread join, lines 20 and 21. After both threads are stopped we check if the two ids are unique line 22. To test all thread interleavings we put the complete test in a while loop iterating over all thread interleavings using the class AllInterleavings from <a href="https://vmlens.com">vmlens</a>, line 15.</p> <p>Running the test we see the following error:</p> <p><pre class="brush: java">java.lang.AssertionError: at org.junit.Assert.fail(Assert.java:91) at org.junit.Assert.assertTrue(Assert.java:43)</pre></p> <p>The reason for the error is that since the operation ++ is not atomic the two threads can override the result of the other thread. We can see this in the report from vmlens:</p> <p><img style="width: 100%; height: auto;" src="../../../img/articles/cp/uniqueIdError.png" /></p> <p>In the case of the error, both threads first read the variable counter in parallel. And then both create the same id. To solve this problem we make the method atomic by using a synchronized block:</p> <p><pre class="brush: java">private final Object LOCK = new Object(); public long nextId() { synchronized(LOCK) { return counter++; } }</pre></p> <p>Now the method is atomic. The synchronized block makes sure that other threads can not see the intermediate state of the method.</p> <p>Methods which do not access shared state are automatically atomic. The same is true for classes with read-only state. Therefore stateless and immutable classes are an easy way to implement thread-safe classes. All their methods are automatically atomic.</p> <p>Not all usages of atomic methods are automatically thread safe. Combining multiple atomic methods for the same values typical leads to race conditions. Let us look at the atomic method get and put from ConcurrentHashMap to see why. Let us use those methods to insert a value in the map when no previous mapping exists:</p> <p><pre class="brush: java">public class TestUpdateTwoAtomicMethods { public void update(ConcurrentHashMap&lt;Integer,Integer&gt; map) { Integer result = map.get(1); if( result == null ) { map.put(1, 1); } else { map.put(1, result + 1 ); } } @Test public void testUpdate() throws InterruptedException { try (AllInterleavings allInterleavings = new AllInterleavings(&quot;TestUpdateTwoAtomicMethods&quot;);) { while(allInterleavings.hasNext()) { final ConcurrentHashMap&lt;Integer,Integer&gt; map = new ConcurrentHashMap&lt;Integer,Integer&gt;(); Thread first = new Thread( () -&gt; { update(map); } ) ; Thread second = new Thread( () -&gt; { update(map); } ) ; first.start(); second.start(); first.join(); second.join(); assertEquals( 2 , map.get(1).intValue() ); } } } }</pre></p> <p>The test is similar to the previous test. Again we use two threads, to test if our method is thread-safe, lines 18 and 19. An again we test after both threads finished if the result is correct, line 24. Running the test we see the following error:</p> <p><pre class="brush: java">java.lang.AssertionError: expected:&lt;2&gt; but was:&lt;1&gt; at org.junit.Assert.fail(Assert.java:91) at org.junit.Assert.failNotEquals(Assert.java:645)</pre></p> <p>The reason for the error is that the combination of the two atomic methods, get and put is not atomic. So the two threads can override the result of the other thread. We can see this in the report from vmlens:</p> <p><img style="width: 100%; height: auto;" src="../../../img/articles/cp/nonAtomicUpdate.png" /></p> <p>In the case of the error, both threads first get the value in parallel. And then both create the same value and put it into the map. To solve this race condition we need to use one method instead of two. In our case we can use the single method compute instead of the two methods get and put:</p> <p><pre class="brush: java">public void update() { map.compute(1, (key, value) -&gt; { if (value == null) { return 1; } return value + 1; }); }</pre></p> <p>This solves the race condition since the method compute is atomic. While all operations which operate on the same element of ConcurrentHashMap are atomic, operations that operate on the complete map like size are quiescent. So let us see what quiescent means.</p> <h1><a href="#meaning-of-quiescent" id="meaning-of-quiescent"></a>Meaning of Quiescent</h1> <p>Quiescent means that we need to make sure that no other methods are currently running when we call the quiescent method. The following example shows how to use the quiescent method size of the ConcurrentHashMap:</p> <p><pre class="brush: java">ConcurrentHashMap&lt;Integer,Integer&gt; map = new ConcurrentHashMap&lt;Integer,Integer&gt;(); Thread first = new Thread(() -&gt; { map.put(1,1);}); Thread second = new Thread(() -&gt; { map.put(2,2);}); first.start(); second.start(); first.join(); second.join(); assertEquals( 2 , map.size());</pre></p> <p>By waiting till all threads are finished using thread join, we make sure that no other threads are accessing the ConcurrentHashMap when we call the method size.</p> <p>The method size uses a mechanism also used in the class java.util.concurrent.atomic.LongAdder, LongAccumulator, DoubleAdder, and DoubleAccumulator to avoid contention. Instead of using a single variable for storing the current size it uses an array. Different threads update different parts of the array thereby avoiding contention. The algorithm is explained in more <a href="http://hg.openjdk.java.net/jdk9/jdk9/jdk/file/f398670f3da7/src/java.base/share/classes/java/util/concurrent/atomic/Striped64.java">detail in the java doc of Striped64</a></p> <p>The quiescent classes and methods are useful for collecting statistics under high contention. After you collected the data you can use a single thread to evaluate the collected statistics.</p> <h1><a href="#why-no-other-types-of-thread-safe-methods-in-java" id="why-no-other-types-of-thread-safe-methods-in-java"></a>Why no other types of thread safe methods in java?</h1> <p>In theoretical computer science, thread safety means that a data structure full fills a correctness criterion. The most common used correctness criterion is linearizable, which means that the methods of the data structure are atomic. For common data structures exists a provable linearizable concurrent data structures, see the book <a href="https://www.elsevier.com/books/the-art-of-multiprocessor-programming-revised-reprint/herlihy/978-0-12-397337-5">The Art of multiprocessor programming by Maurice Herlihy and Nir Shavit</a>. But to make a data structure linearizable an expensive synchronization mechanism like compare and swap is needed, see the paper <a href="https://www.cs.bgu.ac.il/~hendlerd/papers/p168-expensiveSynch.pdf">Laws of Order: Expensive Synchronization in Concurrent Algorithms Cannot be Eliminated</a>.</p> <p>Therefore other correctness criteria like quiescent are investigated. So I think the question is not, why are there no other types of thread-safe methods in java but rather when will there be other types of thread safety available in java.</p> <h1><a href="#conclusion" id="conclusion"></a>Conclusion</h1> <p>Thread safety in java means that the methods of a class are either atomic or quiescent. A method is atomic when the method call appears to take effect instantaneously. Quiescent means that we need to make sure that no other methods are currently running when we call the quiescent method.</p> <p>Currently, quiescent methods are only used to collect statistics like the size of the concurrentHashMap. For all other use cases atomic methods are used. Let us see if the future brings other types of thread-safe methods.</p> Tue, 26 Nov 2019 23:00:00 GMT http://vmlens.com/articles/cp/java_thread_safety 2019-11-26T23:00:00Z How are multi-threaded tests in Java possible? http://vmlens.com/articles/ct/java_memory_model_enables_tests Testing multi-threaded software is hard since bugs depend on specific thread interleaving. So to write meaningful tests we need a way to test all thread interleavings. Here comes the Java Memory Model into the play. The Java Memory Model not only enables us to write multi-threaded software but also allows us to write a test for this multi-threaded software. <p>How are multi-threaded tests in Java possible?</p> <p>Testing multi-threades software seams impossible. Bugs depend on a specific timing of events, sometimes also on specific compiler or microprocessor optimizations. So how is it possible to write tests for multi-threaded software in Java?</p> <p>Here comes the Java Memory Model into the play. The Java Memory Model is part of the specification of the Java Virtual Machine. It is described here.</p> <p>The Java Memory Model defines rules to correctly synchronize our application. As long as we follow those rules our application behaves the same on any plattform a Java Virtual Machine is running. So basically it allows us to write multi-threaded software which is plattform independent.</p> <p>But not only this. It also allows us to write a test for this multi-threaded software.</p> <p>The Java Memory Model defines synchronization actions like locking a monitor or reading and writing of a volatile variable. And it guarantees that if all reads and writes to shared variables can be ordered through those synchronization actions <q>there is a total order over all individual actions (such as reads and writes) which is consistent with the order of the program, and each individual action is atomic and is immediately visible to every thread.</q></p> <p>On the other side, if the reads and writes to shared variables can not be ordered, a data race exists in the program. A data race means that the order is not defined by the Java Memory Model. Components like the compiler or the CPU can reorder the reads and writes in nonintuitive ways and the threads can read stale values.</p> <p>The idea to test multi-threaded software is first to check for those data races. If such data race exists, the test contains an error. And second test all relations of synchronization actions. For example if we have one thread reading and on thread writing to the same volatile variable. Then we must execute a test run where first the reading thread reads and then the writing thread writes, and a test run where first the writing thread writes and then the reading thread reads.</p> <h1><a href="#how-to-test-all-thread-interleavings" id="how-to-test-all-thread-interleavings"></a>How to test all thread interleavings?</h1> <p>So to test all interleavings in a systematic way we need to:</p> <ol> <li>For each related sync action a and b create a run with a before b and b before a. Two synchronization actions are related if they can create a synchronized-with relation defined by the JavaMemory Model. For example, a write to a volatile variable creates a synchronized-with relation with a subsequent read from this variable. So the read and the write to the same volatile variable are related.</li> <li>Check for data races and flag them as error</li> </ol> <p>Let us look at an example to see how this works in practice. Let us use one thread, thread A, to write to volatile variable v and one thread, thread B, to read from this volatile variable. We have the following two thread interleavings:</p> <p>First Thread A than B <ul style="list-style-type:none;"></p> <li>Thread A writes to v</li> <li>Thread B reads from v</li> </ul> And second, thread B than A: <ul style="list-style-type:none;"> <li>Thread B reads from v</li> <li>Thread A writes to v</li> </ul> <p>Only the first interleaving, thread A writes than thread B reads, creates a synchronized with relation. If we add a read and write of a normal variable we will see that the second interleaving leads to data races.</p> <p>Let us see how this looks like by letting Thread A write to the normal field n before writing to the volatile variable v. And let thread B first write from the volatile variable v and then read from the normal field n. Now we have the following four potential thread interleavings:</p> <p>First: <ul style="list-style-type:none;"></p> <li>Thread A writes to n</li> <li>Thread A writes to v</li> <li>Thread B reads from v</li> <li>Thread B reads from n</li> </ul> Second: <ul style="list-style-type:none;"> <li>Thread B reads from v</li> <li>Thread A writes to n</li> <li>Thread A writes to v</li> <li>Thread B reads from n</li> </ul> Third: <ul style="list-style-type:none;"> <li>Thread A writes to n</li> <li>Thread B reads from v</li> <li>Thread A writes to v</li> <li>Thread B reads from n</li> </ul> Fourth: <ul style="list-style-type:none;"> <li>Thread A writes to n</li> <li>Thread B reads from v</li> <li>Thread B reads from n</li> <li>Thread A writes to v</li> </ul> <p>Only the first interleaving is data race free.</p> <p>The example shows to find all data races we need to test all relations between related sync actions. So the two parts, testing of all sync action and checking for data races depend on each other. To find all data races we need to create runs for every combination of related sync actions. And to make sure that our test and thereby our program does not depend on external components like the uses compiler or CPU our program must be data race free.</p> <h1><a href="#how-to-write-such-tests" id="how-to-write-such-tests"></a>How to write such tests?</h1> <p>The above described two steps to test multi-threaded software are implemented in vmlens. vmlens traces a test using a java agent with bytecode transformation.<br /> It records each synchronization action and memory access to find data races. And it calculates the different thread interleavings using the recorded synchronization actions. To enforce a specific thread interleaving vmlens inserts wait instruction in to the tested bytecode. In practice a test with vmlens looks like this:</p> <h1><a href="#summary" id="summary"></a>Summary</h1> <p>The Java Memory Model allows us to write a test for this multi-threaded software. It defines which thread interleavings exist and gives us a way to create all different thread interleavings.</p> Mon, 25 Nov 2019 23:00:00 GMT http://vmlens.com/articles/ct/java_memory_model_enables_tests 2019-11-25T23:00:00Z Java Concurrency: Copy On Write http://vmlens.com/articles/cp/copy_on_write Copy on write is a technique which allows you to update a data structure in a thread-safe way. The main advantage of copy on write is that reading threads get never blocked. Why do we need this technique? And how to use this technique correctly? <p>Copy on write is a technique which allows you to update a data structure in a thread-safe way. The main advantage of copy on write is that reading threads get never blocked.</p> <p>Why do we need this technique? And how to use this technique correctly?</p> <h1><a href="#why-copy-on-write" id="why-copy-on-write"></a>Why copy on write?</h1> <p>In the following, I want to implement a thread-safe class representing an address. To make the example short the address consists only of the street, the city, and the phone number:</p> <p>You can download the source code of all examples from <a href="https://github.com/vmlens/tutorial-copy-on-write">GitHub here</a>.</p> <p><pre class="brush: java">public class MutableAddress { private volatile String street; private volatile String city; private volatile String phoneNumber; public MutableAddress(String street, String city, String phoneNumber) { this.street = street; this.city = city; this.phoneNumber = phoneNumber; } public String getStreet() { return street; } public String getCity() { return city; } public void updatePostalAddress(String street ,String city ) { this.street = street; this.city = city; } @Override public String toString() { return &quot;street=&quot; + street + &quot;,city=&quot; + city + &quot;,phoneNumber=&quot; + phoneNumber; } }</pre></p> <p>I use volatile fields, line 2 till 4, to make sure that the threads always see the current values, as explained in <a href="http://vmlens.com/articles/cp/why_volatile/">greater detail here</a>.</p> <p>To check if this class is thread-safe I use the following test:</p> <p><pre class="brush: java">public class ConcurrencyTestReadWrite { private final MutableAddress address = new MutableAddress(&quot;E. Bonanza St.&quot; , &quot;South Park&quot; , &quot;456 77 99&quot;); private String readAddress; @Interleave(ConcurrencyTestReadWrite.class) private void updatePostalAddress() { address.updatePostalAddress(&quot;Evergreen Terrace&quot; , &quot;Springfield&quot;); } @Interleave(ConcurrencyTestReadWrite.class) private void read() { readAddress = address.toString(); } @Test public void test() throws InterruptedException { Thread first = new Thread( () -&gt; { updatePostalAddress(); } ) ; Thread second = new Thread( () -&gt; { read(); } ) ; first.start(); second.start(); first.join(); second.join(); assertTrue( &quot;readAddress:&quot; + readAddress , readAddress.equals( &quot;street=E. Bonanza St.,city=South Park,phoneNumber=456 77 99&quot;) || readAddress.equals( &quot;street=Evergreen Terrace,city=Springfield,phoneNumber=456 77 99&quot;) ); } }</pre></p> <p>I need two threads to test if the class is thread-safe, created in line 15 and 16. I start those two threads, line 17 and 18. And then wait till both are ended using thread join, line 19 and 20. After both threads are stopped I check if the read address equals either the value before or after the update, line 21 till 25.</p> <p>To test all thread interleavings I use the annotation Interleave, line 5 and 9, from <a href="http://vmlens.com">vmlens</a>. The Interleave annotation tells vmlens to test all thread interleavings for the annotated method. Running the test we see the following error:</p> <p><pre class="brush: java">java.lang.AssertionError: readAddress: street=Evergreen Terrace,city=South Park,phoneNumber=456 77 99</pre></p> <p>We read a mixture between the initial address, e.g. the city South Park and the updated address e.g. the street Evergreen Terrace. To see what went wrong let us look at the vmlens report:</p> <p><img style="width: 100%; height: auto;" src="../../../img/articles/cp/readWrite.png" /></p> <p>So first the writing thread, thread id 13, updates the street. Then the reading thread, thread id 14, reads the street, city and phone number. Thereby reading the already updated street but the initial city.</p> <h1><a href="#copy-on-write" id="copy-on-write"></a>Copy on write</h1> <p>To solve this bug I use the copy on write technique. The idea is to create a new copy of the object when writing. Then change the values in the newly created object and publish the copied object. Since I need to copy the object I can make it immutable. The address using the copy on write technique then consists of the following two classes:</p> <p>First, the immutable class to represent the current address:</p> <p><pre class="brush: java">public class AddressValue { private final String street; private final String city; private final String phoneNumber; public AddressValue(String street, String city, String phoneNumber) { super(); this.street = street; this.city = city; this.phoneNumber = phoneNumber; } public String getStreet() { return street; } public String getCity() { return city; } public String getPhoneNumber() { return phoneNumber; } }</pre></p> <p>Second, the mutable class to implement the copy on write technique:</p> <p><pre class="brush: java">public class AddressUsingCopyOnWrite { private volatile AddressValue addressValue; private final Object LOCK = new Object(); @Override public String toString() { AddressValue local = addressValue; return &quot;street=&quot; + local.getStreet() + &quot;,city=&quot; + local.getCity() + &quot;,phoneNumber=&quot; + local.getPhoneNumber(); } public AddressUsingCopyOnWrite(String street, String city, String phone) { this.addressValue = new AddressValue( street, city, phone); } public void updatePostalAddress(String street ,String city ) { synchronized(LOCK){ addressValue = new AddressValue( street, city, addressValue.getPhoneNumber() ); } } public void updatePhoneNumber( String phoneNumber) { synchronized(LOCK){ addressValue = new AddressValue( addressValue.getStreet(), addressValue.getCity(), phoneNumber ); } } }</pre></p> <p>An update now consists of creating a new copy of AddressValue, line 16 and 17 for updating the postal address and line 22 and 23 to update the phone number.</p> <p>Using those two classes the tests succeeds, making the address thread-safe.</p> <h1><a href="#why-using-a-local-variable-when-reading" id="why-using-a-local-variable-when-reading"></a>Why using a local variable when reading</h1> <p>As you see in the toString method I store the addressValue variable in the local variable local, line 6. Why?</p> <p>Let us see what happens when we directly access the variable addressValue instead of using a local variable:</p> <p><pre class="brush: java">public String toStringNotThreadSafe() { return &quot;street=&quot; + addressValue.getStreet() + &quot;,city=&quot; + addressValue.getCity() + &quot;,phoneNumber=&quot; + addressValue.getPhoneNumber(); }</pre></p> <p>Running the test we see the following error:</p> <p><pre class="brush: java">java.lang.AssertionError: readAddress: street=E. Bonanza St.,city=Springfield,phoneNumber=456 77 99</pre></p> <p>So we again read an inconsistent address. We can again see in the vmlens report what went wrong:</p> <p><img style="width: 100%; height: auto;" src="../../../img/articles/cp/notLocal.png" /></p> <p>The reading thread, thread id 14, first reads the variable addressValue to get the street. Then the writing thread, thread id 14, update the variable addressValue. Now the reading threads reads the variable addressValue to get the city and phone number. So the reading thread reads partially the initial and partially the updated address.</p> <h1><a href="#why-synchronized-block-when-writing" id="why-synchronized-block-when-writing"></a>Why synchronized block when writing</h1> <p>The second part to make the copy on write technique thread-safe is a synchronized block when we write to the variable addressValue. Why?</p> <p>Let us see what happens when we remove the synchronized block</p> <p><pre class="brush: java">public void updatePostalAddress(String street ,String city ) { addressValue = new AddressValue( street, city, addressValue.getPhoneNumber() ); } public void updatePhoneNumber( String phone) { addressValue = new AddressValue( addressValue.getStreet(), addressValue.getCity(), phone ); }</pre></p> <p>Running the test we see the following:</p> <p><pre class="brush: java">[INFO] BUILD SUCCESS</pre></p> <p>No error. The test still succeeds.</p> <p>To see why we need the synchronized block we need a different test We need to test what happens when we update different parts of our address from different threads. So we use the following test:</p> <p><pre class="brush: java">public class ConcurrencyTestTwoWrites { private final AddressUsingCopyOnWriteWithoutSynchronized address = new AddressUsingCopyOnWriteWithoutSynchronized(&quot;E. Bonanza St.&quot; , &quot;South Park&quot; , &quot;456 77 99&quot;); @Interleave(ConcurrencyTestTwoWrites.class) private void updatePostalAddress() { address.updatePostalAddress(&quot;Evergreen Terrace&quot; , &quot;Springfield&quot;); } @Interleave(ConcurrencyTestTwoWrites.class) private void updatePhoneNumber() { address.updatePhoneNumber(&quot;99 55 2222&quot;); } @Test public void test() throws InterruptedException { Thread first = new Thread( () -&gt; { updatePostalAddress();} ) ; Thread second = new Thread( () -&gt; { updatePhoneNumber(); } ) ; first.start(); second.start(); first.join(); second.join(); assertEquals( &quot;street=Evergreen Terrace, city=Springfield,phoneNumber=99 55 2222&quot; , address.toString() ); } }</pre></p> <p>In this test, the first thread updates the postal address, line 15 and the second thread updates the phone number, line 16. After both threads are stopped I check if the read address contains the new phone number and postal address, line 21 till 23.</p> <p>If we run this test we see the following error:</p> <p><pre class="brush: java">org.junit.ComparisonFailure: expected:&lt;...ngfield,phoneNumber=[99 55 2222]&gt; but was:&lt;...ngfield,phoneNumber=[456 77 99]&gt;</pre></p> <p>The problem is that without synchronization a thread overrides the update from another thread leading to a race condition. By surrounding every write to the variable addressValue we avoid this race and this test also succeeds.</p> <h1><a href="#comparison-to-read-write-locks" id="comparison-to-read-write-locks"></a>Comparison to read-write locks</h1> <p>Using copy on write, only writing threads get blocked by other writing threads. All other combinations are non-blocking. So reading threads get never blocked and writing threads are not blocked by a reading thread.</p> <p>Compare this to read-write locks where reading threads get blocked by writing threads. And where writing threads not only get blocked by other writing threads but also by reading threads.</p> <h1><a href="#conclusion" id="conclusion"></a>Conclusion</h1> <p>Copy on write let us update a class in a thread-safe way. The main advantage of this technique is that reading threads never block and that writing threads only get blocked by other writing threads. When you use this technique make sure that you always use a local variable when reading and a synchronized block when writing.</p> Sat, 31 Aug 2019 22:00:00 GMT http://vmlens.com/articles/cp/copy_on_write 2019-08-31T22:00:00Z Java Concurrency: AtomicInteger http://vmlens.com/articles/cp/atomic_integer AtomicInteger is a class specially designed to update integers in a thread-safe way. Why do we need this a class? Why can we not simply use a volatile int? And how to use AtomicInteger? <p>AtomicInteger is a class specially designed to update integers in a thread-safe way. Why do we need this a class? Why can we not simply use a volatile int? And how to use AtomicInteger?</p> <h1><a href="#why-atomicinteger" id="why-atomicinteger"></a>Why AtomicInteger?</h1> <p>The following shows an example of a not thread-safe counter using a volatile int:</p> <p><pre class="brush: java">public class CounterNotThreadSafe { private volatile int count = 0; public void increment() { count++; } public int getCount() { return count; } }</pre></p> <p>You can download the source code of all examples from <a href="https://github.com/vmlens/tutorial-atomic-integer">GitHub here</a>.</p> <p>We store the count in the volatile int count, line 2. We need the volatile keyword to make sure that the threads always see the current values, as explained in <a href="https://vmlens.com/articles/cp/why_volatile/">greater detail here</a>. We increment the counter by using the ++ operation, line 4. To check if the class is thread-safe we use the following test:</p> <p><pre class="brush: java">public class ConcurrencyTestCounter { private final CounterNotThreadSafe counter = new CounterNotThreadSafe(); @Interleave private void increment() { counter.increment(); } @Test public void testCounter() throws InterruptedException { Thread first = new Thread( () -&gt; { increment(); } ) ; Thread second = new Thread( () -&gt; { increment(); } ) ; first.start(); second.start(); first.join(); second.join(); assertEquals( 2 , counter.getCount()); } }</pre></p> <p>To test if the counter is thread-safe we need two threads, created in line 9 and 10. We start those two threads, line 11 and 12. And then wait till both are ended using thread join, line 13 and 14. After both threads are stopped we check if the count is two, line 15.</p> <p>To test all thread interleavings we use the annotation Interleave, line 3, from vmlens. The Interleave annotation tells vmlens to test all thread interleavings for the annotated method. Running the test we see the following error:</p> <p><pre class="brush: java">ConcurrencyTestCounter.testCounter:22 expected:&lt;2&gt; but was:&lt;1&gt;</pre></p> <p>The reason for the error is that since the operation ++ is not atomic the two threads can override the result of the other thread. We can see this in the report from vmlens:</p> <p><img style="width: 100%; height: auto;" src="../../../img/articles/cp/incorrectCounter.png" /></p> <p>In the case of the error, both threads first read the variable count in parallel. And then both write to the variable. This leads to the wrong value 1.</p> <p>To fix this bug we use the class AtomicInteger:</p> <p><pre class="brush: java">public class CounterUsingIncrement { private final AtomicInteger count = new AtomicInteger(); public void increment() { count.incrementAndGet(); } public int getCount() { return count.get(); } }</pre></p> <p>Instead of using an int we use AtomicInteger for the variable count, line 2. And instead of using the operation ++ we use the method incrementAndGet, line 4.</p> <p>Now since the method incrementAndGet is atomic, e.g. the other thread always see the value either before or after the method call, the threads can not override the value of their calculation. So the count is now always 2, for all thread interleavings.</p> <h1><a href="#how-to-use-atomicinteger" id="how-to-use-atomicinteger"></a>How to use AtomicInteger</h1> <p>The class AtomicInteger has multiple methods which allow us to update the AtomicInteger atomically. For example, the method incrementAndGet atomically increment the AtomicInteger and decrementAndGet decrement the AtomicInteger.</p> <p>But the method compareAndSet is special. This method allows us to implement arbitrary calculations atomically. The compareAndSet method takes two parameters, the expected current value, and the new value. The method atomically checks if the current value equals the expected value. If yes the method updates the value to the new value and return true. If not the method leaves the current value unchanged and returns false.</p> <p>The idea to use this method is to let compareAndSet check if the current value was changed by another thread while we calculated the new value. If not we can safely update the current value. Otherwise, we need to recalculate the new value with the changed current value.</p> <p>The following example shows how to use the compareAndSet to implement our counter:</p> <p><pre class="brush: java">public void increment() { int current = count.get(); int newValue = current + 1; while( ! count.compareAndSet( current , newValue ) ) { current = count.get(); newValue = current + 1; } }</pre></p> <p>We first read the current value, line 2. Then we calculate the new value, line 3. And then we check using compareAndSet if another thread changed the current value, line 4. compareAndSet will update the current value if the current value is unchanged and return true. Otherwise, if the value was changed compareAndSet will return false. Since this test might fail multiple times, we need to use a while loop. If the value was changed by another thread, we need to get the current changed value, line 5. And then recalculate the new value, line 6 and try to update again.</p> <h1><a href="#conclusion" id="conclusion"></a>Conclusion</h1> <p>AtomicInteger let us update integers in a thread-safe way. Use atomic methods like incrementAndGet or decrementAndGet for simple types of calculations. And use the methods get and compareAndSet for all other types of calculations.</p> Wed, 24 Jul 2019 22:00:00 GMT http://vmlens.com/articles/cp/atomic_integer 2019-07-24T22:00:00Z Why is combining thread-safe methods an error? http://vmlens.com/articles/cp/do_not_combine_thread_safe_methods Combining dependent thread-safe methods leads to race conditions. Only when the methods do not depend on each other, we can combine them in a thread-safe way. Why is combining thread-safe methods an error? <p>Combining dependent thread-safe methods leads to race conditions. Only when the methods do not depend on each other, we can combine them in a thread-safe way.</p> <p>Why is combining thread-safe methods an error? And what does this tell us about how to use thread-safe classes?</p> <h1><a href="#thread-safe-methods-must-be-atomic" id="thread-safe-methods-must-be-atomic"></a>Thread safe methods must be atomic</h1> <p>A method is thread-safe if it can be called from multiple threads without external synchronization. To make this possible the thread-safe method must be atomic, e.g. other threads only see the state before or after the method call nothing in between. The following example shows why it is necessary that a thread-safe method is atomic:</p> <p><pre class="brush: java">public class TestCounter { private volatile int i = 0; @Interleave public void increment() { i++; } @Test public void testUpdate() throws InterruptedException { Thread first = new Thread( () -&gt; {increment();} ) ; Thread second = new Thread( () -&gt; {increment();} ) ; first.start(); second.start(); first.join(); second.join(); } @After public void checkResult() { assertEquals( 2 , i ); } }</pre></p> <p>You can download the source code of all examples from <a href="https://github.com/vmlens/examples">Github here</a>.</p> <p>To test this I use a method which increments a counter, line 4. I use two threads which call the increment method, line 9 and 10. To test all thread interleavings I use the annotation Interleave, line 3, from <a href="https://vmlens.com">vmlens</a>. vmlens is a tool I have written to test multi-threaded Java software. The Interleave annotation tells vmlens to test all thread interleavings for the annotated method. Running the test we see the following error:</p> <p><pre class="brush: java">java.lang.AssertionError: expected:&lt;2&gt; but was:&lt;1&gt;</pre></p> <p>The reason for the error is that since the operation i++ is not atomic the two threads override the result of the other thread. We can see this in the report from vmlens:</p> <p><img style="width: 100%; height: auto;" src="../../../img/articles/cp/not2MethodsCounter.png" /></p> <p>So to make methods thread safe we must make them atomic.</p> <h1><a href="#combining-two-dependent-thread-safe-methods-leads-to-race-conditions" id="combining-two-dependent-thread-safe-methods-leads-to-race-conditions"></a>Combining two dependent thread safe methods leads to race conditions</h1> <p>Now let us see what happens when we combine 2 atomic methods:</p> <p><pre class="brush: java">public class TestTwoAtomicMethods { private final ConcurrentHashMap&lt;Integer,Integer&gt; map = new ConcurrentHashMap&lt;Integer,Integer&gt;(); @Interleave public void update() { Integer result = map.get(1); if( result == null ) { map.put(1, 1); } else { map.put(1, result + 1 ); } } @Test public void testUpdate() throws InterruptedException { Thread first = new Thread( () -&gt; { update(); } ); Thread second = new Thread( () -&gt; { update(); } ); first.start(); second.start(); first.join(); second.join(); } @After public void checkResult() { assertEquals( 2 , map.get(1).intValue() ); } }</pre></p> <p>I use two methods from ConcurrentHashMap for the same key 1. The method update, line 6 till 12, first gets the value from the ConcurrentHashMap using the method get line 6. Than update increments the value and put it back using the method put, line 8 and 11.</p> <p>Running the test we see the following error:</p> <p><pre class="brush: java">java.lang.AssertionError: expected:&lt;2&gt; but was:&lt;1&gt; </pre></p> <p>The reason for the error is that the combination of two atomic methods is not atomic. So for specific thread interleavings, one thread overrides the result of the other thread. We can see this in the report from vmlens:</p> <p><img style="width: 100%; height: auto;" src="../../../img/articles/cp/update_wrong.png" /></p> <h1><a href="#only-use-one-atomic-method" id="only-use-one-atomic-method"></a>Only use one atomic method</h1> <p>To fix this race condition we must replace the two methods with one atomic method. For our example, we can use the method compute witch executes the get and put in one atomic method:</p> <p><pre class="brush: java">public void update() { map.compute(1, (key, value) -&gt; { if (value == null) { return 1; } return value + 1; }); }</pre></p> <h1><a href="#conclusion" id="conclusion"></a>Conclusion</h1> <p>The thread-safe methods from the classes in the package java.util.concurrent only update a small amount of state atomically. This allows multiple threads to update different parts of the data structure simultaneously. They are carefully designed to allow simultaneous reads and writes to the same element without blocking. To use them correctly we must find the one atomic method which fits our need.</p> Thu, 20 Jun 2019 22:00:00 GMT http://vmlens.com/articles/cp/do_not_combine_thread_safe_methods 2019-06-20T22:00:00Z Concurrent programming: Two techniques to avoid shared state http://vmlens.com/articles/cp/2_techniques_to_avoid_shared_state The more I learn about multi-threaded programming, the harder it gets. Coordinating multiple threads modifying the same data is complicated. The java.util.concurrent.ConcurrentHashMap for example needs with 4264 lines of code two times more lines of code than its single threaded counter part the java.uti.HashMap with 1617 lines of code. So I think the best solution for concurrent programming is to avoid shared state. <p>The more I learn about multi-threaded programming, the harder it gets. Coordinating multiple threads modifying the same data is complicated. The java.util.concurrent.ConcurrentHashMap, for example, needs with 4264 lines of code two times more lines of code than its single threaded counterpart the java.uti.HashMap with 1617 lines of code.</p> <p>So I think the best solution for concurrent programming is to avoid shared state.</p> <p>The first technique to avoid shared state is to copy the data before each modification. This technique relies on the fact that updating a single variable is easy. Only when we need to update multiple variables things get complicated.</p> <h1><a href="#copy-before-modification" id="copy-before-modification"></a>Copy before modification</h1> <p>The following shows this technique using a <a href="https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/locks/ReentrantLock.html">java.util.concurrent.locks.ReentrantLock</a> for writing and a volatile field for reading:</p> <p><pre class="brush: java">final transient ReentrantLock lock = new ReentrantLock(); private transient volatile Object[] array; public E get(int index) { return (E) array[index]; } public E set(int index, E element) { final ReentrantLock lock = this.lock; lock.lock(); try { Object[] elements = array; E oldValue = (E) elements[index]; if (oldValue != element) { int len = elements.length; Object[] newElements = Arrays.copyOf(elements, len); newElements[index] = element; array = newElements; } else { // Not quite a no-op; ensures volatile write semantics array = elements; } return oldValue; } finally { lock.unlock(); } } </pre></p> <p>The example is based on the class <a href="https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/CopyOnWriteArrayList.html">java.util.concurrent.CopyOnWriteArrayList</a>.</p> <p>Reading consists of reading the volatile variable array and then the ith array element, line 4. Writing consists of Locking the ReentrantLock, line 7. Then if the current element and the new array element are no the same, creating a local copy of the array, line 14, modifying the local copy, line 15 and finally setting the array variable to the newly created array, line 16. Why we need a volatile variable instead of a normal variable is explained in <a href="../why_volatile/">this blog post</a>.</p> <p>The technique implies to copy the data structure each time we want to modify the data structure. Using so-called persistent data structures we can avoid copying the complete data structures. Persistent data structures keep the old state together with the modification. <a href="https://github.com/lacuna/bifurcan/blob/master/doc/comparison.md">This post</a> by the bifurcan project compares different libraries which implement persistent data structures.</p> <p>Copying before modifying works best for workloads consisting of multiple reads and few writes. The second technique work best for the opposite type of workloads, multiple writes and only a few reads:</p> <h1><a href="#asynchronous-modification-using-a-single-thread" id="asynchronous-modification-using-a-single-thread"></a>Asynchronous modification using a single thread</h1> <p>The idea of this technique is to use a single thread which owns the state and a messaging queue to let other threads modify the state through this thread. The following example shows how to implement this technique using the ExecutorService:</p> <p><pre class="brush: java">ExecutorService executor = Executors.newFixedThreadPool(1); executor.execute( () -&gt; modifyState() ); executor.execute( () -&gt; modifyState() ); executor.shutdown(); executor.awaitTermination(10, TimeUnit.MINUTES);</pre></p> <p>To implement this technique with a single threaded ExecutorService we first need to create one, line 1. Then we can modify the state asynchronously using the execute method, line 3 and 4. When we are done we need to stop the ExecutorService, line 4 and wait till the service is terminated, line 5.</p> <p>Martin Thompson recommends this technique in <a href="https://mechanical-sympathy.blogspot.com/2011/09/single-writer-principle.html">this blog post</a> for highly contended data structures. The single event processing thread of UI libraries like SWING or SWT is another example usage of this technique. And the idea behind the <a href="https://akka.io/">Akka</a> framework to use messaging between actors instead of shared state is similar to this technique.</p> <h1><a href="#what-is-next" id="what-is-next"></a>What is next?</h1> <p>The two techniques presented here are described in further detail in the book <a href="http://jcip.net/">Java Concurrency in Practice by Brian Goetz et al</a>. This book is the best resource to learn concurrent programming.</p> <p>To start with avoiding shared state we need to know which parts of our application uses shared state. I added a report in <a href="https://vmlens.com">vmlens</a>, a tool I developed to test multi-threaded Java, which shows which method uses shared state. This report is <a href="../../../help/manual/#report_shared_state">described here</a>.</p> <p>And if you are interested in writing concurrent collections like the mentioned ConcurrentHashMap read the book <a href="https://www.elsevier.com/books/the-art-of-multiprocessor-programming-revised-reprint/herlihy/978-0-12-397337-5">The Art of multiprocessor programming by Maurice Herlihy and Nir Shavit</a>. This book explains how commonly used data structures can be implemented for multi-threaded access.</p> Thu, 30 May 2019 22:00:00 GMT http://vmlens.com/articles/cp/2_techniques_to_avoid_shared_state 2019-05-30T22:00:00Z Why do we need the volatile keyword? http://vmlens.com/articles/cp/why_volatile What fascinates me about the volatile keyword is that it is necessary because my software still runs on a silicon chip. Even if my application runs in the cloud on a virtual machine in the Java virtual machine. But despite all of those software layers abstracting away the underlying hardware, the volatile keyword is still needed because of the cache of the processor my software runs on. <p>What fascinates me about the volatile keyword is that it is necessary because my software still runs on a silicon chip. Even if my application runs in the cloud on a virtual machine in the Java virtual machine. But despite all of those software layers abstracting away the underlying hardware, the volatile keyword is still needed because of the cache of the processor my software runs on.</p> <h1><a href="#the-volatile-keyword-and-the-cache-of-modern-processors" id="the-volatile-keyword-and-the-cache-of-modern-processors"></a>The volatile keyword and the cache of modern processors</h1> <p>Processors cache the values from the main memory in per-core caches to improve the memory access performance. While a read from a CPU register takes approximately 300 picoseconds a read from the main memory take 50 - 100 nanosecond. By using a cache this time can be reduced to approximately one nanosecond. Numbers are taken from Computer Architecture, A Quantitative Approach, JL Hennessy, DA Patterson, 5th edition, page 72.</p> <p>As pointed out by Reddit Commenters this Level 1 cache was already used in the i486 Procesor family.</p> <p>Now the question is when should a core check if the cached value was modified in the cache of another core. This is done by the volatile field annotation. By declaring a field as volatile we tell the JVM that when a thread reads the volatile field we want to see the latest written value. The JVM than uses special instructions to tell the CPU that it should synchronize its caches. For the x86 processor family those instructions are called memory fences <a href="https://mechanical-sympathy.blogspot.com/2011/07/memory-barriersfences.html">as described here</a>.</p> <p>The processor not only synchronizes the value of the volatile field but the complete cache. So if we read from a volatile field we see all writes on other cores to this variable and also the values which were written on those cores before the write to the volatile variable.</p> <h1><a href="#the-volatile-field-in-action" id="the-volatile-field-in-action"></a>The volatile field in action</h1> <p>Now let us look at how this works in practice. Let us see if we read stale values when we use a field without volatile annotation:</p> <p><pre class="brush: java">public class Termination { private int v; public void runTest() throws InterruptedException { Thread workerThread = new Thread( () -&gt; { while(v == 0) { // spin } }); workerThread.start(); v = 1; workerThread.join(); // test might hang up here } public static void main(String[] args) throws InterruptedException { for(int i = 0 ; i &lt; 1000 ; i++) { new Termination().runTest(); } } } </pre></p> <p>When the writing thread updates the field v in one core and the reading thread reads the field v in another thread, the test should hang up and run forever. But at least when I run the test on my machine, the test never hangs up. The reason is that the test needs so few CPU cycles that both threads typically run on the same core. And when both threads run on the same core they read and write to the same cache.</p> <p>Luckily the OpenJDK provides a tool, <a href="http://openjdk.java.net/projects/code-tools/jcstress/">jcstress</a>, which helps with this type of tests. jcstress uses multiple tricks that the threads of the tests run on different cores. Here the above example is rewritten as a jcstress test:</p> <p><pre class="brush: java">@JCStressTest(Mode.Termination) @Outcome(id = &quot;TERMINATED&quot;, expect = Expect.ACCEPTABLE, desc = &quot;Gracefully finished.&quot;) @Outcome(id = &quot;STALE&quot;, expect = Expect.ACCEPTABLE_INTERESTING, desc = &quot;Test hung up.&quot;) @State public class APISample_03_Termination { int v; @Actor public void actor1() { while (v == 0) { // spin } } @Signal public void signal() { v = 1; } }</pre></p> <p>This test is from the <a href="http://hg.openjdk.java.net/code-tools/jcstress/file/64f2cf32fa0a/jcstress-samples/src/main/java/org/openjdk/jcstress/samples/APISample_03_Termination.java">jcstress examples</a>. By annotating the class with the annotation @JCStressTest we tell jcstress that this class is a jcstress test. jcstress runs the methods annotated with @Actor and @Signal in a separate thread. jcstress first starts the actor thread and then runs the signal thread. If the test exits in a reasonable time, jcstress records the &quot;TERMINATED&quot; result, otherwise the result &quot;STALE&quot;.</p> <p>I have run this test on my development machine, once with a normal and once with a volatile field v. The test for the volatile field looked like this:</p> <p><pre class="brush: java">public class APISample_03_Termination { volatile int v; // methods omitted }</pre></p> <p>jcstress runs the test case multiple times with different JVM parameters. Here are the results of this test on my development machine an Intel i5 4 core CPU using the test mode stress:</p> <p><table width=100% border="1"> <tr> <th>JVM options</th> <th class="text-right">Observed state</th> <th class="text-right">Occurrence non volatile</th> <th class="text-right">Occurrence volatile</th> </tr> <tr> <td rowspan="2" style="font-style: italic;" >-client</td> <td class="text-right">TERMINATED</td> <td class="text-right">10</td> <td class="text-right">8980294</td> </tr> <tr> <td class="text-right" style="color : #ff0000;">STALE</td> <td class="text-right" style="color : #ff0000;">10</td> <td class="text-right">0</td> </tr> <tr> <td rowspan="2" style="font-style: italic;" >-server</td> <td class="text-right">TERMINATED</td> <td class="text-right">11</td> <td class="text-right">9040080</td> </tr> <tr> <td class="text-right" style="color : #ff0000;">STALE</td> <td class="text-right" style="color : #ff0000;">10</td> <td class="text-right">0</td> </tr> <tr> <td>-XX:TieredStopAtLevel=1</td> <td class="text-right">TERMINATED</td> <td class="text-right">8858074</td> <td class="text-right">9052777</td> </tr> <tr> <td>-Xint</td> <td class="text-right">TERMINATED</td> <td class="text-right">8035685</td> <td class="text-right">8454639</td> </tr> <tr> <td rowspan="2" style="font-style: italic;" >-server, -XX:-TieredCompilation</td> <td class="text-right">TERMINATED</td> <td class="text-right">0</td> <td class="text-right">8563250</td> </tr> <tr> <td class="text-right" style="color : #ff0000;">STALE</td> <td class="text-right" style="color : #ff0000;">10</td> <td class="text-right">0</td> </tr> <tr> <td rowspan="2" style="font-style: italic;" >-client, -XX:-TieredCompilation</td> <td class="text-right">TERMINATED</td> <td class="text-right">3</td> <td class="text-right">8719757</td> </tr> <tr> <td class="text-right" style="color : #ff0000;">STALE</td> <td class="text-right" style="color : #ff0000;">10</td> <td class="text-right">0</td> </tr> </table></p> <p>As we see using fields without volatile annotation lead indeed to hung threads. The percentage of hung threads depends on the JVM flags and the environment, JDK version and so on. Please run this on your PC, you should see a different distribution between hung and completed runs.</p> <h1><a href="#when-to-use-volatile-fields" id="when-to-use-volatile-fields"></a>When to use volatile fields</h1> <p>The volatile field is most often used as a flag to signal a specific condition like in the test above. Another usage of volatile fields is to use the volatile field for reading and locks for writing. Or you can use them with the JDK 9 VarHandle to achieve atomic operations. How to implement those techniques is <a href="../../3_tips_volatile_fields/">described here</a>.</p> <h1><a href="#the-volatile-field-as-an-example-of-a-happens-before-relation" id="the-volatile-field-as-an-example-of-a-happens-before-relation"></a>The volatile field as an example of a happens-before relation</h1> <p>But typical I do not use volatile fields directly. I rather use data structures from the java.util.concurrent package for concurrent programming. Which internally use the volatile fields.</p> <p>In the documentation of those classes we often read something about memory consistency effects and happens-before relation like in the following from the interface <a href="https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/Future.html">Future</a>:</p> <blockquote> <p>Memory consistency effects: Actions taken by the asynchronous computation happen-before actions following the corresponding Future.get() in another thread.</p> </blockquote> <p>Now with our knowledge about the volatile field, we can decode this documentation. If we read from a volatile field we see all writes on other cores to this variable. In the words of the java.util.concurrent documentation we would say the read to a volatile variable creates a happen-before relation to the write to this variable. The term happen-before comes from the mathematical model which formalizes the effect of the volatile field. This model is <a href="https://docs.oracle.com/javase/specs/jls/se8/html/jls-17.html#jls-17.4.5">described here</a>.</p> <p>So the above statement means that a Thread which calls Future.get() always sees the latest written values which were written by other Threads before calling another method of the interface Future.</p> <p>Let us use the class FutureTask to transfer data between two threads as an example. FutureTask implements the interface Future so calling the method FutureTask.get() always sees the latest written value by another method, for example, FutureTask.set().</p> <p>Here is a potential program flow to explain this: Thread A set variable x and y of object OA to one and calls FutureTask.set(OA). Now Thread B reads this object calling FutureTask.get() into the variable OB. To make the example more interesting Thread A now sets variable x to two. If Thread B reads variable y it surely sees value one, since the cache was synchronized between the call to FutureTask.set(OA) and FutureTask.get(). But for variable y Thread B reads one or two, depending on which cores the two Threads were running on.</p> <p>In pseudo code this looks like this:</p> <p><table width=100% border="1"> <tr> <th>Thread A</th> <th>Thread B</th> </tr> <tr> <td>OA.x = 1</td> <td></td> </tr> <tr> <td>OA.y = 1</td> <td></td> </tr> <tr> <td>FutureTask.set(OA)</td> <td></td> </tr> <tr> <td></td> <td>OB = FutureTask.get()</td> </tr> <tr> <td>OA.y = 2</td> <td>OB.x == 1 </td> </tr> <tr> <td></td> <td>OB.y == 1 or OB.y == 2</td> </tr> </table></p> <h1><a href="#tools-to-detect-missing-volatile-annotations" id="tools-to-detect-missing-volatile-annotations"></a>Tools to detect missing volatile annotations</h1> <p>If you forget to declare a field as volatile a thread might read a stale value. But the chance to see this during tests is rather low. Since the read and the write must happen at almost the same time and on different cores to read a stale value, this happens only under heavy load and after a long run time, e.g. in production.</p> <p>So it is no surprise that there exist tools to detect such a problem in test runs:</p> <dl> <dt><a href="https://github.com/google/sanitizers/wiki/ThreadSanitizerCppManual">ThreadSanitizer</a></dt> <dd>ThreadSanitizer can detect missing volatile annotations in C++ programs. There is a draft for a Java enhancement proposal, <a href="http://openjdk.java.net/jeps/8208520">JEP draft: Java Thread Sanitizer</a> to include ThreadSanitizerinto the OpenJDK JVM. This would allow us to find missing volatile annotations in the JVM and also in the by the JVM executed Java application.</dd> <dt><a href="https://vmlens.com">vmlens</a></dt> <dd>vmlens, a tool I have written to test concurrent java, can detect missing volatile annotations in Java test runs.</dd> </dl> <h1><a href="#conclusion" id="conclusion"></a>Conclusion</h1> <p>The volatile field is needed to make sure that multiple threads always see the newest value. Even when the cache system or compiler optimizations are at work. Reading from a volatile variable always returns the latest written value from this variable. The methods of most classes in the java.util.concurrent package also has this property. Often by using volatile fields internally.</p> Wed, 15 May 2019 22:00:00 GMT http://vmlens.com/articles/cp/why_volatile 2019-05-15T22:00:00Z Detecting visibility bugs in concurrent Java http://vmlens.com/articles/ct/visibility_bugs Chances to detect visibility bugs vary. The following visibility bug can in the best case detected in 90 percent of all cases. In the worst case, the chance to detect the bug is lower than one in a million. <p>Chances to detect visibility bugs vary. The following visibility bug can in the best case detected in 90 percent of all cases. In the worst case, the chance to detect the bug is lower than one in a million.<br /> But first what are visibility bugs?</p> <h1><a href="#what-are-visibility-bugs" id="what-are-visibility-bugs"></a>What are visibility bugs?</h1> <p>A visibility bug happens when a thread reads a stale value. In the following example a thread signals another thread to stop the processing of its while loop:</p> <p><pre class="brush: java">public class Termination { private int v; public void runTest() throws InterruptedException { Thread workerThread = new Thread( () -&gt; { while(v == 0) { // spin } }); workerThread.start(); v = 1; workerThread.join(); // test might hang up here } public static void main(String[] args) throws InterruptedException { for(int i = 0 ; i &lt; 1000 ; i++) { new Termination().runTest(); } } } </pre></p> <p>The bug is that the worker thread might never see the update of the variable v and therefore runs forever.</p> <p>One reason, for reading stale values, is the cache of the CPU cores. Each core of modern CPUs has his own cache. So if the reading and writing thread runs on different cores the reading thread sees a cached value and not the value written by the writing thread. The following shows the cores and caches inside an Intel Pentium 4 CPU, from <a href="https://superuser.com/a/507477">this superuser answer</a>:</p> <p><img src="../../../img/articles/ct/intel_cache.png" /></p> <p>Each core of an Intel Pentium 4 CPU has its own level 1 and level 2 cache. All cores share a large level 3 cache. The reason for those caches is performance. The following numbers show the time needed to access the memory, from Computer Architecture, A Quantitative Approach, JL Hennessy, DA Patterson, 5th edition, page 72:</p> <ul> <li>CPU Register ~ 300 picosecond</li> <li>Level 1 Cache ~ 1 nanosecond</li> <li>Main Memory ~ 50 - 100 nanosecond</li> </ul> <p>Reading and writing to a normal field does not invalidate the cache. So if two threads on different cores read and write to the same variable they see stale values. Let us see if we can reproduce this bug.</p> <h1><a href="#how-to-reproduce-a-visibility-bug" id="how-to-reproduce-a-visibility-bug"></a>How to reproduce a visibility bug</h1> <p>If you have run the above example chances are high that the test does not hang up. The test needs so few CPU cycles that both threads typically run on the same core. And when both threads run on the same core they read and write to the same cache. Luckily the OpenJDK provides a tool, <a href="http://openjdk.java.net/projects/code-tools/jcstress/">jcstress</a>, which helps with this type of tests. jcstress uses multiple tricks that the threads of the tests run on different cores. Here the above example is rewritten as jcstress test:</p> <p><pre class="brush: java">@JCStressTest(Mode.Termination) @Outcome(id = &quot;TERMINATED&quot;, expect = Expect.ACCEPTABLE, desc = &quot;Gracefully finished.&quot;) @Outcome(id = &quot;STALE&quot;, expect = Expect.ACCEPTABLE_INTERESTING, desc = &quot;Test hung up.&quot;) @State public class APISample_03_Termination { int v; @Actor public void actor1() { while (v == 0) { // spin } } @Signal public void signal() { v = 1; } }</pre></p> <p>This test is from the <a href="http://hg.openjdk.java.net/code-tools/jcstress/file/64f2cf32fa0a/jcstress-samples/src/main/java/org/openjdk/jcstress/samples/APISample_03_Termination.java">jcstress examples</a>. By annotating the class with the annotation @JCStressTest we tell jcstress that this class is a jcstress test. jcstress runs the methods annotated with @Actor and @Signal in a separate thread. jcstress first starts the actor thread and then runs the signal thread. If the test exits in a reasonable time, jcstress records the &quot;TERMINATED&quot; result, otherwise the result &quot;STALE&quot;.</p> <p>jcstress runs the test case multiple times with different JVM parameters. Here are the results of this test on my development machine an Intel i5 4 core CPU using the test mode stress.</p> <p><table width=100% border="1"> <tr> <th>JVM options</th> <th class="text-right">Observed state</th> <th class="text-right">Occurrence</th> </tr> <tr> <td rowspan="2" style="font-style: italic;" >None</td> <td class="text-right">TERMINATED</td> <td class="text-right">16</td> </tr> <tr> <td class="text-right" style="color : #ff0000;">STALE</td> <td class="text-right" style="color : #ff0000;">10</td> </tr> <tr> <td rowspan="2" > -XX:-TieredCompilation</td> <td class="text-right">TERMINATED</td> <td class="text-right">1</td> </tr> <tr> <td class="text-right" style="color : #ff0000;">STALE</td> <td class="text-right" style="color : #ff0000;">10</td> </tr> <tr> <td>-XX:TieredStopAtLevel=1</td> <td class="text-right">TERMINATED</td> <td class="text-right">8776026</td> </tr> <tr> <td>Xint</td> <td class="text-right">TERMINATED</td> <td class="text-right">9058042</td> </tr> </table></p> <p>As we see for the JVM parameter-XX:-TieredCompilation the thread hangs up in 90 percent of all cases. But for the JVM flags -XX:TieredStopAtLevel=1 and -Xint the thread terminated in all runs.</p> <p>After confirming that indeed our example contains a bug, how can we fix it?</p> <h1><a href="#how-to-avoid-visibility-bugs" id="how-to-avoid-visibility-bugs"></a>How to avoid visibility bugs?</h1> <p>Java has specialized instructions which guarantee that a thread always sees the latest written value. One such instruction is the volatile field modifier When reading a volatile field a thread is guaranteed to see the last written value. The guarantee not only applies to the value of the field but to all values written by the writing thread before the write to the volatile variable. Adding the field modifier volatile to the field v from the above example makes sure that the while loop terminates always. Even if run it in a test with jcstress.</p> <p><pre class="brush: java">public class Termination { volatile int v; // methods omitted }</pre></p> <p>The volatile field modifier is not the only instruction which gives such visibility guarantees. For example the synchronized statement and classes in the package java.util.concurrent give the same guarantees. A good read to learn about techniques to avoid visibility bugs is the book JavaConcurrency In Practice by Brian Goetz et al.</p> <p>After seeing why visibility bugs happen and how to reproduce and avoid them let us look at how to find them.</p> <h1><a href="#how-to-find-visibility-bugs" id="how-to-find-visibility-bugs"></a>How to find visibility bugs?</h1> <p>The <a href="https://docs.oracle.com/javase/specs/jls/se8/html/jls-17.html">Java Language Specification Chapter 17. Threads and Locks</a> defines the visibility guarantees of the Java instructions formally. This specification defines a so-called happens before relation to define the visibility guarantees:</p> <blockquote> <p>Two actions can be ordered by a happens-before relationship. If one action happens-before another, then the first is visible to and ordered before the second.</p> </blockquote> <p>And the reading from and writing to a volatile field creates such a happens-before relation:</p> <blockquote> <p>A write to a volatile field (§8.3.1.4) happens-before every subsequent read of that field.</p> </blockquote> <p>Using this specification we can check if a program contains visibility bugs, called data race in the specification.</p> <blockquote> <p>When a program contains two conflicting accesses (§17.4.1) that are not ordered by a happens-before relationship, it is said to contain a data race. Two accesses to (reads of or writes to) the same variable are said to be conflicting if at least one of the accesses is a write.</p> </blockquote> <p>Looking at our example we see that there is no happens-before relation between the read and the write to the shared variable v. So this example contains a data race according to the specification.</p> <p>Of course, this reasoning can be automated. The following two tools use this rules to automatically detect visibility bugs:</p> <ul> <li><a href="https://github.com/google/sanitizers/wiki/ThreadSanitizerCppManual">ThreadSanitizer</a> uses the rules of the <a href="https://en.cppreference.com/w/cpp/language/memory_model">c++ memory model</a> to find visibility bugs in c++ applications. The c++ memory model consists of formal rules to specify the visibility guarantees of c++ instructions similar to what the Java Language Specification does for the Java instructions. There is a draft for a Java enhancement proposal, <a href="http://openjdk.java.net/jeps/8208520">JEP draft: Java Thread Sanitizer</a> to include ThreadSanitizerinto the OpenJDK JVM. The use of ThreadSanitizer should be enabled through a command line flag.</li> <li><a href="https://vmlens.com">vmlens</a>, a tool I have written to test concurrent java, uses the Java Language Specification to automatically check if a Java test run contains visibility bugs.</li> </ul> Tue, 21 Aug 2018 22:00:00 GMT http://vmlens.com/articles/ct/visibility_bugs 2018-08-21T22:00:00Z AtomicReference, a sometimes easier alternative to synchronized blocks http://vmlens.com/articles/cp/examples_atomicReference Brian Goetz lists AtomicReference in his the book Java Concurrency in Practice in the in the section advanced topics. Yet we will see that AtomicReference are, for specific use cases, easier to use than synchronized blocks. <p>Brian Goetz lists <a href="https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/atomic/AtomicReference.html">AtomicReference</a> in his the book <a href="http://jcip.net/">Java Concurrency in Practice</a> in the in the section advanced topics. Yet we will see that AtomicReference are, for specific use cases, easier to use than synchronized blocks. And the new JDK 8 getAndUpdate and updateAndGet methods make AtomicReference, even more, easier to use.</p> <p>Let us start with the first topic, a use case which can be easier implemented by an AtomicReference than by a synchronized block: A concurrent state machine.</p> <h1><a href="#how-to-use-compareandset-a-concurrent-state-machine" id="how-to-use-compareandset-a-concurrent-state-machine"></a>How to use compareAndSet: A concurrent state machine</h1> <p>The class <a href="https://github.com/apache/maven-surefire/blob/master/surefire-api/src/main/java/org/apache/maven/surefire/booter/CommandReader.java">CommandReader</a> from the <a href="https://github.com/apache/maven-surefire">maven surefire plugin</a> uses the compareAndSet method to implement a concurrent state machine:</p> <p><pre class="brush: java">public final class CommandReader { private static final CommandReader READER = new CommandReader(); private final Thread commandThread = newDaemonThread( new CommandRunnable(), &quot;surefire-forkedjvm-command-thread&quot; ); private final AtomicReference&lt;Thread.State&gt; state = new AtomicReference&lt;Thread.State&gt;( NEW ); public static CommandReader getReader() { final CommandReader reader = READER; if ( reader.state.compareAndSet( NEW, RUNNABLE ) ) { reader.commandThread.start(); } return reader; } }</pre></p> <p>The class AtomicReference wraps another class to enrich a variable with atomic update functionality. In line 5 the AtomicReference represents an atomic variable of the Enum type Thread.State. The AtomicReference gets initialized in line 6 to the value NEW.</p> <p>The method getReader must start the commandThread when the current state is NEW and update its value to RUNNABLE. Since the method can be called by multiple threads in parallel, setting and checking must be done atomically. This is done by the method compareAndSet, line 9. The compareAndSet method only updates its value to the new value when the current value is the same as the expected. In the example, it only updates the variable to RUNNING when the current value is NEW. If the update succeed the method returns true and the thread gets started, otherwise, it returns false and nothing happens. The check and update are done atomically.</p> <p>Here is, as a comparison, the same functionality implemented with a synchronized block.</p> <p><pre class="brush: java">public final class CommandReader { private static final CommandReader READER = new CommandReader(); private final Thread commandThread = newDaemonThread( new CommandRunnable(), &quot;surefire-forkedjvm-command-thread&quot; ); private final Thread.State state = NEW; private final Object LOCK = new Object(); public static CommandReader getReader() { final CommandReader reader = READER; synchronized(reader.LOCK) { if(reader.state == NEW) { reader.commandThread.start(); reader.state = RUNNABLE; } } return reader; } }</pre></p> <p>We use a synchronized block around the check and update of the variable state, line 10. This example shows that the check and update must be atomic. Without synchronization, two threads might read a state NEW calling the start method from the commandThread multiple times.</p> <p>As we see we can replace the synchronized block, the if statement and the write to the state by one method call to compareAndSet. In the next example, we see how to use the compareAndSet method to update values.</p> <h1><a href="#updating-values-retry-till-success" id="updating-values-retry-till-success"></a>Updating values: Retry till success</h1> <p>The idea behind using compareAndSet for updates is to retry till the update succeeds. The class AsyncProcessor from RXJava uses this technique to update an array of subscribers in the method add:</p> <p><pre class="brush: java">final AtomicReference&lt;AsyncSubscription&lt;T&gt;[]&gt; subscribers; boolean add(AsyncSubscription&lt;T&gt; ps) { for (;;) { AsyncSubscription&lt;T&gt;[] a = subscribers.get(); if (a == TERMINATED) { return false; } int n = a.length; @SuppressWarnings(&quot;unchecked&quot;) AsyncSubscription&lt;T&gt;[] b = new AsyncSubscription[n + 1]; System.arraycopy(a, 0, b, 0, n); b[n] = ps; if (subscribers.compareAndSet(a, b)) { return true; } } }</pre></p> <p>The update is retried using a for loop, line 3. The loop is only terminated if either the subscriber array is in the state terminated, line 6, or the compareAndSet operation succeeds, line 14. In all other cases, the update is repeated on a copy of the array.</p> <p>Starting with JDK 8 the class AtomicReference provides this functionality in the two utility methods getAndUpdate and updateAndGet. The following shows the implementation of the getAndUpdate method in JDK 8:</p> <p><pre class="brush: java">public final V getAndUpdate(UnaryOperator&lt;V&gt; updateFunction) { V prev, next; do { prev = get(); next = updateFunction.apply(prev); } while (!compareAndSet(prev, next)); return prev; }</pre></p> <p>The method uses the same technique as the add method from the class AsyncProcessor. It retries the compareAndSet method in a loop, line 6. The updateFunction will be called multiple times when the update fails. So this function must be either side effect free.</p> <p>And here is the add method from above implemented with the new updateAndGet method:</p> <p><pre class="brush: java">boolean add(AsyncSubscription&lt;T&gt; ps) { AsyncSubscription&lt;T&gt;[] result = subscribers.updateAndGet( ( a ) -&gt; { if (a != TERMINATED) { int n = a.length; @SuppressWarnings(&quot;unchecked&quot;) AsyncSubscription&lt;T&gt;[] b = new AsyncSubscription[n + 1]; System.arraycopy(a, 0, b, 0, n); b[n] = ps; return b; } else { return a; } }); return result != TERMINATED; }</pre></p> <p>As we see the while loop is hidden in the updateAndGet method. We only need to implement a function calculating a new value from an old one.</p> <h1><a href="#conclusion-and-next-steps" id="conclusion-and-next-steps"></a>Conclusion and next steps</h1> <p>We have seen two examples of compareAndSet. If you are interested in more examples take a look at the book <a href="http://booksite.elsevier.com/9780123705914/?ISBN=9780123705914%20">The Art of Multiprocessor Programming</a>. It shows you how to implement typical concurrent data structures using compareAndSet. <a href="../../../help/testAtomic.html">And this article shows how to test atomic updates.</a></p> <p>I would be glad to hear from you about how you use AtomicReference in your application.</p> Wed, 25 Apr 2018 22:00:00 GMT http://vmlens.com/articles/cp/examples_atomicReference 2018-04-25T22:00:00Z Why do we need 4 classes for atomic updates in Java? http://vmlens.com/articles/cp/4_atomic_updates Atomic updates are an advanced technique, typically used in high performant concurrent data structures. So why do we need 4 classes for atomic updates in Java? <p>Atomic updates are an advanced technique, typically used in high performant concurrent data structures. Atomic updates are for example heavily used in the package java.util.concurrent. So why do we need 4 classes for atomic updates in Java?</p> <p>By looking at each class we will see which class should be used for what. And by looking at the reason for those 4 classes we see what to look for when implementing a high and low-level API. But let us start with the easiest of the four classes: AtomicReference</p> <h1><a href="#first-class-atomicreference" id="first-class-atomicreference"></a>First Class: AtomicReference</h1> <p>The easiest way to call compare and set is to use the method compareAndSet from the class <a href="https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/atomic/AtomicReference.html">AtomicReference</a>. The class <a href="https://github.com/apache/maven-surefire/blob/master/surefire-api/src/main/java/org/apache/maven/surefire/booter/CommandReader.java">CommandReader</a> from the <a href="https://github.com/apache/maven-surefire">maven surefire plugin</a> uses the compareAndSet method of AtomicReference to implement a concurrent state machine:</p> <p><pre class="brush: java">public final class CommandReader { private static final CommandReader READER = new CommandReader(); private final Thread commandThread = newDaemonThread( new CommandRunnable(), &quot;surefire-forkedjvm-command-thread&quot; ); private final AtomicReference&lt;Thread.State&gt; state = new AtomicReference&lt;Thread.State&gt;( NEW ); public static CommandReader getReader() { final CommandReader reader = READER; if ( reader.state.compareAndSet( NEW, RUNNABLE ) ) { reader.commandThread.start(); } return reader; } }</pre></p> <p>The class AtomicReference wraps another class to enrich a variable with atomic update functionality. In line 5 the AtomicReference represents an atomic variable of the Enum type Thread.State. The AtomicReference gets initialized in line 6 to the value NEW. The compareAndSet method only updates its value to the new value when the current value is the same as the expected. In the example, it only updates the variable to RUNNING when the current value is NEW. If the updated succeed the method returns true and the thread gets started, otherwise, it returns false and nothing happens.</p> <p>The disadvantage of AtomicReference is that for each Object we want to update atomically we need a separate AtomicReference instance. With the <a href="http://openjdk.java.net/projects/code-tools/jol/">OpenJDK tool jol</a>, we see that for our example the AtomicReference costs two-thirds of the object Thread.State:</p> <p><pre class="brush: java">public static void main(String[] args) { out.println(VM.current().details()); out.println(ClassLayout.parseClass(AtomicReference.class).toPrintable()); out.println(ClassLayout.parseClass(Thread.State.class).toPrintable()); } </pre></p> <p>And here the output:</p> <p><pre class="brush: java"># Running 64-bit HotSpot VM. java.util.concurrent.atomic.AtomicReference object internals: Instance size: 16 bytes java.lang.Thread$State object internals: Instance size: 24 bytes</pre></p> <p>Therefore Java provides a second class, AtomicReferenceFieldUpdater, to call compareAndSet using reflection.</p> <h1><a href="#second-class-atomicreferencefieldupdater" id="second-class-atomicreferencefieldupdater"></a>Second Class: AtomicReferenceFieldUpdater</h1> <p>The <a href="https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/atomic/AtomicReferenceFieldUpdater.html">AtomicReferenceFieldUpdater</a> uses reflection to access the field to update atomically:</p> <p><pre class="brush: java">private volatile Thread.State state = NEW; private static final AtomicReferenceFieldUpdater&lt;AtomicReferenceExample, Thread.State&gt; ATOMIC_STATE_UPDATER = AtomicReferenceFieldUpdater. newUpdater(AtomicReferenceExample.class, Thread.State.class, &quot;state&quot;); public void update() { ATOMIC_STATE_UPDATER.compareAndSet(this, NEW, RUNNABLE); }</pre></p> <p>The volatile field which should be updated atomically is declared in line 1. A new AtomicReferenceFieldUpdater is created with the name of the field the class of the field and the class containing the field, line 3. The field holding the AtomicReferenceFieldUpdater is static final since we need only one AtomicReferenceFieldUpdater for all objects. To update a field we call compareAndSet with the object we want to update, the expected and the new value, line 5.</p> <h1><a href="#third-class-sunmiscunsafe" id="third-class-sunmiscunsafe"></a>Third Class: sun.misc.Unsafe</h1> <p>To see the third way let us look at the implementation of AtomicReferenceFieldUpdater.</p> <p><pre class="brush: java"> private static final class AtomicReferenceFieldUpdaterImpl&lt;T,V&gt; extends AtomicReferenceFieldUpdater&lt;T,V&gt; { private static final Unsafe unsafe = Unsafe.getUnsafe(); private final long offset; AtomicReferenceFieldUpdaterImpl(final Class&lt;T&gt; tclass, final Class&lt;V&gt; vclass, final String fieldName, final Class&lt;?&gt; caller) { final Field field; field = AccessController.doPrivileged( new PrivilegedExceptionAction&lt;Field&gt;() { public Field run() throws NoSuchFieldException { return tclass.getDeclaredField(fieldName); } }); // parameter checks and exception handling omitted offset = unsafe.objectFieldOffset(field); } public boolean compareAndSet(T obj, V expect, V update) { if (obj == null || obj.getClass() != tclass || cclass != null || (update != null &amp;&amp; vclass != null &amp;&amp; vclass != update.getClass())) updateCheck(obj, update); return unsafe.compareAndSwapObject(obj, offset, expect, update); } }</pre></p> <p>AtomicReferenceFieldUpdater is a wrapper around the class sun.misc.Unsafe, adding security checks and making the API easier. To use the method compareAndSwapObject from sun.misc.Unsafe we need the offset of the field we want to update. The offset is calculated in the constructor line 17. To update an object we call compareAndSwapObject with the object we want to update, the offset, the expected state and the new state, line 24.</p> <h1><a href="#usage-count" id="usage-count"></a>Usage Count</h1> <p>Many people do what we did. They look at the implementation of AtomicReference and AtomicReferenceFieldUpdater. And sometimes they end up using sun.misc.Unsafe instead of AtomicReference or AtomicReferenceFieldUpdater. To see how often, let us look at the GitHub open source projects using google bigquery.<br /> <a href="https://medium.com/google-cloud/github-on-bigquery-analyze-all-the-code-b3576fd2b150">google bigquery allows you to query GitHub projects using SQL</a>. We use the following query to get the usage count:</p> <p><pre class="brush: java">SELECT count(*) FROM ( SELECT SPLIT(repo.content, '\n') line FROM [fh-bigquery:github_extracts.contents_java] as repo HAVING REGEXP_MATCH(line, [Search Key]) )</pre></p> <p>The used table fh-bigquery:github_extracts.contents_java was last updated Jan 19, 2017. Here are the results for the three search keys:</p> <table class="table"> <tr> <td><b>Search Key</b></td> <td><b>Count</b></td> </tr> <tr> <td>java.util.concurrent.atomic.AtomicReference;</td> <td>20746</td> </tr> <tr> <td>compareAndSwapObject</td> <td>3454</td> </tr> <tr> <td>java.util.concurrent.atomic.AtomicReferenceFieldUpdater;</td> <td>896</td> </tr> </table> <p>AtomicReference is most often used, followed by the usage of Unsafe. AtomicReferenceFieldUpdater, on the other hand, is only seldom used.</p> <p>I use compareAndSwapObject as the search key for sun.misc.Unsafe since The sun.misc.Unsafe provides methods for multiple use cases. And compareAndSwapObject uniquely identifies the usage sun.misc.Unsafe for an atomic update of a reference variable.</p> <h1><a href="#fourth-class-varhandle" id="fourth-class-varhandle"></a>Fourth Class: VarHandle</h1> <p>But with the release of JDK 9, Oracle wanted to clean up. The first idea to simply forbid the usage of sun.misc.Unsafe using the new module system <a href="https://adtmag.com/blogs/watersworks/2015/08/java-9-hack.aspx">led to a public outcry in the community</a>. This outcry led to an update to the <a href="http://openjdk.java.net/jeps/260">JDK Enhancement Proposal 260: Encapsulate Most Internal APIs</a>. It now states that all methods which have a replacement in JDK 9 will be deprecated and removed in a later release. The replacement in JDK 9 for objectFieldOffset of sun.misc.Unsafe is the class <a href="https://docs.oracle.com/javase/9/docs/api/java/lang/invoke/VarHandle.html">VarHandle</a>.</p> <p>The JDK Enhancement Proposal for the class VarHandle is <a href="http://openjdk.java.net/jeps/193">JEP 193 : Variable Handles</a>. I think the most important sentence for a higher usage of the class VarHandle instead of sun.misc.Unsafe from the community is the penultimate sentence of this JEP:</p> <blockquote> <p>The classes in java.util.concurrent (and other areas identified in the JDK) will be migrated from sun.misc.Unsafe to VarHandle.</p> </blockquote> <p>The following shows the usage of a VarHandle to update our state variable:</p> <p><pre class="brush: java">private volatile Thread.State state = NEW; private static final VarHandle ATOMIC_STATE_UPDATER; static { try { MethodHandles.Lookup l = MethodHandles.lookup(); ATOMIC_STATE_UPDATER = l.findVarHandle(VarHandleExample.class, &quot;state&quot;, Thread.State.class); } catch (ReflectiveOperationException e) { throw new Error(e); } } public void update() { ATOMIC_STATE_UPDATER.compareAndSet(this, NEW, RUNNABLE); } </pre></p> <p>A VarHandle works almost the same as AtomicReferenceFieldUpdater. The volatile field which should be updated atomically is declared in line 1. We create a new VarHandle with reflection by calling findVarHandle of the class MethodHandles.Lookup, line 7. We need only one VarHandle for all Instances of VarHandleExample, so we can declare the variable ATOMIC_STATE_UPDATER as static final, line 2. To update a field we call compareAndSet with the object we want to update, the expected and the new value, line 14.</p> <h1><a href="#summary-and-next-steps" id="summary-and-next-steps"></a>Summary and Next Steps</h1> <p>If you want to use compare and set in your application use AtomicReference. It is the easiest class of the four. <a href="http://vmlens.com/articles/cp/examples_atomicReference/">This post</a> shows that AtomicReference is sometimes even easier to use than a synchronized block. And if you want to learn how to implement hight performant concurrent data structures using AtomicReference read the book <a href="http://booksite.elsevier.com/9780123705914/?ISBN=9780123705914%20">The Art of Multiprocessor Programming</a>.</p> <p>Our 4 classes show that if you want to provide low and high-level API, do not offer an API with medium complexity like AtomicReferenceFieldUpdater. And the low-level API should be so useful that it is used in your own classes, like VarHandle in the package java.util.concurrent. But I think the main lesson is, nobody gets all classes right all the time.</p> Mon, 23 Apr 2018 22:00:00 GMT http://vmlens.com/articles/cp/4_atomic_updates 2018-04-23T22:00:00Z Waiting for tasks with Phaser http://vmlens.com/articles/examples_phaser The class Phaser let you wait for a flexible amount of tasks executed in other threads. We will see how to use it by looking at a real-life example. <p>The class <a href="https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/Phaser.html">Phaser</a> let you wait for a flexible amount of tasks executed in other threads. Use the method register to add a task you want to wait for. Call arrive to signal that a registered task is finished. And call awaitAdvance to wait till all registered tasks are finished. We will see how to use it by looking at a real-life example. But first, how does it work?</p> <h1><a href="#how-does-phaser-work" id="how-does-phaser-work"></a>How does Phaser work?</h1> <p>The class Phaser works in phases. You register tasks for all phases by calling the method register. You signal that a task is finished for this phase by calling arrive. When all registered tasks have arrived for this phase the Phaser starts a new phase and you can start over again. The following shows this for the first phase, phase zero:</p> <p><pre class="brush: java"> Phaser phaser = new Phaser(); assertEquals( 0 , phaser.register() ); assertEquals( 0 , phaser.arrive() );</pre></p> <p>Both methods return the current phase, phase 0 in the example. By calling arrive, line 3 in the example above, the Phaser starts a new phase 1:</p> <p><pre class="brush: java"> assertEquals( 1 , phaser.getPhase() ); </pre></p> <p>And we can start over again calling arrive to start a new phase:</p> <p><pre class="brush: java"> assertEquals( 1 , phaser.arrive() ); assertEquals( 2 , phaser.getPhase() );</pre></p> <p>The Phaser let us wait for other threads by the method awaitAdvance. awaitAdvance let a thread wait till the Phaser reaches a new phase. You call the awaitAdvance method with the current phase to wait for other threads:</p> <p><pre class="brush: java">phaser.awaitAdvance( phaser.getPhase() ); // waits for phaser.arrive() in other threads</pre></p> <p>awaitAdvance returns immediately if the current phase is not equal to the given phase value:</p> <p><pre class="brush: java">phaser.awaitAdvance( phaser.getPhase() + 1); // returns immediately</pre></p> <h2><a href="#deregistering-tasks-and-the-terminal-state" id="deregistering-tasks-and-the-terminal-state"></a>Deregistering Tasks and the terminal state</h2> <p>The Phaser let you deregister tasks by calling arriveAndDeregister:</p> <p><pre class="brush: java">phaser.arriveAndDeregister();</pre></p> <p>When all registered tasks are deregistered the Phaser gets terminated:</p> <p><pre class="brush: java">assertEquals( true , phaser.isTerminated() );</pre></p> <p>When the Phaser is terminated arrive and register have no effect and return a negative number. And the method await returns immediately. You can change this behavior by overriding the method onAdvance:</p> <p><pre class="brush: java"> Phaser phaser = new Phaser() { protected boolean onAdvance(int phase, int parties) { return false; } }</pre></p> <p>By always returning false as in the example above, the Phaser can only be terminated by calling the method forceTermination explicitly.</p> <h1><a href="#example-waiting-for-other-threads" id="example-waiting-for-other-threads"></a>Example: Waiting for other threads</h1> <p>The class <a href="https://github.com/JetBrains/intellij-community/blob/master/platform/lang-impl/src/com/intellij/util/indexing/FileBasedIndexImpl.java">ChangedFilesCollector</a> from the <a href="https://github.com/JetBrains/intellij-community">IntelliJ community edition</a> uses the Phaser to wait till all threads have reached a specific state. The used Phaser overrides the onAdvance method, to allow to reuse the Phaser when all tasks are deregistered:</p> <p><pre class="brush: java"> private final Phaser myWorkersFinishedSync = new Phaser() { @Override protected boolean onAdvance(int phase, int registeredParties) { return false; } };</pre></p> <p>In the method processFilesInReadAction this Phaser is used to wait till all threads have finished their tasks:</p> <p><pre class="brush: java"> private void processFilesInReadAction() { assert ApplicationManager.getApplication().isReadAccessAllowed(); myWorkersFinishedSync.register(); int phase = myWorkersFinishedSync.getPhase(); try { // other statements omitted } finally { myWorkersFinishedSync.arriveAndDeregister(); } myWorkersFinishedSync.awaitAdvance(phase); } </pre></p> <p>In line 3 the method register registers a new task. The variable phase remembers the current phase in line 4. In line 9 the method arriveAndDeregister signals that the task is done and deregisters the task. And in line 11 we wait for the other threads using the method awaitAdvance.</p> <h1><a href="#other-classes-to-wait-for-threads" id="other-classes-to-wait-for-threads"></a>Other classes to wait for threads</h1> <p>Java provides three classes to wait for other threads: Phaser, <a href="http://vmlens.com/articles/examples_countDownLatch/">CountDownLatch</a>, and <a href="http://vmlens.com/articles/examples_cyclicBarrier/">CyclicBarrier</a>. Use Phaser when you need to wait for a flexible amount of threads. When you need to wait for a fixed amount of tasks done in other threads use CountDownLatch instead. And use CyclicBarrier when you do the work and need to wait in the same threads for a fixed amount of threads.</p> <h1><a href="#next-steps" id="next-steps"></a>Next Steps</h1> <p>This was the last java.util.concurrent class to wait for other threads. In the next blog post, we will look at the classes in the package java.util.concurrent.atomic.</p> <p>I would be glad to hear from you about how you use Phaser in your application.</p> Tue, 03 Apr 2018 22:00:00 GMT http://vmlens.com/articles/examples_phaser 2018-04-03T22:00:00Z Waiting for another thread with CyclicBarrier: 2 Real-life examples http://vmlens.com/articles/examples_cyclicBarrier The CyclicBarrier let a set of threads wait till they have all reached a specific state. We will see how to use it by looking at two real-life examples. <p>The <a href="https://docs.oracle.com/javase/7/docs/api/java/util/concurrent/CyclicBarrier.html">CyclicBarrier</a> let a set of threads wait till they have all reached a specific state. Initialize the CyclicBarrier with the number of threads which need to wait for each other. Call await to signal that you are ready to proceed and wait for the other threads. We will see how to use it by looking at two real-life examples.</p> <h1><a href="#how-to-use-cyclicbarrier" id="how-to-use-cyclicbarrier"></a>How to use CyclicBarrier</h1> <p>The first use case of CyclicBarrier is to signal that we are ready to proceed and wait for the other threads. The class <a href="https://github.com/blazegraph/database/blob/master/bigdata-core/bigdata/src/java/com/bigdata/service/DistributedTransactionService.java">DistributedTxCommitTask</a> from the Blazegraph open source graph database uses the CyclicBarrier to coordinate multiple threads. The following shows the creation of the CyclicBarrier:</p> <p><pre class="brush: java">preparedBarrier = new CyclicBarrier(nservices, new Runnable() { public void run() { // Statements omitted } });</pre></p> <p>We need to wait for nservices threads so we initialize the CyclicBarrier with this variable, line 1. To execute an action after all threads have called await, we initialize the CyclicBarrier with a Runnable, line 2. The run method of the Runnable class will be called by the last thread calling await. Only after the run method was executed the threads can continue.</p> <p>When we are ready we call await, signaling that we are ready and waiting for the other threads:</p> <p><pre class="brush: java">try { // Statements omitted preparedBarrier.await(); // Statements omitted } finally { if (preparedBarrier != null) preparedBarrier.reset(); }</pre></p> <p>As we see, we execute the task of the thread a try block, line 1 to 4. And we call the reset method of the CyclicBarrier in a finally block, line 7. This makes sure that in the case of an exception other threads are not waiting infinitely but receive a BrokenBarrierException exception.</p> <h2><a href="#handling-of-exceptions" id="handling-of-exceptions"></a>Handling of exceptions</h2> <p>The BrokenBarrierException allows us to signal the other waiting threads that we can not finish our task and it does not make sense to wait any longer. The BrokenBarrierException gets thrown when</p> <ul> <li>Another thread was interrupted by calling Thread.interrupt while waiting</li> <li>CyclicBarrier reset was called while other threads were still waiting at the barrier.</li> </ul> <p>In the case of Thread.interrupt, the thread who gets interrupted, will receive an InterruptedException while waiting and all other threads waiting will receive a BrokenBarrierException. So both exception signal that we should give up our task.</p> <h1><a href="#waiting-in-cycles" id="waiting-in-cycles"></a>Waiting in cycles</h1> <p>The second use case of CyclicBarrier is to wait in cycles. The <a href="https://github.com/google/guava/blob/master/guava-tests/test/com/google/common/util/concurrent/MoreExecutorsTest.java">MoreExecutorsTest</a> test from guava, the Google core libraries for Java, shows how to do this. The following shows the first two cycles of this test:</p> <p><pre class="brush: java"> public void testDirectExecutorServiceServiceTermination() throws Exception { final ExecutorService executor = newDirectExecutorService(); final CyclicBarrier barrier = new CyclicBarrier(2); Thread otherThread = new Thread( new Runnable() { public void run() { try { Future&lt;?&gt; future = executor.submit( new Callable&lt;Void&gt;() { public Void call() throws Exception { // WAIT #1 barrier.await(1, TimeUnit.SECONDS); // WAIT #2 barrier.await(1, TimeUnit.SECONDS); assertTrue(executor.isShutdown()); assertFalse(executor.isTerminated()); // Next cycle omitted } }); // checks omitted } catch (Throwable t) { throwableFromOtherThread.set(t); } } }); otherThread.start(); // WAIT #1 barrier.await(1, TimeUnit.SECONDS); assertFalse(executor.isShutdown()); assertFalse(executor.isTerminated()); executor.shutdown(); assertTrue(executor.isShutdown()); assertFalse(executor.isTerminated()); // WAIT #2 barrier.await(1, TimeUnit.SECONDS); // Next cycle omitted }</pre></p> <p>We initialize the CyclicBarrier for two threads, line 3. The first cycle ends when both threads call await, line 14 and 30. This also starts the second cycle. The second cycle ends when again both threads call await, line 16 and line 37. This allows you two test a process in multiple steps.</p> <h1><a href="#other-classes-to-wait-for-threads" id="other-classes-to-wait-for-threads"></a>Other classes to wait for threads</h1> <p>Java provides three classes to wait for other threads: CyclicBarrier, <a href="http://vmlens.com/articles/examples_countDownLatch/">CountDownLatch</a>, and <a href="http://vmlens.com/articles/examples_phaser/">Phaser</a>. Use CyclicBarrier when you do the work and need to wait in the same threads. When you need to wait for tasks done in other threads use CountDownLatch instead. To use CyclicBarrier or CountDownLatch you need to know the number of threads when you call the constructor. If you need to add threads after construction time, use the class Phaser.</p> <h1><a href="#summary-and-next-steps" id="summary-and-next-steps"></a>Summary and next steps</h1> <p>Use the following steps to wait till a set of threads have reached a specific state with CyclicBarrier:</p> <ol> <li>Initialize the CountDownLatch with the number of threads you are waiting for.</li> <li>Call await to signal that you are ready and wait for the other threads.</li> <li>Call reset in the finally block. This makes sure that other threads receive a BrokenBarrierException instead of waiting infinitely when this thread can not finish its task.</li> <li>BrokenBarrierException and InterruptedException both signal that we should give up our task.</li> </ol> <p>To use CyclicBarrier you need to know the number of threads working on the task in advance. We will look at the class Phaser next, which allows us to register threads after construction..</p> <p>I would be glad to hear from you about how you use CyclicBarrier in your application.</p> Wed, 28 Mar 2018 22:00:00 GMT http://vmlens.com/articles/examples_cyclicBarrier 2018-03-28T22:00:00Z Waiting for another thread with CountDownLatch: 2 Real-life examples http://vmlens.com/articles/examples_countDownLatch CountDownLatch is an easy way to wait till another thread has finished a task. We will see how to use it by looking at two real-life examples. <p><a href="https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/CountDownLatch.html">CountDownLatch</a> is an easy way to wait till another thread has finished a task. The CountDownLatch is initialized by the number of threads we need to wait for. After a thread has finished its task it calls countDown, counting down the countdown. A thread can wait till the countdown reaches zero by calling await. We will see how to use it by looking at two real-life examples.</p> <h1><a href="#how-to-use-countdownlatch-waiting-for-initialization" id="how-to-use-countdownlatch-waiting-for-initialization"></a>How to use CountDownLatch: Waiting for initialization</h1> <p>The apache ignite <a href="https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/datastructures/CacheDataStructuresManager.java">CacheDataStructuresManager</a> needs to wait till it is initialized. One thread initializes the CacheDataStructuresManager so we need a CountDownLatch with count one:</p> <p><pre class="brush: java">private final CountDownLatch initLatch = new CountDownLatch(1);</pre></p> <p>The initialization is done in the onKernalStart0() method:</p> <p><pre class="brush: java"> @Override protected void onKernalStart0() throws IgniteCheckedException { try { queueHdrView = cctx.cache(); initFlag = true; } finally { initLatch.countDown(); } }</pre></p> <p>The countDown method is called in the finally block. So we guarantee that the countdown eventually reaches zero and we do not wait forever. To signal that the initialization was successful we use the boolean flag initFlag.</p> <p>The method waitInitialization is used to check if the initialization was successful:</p> <p><pre class="brush: java"> private void waitInitialization() throws IgniteCheckedException { if (initLatch.getCount() &gt; 0) U.await(initLatch); if (!initFlag) throw new IgniteCheckedException( &quot;DataStructures manager was not properly initialized.&quot;); }</pre></p> <p>We check the count of the CountDownLatch waiting if necessary in the await method from IgniteUtils. After that, we check the initFlag if the initialization was successful.</p> <p>The method await from <a href="https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/util/IgniteUtils.java">IgniteUtils</a> handles the InterruptedException from the CountDownLatch await method:</p> <p><pre class="brush: java"> public static void await(CountDownLatch latch) throws IgniteInterruptedCheckedException { try { if (latch.getCount() &gt; 0) latch.await(); } catch (InterruptedException e) { Thread.currentThread().interrupt(); throw new IgniteInterruptedCheckedException(e); } }</pre></p> <p>In the example the interrupt flag is restored by calling Thread.currentThread().interrupt() and a new checked Exception is thrown.</p> <h2><a href="#handling-of-interruptedexception" id="handling-of-interruptedexception"></a>Handling of InterruptedException</h2> <p>The await method of the CountDownLatch throws an InterruptedException when the method Thread.interrupt was called. the CountDownLatch clears the interrupt flag when it throws the InterruptedException. Therefore the book <a href="http://jcip.net/">Java Concurrency in Practice - Brian Goetz, et al.</a> recommends to either propagate the InterruptedException or to restore the interrupt flag.</p> <p>The guava library provides the utility method <a href="https://google.github.io/guava/releases/19.0/api/docs/com/google/common/util/concurrent/Uninterruptibles.html#awaitUninterruptibly(java.util.concurrent.CountDownLatch)">awaitUninterruptibly</a> in the class Uninterruptibles which implements the second recommendation. It restores the interrupt flag. Using this method you make sure that another blocking method will again throw an InterruptedException.</p> <h1><a href="#how-to-use-await-with-a-timeout-testing-multithreaded-software" id="how-to-use-await-with-a-timeout-testing-multithreaded-software"></a>How to use await with a timeout: Testing multithreaded software.</h1> <p>The next example shows how to call the second method to wait for other threads, await with a timeout: This method is typically used in tests, like in this example in the <a href="https://github.com/eclipse/jetty.project/blob/jetty-9.4.x/jetty-servlet/src/test/java/org/eclipse/jetty/servlet/PostServletTest.java">PostServletTest</a> from jetty:</p> <p><pre class="brush: java"> @Test public void testBadPost() throws Exception { StringBuilder req = new StringBuilder(16*1024); // creation of the request String omitted String resp = connector.getResponse(req.toString()); assertThat(resp,startsWith(&quot;HTTP/1.1 200 OK&quot;)); assertTrue(complete.await(5,TimeUnit.SECONDS)); assertThat(ex0.get(),not(nullValue())); assertThat(ex1.get(),not(nullValue())); }</pre></p> <p>The await method gets called with the amount we want to wait together with the unit for the amount, line 8. In our example, we want to wait no more than 5 seconds. The test fails when the await method returns false because of the timeout.</p> <h1><a href="#other-classes-to-wait-for-threads" id="other-classes-to-wait-for-threads"></a>Other classes to wait for threads</h1> <p>Java provides three classes to wait for other threads: CountDownLatch, <a href="http://vmlens.com/articles/examples_cyclicBarrier/">CyclicBarrier</a>, and <a href="http://vmlens.com/articles/examples_phaser/">Phaser</a>. When you need to wait for tasks done in other threads use CountDownLatch. Use CyclicBarrier when you do the work and need to wait in the same threads. To use CyclicBarrier or CountDownLatch you need to know the number of threads when you call the constructor. If you need to add threads after construction time, use the class Phaser.</p> <h1><a href="#summary-and-next-steps" id="summary-and-next-steps"></a>Summary and next steps</h1> <p>When you want to wait for another thread with CountDownLatch follow these steps:</p> <ol> <li>Initialize the CountDownLatch with the number of threads you are waiting for.</li> <li>Call the countDown method in the finally block when a thread has finished its task.</li> <li>Use a flag or counter to signal that the tasks were successful.</li> <li>Wait for the threads with the method await. Propagate the InterruptedException thrown by await or restore the interrupted flag when you catch the InterruptedException.</li> </ol> <p>The CountDownLatch can only be used once. We will look at the CyclicBarrier next, which can be used multiple times. I would be glad to hear from you about how you use CountDownLatch in your application.</p> Tue, 20 Mar 2018 23:00:00 GMT http://vmlens.com/articles/examples_countDownLatch 2018-03-20T23:00:00Z Concurrent Java: Low scalability, the risk of deadlocks or garbage creation, you can not avoid all http://vmlens.com/articles/tradeoffs_concurrent_programming On the example of three concurrent map implementations, we will see that we either have low scalability, the risk of deadlocks or unnecessary garbage creation. <p>On the example of three concurrent map implementations, we will see that we either have low scalability, the risk of deadlocks or unnecessary garbage creation. Locks lead to low scalability when we use only one, or the risk to deadlock when using multiple locks. Compare and swap operations and immutable data structures on the other side lead to unnecessary object creation which causes more garbage collection cycles.</p> <p>So what technique to choose? If we have a small data structure we can use an immutable class. And if our data structure is not performance critical, we can use a single lock. In all other cases, I would start with multiple locks since it is easier to implement than compare and swap operations. But let us first start with the easiest case, a hash map synchronized with a single lock.</p> <h1><a href="#a-single-lock-low-scalability" id="a-single-lock-low-scalability"></a>A single lock: Low scalability</h1> <p>As the first example we look at a concurrent map created by Collections.synchronizedMap:</p> <p><pre class="brush: java">private Map&lt;Integer,Integer&gt; map = Collections.synchronizedMap(new HashMap&lt;Integer,Integer&gt;());</pre></p> <p><a style="color: inherit;" name="update12">In our examples we use the compute and put method to update our concurrent maps: </a></p> <p><pre class="brush: java">public void update12() { map.compute( 1, ( key ,value ) -&gt; { map.put( 2 , 1); return value; }); }</pre></p> <p>When we look at the implementation of the compute method we see it is a wrapper which puts a synchronization block around an underlying map:</p> <p><pre class="brush: java">public V compute(K key, BiFunction&lt;? super K, ? super V, ? extends V&gt; remappingFunction) { synchronized (mutex) {return m.compute(key, remappingFunction);} }</pre></p> <p>The synchronization block acts as a lock. Since we are only using one monitor, the object in the mutex variable, we have only one lock. The lock makes sure that only one thread at a time accesses the underlying map. This makes sure that threads do not see an inconsistent state and that only one thread at a time modifies the data structure. But it also leads to low scalability, since the threads must wait for each other.</p> <h1><a href="#multiple-locks-risk-of-deadlocks" id="multiple-locks-risk-of-deadlocks"></a>Multiple locks: Risk of deadlocks</h1> <p>As next example let us look at a map implementation using one lock per hash array element, the ConcurrentHashMap.</p> <p><pre class="brush: java">private Map&lt;Integer,Integer&gt; map = new ConcurrentHashMap&lt;Integer,Integer&gt;();</pre></p> <p>The ConcurrentHashMap stores all its data in a hash array. The ConcurrentHashMap uses those array elements as locks when we update the map. Both put and compute methods use locks. Let us again look at the compute method to see how this is done:</p> <p><pre class="brush: java"> public V compute(K key, BiFunction&lt;? super K, ? super V, ? extends V&gt; remappingFunction) { // Parameter checks omitted int h = spread(key.hashCode()); // Spreads higher bits of hash to lower V val = null; int delta = 0; int binCount = 0; for (Node&lt;K,V&gt;[] tab = table;;) { Node&lt;K,V&gt; f; int n, i, fh; if (tab == null || (n = tab.length) == 0) tab = initTable(); else if ((f = tabAt(tab, i = (n - 1) &amp; h)) == null) { // Handling of missing elements omitted } else if ((fh = f.hash) == MOVED) tab = helpTransfer(tab, f); else { synchronized (f) { // Statements omitted // The remappingFunction method gets called in this block } } // Incrementing of hash map size omitted return val; }</pre></p> <p>Here again a synchronized block, in line 18, acts as the lock. But this time multiple locks are used since the synchronization happens one multiple objects held by the variable f. The variable f contains the return value from the method tabAt, line 12. This method returns the hash array element at index (n-1) &amp; h. <a href="http://vmlens.com/articles/5_tips_from_concurrent_hashmap/">The operation index (n-1) &amp; gives you the hash code of the key modulo the size of the hash array.</a></p> <p>So which lock is used depend on the key we use. If we use a new method which updates the values in reverse order like the following <a style="color: inherit;" name="update21">update21</a> method, <a href="http://vmlens.com/articles/detect_deadlocks/">we create a deadlock</a>.</p> <p><pre class="brush: java">public void update21() { map.compute( 2, ( key ,value ) -&gt; { map.put( 1 , 1); return value; }); }</pre></p> <p><a href="#update21">update21</a> locks array element 2, the key for the compute call and requests a lock on element 1, the key used for the put call. But <a href="#update12">update12</a> holds a lock on array element 1, by the compute call, and requests a lock on array element 2 while, for the put call. If we call both methods in parallel two threads are requesting a lock held by another thread: A deadlock.</p> <h2><a href="#how-to-avoid-deadlocks" id="how-to-avoid-deadlocks"></a>How to avoid deadlocks</h2> <p>The example gives us a hint what rules to follow to avoid deadlock. The first rule:</p> <blockquote> <p>try to minimize the number of potential locking interactions, and follow and document a lock ordering protocol for locks that may be acquired together. — <a href="http://jcip.net/">Java Concurrency in Practice - Brian Goetz, et al.</a></p> </blockquote> <p>We violated this rule in our example. Our update21 method even accessed the locks in reverse order on purpose. And the second:</p> <blockquote> <p>Release all locks before invoking unknown code. — <a href="https://www.kernel.org/pub/linux/kernel/people/paulmck/perfbook/perfbook.html">Is Parallel Programming Hard, And, If So, What Can You Do About It? - Paul E. McKenney</a></p> </blockquote> <p>That the compute method of the ConcurrentHashMap does not release all its locks before calling the callback function is a clear disadvantage of the compute method. At least it is documented in the <a href="https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/ConcurrentHashMap.html#compute-K-java.util.function.BiFunction-">javadoc of the method.</a></p> <h1><a href="#compare-and-swap-operation-garbage-creation" id="compare-and-swap-operation-garbage-creation"></a>Compare and swap operation: Garbage creation</h1> <p>As the third example we look at a map implementation using a compare and swap operation for updating, the ConcurrentSkipListMap:</p> <p><pre class="brush: java">private Map&lt;Integer,Integer&gt; map = new ConcurrentSkipListMap&lt;Integer,Integer&gt;();</pre></p> <p>All modern CPU provide instructions to atomically compare and swap or increment and get values. Those operations are used internally by the JVM to implement synchronized locks.</p> <p>The compare and swap method checks the value of a variable and only updates it when the value is the same as the expected value. <a href="http://vmlens.com/articles/3_tips_volatile_fields/#3-use-with-jdk-9-varhandle-for-atomic-operations">Compare and swap operations can be called in Java by using either the classes in java .util.concurrent.atomic or sun.mis.Unsafe or in JDK 9 VarHandles.</a> Let us look at the compute method to see how this operation is used.</p> <p><pre class="brush: java"> public V compute(K key, BiFunction&lt;? super K, ? super V, ? extends V&gt; remappingFunction) { // Parameter checks omitted for (;;) { Node&lt;K,V&gt; n; Object v; V r; if ((n = findNode(key)) == null) { // Handling of missing elements omitted } else if ((v = n.value) != null) { @SuppressWarnings(&quot;unchecked&quot;) V vv = (V) v; if ((r = remappingFunction.apply(key, vv)) != null) { if (n.casValue(vv, r)) return r; } else if (doRemove(key, vv) != null) break; } } return null; }</pre></p> <p>In line 12 the compare and swap operation is called through the method casValue. This method only updates the value of the variable n to the new value r when its current value is still vv. The variable n holds the node for our key, set in line 6. The new value r is calculated using the remappingFunction in line 11. And the variable vv holds the old value of the node n. If the method casValue successfully updates the value of n it returns true otherwise false.</p> <p>So we first remember the old value of the node we want to update. Then we create the new value we want to set. And then we update the node only when it is still containing the old value. So we make sure that no other thread has updated our node in between. We repeat this till we successful update our node through the for loop in line 4.</p> <p>The casValue method uses the sun.misc.Unsafe method internally to call the compare and swap operation:</p> <p><pre class="brush: java">boolean casValue(Object cmp, Object val) { return UNSAFE.compareAndSwapObject(this, valueOffset, cmp, val); }</pre></p> <p>The sun.misc.Unsafe compareAndSwapObject method is a very low-level method. It takes as parameters the object which contains the variable to update, a memory offset of the variable to update and the expected value cmp and the new value val.</p> <p>This is a very performant way to make sure that we never overwrite a value written by another thread. But it leads to unnecessary object creation when the update fails. The next technique creates even more garbage, immutable classes.</p> <h1><a href="#immutable-data-even-more-garbage-creation" id="immutable-data-even-more-garbage-creation"></a>Immutable Data: Even more garbage creation</h1> <p>Immutable classes do not change their value in the middle of an operation without using synchronized blocks. By avoiding synchronization blocks you avoid deadlocks. And since you are always working with an unchangeable consistent state you avoid race conditions. But modifying immutable state consists of copying the immutable object. So each modification leads to the creation of garbage. <a href="http://vmlens.com/articles/_immutable_classes/">Read here more</a> about how to use immutable classes for concurrent programming.</p> <h1><a href="#it-is-your-choice" id="it-is-your-choice"></a>It is your choice</h1> <p>Immutable data is easy to implement and use. But each modification requires a copy of the data structure. So you only use it when your data structure is small enough or the reads vastly outnumber the writes. In all other cases, you need to choose between compare and swap operations and locks. I typically prefer locks since they are easier to use.</p> <h1><a href="#what-is-next" id="what-is-next"></a>What is next?</h1> <p>All maps we looked at are synchronous. When we call a compute method the method is executed in the calling thread. Therefore we need to coordinate the different threads calling the method in parallel. Each of those mechanisms like locks or compare and swap operations have it specific tradeoffs.</p> <p>In the next blog post, we will look at the tradeoffs for a map with can be updated asynchronously. I would be glad to hear from you about which synchronization mechanism you use in your application.</p> Mon, 12 Mar 2018 23:00:00 GMT http://vmlens.com/articles/tradeoffs_concurrent_programming 2018-03-12T23:00:00Z How to implement values with immutable classes http://vmlens.com/articles/values Classes representing values should have the same properties than primitive types. In the following, we will see that those two properties are also useful for classes representing properties. <p>Classes representing values should have the same properties than primitive types. Primitive types represent basic values like int or char in Java. Primitive types have no identity and they are immutable.</p> <p>In the following, we will see that those two properties are also useful for classes representing properties.</p> <h1><a href="#no-identity" id="no-identity"></a>No identity</h1> <p>Two values are equal if they have the same external visible state: For example, if you have two variables a and b with the value 5, they are equal:</p> <p><pre class="brush: java"> int a = 5; int b = 5; a == b // true</pre></p> <h1><a href="#immutable" id="immutable"></a>Immutable</h1> <p>Values are immutable. If you a modify a value it becomes a new value. In the following example, we modify the variable a which leads to a new value b. A and b are nor equal.</p> <p><pre class="brush: java"> int a = 5 int b = a + 2; a == b // false</pre></p> <h1><a href="#a-value-class" id="a-value-class"></a>A value class</h1> <p>Now let us look at a class representing a value with those two properties, the java.time.Instant class. This class represents an instant in time. Let us first look at the field declaration of this class:</p> <p><pre class="brush: java">package java.time; public final class Instant { private final long seconds; private final int nanos; // static fields and methods omitted } </pre></p> <p>Declaring the field final makes this class immutable. Declaring a field as final as the two fields in the example, line 5 and 6, lets the compiler check that the fields are not modified after the constructor of the class was called. Note that final is a field modifier. It makes the field itself immutable not the object the field references to. So the type of the final field must also be immutable or a primitive type like in the example.</p> <p>Next, let us look at the equal method to see how equality is implemented for this class:</p> <p><pre class="brush: java">public boolean equals(Object otherInstant) { if (this == otherInstant) { return true; } if (otherInstant instanceof Instant) { Instant other = (Instant) otherInstant; return this.seconds == other.seconds &amp;&amp; this.nanos == other.nanos; } return false; } </pre></p> <p>As we see two Instant objects are equal if their external visible state is equal.</p> <h1><a href="#immutability-but-an-identity" id="immutability-but-an-identity"></a>Immutability but an identity</h1> <p>Now let us look at a class with is immutable but uses its identity for equality, the Object class. The Object class is useful when you only need an identity but no state. Like in the following example, from the JDK 9 java.util.concurrent.CopyOnWriteArrayList, where we need the identity as a monitor for synchronization:</p> <p><pre class="brush: java">public class CopyOnWriteArrayList&lt;E&gt; { final transient Object lock = new Object() public boolean add(E e) { synchronized (lock) { // other statements omitted } } }</pre></p> <p>To represent a value a class with identity is not useful.</p> <h1><a href="#no-identity-but-mutable" id="no-identity-but-mutable"></a>No identity but mutable</h1> <p>The following shows a mutable value class, the java.util.Date class:</p> <p><pre class="brush: java">public class Date { private transient BaseCalendar.Date cdate; private transient long fastTime; public void setTime(long time) { fastTime = time; cdate = null; } public boolean equals(Object obj) { return obj instanceof Date &amp;&amp; getTime() == ((Date) obj).getTime(); } public long getTime() { return getTimeImpl(); } private final long getTimeImpl() { if (cdate != null &amp;&amp; !cdate.isNormalized()) { normalize(); } return fastTime; } // static fields and other methods omitted }</pre></p> <p>The two fields cdate and fastTime are not final and can be changed by the setTime method making the class mutable. The equals method checks the externally visible state for equality. While it is possible to implement values with mutable classes, immutable classes are easier to use.</p> <h1><a href="#advantages-of-immutable-value-classes" id="advantages-of-immutable-value-classes"></a>Advantages of immutable value classes</h1> <p>Immutable classes cannot change their state. This property is useful in specific scenarios in single-threaded programs, for example when you use them as keys in hash maps. <a href="http://vmlens.com/articles/_immutable_classes/">And it makes writing multi-threaded programs much easier.</a></p> <p>Immutable classes do not change their value in the middle of an operation without using synchronized blocks. By avoiding synchronization blocks you avoid deadlocks. And since you are always working with an unchangeable consistent state you avoid race conditions.</p> <h1><a href="#usage-of-identity" id="usage-of-identity"></a>Usage of identity</h1> <p>While it is still possible to access the identity of a value object it is probably an error. For example, the use of == instead of equals is probably incorrect:</p> <p><pre class="brush: java">Integer a = new Integer(5); Integer b = new Integer(5); a.equals(b) // true a == b // false </pre></p> <p>You probably expect true when you compare two Integer of value 5, so the use of == is incorrect. The following methods are using the identity of the object and should be avoided when using value classes:</p> <ul> <li>synchronized statement</li> <li>System.identityHashCode</li> <li>Object.notify and wait</li> </ul> <h1><a href="#the-future-jep-169-value-objects" id="the-future-jep-169-value-objects"></a>The future: JEP 169, Value Objects</h1> <p><a href="https://stackoverflow.com/questions/258120/what-is-the-memory-consumption-of-an-object-in-java">Implementing values using classes requires more memory than their primitive counterparts.</a> A solution to this problem is implemented by <a href="http://openjdk.java.net/jeps/169">The Java enhancement proposal 169, Value Objects,</a>. It will allow you to create value classes with similar memory characteristics as primitive types.<br /> The idea is to implement a new operator lockPermanently which converts an object into a new state with memory consumption similar to a primitive type. Using an operation which requires the identity of the value object like == or synchronized on such a locked object will be forbidden.</p> <h1><a href="#conclusion-and-what-is-next" id="conclusion-and-what-is-next"></a>Conclusion and what is next?</h1> <p>Primitive types represent basic values. Primitive types are immutable and have no identity. We have seen how to implement classes with the same properties. The usage of the identity of a value class, while still possible, is probably an error. Making value classes immutable makes them easier to use, especially for thread safe software.</p> <p>In the next blog post, we will look at one type of value classes, messages, to write thread safe software. I would be glad to hear from you if you use value classes in your application.</p> Sun, 04 Mar 2018 23:00:00 GMT http://vmlens.com/articles/values 2018-03-04T23:00:00Z How to use immutable classes for concurrent programming in Java http://vmlens.com/articles/_immutable_classes Immutable classes make concurrent programming easier. In the following, we will look at how to use immutable classes for concurrent programming in Java. <p>Immutable classes make concurrent programming easier. Immutable classes make sure that the values are not changed in the middle of an operation without using synchronized blocks. By avoiding synchronization blocks you avoid deadlocks. And since you are always working with an unchangeable consistent state you avoid race conditions. In the following, we will look at how to use immutable classes for concurrent programming in Java.</p> <h1><a href="#how-to-write-an-immutable-class" id="how-to-write-an-immutable-class"></a>How to write an immutable class?</h1> <p>As an example for an immutable class we implement a class storing the login credentials of a user:</p> <p><pre class="brush: java">public class ImmutableLogin { private final String userName; private final String password; public ImmutableLogin(String userName, String password) { super(); this.userName = userName; this.password = password; } public String getUserName() { return userName; } public String getPassword() { return password; } }</pre></p> <p>When you implement an immutable class you declare its fields as final as the two fields, line 2 and 3, in the example. This lets the compiler check that the fields are not modified after the constructor of the class was called. Note that final is a field modifier. It makes the field itself immutable not the object the field references to. So the type of the final field must also be immutable like in the example the class String.</p> <p>The following shows a mutable class storing the same information:</p> <p><pre class="brush: java">public class MutableLogin { private String userName; private String password; public String getUserName() { return userName; } public void setUserName(String userName) { this.userName = userName; } public String getPassword() { return password; } public void setPassword(String password) { this.password = password; } }</pre></p> <h1><a href="#how-to-use-immutable-classes" id="how-to-use-immutable-classes"></a>How to use immutable classes?</h1> <p>First, let us see how we can share the immutable login data between multiple threads using a java.util.concurrent.ConcurrentHashMap. To change the login data we use the method compute:</p> <p><pre class="brush: java">private final ConcurrentHashMap&lt;String,ImmutableLogin&gt; mapImmutableLogin = new ConcurrentHashMap&lt;String,ImmutableLogin&gt;(); public void changeImmutableLogin() { mapImmutableLogin.compute(&quot;loginA&quot;, (String key , ImmutableLogin login ) -&gt; { return new ImmutableLogin(login.getUserName() , &quot;newPassword&quot;); }); }</pre></p> <p>As you probably expected, we need to copy the ImmutableLogin to change it. The compute method uses a synchronized block internally to make sure that a value for a given key is not changed in parallel by multiple threads.</p> <p>The following shows an example for reading the changed login data from the ConcurrentHashMap using get:</p> <p><pre class="brush: java"> public void readImmutableLogin() { ImmutableLogin immutableLogin = mapImmutableLogin.get(&quot;loginA&quot;); // read from the object immutableLogin }</pre></p> <p>Reading the data can directly operate on the ImmutableLogin class without synchronization block.</p> <p>Now we look how we can achieve the same using the mutable login class.Again changing the password in the ConcurrentHashMap:</p> <p><pre class="brush: java">private final ConcurrentHashMap&lt;String,MutableLogin&gt; mapMutableLogin = new ConcurrentHashMap&lt;String,MutableLogin&gt;(); public void changeMutableLogin() { MutableLogin mutableLogin = mapMutableLogin.get(&quot;loginA&quot;); synchronized(mutableLogin) { mutableLogin.setPassword(&quot;newPassword&quot;); } }</pre></p> <p>and reading the data:</p> <p><pre class="brush: java"> public void readMutableLogin() { MutableLogin mutableLogin = mapMutableLogin.get(&quot;loginA&quot;); synchronized(mutableLogin) { // read from the object mutableLogin } }</pre></p> <p>To make sure that the MutableLogin object does not get changed while reading we need to synchronize the reading and writing using the same monitor. In the examples, we use the MutableLogin object as the monitor. To avoid a nested synchronized block we use the get method for modifying the MutableLogin instead of the compute method.</p> <h1><a href="#separating-identity-and-state" id="separating-identity-and-state"></a>Separating identity and state</h1> <p>In the above examples, the keys of the ConcurrentHashMap defined the identity of the different logins and the values the current state of the login. In the case of the MutableLogin class, each key has exactly one MutableLogin object. In the case of the ImmutableLogin, each key has different ImmutableLogin objects at different points in time. Mutable classes represent both identity and state while immutable classes represent only the state and we need a separate class to represent the identity. So using Immutable classes leads to a separation of identity and state.</p> <p>The following shows how to encode the identity of the login in the class Login and the state in the class ImmutableLogin:</p> <p><pre class="brush: java">public class Login { private volatile ImmutableLogin state; public Login(ImmutableLogin state) { super(); this.state = state; } public synchronized void change(Function&lt;ImmutableLogin,ImmutableLogin&gt; update ) { state = update.apply(state); } public ImmutableLogin getState() { return state; } }</pre></p> <p>The change function uses a synchronized block to make sure that only one thread is updating the Login object at a given time. The field state must be declared as volatile to make sure that you read always the latest written value.</p> <h1><a href="#when-to-use-immutable-classes" id="when-to-use-immutable-classes"></a>When to use immutable classes?</h1> <p>Modifying immutable state consists of a copying the immutable object. This allows us to read the state without the need of synchronization blocks. So you should use immutable classes when it is feasible to copy the object for every modification and you need read-only access to the data.</p> <h1><a href="#what-is-next" id="what-is-next"></a>What is next?</h1> <p>So far all examples we have looked at used synchronized blocks for changing the state of the class holding the immutable object. In the next blog post, we will see how to implement a concurrent hash map with immutable classes for the hash array elements using compare-and-swap operations instead.</p> <p>In the meantime, if you want to test whether your application is thread-safe, try <a href="http://vmlens.com">vmlens</a> for free. I would be glad to hear from you about how you use immutable classes.</p> Mon, 12 Feb 2018 23:00:00 GMT http://vmlens.com/articles/_immutable_classes 2018-02-12T23:00:00Z A specialized high performant concurrent queue http://vmlens.com/articles/vmlens_queue_benchmark Using a specialized algorithm it is possible to achieve up to four times better performance than java.util.concurrent.BlockingQueue for multiple writing and a single reading thread. <p>Using a specialized algorithm it is possible to achieve up to four times better performance than java.util.concurrent.BlockingQueue for multiple writing and a single reading thread. Such a blocking queue supporting multiple writers but only one reader is useful if you have to access a single resource from multiple threads. Instead of writing directly to the resource you to put the data into a queue and let a single thread write the data asynchronously to the resource.</p> <p>In vmlens, this queue is used to write trace events to the file system.</p> <h1><a href="#the-queue" id="the-queue"></a>The Queue</h1> <p>The main idea is to use not one single queue but many. We use one queue per writing thread stored in a thread local field. A thread local queue is then a simple linked list using a volatile field for the next element and final for the value. Reading iterates over all writing threads to collect the written data. The algorithm is explained in more detail <a href="http://vmlens.com/articles/lock_free_executor_service/">here</a>.</p> <p>You can download the source code for the queue from GitHub <a href="https://github.com/vmlens/executor-service">here</a>.</p> <h1><a href="#the-benchmark" id="the-benchmark"></a>The Benchmark</h1> <p>Here is the source code of the benchmark:</p> <p><pre class="brush: java">@State(Scope.Group) public class BlockingQueueBenchmark { private static final int WRITING_THREAD_COUNT = 5; private static final int VMLENS_QUEUE_LENGTH = 1000; private static final int JDK_QUEUE_LENGTH = 4000; EventBusImpl eventBus; Consumer consumer; ProzessAllListsRunnable prozess; TLongObjectHashMap&lt;ProzessOneList&gt; threadId2ProzessOneRing; LinkedBlockingQueue jdkQueue; private long jdkCount = 0; private long vmlensCount = 0; @Setup() public void setup() { eventBus = new EventBusImpl(VMLENS_QUEUE_LENGTH); consumer = eventBus.newConsumer(); prozess = new ProzessAllListsRunnable(new EventSink() { public void execute(Object event) { vmlensCount++; } public void close() { } public void onWait() { } }, eventBus); threadId2ProzessOneRing = new TLongObjectHashMap&lt;ProzessOneList&gt;(); jdkQueue = new LinkedBlockingQueue(JDK_QUEUE_LENGTH); } @Benchmark @Group(&quot;vmlens&quot;) @GroupThreads(WRITING_THREAD_COUNT) public void offerVMLens() { consumer.accept(&quot;event&quot;); } @Benchmark @Group(&quot;vmlens&quot;) @GroupThreads(1) public void takeVMLens() { prozess.oneIteration(threadId2ProzessOneRing); } @Benchmark @Group(&quot;jdk&quot;) @GroupThreads(WRITING_THREAD_COUNT) public void offerJDK() { try { jdkQueue.put(&quot;event&quot;); } catch (Exception e) { e.printStackTrace(); } } @Benchmark @Group(&quot;jdk&quot;) @GroupThreads(1) public void takeJDK() { try { jdkQueue.poll(100, TimeUnit.SECONDS); jdkCount++; } catch (Exception e) { e.printStackTrace(); } } @TearDown(Level.Trial) public void printCounts() { System.out.println(&quot;jdkCount &quot; + jdkCount); System.out.println(&quot;vmlensCount &quot; + vmlensCount); } }</pre></p> <p>The benchmark uses <a href="http://openjdk.java.net/projects/code-tools/jmh/">jmh</a>, an OpenJDK framework for micro-benchmarks. The benchmark consists of publishing events to the queues, line 35 for the vmlens queue and line 48 for the JDK queue using WRITING_THREAD_COUNT threads. The events are read in line 41 for vmlens and line 58 for JDK using one thread. The vmlens queue reads all currently available events in one call and calls the callback function execute, line 20, for each event.</p> <p>You can download the source code of the benchmark from GitHub <a href="https://github.com/ThomasKrieger/benchmarks">here</a>.</p> <h1><a href="#results" id="results"></a>Results</h1> <p>The benchmark was run on an Intel i5 4 core CPU. All tests were run with the following jmh parameters: <code>-wi 10 -i 50 -f 5 -tu ms</code>. The following graph shows the throughput in operation per milliseconds for one to 8 writing threads:</p> <p><img class="center-block img-responsive" src="/img/articles/vmlens_queue_benchmark/time.png" alt="reentrantlock cheat sheet" /></p> <h1><a href="#conclusion-and-next-steps" id="conclusion-and-next-steps"></a>Conclusion and next steps</h1> <p>As we could see it is possible to achieve better throughput using a specialized queue than the generic java.util.concurrent.BlockingQueue for a blocking multiple writer single reader queue. When we use this queue for writing to the file system the limiting factor is the reading thread. This thread not only needs to collect all the data but also write it to the file system. So to improve the performance further I plan to collect the data and probably zip it in the blocked writing threads.</p> <p>I would be glad to hear from you about the techniques you use to write to the file system.</p> Tue, 30 Jan 2018 23:00:00 GMT http://vmlens.com/articles/vmlens_queue_benchmark 2018-01-30T23:00:00Z 5 Tips for Performant, Thread-Safe Java From ConcurrentHashMap http://vmlens.com/articles/5_tips_from_concurrent_hashmap java.util.concurrent.ConcurrentHashMap is a highly optimized concurrent hash map implementation. Here are 5 tips we can learn from its implementation: <p>java.util.concurrent.ConcurrentHashMap is a highly optimized concurrent hash map implementation. Here are 5 tips we can learn from its implementation:</p> <p><strong>Disclaimer: The techniques described here increase the complexity of the code, making it harder to reason about it and test. So please only apply have them when you have seen through profiling that your code is on the hot path.</strong></p> <h1><a href="#use-bit-operations-instead-of-expensive-mathematical-operations" id="use-bit-operations-instead-of-expensive-mathematical-operations"></a>Use bit operations instead of expensive mathematical operations</h1> <p>An example of this technique is the use of the and operation instead modulo in ConcurrentHashMap. ConcurrentHashMap stores all values in an array. To calculate the position where an entry should be stored we need to calculate the hash code modulo the array size. When the array size is a power of two as in the case of the ConcurrentHashmap we can write hash code and ( array size - 1 ) instead of hash code modulo size. The following shows this for a size of 8 and a hash code of 23:</p> <p><pre class="brush: java"> int sizeMinusOne = 0b00000111; // 7 int hashCode = 0b00010111; // 23 int positionInArray = hashCode &amp; sizeMinusOne; // 7 </pre></p> <p>This is for example done in the get method:</p> <p><pre class="brush: java"> public V get(Object key) { Node&lt;K,V&gt;[] tab; Node&lt;K,V&gt; e, p; int n, eh; K ek; int h = spread(key.hashCode()); if ((tab = table) != null &amp;&amp; (n = tab.length) &gt; 0 &amp;&amp; (e = tabAt(tab, (n - 1) &amp; h)) != null) { // other code omitted } return null; }</pre></p> <p>In line 5 the array position is calculated using (n - 1) &amp; h. <a href="https://lustforge.com/2016/05/08/modulo-operator-performance-impact/">The and operation is about 20 percent faster than the modulo operation.</a> So if you have an expensive mathematical operation inside your critical path it is a good idea to see if you can replace it with a bit operation.</p> <h1><a href="#use-fine-grained-locks" id="use-fine-grained-locks"></a>Use fine-grained locks</h1> <p>If multiple threads are accessing the same lock it becomes a bottleneck. A solution to this problem is to use fine-grained locking. As can be seen in the putVal method ConcurrentHashMap uses the array elements as monitors for synchronized locks:</p> <p><pre class="brush: java"> final V putVal(K key, V value, boolean onlyIfAbsent) { if (key == null || value == null) throw new NullPointerException(); int hash = spread(key.hashCode()); int binCount = 0; for (Node&lt;K,V&gt;[] tab = table;;) { Node&lt;K,V&gt; f; int n, i, fh; if (tab == null || (n = tab.length) == 0) tab = initTable(); else if ((f = tabAt(tab, i = (n - 1) &amp; hash)) == null) { if (casTabAt(tab, i, null, new Node&lt;K,V&gt;(hash, key, value, null))) break; // no lock when adding to empty bin } else if ((fh = f.hash) == MOVED) tab = helpTransfer(tab, f); else { V oldVal = null; synchronized (f) { // other code omitted } } addCount(1L, binCount); return null; }</pre></p> <p>If we need to change a non-empty array element it is locked using a synchronized block with the array element as the monitor. This is done with the synchronized block in line 18 on the array element received in line 9. If the hash function used disperses the elements properly among the array elements, threads access different array elements and therefore synchronize on different monitors.</p> <p>So to improve the scalability use the smallest independent value as the lock. Using this technique leads to many lock objects. Since reentrant locks consume more memory than the usage of synchronized blocks, this technique should be used with synchronized blocks instead of reentrant locks.</p> <h1><a href="#use-volatile-fields-for-reading-and-locks-for-writing" id="use-volatile-fields-for-reading-and-locks-for-writing"></a>Use volatile fields for reading and locks for writing</h1> <p>As we have seen writing in the putVal method uses a lock on the array element. As can be seen in the get method reading do not uses locks but only consists of reading from a volatile field:</p> <p><pre class="brush: java"> public V get(Object key) { Node&lt;K,V&gt;[] tab; Node&lt;K,V&gt; e, p; int n, eh; K ek; int h = spread(key.hashCode()); if ((tab = table) != null &amp;&amp; (n = tab.length) &gt; 0 &amp;&amp; (e = tabAt(tab, (n - 1) &amp; h)) != null) { if ((eh = e.hash) == h) { if ((ek = e.key) == key || (ek != null &amp;&amp; key.equals(ek))) return e.val; } else if (eh &lt; 0) return (p = e.find(h, key)) != null ? p.val : null; while ((e = e.next) != null) { if (e.hash == h &amp;&amp; ((ek = e.key) == key || (ek != null &amp;&amp; key.equals(ek)))) return e.val; } } return null; }</pre></p> <p>The array element is received in line 5 using the method tabAt. The volatile read is done in the method tabAt as can be seen here, using the method getObjectVolatile from sun.misc.Unsafe :</p> <p><pre class="brush: java"> static final &lt;K,V&gt; Node&lt;K,V&gt; tabAt(Node&lt;K,V&gt;[] tab, int i) { return (Node&lt;K,V&gt;)U.getObjectVolatile(tab, ((long)i &lt;&lt; ASHIFT) + ABASE); }</pre></p> <p>When you have a class with many reads and some writes use this technique. Reading simply consists reading from a volatile field while writing uses a lock. In ConcurrentHashMap the values of the Node are directly modified. This makes it necessary to declare the fields of this class also as volatile:</p> <p><pre class="brush: java"> static class Node&lt;K,V&gt; implements Map.Entry&lt;K,V&gt; { final int hash; final K key; volatile V val; volatile Node&lt;K,V&gt; next; // methods omitted }</pre></p> <p>This has the disadvantage that the values can change while you read them. An easier variation of this technique is used in <a href="http://vmlens.com#2-use-volatile-fields-for-reading-and-locks-for-writing">java.util.concurrent.CopyOnWriteArrayList which include a copy of the data during the write.</a></p> <h1><a href="#lazily-create-immutable-data" id="lazily-create-immutable-data"></a>Lazily create immutable data</h1> <p>ConcurrentHashMap has multiple functions to create a view on this collection, like the following</p> <p><pre class="brush: java">private transient KeySetView&lt;K,V&gt; keySet; public KeySetView&lt;K,V&gt; keySet() { KeySetView&lt;K,V&gt; ks; return (ks = keySet) != null ? ks : (keySet = new KeySetView&lt;K,V&gt;(this, null)); } public static class KeySetView&lt;K,V&gt; extends CollectionView&lt;K,V,K&gt; implements Set&lt;K&gt;, java.io.Serializable { private static final long serialVersionUID = 7249069246763182397L; private final V value; KeySetView(ConcurrentHashMap&lt;K,V&gt; map, V value) { // non-public super(map); this.value = value; } // methods omitted } </pre></p> <p>As we can see in line 1 the field keySet is not volatile and the method keySet(), line 3 till 6 is not synchronized. This leads to two problems, first the KeySetView object is not correctly published and second it might lead to the creation of multiple KeySetView objects. Through the usage of the final field in line 11, we make sure <a href="https://shipilev.net/blog/2014/jmm-pragmatics/#_part_v_finals">that the object gets correctly initialized even when they are incorrectly published</a>. And the creation of multiple objects does not lead to inconsistent state since they are immutable.</p> <h1><a href="#use-javautilconcurrentatomiclongadder-for-counting" id="use-javautilconcurrentatomiclongadder-for-counting"></a>Use java.util.concurrent.atomic.LongAdder for counting</h1> <p>Every writing and deleting threads needs to update the size counter of the collection. So the modification of the size becomes a bottleneck even if we use the atomic methods from java.util.concurrent.atomic.AtomicLong. To solve this problem ConcurrentHashMap uses the class CounterCell:</p> <p><pre class="brush: java"> /** * A padded cell for distributing counts. Adapted from LongAdder * and Striped64. See their internal docs for explanation. */ @sun.misc.Contended static final class CounterCell { volatile long value; CounterCell(long x) { value = x; } }</pre></p> <p>To implement this mechanism yourself is probably not a good idea since it is rather complicated. Luckily the JDK provides an implementation of the technique used, the class java.util.concurrent.atomic.LongAdder. The idea behind the counter cells is described in the java doc of java.util.concurrent.atomic.LongAdder:</p> <blockquote> <p>One or more variables that together maintain an initially zero long sum. When updates (method add(long)) are contended across threads, the set of variables may grow dynamically to reduce contention.</p> </blockquote> <h1><a href="#what-is-next" id="what-is-next"></a>What is next?</h1> <p>As we could see ConcurrentHashMap is full of ticks to write high-performance yet still thread-safe java. The next time we will look at java.util.concurrent.ConcurrentSkipListMap. I would be glad to hear from you about the techniques you use to achieve high-performance thread-safe classes.</p> Thu, 18 Jan 2018 23:00:00 GMT http://vmlens.com/articles/5_tips_from_concurrent_hashmap 2018-01-18T23:00:00Z 7 Techniques for thread-safe classes http://vmlens.com/articles/techniques_for_thread_safety Almost every Java application uses threads. It is, therefore, necessary to write classes in a thread-safe way, which can be achieved by one of the following techniques: <h1><a href="#7-techniques-for-thread-safe-classes" id="7-techniques-for-thread-safe-classes"></a>7 Techniques for thread-safe classes</h1> <p>Almost every Java application uses threads. A web server like Tomcat process each request in a separate worker thread, fat clients process long-running requests in dedicated worker threads, and even batch processes use the java.util.concurrent.ForkJoinPool to improve performance.</p> <p>It is, therefore, necessary to write classes in a thread-safe way, which can be achieved by one of the following techniques:</p> <h2><a href="#no-state" id="no-state"></a>No state</h2> <p>When multiple threads access the same instance or static variable you must somehow coordinate the access to this variable. The easiest way to do this is simply by avoiding instance or static variables. Methods in classes without instance variables do only use local variables and method arguments. The following example shows such a method which is part of the class java.lang.Math:</p> <p><pre class="brush: java">public static int subtractExact(int x, int y) { int r = x - y; if (((x ^ y) &amp; (x ^ r)) &lt; 0) { throw new ArithmeticException(&quot;integer overflow&quot;); } return r; }</pre></p> <h2><a href="#no-shared-state" id="no-shared-state"></a>No shared state</h2> <p>If you can not avoid state do not share the state. The state should only be owned by a single thread. <a href="http://flylib.com/books/en/2.558.1/why_are_guis_single_threaded_.html">An example of this technique is the event processing thread of the SWT or Swing graphical user interface frameworks</a>.</p> <p>You can achieve thread-local instance variables by extending the thread class and adding an instance variable. In the following example, the field pool and workQueue are local to a single worker thread. <pre class="brush: java">package java.util.concurrent; public class ForkJoinWorkerThread extends Thread { final ForkJoinPool pool; final ForkJoinPool.WorkQueue workQueue; }</pre></p> <p>The other way to achieve thread-local variables is to use the class java.lang.ThreadLocal for the fields you want to make thread-local. Here is an example of an instance variable using java.lang.ThreadLocal:</p> <p><pre class="brush: java">public class CallbackState { public static final ThreadLocal&lt;CallbackStatePerThread&gt; callbackStatePerThread = new ThreadLocal&lt;CallbackStatePerThread&gt;() { @Override protected CallbackStatePerThread initialValue() { return getOrCreateCallbackStatePerThread(); } }; }</pre></p> <p>You wrap the type of your instance variable inside the java.lang.ThreadLocal. You can provide an initial value for your java.lang.ThreadLocal through the method initialValue().</p> <p>The following shows how to use the instance variable: <pre class="brush: java">CallbackStatePerThread callbackStatePerThread = CallbackState.callbackStatePerThread.get();</pre></p> <p>Through calling the method get() you receive the object associated with the current thread.</p> <p>Since in application servers a pool of many threads is used to process requests, java.lang.ThreadLocal leads to a high memory consumption in this environment. java.lang.ThreadLocal is therefore not recommended for classes executed by the request processing threads of an application server.</p> <h2><a href="#message-passing" id="message-passing"></a>Message passing</h2> <p>If you do not share state using the above techniques you need a way for the threads to communicate. A technique to do this is by passing messages between threads. You can implement message passing using a concurrent queue from the package java.util.concurrent. Or, better yet, use a framework like <a href="https://akka.io/">Akka</a>, a framework for actor style concurrency. The following example shows how to send a message with Akka:</p> <p><pre class="brush: java">target.tell(message, getSelf());</pre></p> <p>and receive a message:</p> <p><pre class="brush: java">@Override public Receive createReceive() { return receiveBuilder() .match(String.class, s -&gt; System.out.println(s.toLowerCase())) .build(); }</pre></p> <h2><a href="#immutable-state" id="immutable-state"></a>Immutable state</h2> <p>To avoid the problem that the sending thread changes the message during the message is read by another thread, messages should be immutable. <a href="https://doc.akka.io/docs/akka/2.5.5/java/actors.html#messages-and-immutability">The Akka framework, therefore, has the convention that all messages have to be immutable</a></p> <p>When you implement an immutable class you should declare its fields as final. This not only makes sure that the compiler can check that the fields are in fact immutable <a href="https://shipilev.net/blog/2014/jmm-pragmatics/#_part_v_finals">but also makes them correctly initialized even when they are incorrect published.</a> Here is an example of a final instance variable:</p> <p><pre class="brush: java">public class ExampleFinalField { private final int finalField; public ExampleFinalField(int value) { this.finalField = value; } }</pre></p> <p>final is a field modifier. It makes the field immutable not the object the field references to. So the type of the final field should be a primitive type like in the example or also an immutable class.</p> <h2><a href="#use-the-data-structures-from-javautilconcurrent" id="use-the-data-structures-from-javautilconcurrent"></a>Use the data structures from java.util.concurrent</h2> <p>Message passing uses concurrent queues for the communication between threads. Concurrent Queues are one of the data structures provided in the package java.util.concurrent. This package provides classes for concurrent maps, queues, dequeues, sets and lists. Those data structures are highly optimized and tested for thread safety.</p> <h2><a href="#synchronized-blocks" id="synchronized-blocks"></a>Synchronized blocks</h2> <p>If you can not use one of the above techniques use synchronized locks. By putting a block inside a synchronized block you make sure that only one thread at a time can execute this section.</p> <p><pre class="brush: java">synchronized(lock) { i++; }</pre></p> <p>Beware that when you use multiple nested synchronize blocks you risk deadlocks. <a href="http://vmlens.com/articles/detect_deadlocks/">A deadlock happens when two threads are trying to acquire a lock held by the other thread.</a></p> <h2><a href="#volatile-fields" id="volatile-fields"></a>Volatile fields</h2> <p>Normal, nonvolatile fields, can be cached in registers or caches. Through the declaration of a variable as volatile, you tell the JVM and the compiler to always return the latest written value. This not only applies to the variable itself but to all values written by the thread which has written to the volatile field. The following shows an example of a volatile instance variable:</p> <p><pre class="brush: java">public class ExampleVolatileField { private volatile int volatileField; }</pre></p> <p><a href="http://vmlens.com/articles/3_tips_volatile_fields/">You can use volatile fields if the writes do not depend on the current value. Or if you can make sure that only one thread at a time can update the field.</a></p> <p>volatile is a field modifier. It makes the field itself volatile not the object it references. In case of an array you need to use java.util.concurrent.atomic.AtomicReferenceArray to access the array elements in a volatile way. See the <a href="http://vmlens.com/articles/org_springframework_util_ConcurrentReferenceHashMap_is_not_thread_safe/">race condition in org.­springframework.­util.­ConcurrentReferenceHashMap</a> as an example of this error.</p> <h2><a href="#even-more-techniques" id="even-more-techniques"></a>Even more techniques</h2> <p>I excluded the following more advanced techniques from this list: Atomic updates, a technique in which you call atomic instructions like compare and set provided by the CPU, java.util.concurrent.locks.ReentrantLock, a lock implementation which provides more flexibility than synchronized blocks, java.util.concurrent.locks.ReentrantReadWriteLock, a lock implementation in which reads do not block reads and java.util.concurrent.locks.StampedLock a nonreeantrant Read-Write lock with the possibility to optimistically read values.</p> <h1><a href="#conclusion" id="conclusion"></a>Conclusion</h1> <p>The best way to achieve thread safety is to avoid shared state. For the state, you need to share you can either use message parsing together with immutable classes or the concurrent data structures together with synchronized blocks and volatile fields.</p> <p>I would be glad to hear from you about the techniques you use to achieve thread-safe classes.</p> Wed, 10 Jan 2018 23:00:00 GMT http://vmlens.com/articles/techniques_for_thread_safety 2018-01-10T23:00:00Z 3 Tips for volatile fields in java http://vmlens.com/articles/3_tips_volatile_fields Volatile fields are one of built-in mechanism to write multi-threaded java. In the following I collected three tips on when and how to use volatile fields in practice... <h1><a href="#3-tips-for-volatile-fields-in-java" id="3-tips-for-volatile-fields-in-java"></a>3 Tips for volatile fields in java</h1> <p>Volatile fields are one of built-in mechanism to write multi-threaded java.</p> <blockquote> <p>Volatile variables are not cached in registers or in caches where they are hidden from other processors, so a read of a volatile variable always returns the most recent write by any thread. ... The visibility effects of volatile variables extend beyond the value of the volatile variable itself. When thread A writes to a volatile variable and subsequently thread B reads that same variable, the values of all variables that were visible to A prior to writing to the variable become visible to B after reading the volatile variable.<br /> — <a href="http://jcip.net/">Java Concurrency in Practice - Brian Goetz, et al.</a></p> </blockquote> <p>In the following I collected three tips on when and how to use volatile fields in practice:</p> <h1><a href="#1-use-volatile-fields-when-writes-do-not-depend-on-its-current-value" id="1-use-volatile-fields-when-writes-do-not-depend-on-its-current-value"></a>1) Use volatile fields when writes do not depend on its current value.</h1> <p>An example is a flag to stop a worker thread from another thread:</p> <pre class="brush: java"> public class WorkerThread extends Thread { private volatile boolean isRunning = true; @Override public void run() { while(isRunning) { // execute a task } } public void stopWorker() { isRunning = false; } } </pre> <p>The WorkerThread executes his tasks in a while loop, line 5. It checks the volatile field isRunning in each iteration and stops processing if the field is false. This allows other threads to stop the WorkerThread by calling the method stopWorker which sets the value of the field to false. Since a thread can call the method stopWorker even if the WorkerThread is already stopped, the write to the field can be executed independently of its current value.</p> <p>By declaring the field volatile we make sure that the WorkerThread sees the update done in another Thread and does not run forever.</p> <h1><a href="#2-use-volatile-fields-for-reading-and-locks-for-writing" id="2-use-volatile-fields-for-reading-and-locks-for-writing"></a>2) Use volatile fields for reading and locks for writing</h1> <p>The java.util.concurrent.CopyOnWriteArrayList get and set methods are an example of this tip:</p> <pre class="brush: java"> public class CopyOnWriteArrayList&lt;E&gt; implements List&lt;E&gt;, RandomAccess, Cloneable, java.io.Serializable { private transient volatile Object[] array; final Object[] getArray() { return array; } final void setArray(Object[] a) { array = a; } private E get(Object[] a, int index) { return (E) a[index]; } public E get(int index) { return get(getArray(), index); } public E set(int index, E element) { final ReentrantLock lock = this.lock; lock.lock(); try { Object[] elements = getArray(); E oldValue = get(elements, index); if (oldValue != element) { int len = elements.length; Object[] newElements = Arrays.copyOf(elements, len); newElements[index] = element; setArray(newElements); } else { // Not quite a no-op; ensures volatile write semantics setArray(elements); } return oldValue; } finally { lock.unlock(); } } // Other fields and methods omitted } </pre> <p>The get method, line 13, simply reads the volatile field array and returns the value at the position index. Writing uses a lock to ensure that only one thread can modify the array at a given time. Line 18 acquires the lock and line 33 releases the lock. Writing requires copying the array when an element is changed, line 24 so that the reading threads do not see an inconsistent state. The writing thread then updates the array, line 25 and set the new array to the volatile field array, line 26.</p> <p>Using this tip only writes block writes. Compare this to using synchronized set and get methods where each operation block all other operations. <a href="https://www.javaspecialists.eu/archive/Issue165.html">Or java.util.concurrent.locks.ReentrantReadWriteLock where too many readers can lead to starvation of writers.</a></p> <p>This is especially a problem for older JDKs. Here are the number from Akhil Mittal in a <a href="https://dzone.com/articles/3-tips-for-volatile-fields-in-java">DZone</a> comment to this article:</p> <pre style="white-space: pre-wrap; word-break: keep-all;"> Java 6 RO= 4, RW= 4, fair=false 4,699,560 584,991 Java 9 RO= 4, RW= 4, fair=false 2,133,904 3,289,220 </pre> <h1><a href="#3-use-with-jdk-9-varhandle-for-atomic-operations" id="3-use-with-jdk-9-varhandle-for-atomic-operations"></a>3) Use with JDK 9 VarHandle for atomic operations.</h1> <p>All modern CPU provide instructions to atomically compare and set or increment and get values. Those operations are used internally by the JVM to implement synchronized locks. Prior to JDK 1.9, they were available for Java applications only through classes in the java.util.concurrent.atomic package or by using the private java API sun.misc.Unsafe. With the new JDK 9 VarHandle, it is now possible to execute such operations directly on volatile fields. The following shows the AtomicBoolean compareAndSet method implemented using VarHandles:</p> <pre class="brush: java"> public class AtomicBoolean implements java.io.Serializable { private static final VarHandle VALUE; static { try { MethodHandles.Lookup l = MethodHandles.lookup(); VALUE = l.findVarHandle(AtomicBoolean.class, "value", int.class); } catch (ReflectiveOperationException e) { throw new Error(e); } } private volatile int value; public final boolean compareAndSet(boolean expectedValue, boolean newValue) { return VALUE.compareAndSet(this, (expectedValue ? 1 : 0), (newValue ? 1 : 0)); } // Other fields and methods omitted } </pre> <p>The VarHandle works similar to the class java.lang.reflect.Field. You need to lookup a VarHandle from the class which contains the field using the name of the field, line 6. To execute a compareAndSet operation on the field we need to call the VarHandle with the object of the field, the expected and the new value, line 13.</p> <h1><a href="#conclusion" id="conclusion"></a>Conclusion</h1> <p>You can use volatile fields if the writes do not depend on the current value as in tip one. Or if you can make sure that only one thread at a time can update the field as in tip two and three. I think that those three tips cover the more common ways to use volatile fields. <a href="http://vmlens.com/articles/lock_free_executor_service/">Read here</a> about a more exotic way to use volatile fields to implement a concurrent queue.</p> <p>I would be glad to hear from you about other ways to use volatile fields to achieve thread-safety.</p> Mon, 11 Dec 2017 23:00:00 GMT http://vmlens.com/articles/3_tips_volatile_fields 2017-12-11T23:00:00Z org.springframework.util.ConcurrentReferenceHashMap is not thread-safe http://vmlens.com/articles/org_springframework_util_ConcurrentReferenceHashMap_is_not_thread_safe The class org.springframework.util.ConcurrentReferenceHashMap which is part of the spring framework is not thread-safe. <p>The class org.springframework.util.ConcurrentReferenceHashMap which is part of the spring framework is not thread-safe. We see this in the following junit test:</p> <pre class="brush: java"> @RunWith(ConcurrentTestRunner.class) public class TestSpringConcurrentReferenceHashMap { private ConcurrentReferenceHashMap map = new ConcurrentReferenceHashMap(); private KeyAndValue[] keyAndValue; private static final int SIZE = 100; public TestSpringConcurrentReferenceHashMap() { keyAndValue = new KeyAndValue[SIZE]; for(int i = 0 ; i&lt; SIZE ; i++) { keyAndValue[i] = new KeyAndValue(i); } } @Test public void test() { for(int i = 0 ; i &lt; SIZE ; i++) { AtomicInteger result = (AtomicInteger)map.get(i); if( result == null ) { result = (AtomicInteger) map.putIfAbsent( keyAndValue[i].key , keyAndValue[i].value ); } if( result != null ) { result.addAndGet(1); } } } class KeyAndValue { KeyAndValue(int key) { this.key = key; value= new AtomicInteger(); } Integer key; AtomicInteger value; } } </pre> <p>The ConcurrentTestRunner runs the test by 4 threads in parallel. <a href="http://vmlens.com">vmlens</a>, a tool to detect <a href="http://vmlens.com/articles/detect_deadlocks/">deadlocks</a> and <a href="http://vmlens.com/articles/detecting-java-race-conditions-with-tests/">race conditions</a> during the test run, shows the following race conditions:</p> <p><img src="/img/articles/org_springframework_util_ConcurrentReferenceHashMap_is_not_thread_safe/races.png" /></p> <p>For example, the first race is the access to a value of an array without correct synchronization. In the method getReference the value of the array is read in line 12</p> <pre class="brush: java"> @Nullable public Reference&lt;K, V&gt; getReference(@Nullable Object key, int hash, Restructure restructure) { if (restructure == Restructure.WHEN_NECESSARY) { restructureIfNecessary(false); } if (this.count == 0) { return null; } // Use a local copy to protect against other threads writing Reference&lt;K, V&gt;[] references = this.references; int index = getIndex(hash, references); Reference&lt;K, V&gt; head = references[index]; return findInChain(head, key, hash); } </pre> <p>And in the method add the value of the array is written in line 22.</p> <pre class="brush: java"> @Nullable public &lt;T&gt; T doTask(final int hash, final Object key, final Task&lt;T&gt; task) { boolean resize = task.hasOption(TaskOption.RESIZE); if (task.hasOption(TaskOption.RESTRUCTURE_BEFORE)) { restructureIfNecessary(resize); } if (task.hasOption(TaskOption.SKIP_IF_EMPTY) &amp;&amp; this.count == 0) { return task.execute(null, null, null); } lock(); try { final int index = getIndex(hash, this.references); final Reference&lt;K, V&gt; head = this.references[index]; Reference&lt;K, V&gt; reference = findInChain(head, key, hash); Entry&lt;K, V&gt; entry = (reference != null ? reference.get() : null); Entries entries = new Entries() { @Override public void add(V value) { @SuppressWarnings(&quot;unchecked&quot;) Entry&lt;K, V&gt; newEntry = new Entry&lt;&gt;((K) key, value); Reference&lt;K, V&gt; newReference = Segment.this.referenceManager.createReference(newEntry, hash, head); Segment.this.references[index] = newReference; Segment.this.count++; } }; return task.execute(reference, entry, entries); } finally { unlock(); if (task.hasOption(TaskOption.RESTRUCTURE_AFTER)) { restructureIfNecessary(resize); } } } </pre> <p>The field references is declared as volatile. But the access to the value of the array references[index] is not volatile. To make the access to an array volatile we need to use java.util.concurrent.atomic.AtomicReferenceArray.</p> <h1><a href="#the-consequence-of-the-race-conditions" id="the-consequence-of-the-race-conditions"></a>The consequence of the race conditions</h1> <p>To see the consequence of the race conditions we use <a href="https://wiki.openjdk.java.net/display/CodeTools/jcstress">jcstress</a>, an open JDK code tool: <q cite="https://wiki.openjdk.java.net/display/CodeTools/jcstress">The Java Concurrency Stress tests (jcstress) is an experimental harness and a suite of tests to aid the research in the correctness of concurrency support in the JVM, class libraries, and hardware.</q></p> <p>I use the following <a href="https://github.com/vmlens/jcstress-examples/blob/master/src/main/java/com/vmlens/stressTest/tests/BigDecimalToString.java">test class:</a></p> <pre class="brush: java"> @JCStressTest @Outcome(id = &quot;10&quot;, expect = Expect.ACCEPTABLE, desc = &quot;Default outcome.&quot;) @State public class SpringConcurrentHashMap { private final ConcurrentReferenceHashMap map = new ConcurrentReferenceHashMap(); private final KeyAndValue[] keyAndValue; private final int SIZE = 10; class KeyAndValue { KeyAndValue(int key) { this.key = key; value = new AtomicInteger(); } final Integer key; final AtomicInteger value; } public SpringConcurrentHashMap() { keyAndValue = new KeyAndValue[SIZE]; for (int i = 0; i &lt; SIZE; i++) { keyAndValue[i] = new KeyAndValue(i); } } public void test() { for (int i = 0; i &lt; SIZE; i++) { AtomicInteger result = (AtomicInteger) map.get(i); if (result == null) { result = (AtomicInteger) map.putIfAbsent(keyAndValue[i].key, keyAndValue[i].value); } if (result != null) { result.addAndGet(1); } } } @Actor public void actor1() { test(); } @Actor public void actor2() { test(); } @Arbiter public void arbiter(IntResult1 r) { int sum = 0; for (int i = 0; i &lt; SIZE; i++) { sum += ((AtomicInteger) map.get(i)).get(); } r.r1 = sum; } } </pre> <p>The jcstress tool runs each actor in a separate thread, repeating the test multiple times. We have two actor method, each creating a new AtomicInteger in the hash map if no value is there or incrementing the AtomicInteger if a value existed. Therefore we expect the sum for all values in the hash map after the thread was run 10. If I run this test using the stress mode on my development machine, an intel i5 4 core CPU with oracle JDK 8, I see the following result:</p> <pre class="brush: java"> 2 matching test results. [FAILED] com.vmlens.stressTest.tests.SpringConcurrentHashMap (JVM args: [-client]) Observed state Occurrences Expectation Interpretation 10 92,597,143 ACCEPTABLE Default outcome. 9 7 FORBIDDEN No default case provided, assume FORBIDDEN [FAILED] com.vmlens.stressTest.tests.SpringConcurrentHashMap (JVM args: [-server]) Observed state Occurrences Expectation Interpretation 10 98,517,611 ACCEPTABLE Default outcome. 9 9 FORBIDDEN No default case provided, assume FORBIDDEN </pre> <p>In a small percentage of all cases, 7 cases for -client and 9 cases for -server the count is less than 10. So in some cases, a value in the map is overridden by a new value.</p> Mon, 27 Nov 2017 23:00:00 GMT http://vmlens.com/articles/org_springframework_util_ConcurrentReferenceHashMap_is_not_thread_safe 2017-11-27T23:00:00Z How to test if your spring application is thread-safe http://vmlens.com/articles/test_spring_application In the following, I want to show you how to test if your spring application is thread-safe. As an example application, I use the spring petclinic project. <p>In the following, I want to show you how to test if your spring application is thread-safe. As an example application, I use the spring <a href="https://github.com/spring-projects/spring-petclinic">petclinic</a> project.</p> <p>To detect concurrency bugs during our tests we use <a href="http://vmlens.com">vmlens</a>. vmlens traces the test execution and analyzes the trace afterward. It detects <a href="http://vmlens.com/articles/detect_deadlocks/">deadlocks</a> and <a href="http://vmlens.com/articles/detecting-java-race-conditions-with-tests/">race conditions</a> during the test run.</p> <h1><a href="#testing" id="testing"></a>Testing</h1> <p>To test the spring project we parallelize the existing unit tests. The following shows test method runs the existing test shouldFindAllPetTypes of the ClinicServiceTests class in parallel:</p> <pre class="brush: java"> public class ClinicServiceTests { @Test public void testMultithreaded() throws InterruptedException { TestUtil.runMultithreaded( new Runnable() { public void run() { try{ shouldFindAllPetTypes(); } catch(Exception e) { e.printStackTrace(); } } } , 5); } </pre> <p>The TestUtil.runMultithreaded method runs the runnable with n threads in parallel:</p> <pre class="brush: java"> public static void runMultithreaded(Runnable runnable, int threadCount) throws InterruptedException { List&lt;Thread&gt; threadList = new LinkedList&lt;Thread&gt;(); for(int i = 0 ; i &lt; threadCount; i++) { threadList.add(new Thread(runnable)); } for( Thread t : threadList) { t.start(); } for( Thread t : threadList) { t.join(); } } </pre> <p>You can find the source of the class TestUtil at GitHub <a href="https://github.com/ThomasKrieger/concurrent-junit/blob/master/concurrent-junit/src/main/java/com/anarsoft/vmlens/concurrent/junit/TestUtil.java">here</a>. After running the junit test we see the following report in vmlens:</p> <p><img src="/img/articles/test_spring_application/race_conditions.png" class="img-responsive center-block" alt="race conditions in spring petclinic" style="width: 100%" /></p> <h1><a href="#analyzing" id="analyzing"></a>Analyzing</h1> <p>Let us look at one of the races found, the race at accessing the field org.hsqldb.HsqlNameManager.sysNumber.</p> <p><img src="/img/articles/test_spring_application/reading.png" class="img-responsive center-block" alt="race condition at org.hsqldb.HsqlNameManager.sysNumber, reading thread" /></p> <p><img src="/img/articles/test_spring_application/writing.png" class="img-responsive center-block" alt="race condition at org.hsqldb.HsqlNameManager.sysNumber, writing thread" /></p> <p>The access to the field is locked in the methods org.hsqldb.StatementManager.compile, org.hsqldb.Session.execute and org.hsqldb.jdbc.JDBCConnection.prepareStatement but each thread uses a different monitor. Here is as an example the JDBCConnection prepareStatement method:</p> <pre class="brush: java"> public synchronized PreparedStatement prepareStatement( String sql) throws SQLException { checkClosed(); try { return new JDBCPreparedStatement(this, sql, JDBCResultSet.TYPE_FORWARD_ONLY, JDBCResultSet.CONCUR_READ_ONLY, rsHoldability, ResultConstants.RETURN_NO_GENERATED_KEYS, null, null); } catch (HsqlException e) { throw JDBCUtil.sqlException(e); } } </pre> <p>The problem is that the synchronization happens on the PreparedStatement and Session which is created for each thread, while HsqlNameManager is shared between all threads. That HsqlNameManager is shared between all threads can be seen in the method org.hsqldb.Table.createPrimaryKey:</p> <pre class="brush: java"> public void createPrimaryKey(HsqlName indexName, int[] columns, boolean columnsNotNull) { if (primaryKeyCols != null) { throw Error.runtimeError(ErrorCode.U_S0500, "Table"); } if (columns == null) { columns = ValuePool.emptyIntArray; } for (int i = 0; i < columns.length; i++) { getColumn(columns[i]).setPrimaryKey(true); } primaryKeyCols = columns; setColumnStructures(); primaryKeyTypes = new Type[primaryKeyCols.length]; ArrayUtil.projectRow(colTypes, primaryKeyCols, primaryKeyTypes); primaryKeyColsSequence = new int[primaryKeyCols.length]; ArrayUtil.fillSequence(primaryKeyColsSequence); HsqlName name = indexName; if (name == null) { name = database.nameManager.newAutoName("IDX", getSchemaName(), getName(), SchemaObject.INDEX); } createPrimaryIndex(primaryKeyCols, primaryKeyTypes, name); setBestRowIdentifiers(); } </pre> <p>Line 20 shows that the nameManager is part of the database object which is shared between all threads. The race happens in the method org.hsqldb.HsqlNameManager.newAutoName where the field sysNumber is read and written by the two threads:</p> <pre class="brush: java"> public HsqlName newAutoName(String prefix, String namepart, HsqlName schema, HsqlName parent, int type) { StringBuffer sb = new StringBuffer(); if (prefix != null) { if (prefix.length() != 0) { sb.append("SYS_"); sb.append(prefix); sb.append('_'); if (namepart != null) { sb.append(namepart); sb.append('_'); } sb.append(++sysNumber); } } else { sb.append(namepart); } HsqlName name = new HsqlName(this, sb.toString(), type, false); name.schema = schema; name.parent = parent; return name; } </pre> <p>In line 13 the field sysNumber is incremented by ++. The operation ++ is not one atomic operation but 6 byte code operations:</p> <pre class="brush: java">ALOAD 0: this DUP GETFIELD Counter.count : int ICONST_1 IADD PUTFIELD Counter.count : int </pre> <p>If two threads execute this in parallel, this might lead to a scenario where both threads read the same value and then both increment the value, leading to duplicate values. And since the field sysNumber is used to generate the primary key the race condition leads to duplicated primary keys.</p> Wed, 22 Nov 2017 23:00:00 GMT http://vmlens.com/articles/test_spring_application 2017-11-22T23:00:00Z An algorithm for a concurrent queue using only thread local and volatile fields http://vmlens.com/articles/lock_free_executor_service In the following, I want to show you an algorithm for a concurrent queue which supports one reading and multiple writing threads. The writing threads need only a read from a thread local field and a write to a volatile field to publish an event to the queue. <p>In the following, I want to show you an algorithm for a concurrent queue which supports one reading and multiple writing threads. The writing threads need only a read from a thread local field and a write to a volatile field to publish an event to the queue. Writing does not need &quot;compare and swap&quot; operations like the standard JDK concurrent queues, leading to an easier and potentially faster algorithm. A usage example is a background thread writing log events asynchronously to a file.</p> <h1><a href="#writing" id="writing"></a>Writing</h1> <p>The main idea is to use not one single queue but many. We use one queue per writing thread stored in a thread local field. Then the queue is a simple linked list using a volatile field for the next element and final for the value:</p> <pre class="brush: java"> public class LinkedList&lt;T&gt; implements Consumer&lt;T&gt; { volatile ListElementPointer&lt;T&gt; lastRead; private LinkedListElement&lt;T&gt; lastWritten; // Constructor omitted @Override public void accept(T event) { // Queue stopped logic omitted LinkedListElement&lt;T&gt; linkedListElement= new LinkedListElement&lt;T&gt;(event); if( lastWritten == null ) { lastWritten = linkedListElement; lastRead= new ListElementPointer&lt;T&gt;(lastWritten); } else { lastWritten.next = linkedListElement; lastWritten = lastWritten.next; } } } class LinkedListElement&lt;T&gt; { volatile LinkedListElement&lt;T&gt; next; final T event; // Constructor omitted } </pre> <p>Writing an element to the queue is implemented in the accept method, line 6. When it is the first element written, lastWritten and lastRead will be set to the new LinkedListElement line 10 till 13. Otherwise, the list is extended by the new LinkedListElement and lastWritten is moved to the end of the list, line 16 and 17.</p> <p>And here is the class storing each queue, called Consumer, in a thread local field:</p> <pre class="brush: java"> public class ThreadLocalConsumer&lt;T&gt; implements Consumer&lt;T&gt; { private final EventBus&lt;T&gt; theBus; private ThreadLocal&lt;Consumer&lt;T&gt;&gt; threadLocal = new ThreadLocal&lt;Consumer&lt;T&gt;&gt;(); // Constructor omitted public void accept(T event) { Consumer&lt;T&gt; consumer = threadLocal.get(); if( consumer == null ) { consumer = theBus.newConsumerForThreadLocalStorage(Thread.currentThread()); threadLocal.set( consumer ); } consumer.accept(event); } } </pre> <h1><a href="#reading" id="reading"></a>Reading</h1> <p>When reading the elements we must remember the last elements we read. We do this in the field lastRead in the LinkedList. The following shows the method used for reading elements from one queue:</p> <pre class="brush: java"> public void prozessWithoutReadCount(EventSink&lt;T&gt; eventSink) { if(list.lastRead != null ) { // First element read if( ! list.lastRead.isRead ) { eventSink.execute( list.lastRead.element.event ); list.lastRead.isRead = true; } LinkedListElement&lt;T&gt; current = list.lastRead.element; while( current.next != null ) { eventSink.execute( current.event ); current = current.next; } list.lastRead.element = current; } } </pre> <p>And here is the class ListElementPointer used to store the last read element:</p> <pre class="brush: java"> public class ListElementPointer&lt;T&gt; { LinkedListElement&lt;T&gt; element; boolean isRead; // Constructor omitted } </pre> <p>The field lastRead is initialized by the writing thread and afterward only modified by the reading thread.</p> <p>I skip the logic for creating new LinkedLists and reading multiple LinkedLists in the reading thread. You can see the complete source code <a href="https://github.com/vmlens/executor-service">here</a>.</p> <h1><a href="#usage" id="usage"></a>Usage</h1> <p>The queue is open source and the source code is available on GitHub <a href="https://github.com/vmlens/executor-service">here</a>. We use this queue in to write events asynchronously to a file for later analysis.</p> Wed, 01 Nov 2017 23:00:00 GMT http://vmlens.com/articles/lock_free_executor_service 2017-11-01T23:00:00Z How to test if your tomcat web application is thread safe http://vmlens.com/articles/test_tomcat_web_applications In the following, I want to show you how to test if your tomcat web application is thread-safe. As an example application, I use Jenkins deployed on an apache tomcat 9.0. <p>In the following, I want to show you how to test if your tomcat web application is thread-safe. As an example application, I use Jenkins deployed on an apache tomcat 9.0.</p> <p>To detect concurrency bugs during our tests we use <a href="http://vmlens.com">vmlens</a>. vmlens traces the test execution and analyzes the trace afterward. It detects <a href="http://vmlens.com/articles/detect_deadlocks/">deadlocks</a> and <a href="http://vmlens.com/articles/detecting-java-race-conditions-with-tests/">race conditions</a> during the test run.</p> <h1><a href="#testing" id="testing"></a>Testing</h1> <p>To enable vmlens we add it as java agent to the CATALINA_OPTS in catalina.sh on Linux or catalina.bat on windows:</p> <pre class="brush: java"> CATALINA_OPTS=&quot;-javaagent:&lt;Path to agent&gt; -Xmx8g&quot; </pre> <p>We also set a high enough heap size. After running Jenkins and executing some build jobs we see the following report in vmlens:</p> <p><img src="/img/articles/test_tomcat_web_applications/races.png" class="img-responsive center-block" alt="race condition in jenkins" /></p> <h1><a href="#analyzing" id="analyzing"></a>Analyzing</h1> <p>Let us look at one of the races found, the race at accessing the field hudson.UDPBroadcastThread.shutdown.</p> <p><img src="/img/articles/test_tomcat_web_applications/race_shutdown.png" class="img-responsive center-block" alt="race condition accessing UDPBroadcastThread.shutdown" /></p> <p>The thread &quot;Jenkins UDP 33848 monitoring thread&quot; reads the field in the race and the thread &quot;localhost-startStop-2&quot; writes it. Let us look at the class and the reading method run() and writing method shutdown().</p> <pre class="brush: java"> public class UDPBroadcastThread extends Thread { private boolean shutdown; public void run() { try { mcs.joinGroup(MULTICAST); ready.signal(); while(true) { byte[] buf = new byte[2048]; DatagramPacket p = new DatagramPacket(buf,buf.length); mcs.receive(p); SocketAddress sender = p.getSocketAddress(); // prepare a response TcpSlaveAgentListener tal = jenkins.getTcpSlaveAgentListener(); StringBuilder rsp = new StringBuilder("<hudson>"); tag(rsp,"version", Jenkins.VERSION); tag(rsp,"url", jenkins.getRootUrl()); tag(rsp,"server-id", jenkins.getLegacyInstanceId()); tag(rsp,"slave-port",tal==null?null:tal.getPort()); for (UDPBroadcastFragment f : UDPBroadcastFragment.all()) f.buildFragment(rsp,sender); rsp.append("</hudson>"); byte[] response = rsp.toString().getBytes("UTF-8"); mcs.send(new DatagramPacket(response,response.length,sender)); } } catch (ClosedByInterruptException e) { // shut down } catch (SocketException e) { if (shutdown) { // forcibly closed return; } // if we failed to listen to UDP, just silently abandon it, as a stack trace // makes people unnecessarily concerned, for a feature that currently does no good. LOGGER.log(Level.INFO, "Cannot listen to UDP port {0}, skipping: {1}", new Object[] {PORT, e}); LOGGER.log(Level.FINE, null, e); } catch (IOException e) { if (shutdown) return; // forcibly closed LOGGER.log(Level.WARNING, "UDP handling problem",e); udpHandlingProblem = true; } } public void shutdown() { shutdown = true; mcs.close(); interrupt(); } } </pre> The field shutdown is a nonvolatile field. It is read in line 28 and 35 in the method run and written in line 41 in the method shutdown <p>Since the field hudson.UDPBroadcastThread.shutdown is not volatile, it is not guaranteed that the &quot;Jenkins UDP 33848 monitoring thread&quot; sees the values set by the &quot;localhost-startStop-2&quot; thread.</p> <p>The &quot;Jenkins UDP 33848 monitoring thread&quot; might for example run on the first core while &quot;localhost-startStop-2&quot; on the second core of a multi-core CPU. The write to a normal field does not invalidate the cache of the cores. Therefore the &quot;Jenkins UDP 33848 monitoring thread&quot; still sees the cached old value.</p> Tue, 24 Oct 2017 22:00:00 GMT http://vmlens.com/articles/test_tomcat_web_applications 2017-10-24T22:00:00Z A new way to detect deadlocks during tests http://vmlens.com/articles/detect_deadlocks In the following, I want to show you a new way to detect deadlocks during tests. A deadlock happens when two threads are trying to acquire a lock held by the other thread. <p>In the following, I want to show you a new way to detect deadlocks during tests. A deadlock happens when two threads are trying to acquire a lock held by the other thread. To detect a deadlock you need to reach the exact time point when both threads are waiting for the other lock. Let us look at an example:</p> <pre class="brush: java"> import java.util.concurrent.ConcurrentHashMap; import java.util.function.BiFunction; import org.junit.Test; import org.junit.runner.RunWith; import com.anarsoft.vmlens.concurrent.junit.ConcurrentTestRunner; @RunWith(ConcurrentTestRunner.class) public class TestConcurrentHashMapCompute { private final ConcurrentHashMap&lt;Integer,Integer&gt; map = new ConcurrentHashMap&lt;Integer,Integer&gt;(); public TestConcurrentHashMapCompute() { map.put(1, 1); map.put(2, 2); } @Test public void update12() { map.compute( 1 , new BiFunction&lt;Integer,Integer,Integer&gt;() { public Integer apply(Integer k, Integer v) { map.put( 2 , 1); return v; } } ); } @Test public void update21() { map.compute( 2 , new BiFunction&lt;Integer,Integer,Integer&gt;() { public Integer apply(Integer k, Integer v) { map.put( 1 , 1); return v; } } ); } } </pre> <p>The ConcurrentTestRunner runs the JUnit test methods by 4 threads in parallel. The test succeeds at least almost all the time.</p> <p>One way to see the deadlock hidden in this test is to execute the test by more threads multiple times by a machine with many cores. The other way is to trace the test execution and to analyze the lock order afterward.</p> <p>We can do this by adding the <a href="http://vmlens.com">vmlens</a> agent to the VM arguments of our test. After analyzing the test run vmlens reports the following deadlock:</p> <pre class="brush: java"> - deadlock: Monitor@java.util.concurrent.ConcurrentHashMap.putVal()&lt;-&gt;Monitor@java.util.concurrent.ConcurrentHashMap.putVal() parent2Child: thread: Thread-4 stack: - java.util.concurrent.ConcurrentHashMap.putVal &lt;&lt;Monitor@java.util.concurrent.ConcurrentHashMap.putVal()&gt;&gt; - java.util.concurrent.ConcurrentHashMap.put - com.anarsoft.vmlens.concurrent.example.TestConcurrentHashMapCompute$2.apply - com.anarsoft.vmlens.concurrent.example.TestConcurrentHashMapCompute$2.apply - java.util.concurrent.ConcurrentHashMap.compute &lt;&lt;Monitor@java.util.concurrent.ConcurrentHashMap.putVal()&gt;&gt; - com.anarsoft.vmlens.concurrent.example.TestConcurrentHashMapCompute.update21 child2Parent: thread: Thread-1 stack: - java.util.concurrent.ConcurrentHashMap.putVal &lt;&lt;Monitor@java.util.concurrent.ConcurrentHashMap.putVal()&gt;&gt; - java.util.concurrent.ConcurrentHashMap.put - com.anarsoft.vmlens.concurrent.example.TestConcurrentHashMapCompute$1.apply - com.anarsoft.vmlens.concurrent.example.TestConcurrentHashMapCompute$1.apply - java.util.concurrent.ConcurrentHashMap.compute &lt;&lt;Monitor@java.util.concurrent.ConcurrentHashMap.putVal()&gt;&gt; - com.anarsoft.vmlens.concurrent.example.TestConcurrentHashMapCompute.update12 </pre> <p>Here is the putVal method. The synchronized statement leading to the deadlock is in line 18:</p> <pre class="brush: java"> final V putVal(K key, V value, boolean onlyIfAbsent) { if (key == null || value == null) throw new NullPointerException(); int hash = spread(key.hashCode()); int binCount = 0; for (Node&lt;K,V&gt;[] tab = table;;) { Node&lt;K,V&gt; f; int n, i, fh; if (tab == null || (n = tab.length) == 0) tab = initTable(); else if ((f = tabAt(tab, i = (n - 1) &amp; hash)) == null) { if (casTabAt(tab, i, null, new Node&lt;K,V&gt;(hash, key, value, null))) break; // no lock when adding to empty bin } else if ((fh = f.hash) == MOVED) tab = helpTransfer(tab, f); else { V oldVal = null; synchronized (f) { // ... omitted } if (binCount != 0) { if (binCount &gt;= TREEIFY_THRESHOLD) treeifyBin(tab, i); if (oldVal != null) return oldVal; break; } } } addCount(1L, binCount); return null; } </pre> <p>and the compute method. The synchronized statement leading to the deadlock is in line 19:</p> <pre class="brush: java"> public V compute(K key, BiFunction&lt;? super K, ? super V, ? extends V&gt; remappingFunction) { if (key == null || remappingFunction == null) throw new NullPointerException(); int h = spread(key.hashCode()); V val = null; int delta = 0; int binCount = 0; for (Node&lt;K,V&gt;[] tab = table;;) { Node&lt;K,V&gt; f; int n, i, fh; if (tab == null || (n = tab.length) == 0) tab = initTable(); else if ((f = tabAt(tab, i = (n - 1) &amp; h)) == null) { // ... omitted contains call to remappingFunction function } else if ((fh = f.hash) == MOVED) tab = helpTransfer(tab, f); else { synchronized (f) { // ... omitted contains call to remappingFunction function } if (binCount != 0) { if (binCount &gt;= TREEIFY_THRESHOLD) treeifyBin(tab, i); break; } } } if (delta != 0) addCount((long)delta, binCount); return val; } </pre> Wed, 18 Oct 2017 22:00:00 GMT http://vmlens.com/articles/detect_deadlocks 2017-10-18T22:00:00Z How to use atomic methods to write thread-safe classes http://vmlens.com/articles/use_atomic_methods The classes in the package java.util.concurrent use atomic methods to update their internal state in a thread-safe way. In the following, you will see how to write and use such atomic methods to create thread-safe classes. <p>The classes in the package java.util.concurrent use atomic methods to update their internal state in a thread-safe way. In the following, you will see how to write and use such atomic methods to create thread-safe classes.</p> <h1><a href="#atomic-methods" id="atomic-methods"></a>Atomic methods</h1> <p>A method is atomic if it is &quot;all or nothing&quot;. If a thread reads the data the thread can only see the state before or after the execution of the atomic method, no intermediate state. After the atomic method was executed successfully the changes are visible to all threads. The atomic method only modifies data of its own object without side effects.<br /> Here are some examples of atomic methods:</p> <div style="padding-left : 40px;"> <h3><a href="#javautilconcurrentatomicatomicinteger" id="javautilconcurrentatomicatomicinteger"></a>java.util.concurrent.atomic.AtomicInteger</h3> <div style="padding-left : 40px;"> <h4><a href="#int-get" id="int-get"></a>int <span class="javadoc_blue">get</span>()</h4> <p>Gets the current value.</p> <h4><a href="#void-setint-newvalue" id="void-setint-newvalue"></a>void <span class="javadoc_blue">set</span>(int newValue)</h4> <p>Sets to the given value.</p> <h4><a href="#int-addandgetint-delta" id="int-addandgetint-delta"></a>int <span class="javadoc_blue">addAndGet</span>(int delta)</h4> <p>Atomically adds the given value to the current value.</p> </div> </div> <div style="padding-left : 40px;"> <h3><a href="#javautilconcurrentconcurrenthashmap" id="javautilconcurrentconcurrenthashmap"></a>java.util.concurrent.ConcurrentHashMap</h3> <div style="padding-left : 40px;"> <h4><a href="#v-computek-key-bifunction-super-k-super-v-extends-v-remappingfunction" id="v-computek-key-bifunction-super-k-super-v-extends-v-remappingfunction"></a>V <span class="javadoc_blue">compute</span>(K key, BiFunction&lt;? super K,? super V,? extends V&gt; remappingFunction)</h4> <p>Attempts to compute a mapping for the specified key and its current mapped value (or null if there is no current mapping). The entire method invocation is performed atomically. Some attempted update operations on this map by other threads may be blocked while computation is in progress, so the computation should be short and simple, and must not attempt to update any other mappings of this Map.</p> </div> </div> <h1><a href="#writing-atomic-methods" id="writing-atomic-methods"></a>Writing atomic methods</h1> <p>The easiest way to implement atomic methods is to use synchronized blocks. For example, the backport for AtomicInteger for JDKs prior to version 1.5 uses synchronized blocks and volatile fields:</p> <pre class="brush: java">package edu.emory.mathcs.backport.java.util.concurrent.atomic; public class AtomicInteger extends Number implements java.io.Serializable { private volatile int value; public final int get() { return value; } public final synchronized void set(int newValue) { value = newValue; } public final synchronized int addAndGet(int delta) { return value += delta; } } </pre> <p>To write lock-free atomic methods you can use atomic operations of the CPU. In JAVA atomic operations can be accessed using sun.misc.Unsafe like in the implementation of AtomicInteger below. Or you can use the classes of the java.util.concurrent.atomic package, which provides atomic operations for each field type.</p> <pre class="brush: java">package java.util.concurrent.atomic; import sun.misc.Unsafe; public class AtomicInteger extends Number implements java.io.Serializable { private static final Unsafe unsafe = Unsafe.getUnsafe(); private static final long valueOffset; static { try { valueOffset = unsafe.objectFieldOffset (AtomicInteger.class.getDeclaredField("value")); } catch (Exception ex) { throw new Error(ex); } } private volatile int value; public final int get() { return value; } public final void set(int newValue) { value = newValue; } public final int addAndGet(int delta) { return unsafe.getAndAddInt(this, valueOffset, delta) + delta; } }</pre> <h1><a href="#using-atomic-methods" id="using-atomic-methods"></a>Using atomic methods</h1> <p>To see how to use atomic methods let us update our AtomicInteger from two threads. We can simulate this using the following two methods. Calling AtomicInteger sequential:</p> <pre class="brush: java"> public void testSetAndGetSequential() throws Exception { AtomicInteger atomicInteger= new AtomicInteger(0); int threadA = atomicInteger.get(); atomicInteger.set(threadA + 5); int threadB = atomicInteger.get(); atomicInteger.set(threadB + 5); assertEquals( atomicInteger.get() , 10 ); } </pre> <p>and calling AtomicInteger in parallel:</p> <pre class="brush: java"> public void testSetAndGetParallel() throws Exception { AtomicInteger atomicInteger= new AtomicInteger(0); int threadA = atomicInteger.get(); int threadB = atomicInteger.get(); atomicInteger.set(threadA + 5); atomicInteger.set(threadB + 5); assertEquals( atomicInteger.get() , 5 ); } </pre> <p>Each method simulates a different thread interleaving. As we can see the use of get and set leads for a specific thread interleaving to an incorrect result. To update the AtomicInteger correctly we need to use a method which combines the set and get operation in an atomic method, the addAndGet method. Now the result is independent of the thread interleaving:</p> <pre class="brush: java"> public void testUpdate() throws Exception { AtomicInteger atomicInteger= new AtomicInteger(0); atomicInteger.addAndGet(5); // Thread A atomicInteger.addAndGet(5); // Thread B assertEquals( atomicInteger.get() , 10 ); } </pre> <p>Care must be taken if we want to use callback functions inside atomic methods like in the compute method of ConcurrentHashMap. Those callback methods should not modify the same object, as in the test below. The following usage leads to a deadlock. To test this we need a multi-threaded unit test. This junit test uses <a href="https://github.com/ThomasKrieger/concurrent-junit">concurrent-junit</a> to run the test in parallel. The ConcurrentTestRunner runs each test method in parallel by 4 threads.</p> <pre class="brush: java"> import java.util.concurrent.ConcurrentHashMap; import java.util.function.BiFunction; import org.junit.Test; import org.junit.runner.RunWith; import com.anarsoft.vmlens.concurrent.junit.ConcurrentTestRunner; @RunWith(ConcurrentTestRunner.class) public class TestConcurrentHashMapCompute { private final ConcurrentHashMap&lt;Integer,Integer&gt; map = new ConcurrentHashMap&lt;Integer,Integer&gt;(); public TestConcurrentHashMapCompute() { map.put(1, 1); map.put(2, 2); } @Test public void update12() { map.compute( 1 , new BiFunction&lt;Integer,Integer,Integer&gt;() { public Integer apply(Integer k, Integer v) { map.put( 2 , 1); return v; } } ); } @Test public void update21() { map.compute( 2 , new BiFunction&lt;Integer,Integer,Integer&gt;() { public Integer apply(Integer k, Integer v) { map.put( 1 , 1); return v; } } ); } } </pre> <p>If we run the test we see the following deadlocks. The trace was generated by <a href="http://vmlens.com">vmlens.com</a>:</p> <pre class="brush: java"> - deadlock: Monitor@java.util.concurrent.ConcurrentHashMap.putVal()&lt;-&gt;Monitor@java.util.concurrent.ConcurrentHashMap.putVal() parent2Child: thread: Thread-4 stack: - java.util.concurrent.ConcurrentHashMap.putVal &lt;&lt;Monitor@java.util.concurrent.ConcurrentHashMap.putVal()&gt;&gt; - java.util.concurrent.ConcurrentHashMap.put - com.anarsoft.vmlens.concurrent.example.TestConcurrentHashMapCompute$2.apply - com.anarsoft.vmlens.concurrent.example.TestConcurrentHashMapCompute$2.apply - java.util.concurrent.ConcurrentHashMap.compute &lt;&lt;Monitor@java.util.concurrent.ConcurrentHashMap.putVal()&gt;&gt; - com.anarsoft.vmlens.concurrent.example.TestConcurrentHashMapCompute.update21 child2Parent: thread: Thread-1 stack: - java.util.concurrent.ConcurrentHashMap.putVal &lt;&lt;Monitor@java.util.concurrent.ConcurrentHashMap.putVal()&gt;&gt; - java.util.concurrent.ConcurrentHashMap.put - com.anarsoft.vmlens.concurrent.example.TestConcurrentHashMapCompute$1.apply - com.anarsoft.vmlens.concurrent.example.TestConcurrentHashMapCompute$1.apply - java.util.concurrent.ConcurrentHashMap.compute &lt;&lt;Monitor@java.util.concurrent.ConcurrentHashMap.putVal()&gt;&gt; - com.anarsoft.vmlens.concurrent.example.TestConcurrentHashMapCompute.update12 </pre> <h1><a href="#conclusion" id="conclusion"></a>Conclusion</h1> <p>Atomic methods let you use classes in a thread-safe way without knowing the implementation details. To test a specific thread interleaving we can simply order the calls accordingly. Callback functions inside atomic methods should not modify the same object since this might lead to deadlocks.</p> Wed, 20 Sep 2017 22:00:00 GMT http://vmlens.com/articles/use_atomic_methods 2017-09-20T22:00:00Z Opinion: Use atomic methods for thread safety http://vmlens.com/articles/atomic_methods When we update rows in our database we use transactions. By using transactions we can safely modify our data even when multiple clients are accessing the database concurrently. In java, we have many techniques for modifying data concurrently, but we are missing a high-level abstraction like transactions in a database to concurrently modify data in a safe way. <p>When we update rows in our database we use transactions. By using transactions we can safely modify our data even when multiple clients are accessing the database concurrently. In java, we have many techniques for modifying data concurrently, volatile fields, synchronized blocks, classes from java.util.concurrent.</p> <p>But we are missing a high-level abstraction like transactions in a database to concurrently modify data in a safe way. I think atomic methods are a good fit for such a high-level abstraction.</p> <h1><a href="#atomic-methods" id="atomic-methods"></a>Atomic methods</h1> <p>A method is atomic if it is &quot;all or nothing&quot;. If another thread reads the data the other thread can only see the state before or after the execution of the atomic block, no intermediate state. After the atomic method was successful the changes are visible to all other threads. The atomic method only modifies data of its own object without side effects.<br /> Here is an example of a class with atomic methods implemented by the synchronized statement:</p> <pre class="brush: java"> public class AtomicPositiveValue { private int value; public AtomicPositiveValue(int newValue) throws Exception { if( newValue < 0 ) { throw new Exception("value is negative"); } this.value = newValue; } public synchronized int get() { return value; } public synchronized void set(int newValue) throws Exception { if( newValue < 0 ) { throw new Exception("value is negative"); } value = newValue; } } </pre> <h1><a href="#using-atomic-methods" id="using-atomic-methods"></a>Using atomic methods</h1> <p>Let us modify an instance of our class from multiple threads. The main advantage of atomic methods is that we can simulate this by a single threaded method as the following.</p> <pre class="brush: java"> public void testSetAndGetParallel() throws Exception { AtomicPositiveValue atomicPositiveValue= new AtomicPositiveValue(0); int threadA = atomicPositiveValue.get(); int threadB = atomicPositiveValue.get(); atomicPositiveValue.set(threadA + 5); atomicPositiveValue.set(threadB + 5); assertEquals( atomicPositiveValue.get() , 5 ); } public void testSetAndGetSequentiell() throws Exception { AtomicPositiveValue atomicPositiveValue= new AtomicPositiveValue(0); int threadA = atomicPositiveValue.get(); atomicPositiveValue.set(threadA + 5); int threadB = atomicPositiveValue.get(); atomicPositiveValue.set(threadB + 5); assertEquals( atomicPositiveValue.get() , 10 ); } </pre> <p>This is not the result we wanted. The chaining of the get and set method leads to a non atomic update and the result depends on the order of the set and get calls from the different threads. We need an atomic update method:</p> <pre class="brush: java"> public synchronized int update(int delta) throws Exception { int temp = value + delta; if( temp &lt; 0 ) { throw new Exception(&quot;value is negative&quot;); } value = temp; return value; } </pre> <p>Now we always achieve the same result:</p> <pre class="brush: java"> public void testUpdate() throws Exception { AtomicPositiveValue atomicPositiveValue= new AtomicPositiveValue(0); atomicPositiveValue.update(5); // Thread A atomicPositiveValue.update(5); // Thread B assertEquals( atomicPositiveValue.get() , 10 ); } </pre> <p>As we have seen chaining atomic methods from the same object typically leads to nonatomic methods. Therefore we need to provide atomic methods for all use cases.</p> <h1><a href="#composing-atomic-methods" id="composing-atomic-methods"></a>Composing atomic methods</h1> <p>Now let us see what happens when we compose atomic methods. For example let us create a method which transfers an amount from one instance to another.</p> <pre class="brush: java"> public synchronized void transfer(AtomicPositiveValue other, int amount) throws Exception { other.update( -1 * amount ); update(amount); } </pre> <p>To test this method we need a multi-threaded unit test. The ConcurrentTestRunner runs each test method parallel by 4 threads.</p> <pre class="brush: java"> @RunWith(ConcurrentTestRunner.class) public class TestDeadlockAtomicValue { private final AtomicPositiveValue first; private final AtomicPositiveValue second; public TestDeadlockAtomicValue() throws Exception { first = new AtomicPositiveValue(1000); second = new AtomicPositiveValue(1000); } @Test public void testTransferFirstToSecond() throws Exception { second.transfer( first , 1); } @Test public void testTransferSecondToFirst() throws Exception { first.transfer( second , 1); } } </pre> <p>If we run the test we see deadlocks:</p> <pre class="brush: java"> - deadlock: Monitor@com.anarsoft.vmlens.concurrent.example.AtomicPositiveValue.update()&lt;-&gt;Monitor@com.anarsoft.vmlens.concurrent.example.AtomicPositiveValue.update() parent2Child: thread: Thread-4 stack: - com.anarsoft.vmlens.concurrent.example.AtomicPositiveValue.update &lt;&lt;Monitor@com.anarsoft.vmlens.concurrent.example.AtomicPositiveValue.update()&gt;&gt; - com.anarsoft.vmlens.concurrent.example.AtomicPositiveValue.transfer &lt;&lt;Monitor@com.anarsoft.vmlens.concurrent.example.AtomicPositiveValue.update()&gt;&gt; - com.anarsoft.vmlens.concurrent.example.TestDeadlockAtomicValue.testTransferSecondToFirst - sun.reflect.NativeMethodAccessorImpl.invoke - sun.reflect.DelegatingMethodAccessorImpl.invoke - java.lang.reflect.Method.invoke - org.junit.runners.model.FrameworkMethod$1.runReflectiveCall - org.junit.internal.runners.model.ReflectiveCallable.run - org.junit.runners.model.FrameworkMethod.invokeExplosively - org.junit.internal.runners.statements.InvokeMethod.evaluate - com.anarsoft.vmlens.concurrent.junit.internal.ConcurrentStatement.evaluateStatement - com.anarsoft.vmlens.concurrent.junit.internal.ConcurrentStatement.evaluate - com.anarsoft.vmlens.concurrent.junit.internal.ParallelExecutorThread.run child2Parent: thread: Thread-1 stack: - com.anarsoft.vmlens.concurrent.example.AtomicPositiveValue.update &lt;&lt;Monitor@com.anarsoft.vmlens.concurrent.example.AtomicPositiveValue.update()&gt;&gt; - com.anarsoft.vmlens.concurrent.example.AtomicPositiveValue.transfer &lt;&lt;Monitor@com.anarsoft.vmlens.concurrent.example.AtomicPositiveValue.update()&gt;&gt; - com.anarsoft.vmlens.concurrent.example.TestDeadlockAtomicValue.testTransferFirstToSecond - sun.reflect.NativeMethodAccessorImpl.invoke - sun.reflect.DelegatingMethodAccessorImpl.invoke - java.lang.reflect.Method.invoke - org.junit.runners.model.FrameworkMethod$1.runReflectiveCall - org.junit.internal.runners.model.ReflectiveCallable.run - org.junit.runners.model.FrameworkMethod.invokeExplosively - org.junit.internal.runners.statements.InvokeMethod.evaluate - com.anarsoft.vmlens.concurrent.junit.internal.ConcurrentStatement.evaluateStatement - com.anarsoft.vmlens.concurrent.junit.internal.ConcurrentStatement.evaluate - com.anarsoft.vmlens.concurrent.junit.internal.ParallelExecutorThread.run </pre> The trace was generated by vmlens. <p>Chaining of atomic methods leads to deadlock. At least when they are implemented by synchronized statements. And since how the atomic method is implemented should by hidden, we need to avoid chaining of atomic methods.</p> <h1><a href="#implementing-with-compareandset" id="implementing-with-compareandset"></a>Implementing with compareAndSet</h1> <p>To see that we can easily change the implementation of our AtomicPositiveValue, let us see how it can be implemented with compareAndSet. Suppose in our performance test we see a bottleneck at the get method. And we decide to use the following faster implementation using AtomicInteger with compareAndSet:</p> <pre class="brush: java"> public class AtomicPositiveValueUsingAtomicInteger { private final AtomicInteger value; public AtomicPositiveValueUsingAtomicInteger(int newValue) throws Exception { if (newValue &lt; 0) { throw new Exception(&quot;value is negative&quot;); } value = new AtomicInteger(newValue); } public int get() { return value.get(); } public int update(int delta) throws Exception { int current = value.get(); int update = current + delta; if (update &lt; 0) { throw new Exception(&quot;value is negative&quot;); } while (!value.compareAndSet(current, update)) { update = current + delta; if (update &lt; 0) { throw new Exception(&quot;value negative&quot;); } } return update; } } </pre> <p>The behaviour of our class is the same as the synchronized implementation. It is only faster.</p> <h1><a href="#conclusion" id="conclusion"></a>Conclusion</h1> <p>Atomic methods let us use classes in a thread safe way without knowing the implementation details, similar to database transactions. If we want to test if our usage is correct we can simply chain the method calls. To test a specific thread interleaving we can simply order the calls accordingly. In contrast to database transactions, which have automatic deadlock detection, we can not chain atomic methods.</p> Tue, 12 Sep 2017 22:00:00 GMT http://vmlens.com/articles/atomic_methods 2017-09-12T22:00:00Z java.math.BigDecimal toString is not thread safe http://vmlens.com/articles/java-math-BigDecimal-toString-is-not-threadsafe java.math.BigDecimal toString is not thread safe. Calling it from multiple threads leads to strange results. <p>BigDecimal is an immutable data type. So every method of this class should be thread safe. But this not the case for the method toString. Calling it from multiple threads leads to strange results.</p> <p>To see this, let us look at the source code:</p> <pre class="brush: java"> @Override public String toString() { String sc = stringCache; if (sc == null) stringCache = sc = layoutChars(true); return sc; } /** * Used to store the canonical string representation, if computed. */ private transient String stringCache; </pre> <p>As we see a non-volatile field stringCache is used to cache the String computed in the method layoutChars. The method layoutChars uses a thread local StringBuffer to compute a String representation of this BigDecimal. In line 3 the instance variable stringCache is read and line 5 written This makes the class BigDecimal mutable and the method toString not thread safe.</p> <h1><a href="#the-race-condition" id="the-race-condition"></a>The race condition</h1> <p>If the code is executed in the given order everything is o.k. But if some component reorders the statements, the cached String is not completely initialized. In pseudo code the method toString together with layoutChars looks like this:</p> <pre class="brush: java"> store stringCache in local Variable sc if sc is null { call layoutChars { compute String with thread local StringBuilder call StringBuilder toString { create String initialize String in Constructor of class String } } store result in instance Variable stringCache } </pre> <p>If the statements get reordered a thread sees an uninitialized String:</p> <table class="table"> <tbody> <tr> <td>Thread A</td> <td>store stringCache in local Variable sc</td> <td></td> </tr> <tr> <td>Thread A</td> <td>if sc is null</td> <td></td> </tr> <tr> <td>Thread A</td> <td>create String</td> <td></td> </tr> <tr> <td>Thread A</td> <td>store result in instance Variable stringCache</td> <td></td> </tr> <tr style=" background-color: rgb(204, 204, 204);"> <td>Thread B</td> <td>store stringCache in local Variable sc</td> <td></td> </tr> <tr style=" background-color: rgb(204, 204, 204);"> <td>Thread B</td> <td>Thread B sees an uninitialized String</td> <td></td> </tr> <tr> <td>Thread A</td> <td>compute String with thread local StringBuilder</td> <td></td> </tr> <tr> <td>Thread A</td> <td>initialize String in Constructor of class String</td> <td></td> </tr> <tr> </tbody> </table> <p>One component which reorders statements is the cache system of the CPU. <a href="https://en.wikipedia.org/wiki/ARM_architecture">ARM</a> compatible processors like in smartphones or the Raspberry Pi reorder reads and writes to improve performance, leading to a scenario as described above.</p> <h1>Reproducing the error</h1> To reproduce the error I use <a href="https://wiki.openjdk.java.net/display/CodeTools/jcstress">jcstress</a>, an open JDK code tool: <q cite="https://wiki.openjdk.java.net/display/CodeTools/jcstress">The Java Concurrency Stress tests (jcstress) is an experimental harness and a suite of tests to aid the research in the correctness of concurrency support in the JVM, class libraries, and hardware.</q> <p>I use the following <a href="https://github.com/vmlens/jcstress-examples/blob/master/src/main/java/com/vmlens/stressTest/tests/BigDecimalToString.java">test class:</a></p> <pre class="brush: java"> package com.vmlens.stressTest.tests; import java.math.BigDecimal; import org.openjdk.jcstress.annotations.*; import org.openjdk.jcstress.infra.results.IntResult1; @JCStressTest @Outcome(id = "0", expect = Expect.ACCEPTABLE, desc = "Default outcome.") @State public class BigDecimalToString { private final BigDecimal testBigDecimal = new BigDecimal("0.56"); @Actor public void actor1(IntResult1 r) { testBigDecimal.toString().length(); } @Actor public void actor2(IntResult1 r) { testBigDecimal.toString().length(); } } </pre> <p>Jcstress runs this test multiple times always calling the method actor1 and actor2 from separate threads. When I call this test on a raspberry pi, I see the following null pointer exception:</p> <pre class="brush: java"> java.lang.NullPointerException at java.lang.String.length(String.java:623) at com.vmlens.stressTest.tests.BigDecimalToString.actor1(BigDecimalToString.java:12) at com.vmlens.stressTest.tests.BigDecimalToString_jcstress.actor1(BigDecimalToString_jcstress.java:145) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) </pre> <p>If we look at line 623 of class String we see that the String was indeed not completely initialized. The instance variable value storing the character array is null:</p> <pre> return value.length; </pre> <h1><a href="#where-did-i-find-the-race-condition" id="where-did-i-find-the-race-condition"></a>Where did I find the race condition?</h1> <p>I found the race condition while testing geronimo web service with <a href="https://vmlens.com">vmlens</a>. vmlens detects race conditions during test runs. vmlens created the following trace during my tests:</p> <pre class="brush: java"> - variable: java.math.BigDecimal.stringCache reading: thread: DefaultThreadPool 197 stack: - java.math.BigDecimal.toString - org.apache.axis.encoding.ser.SimpleSerializer.getValueAsString - org.apache.axis.encoding.ser.SimpleSerializer.serialize - org.apache.axis.encoding.SerializationContext.serializeActual - org.apache.axis.encoding.SerializationContext.serialize - org.apache.axis.encoding.SerializationContext.serialize - org.apache.axis.encoding.ser.BeanSerializer.serialize - org.apache.axis.encoding.SerializationContext.serializeActual - org.apache.axis.encoding.SerializationContext.serialize - org.apache.axis.encoding.SerializationContext.serialize - org.apache.axis.message.RPCParam.serialize - org.apache.axis.message.RPCElement.outputImpl - org.apache.axis.message.MessageElement.output - org.apache.axis.message.SOAPBody.outputImpl - org.apache.axis.message.SOAPEnvelope.outputImpl - org.apache.axis.message.MessageElement.output - org.apache.axis.SOAPPart.writeTo ---- Stack Trace shortened ---- writing: thread: DefaultThreadPool 196 stack: - java.math.BigDecimal.toString - org.apache.axis.encoding.ser.SimpleSerializer.getValueAsString - org.apache.axis.encoding.ser.SimpleSerializer.serialize ---- Stack Trace shortened ---- </pre> <p>This shows that toString is used for serializing BigDecimals to SOAP messages.</p> <h1><a href="#conclusion" id="conclusion"></a>Conclusion</h1> <p>The caching of the computed String in an instance variable in the method toString makes the class BigDecimal mutable and the method toString not thread safe.This leads to NullPointer exceptions on ARM compatible processors. One usage of the toString method is the serialization of BigDecimals to SOAP messages.</p> Mon, 28 Aug 2017 22:00:00 GMT http://vmlens.com/articles/java-math-BigDecimal-toString-is-not-threadsafe 2017-08-28T22:00:00Z java.lang.reflect.TypeVariable getBounds is not thread safe http://vmlens.com/articles/java-lang-reflect-typevariable-getbounds-is-not-thread-safe java.lang.reflect.TypeVariable getBounds is not thread safe. Calling it from multiple threads might even crash your JVM. <p>java.lang.reflect.TypeVariable getBounds is not thread safe. Calling it from multiple threads might even crash your JVM</p> <p>The following method shows you the use of getBounds(). The method getBounds is used to get the upper bound(s) of a generic type:</p> <pre class="brush: java">public void testGetBounds() { Class cl = GenericInterface.class; TypeVariable typeVariable = cl.getTypeParameters()[0]; typeVariable.getBounds()[0].getTypeName(); } </pre> <p>To make the examples and tests shorter, I do not iterate over the returned array but simply use the first element. Here is the generic interface used in the example:</p> <pre class="brush: java">package com.vmlens.stressTest.util; public interface GenericInterface&gt; { } </pre> <p>If you call testGetBounds from multiple threads, calling getBounds leads to a race condition.</p> <h1>The race condition</h1> Since the array of TypeVariables returned by getTypeParameters is cached in the volatile field genericInfo, each thread works on the same TypeVariable instance. And the class TypeVariableImpl implementing the TypeVariable interface modifies the not volatile field bounds without synchronization: <pre class="brush: java">package sun.reflect.generics.reflectiveObjects; // import statements omitted public class TypeVariableImpl extends LazyReflectiveObjectGenerator implements TypeVariable { // upper bounds - evaluated lazily private Type[] bounds; public Type[] getBounds() { // lazily initialize bounds if necessary if (bounds == null) { FieldTypeSignature[] fts = getBoundASTs(); // get AST // allocate result array; note that // keeping ts and bounds separate helps with threads Type[] ts = new Type[fts.length]; // iterate over bound trees, reifying each in turn for ( int j = 0; j &lt; fts.length; j++) { Reifier r = getReifier(); fts[j].accept(r); ts[j] = r.getResult(); } // cache result bounds = ts; // could throw away bound ASTs here; thread safety? } return bounds.clone(); // return cached bounds } // other fields and methods omitted } </pre> <p>In line 15 and 30 the field <q>bounds</q> is read and in line 27 it is written. If the code is executed in the given order everything is o.k. But if some component reorders the statements, the array is not completely initialized. In pseudo code the method getBounds looks like this:</p> <pre class="brush: java">if instance variable bounds is null { set local variable ts to new Array initialize the array set instance variable bounds to the local variable ts } return instance variable bounds.clone </pre> <p>If the statements get reordered another thread sees an uninitialised array: <table class="table"> <tbody> <tr></p> <td>Thread A</td> <td>set local variable ts to new Array</td> <td></td> </tr> <tr> <td>Thread A</td> <td>set instance variable bounds to the local variable ts</td> <td></td> </tr> <tr> <td>Thread B</td> <td>if instance variable bounds is null</td> <td></td> </tr> <tr> <td>Thread B</td> <td>Thread B return instance variable bounds.clone // the array is not yet completely initialized</td> <td></td> </tr> </tbody> </table> <p>One such component is the cache system of the CPU. <a href="https://en.wikipedia.org/wiki/ARM_architecture">ARM</a> compatible processors like in smartphones or the Raspberry Pi reorder reads and writes to improve performance, leading to a scenario as described above.</p> <h1>Reproducing the error</h1> To reproduce the error I use <a href="https://wiki.openjdk.java.net/display/CodeTools/jcstress">jcstress</a>, an open JDK code tool: <q cite="https://wiki.openjdk.java.net/display/CodeTools/jcstress">The Java Concurrency Stress tests (jcstress) is an experimental harness and a suite of tests to aid the research in the correctness of concurrency support in the JVM, class libraries, and hardware.</q> <p>I use the following <a href="https://github.com/vmlens/jcstress-examples/blob/master/src/main/java/com/vmlens/stressTest/tests/TypeVariableGetBounds.java">test class:</a></p> <pre class="brush: java">@JCStressTest @Outcome(id = "0, 0", expect = Expect.ACCEPTABLE, desc = "Default outcome.") @State public class TypeVariableGetBounds { private final Class cl; public TypeVariableGetBounds() { try { cl = (new StressTestClassLoader(TypeVariableGetBounds.class.getClassLoader())) .loadClass("com.vmlens.stressTest.util.GenericInterface"); } catch (Exception e) { throw new RuntimeException("Test setup incorrect", e); } } public void callContainsDataRace() { TypeVariable typeVariable = cl.getTypeParameters()[0]; typeVariable.getBounds()[0].getTypeName(); } @Actor public void actor1(IntResult2 r) { callContainsDataRace(); } @Actor public void actor2(IntResult2 r) { callContainsDataRace(); } } </pre> <p>Jcstress runs this test multiple times always calling the method actor1 and actor2 from separate threads. To separate the tests, I use a <a href="https://github.com/vmlens/jcstress-examples/blob/master/src/main/java/com/vmlens/stressTest/util/StressTestClassLoader.java">special classloader</a>, which always reloads a class.</p> <p>When I call this test on a raspberry pi using the test mode tough or stress, I see a crash of the JVM:</p> <pre> # # A fatal error has been detected by the Java Runtime Environment: # # SIGSEGV (0xb) at pc=0x7665b4c0, pid=16112, tid=1680598112 # # JRE version: Java(TM) SE Runtime Environment (8.0_65-b17) (build 1.8.0_65-b17) # Java VM: Java HotSpot(TM) Client VM (25.65-b01 mixed mode linux-arm ) # Problematic frame: # V [libjvm.so+0x27a4c0] </pre> The JVM error log shows the following: <pre>Stack: [0x6426f000,0x642bf000], sp=0x642bd8b8, free space=314k Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code) V [libjvm.so+0x27a4c0] Java frames: (J=compiled Java code, j=interpreted, Vv=VM code) J 1303 java.lang.Object.clone()Ljava/lang/Object; (0 bytes) @ 0x742ba71c [0x742ba6e0+0x3c] J 1313 C1 sun.reflect.generics.repository.GenericDeclRepository.getTypeParameters()[Ljava/lang/reflect/TypeVariable; (80 bytes) @ 0x742b8210 [0x742b7f40+0x2d0] J 1229 C1 com.vmlens.stressTest.tests.TypeVariableGetBounds_jcstress.actor1()Ljava/lang/Void; (109 bytes) @ 0x742a6cb8 [0x742a6bf0+0xc8] j com.vmlens.stressTest.tests.TypeVariableGetBounds_jcstress$$Lambda$6.call()Ljava/lang/Object;+4 j java.util.concurrent.FutureTask.run()V+42 j java.util.concurrent.ThreadPoolExecutor.runWorker(Ljava/util/concurrent/ThreadPoolExecutor$Worker;)V+95 j java.util.concurrent.ThreadPoolExecutor$Worker.run()V+5 j java.lang.Thread.run()V+11 v ~StubRoutines::call_stub </pre> <p>It seems that the native array clone method can not cope with an uninitialized array.</p> <p>So far I could not reproduce this error on my intel i5 workstation.</p> <h1>Conclusion</h1> java.lang.reflect.TypeVariable getBounds is not thread safe. Calling it from multiple threads might lead, depending on the java platform you are using, to strange errors. Sun, 19 Mar 2017 23:00:00 GMT http://vmlens.com/articles/java-lang-reflect-typevariable-getbounds-is-not-thread-safe 2017-03-19T23:00:00Z How to crash the java virtual machine with a race condition http://vmlens.com/articles/crash_jvm_with_race_condition This is a how-to guide for crashing the java virtual machine. It gives you an introduction to race conditions and shows you what errors can happen if your code contains such bugs. <p>This is a how-to guide for crashing the java virtual machine. It gives you an introduction to race conditions and shows you what errors can happen if your code contains such bugs.</p> <h1>Create a race condition</h1> Let us start with the following <a href="https://github.com/vmlens/stress-test/blob/master/src/main/java/com/vmlens/stressTest/examples/dataRace/DataRaceTest.java">method</a>: <pre class="brush: java">public class DataRaceTest implements Runnable { private Type[] instance; @Override public void run() { if (instance == null) { Type[] ts = new Type[1]; ts[0] = Object.class; instance = ts; } instance[0].getTypeName(); } } </pre> <p>If this method is executed by multiple threads it leads to a race condition. More specifically it leads to a data race. Data races are defined in the <a href="https://docs.oracle.com/javase/specs/jls/se7/html/jls-17.html#jls-17.4">java memory model</a> as an access to a shared field without correct synchronization. According to the java memory model data races lead to platform dependent undefined behavior:</p> <blockquote cite="https://docs.oracle.com/javase/specs/jls/se7/html/jls-17.html#jls-17.4.5">Without correct synchronization, very strange, confusing and counterintuitive behaviors are possible.</blockquote> If the code is executed in the given order everything is o.k. But if some component reorders the statements, the array might not be completely initialized. One such component is the cache system of the CPU. Intel Processors, for example, have a cache system with a strong memory guarantees. The cache system makes sure that the values written by the cores are almost always seen in the same order as they have been written by the other cores. But <a href="https://en.wikipedia.org/wiki/ARM_architecture">ARM</a> compatible processors like in smartphones or the Raspberry Pi do not give this guarantee. <h1>Create a Nullpointer Exception</h1> To see what is happening, I execute the method multiple times by multiple threads To do this I use a tool to reproduce race conditions, called <a href="https://github.com/vmlens/stress-test">stress test</a>. I run it with the following command line options: <pre>java -cp stress-test-0.0.1-SNAPSHOT-jar-with-dependencies.jar com.vmlens.StressTest -i 5 -w 16 com.vmlens.stressTest.examples.dataRace.DataRaceTestSetup </pre> <p>using the following <a href="https://github.com/vmlens/stress-test/blob/master/src/main/java/com/vmlens/stressTest/examples/dataRace/DataRaceTestSetup.java">test setup class</a>:</p> <pre class="brush: java">public class DataRaceTestSetup implements TestSetup { @Override public Runnable createTest() { return new DataRaceTest(); } } </pre> <p>This runs the method run of DataRaceTest for 5 iterations. Each iteration consists of 16.000 tests run by 16 threads.</p> <p>If I run this on my intel i5 workstation, I could not create an exception. If I run this on my Raspberry Pi, I see the following Nullpointer Exception for every 2000 tests:</p> <pre>java.lang.NullPointerException at com.vmlens.stressTest.examples.dataRace.DataRaceTest.run(DataRaceTest.java:78) at com.vmlens.stressTest.internal.TestCall.call(TestCall.java:36) at com.vmlens.stressTest.internal.TestCall.call(TestCall.java:7) at com.vmlens.stressTest.internal.WorkerThread.run(WorkerThread.java:22) </pre> <h1>Crash the java virtual machine</h1> Now let us add some native call. Let us clone the <a href="https://github.com/vmlens/stress-test/blob/master/src/main/java/com/vmlens/stressTest/examples/clone/CloneTest.java">array</a>: <pre class="brush: java">@Override public void run() { if (instance == null) { Type[] ts = new Type[1]; ts[0] = Object.class; instance = ts; } Type[] clonedInstance = instance.clone(); clonedInstance[0].getTypeName(); } </pre> <p>This time I run the test till I see at least one error. This is done by using the -e option:</p> <pre class="brush: java">java -cp stress-test-0.0.1-SNAPSHOT-jar-with-dependencies.jar com.vmlens.StressTest -e 1 -w 16 com.vmlens.stressTest.examples.dataRace.DataRaceTestSetup </pre> <p>If I run this code on a Raspberry Pi, I see a java virtual machine crash after some time. It takes between half an hour and a day:</p> <pre># # A fatal error has been detected by the Java Runtime Environment: # # SIGSEGV (0xb) at pc=0x741ab340, pid=26364, tid=1682633824 # # JRE version: Java(TM) SE Runtime Environment (8.0_65-b17) (build 1.8.0_65-b17) # Java VM: Java HotSpot(TM) Client VM (25.65-b01 mixed mode linux-arm ) # Problematic frame: # J 38 C1 com.vmlens.stressTest.internal.TestSetupCall.call()Lcom/vmlens/stressTest/internal/Result; (45 bytes) @ 0x741ab340 [0x741ab250+0xf0] </pre> <h1>Conclusion</h1> Data races are hard to reproduce. You need different types of hardware. And even then it takes time till an error occurs. <p>That is the reason I developed <a href="http://vmlens.com">vmlens</a>, a tool to detect data races in the execution trace of an application.</p> Thu, 02 Mar 2017 23:00:00 GMT http://vmlens.com/articles/crash_jvm_with_race_condition 2017-03-02T23:00:00Z java.util.concurrent.locks.ReentrantLock Cheat Sheet http://vmlens.com/articles/reentrant-lock-cheat-sheet The reentrant lock is a replacement for the easier to use synchronized statement. In the cheat sheet, I have summarized the reentrant lock techniques. And in the blog post, I give a detailed description of those techniques. <p>The reentrant lock is a replacement for the easier to use synchronized statement when you need one of the following advanced techniques: lockInterrupibly, tryLock, lock coupling, multiple conditions, or fair locks In the cheat sheet, I have summarized each technique. And in the blog post, I give a detailed description of those techniques.</p> <p><img class="center-block img-responsive" src="http://vmlens.com/WordPress_01/wp-content/uploads/2016/09/ReentrantLock-Cheat-Sheet-Small.png" alt="reentrantlock cheat sheet" /></p> <p>Similar as synchronized this lock is reentrant, which means the same thread can acquire the lock multiple times.</p> <h1>Lock Interruptible</h1> <pre class="brush: java">private final ReentrantLock lock = new ReentrantLock(); public void m() throws InterruptedException { lock.lockInterruptibly(); try { // ... method body } finally { lock.unlock() } } </pre> <p>Throws an InterruptedException, when the thread.interrupt method was called by another thread. This allows you to interrupt the processing even while trying to acquire a lock. Something which is not possible with the synchronized statement.</p> <h1>Try Lock</h1> <pre class="brush: java">private final ReentrantLock lock = new ReentrantLock(); public boolean m() throws InterruptedException { if(! lock.tryLock(2 , TimeUnit.MILLISECONDS ) ) { return false; } try { // ... method body } finally { lock.unlock() } return true; } </pre> <p>Similar to aqcuireInterrubtable throws a thread interrupted exception when the thread was interrupted by another thread. Additionally, it allows you to give a time span how long to try to acquire the lock. Useful when you have a task which is only valid for a specific amount of time. You can also use it to avoid deadlocks. When you need to acquire two locks simultaneously. Acquire the locks with tryLock. If you can not acquire one of the locks at the given interval release both locks, wait a little bit and try again.</p> <h1>Lock Coupling</h1> The reentrant lock allows you to unlock a lock immediately after you have successfully acquired another lock leading to a technique called lock coupling.This can, for example, be used to lock a linkedlist. You can see an implementation <a href="https://github.com/gramoli/synchrobench/blob/master/java/src/linkedlists/lockbased/LockCouplingListIntSet.java">here</a>. <h1>Multiple Conditions</h1> Create: <pre class="lang:java">Condition notFull = lock.newCondition();</pre> Wait for Condition to become true <pre class="brush: java">lock.lock() { while( condition not fullfilled) { notFull.await(); } } finally { lock.unlock() } </pre> <p>Signal that condition has become true <pre class="brush: java">lock.lock() { notFull.signal(); } finally { lock.unlock() }</p> </pre> Be cautious that you do not mix the methods from Condition, 'await', 'signal', with the methods from java.lang.Object wait, notify. Useful when you need to wait for different conditions. ArrayBlockingQueue, for example, uses two conditions one to wait that the queue becomes not full the other to wait till the queue becomes not empty. <h1>Fair locks</h1> <pre class="lang:java">private final ReentrantLock lock = new ReentrantLock(true);</pre> Threads get the lock in the order they requested it. This has a high-performance penalty, see the following benchmark for details <h1>Benchmark</h1> <img class="center-block img-responsive" src="http://vmlens.com/WordPress_01/wp-content/uploads/2016/09/benchmark.png" alt="synchronized vs. ReentrantLock" /> <p>The figure shows the throughput of synchronized vs reentrant lock in a fair and unfair mode for different thread counts . The benchmark was run on JDK 8 on an intel i5 4 core CPU using jmh. The source of the benchmark can be downloaded <a href="https://github.com/ThomasKrieger/benchmarks">here</a>.</p> Mon, 26 Sep 2016 22:00:00 GMT http://vmlens.com/articles/reentrant-lock-cheat-sheet 2016-09-26T22:00:00Z A new high throughput java executor service http://vmlens.com/articles/a-new-high-throughput-java-executor-service The vmlens executor service is a high throughput executor service. It achieves three times higher throughput than the standard JDK executor service. <p>The <a href="http://executorservice.org/">vmlens executor service</a> is a high throughput executor service. It achieves three times higher throughput than the standard JDK executor service. The tradeoff is that the latency is much higher than that of the standard JDK executor service. Here are my benchmark results:</p> <p><img class="center-block img-responsive" src="http://vmlens.com/WordPress_01/wp-content/uploads/2016/09/throughput.png" alt="throughput of vmlens executor service" /></p> <p>The figure shows the throughput of the vmlens executor Service compared to the standard JDK executor service for different threads on JDK 8.</p> <p><img class="center-block img-responsive" src="http://vmlens.com/WordPress_01/wp-content/uploads/2016/09/latency.png" alt="latency of vmlens executor service" /></p> <p>The figure shows the latency of the vmlens executor Service compared to the standard JDK executor service for different threads on JDK 8. Both benchmarks were run with jmh on an intel i5 4 core CPU. The source of the benchmark can be downloaded <a href="https://github.com/ThomasKrieger/benchmarks">here</a></p> <h2>A wait-free algorithm for writing</h2> The basic Idea is that writing should be as fast as possible. The vmlens executor Service uses a thread local field to store its last written queue node. So writing consists of creating a new QueueNode, writing to a thread local field and writing to the volatile QueueNode next field. <pre class="brush: java">public class QueueManyWriters private final ThreadLocal lastWrittenQueueNode = new ThreadLocal(); public void accept(E element) { if( dispatcherThread.stop ) { throw new RejectedExecutionException(); } QueueNode current = new QueueNode(element); if( lastWrittenQueueNode.get() == null ) { writingThreads.append(current,Thread.currentThread().getId()); lastWrittenQueueNode.set(new LastWrittenQueueNode(current)); } else { lastWrittenQueueNode.get().last.next = current; lastWrittenQueueNode.get().last = current; } } ... } public class QueueNode { final E element; volatile QueueNode next; public QueueNode(E element) { super(); this.element = element; } } </pre> Reading is done by a single dispatcher Thread. The dispatcher collects the tasks and pushes them to one of the worker thread. For collecting the tasks the dispatcher thread uses a local list. The list contains the last read element for each writing thread. <h2>Conclusion</h2> The wait-free algorithm for writing leads to a three time higher throughput than the standard JDK executor service. The vmlens executor service is used in vmlens, a tool to test multithreaded application, for asynchronously writing events to disk. Whatever type of execution service you use you should test the multithreaded part of your application. <a href="http://vmlens.com/">Read more about testing multithreaded java code here</a>. Sun, 11 Sep 2016 22:00:00 GMT http://vmlens.com/articles/a-new-high-throughput-java-executor-service 2016-09-11T22:00:00Z Performance Improvements of Contended Java Monitor from JDK 6 to 9 http://vmlens.com/articles/performance-improvements-of-java-monitor The new JDK™ 9 early access release contains a JDK enhancement proposal JEP 143, Improve Contended Locking, to improve the performance of contended monitors. So let us look at the performance improvements of contended monitors. <p>The new JDK™ 9 early access release contains a JDK enhancement proposal <a href="http://openjdk.java.net/jeps/143">JEP 143, Improve Contended Locking</a>, to improve the performance of contended monitors. Monitors are used by the java synchronized statement to lock the access to a code block. If the synchronized block is called by many threads, the monitor becomes contended. This can degrade the performance dramatically. So let us look at the performance improvements of contended monitors.</p> <p><img class="center-block img-responsive" src="http://vmlens.com/WordPress_01/wp-content/uploads/2016/08/per_jdk1.png" alt="Contended Monitor Performance for Different JDKs" /></p> <p>The graphic shows the time of one method call. Lower means better performance. The test consists of 8 threads accessing a synchronized block. All threads are accessing the same monitor. You can download the test <a href="https://github.com/ThomasKrieger/benchmarks">here</a>. The test was run on an intel i5 4 core CPU As we see JDK 9 improves the performance of contended monitors. Let us now look at a direct comparison between JDK 8 and 9.</p> <h1>Comparison between JDK 8 and JDK 9</h1> The following shows how much the switch from JDK 8 to JDK 9 will bring. <p><img class="center-block img-responsive" src="http://vmlens.com/WordPress_01/wp-content/uploads/2016/08/jdk8_vs_jdk9.png" alt="jdk8_vs_jdk9" /></p> <p>The graphic shows the time of one method call for a contended monitor at different threads. As we see the performance of JDK 9 degrades mutch slower than the performance from JDK 8. For 16 Threads JDK 8 needs 2580 ns while JDK 9 only needs 1655 ns. This is an improvement by more than 60 percent. One advice to improve the performance is to use reentrant locks instead of synchronized blocks. So let us see if this advice is still true for JDK 9.</p> <h1>Reentrant Locks vs synchronized</h1> Let us look at the performance of a contended reentrant lock vs a contended monitor of a synchronized block. <p><img class="center-block img-responsive" src="http://vmlens.com/WordPress_01/wp-content/uploads/2016/08/monitor_vs_lock_8.png" alt="monitor_vs_lock_8" /></p> <p><img class="center-block img-responsive" src="http://vmlens.com/WordPress_01/wp-content/uploads/2016/08/monitor_vs_lock_9.png" alt="monitor_vs_lock_9" width="640" height="480" /></p> <p>The graphics show the time of one method call at different threads. As we see in JDK 9 the performance of the synchronized statements gets almost as fast as reentrant locks.</p> <h1>Conclusion</h1> As we have seen JDK 9 improves the performance of contended monitors. In JDK 9 contended monitors are almost as fast as contended reentrant locks. But JDK 9 also has a JDK enhancement proposal <a href="http://openjdk.java.net/jeps/285">JEP 285, Spin-Wait Hints</a>, to improve the performance of locks. I will look at this in the next blog article at this blog. If you have a question or remark please add a comment below. Sun, 14 Aug 2016 22:00:00 GMT http://vmlens.com/articles/performance-improvements-of-java-monitor 2016-08-14T22:00:00Z Detecting Java Race Conditions With Tests, Part 2 http://vmlens.com/articles/detecting-java-race-conditions-with-tests-part-2 If you update a field from different threads, you must make sure that between the read and the write from one thread the field is not updated by another thread. <p>If you update a field from different threads, you must make sure that between the read and the write from one thread the field is not updated by another thread. You can achieve this by using a synchronized block or the atomic compareAndSet method. But how can you test it?</p> <h1>Example of a Test</h1> Let us look at an example of an incorrect synchronized class: <pre class="brush: java">class Account { private int amount = 0; public synchronized int getAmount() { return amount; } public synchronized void setAmount(int amount) { this.amount = amount; } } </pre> To test this class we use the following junit test case: <pre class="brush: java">@RunWith(ConcurrentTestRunner.class) public class TestAccount { private final Account account = new Account(); private final static int THREAD_COUNT = 2; @Test @ThreadCount(THREAD_COUNT) public void testAdd() { account.setAmount( account.getAmount() + 20 ); } @After public void chechBalance() { assertEquals( "" , THREAD_COUNT * 20 , account.getAmount() ); } } </pre> This junit test uses <a href="https://github.com/ThomasKrieger/concurrent-junit">concurrent-junit</a> to run the test in parallel. The test sometimes succeed, sometimes fails. Every time the get method is called immediately one after the other the sum is incorrect. In the <a href="http://vmlens.com">vmlens</a> Method Explorer you can see the order of the two methods. In case of an error you will see the following: <p><img class="center-block img-responsive" src="http://vmlens.com/WordPress_01/wp-content/uploads/2016/06/artikel_11.png" alt="Java Race Condition Non Atomic Update" /></p> <h1>The Solution: Waitpoints</h1> For a test this is rather inconvenient. If a test fails, which is bad enough, it should at least always fail. To achieve this, a thread needs to wait for the other threads before calling the set method. In <a href="http://vmlens.com">vmlens</a> this can be done by using a “waitpoint”. You can set waitepoints at the beginning of a monitor block or a volatile field access: If we set the waitepoint at the synchronized block of the set method the test always fails: <p><img class="center-block img-responsive" src="http://vmlens.com/WordPress_01/wp-content/uploads/2016/06/artikel_22.png" alt="vmlens Waitpoint" /></p> <h1>Conclusion</h1> In the <a href="http://vmlens.com/articles/detecting-java-race-conditions-with-tests/">first part</a> you have seen how to detect lost updates with tests. This second part shows how to detect non atomic updates. In the next part we will see, how to use this technique to test for thread safety. If you have any questions or remarks please comment below. Tue, 26 Jul 2016 22:00:00 GMT http://vmlens.com/articles/detecting-java-race-conditions-with-tests-part-2 2016-07-26T22:00:00Z 5 Tips to make your classes thread safe http://vmlens.com/articles/5-tips-to-make-your-classes-thread-safe While testing vmlens, a tool to find data races in java applications, on open source projects, I found the following 5 tricks how to make classes thread safe. <p>While testing <a href="http://vmlens.com/">vmlens</a>, a tool to find data races in java applications, on open source projects, I found the following 5 tricks how to make classes thread safe.</p> <h2>1) Declare immutable member variables as final</h2> Always declare immutable member variables as final. This makes sure that your class behaves correctly independent on how it is used. Take for example the field fieldAccessor in the class java.lang.reflect.Field. <pre class="brush: java"> private FieldAccessor fieldAccessor; private FieldAccessor getFieldAccessor(Object obj) throws IllegalAccessException { boolean ov = override; FieldAccessor a = (ov) ? overrideFieldAccessor : fieldAccessor; return (a != null) ? a : acquireFieldAccessor(ov); } </pre> <p>Since it is not synchronized and not declared volatile, a thread reading this field might not see a completely initialized object as described in <a href="http://www.cs.umd.edu/~pugh/java/memoryModel/DoubleCheckedLocking.html">DoubleCheckedLocking</a> But since the created object type sun.reflect.UnsafeQualifiedIntegerFieldAccessorImpl only uses final problem, there is no problem. Threads reading this field will always see a fully initialized object or null.</p> <h2>2) Create objects eagerly</h2> Using final fields forces you to initialize your objects in the constructor. Lazy initialization of your objects on the other side is almost never a good idea in concurrent programs. <p>Take for example the old version from org.apache.commons.lang.StringEscapeUtils. It uses the lazy initialized class org.apache.commons.lang.Entities$LookupEntityMap:</p> <pre class="brush: java"> private String[] lookupTable() { if (lookupTable == null) { createLookupTable(); } return lookupTable; } </pre> <p>This only works with locks or synchronization. Much better is the new version org.apache.commons.lang3.StringEscapeUtils witch eagerly creates the lookup tables and also uses a final field.</p> <pre class="lang:java"> public static final CharSequenceTranslator ESCAPE_XML10 = new AggregateTranslator( ... </pre> <h2>3) Use volatile for mutable boolean variables</h2> Mutable boolean fields are often used for controlling the flow of your application. For example to control the life cycle of a thread the following pattern can be used: <pre class="brush: java">private volatile boolean isWorking; while(isWorking) { // do something } </pre> Use a volatile field to make the changes done in one thread visible in other threads. <h2>4) Check 3rd party classes</h2> A typical example for not doing so, is the use of the non thread safe java.util.date as member variable without synchronization. Therefore always check if the class is documented as thread safe. If not chances are high that it is not. <h2>5) Test</h2> Like all other features of your application, concurrency must be tested. In my next Blog post I will write how to test concurrency. In the meanwhile you can give <a href="http://vmlens.com/">vmlens</a> a trial, which helps you to detect data races during testing. Tue, 26 Jul 2016 22:00:00 GMT http://vmlens.com/articles/5-tips-to-make-your-classes-thread-safe 2016-07-26T22:00:00Z 3 Synchronization idioms http://vmlens.com/articles/3-synchronization-idioms-2 While testing vmlens on open source projects, I found the following 3 synchronization idioms. Each synchronization idiom is useful for a specific access pattern: <h1>Signal</h1> <h2>Intent</h2> Change the behavior of a thread based on an event in another thread. <h2>Motivation</h2> In GUI Frameworks like swing the event thread handles all user events. Some events do not depend on the current state of other threads. An example is the cancel event for a long running operation. To notify worker threads an event handler uses the signal idiom. <h2>Structure</h2> <img class="center-block img-responsive" src="http://vmlens.com/WordPress_01/wp-content/uploads/2015/10/volatile.png" alt="volatile" /> <p>The Signal idiom consists of a volatile field and at least on thread reading and one thread writing this field. The field must never be read and written in the same thread.</p> <h2>Sample Code</h2> <pre class="brush: java">public class WorkerThread extends Thread { public volatile boolean canceled; public void run() { while( ! canceled ) { // do some work } } } </pre> <h1>Snapshot</h1> <h2>Intent</h2> See a consistent state of an object. <h2>Motivation</h2> A thread needs to make a long running computation with data which is potentially changed by other threads in the middle of the computation. An example is the deployment descriptor of a web application in a web server. Through automatic reloads the deployment descriptor might change in the middle of the processing of a web request. To see a consistent state the web request processing thread uses a snapshot of the deployment descriptor. <h2>Structure</h2> <img class="center-block img-responsive" src="http://vmlens.com/WordPress_01/wp-content/uploads/2015/10/snapshot.png" alt="snapshot" /> <p>The Snapshot idiom consists of a class holding the current snapshot. A thread reading values gets the current snapshot and reads values from this snapshot. A thread writing values clones the current snapshot, changes this copy and set the clone as new current snapshot.</p> <h2>Sample Code</h2> <pre class="brush: java">public class CopyOnWriteArrayList { final transient ReentrantLock lock = new ReentrantLock(); // usa a volatile field for the snapshot reference private transient volatile Object[] array; final Object[] getArray() { return array; } final void setArray(Object[] a) { array = a; } public boolean add(E e) { final ReentrantLock lock = this.lock; lock.lock(); try { Object[] elements = getArray(); int len = elements.length; // clone the object Object[] newElements = Arrays.copyOf(elements, len + 1); // change the local copy newElements[len] = e; // set the copy as new snapshot setArray(newElements); return true; } finally { lock.unlock(); } } public void forEach(Consumer action) { if (action == null) throw new NullPointerException(); // work with the current snapshot Object[] elements = getArray(); int len = elements.length; for (int i = 0; i &lt; len; ++i) { @SuppressWarnings("unchecked") // all read operations are done on this snapshot E e = (E) elements[i]; action.accept(e); } } } </pre> Taken from java.util.concurrent.CopyOnWriteArrayList. Comments are mine. <h1>Put if absent</h1> <h2>Intent</h2> Get an object out of a map in a multithreaded environment. Create it if it does not exist. <h2>Motivation</h2> You have a map of objects which is accessed from many threads. Each thread behaves the same, checking if a value exists for a key, if not creating the value. You have many concurrent reads. A read should not be blocked by a write to a different key. For example in a web application the language specific formats are stored in a map, using the language as key. Each worker thread checks if the language specific formats are available in the map, if not the thread creates a new one. <h2>Structure</h2> <img class="center-block img-responsive" src="http://vmlens.com/WordPress_01/wp-content/uploads/2015/10/putIfAbsent.png" alt="putIfAbsent" /> <p>A thread tries to get a value for a key calling get on a concurrent map. If the “get” method returns null it creates the missing value and calls putIfAbsent.</p> <h2>Sample Code</h2> <pre class="brush: java">public Set getLanguageTagSet(String category) { // get the value Set tagset = langtagSets.get(category); // if value is null create one if (tagset == null) { tagset = createLanguageTagSet(category); // call putIfAbsent Set ts = langtagSets.putIfAbsent(category, tagset); // if putIfAbsent returns a value a other thread has created a new value in between if (ts != null) { tagset = ts; } } return tagset; } </pre> Taken from sun.util.locale.provider.JRELocaleProviderAdapter. Comments are mine. <h1>Conclusion</h1> Each synchronization idiom can only be used for a specific access pattern. Using it outside this access pattern will lead to race conditions in your application. Therefore always use a tool like <strong><a href="http://vmlens.com/">vmlens</a></strong> to detect race conditions during development and testing. Tue, 26 Jul 2016 22:00:00 GMT http://vmlens.com/articles/3-synchronization-idioms-2 2016-07-26T22:00:00Z How to test if your multi threaded java rest service is thread safe? http://vmlens.com/articles/how-to-test-if-your-multi-threaded-java-rest-service-is-thread-safe In the following you will see, how to test if your rest service is thread safe. Let us start with a simple example, a counter which contains a race condition. <h1>A not thread safe counter</h1> As an example we use the following jersey rest service. It consists of a resource which increments a counter for each post call and returns the new value: <pre class="brush: java">@Path("counter") public class Counter { private static int i = 0; @POST @Produces(MediaType.TEXT_PLAIN) public String addOne() { return new Integer(i++).toString(); } } </pre> This is clearly not thread safe. The access to the variable counter is not synchronized, which will lead to a race condition, if the method "addOne" is called from too many threads in parallel. Let us see, if we can detect this bug with a test. <h1>How to test</h1> To test if this service is thread safe, we need a multi threaded test, like the following: <pre class="brush: java">@RunWith(ConcurrentTestRunner.class) public class CounterTest { private HttpServer server; private WebTarget target; @Before public void setUp() throws Exception { server = Main.startServer(); Client c = ClientBuilder.newClient(); target = c.target(Main.BASE_URI); } @After public void tearDown() throws Exception { server.shutdown(); } @Test public void testAddOne() { String responseMsg = target.path("counter").request().post(Entity.json(null) , String.class); /* * * Checking the responseMsg left out for brevity... * */ } } </pre> The concurrent test runner runs the test method in parallel with THREAD_COUNT threads. To detect race conditions, we need a tool, which can detect race conditions during tests. One such tool is <a href="http://vmlens.com/">vmlens</a>. We can enable it, by adding the <a href="http://vmlens.com/">vmlens</a> agent path to the vm arguments. After running we will see the race condition in <a href="http://vmlens.com/">vmlens</a>: <p><img class="center-block img-responsive" src="http://vmlens.com/WordPress_01/wp-content/uploads/2016/07/artikel_screen_17.07.161.png" alt="Race Condition in REST Service" /></p> <p>After we have found the race condition, we want to fix the race.</p> <h1>Making the rest service thread safe</h1> The easiest way to do this, is to use a java.util.concurrent.atomic.AtomicInteger. AtomicInteger uses a volatile Field internally, making updates visible to all threads. And the used method "addAndGet" is made atomic, by using compareAndSet. <pre class="brush: java">@Path("counter") public class Counter { private static AtomicInteger i = new AtomicInteger(); @POST @Produces(MediaType.TEXT_PLAIN) public String addOne() { return new Integer(i.incrementAndGet()).toString(); } } </pre> <h1>Conclusion</h1> To test a multi threaded java rest service we need two things, a multi threaded test and a tool which can detect java race conditions. For the multi threaded test I used <a href="https://github.com/ThomasKrieger/concurrent-junit">concurrent-junit</a>, for the race condition detection I used <a href="http://vmlens.com/">vmlens</a>. If you have a question or remark please add a comment below. Sun, 17 Jul 2016 22:00:00 GMT http://vmlens.com/articles/how-to-test-if-your-multi-threaded-java-rest-service-is-thread-safe 2016-07-17T22:00:00Z A new way to junit test your multithreaded java code http://vmlens.com/articles/a-new-way-to-junit-test-your-multithreaded-java-code In the following you see a simple example on how to test multithreaded java code with JUnit. Suppose we want to create a counter which can be used concurrently. <p>In the following you see a simple example on how to test multithreaded java code with <a href="http://junit.org/">JUnit</a>. Suppose we want to create a counter which can be used concurrently. So we start with the class Counter and the <a href="http://junit.org/">JUnit</a> test TestCounter:</p> <pre class="brush: java">public class Counter { private int count=0; public void addOne() { count++; } public int getCount() { return count; } } </pre> <pre class="brush: java">import static org.junit.Assert.assertEquals; import org.junit.After; import org.junit.Test; import org.junit.runner.RunWith; import com.anarsoft.vmlens.concurrent.junit.ConcurrentTestRunner; @RunWith(ConcurrentTestRunner.class) public class TestCounter { private Counter counter = new Counter(); @Test public void addOne() { counter.addOne(); } @After public void testCount() { assertEquals("4 Threads running addOne in parallel should lead to 4" , 4 , counter.getCount()); } } </pre> <p>By using the RunWith annotation the <a href="http://junit.org/">JUnit</a> test is run by a special <a href="https://github.com/ThomasKrieger/concurrent-junit">ConcurrentTestRunner</a>. This test runner runs the method annotated with &quot;Test&quot; parallel in 4 threads. After that it executes the methods marked with the annotation After&quot; in the main thread. <img class="size-full wp-image-854 aligncenter" src="http://vmlens.com/WordPress_01/wp-content/uploads/2015/11/flow.png" alt="flow" width="860" height="549" /> If we run the test case with a race condition catcher like <a href="http://vmlens.com/">vmlens</a> we see the following:</p> <p><img class="alignnone size-full wp-image-2771" src="http://vmlens.com/WordPress_01/wp-content/uploads/2016/06/artikel_new_way_1.png" alt="Race Condition at Count" width="1248" height="600" /></p> <p>We have a race condition accessing the field count. To solve this we declare count as volatile and run the test again.</p> <pre class="brush: java">private volatile int count=0;</pre> <p>Now the test case succeeds. At least almost all the time. If you run the test case very often, you will sometimes see an assertion failed exception.</p> <pre class="brush: java">java.lang.AssertionError: 4 Threads running addOne in parallel should lead to 4 expected:&lt;4&gt; but was:&lt;3&gt; at org.junit.Assert.fail(Assert.java:88) at org.junit.Assert.failNotEquals(Assert.java:834) at org.junit.Assert.assertEquals(Assert.java:645) at TestCounter.testCount(TestCounter.java:21) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) at com.anarsoft.vmlens.concurrent.junit.internal.InvokeListOfMethods.evaluate(InvokeListOfMethods.java:23) at com.anarsoft.vmlens.concurrent.junit.internal.ConcurrentStatement.evaluateStatement(ConcurrentStatement.java:12) at com.anarsoft.vmlens.concurrent.junit.ConcurrentTestRunner.evaluateStatement(ConcurrentTestRunner.java:212) at com.anarsoft.vmlens.concurrent.junit.ConcurrentTestRunner.runChildrenConcurrently(ConcurrentTestRunner.java:172) at com.anarsoft.vmlens.concurrent.junit.ConcurrentTestRunner.access$0(ConcurrentTestRunner.java:78) at com.anarsoft.vmlens.concurrent.junit.ConcurrentTestRunner$1.evaluate(ConcurrentTestRunner.java:72) at org.junit.runners.ParentRunner.run(ParentRunner.java:363) at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:86) at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:459) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:675) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:382) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:192) </pre> <p>Actually count++ is not one operation but 6 byte code operations, containing one read and one write to the field count:</p> <pre class="brush: java">ALOAD 0: this DUP GETFIELD Counter.count : int ICONST_1 IADD PUTFIELD Counter.count : int </pre> <p>If we look at the volatile field in vmlens we see the order of read and writes:</p> <p><img class="center-block img-responsive" src="http://vmlens.com/WordPress_01/wp-content/uploads/2016/06/artikel_new_way_2.png" alt="Order of events in case of race condition" width="899" height="366" /></p> <p>To make sure that the test case always fails we can use a “wait point” at the write to the field:</p> <p><img class="center-block img-responsive" src="http://vmlens.com/WordPress_01/wp-content/uploads/2016/06/artikel_new_way_3.png" alt="Wait Point" width="897" height="361" /></p> <p>By letting the threads wait before the write operation, we make sure that two threads read the field in parallel. And reading in parallel causes the bug.</p> <p>To solve this we need to make those methods atomic. This can be done by using java.util.concurrent.atomic.AtomicInteger:</p> <pre class="brush: java">import java.util.concurrent.atomic.AtomicInteger; public class Counter { private final AtomicInteger count= new AtomicInteger(); public void addOne() { count.incrementAndGet(); } public int getCount() { return count.get(); } } </pre> <p>Now the test case always succeed. As test runner I used <a href="https://github.com/ThomasKrieger/concurrent-junit">concurrent-junit</a>, as race condition catcher I used <a href="http://vmlens.com/">vmlens</a></p> <p>And although it is a fairly simple method, I realize that there are some tricky parts. Especially when you have to test more complicated classes. So, if you have any questions, just ask them in a comment below.</p> Thu, 07 Jul 2016 22:00:00 GMT http://vmlens.com/articles/a-new-way-to-junit-test-your-multithreaded-java-code 2016-07-07T22:00:00Z 5 Ways to thread safe update a field in java http://vmlens.com/articles/5-ways-to-thread-safe-update-a-field-in-java-2 Here are the 5 possibilities to update a field in java in a thread safe way. But before we start, what do you have to make look at? <p>Here are the 5 possibilities to update a field in java in a thread safe way. But before we start, what do you have to make look at? If you access a field from many threads, you must make sure that: <ol style="list-style-type: lower-alpha;"> <li style="list-style-type: none"> <ol style="list-style-type: lower-alpha;"> <li>changes are made visible to all threads, and</li> <li>the value is not changed during the update by the other thread, and</li> <li>reading threads do not see the inconsistent intermediate state.</li></p> </ol> </li> </ol> You can achieve this by one of the following 5 ways: <h2>1) volatile Field</h2> <h3>When to use?</h3> You can use a volatile field when you have only one thread updating and many threads reading a single-valued field. Or use it, when the writing threads do not read the field. You can use it only for single valued fields like boolean or int. If you want to update object graphs or collections, use copy on write, as described below, instead. <h3>Example</h3> The following example code shows a worker thread, which stops processing based on a volatile field. This makes it possible, that other threads, like an event dispatching thread, stop the worker thread. <pre class="brush: java">public class WorkerThread extends Thread { private volatile boolean canceled = false; public void cancelThread() { this.canceled = true; } @Override public void run() { while( ! canceled ) { // Do Some Work } } } </pre> <h3>How does it work?</h3> Declaring the field volatile makes changes made by one thread visible to all other threads. As a writing thread do not read the value, point b “the value is not changed during the update by the other thread” is fulfilled. Since the field is a single value point c “reading threads do not see inconsistent intermediate state” is also fulfilled. <h3>How to test?</h3> By using <a href="http://vmlens.com/">vmlens</a>, an eclipse plugin to test multi-threaded software and to detect java race conditions, we can find fields, which should be declared volatile. After declaring the field volatile, we can check in the “order of event” view of <a href="http://vmlens.com/">vmlens</a>, that the field is correctly read and written: <p><img class="center-block img-responsive" src="http://vmlens.com/WordPress_01/wp-content/uploads/2016/07/5way_screen_11.png" alt="Order of Events in vmlens" /></p> <h2>2) copy on write</h2> <h3>When to use?</h3> Use copy on write, if you want to update a graph of objects or a collection and the threads mostly read and only rarely update. <h3>Example</h3> The following shows the add and get Method from java.util.concurrent.CopyOnWriteArrayList <pre class="brush: java">private transient volatile Object[] array; final Object[] getArray() { return array; } public boolean add(E e) { final ReentrantLock lock = this.lock; lock.lock(); try { Object[] elements = getArray(); int len = elements.length; Object[] newElements = Arrays.copyOf(elements, len + 1); newElements[len] = e; setArray(newElements); return true; } finally { lock.unlock(); } } public E get(int index) { return get(getArray(), index); } </pre> <h3>How does it work?</h3> Again declaring the field volatile makes changes made by one thread visible to the other threads. By using a lock around the updating method, we make sure that the value is not changed during the updating process. Since we copy the data before changing it, reading threads do not see the inconsistent intermediate state. <h3>How to test?</h3> We can test this by using a multithreaded test and adding a wait point inside <a href="http://vmlens.com/">vmlens</a> at the read of the field. <p><img class="center-block img-responsive" src="http://vmlens.com/wp-content/WordPress_01/uploads/2016/07/5ways_screen_2.png" alt="Wait Point in vmlens" /></p> <h2>3) lock based atomic update</h2> <h3>When to use?</h3> Use locks, when updates and reads happen equally often. Use it until the lock becomes a bottleneck and you need the more performant solution compareAndSet as described below. <h3>Example</h3> The following example shows a lock based counter: <pre class="brush: java">public class LockBasedCounter { private int i = 0; public synchronized void addOne() { i++; } public synchronized int get() { return i; } } </pre> <h3>How does it work?</h3> The synchronize statements make sure that the changes made by one thread are seen by the other threads. Since only one thread can execute the methods protected by the lock at a given time, the value can not be changed during the update by another thread and the other threads can not see an intermediate inconsistent state. <h3>How to test?</h3> We can test this by using a multi-threaded test and adding a wait point at the updating method. <p><img class="center-block img-responsive" src="http://vmlens.com/WordPress_01/wp-content/uploads/2016/07/5way_screen_31.png" alt="Wait Point in vmlens" /></p> <h2>4) compare And Set based atomic update</h2> <h3>When to use?</h3> Use compareAndSet, when the lock in the solution described above becomes a bottleneck. Or use it, if there exists a ready-made solution, as for example the AtomicInteger as shown below. <h3>Example</h3> The following implements a counter based on Atomic Integer. <pre class="brush: java">public class AtomicIntegerCounter { private final AtomicInteger i = new AtomicInteger(); public void addOne() { i.incrementAndGet(); } public int get() { return i.get(); } } </pre> The Atomic Integer uses compareAndSet Internally in the incrementAndGet method: <pre class="brush: java">public final int incrementAndGet() { for (;;) { int current = get(); int next = current + 1; if (compareAndSet(current, next)) return next; } } </pre> <h3>How does it work?</h3> Again declaring the field volatile makes changes made by one thread visible to the other threads. You are optimistically calculating the new value and only set the calculated value when the value of the field is still the same as at the beginning of the calculation. Thereby you make sure that the value is only written if it was not changed by another thread. If your field points to a collection or graph of objects, you must create a copy before your update, similar as copy on write. <h3>How to test?</h3> We can test this by using a multi-threaded test and adding a wait point in <a href="http://vmlens.com/">vmlens</a> at the compareAndSet method. <h2>5) Benign Data Race</h2> <h3>When to use?</h3> Only use this when you can sacrifice correctness for performance. <h3>Example</h3> The following example shows a counter used in to switch between different implementations in the class sun.reflect.NativeMethodAccessorImpl <pre class="brush: java">class NativeMethodAccessorImpl extends MethodAccessorImpl { private Method method; private DelegatingMethodAccessorImpl parent; private int numInvocations; NativeMethodAccessorImpl(Method method) { this.method = method; } public Object invoke(Object obj, Object[] args) throws IllegalArgumentException, InvocationTargetException { if (++numInvocations &gt; ReflectionFactory.inflationThreshold()) { MethodAccessorImpl acc = (MethodAccessorImpl) new MethodAccessorGenerator(). generateMethod(method.getDeclaringClass(), method.getName(), method.getParameterTypes(), method.getReturnType(), method.getExceptionTypes(), method.getModifiers()); parent.setDelegate(acc); } return invoke0(method, obj, args); } ... } </pre> <h3>How does it work?</h3> This way of updating the field does neither guarantee that changes are visible in other threads, nor that other threads are not changing the field between an update. But sometimes, as in the above example, you can live with incorrect results for higher performance. <h3>How to test?</h3> You can test this only with a single threaded test since multiple threads lead to non-deterministic behavior. <h2>Conclusion</h2> Which of the 5 ways to update a field in a thread safe way you use, depends on your performance and safety requirements. Independent of which way you use, you should test it. Read more about unit testing multi-threaded software with <a href="http://vmlens.com/">vmlens</a> and <a href="https://github.com/ThomasKrieger/concurrent-junit">concurrent-junit</a> in <a href="http://vmlens.com/articles/a-new-way-to-junit-test-your-multithreaded-java-code/">a new way to junit test your multithreaded java code</a> . If you have a question or remark please add a comment below. Fri, 24 Jun 2016 22:00:00 GMT http://vmlens.com/articles/5-ways-to-thread-safe-update-a-field-in-java-2 2016-06-24T22:00:00Z Detecting Java Race Conditions With Tests Part 1 http://vmlens.com/articles/detecting-java-race-conditions-with-tests Detecting java race conditions is hard, detecting them during tests is impossible. Really? <p class="p1"><span class="s1">Detecting java race conditions is hard, detecting them during tests is impossible. Really? </span></p> <p class="p1"><span class="s1">In the following I want to show you, that it is actually rather easy. In this first part you will see how to detect lost updates, the first type of java race conditions. In the second part we will look at the second type of java race conditions: Non atomic access.</span></p> <h3 class="p3"><span class="s1"><b>So what are visibility problems anyway?<span class="Apple-converted-space"> </span></b></span></h3> <p class="p1"><span class="s1">If you read and write to the same field from different threads without synchronization, you will lose updates.</span></p> <p class="p1" style="text-align: center;"><img class="alignnone size-full wp-image-2697" src="http://vmlens.com/WordPress_01/wp-content/uploads/2016/03/artikel_17.03_bild2.1.png" alt="Multi Core Memory Architecture" width="720" height="416" /></p> <p style="text-align: center;"></p> <p style="text-align: center;"></p> <p class="p1"><span class="s1">Why? In multi core computers every core has a cache. If you write to a field, another thread sees the old value if the thread runs on another core. Or if the thread runs on the same core, you will see the new value. Only if the cache was invalidated, the thread will always see the new value. Without invalidating the cache you lose updates. And invalidating the cache is what a synchronization statement or writing to a volatile field does.</span></p> <h3 class="p1" style="text-align: left;"><span class="s1"><b>O.K. so what can I do?</b></span></h3> <p class="p1" style="text-align: center;"><img class="size-full wp-image-2465 alignright" src="http://vmlens.com/WordPress_01/wp-content/uploads/2016/03/artikel_17.03_bild31.png" alt="Java Duke" width="200" height="181" /></p> <p class="p1" style="text-align: center;"></p> <p class="p1"><span class="s1">Simply run a multithreaded test. Afterwards check if each concurrent field access is correctly synchronized. The java memory model formally specifies which statements correctly synchronize a field access.<span class="Apple-converted-space"> </span>For example the following is correctly synchronized since between the write and read is a thread start:</span></p> <pre class="brush: java">public class ThreadStart extends Thread { private int i = 0; public static void main(String[] args ) { ThreadStart threadStart = new ThreadStart(); threadStart.i = 8; /* * * see https://docs.oracle.com/javase/specs/jls/se7/html/jls-17.html#jls-17.4.4 * An action that starts a thread synchronizes-with the first action in the thread * it starts. */ threadStart.start(); } @Override public void run() { int j = i; } } </pre> <p class="p1"><span class="s1">So let’s get started with a simple example, a counter:</span></p> <pre class="brush: java">public class Counter { private int i = 0; public void addOne() { i++; } } </pre> <p class="p1"><span class="s1">To test the counter, we use the following junit test. The <a href="https://github.com/ThomasKrieger/concurrent-junit" target="_blank" rel="noopener noreferrer">concurrent test runner</a> runs the test method from four different threads. </span></p> <pre class="brush: java">import org.junit.Test; import org.junit.runner.RunWith; import com.anarsoft.vmlens.concurrent.junit.ConcurrentTestRunner; @RunWith(ConcurrentTestRunner.class) public class TestCounter { private final Counter counter = new Counter(); @Test public void testAdd() { counter.addOne(); } } </pre> <p class="p1"><span class="s1">To trace the fields and synchronization actions, add the <a href="http://vmlens.com" target="_blank" rel="noopener noreferrer">vmlens agent</a> path to the virtual machine arguments. After the run, <a href="http://vmlens.com" target="_blank" rel="noopener noreferrer">vmlens </a>checks all field accesses. If <a href="http://vmlens.com" target="_blank" rel="noopener noreferrer">vmlens </a>finds a field access which is not correctly synchronized, we have detected a java race condition.</span></p> <h3 class="p1" style="text-align: left;"><strong><span class="s1">It is not a bug, it is a feature</span></strong></h3> <p class="p1"><span class="s1">Probably you have noticed the detected java race condition accessing the field numInvocations in the class sun.reflect.NativeMethodAccessorImpl. If we look at the source code, we see that it is used to switch to a faster implementation, if a threshold is reached. </span></p> <pre class="brush: java">public Object invoke(Object obj, Object[] args) throws IllegalArgumentException, InvocationTargetException { if (++numInvocations &gt; ReflectionFactory.inflationThreshold()) { MethodAccessorImpl acc = (MethodAccessorImpl) new MethodAccessorGenerator(). generateMethod(method.getDeclaringClass(), method.getName(), method.getParameterTypes(), method.getReturnType(), method.getExceptionTypes(), method.getModifiers()); parent.setDelegate(acc); } return invoke0(method, obj, args); } </pre> <span class="s1">This is an example for a java race condition, which is not a bug but a feature. The performance loss to make the numInvocations count thread safe is worse than to lose some values.<span class="Apple-converted-space"> </span></span> <h3><strong><span class="s1">If it is not tested it is probably broken</span></strong></h3> <p class="p1"><span class="s1">Since detecting java race conditions with testing is rather new, you can expect many undetected race conditions. Here is an example of race conditions during the start and stop of <a href="https://jenkins-ci.org/">Jenkins</a>, an open source continuous integration server:</span></p> <p class="p1"><img class="alignnone size-full wp-image-2466" src="http://vmlens.com/WordPress_01/wp-content/uploads/2016/03/artikel_17.03_bild5-Kopie.png" alt="Java Race Conditions in Jenkins " width="1014" height="599" /></p> <p class="p1">So detecting lost updates with tests is possible. As test runner I used <a href="https://github.com/ThomasKrieger/concurrent-junit">concurrent-junit</a>, as race condition catcher I used <a href="http://vmlens.com">vmlens. </a>In the next part we will look at <a href="http://vmlens.com/articles/detecting-java-race-conditions-with-tests-part-2/">testing non atomic updates</a>.</p> Thu, 10 Mar 2016 23:00:00 GMT http://vmlens.com/articles/detecting-java-race-conditions-with-tests 2016-03-10T23:00:00Z Is your java eclipse plugin thread safe? http://vmlens.com/articles/is-your-java-eclipse-plugin-thread-safe A race condition happens when the outcome of the program depends on the sequence or timing of other uncontrollable events. <h2>Or does it contain race conditions?</h2> <a href="https://en.wikipedia.org/wiki/Race_condition">According to wikipedia:</a> <blockquote>A race condition happens when the outcome of the program depends on the sequence or timing of other uncontrollable events. It becomes a bug when events do not happen in the order the programmer intended</blockquote> We searched inside eclipse for race conditions to see what are the most common types of race conditions inside eclipse and its plugins. <p><img class="alignnone size-full wp-image-395" src="http://vmlens.com/WordPress_01/wp-content/uploads/2015/08/races.png" alt="races" width="600" height="400" /></p> <p>The following types of race conditions were found by <a href="http://vmlens.com/">vmlens </a>inside eclipse luna during startup and debugging of a java project:</p> <h2>No synchronization at all</h2> The most common cause for race conditions was accessing the same field from different threads without any synchronization at all. <table> <tbody> <tr> <th>Object</th> <th>Count</th> </tr> <tr> <td>Concurrently Accessed Fields</td> <td>2065</td> </tr> <tr> <td>Fields Traced</td> <td>27114</td> </tr> <tr> <td>Monitors</td> <td>7162</td> </tr> <tr> <td>Locks</td> <td>427</td> </tr> <tr> <td>Threads</td> <td>52</td> </tr> <tr> <td>Volatile Fields</td> <td>2836</td> </tr> </tbody> </table> During this run 2065 different fields were accessed by more than one thread, 4 of them without synchronization. <p>2836 volatile fields were used. For 3 more fields it would have been necessary to declare them as volatile. This leads to the second type of race conditions found, visibility problems.</p> <h2>Visibility</h2> A field is accessed by many threads, but not declared as volatile. <p>The jvm does not directly write field updates into the main memory but first in registers or a cache of your cpu. As long as your program is running on one core of your pc, this is not a problem. But if the threads runs on different cores, they probably will not see the updates to the fields.</p> <p>This problem appears most often with boolean flags. Like the terminated field of org.eclipse.equinox.internal.util.impl.tpt.timer.TimerImpl in the run method: <pre class="brush: java">public void run() { TimerQueueNode n = null; while (!terminated) { synchronized (sync) { if (n == null &amp;&amp; queue.isEmpty()) { try { sync.wait(); } catch (Exception e) { } // todo check if isEmpty is necessary if (queue.isEmpty() || terminated) { continue; } } }</p> </pre> <h2>Conclusion</h2> For 2065 concurrently accessed fields, <a href="http://vmlens.com/">vmlens</a> found 7 race conditions. All other were correctly synchronized by 7162 monitors or declared as volatile. Fri, 11 Dec 2015 23:00:00 GMT http://vmlens.com/articles/is-your-java-eclipse-plugin-thread-safe 2015-12-11T23:00:00Z Synchronized java.util.HashMap vs. java.util.concurrent.ConcurrentHashMap http://vmlens.com/articles/synchronized-java-util-hashmap-vs-java-util-concurrent-concurrenthashmap Using java.util.HashMap from many threads without any synchronization leads to race conditions. <p>Using java.util.HashMap from many threads without any synchronization leads to race conditions. So we need some kind of synchronization. The easiest way is to synchronize the complete class. Another way is to use a java.util.concurrent.ConcurrentHashMap. But what does this mean to the performance of the application? As an example we use a HashMap to count the occurrence of a String: <pre class="brush: java">import java.math.BigInteger; import java.security.SecureRandom; import org.openjdk.jmh.annotations.Benchmark; import org.openjdk.jmh.annotations.Scope; import org.openjdk.jmh.annotations.State; @State(Scope.Benchmark) public class MyBenchmark { private HashMapBasedCollectionmap = new HashMapBasedCollection(); private String nextSessionId(SecureRandom random) { return new BigInteger(130, random).toString(32); } public String[] buildNames(int size) { SecureRandom random = new SecureRandom(); String[] result = new String[size]; for (int i = 0; i &lt; size; i++) { result[i] = nextSessionId(random); } return result; } @Benchmark public void testMethod() { String[] array = buildNames(40); for (int j = 0; j &lt; 200; j++) { for (int i = 0; i &lt; 40; i++) { map.addOne(array[i]); } } } }</p> </pre> <pre class="brush: java">import java.util.HashMap; import java.util.function.BiFunction; public class HashMapBasedCollection { private HashMap&lt;string,integer&gt; map = new HashMap&lt;string,integer&gt;(); public static final BiFunction&lt;string, integer,="" integer=""&gt; fun = new BiFunction&lt;string, integer,="" integer=""&gt;() { @Override public Integer apply(String t, Integer u) { if( u == null ) { return new Integer(0); } return u + 1; } }; /** * * Warning: not thread safe * */ public void addOne(String key) { map.compute(key, fun); } } </pre> As expected if we run the method “testMethod()” with two threads we see a race condition. Output from <a href="http://vmlens.com" >vmlens</a>, a tool to detect race conditions: <img class="center-block img-responsive" src="http://vmlens.com/WordPress_01/wp-content/uploads/2015/11/race_hash_map.png" alt="race_hash_map" /> So we need to use java.util.HashMap with synchronization: <pre class="brush: java">public class SynchronizedHashMapBasedCollection { private HashMap&lt;string,integer&gt; map = new HashMap&lt;string,integer&gt;(); public synchronized void addOne(String key) { map.compute(key, HashMapBasedCollection.fun); } } </pre> Or a java.util.concurrent.ConcurrentHashMap: <pre class="brush: java">public class ConcurrentHashMapBasedCollection { private final ConcurrentHashMap&lt;string,integer&gt; map = new ConcurrentHashMap&lt;string,integer&gt;(); public void addOne(String key) { map.compute(key, HashMapBasedCollection.fun ); } } </pre> Below you see the throughput for the two implementations for different counts of thread. <img class="center-block img-responsive" src="http://vmlens.com/WordPress_01/wp-content/uploads/2015/11/concurrentVsSynchronized.png" alt="concurrentVsSynchronized" /> That the performance of the ConcurrentHashMap for many threads is better is probably no surprise. What surprised me is that the performance for one thread is the same. This means ConcurrentHashMap is a good alternative to synchronized HashMap even for few threads. The benchmark was created using <a href="http://openjdk.java.net/projects/code-tools/jmh/" >jmh</a>. The race condition was detected by <a href="http://vmlens.com" target="_blank" >vmlens</a>. Sat, 14 Nov 2015 23:00:00 GMT http://vmlens.com/articles/synchronized-java-util-hashmap-vs-java-util-concurrent-concurrenthashmap 2015-11-14T23:00:00Z