Tests for thread safety differ from typical single-threaded tests. To test if a method is thread-safe we need to call the method in parallel from multiple threads. We need to do this for all potential thread interleavings. And afterward, we need to check if the result is correct.

Those three requirements for our test lead to a special type of tests for thread safety which differ from typical single-threaded tests. Since we want to test all thread interleavings our test must be repeatable and run automatically. And since the methods run in parallel the potential result is a combination of different outcomes.

Let us look at an example to see how this looks in practice.

A test for thread safety

Suppose we want to test if the following class representing an Address is thread-safe. It offers one method to update the street and city, the method update and one method to read the complete Address, the method toString:

public class MutableAddress {
	private volatile String street;
	private volatile String city;
	private volatile String phoneNumber;
	public MutableAddress(String street, String city, 
		String phoneNumber) {
		this.street = street;
		this.city = city;
		this.phoneNumber = phoneNumber;
	}
	public void update(String street ,String city ) {
		this.street = street;
		this.city = city;
	}
	public String toString() {
		return "street=" + street + ",city=" + city + ",
		phoneNumber=" + phoneNumber;
	}
}

I use volatile fields, line 2 till 4, to make sure that the threads always see the current values, as explained in greater detail here. You can download the source code of all examples from GitHub here.

Now let us first see if the combination of toString and update is thread-safe. Here is the test:

import com.vmlens.api.AllInterleavings;
public class TestToStringAndUpdate {
	@Test
	public void testMutableAddress() throws InterruptedException {
		try (AllInterleavings allInterleavings = 
			new AllInterleavings("TestToStringAndUpdate_Not_Thread_Safe");) {
			while (allInterleavings.hasNext()) {
				MutableAddress address = new MutableAddress("E. Bonanza St.",
					 "South Park", "456 77 99");
				String readAddress = null;
				Thread first = new Thread(() -> {
					address.update("Evergreen Terrace", "Springfield");
				});
				first.start();
				readAddress = address.toString();
				first.join();
				assertTrue("readAddress:" + readAddress,readAddress.equals(
		"street=E. Bonanza St.,city=South Park,phoneNumber=456 77 99") 
					|| readAddress.equals(
		"street=Evergreen Terrace,city=Springfield,phoneNumber=456 77 99"));
			}
		}
	}
}

The test executes the two methods in parallel from two threads. To test all thread interleavings we put the complete test in a while loop iterating over all thread interleavings using the class AllInterleavings from vmlens, line 7. To see if the class is thread-safe we compare the result against the to potential outcomes, the value before the update and after the update, lines 17 till 20.

Running the test leads to the following error:

java.lang.AssertionError: readAddress:street=Evergreen Terrace
		,city=South Park,phoneNumber=456 77 99
	at com.vmlens.tutorialCopyOnWrite.TestToStringAndUpdate.
		testMutableAddress(TestToStringAndUpdate.java:22)

To see what went wrong we look at the report vmlens generated.

The problem is that for one thread interleaving the thread with Thread id 30 first updates the street name and then the main thread, thread id 1, reads the street and city name. So the main thread reads a partial updated address which leads to the error.

To make the address class thread-safe we copy the address value every time we update the address. Here is a thread-safe implementation using this technique. It consists of two classes, an immutable value, and a mutable container.

First the immutable value class:

public class AddressValue {
	private final String street;
	private final String city;
	private final String phoneNumber;
	public AddressValue(String street, String city, 
				String phoneNumber) {
		super();
		this.street = street;
		this.city = city;
		this.phoneNumber = phoneNumber;
	}
	public String getStreet() {
		return street;
	}
	public String getCity() {
		return city;
	}
	public String getPhoneNumber() {
		return phoneNumber;
	}
}

Second the mutable container class:

public class AddressUsingCopyOnWrite {
	private volatile AddressValue addressValue;
	private final Object LOCK = new Object();
	public AddressUsingCopyOnWrite(String street, 
			String city, String phone) {
		this.addressValue = new AddressValue( street, 
				city,  phone);
	}
	public void update(String street ,String city ) {
		synchronized(LOCK){
			addressValue = new AddressValue(  street,  city,  
					addressValue.getPhoneNumber() );
		}
	}
	public String toString() {
		AddressValue local = addressValue;
		return "street=" + local.getStreet()
		+ ",city=" + 	local.getCity() + 
		",phoneNumber=" + local.getPhoneNumber();
	}
}

The class AddressUsingCopyOnWrite creates a new address value every time it updates the variable addressValue. This makes sure that we always read a consistent address, either the value before or after the update.

If we run the test with those two classes, the test succeeds.

What do we need to test?

So far we tested the combination of toString and update for thread safety. To test if a class is thread-safe we need to test all combinations of modifying methods and all combinations of read-only methods together with modifying methods. So for our example class, we need to test the following two combinations:

1. update and update
2. toString and update

Since the combinations of read-only methods are automatically thread-safe we do not need to test the combination of the method toString with itself.

Data Races

So far we used volatile fields to avoid data races. Let us see what happens when we use normal fields instead. So in our thread-safe class AddressUsingCopyOnWrite we remove the volatile modifier and re-run our test. Now, vmlens reports a data race in the file target/interleave/issues.html

A data race is an access to a field where a thread might read a stale value. If the thread indeed reads a stale value depends on external factors like which optimizations the compiler is using or on which hardware architecture the JVM is running and on which cores the threads are running. To make it possible to always detect such a data race independent of those external factors, vmlens searches for data races in the execution trace of the test run. And if vmlens have found one as in the example it reports them in the issue report.

Summary

Tests for thread safety differ from typical single-threaded tests. To test if the combination of two methods, a and b, is thread-safe call them from two different threads. Put the complete test in a while loop iterating over all thread interleavings with the help from the class AllInterleavings from vmlens. Test if the result is either a after b or b after a. And to test if a class is a thread-safe test all combinations of modifying methods and all combinations of read-only methods together with modifying methods.