In the following, I want to show you how to test if your spring application is thread-safe. As an example application, I use the spring petclinic project.

To detect concurrency bugs during our tests we use vmlens. vmlens traces the test execution and analyzes the trace afterward. It detects deadlocks and race conditions during the test run.

Testing

To test the spring project we parallelize the existing unit tests. The following shows test method runs the existing test shouldFindAllPetTypes of the ClinicServiceTests class in parallel:

public class ClinicServiceTests {
    @Test
    public void testMultithreaded() throws InterruptedException 
    {
    	TestUtil.runMultithreaded( new Runnable() {
			public void run() {
				try{
					shouldFindAllPetTypes();
				}
				catch(Exception e)
				{
					e.printStackTrace();
				}
			}
    	}
    	, 5);
    }

The TestUtil.runMultithreaded method runs the runnable with n threads in parallel:

	public static void runMultithreaded(Runnable  runnable, int threadCount) throws InterruptedException
	{
		List<Thread>  threadList = new LinkedList<Thread>();	
		for(int i = 0 ; i < threadCount; i++)
		{
			threadList.add(new Thread(runnable));
		}
		for( Thread t :  threadList)
		{
			t.start();
		}
		for( Thread t :  threadList)
		{
			t.join();
		}
	}

You can find the source of the class TestUtil at GitHub here. After running the junit test we see the following report in vmlens:

Analyzing

Let us look at one of the races found, the race at accessing the field org.hsqldb.HsqlNameManager.sysNumber.

The access to the field is locked in the methods org.hsqldb.StatementManager.compile, org.hsqldb.Session.execute and org.hsqldb.jdbc.JDBCConnection.prepareStatement but each thread uses a different monitor. Here is as an example the JDBCConnection prepareStatement method:

  public synchronized PreparedStatement prepareStatement(
            String sql) throws SQLException {
        checkClosed();
        try {
            return new JDBCPreparedStatement(this, sql,
                    JDBCResultSet.TYPE_FORWARD_ONLY,
                    JDBCResultSet.CONCUR_READ_ONLY, rsHoldability,
                    ResultConstants.RETURN_NO_GENERATED_KEYS, null, null);
        } catch (HsqlException e) {
            throw JDBCUtil.sqlException(e);
        }
    }

The problem is that the synchronization happens on the PreparedStatement and Session which is created for each thread, while HsqlNameManager is shared between all threads. That HsqlNameManager is shared between all threads can be seen in the method org.hsqldb.Table.createPrimaryKey:

 public void createPrimaryKey(HsqlName indexName, int[] columns,
                                 boolean columnsNotNull) {
        if (primaryKeyCols != null) {
            throw Error.runtimeError(ErrorCode.U_S0500, "Table");
        }
        if (columns == null) {
            columns = ValuePool.emptyIntArray;
        }
        for (int i = 0; i < columns.length; i++) {
            getColumn(columns[i]).setPrimaryKey(true);
        }
        primaryKeyCols = columns;
        setColumnStructures();
        primaryKeyTypes = new Type[primaryKeyCols.length];
        ArrayUtil.projectRow(colTypes, primaryKeyCols, primaryKeyTypes);
        primaryKeyColsSequence = new int[primaryKeyCols.length];
        ArrayUtil.fillSequence(primaryKeyColsSequence);
        HsqlName name = indexName;
        if (name == null) {
            name = database.nameManager.newAutoName("IDX", getSchemaName(),
                    getName(), SchemaObject.INDEX);
        }
        createPrimaryIndex(primaryKeyCols, primaryKeyTypes, name);
        setBestRowIdentifiers();
    }

Line 20 shows that the nameManager is part of the database object which is shared between all threads. The race happens in the method org.hsqldb.HsqlNameManager.newAutoName where the field sysNumber is read and written by the two threads:

    public HsqlName newAutoName(String prefix, String namepart,
                                HsqlName schema, HsqlName parent, int type) {
        StringBuffer sb = new StringBuffer();
        if (prefix != null) {
            if (prefix.length() != 0) {
                sb.append("SYS_");
                sb.append(prefix);
                sb.append('_');
                if (namepart != null) {
                    sb.append(namepart);
                    sb.append('_');
                }
                sb.append(++sysNumber);
            }
        } else {
            sb.append(namepart);
        }
        HsqlName name = new HsqlName(this, sb.toString(), type, false);
        name.schema = schema;
        name.parent = parent;
        return name;
    }

In line 13 the field sysNumber is incremented by ++. The operation ++ is not one atomic operation but 6 byte code operations:

ALOAD 0: this
DUP
GETFIELD Counter.count : int
ICONST_1
IADD
PUTFIELD Counter.count : int

If two threads execute this in parallel, this might lead to a scenario where both threads read the same value and then both increment the value, leading to duplicate values. And since the field sysNumber is used to generate the primary key the race condition leads to duplicated primary keys.