Top 5 tips for debugging Java thread-safety, multi-threading, or concurrency issues

Tutorial style debugging of Java thread safety issues extending general tips outlined in Identifying and fixing Java concurrency issues

#1: Manually reviewing the code for any obvious thread-safety issues. Good knowledge of multi-threading is required.

#2: List all possible causes and add extensive log statements and write test cases to prove or disprove your theories. The log statements will have something like

log.info(Thread.currentThread().getName() + " produced: " + count);
System.out.println(Thread.currentThread().getName() + " consumed: " + consumed);

#3: Using your IDE debugging capability by setting a conditional break point.  Thread.currentThread().getName().equals("Thread-0"). For example, stopping for particular thread as demonstrated below step by step in eclipse IDE. The working code can be found at Java producer consumer working example.

The above code continuously produces output like:

Thread-0 produced: 1
Thread-1 consumed: 1
Thread-0 produced: 2
Thread-1 consumed: 2
Thread-0 produced: 3
Thread-1 consumed: 3
Thread-0 produced: 4
Thread-1 consumed: 4

You can add a break point to the ProducerConsumer that is used by both worker threads ProducerThread and ConsumerThread. Both these worker threads are spawned by the default main thread.

Step 1: Create a conditional debug point as shown below in the first line of the produce method.

Step 2: Run the ProducerConsumerTest in debug mode. The execution stops on the break point when worker "Thread-0" enters the break point. In the above example only one thread enters produce( ) method. But in industrial applications you can have many threads.

You also have options to suspend and resume the threads you want as shown below with right-click context menu in the debug window. When you are suspended, you can also copy the stack at that suspended point in time.

You can also inspect and watch shared variables to ascertain any thread-safety issues.

The above diagram adds a watch expression on a shared variable.


#4: Thread dumps are very useful for diagnosing synchronization problems such as deadlocks. The trick is to take 5 or 6 sets of thread dumps at an interval of 5 seconds between each to have a log file that has 25 to 30 seconds worth of run-time action. For thread dumps, use kill -3 in Unix and CTRL+BREAK in Windows. There are tools like Thread Dump Analyzer (TDA), Samurai, etc. to derive useful information from the thread dumps to find where the problem is. For example, Samurai colors idle threads in grey, blocked threads in red, and running threads in green. You must pay more attention to those red threads.

Creating a thread dump in windows

Step 1: While the ProducerConsumerTest  is running, open a DOS command prompt at type jconsole.

Note down the process id: 4800. Connect to 4800, and 

Step 2: You can detect any deadlocks by clicking on the "Detect Deadlock" button in the threads tab.

Step 3: To get a thread dump, open a DOS command prompt and type jstat [pid]

jstat 4800

This will produce a stack trace. The stack trace looks something like

You need to pay attention to blocked threads, and there are tools like  Thread Dump Analyzer (TDA), Samurai, etc to analyze thread dumps.

jstack and jconsole are provided with your JDK installation under jdk[version]/bin.

#5: There are static analysis tools like Sonar, ThreadCheck, etc for catching concurrency bugs at compile-time by analyzing the byte code. Sonar produces reports with recommendations.