The question is around the discussion "Multiple Java threads seemingly locking same monitor". In our application, we are facing similar issue. Sometimes the application is running extremely slow. Multiple thread dumps have been captured. The thread dumps indicate that 2/3 threads have acquired the same lock object at the same point of time and are in the BLOCKED state. Other threads (10 to 20 in number at different point of time) are BLOCKED while waiting for the very same lock object. Pseudo thread dump looks like the following:
"MyThread-91" prio=3 tid=0x07552800 nid=0xc7 waiting for monitor entry [0xc4dff000]
java.lang.Thread.State: BLOCKED (on object monitor)
at com.myCompany.abc.util.MySharedLinkedList$MySharedIterator.hasNext(MySharedLinkedList.java:177)
- locked <0xce1fb810> (a com.myCompany.abc.util.MySharedLinkedList)
at com.myCompany.abc.util.MyEventProcessor.notifyListeners(MyEventProcessor.java:2644)
...............................................................................................
"MyThread-2" prio=3 tid=0x07146400 nid=0x6e waiting for monitor entry [0xc6aef000]
java.lang.Thread.State: BLOCKED (on object monitor)
at com.myCompany.abc.util.MySharedLinkedList$MySharedIterator.hasNext(MySharedLinkedList.java:177)
- locked <0xce1fb810> (a com.myCompany.abc.util.MySharedLinkedList)
at com.myCompany.abc.util.MyEventProcessor.notifyListeners(MyEventProcessor.java:2644)
................................................................................................
"MyThread-14" prio=3 tid=0x074b9400 nid=0x7a waiting for monitor entry [0xc64ef000]
java.lang.Thread.State: BLOCKED (on object monitor)
at com.myCompany.abc.util.MySharedLinkedList$MySharedIterator.next(MySharedLinkedList.java:194)
- waiting to lock <0xce1fb810> (a com.myCompany.abc.util.MySharedLinkedList)
at com.myCompany.abc.util.MyEventProcessor.notifyListeners(MyEventProcessor.java:2646)
................................................................................................
MyThread-91 and MyThread-2 are BLOCKED while having lock on <0xce1fb810>. MyThread-14 in BLOCKED while waiting for the same lock <0xce1fb810>.
We are not running into any thread deadlock issue here for sure. Please note, threads which are BLOCKED on the lock (0xce1fb810) at any point of time are releasing it subsequently. But, some other threads are getting BLOCKED after acquiring the same lock object. According to the discussion mentioned above (& sample code provided by Gray), it could be because of invoking wait() within a synchronized block. But, We inspected our code and we don't see any wait() getting invoked within the synchronized block. In our case, it's an internal implementation of Linked List which in turn has an inner class implementing iterator. The next() and hasNext() of the iterator implementation locks on the same instance of the outer class, i.e., the instance of custom linked list. When multiple threads are invoking next() and hasNext(), they are moving into the BLOCKED state after "acquiring" the same lock.
Here goes the pseudo code:
public final class MySharedLinkedList<E> implements Collection<E> {
/**
* Represents an entry in the list.
*/
private static final class Entry<E> {
//Instance variables and methods for Entry goes here.
}
/**
* Non fail-fast implementation of iterator for this list.
*/
public final class MySharedIterator implements Iterator<E> {
public boolean hasNext() {
//Some code goes here.
synchronized (MySharedLinkedList.this) {
//Some code goes here.
}
}
public E next() {
//Some code goes here.
synchronized (MySharedLinkedList.this) {
//Some code goes here.
}
}
}
public synchronized Iterator<E> iterator() {
//Returns a new instance of the iterator.
}
}
/**
* Singleton Instance
*/
public class MyEventProcessor {
//listeners contains a number of Node objects
private final SharedLinkedList<Node> listeners = new SharedLinkedList<Node>();
private void notifyListeners() {
final SharedLinkedList<ProvAPIEventNode>.SharedIterator iterator = listeners.sharedIterator();
try {
while (iterator.hasNext()) {
final Node node = iterator.next();
//Lots of other things go here
} catch (Exception e) {
//Handle the exception
}
}
}
}
So, the question is what else (other than wait()) might lead to this scenario?
This blog talks about a similar situation (under the section "Example 2: When the Processing Performance is Abnormally Slow"). But not sure if something similar is happening here.
Don't close this thread as a duplicate of this or this. As mentioned, the behavior is similar, but I guess root cause might not be.
Thoughts??
