Yet another C# Deadlock Debugging Question

Question

I have a multi-threaded application build in C# using VS2010 Professional. It's quite a large application and we've experienced the classing GUI cross-threading and deadlock issues before, but in the past month we've noticed the appears to lock up when left idle for around 20-30 minutes.

The application is irresponsive and although it will repaint itself when other windows are dragged in front of the application and over it, the GUI still appears to be locked... interstingly (unlike if the GUI thread is being used for a considerable amount of time) the Close, Maximise and minimise buttons are also irresponsive and when clicked the little (Not Responding...) text is not displayed in the title of the application i.e. Windows still seems to think it's running fine.

If I break/pause the application using the debugger, and view the threads that are running. There are 3 threads of our managed code that are running, and a few other worker threads whom the source code cannot be displayed for.

The 3 threads that run are:

1. The main/GUI thread
2. A thread that loops indefinitely
3. A thread that loops indefinitely

If I step into threads 2 and 3, they appear to be looping correctly. They do not share locks (even with the main GUI thread) and they are not using the GUI thread at all. When stepping into the main/GUI thread however, it's broken on Application.Run...

This problem screams deadlock to me, but what I don't understand is if it's deadlock, why can't I see the line of code the main/GUI thread is hanging on?

Any help will be greatly appreciated! Let me know if you need more information...

Cheers,

Roo

-----------------------------------------------------SOLUTION--------------------------------------------------

Okay, so the problem is now solved. Thanks to everyone for their suggestions! Much appreciated! I've marked the answer that solved my initial problem of determining where on the main/UI thread the application hangs (I handn't turned off the "Enable Just My Code" option).

The overall issue I was experiencing was indeed Deadlock, however. After obtaining the call-stack and popping the top half of it into Google I came across this which explains exactly what I was experiencing...

http://timl.net/

This references a lovely guide to debugging the issue...

http://www.aaronlerch.com/blog/2008/12/15/debugging-ui/

This identified a control I was constructing off the GUI thread. I did know this, however, and was marshalling calls correctly, but what I didn't realise was that behind the scenes this Control was subscribing to an event or set of events that are triggered when e.g. a Windows session is unlocked or the screensaver exits. These calls are always made on the main/UI thread and were blocking when it saw the call was made on the incorrect thread. Kim explains in more detail here...

http://krgreenlee.blogspot.com/2007/09/onuserpreferencechanged-hang.html

In the end I found an alternative solution which did not require this Control off the main/UI thread.

That appears to have solved the problem and the application no longer hangs. I hope this helps anyone who's confronted by a similar problem.

Thanks again to everyone on here who helped! (and indirectly, the delightful bloggers I've referenced above!)

Roo

-----------------------------------------------------SOLUTION II--------------------------------------------------

Aren't threading issues delightful...you think you've solved it, and a month down the line it pops back up again. I still believe the solution above resolved an issue that would cause simillar behaviour, but we encountered the problem again.

As we spent a while debugging this, I thought I'd update this question with our (hopefully) final solution:

The problem appears to have been a bug in the Infragistics components in the WinForms 2010.1 release (no hot fixes). We had been running from around the time the freeze issue appeared (but had also added a bunch of other stuff too). After upgrading to WinForms 2010.3, we've yet to reproduce the issue (deja vu). See my question here for a bit more information: '.NET 4.0 and the dreaded OnUserPreferenceChanged Hang'. Hans has given a nice summary of the general issue.

I hope this adds a little to the suggestions/information surrounding the nutorious OnUserPreferenceChanged Hang (or whatever you'd like to call it).

Cheers,

Roo

Answer 1

I would check to make sure there is no modal dialog that is being launched behind the main form. This can happen if the dialog is launched and some event causes the main form to steal back focus. You can avoid this by making sure the modal dialog has the main form as its Owner when you create it.

Answer 2

Check where it's hanging in Application.Run; disable Just My Code in Debugger Options, then look at the call stack.

I've fixed an application that deadlocked in Application.Run because it was trying to remove an event handler from an object that a different thread was locked on. (Until the C# 4.0 compiler, event handler accessors implicitly lock(this))

However, if you're using VS2010, that isn't the exact issue.