I've implemented a specialised tree data structure for a ray tracing application. I'm using an std::list in each of the tree's nodes to store data items. I have a problem where running the application, with this data structure in use, crashes not only the application, but freezes the whole OS (Windows 7). This occurs in release mode only; in debug mode it takes longer (as is expected) but works fine. I'm using Visual Studio 2010. Running (Ctrl+F5) or debugging (F5) in release mode both caused the full system crash.
From what I've found so far, different behaviour between release and debug mode seems to be attributed to memory leaks, as debug mode (from what I've understood) is a little more forgiving. However I've found nothing about a C++ application crashing the OS.
I'd just like to know under what circumstances such behaviour can occur, so I know where to look. It's difficult to reduce the problem to something simple and trace the issue, because it's in the nature of ray tracing to be highly parallel and work with a lot of data.
Oh, and the problem is not an infinite recursion causing a stack overflow. I made that happen intentionally and it did not crash the OS.