Behavior of DebugBreak differs between unmanaged a

Question

Take the following simple source (name it test.cpp):

#include <windows.h>

void main()
{
DebugBreak();
}

Compile and link this using the following commands:

cl /MD /c test.cpp
link /debug test.obj

If TEST.EXE is now run (on a 64-bit Windows 7 system), you get the following dialog:

Now add the following source file (name it test2.cpp):

void hello()
{
}

And compile and link this together with the first source, like this:

cl /MD /c       test.cpp
cl /MD /c /clr  test2.cpp
link test.obj test2.obj

Notice that we didn't even call the hello-function, we just linked it in.

Now run TEST.EXE again (on the same 64-bit Windows 7 system). Instead of the dialog shown above, you get this:

Apparently, linking in the .Net framework makes DebugBreak behave differently. Why is this? And how can I get the old DebugBreak behavior back again? Is this possibly a Windows 7 or 64-bit specific behavior?

A side-remark to make clear why I want to use DebugBreak: we have a custom assert-framework (something like the SuperAssert from John Robbin's Debugging Windows Applications book), and I use the DebugBreak function so the developer can jump into the debugger (or open a new debugger) if there is a problem. Now there is only the simple popup and no possibility to jump to the debugger anymore.

As an alternative solution I could perform a divide-by-zero or a write to invalid address, but I find this a less clean solution.

EDIT: This is the call stack in the second test (the simple dialog):

ntdll.dll!_NtRaiseHardError@24()  + 0x12 bytes  
ntdll.dll!_NtRaiseHardError@24()  + 0x12 bytes  
clrjit.dll!Compiler::compCompile()  + 0x5987 bytes  
clr.dll!RaiseFailFastExceptionOnWin7()  + 0x6b bytes    
clr.dll!WatsonLastChance()  + 0x1b8 bytes   
clr.dll!InternalUnhandledExceptionFilter_Worker()  + 0x29c bytes    
clr.dll!InitGSCookie()  + 0x70062 bytes 
clr.dll!__CorExeMain@0()  + 0x71111 bytes   
msvcr100_clr0400.dll!@_EH4_CallFilterFunc@8()  + 0x12 bytes 
msvcr100_clr0400.dll!__except_handler4_common()  + 0x7f bytes   
clr.dll!__except_handler4()  + 0x20 bytes   
ntdll.dll!ExecuteHandler2@20()  + 0x26 bytes    
ntdll.dll!ExecuteHandler@20()  + 0x24 bytes 
ntdll.dll!_KiUserExceptionDispatcher@8()  + 0xf bytes   
KernelBase.dll!_DebugBreak@0()  + 0x2 bytes 
test_mixed.exe!01031009()   

This is the call stack in the first test (dialog with choices "close" and "debug"):

ntdll.dll!_ZwWaitForMultipleObjects@20()  + 0x15 bytes  
ntdll.dll!_ZwWaitForMultipleObjects@20()  + 0x15 bytes  
kernel32.dll!_WaitForMultipleObjectsExImplementation@20()  + 0x8e bytes 
kernel32.dll!_WaitForMultipleObjects@16()  + 0x18 bytes 
kernel32.dll!_WerpReportFaultInternal@8()  + 0x124 bytes    
kernel32.dll!_WerpReportFault@8()  + 0x49 bytes 
kernel32.dll!_BasepReportFault@8()  + 0x1f bytes    
kernel32.dll!_UnhandledExceptionFilter@4()  + 0xe0 bytes    
ntdll.dll!___RtlUserThreadStart@8()  + 0x369cc bytes    
ntdll.dll!@_EH4_CallFilterFunc@8()  + 0x12 bytes    
ntdll.dll!ExecuteHandler2@20()  + 0x26 bytes    
ntdll.dll!ExecuteHandler@20()  + 0x24 bytes 
ntdll.dll!_KiUserExceptionDispatcher@8()  + 0xf bytes   
KernelBase.dll!_DebugBreak@0()  + 0x2 bytes 
test_native.exe!00af1009()  

The difference starts in ntdll.dll!Executehandler2@20. In a non-.net application it calls ntdll.dll!@_EH4_CallFilterFunc. In a .net application is calls clr.dll!__except_handler4.

Answer 1

I found the solution on the following page: http://www.codeproject.com/KB/debug/DebugBreakAnyway.aspx.

Instead of just writing DebugBreak, you have to embed the DebugBreak call between a __try/__except construction, like this:

__try
   {
   DebugBreak();
   }
__except (UnhandledExceptionFilter(GetExceptionInformation()))
   {
   }

Apparently, the UnhandledExceptionFilter function handles the DebugBreak exception by default, which seems to be overruled in a mixed-mode appliation.

Now you get the original dialog back again.