Stepping into a P/Invoke call in disassemby view

Question

My C# code is calling an unmanaged third-party library function via P/Invoke, and the unmanaged function is having some strange side effects. I want to debug into it and see what it's doing.

If I debug my C# code, and try to "Step Into" the P/Invoke call, it steps over instead. No surprise there -- I expected that; it doesn't have the source for this DLL, and I didn't tell it I was okay with seeing the disassembly view.

So I switch the debugger to disassembly view (Debug > Windows > Disassembly). Now I see the individual x86 instructions in my JITted code. Again I try to step into the P/Invoke call. Again, it steps over instead -- even though I clearly told it to Step Into an x86 CALL instruction. How hard is it to step into an x86 CALL?

My Googling thus far has shown me a couple of options that can affect this, and I've already set them:

• In Tools > Options > Debugging > General, "Enable Just My Code" is unchecked.
• In Project > Properties > Debug tab, "Enable unmanaged code debugging" is checked.

No good. Visual Studio still refuses to step in.

I don't have a PDB for the third-party DLL, but that shouldn't matter. I don't care about source code or symbol information. (Well, actually they'd be really nice, but I already know I'm not going to get them.) Visual Studio can do x86 debugging (that's what the Disassembly view is for), and all I want to do is step into the x86 code.

What else do I need to do to get VS to allow me to step into the x86 instructions inside a P/Invoke call?

Answer 1

This may help you solve the problem: (by Graviton)

CallingConvention = CallingConvention.Cdecl

Also this mentions that you need to detach managed debugger and re-attach unmanaged when crossing the boundaries. You might need to check the capabilities of mixed debugger and it's preferences from MSDN.

And finally, using Ed Dore' answer:

Under Tools.Options dialog, select the Debugging category, and make sure the "Enable Just My Code" setting is unchecked. From the Project properties, select the Debug tab, and then ensure that "Enable unmanaged code debugging" is checked.

Once you've got these squared away, you should get the mixed mode debugging support working.

Also, if you use "Debug.Attach To Process" , be sure to hit the "Select..." button in the "Attach To Process" dialog, and select both Managed and Native debugging support.

Answer 2

One thing I would try is going from C# to C++/CLI code, and then from C++ to the third-party code. Once you're in C++ (and free of the P/Invoke infrastructure), you might have better luck with the disassembly view.

Answer 3

I had a similar issue where I was debugging a C# exe that called my own C++ dll via PInvoke, all part of the same solution. Enabling native code debugging in my c# project allowed me to debug my C++ code.

Answer 4

In your C# project properties, in the Debug tab, check Enable native code debugging. Worked for me in VS 2012.

Credit goes to billb.

Also, since it's a third party library, ensure Enable Just My Code is unchecked in Options > Debugging.