Debugging Visual Studio 2010 DLL Project

Question

I'm trying to debug a C/C++ native DLL project from Visual Studio 2010. I'm attempting to follow these instructions: http://msdn.microsoft.com/en-us/library/c91k1xcf(v=VS.100).aspx

I want to use the built-in debugger and be able to step code, examine structures, etc. as I would do with a regular .exe project. The instructions on the page above describe a Debugging category under Configuration Properties which I do not see.

http://img707.imageshack.us/img707/4402/lalasz.png

Just pressing F5 to debug results in the following error:

Unable to start program 'C:\Users.......Test.dll'

I've used the debugger for regular .exe projects many times and it works fine on this computer. I'm not sure if I'm just missing something very obvious right now though.

Edit: Since I didn't make it clear from the start, I want Visual Studio to LoadLibrary my DLL into a stub process and let me debug at a source level from there, much like how OllyDbg does it.

My DLL is not the type that holds a bunch of functions to be exported and called. Instead it does a switch/case in the DllMain and on DLL_PROCESS_ATTACH will spawn a new thread. Therefore all I need Visual Studio to do is to load my DLL into a stub executable and allow me to set breakpoints, etc.

Answer 1

You probably have as the startup project the one that produces the dll.

You have two choices: either change the startup project to another project that produces an executable that uses that DLL, or configure from project properties the debug settings for the dll project to start an external application that uses that dll (Project Properties/Debug/Command).

Answer 2

Native DLLs cannot be run standalone - they must run in the context of some program. See this part of the instructions page you referenced.

"If you start debugging from the project that creates the DLL, you must specify the executable you want to use in debugging the DLL."

Answer 3

You would run into this problem from a Managed project also. What Visual Studio is telling you is that it cannot run the DLL, just the same as you cannot double-click a DLL from Explorer, and have a program run.

In order to debug the DLL, write a small console application which calls functions from your DLL and exercises your code. If your DLL has a function foo(), call foo() from main in your console application. Set the console application as the "Startup" project, by right clicking the project name in the Solution Explorer and selecting its option.

Then, when you press F5, you will run the console application, which will call the DLL.

Answer 4

You right-clicked the solution name in the Solution Explorer window and got the solution properties. Note that the window says "Solution Test Property Pages".

Right-click the project name (Test in bold) instead to set the project options.