I have always found this to be a very useful feature in Visual Studio. For those who don't know about it, it allows you to edit code while you are debugging a running process, re-compile the code while the binary is still running and continue using the application seamlessly with the new code, without the need to restart it.
How is this feature implemented? If the code I am modifying is in a DLL loaded by the application, does the application simply unload the DLL and reload it again? This would seem to me like it would be prone to instability issues, so I assume it would be smarter than this. Any ideas?
My guess is all objects are aligned to a 4096 byte memory boundary. So if you make small changes to some code then the objects will still be within those boundaries and therefore run as before.
I've had instances where changing a couple of lines will cause a full recompile and link and others where a fairly substantial refactoring of a function will e&c just fine.
My understanding is that when the app is compiled with support for Edit and Continue enabled, the compiler leaves extra room around the functions in the binary image to allow for adding additional code. Then the debugger can compile a new version of the function, replace the existing version (using the padding space as necessary), fix up the stack, set the instruction pointer, and keep going. That way you don't have to fix up any jump pointers, as long as you have enough padding.
Note that Edit and Continue doesn't usually work on code in libs/dlls, only with the main executable code.
My guess is that it recompiles the app (and for small changes this wouldn't mean very much would have to be recompiled). Then since Microsoft makes both the compiler and debugger they can make guarantees about how memory and the like are laid out. So, they can use the debugging API to re-write the code segments with the new ones as long as the changes are small enough.
If the changes redirect to entirely new code, this can obviously be loaded into memory in a similar style as DLLs.
Microsoft also has a mechanism for "hot-patching". Functions have a 2 byte no-op instruction usually something like "mov edx, edx" before any real code. This allows them to redirect the execution of a function cleanly. This may be an option as well.
The key thing to remember is that the application isn't "running", all it's threads are in the stopped state. So as far as the process is concerned any modifications the debugger makes are entirely atomic.
Of course, this is all speculation ;)