What is your favourite anti-debugging trick?

Question

At my previous employer we used a third party component which basically was just a DLL and a header file. That particular module handled printing in Win32. However, the company that made the component went bankcrupt so I couldn't report a bug I'd found.

So I decided to fix the bug myself and launched the debugger. I was surprised to find anti-debugging code almost everywhere, the usual IsDebuggerPresent, but the thing that caught my attention was this:

    ; some twiddling with xor 
    ; and data, result in eax 
    jmp eax 
    mov eax, 0x310fac09 
    ; rest of code here 

At the first glance I just stepped over the routine which was called twice, then things just went bananas. After a while I realized that the bit twiddling result was always the same, i.e. the jmp eax always jumped right into the mov eax, 0x310fac09 instruction. I dissected the bytes and there it was, 0f31, the rdtsc instruction which was used to measure the time spent between some calls in the DLL.

So my question to SO is: What is your favourite anti-debugging trick?

Answer 1

My personal favourite was on the Amiga, where there is a coprocessor (the Blitter) doing large data transfers independent from the processor; this chip would be instructed to clear all memory, and reset from a timer IRQ.

When you attached an Action Replay cartridge, stopping the CPU would mean that the Blitter would continue clearing the memory.

Answer 2

Using nop to remove assembly via the debugger is a useful trick. Of course, putting the code back is a lot harder!!!

Answer 3

I second the virtual machine suggestion. I implemented a MIPS I simulator that (now) can execute binaries generated with mipsel-elf-gcc. Add to that code/data encryption capabilities (AES or with any other algorithm of your choice), the ability of self-simulation (so you can have nested simulators) and you have a pretty good code obfuscator.

The nice feature of choosing MIPS I is that 1) it's easy to implement, 2) I can write code in C, debug it on my desktop and just cross-compile it for MIPS when it's done. No need to debug custom opcodes or manually write code for a custom VM..

Answer 4

I would prefer that people write software that is solid, reliable and does what it is advertised to do. That they also sell it for a reasonable price with a reasonable license.

I know that I have wasted way too much time dealing with vendors that have complicated licensing schemes that only cause problems for the customers and the vendors. It is always my recommendation to avoid those vendors. Working at a nuclear power plant we are forced to use certain vendors products and thus are forced to have to deal with their licensing schemes. I wish there was a way to get back the time that I have personally wasted dealing with their failed attempts to give us a working licensed product. It seems like a small thing to ask, but yet it seems to be a difficult thing for people that get too tricky for their own good.