Detecting, and Shirking Off, the Debugger

问题:

An interesting feature in the new iTunes is it's inability to accept debugger processes that are attached to it (crippling tools like F-Script) Not only would this involve a detection method, but it would require some kind of process that was either checking for the debugger attaching itself mid-run, or an entry-point method that the debugger would emit when it attempts to attach itself. In addition, it would need a way to tell the debugger to go away (as it were) without terminating the process. The question is: How? Clearly, polling for a debugger every X number of seconds is inefficient, and not allowing it to attach to a given process (sans override like ptrace()) seems intensely private.

回答1:

iTunes is calling ptrace(PT_DENY_ATTACH) which sets the P_LNOATTACH flag which stops debuggers (and other processes, e.g. F-Script and DTrace) from attaching to the process.

See Is it possible to conceal a OS X app from DTrace? for more information.

I wouldn't be surprised if iTunes is also actively using detection methods to identify debuggers. Apple have gone to great lengths to try to protect the DRM in iTunes.

There are a number of books that have methods of securing Cocoa applications, including detecting debuggers. Some potential titles that spring to mind (I haven't double checked the contents of these so don't assume they have detection methods): "Mac Hacker's Handbook", "Hacking and Securing iOS Applications", "Professional Cocoa Application Security" and "Secure Programming Cookbook for C & C++".

"Mac OS X Internals" and "Mac OS X and iOS Internals" might have something on PT_DENY_ATTACH.