ptrace in solaris

Question

I'm trying to port a program that uses ptrace from linux to solaris, but no luck, as it complains that sys/ptrace.h is not found. Any idea how to port it?

Answer 1

EDIT: added and removed stale info on how to contact me and how I would license the code to you

I ended up using /proc/<pid>/ctrl and the other various interfaces in /proc/<pid> to write my own library for doing ptrace() like things. Unfortunately, that interface (at least at the time) was marked as not safe to use directly, could change at any time etc. but it seemed stable in practice. No idea if it has since changed ,this was circa 2011 or so I think...

To develop the basic ptrace equivalent functionality, start with:

man 4 proc

Using the /proc/<pid>/ctrl interface, you can do just about everything (and more) than you can with ptrace such as read/write memory/registers, attach/detach, set syscall breakpoints, etc, etc. With not too much work, you can write your own ptrace emulation API. These are the prototypes for the low-level part of the API I wrote:

procfs_ctl_PCSFAULT (pid_t pid, fltset_t * fltset);
procfs_ctl_PCWATCH (pid_t pid, prwatch_t * prwatch);
procfs_ctl_PCSTRACE (pid_t pid, sigset_t * sigset);
procfs_ctl_PCSEXIT (pid_t pid, sysset_t * sysset);
procfs_ctl_PCSENTRY (pid_t pid, sysset_t * sysset);
procfs_ctl_PCWSTOP (pid_t pid);
procfs_ctl_PCSVADDR (pid_t pid, long vaddr);
procfs_ctl_PCSTOP (pid_t pid);
procfs_ctl_PCWRITE (pid_t pid, off_t off, void *buff, size_t count);
procfs_ctl_PCTWSTOP (pid_t pid, long delay);
procfs_ctl_PCSCRED (pid_t pid, prcred_t * cred);
procfs_ctl_PCREAD (pid_t pid, off_t off, void *buff, size_t count);
procfs_ctl_PCSSIG (pid_t pid);
procfs_ctl_PCRUN (pid_t pid, long flags);
procfs_ctl_PCDSTOP (pid_t pid);
procfs_ctl_PCCSIG (pid_t pid);
procfs_ctl_PCCFAULT (pid_t pid);
procfs_ctl_PCKILL (pid_t pid, long signal);
procfs_ctl_PCSREG (pid_t pid, long lwpid, prgregset_t * regs);

You can probably see how easily it would be to then write a compatibility layer for ptrace on top of these functions. The book Solaris Internals was really useful while doing this- while the chapter on the proc interface was pretty much a verbatim copy of the man pages, it was nice to have to flip through quickly.

Ultimately, I ended up not producing a ptrace compatible API- I skipped that step and implemented functions that performed much higher level functions- an example of the high level code that used these lower level /proc/<pid>/ctrl based functions, here is a list of some example high-level functions I implemented based on these lower-level functions that demonstrates all of the building blocks they provide (pretty much everything, as I said)

NOTE: These were designed for a specific program that needed to do exotic/dangerous/unsupported things to a running binary (without symbols) like find string and address references, locate and call existing functions, inject and run position independent code, hook system calls and modify arguments and read return values, etc. so there are quite a few weird functions here.

Also, the ones involving searching for references to strings and pointers are not very well written- they were just written so that they would work on the application I was working with. You should really use something like a real binary analysis library for things like that, but this just demonstrates how powerful the proc interface is. This was all for Solaris 9/10 SPARC. Examples:

TrapPostSyscall (pid, SYS_dup);
SingleStep(pid,lwpid,flags);
AttachProcess (pid);
ContinueProcess (pid, PRCFAULT | PRCSIG | PRSABORT);
DetachProcess (pid);
GetFaultAddress(pid,1);
GetMainDataSection (pid, &datamap);
WhyStopped (pid, 1));
PrintRegisters(regs);
ReadGeneralRegisterSet (pid, 1, &saved_regs);
SetProgramCounter (pid, 1, mapping - 4);
StopProcess (pid);
TraceFault (pid, FLTWATCH);
TrapPostSyscall (pid, SYS_dup);
TrapPostSyscall (pid, SYS_execve);
TrapSyscallClearall (pid);
WaitForProcessStop (pid);
WatchMemory (pid, datamap.pr_vaddr, datamap.pr_size,WA_TRAPAFTER | WA_WRITE);
WriteGeneralRegisterSet (pid, 1, &saved_regs);
WriteToProcess (pid, mapping, call_pic, sizeof (call_pic));
ReadFromProcess (pid, prmap.pr_vaddr, buff, size);
GetIndividualRegister (pid, 1, R_O0, (long *) fd);
SetIndividualRegister (pid,1,R_O0,fd);
IsProcessStopped (pid);
SearchProcessForReference (pid, stringat);
SearchProcessMem (pid,needle,strlen(needle)+1);
GetPreviousSave (pid, SearchProcessForReference (pid, stringat));
GetMapFromVirtualAddress (pid, base, &prmap);
GetNextProcessMapping (pid, &prmap);
SearchProcessWordRef (pid, addr);
GetReadWriteExecMapping (pid);
CreateRemoteMapping (pid, NULL, PAGESIZE)
TrapSyscall (pid, SYS_dup);
TrapSyscallClearall(pid);
TrapPostSyscall (pid, SYS_dup);
TrapPostSyscallClearall (pid);

If you have interest in this, I can GPL the low-level or high-level stuff and you can very easily produce ptrace compatible wrappers around them. Send me a note and I'll share under the GPLv2

Answer 2

At least on the solaris system I have access to, man ptrace says to include

#include <unistd.h>
#include <sys/types.h>

for access to the ptrace prototype and constants. However, there is a usage note that states that ptrace is available only with the 32-bit libc, and that 64 bit clients should use the /proc debugging interfaces instead, so I'm not sure how far this will get you.

Answer 3

Some, such as Solaris, have removed ptrace as a system call altogether, retaining it as a library call that reinterprets calls to ptrace in terms of the platform's procfs

http://en.wikipedia.org/wiki/Ptrace

Also, it seems that strace utility is not avaliable on Solaris, instead they have one that is called truss, see if you have that on your system.