This might be a silly question, but I was debugging a binary with gdb trying to "reverse engineer" it and reached an instruction that makes a syscall after which the effect I want to reverse engineer appears. I assume that another process is taking over and does the job so I was wondering if it was possible to debug the kernel code that handles the syscall with gdb.
Here is the x86 assembly snippet that makes the syscall (it appears that it is sys_getpid):
0x00007ffff7660d3e <+14>: movsxd rdx,edx
0x00007ffff7660d41 <+17>: movsxd rdi,edi
0x00007ffff7660d44 <+20>: mov eax,0x14
0x00007ffff7660d49 <+25>: syscall
The syscall (or sysenter or int 0x80 etc...) machine instruction is for making syscalls which by definition are handled by the Linux kernel. Details are defined in the x86-64 ABI specification. Read Advanced Linux Programming to get an overview of most of them. See also Linux Assembly HowTo.
From the point of view of a user application, a syscall is a virtual atomic instruction.
No specific userland process is handling syscalls, it is the job of the kernel to handle them, and it is nearly the sole way for an application to interact with the kernel.
the processing of syscalls by the kernel for a given process is accounted as system CPU time, e.g. by time(1)
The list of documented syscalls is given in syscalls(2). See also <asm/unistd.h> and <asm/unistd_64.h> etc... headers.
You could use strace(1) to understand the sequence of syscalls done by a particular run (of some process).
See also vdso(7).
