What are the calling conventions for UNIX & Linux

Question

Following links explain x86-32 system call conventions for both UNIX (BSD flavor) & Linux:

• http://www.int80h.org/bsdasm/#system-calls

• http://www.freebsd.org/doc/en/books/developers-handbook/x86-system-calls.html

But what are the x86-64 system call conventions on both UNIX & Linux?

Answer 1

In addition to the link that Jonathan Leffler provides in his answer, also Agner Fog's Calling Conventions pdf may be useful to you.

Answer 2

Perhaps you're looking for the x86_64 ABI?

• www.x86-64.org/documentation/abi.pdf (404 at 2018-11-24)
• www.x86-64.org/documentation/abi.pdf (via Wayback Machine at 2018-11-24)
• Where is the x86-64 System V ABI documented? - https://github.com/hjl-tools/x86-psABI/wiki/X86-psABI is kept up to date (by HJ Lu, one of the ABI maintainers) with links to PDFs of the official current version.

If that's not precisely what you're after, use 'x86_64 abi' in your preferred search engine to find alternative references.

Answer 3

Calling conventions defines how parameters are passed in the registers when calling or being called by other program. And the best source of these convention is in the form of ABI standards defined for each these hardware. For ease of compilation, the same ABI is also used by userspace and kernel program. Linux/Freebsd follow the same ABI for x86-64 and another set for 32-bit. But x86-64 ABI for Windows is different from Linux/FreeBSD. And generally ABI does not differentiate system call vs normal "functions calls". Ie, here is a particular example of x86_64 calling conventions and it is the same for both Linux userspace and kernel: http://eli.thegreenplace.net/2011/09/06/stack-frame-layout-on-x86-64/ (note the sequence a,b,c,d,e,f of parameters):

Performance is one of the reasons for these ABI (eg, passing parameters via registers instead of saving into memory stacks)

For ARM there is various ABI:

http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.subset.swdev.abi/index.html

https://developer.apple.com/library/ios/documentation/Xcode/Conceptual/iPhoneOSABIReference/iPhoneOSABIReference.pdf

ARM64 convention:

http://infocenter.arm.com/help/topic/com.arm.doc.ihi0055b/IHI0055B_aapcs64.pdf

For Linux on PowerPC:

http://refspecs.freestandards.org/elf/elfspec_ppc.pdf

http://www.0x04.net/doc/elf/psABI-ppc64.pdf

And for embedded there is the PPC EABI:

http://www.freescale.com/files/32bit/doc/app_note/PPCEABI.pdf

This document is good overview of all the different conventions:

http://www.agner.org/optimize/calling_conventions.pdf

Answer 4

I verified these using GNU Assembler (gas) on Linux.

Kernel Interface

x86-32 Linux System Call convention:

In x86-32 parameters for Linux system call are passed using registers. %eax for syscall_number. %ebx, %ecx, %edx, %esi, %edi, %ebp are used for passing 6 parameters to system calls.

The return value is in %eax. All other registers (including EFLAGS) are preserved across the int $0x80.

I took following snippet from the Linux Assembly Tutorial but I'm doubtful about this. If any one can show an example, it would be great.

If there are more than six arguments, %ebx must contain the memory location where the list of arguments is stored - but don't worry about this because it's unlikely that you'll use a syscall with more than six arguments.

For an example and a little more reading, refer to http://www.int80h.org/bsdasm/#alternate-calling-convention

There is faster way to make 32bit system calls: using sysenter. The kernel maps a page of memory into every process (the vdso), with the user-space side of the sysenter, which has to cooperate with the kernel for it to be able to find the return address. arg to register mapping is the same as for int $0x80, but instead of that instruction, code should call a function in the vdso. (TODO: update this with a link and/or specific info).

x86-32 [Free|Open|Net|DragonFly]BSD UNIX System Call convention:

Parameters are passed on the stack. Push the parameters (last parameter pushed first) on to the stack. Then push an additional 32-bit of dummy data (Its not actually dummy data. refer to following link for more info) and then give a system call instruction int $0x80

http://www.int80h.org/bsdasm/#default-calling-convention

---

x86-64 Linux System Call convention:

x86-64 Mac OS X is similar but different. TODO: check what *BSD does.

Refer to section: "A.2 AMD64 Linux Kernel Conventions" of System V Application Binary Interface AMD64 Architecture Processor Supplement. The latest versions of the i386 and x86-64 System V psABIs can be found linked from this page in the ABI maintainer's repo. (See also the x86 tag wiki for up-to-date ABI links and lots of other good stuff about x86 asm.)

Here is the snippet from this section:

1. User-level applications use as integer registers for passing the sequence %rdi, %rsi, %rdx, %rcx, %r8 and %r9. The kernel interface uses %rdi, %rsi, %rdx, %r10, %r8 and %r9.
2. A system-call is done via the syscall instruction. This clobbers %rcx and %r11, as well as %rax, but other registers are preserved.
3. The number of the syscall has to be passed in register %rax.
4. System-calls are limited to six arguments, no argument is passed directly on the stack.
5. Returning from the syscall, register %rax contains the result of the system-call. A value in the range between -4095 and -1 indicates an error, it is -errno.
6. Only values of class INTEGER or class MEMORY are passed to the kernel.

Remember this is from the Linux-specific appendix to the ABI, and even for Linux it's informative not normative. (But it is in fact accurate.)

User Interface

x86-32 Function Calling convention:

In x86-32 parameters were passed on stack. Last parameter was pushed first on to the stack until all parameters are done and then call instruction was executed. This is used for calling C library (libc) functions on Linux from assembly.

---

x86-64 Function Calling convention:

x86-64 passes args in registers, which is more efficient than i386 System V's stack args convention. It avoids the latency and extra instructions of storing args to memory (cache) and then loading them back again in the callee. This works well because there are more registers available, and is better for modern high-performance CPUs where latency and out-of-order execution matter. (The i386 ABI is very old).

In this new mechanism: First the parameters are divided into classes. The class of each parameter determines the manner in which it is passed to the called function.

For complete information refer to : "3.2 Function Calling Sequence" of System V Application Binary Interface AMD64 Architecture Processor Supplement which reads, in part:

Once arguments are classified, the registers get assigned (in left-to-right order) for passing as follows:

1. If the class is MEMORY, pass the argument on the stack.
2. If the class is INTEGER, the next available register of the sequence %rdi, %rsi, %rdx, %rcx, %r8 and %r9 is used

So %rdi, %rsi, %rdx, %rcx, %r8 and %r9 are the registers in order used to pass integer/pointer (i.e. INTEGER class) parameters to any libc function from assembly. %rdi is used for the first INTEGER parameter. %rsi for 2nd, %rdx for 3rd and so on. Then call instruction should be given. The stack (%rsp) must be 16B-aligned when call executes.

If there are more than 6 INTEGER parameters, the 7th INTEGER parameter and later are passed on the stack. (Caller pops, same as x86-32.)

The first 8 floating point args are passed in %xmm0-7, later on the stack. There are no call-preserved vector registers. (A function with a mix of FP and integer arguments can have more than 8 total register arguments.)

Variadic functions (like printf) always need %al = the number of FP register args.

There are rules for when to pack structs into registers (rdx:rax on return) vs. in memory. See the ABI for details, and check compiler output to make sure your code agrees with compilers about how something should be passed/returned.