I'm having trouble finding the good documentation for writing 64-bit assembly on MacOS.
The 64-bit SysV ABI says the following in section A.2.1 and this SO post quotes it:
A system-call is done via the syscall instruction. The kernel destroys registers %rcx and %r11.
Returning from the syscall, register %rax contains the result of the system-call. A value in the range between -4095 and -1 indicates an error, it is -errno.
Those two sentences are ok on Linux but are wrong on macOS Sierra with the following code:
global _start
extern _exit
section .text
_start:
; Align stack to 16 bytes for libc
and rsp, 0xFFFFFFFFFFFFFFF0
; Call write
mov rdx, 12 ; size
mov rsi, hello ; buf
mov edi, 1 ; fd
mov rax, 0x2000004 ; write ; replace to mov rax, 0x1 on linux
syscall
jc .err ; Jumps on error on macOS, but why?
jnc .ok
.err:
mov rdi, -1
call _exit ; exit(-1)
.ok:
; Expect rdx to be 12, but it isn't on macOS!
mov rdi, rdx
call _exit ; exit(rdx)
; String for write
section .data
hello:
.str db `Hello world\n`
.len equ $-hello.str
Compile with NASM:
; MacOS: nasm -f macho64 syscall.asm && ld syscall.o -lc -macosx_version_min 10.12 -e _start -o syscall
; Linux: nasm -f elf64 syscall.asm -o syscall.o && ld syscall.o -lc -dynamic-linker /lib64/ld-linux-x86-64.so.2 -o syscall
Run on macOS:
./syscall # Return value 0
./syscall >&- # Return value 255 (-1)
I found out that:
- A syscall return
errno
an sets the carry flag on error, instead of returning-errno
inrax
rdx
register is clobbered bysyscall
- On Linux, everything works as expected
Why is rdx
clobbered? Why doesn't a syscall return -errno
? Where can I find the real documentation?
The only place I found where someone talks about the carry flag for syscall errors is here