I have found charts online showing various syscalls for x86_64 linux nasm assembly and there appears to be 380ish total syscalls, however every book or tutorial I can find only "how a few of the syscalls work and what they do?" Does anyone know where I can find information on every single syscall for x86_64 linux assembly using the nasm assembler?
Any help would be great.
Look at the Linux man pages (section 2). http://man7.org/linux/man-pages/dir_section_2.html
It doesn't matter what assembler (or C compiler) you use to create x86-64 machine code, the system calls you can make are the same. (Put a call number in RAX and run the syscall instruction; inside the kernel it uses that number to index a table of function pointers. Or returns -ENOSYS if it's out of range.)
Debug your program with strace ./my_program to trace the system calls it makes. This decodes the args and return values into meaningful stuff on a per-call basis, so you can easily see if you passed a bad pointer making the syscall return -EFAULT for example. (System calls don't raise SIGSEGV / segfault, they just return an error.)
/usr/include/asm/unistd_64.h has the actual numbers. (Included by <asm/unistd.h> when compiling for 64-bit). The man pages will document the args in terms of C syntax. Given the C prototype, you can work out the asm ABI according to the x86-64 System V ABI. (Same as the function-call ABI except with R10 instead of RCX for the 4th arg, if present.) What are the calling conventions for UNIX & Linux system calls on i386 and x86-64
syscall(2) is a glibc wrapper function for system calls, and the syscall man page also documents is asm ABI for various Linux platforms (x86-64, SPARC, ARM, etc.), including registers for the call number and ret val, and the instruction for entering the kernel. Note that the function name being the same as the x86-64 syscall instruction is just a coincidence.
Nobody bothers to make exhaustive documentation for every system call for every different flavour of asm syntax - the information is all there in the man pages plus the calling convention doc; the NOTES section of the Linux man pages document differences between the C library wrapper API vs. the underlying asm system call.
See also https://blog.packagecloud.io/eng/2016/04/05/the-definitive-guide-to-linux-system-calls/ for more including VDSO stuff for efficient getpid / clock_gettime without even entering the kernel.
However, some people do compile tables of system call name and Linux x86-64 call number and arg registers. I've never found that useful (the syscall calling convention is so close to the function calling convention that it's easy to remember), but https://blog.rchapman.org/posts/Linux_System_Call_Table_for_x86_64/ is there if you want it.
Notable differences between the POSIX function and the raw Linux system call exist for a couple calls: For example brk / sbrk, and also getpriority where the "nice" level return values are biased so they're not in the -4095..-1 range of error codes. But most system calls have an ABI that exactly matches the C library wrapper prototype in which case the NOTES section doesn't mention anything.
