What is the meaning of parentheses in opcodes in a

Question

When looking at a listing file that was generated by NASM, I see that there are three kinds of opcodes:

1. Without parentheses.
2. With round parentheses.
3. With square parentheses.

What they mean? when each of them is used?

This is an example of a listing file that demonstrate all of the above:

 1                                  section .text
 2                                      extern printf
 3                                      extern fgets
 4 00000000 313233                  str3:   db "123"
 5                                  main:
 6 00000003 68[00000000]                push    str1
 7 00000008 68[09000000]                push    str2
 8 0000000D 68[00000000]                push    str3
 9 00000012 E8(00000000)                call    func1
10 00000017 E8(04000000)                call    func2
11 0000001C E80B000000                  call    func3
12 00000021 E8(00000000)                call    printf
13 00000026 E8(00000000)                call    fgets
14 0000002B C3                          ret
15                                  
16 0000002C 90                      func3:  nop
17 0000002D C3                          ret
18                                  
19                                  
20                                  section .text1
21 00000000 90                      func1:  nop
22 00000001 90                          nop
23 00000002 90                          nop
24 00000003 C3                          ret
25                                  
26 00000004 90                      func2:  nop
27 00000005 90                          nop
28 00000006 90                          nop
29 00000007 C3                          ret
30                                  
31                                  
32                                  section .data
33                                  
34 00000000 313233343536373839      str1:   db "123456789"
35 00000009 313233343536373839      str2:   db "123456789"

Answer 1

The parentheses and brackets mean that the portion of the instruction bytes inside them depends on the location of the referenced object, which isn't yet fully known (may be set or changed by the linker and/or program loader in the OS).

Here, the addresses of the functions aren't known because they aren't defined in this file, so you have all zeroes in the parentheses:

12 00000021 E8(00000000)                call    printf
13 00000026 E8(00000000)                call    fgets

Here, the address of func3 relative to the address of call func3 is known (the distance is 2Ch-21h=0Bh) and isn't going to change, and so it is fixed, no parentheses or brackets:

11 0000001C E80B000000                  call    func3
12 00000021 ...
...
16 0000002C 90                      func3:  nop

Here, the address of str2 is known partially at the time of assemblying as it may change at link time because it depends on the distance between sections, and so there are brackets:

 7 00000008 68[09000000]                push    str2

Offhand I can't tell the difference between the parentheses and brackets, perhaps it's code vs data. NASM documentation is probably a good place to check it. If it's not described there, there's NASM source code available and you can do more "tests" too.

Answer 2

They're showing where relocations will be applied at link time.

[nnnnnnnn] shows an absolute relocation (when the linker performs the relocation, the base address of some section will be added to the offset). e.g. once the binary is fully linked, those bytes in the push str2 instruction will be changed to contain the base address of .data + 0x09.

(nnnnnnnn) shows a PC-relative relocation (used for calls and branches, where the final value must be relative to the address of the next instruction). e.g. the bytes in the call func2 instruction will be updated with the difference between the final address of func2, and the address of the following instruction (call func3).

There are no brackets for the call func3 instruction itself because no relocation is needed in that case - the call instruction needs a relative address, but func3 is within the same section, so the relative address is already known (the following instruction is at .text + 0x21, and func3 is at .text + 0x2c, so the relative address is 0xb regardless of the final address of .text).