Solution needed for building a static IDT and GDT

Question

This question is inspired by a problem many have encountered over the years, especially in x86 operating system development. Recently a related NASM question was bumped up by an edit. In that case the person was using NASM and was getting the assemble time error:

shift operator may only be applied to scalar values

Another related question asks about a problem with GCC code when generating a static IDT at compile time that resulted in the error:

initializer element is not constant

In both cases the issue is related to the fact that an IDT entry requires an address to an exception handler and a GDT may need a base address to another structure like a Task Segment Structure (TSS). Normally this isn't an issue because the linking process can resolve these addresses through relocation fixups. In the case of an IDT entry or GDT Entry, the fields split up the base/function addresses. There are no relocation types that can tell a linker to shift bits around and then place them in memory the way they are laid out in a GDT/IDT entry. Peter Cordes has written a good explanation of that in this answer.

My question is not asking what the issue is, but a request for functional, and practical solutions to the problem. Although I am self-answering this, it is only one of many possible solutions. I only ask that solutions proposed meet these requirements:

• The GDT and IDT should not have their addresses fixed to a specific physical or linear address.
• At a minimum the solution should be able to work with ELF objects and ELF executables. If it work for other formats, even better!
• It doesn't matter if a solution is part of the process of building a final executable/binary or not. If a solution requires build time processing after the executable/binary is generated that is also acceptable.
• The GDT (or IDT) needs to appear as fully resolved when loaded in memory. Solutions must not require run-time fixups.

---

Sample Code that Doesn't Work

I'm providing some sample code in the form of a legacy bootloader¹ that tries to create a static IDT and GDT at assembly time but fails with these errors when assembled with nasm -f elf32 -o boot.o boot.asm:

boot.asm:78: error: `&' operator may only be applied to scalar values
boot.asm:78: error: `&' operator may only be applied to scalar values
boot.asm:79: error: `&' operator may only be applied to scalar values
boot.asm:79: error: `&' operator may only be applied to scalar values
boot.asm:80: error: `&' operator may only be applied to scalar values
boot.asm:80: error: `&' operator may only be applied to scalar values
boot.asm:81: error: `&' operator may only be applied to scalar values
boot.asm:81: error: `&' operator may only be applied to scalar values

The code is:

macros.inc

; Macro to build a GDT descriptor entry
%define MAKE_GDT_DESC(base, limit, access, flags) \
    (((base & 0x00FFFFFF) << 16) | \
    ((base & 0xFF000000) << 32) | \
    (limit & 0x0000FFFF) | \
    ((limit & 0x000F0000) << 32) | \
    ((access & 0xFF) << 40) | \
    ((flags & 0x0F) << 52))

; Macro to build a IDT descriptor entry
%define MAKE_IDT_DESC(offset, selector, access) \
    ((offset & 0x0000FFFF) | \
    ((offset & 0xFFFF0000) << 32) | \
    ((selector & 0x0000FFFF) << 16) | \
    ((access & 0xFF) << 40))

boot.asm:

%include "macros.inc"

PM_MODE_STACK EQU 0x10000

global _start

bits 16

_start:
    xor ax, ax
    mov ds, ax
    mov es, ax
    mov ss, ax
    mov sp, ax                  ; Stack grows down from physical address 0x00010000
                                ; SS:SP = 0x0000:0x0000 wraps to top of 64KiB segment
    cli
    cld
    lgdt [gdtr]                 ; Load our GDT
    mov eax, cr0
    or eax, 1
    mov cr0, eax                ; Set protected mode flag
    jmp CODE32_SEL:start32      ; FAR JMP to set CS

bits 32
start32:
    mov ax, DATA32_SEL          ; Setup the segment registers with data selector
    mov ds, ax
    mov es, ax
    mov ss, ax
    mov esp, PM_MODE_STACK      ; Set protected mode stack pointer

    mov fs, ax                  ; Not currently using FS and GS
    mov gs, ax

    lidt [idtr]                 ; Load our IDT

    ; Test the first 4 exception handlers
    int 0
    int 1
    int 2
    int 3

.loop:
    hlt
    jmp .loop

exc0:
    iret
exc1:
    iret
exc2:
    iret
exc3:
    iret

align 4
gdt:
    dq MAKE_GDT_DESC(0, 0, 0, 0)   ; null descriptor
.code32:
    dq MAKE_GDT_DESC(0, 0x000fffff, 10011010b, 1100b)
                                ; 32-bit code, 4kb gran, limit 0xffffffff bytes, base=0
.data32:
    dq MAKE_GDT_DESC(0, 0x000fffff, 10010010b, 1100b)
                                ; 32-bit data, 4kb gran, limit 0xffffffff bytes, base=0
.end:

CODE32_SEL equ gdt.code32 - gdt
DATA32_SEL equ gdt.data32 - gdt

align 4
gdtr:
    dw gdt.end - gdt - 1        ; limit (Size of GDT - 1)
    dd gdt                      ; base of GDT

align 4
; Create an IDT which handles the first 4 exceptions
idt:
    dq MAKE_IDT_DESC(exc0, CODE32_SEL, 10001110b)
    dq MAKE_IDT_DESC(exc1, CODE32_SEL, 10001110b)
    dq MAKE_IDT_DESC(exc2, CODE32_SEL, 10001110b)
    dq MAKE_IDT_DESC(exc3, CODE32_SEL, 10001110b)
.end:

align 4
idtr:
    dw idt.end - idt - 1        ; limit (Size of IDT - 1)
    dd idt                      ; base of IDT

---

Footnotes

• ¹I chose a bootloader as an example since a Minimal Complete Verifiable Example was easier to produce. Although the code is in a bootloader, similar code is usually written as part of a kernel or other non-bootloader code. The code may often be written in languages other than assembly, like C/C++ etc.

• Because a legacy bootloader is always loaded by the BIOS at physical address 0x7c00, there are other specific solutions for this case that can be done at assembly time. Such specific solutions break the more general use cases in OS development where a developer usually doesn't want to hard code the IDT or GDT addresses to specific linear/physical addresses, as it is preferable to let the linker do that for them.

Answer 1

One solution that I most commonly use is to actually use the GNU linker (ld) to build the IDT and GDT for me. This answer isn't a primer on writing GNU linker scripts, but it does make use of the BYTE, SHORT, and LONG linker script directives to build the IDT, the GDT, the IDT record, and the GDT record. The linker can use expressions involving <<, >>, &, | etc, and do these on the virtual memory addresses (VMA) of symbols it ultimately resolves.

The problem is that the linker scripts are rather dumb. They don't have a macro language so you'd end up having to write the IDT and GDT entries like this:

. = ALIGN(4);
gdt = .;
NULL_SEL = ABSOLUTE(. - gdt);
SHORT(0);
SHORT(0);
BYTE(0 >> 16);
BYTE(0);
BYTE((0 >> 16 & 0x0f) | (0 << 4)); BYTE(0 >> 24);

CODE32_SEL = ABSOLUTE(. - gdt);
SHORT(0x000fffff);
SHORT(0);
BYTE(0 >> 16);
BYTE(10011010b);
BYTE((0x000fffff >> 16 & 0x0f) | (1100b << 4));
BYTE(0 >> 24);

DATA32_SEL = ABSOLUTE(. - gdt);
SHORT(0x000fffff);
SHORT(0);
BYTE(0 >> 16);
BYTE(10010010b);
BYTE((0x000fffff >> 16 & 0x0f) | (1100b << 4));
BYTE(0 >> 24);
gdt_size = ABSOLUTE(. - gdt);

. = ALIGN(4);
idt = .;
SHORT(exc0 & 0x0000ffff);
SHORT(CODE32_SEL);
BYTE(0x00);
BYTE(10001110b);
SHORT(exc0 >> 16);
SHORT(exc1 & 0x0000ffff);
SHORT(CODE32_SEL);
BYTE(0x00);
BYTE(10001110b);
SHORT(exc1 >> 16);
SHORT(exc2 & 0x0000ffff);
SHORT(CODE32_SEL);
BYTE(0x00);
BYTE(10001110b);
SHORT(exc2 >> 16);
SHORT(exc3 & 0x0000ffff);
SHORT(CODE32_SEL);
BYTE(0x00);
BYTE(10001110b);
SHORT(exc3 >> 16);
idt_size = ABSOLUTE(. - idt);

exc0, exc1, exc2, and exc3 are the exception functions defined and exported from an object file. You can see the IDT entries are using CODE32_SEL for the code segment. The linker is told to compute the selector numbers when building the GDT. Obviously this is very messy and becomes more unwieldy as the GDT and most especially the IDT grow.

You could use a macro processor like m4 to simplify things, but I prefer to use the C preprocessor (cpp) as it is familiar to a lot more developers. Although the C pre-processor is usually used to pre-process C/C++ files, it isn't limited to those files. You can use it on any kind of text file including linker scripts.

You can create a macro file and define a couple macros like MAKE_IDT_DESC and MAKE_GDT_DESC to create GDT and IDT descriptor entries. I use an extension naming convention where ldh stands for (Linker Header), but you can name these files whatever you wish:

macros.ldh:

#ifndef MACROS_LDH
#define MACROS_LDH

/* Linker script C pre-processor macros */

/* Macro to build a IDT descriptor entry */
#define MAKE_IDT_DESC(offset, selector, access) \
    SHORT(offset & 0x0000ffff); \
    SHORT(selector); \
    BYTE(0x00); \
    BYTE(access); \
    SHORT(offset >> 16);

/* Macro to build a GDT descriptor entry */
#define MAKE_GDT_DESC(base, limit, access, flags) \
    SHORT(limit); \
    SHORT(base); \
    BYTE(base >> 16); \
    BYTE(access); \
    BYTE((limit >> 16 & 0x0f) | (flags << 4));\
    BYTE(base >> 24);
#endif

To cut down on the clutter in the main linker script you can create another header file that builds the GDT and IDT (and associated records):

gdtidt.ldh

#ifndef GDTIDT_LDH
#define GDTIDT_LDH

#include "macros.ldh"

/* GDT table */
. = ALIGN(4);
gdt = .;
    NULL_SEL   = ABSOLUTE(. - gdt); MAKE_GDT_DESC(0, 0, 0, 0);
    CODE32_SEL = ABSOLUTE(. - gdt); MAKE_GDT_DESC(0, 0x000fffff, 10011010b, 1100b);
    DATA32_SEL = ABSOLUTE(. - gdt); MAKE_GDT_DESC(0, 0x000fffff, 10010010b, 1100b);
    /* TSS structure tss_entry and TSS_SIZE are exported from an object file */
    TSS32_SEL  = ABSOLUTE(. - gdt); MAKE_GDT_DESC(tss_entry, TSS_SIZE - 1, \
                                                  10001001b, 0000b);
gdt_size = ABSOLUTE(. - gdt);

/* GDT record */
. = ALIGN(4);
SHORT(0);                      /* These 2 bytes align LONG(gdt) on 4 byte boundary */
gdtr = .;
    SHORT(gdt_size - 1);
    LONG(gdt);

/* IDT table */
. = ALIGN(4);
idt = .;
    MAKE_IDT_DESC(exc0, CODE32_SEL, 10001110b);
    MAKE_IDT_DESC(exc1, CODE32_SEL, 10001110b);
    MAKE_IDT_DESC(exc2, CODE32_SEL, 10001110b);
    MAKE_IDT_DESC(exc3, CODE32_SEL, 10001110b);
idt_size = ABSOLUTE(. - idt);

/* IDT record */
. = ALIGN(4);
SHORT(0);                      /* These 2 bytes align LONG(idt) on 4 byte boundary */
idtr = .;
    SHORT(idt_size - 1);
    LONG(idt);

#endif

Now you just have to include gdtidt.ldhin the linker script at a point (inside a section) that you'd like to place the structures:

link.ld.pp:

OUTPUT_FORMAT("elf32-i386");
ENTRY(_start);

REAL_BASE = 0x00007c00;

SECTIONS
{
    . = REAL_BASE;

    .text : SUBALIGN(4) {
        *(.text*);
    }

    .rodata : SUBALIGN(4) {
        *(.rodata*);
    }

    .data : SUBALIGN(4) {
        *(.data);
/* Place the IDT and GDT structures here */
#include "gdtidt.ldh"
    }

    /* Disk boot signature */
    .bootsig : AT(0x7dfe) {
        SHORT (0xaa55);
    }

    .bss : SUBALIGN(4) {
        *(COMMON);
        *(.bss)
    }

    /DISCARD/ : {
        *(.note.gnu.property)
        *(.comment);
    }
}

This linker script is a typical one I use for boot sectors, but all I've done is include the gdtidt.ldh file to allow the linker to generate the structures. The only thing left to do is pre-process the link.ld.pp file. I use the .pp extension for pre-processor files but you could use any extension. To create link.ld from link.ld.pp you can use the command:

cpp -P link.ld.pp >link.ld

The resulting link.ld file that gets generated will look like:

OUTPUT_FORMAT("elf32-i386");
ENTRY(_start);
REAL_BASE = 0x00007c00;
SECTIONS
{
    . = REAL_BASE;
    .text : SUBALIGN(4) {
        *(.text*);
    }
    .rodata : SUBALIGN(4) {
        *(.rodata*);
    }
    .data : SUBALIGN(4) {
        *(.data);
. = ALIGN(4);
gdt = .;
    NULL_SEL = ABSOLUTE(. - gdt); SHORT(0); SHORT(0); BYTE(0 >> 16); BYTE(0); BYTE((0 >> 16 & 0x0f) | (0 << 4)); BYTE(0 >> 24);;
    CODE32_SEL = ABSOLUTE(. - gdt); SHORT(0x000fffff); SHORT(0); BYTE(0 >> 16); BYTE(10011010b); BYTE((0x000fffff >> 16 & 0x0f) | (1100b << 4)); BYTE(0 >> 24);;
    DATA32_SEL = ABSOLUTE(. - gdt); SHORT(0x000fffff); SHORT(0); BYTE(0 >> 16); BYTE(10010010b); BYTE((0x000fffff >> 16 & 0x0f) | (1100b << 4)); BYTE(0 >> 24);;
    TSS32_SEL = ABSOLUTE(. - gdt); SHORT(TSS_SIZE - 1); SHORT(tss_entry); BYTE(tss_entry >> 16); BYTE(10001001b); BYTE((TSS_SIZE - 1 >> 16 & 0x0f) | (0000b << 4)); BYTE(tss_entry >> 24);;
gdt_size = ABSOLUTE(. - gdt);
. = ALIGN(4);
SHORT(0);
gdtr = .;
    SHORT(gdt_size - 1);
    LONG(gdt);
. = ALIGN(4);
idt = .;
    SHORT(exc0 & 0x0000ffff); SHORT(CODE32_SEL); BYTE(0x00); BYTE(10001110b); SHORT(exc0 >> 16);;
    SHORT(exc1 & 0x0000ffff); SHORT(CODE32_SEL); BYTE(0x00); BYTE(10001110b); SHORT(exc1 >> 16);;
    SHORT(exc2 & 0x0000ffff); SHORT(CODE32_SEL); BYTE(0x00); BYTE(10001110b); SHORT(exc2 >> 16);;
    SHORT(exc3 & 0x0000ffff); SHORT(CODE32_SEL); BYTE(0x00); BYTE(10001110b); SHORT(exc3 >> 16);;
idt_size = ABSOLUTE(. - idt);
. = ALIGN(4);
SHORT(0);
idtr = .;
    SHORT(idt_size - 1);
    LONG(idt);
    }
    .bootsig : AT(0x7dfe) {
        SHORT (0xaa55);
    }
    .bss : SUBALIGN(4) {
        *(COMMON);
        *(.bss)
    }
    /DISCARD/ : {
        *(.note.gnu.property)
        *(.comment);
    }
}

With slight modifications to the sample boot.asm file in the question we end up with:

boot.asm:

PM_MODE_STACK      EQU 0x10000 ; Protected mode stack address
RING0_STACK        EQU 0x11000 ; Stack address for transitions to ring0
TSS_IO_BITMAP_SIZE EQU 0       ; Size 0 disables IO port bitmap (no permission)

global _start
; Export the exception handler addresses so the linker can access them
global exc0
global exc1
global exc2
global exc3

; Export the TSS size and address of the TSS so the linker can access them
global TSS_SIZE
global tss_entry

; Import the IDT/GDT and selector values generated by the linker
extern idtr
extern gdtr
extern CODE32_SEL
extern DATA32_SEL
extern TSS32_SEL

bits 16

section .text
_start:
    xor ax, ax
    mov ds, ax
    mov es, ax
    mov ss, ax
    mov sp, ax                  ; Stack grows down from physical address 0x00010000
                                ; SS:SP = 0x0000:0x0000 wraps to top of 64KiB segment

    cli
    cld
    lgdt [gdtr]                 ; Load our GDT
    mov eax, cr0
    or eax, 1
    mov cr0, eax                ; Set protected mode flag
    jmp CODE32_SEL:start32      ; FAR JMP to set CS

bits 32
start32:
    mov ax, DATA32_SEL          ; Setup the segment registers with data selector
    mov ds, ax
    mov es, ax
    mov ss, ax
    mov esp, PM_MODE_STACK      ; Set protected mode stack pointer

    mov fs, ax                  ; Not currently using FS and GS
    mov gs, ax

    lidt [idtr]                 ; Load our IDT

    ; This TSS isn't used in this code since everything is running at ring 0.
    ; Loading a TSS is for demonstration purposes in this case.
    mov eax, TSS32_SEL
    ltr ax                      ; Load default TSS (used for exceptions, interrupts, etc)

    xchg bx, bx                 ; Bochs magic breakpoint

    ; Test the first 4 exception handlers
    int 0
    int 1
    int 2
    int 3

.loop:
    hlt
    jmp .loop

exc0:
    mov word [0xb8000], 0x5f << 8 | '0'   ; Print '0'
    iretd
exc1:
    mov word [0xb8002], 0x5f << 8 | '1'   ; Print '1'
    iretd
exc2:
    mov word [0xb8004], 0x5f << 8 | '2'   ; Print '2'
    iretd
exc3:
    mov word [0xb8006], 0x5f << 8 | '3'   ; Print '3'
    iretd

section .data
; Generate a functional TSS structure
ALIGN 4
tss_entry:
.back_link: dd 0
.esp0:      dd RING0_STACK     ; Kernel stack pointer used on ring0 transitions
.ss0:       dd DATA32_SEL      ; Kernel stack selector used on ring0 transitions
.esp1:      dd 0
.ss1:       dd 0
.esp2:      dd 0
.ss2:       dd 0
.cr3:       dd 0
.eip:       dd 0
.eflags:    dd 0
.eax:       dd 0
.ecx:       dd 0
.edx:       dd 0
.ebx:       dd 0
.esp:       dd 0
.ebp:       dd 0
.esi:       dd 0
.edi:       dd 0
.es:        dd 0
.cs:        dd 0
.ss:        dd 0
.ds:        dd 0
.fs:        dd 0
.gs:        dd 0
.ldt:       dd 0
.trap:      dw 0
.iomap_base:dw .iomap          ; IOPB offset
.iomap: TIMES TSS_IO_BITMAP_SIZE db 0x00
                               ; IO bitmap (IOPB) size 8192 (8*8192=65536) representing
                               ; all ports. An IO bitmap size of 0 would fault all IO
                               ; port access if IOPL < CPL (CPL=3 with v8086)
%if TSS_IO_BITMAP_SIZE > 0
.iomap_pad: db 0xff            ; Padding byte that has to be filled with 0xff
                               ; To deal with issues on some CPUs when using an IOPB
%endif
TSS_SIZE EQU $-tss_entry

The new boot.asm also creates a TSS table (tss_entry) which is used in the linker script to build the GDT entry associated with that TSS.

---

To pre-process the linker script; assemble; link; and generate a binary file that works as a boot sector, the following commands can be used:

cpp -P link.ld.pp >link.ld
nasm -f elf32 -gdwarf -o boot.o boot.asm
ld -melf_i386 -Tlink.ld -o boot.elf boot.o
objcopy -O binary boot.elf boot.bin

To run the boot.bin floppy disk image in QEMU you can use the command:

qemu-system-i386 -drive format=raw,index=0,if=floppy,file=boot.bin

To run it with BOCHS you can use the command:

bochs -qf /dev/null \
        'floppya: type=1_44, 1_44="boot.bin", status=inserted, write_protected=0' \
        'boot: floppy' \
        'magic_break: enabled=0'

The code does these things:

• Load the GDT record with the lgdt instruction.
• Processor is put into 32-bit protected with A20 disabled. All the code in the demonstration resides beneath physical address 0x100000 (1MiB) so enabling A20 isn't required.
• Loads the IDT record with lidt.
• Loads the TSS selector into the task register with ltr.
• Calls each of the exception handlers (exc0, exc1, exc2, and exc3).
• Each exception handler prints a number (0, 1, 2, 3) to the upper left corner of the display.

If it runs correctly in BOCHS the output should look like: