Intercept ELF loader in linux kernel: fs/binfmt_el

I am new to kernel coding and at present I am working with ELF files which have been modified a little bit for the security purposes for which I need to look at some of it's custom section headers and extract the unique code encryption key from it for the CPU to decrypt the contents of the modified ELF.

At present the above logic has been implemented within the load_elf_binary function in the fs/binfmt_elf.c file in the kernel source tree, however it is only about 250 lines of code change for which I need to recompile the whole kernel, so I am looking to improvise this functionality by implementing it as a loadable kernel module(LKM) so that every time an ELF is loaded it checks whether its the modified ELF or not and if it is then it extacts the key from the corresponding section.

EDIT: To summarize it, I am looking at making a loadable kernel module to read through the sections of an ELF and fetch the contents of a custom section that contains the encryption key and related metadata and set those values in CPU registers.

Yes, it's possible, but definitely not easy. There is even a supported kernel facility "kprobes" that allows you to insert calls to your own code from specified locations (see Documentation/kprobes.txt). If inserting calls to your own code is insufficient, I think you would need to use the same sort of mechanisms as kprobe: patching the desired location by overwriting instructions with jmps or calls into your own code.

I once worked at a company whose security product installed its hooks by runtime-patching the Windows kernel. This is pretty much the same thing, though at least with Windows at the time there were a finite number of versions that had to be supported.

So, it's definitely possible, but I wouldn't want to try it. It will be very brittle; you'll be in effect trying to hit a moving target. Every kernel security patch or version upgrade is likely to break your code.