I am learning llvm and wanted to do a proof of concept of an idea I have.
Basically, I want to split my compiler and my runtime. The compiler would give a .bc and the runtime would load it via ParseBitcodeFile and use the ExecutionEngine to run it. This part is working.
Now, to make system calls easily, I want to be able to implement in my runtime C/C++ functions which do all the system calls (file io, stdout printing, etc). My question is, how could I call these functions from the code from my toy compiler, which is compiled in a different step by llvm, and allow it to be used when executed.
Good news: when using the JIT ExecutionEngine, this will just work. When the JIT-er finds an external symbol used by the IR which is not found in the IR itself, it looks in the JIT-ing process itself, so any symbols visible from your host program can be called.
This is explained directly in part 4 of the LLVM tutorial:
Whoa, how does the JIT know about sin and cos? The answer is
surprisingly simple: in this example, the JIT started execution of a
function and got to a function call. It realized that the function was
not yet JIT compiled and invoked the standard set of routines to
resolve the function. In this case, there is no body defined for the
function, so the JIT ended up calling “dlsym("sin")” on the
Kaleidoscope process itself. Since “sin” is defined within the JIT’s
address space, it simply patches up calls in the module to call the
libm version of sin directly.
For the gory details look at lib/ExecutionEngine/JIT/JIT.cpp - in particular its usage of DynamicLibrary.
Eli's answer is great and you should accept it. There's another alternative, though, which is to separately compile your runtime's source files to LLVM modules (e.g. with Clang) and use ExecutionEngine::addModule() to add them.
It's less convenient, and it means compiling the same files twice (once for your host program, another to get Modules from them), but the advantage is that it enables inlining and other cross-function optimizations from your JITted code.
