I'm using valgrind/callgrind to profile my server code for some optimization. The two most used calls that callgrind is reporting to me (using kcachegrind to view) are _dl_lookup_symbol_x and do_lookup_x. However I have no idea what either of these are and can't seem to find any documentation about them.
Could anyone please tell me where these two functions are used and what they do?
_dl_lookup_symbol_x
is an internal function inside the glibc C runtime library. If you browse the source for glibc, you'll find this comment above the_dl_lookup_symbol_x
definition:do_lookup_x
is merely a helper function called within the_dl_lookup_symbol_x
function.I'm no expert on the internals of glibc, but from what I can gather,
_dl_lookup_symbol_x
looks for a symbol (such as a function) inside shared libraries loaded by your program.I don't know why these functions are called so often in your profiling, but at least now you have some clue as to what they do. Your profiling should tell you what functions are responsible for calling
_dl_lookup_symbol_x
so often.Note that it would be normal for
_dl_lookup_symbol_x
to be called many times when the program first starts, as the runtime figures out the addresses of shared library functions with a given name. If you're profiling a very short-lived program, then it's not surprising that you'd see that most of the time is spent in internal "housekeeping" functions rather than your own code.dl_lookup_symbol is called from _dl_fixup which is called from _dl_runtime_resolve_sse
In gdb if you break on the first function call, any call, then in lazy linking, which is default when using gcc, the function's address will not actually be resolved until after that call pretty much returns to main. One of the functions that the process of uses to resolve the address of the function is dl_lookup_symbol, so you should expect to see it once for each function that is called by every program that uses lazy linking, pretty much all of them.
Source code:
Going to gdb:
The following, after stepping into printf, is where lazy linking's magic results show up, after it's done. Here I'm just showing that as I step through this, the program walks through, but once printf is resolved, if I stepi at
the program will go straight to printf. So, next time the function is called jmp QWORD PTR [rip+0x200a72] will take us straight to printf. All code after this point and until the function returns to main is just the resolving of printf's address and only happens the first time that the function is called.
Stepping in. Not sure why gdb doesn't pick these up, they are just instructions in the .plt table. And yes the addresses aren't the same, my system adds 0x555555554000 to everything before we even hit _start
, proof:
From objdump:
gdb
Hey, _dl_runtime_resolve_sse, we're getting closer.The following might look ugly. It's not. It's just saving registers so that the call
0x00007ffff7def11a <+154>: call 0x7ffff7de7970 <_dl_fixup>
halfway through can use whatever registers it needs to, and that's truly where the magic happens. At the end 0x00007ffff7def1b0 <+304>: bnd jmp r11 will take us to printf just this one time that the program goes through this code for this particular function, because it's the only time that the program goes through this code for this particular function.that bnd jmp r11 at the end will take us to printf.
Lastly we step into _dl_fixup, where I singled out the call to _dl_lookup_symbol
This is as far as this journey goes. the dl_lookup_symbol function goes to a land of parsing elf files and working with symbols, that's where the pfm happens.