ld linker question: the --whole-archive option

Question

The only real use of the --whole-archive linker option that I have seen is in creating shared libraries from static ones. Recently I came across Makefile(s) which always use this option when linking with in house static libraries. This of course causes the executables to unnecessarily pull in unreferenced object code. My reaction to this was that this is plain wrong, am I missing something here ?

The second question I have has to do with something I read regarding the whole-archive option but couldn't quite parse. Something to the effect that --whole-archive option should be used while linking with a static library if the executable also links with a shared library which in turn has (in part) the same object code as the static library. That is the shared library and the static library have overlap in terms of object code. Using this option would force all symbols(regardless of use) to be resolved in the executable. This is supposed to avoid object code duplication. This is confusing, if a symbol is refereed in the program it must be resolved uniquely at link time, what is this business about duplication ? (Forgive me if this paragraph is not quite the epitome of clarity)

Thanks

Answer 1

There are legitimate uses of --whole-archive when linking executable with static libraries. One example is building C++ code, where global instances "register" themselves in their constructors (warning: untested code):

main.cc

typedef void (*handler)(const char *protocol);
typedef map<const char *, handler> M;
M m;

void register_handler(const char *protocol, handler) {
   m[protocol] = handler;
}
int main(int argc, char *argv[])
{
   for (int i = 1; i < argc-1; i+= 2) {
      M::iterator it = m.find(argv[i]);
      if (it != m.end()) it.second(argv[i+1]);
   }
}

http.cc (part of libhttp.a)

class HttpHandler {
  HttpHandler() { register_handler("http", &handle_http); }
  static void handle_http(const char *) { /* whatever */ }
};
HttpHandler h; // registers itself with main!

Note that there are no symbols in http.cc that main.cc needs. If you link this as

g++ main.cc -lhttp

you will not get an http handler linked into the main executable, and will not be able to call handle_http(). Contrast this with what happens when you link as:

g++ main.cc -Wl,--whole-archive -lhttp -Wl,--no-whole-archive

The same "self registration" style is also possible in plain-C, e.g. with the __attribute__((constructor)) GNU extension.

Answer 2

I agree that using —whole-archive to build executables is probably not what you want (due to linking in unneeded code and creating bloated software). If they had a good reason to do so they should have documented it in the build system, as now you are left to guessing.

As to your second part of the question. If an executable links both a static library and a dynamic library that has (in part) the same object code as the static library then the —whole-archive will ensure that at link time the code from the static library is preferred. This is usually what you want when you do static linking.

Answer 3

An additional good scenario in which --whole-archive is well-used is when dealing with static libraries and incremental linking.

Let us suppose that:

1. libA implements the a() and b() functions.
2. Some portion of the program has to be linked against libA only, e.g. due to some function wrapping using --wrap (a classical example is malloc)
3. libC implements the c() functions and uses a()
4. the final program uses a() and c()

Incremental linking steps could be:

ld -r -o step1.o module1.o --wrap malloc --whole-archive -lA
ld -r -o step2.o step1.o module2.o --whole-archive -lC
cc step3.o module3.o -o program

Failing to insert --whole-archive would strip function c() which is anyhow used by program, preventing the correct compilation process.

Of course, this is a particular corner case in which incremental linking must be done to avoid wrapping all calls to malloc in all modules, but is a case which is successfully supported by --whole-archive.

Answer 4

Old query, but on your first question ("Why"), I've seen --whole-archive used for in-house libraries as well, primarily to sidestep circular references between those libraries. It tends to hide poor architecture of the libraries, so I'd not recommend it. However it's a fast way of getting a quick trial working.

For your second query, if the same symbol was present in a shared object and a static library, the linker will satisfy the reference with whichever library it meets first.
If the shared library and static library have an exact sharing of code, this may all just work. But where the shared library and the static library have different implementations of the same symbols, your program will still compile but will behave differently based on the order of libraries.

Forcing all symbols to be loaded from the static library is one way of removing confusion as to what is loaded from where. But in general this sounds like solving the wrong problem; you mostly won't want the same symbols in different libraries.

Answer 5

Another legitimate use for --whole-archive is for toolkit developers to distribute libraries containing multiple features in a single static library. In this case, the provider has no idea what parts of the library will be used by the consumer and therefore must include everything.