Why isn't my new operator called

Question

I wanted to see that a dynamically loaded library (loaded with dlopen etc.) really uses its own new an delete operators and not these ones defined in the calling program. So I wrote the following library.cpp

#include <exception>
#include <new>
#include <cstdlib>
#include <cstdio>
#include "base.hpp"
void* operator new(size_t size) {
    std::printf("New of library called\n");
    void *p=std::malloc(size); 
    if (p == 0) // did malloc succeed?
        throw std::bad_alloc(); // ANSI/ISO compliant behavior
    return p;
}
void operator delete(void* p) {
    std::printf("Delete of library called\n");
    std::free(p);
}
class Derived : public Base {
public:
    Derived() : Base(10) { }
};
extern "C" {
    Base* create() {
        return new Derived;
    }
    void destroy(Base* p) {
        delete p;
    }
}

and compiled it with

g++ -g -Wall -fPIC -shared library.cpp -o library.so

or as Employed Russian suggested to try (but in the end nothing changed)

g++ -g -Wall -fPIC -shared -Wl,-Bsymbolic library.cpp -o library.so

The class Base is only holding an int value and a function get_value() to get this value. After that I wrote client.cpp like this

#include <exception>
#include <new>
#include <iostream>
#include <cstdlib>
#include <cstdio>
#include <dlfcn.h>
#include "base.hpp"
void* operator new(size_t size) {
    std::printf("New of client called\n");
    void *p=std::malloc(size); 
    if (p == 0) // did malloc succeed?
        throw std::bad_alloc(); // ANSI/ISO compliant behavior
    return p;
}
void operator delete(void* p) {
    std::printf("Delete of client called\n");
    std::free(p);
}
typedef Base* create_module_t();
typedef void destroy_module_t(Base *);

int main() {
    void* handle = dlopen("./library.so", 
        RTLD_LAZY);
    if (handle == NULL) {
        std::cout << dlerror() << std::endl;
        return 1;
    }
    create_module_t* create_module = NULL;
    void* func = dlsym(handle, "create");
    if (func == NULL) {
        std::cout << dlerror() << std::endl;
        return 1;
    } else create_module = (create_module_t *)func;
    destroy_module_t* destroy_module = NULL;
    func = dlsym(handle, "destroy");
    if (func == NULL) {
        std::cout << dlerror() << std::endl;
        return 1;
    } else destroy_module = (destroy_module_t *)func;
    Base* a = create_module();
    std::cout << "Value: " << a->get_value() << std::endl;
    destroy_module(a);
    return 0;
}

and compiled it with

g++ -Wall -g -o client -ldl client.cpp

Executing client I only get a "New of client called" and a "Delete of client called". Even if I use the compiler switch -Bsymbolic for the library like Employed Russian suggested.

Now: What went wrong? I thought shared library are using their own new/delete and therefore you have to provide next to the factory create a destructor destroy in the library code.

Supplementary question: Why do I need the destroy(Base* p) function? If this function only calls the delete-operator of the client I could also do it by myself, i.e "delete a" instead of destroy_module(a) in the next to last line.

Answer I found: The library can also provide a new/delete-operator pair. So if I use first the library's new and later the client's delete I can probably step into a pitfall. Sadly until now I never saw my library using it's own new or delete... So the original question still isn't answered.

Supplement: I'm only referring to the Linux platform.

Edit: The important parts are in the comments to Employed Russian's Answer. So I'm giving the main clue in a nutshell: If one calls the gcc this way

g++ -Wall -g -fPIC -shared library.cpp -o library.so -Wl,-Bsymbolic

the library will use it's own new/delete operators. Otherwise results

g++ -Wall -g -fPIC -shared library.cpp -o library.so

in a library that's using the new/delete operators of the calling program. Thanks to Employed Russian!

Answer 1

The problem is that on most UNIX platforms (unlike on Win32 and AIX) all symbol references by default bind to the first definition of the symbol visible to the runtime loader.

If you define 'operator new' in the main a.out, everything will bind to that definition (as Neil Butterworth's example shows), because a.out is the very first image runtime loader searches.

If you define it in a library which is loaded after libC.so (or libstdc++.so in case you are using GCC), then your definition will never be used. Since you are dlopen()ing your library after the program has started, libC is already loaded by that point, and your library is the very last one the runtime loader will search; so you lose.

On ELF platforms, you may be able to change the default behavior by using -Bsymbolic. From man ld on Linux:

 -Bsymbolic
   When creating a shared library, bind references to global symbols
   to the definition within the shared library, if any. Normally, it
   is possible for a program linked against a shared library to override
   the  definition within the shared library. This option is only meaningful
   on ELF platforms which support shared libraries.

Note that -Bsymbolic is a linker flag, not a compiler flag. If using g++, you must pass the flag to the linker like this:

  g++ -fPIC -shared library.cpp -o library.so -Wl,-Bsymbolic

Answer 2

The following code works as expected. Are you expecting your dynamic library code to use the new/delete you provide? I think you will be disappointed.

#include <memory>
#include <cstdio>
#include <cstdlib>
using namespace std;;

void* operator new(size_t size) {
        std::printf("New...\n");
        void *p=std::malloc(size); 
        if (p == 0) // did malloc succeed?
                throw std::bad_alloc(); // ANSI/ISO compliant behavior
        return p;
}

void operator delete(void* p) {
        std::printf("Delete...\n");
        std::free(p);
}

int main() {
    int * p = new int(42);
    delete p;
}

Answer 3

Look into RTLD_DEEPBIND

Answer 4

I think the problem is that operator overloading in C++ is a compile-time, not link-time, feature. The DLL has been compiled without knowledge of overloaded new() so it won't work correctly.

Another possibility is that on your platform it would work used with linking (like in your example) but that the DLL does not resolve symbols from your executable file.