I load Python dynamically with dlopen
and RTLD_LOCAL
to avoid collisions with another library which by coincidence contains a few symbols with the same name. Executing my MVCE above on macOS with Xcode fails because it expects _PyBuffer_Type
in the global namespace.
Traceback (most recent call last):
File "...lib/python2.7/ctypes/__init__.py", line 10, in <module>
from _ctypes import Union, Structure, Array
ImportError: dlopen(...lib/python2.7/lib-dynload/_ctypes.so, 2):
Symbol not found: _PyBuffer_Type
Referenced from: ...lib/python2.7/lib-dynload/_ctypes.so
Expected in: flat namespace
in ...lib/python2.7/lib-dynload/_ctypes.so
Program ended with exit code: 255
But why? Does RTLD_LOCAL
overwrite the two-level namespace?
I used otool -hV
to check that _ctypes.so was compiled with the Two-Level namespace option. From my understanding the symbol resolve needs the library name + the symbol name itself. Why does it expect _PyBuffer_Type
in the flat namespace and/or why can't it find it? See TWOLEVEL
by scrolling to the right
> otool -hV /System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload/_ctypes.so
Mach header
magic cputype cpusubtype caps filetype ncmds sizeofcmds flags
MH_MAGIC_64 X86_64 ALL 0x00 BUNDLE 14 1536 NOUNDEFS DYLDLINK TWOLEVEL
Any idea whats going on here?
MVCE
Can be copied to a new Xcode project, simply compile and execute.
#include </System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7/Python.h>
#include <dlfcn.h>
int main(int argc, const char * argv[])
{
auto* dl = dlopen("/System/Library/Frameworks/Python.framework/Versions/2.7/Python", RTLD_LOCAL | RTLD_NOW);
if (dl == nullptr)
return 0;
// Load is just a macro to hide dlsym(..)
#define Load(name) ((decltype(::name)*)dlsym(dl, # name))
Load(Py_SetPythonHome)("/System/Library/Frameworks/Python.framework/Versions/2.7");
Load(Py_Initialize)();
auto* readline = Load(PyImport_ImportModule)("ctypes");
if (readline == nullptr)
{
Load(PyErr_Print)();
dlclose(dl);
return -1;
}
Py_DECREF(readline);
Load(Py_Finalize)();
return 0;
}