I would like to use cffi
(or even ctypes
if I must) to access a C ABI from Python 3 on Linux. The API is implemented by a number of .so
files (let's call them libA.so
, libB.so
and libC.so
), such that libA
contains the main exported functions, and the other libs provide support for libA
.
Now, libA
depends on libB
and libB
depends on libC
. However, there's a problem. There's a global array defined by libA
that libC
expects to be present. So libC
actually depends on libA
- a circular dependency. Trying to use cffi or ctags equivalent to dlopen
to load libA
results in missing symbols from libB
and libC
, but trying to load libC
first results in an error about the missing array (which is in libA
).
Since it's a variable, rather than a function, the RTLD_LAZY option doesn't seem to apply here.
Oddly, ldd libA.so
doesn't show libB
or libC
as dependencies so I'm not sure if that's part of the problem. I suppose that relies on any program that links with these libraries to explicitly specify them all.
Is there a way to get around this? One idea was to create a new shared object (say, "all.so") that is dependent on libA
, libB
and libC
so that dlopen("all.so")
might load everything it needs in one go, but I can't get this to work either.
What's the best strategy to handle this situation? In reality, the ABI I'm trying to access is pretty large, with perhaps 20-30 shared object files.
This (if I understood the problem correctly,) is a perfectly normal usecase on Nix, and should run without problems.
When dealing with problems related to ctypes ([Python 3]: ctypes - A foreign function library for Python), the best (generic) way to tackle them is:
I prepared a small (and dummy) example:
defines.h:
libC:
libC.h:
libC.c:
libB:
libB.h:
libB.c:
libA:
libA.h:
libA.c:
code.py:
Output:
But if your array is declared as static ([CPPReference]: C keywords: static) (and thus, as a consequence it can't be extern as in the example), then you're kind of toasted.
@EDIT0: Extending the example so that it better fits the description.
Since ldd doesn't show dependencies between the .sos, I'm going to assume that each is loaded dynamically.
utils.h:
utils.c:
Below is a modified version of libB.c. Note that the same pattern should also be applied to the original libA.c.
libB.c:
Output:
I believe that this reproduces the problem. Now, if you modify (the 1st part of) code.py to:
you'd get the following output:
Notes:
RTLD_LAZY | RTLD_GLOBAL
are there. if RTLD_LAZY is replaced by RTLD_NOW, it won't work