Dynamic linking with rpath not working under Ubunt

Question

I build an R package which uses Rcpp and links to a third-party shared object (libbarraopt.so) (which also links to other shared objects such as liboptsrvr.so in its own directory). To ensure that it is able to find those shared objects it links against, I put the following variables in ~/.Renviron:

BARRA_OPS_HOME=${HOME}/bin/BarraOptimizer8.5

In the package, I create the following src/Makevars:

BARRA_LIB=$(BARRA_OPS_HOME)/lib/intel64
BARRA_INCLUDE=$(BARRA_OPS_HOME)/include
PKG_CXXFLAGS=-I$(BARRA_INCLUDE)
PKG_CFLAGS=-I$(BARRA_INCLUDE)
PKG_LIBS=-L$(BARRA_LIB) -Wl,-R,$(BARRA_LIB) -lbarraopt

Under Ubuntu 16.04, I can build, load, and use the package successfully without any problem. However, when I test exactly the same package when OS is upgraded to 17.10, the package can be built but it cannot be loaded, saying:

g++ -std=gnu++11 -I/usr/share/R/include -DNDEBUG  -I"/home/renkun/R/x86_64-pc-linux-gnu-library/3.4/Rcpp/include"   -I/home/renkun/bin/BarraOptimizer8.5/include -fpic  -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g -c RcppExports.cpp -o RcppExports.o
** libs
g++ -std=gnu++11 -I/usr/share/R/include -DNDEBUG  -I"/home/renkun/R/x86_64-pc-linux-gnu-library/3.4/Rcpp/include"   -I/home/renkun/bin/BarraOptimizer8.5/include -fpic  -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g -c barraopt.cpp -o barraopt.o
g++ -std=gnu++11 -shared -L/usr/lib/R/lib -Wl,-Bsymbolic-functions -Wl,-z,relro -o barraopt.so RcppExports.o barraopt.o -L/home/renkun/bin/BarraOptimizer8.5/lib/intel64 -Wl,-R,/home/renkun/bin/BarraOptimizer8.5/lib/intel64 -lbarraopt -L/usr/lib/R/lib -lR
installing to /tmp/Rtmpvbb6Io/devtools_install_42a342a07f84/barraopt/libs
* DONE (barraopt)
Error in dyn.load(dllfile) : 
  unable to load shared object '/home/renkun/Workspaces/barraopt/src/barraopt.so':
  liboptsrvr.so: cannot open shared object file: No such file or directory
Calls: suppressPackageStartupMessages ... <Anonymous> -> load_all -> load_dll -> library.dynam2 -> dyn.load
Execution halted

Exited with status 1.

It seems that -Wl,-rpath is not effective here.

Under a machine with Ubuntu 16.04, ldd src/barraopt.so shows that all dynamic linking is corrected resolved. (BARRA_OPS_HOME = /home/ken/bin/BarraOptimizer8.5)

linux-vdso.so.1 =>  (0x00007ffc89a16000)
libbarraopt.so => /home/ken/bin/BarraOptimizer8.5/lib/intel64/libbarraopt.so (0x00007f85dae49000)
libimf.so => /home/ken/bin/BarraOptimizer8.5/lib/intel64/libimf.so (0x00007f85da97f000)
libR.so => /usr/lib/libR.so (0x00007f85da346000)
libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f85d9fc4000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f85d9dae000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f85d99e3000)
liboptsrvr.so => /home/ken/bin/BarraOptimizer8.5/lib/intel64/liboptsrvr.so (0x00007f85d7b10000)
libopsproto.so => /home/ken/bin/BarraOptimizer8.5/lib/intel64/libopsproto.so (0x00007f85d77a1000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f85d7497000)
libintlc.so.5 => /home/ken/bin/BarraOptimizer8.5/lib/intel64/libintlc.so.5 (0x00007f85d7249000)
libblas.so.3 => /usr/lib/libblas.so.3 (0x00007f85d6fe8000)
libreadline.so.6 => /lib/x86_64-linux-gnu/libreadline.so.6 (0x00007f85d6da1000)
libpcre.so.3 => /lib/x86_64-linux-gnu/libpcre.so.3 (0x00007f85d6b31000)
liblzma.so.5 => /lib/x86_64-linux-gnu/liblzma.so.5 (0x00007f85d690f000)
libbz2.so.1.0 => /lib/x86_64-linux-gnu/libbz2.so.1.0 (0x00007f85d66fe000)
libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f85d64e4000)
librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f85d62dc000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f85d60d7000)
libgomp.so.1 => /usr/lib/x86_64-linux-gnu/libgomp.so.1 (0x00007f85d5eb5000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f85d5c98000)
/lib64/ld-linux-x86-64.so.2 (0x000055fb75088000)
libifcore.so.5 => /home/ken/bin/BarraOptimizer8.5/lib/intel64/libifcore.so.5 (0x00007f85d5961000)
libifport.so.5 => /home/ken/bin/BarraOptimizer8.5/lib/intel64/libifport.so.5 (0x00007f85d5732000)
libsvml.so => /home/ken/bin/BarraOptimizer8.5/lib/intel64/libsvml.so (0x00007f85d4e6d000)
libmosek64.so.7.0 => /home/ken/bin/BarraOptimizer8.5/lib/intel64/libmosek64.so.7.0 (0x00007f85d3c63000)
libiomp5.so => /home/ken/bin/BarraOptimizer8.5/lib/intel64/libiomp5.so (0x00007f85d396b000)
libprotobuf.so.6 => /home/ken/bin/BarraOptimizer8.5/lib/intel64/libprotobuf.so.6 (0x00007f85d3668000)
libbridge_common.so => /home/ken/bin/BarraOptimizer8.5/lib/intel64/libbridge_common.so (0x00007f85d3417000)
libsharc_xmlxproto.so => /home/ken/bin/BarraOptimizer8.5/lib/intel64/libsharc_xmlxproto.so (0x00007f85d31a4000)
libboost_thread.so.1.49.0 => /home/ken/bin/BarraOptimizer8.5/lib/intel64/libboost_thread.so.1.49.0 (0x00007f85d2f8a000)
libopenblas.so.0 => /usr/lib/libopenblas.so.0 (0x00007f85d0ef5000)
libtinfo.so.5 => /lib/x86_64-linux-gnu/libtinfo.so.5 (0x00007f85d0ccc000)
libxerces-c-3.1.so => /home/ken/bin/BarraOptimizer8.5/lib/intel64/libxerces-c-3.1.so (0x00007f85d07c4000)
libgfortran.so.3 => /usr/lib/x86_64-linux-gnu/libgfortran.so.3 (0x00007f85d0499000)
libnsl.so.1 => /lib/x86_64-linux-gnu/libnsl.so.1 (0x00007f85d027f000)
libquadmath.so.0 => /usr/lib/x86_64-linux-gnu/libquadmath.so.0 (0x00007f85d0040000)

However, with the same source, under Ubuntu 17.10, ldd shows that the shared objects libbarraopt.so links against are not resolved even though -Wl,-rpath is secified: (BARRA_OPS_HOME = /home/renkun/bin/BarraOptimizer8.5)

linux-vdso.so.1 =>  (0x00007ffe067f5000)
libbarraopt.so => /home/renkun/bin/BarraOptimizer8.5/lib/intel64/libbarraopt.so (0x00007f3dc5f0c000)
libR.so => /usr/lib/libR.so (0x00007f3dc58e4000)
libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f3dc555e000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f3dc5208000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f3dc4ff1000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f3dc4c11000)
liboptsrvr.so => not found
libopsproto.so => not found
libblas.so.3 => /usr/lib/x86_64-linux-gnu/libblas.so.3 (0x00007f3dc49b6000)
libreadline.so.6 => /lib/x86_64-linux-gnu/libreadline.so.6 (0x00007f3dc4770000)
libpcre.so.3 => /lib/x86_64-linux-gnu/libpcre.so.3 (0x00007f3dc44fe000)
liblzma.so.5 => /lib/x86_64-linux-gnu/liblzma.so.5 (0x00007f3dc42d8000)
libbz2.so.1.0 => /lib/x86_64-linux-gnu/libbz2.so.1.0 (0x00007f3dc40c8000)
libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f3dc3eab000)
librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f3dc3ca3000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f3dc3a9f000)
libgomp.so.1 => /usr/lib/x86_64-linux-gnu/libgomp.so.1 (0x00007f3dc3870000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f3dc3651000)
/lib64/ld-linux-x86-64.so.2 (0x00007f3dc6526000)
libopenblas.so.0 => /usr/lib/x86_64-linux-gnu/libopenblas.so.0 (0x00007f3dc13ab000)
libtinfo.so.5 => /lib/x86_64-linux-gnu/libtinfo.so.5 (0x00007f3dc1182000)
libgfortran.so.4 => /usr/lib/x86_64-linux-gnu/libgfortran.so.4 (0x00007f3dc0da3000)
libquadmath.so.0 => /usr/lib/x86_64-linux-gnu/libquadmath.so.0 (0x00007f3dc0b63000)

It looks like only libbarraopt.so is linked with the correct path but shared objects it links against are missing.

I'm wondering what might be wrong with my build configurations that breaks under the toolchain shipped by 17.10. Although using global config such as ldconfig would solve such problem, I prefer not because some .so it relies have conflict with the version the OS ships. I'd rather use a locally configured version without affecting the global config.

Answer 1

It seems that -Wl,-rpath is not effective here.

What likely happened is that your updated linker emits DT_RUNPATH dynamic tag, where the old linker emitted DT_RPATH. (It's also possible that your old linker was GNU-ld, and the new one is Gold.)

The DT_RUNPATH is preferred as more correct, and affects search path of the binary itself, but not of any of the dependent libraries.

The DT_RPATH has global effect, similar to adding the directory to LD_LIBRARY_PATH environment variable.

You can verify this with: readelf -d a.out | grep 'R.*PATH'.

If you do see the RPATH vs. RUNPATH difference, and in fact are using Gold, you can force the "old" behavior with -Wl,--disable-new-dtags (GNU ld also had --disable-new-dtags added to it recently, so it should work for either linker).