Building Python with SSL support in non-standard l

Question

I need to install several Python modules on a RHEL where I don't have root access. At least one of the modules also needs access to Python.h.

In this case I find that the best thing is to install python and it dependencies in ~/local. It usually just works, but this time Python fails to build the SSL module (see details below). Here's the trace of what I'm doing.

So I downloaded python 6 source and off I went:

./configure --prefix=/home/fds/rms/local
make >& make.log

An inspection to log reveals that ssl module has not been compiled, but there is no mention of the cause (no other occurence of ssl in make or configure):

Failed to find the necessary bits to build these modules:
_bsddb             _curses            _curses_panel
_hashlib           _sqlite3           _ssl   <----------

So I figure, python is not finding any ssl library at all (which is strange, but hey...). So I download openssl-0.9.8r and

./config --prefix=/home/fds/rms/local shared
make
make install

Now back to Python, I ./configure and make again. It fails, but this time it's different:

Failed to build these modules:
_hashlib           _ssl

A closer inspection to the log file reveals this:

gcc -pthread -shared build/temp.linux-x86_64-2.6/home/fds/rms/installers/Python-2.6.6/Modules/_ssl.o -L/home/fds/rms/local/lib -L/usr/local/lib -lssl -lcrypto -o build/lib.linux-x86_64-2.6/_ssl.so
*** WARNING: renaming "_ssl" since importing it failed: libssl.so.0.9.8: cannot open shared object file: No such file or directory

So now it's picking up the library but not quite getting it right (the file is there where is should be):

$ find /home/fds/rms/local -iname libssl.so.0.9.8
/home/fds/rms/local/lib/libssl.so.0.9.8

Next thing is tracing make and see where is it looking for the file:

$ strace -f make 2>&1 | grep libssl.so.0.9.8
[pid  5584] open("/lib/libssl.so.0.9.8", O_RDONLY) = -1 ENOENT (No such file or directory)
[pid  5584] open("/usr/lib/libssl.so.0.9.8", O_RDONLY) = -1 ENOENT (No such file or directory)
[pid  5584] open("/lib64/tls/x86_64/libssl.so.0.9.8", O_RDONLY) = -1 ENOENT (No such file or directory)
[pid  5584] open("/lib64/tls/libssl.so.0.9.8", O_RDONLY) = -1 ENOENT (No such file or directory)
[pid  5584] open("/lib64/x86_64/libssl.so.0.9.8", O_RDONLY) = -1 ENOENT (No such file or   directory)
[pid  5584] open("/lib64/libssl.so.0.9.8", O_RDONLY) = -1 ENOENT (No such file or directory)
[pid  5584] open("/usr/lib64/tls/x86_64/libssl.so.0.9.8", O_RDONLY) = -1 ENOENT (No such file or directory)
[pid  5584] open("/usr/lib64/tls/libssl.so.0.9.8", O_RDONLY) = -1 ENOENT (No such file or directory)
[pid  5584] open("/usr/lib64/x86_64/libssl.so.0.9.8", O_RDONLY) = -1 ENOENT (No such file or directory)
[pid  5584] open("/usr/lib64/libssl.so.0.9.8", O_RDONLY) = -1 ENOENT (No such file or directory)
[pid  5584] write(1, "*** WARNING: renaming \"_ssl\" sin"..., 131*** WARNING: renaming "_ssl" since importing it failed: libssl.so.0.9.8: cannot open shared object file: No such file or directory
[pid  5584] open("/lib/libssl.so.0.9.8", O_RDONLY) = -1 ENOENT (No such file or directory)
[pid  5584] open("/usr/lib/libssl.so.0.9.8", O_RDONLY) = -1 ENOENT (No such file or directory)
[pid  5584] open("/lib64/libssl.so.0.9.8", O_RDONLY) = -1 ENOENT (No such file or directory)
[pid  5584] open("/usr/lib64/tls/libssl.so.0.9.8", O_RDONLY) = -1 ENOENT (No such file or directory)
[pid  5584] open("/usr/lib64/libssl.so.0.9.8", O_RDONLY) = -1 ENOENT (No such file or directory)
[pid  5584] write(1, "*** WARNING: renaming \"_hashlib\""..., 135*** WARNING: renaming "_hashlib" since importing it failed: libssl.so.0.9.8: cannot open shared object file: No such file or directory

Mhhh, it's looking in all the wrong places. I try to give a hint:

CPPFLAGS="-I/home/fds/rms/local/include -I/home/fds/rms/local/include/openssl" LDFLAGS="-L/home/fds/rms/local/lib" ./configure --prefix=/home/fds/rms/local

But nothing changes, and make does not seem to try /home/fds/rms/local/lib at all.

I haven't done this in years, so maybe I'm overlooking something. Can anyone help with the problem?

Thanks in advance.

Answer 1

in the Bourne shell (/bin/sh or /bin/bash):

$ LD_LIBRARY_PATH=/usr/local/lib
$ export LD_LIBRARY_PATH
$ make

in the C-shell (/bin/csh or /bin/tcsh):

% setenv LD_LIBRARY_PATH /usr/local/lib
% make

Answer 2

Try adding -Wl,-rpath,/home/fds/rms/local/lib to LDPATH.

Answer 3

Here is the complete process I used with Python 2.7.11.

---

In top level Python2.7.11 source dir:

1. Change Modules/Setup.dist, Modules/Setup : Uncomment _ssl section, comment out _socket (no-op if it's already commented out), uncomment and set SSL appropriately (path to your new ssl lib/includes etc.)

   Note: Modules/Setup file does not exist initially but after first run it get's content from Modules/Setup.dist i believe. Make sure changes are reflected in here before every run.

2. Apply patch: http://gist.github.com/eddy-geek/9604982 (make distclean if previously ran make)

   ./configure --prefix=/root/.local/aks/py-ssl/python2.7 --enable-shared
   
   # modify: Makefile -> set svnversion to ""
   
   make
   
   make altinstall
   

Answer 4

I was getting the same result until I got back to the logs for openssl. There I saw that you need to use -fPIC when building openssl: building '_ssl' extension:

gcc -pthread -fPIC -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/usr/local/ssl/include -I. -IInclude -I./Include -I/usr/local/include -I/home/feramos/Python-2.7.3/Include -I/home/feramos/Python-2.7.3 -c /home/feramos/Python-2.7.3/Modules/_ssl.c -o build/temp.linux-x86_64-2.7/home/feramos/Python-2.7.3/Modules/_ssl.o
gcc -pthread -shared build/temp.linux-x86_64-2.7/home/feramos/Python-2.7.3/Modules/_ssl.o -L/usr/local/ssl/lib -L/usr/local/lib -lssl -lcrypto -o build/lib.linux-x86_64-2.7/_ssl.so
/usr/bin/ld: /usr/local/ssl/lib/libcrypto.a(x86_64cpuid.o): relocation R_X86_64_PC32 against `OPENSSL_cpuid_setup' can not be used when making a shared object; recompile with -fPIC

openssl-0.9.8g]# .config -fPIC

then, make, make install for openssl and then build Python again.