Python HTTPS download fails due to “

2019-08-19 00:50发布

问题:

I'm trying to build a Python 2.7 from source on a pretty minimal system (Ubuntu 16.04 docker image) which doesn't cause HTTPS downloads to fail due to <urlopen error unknown url type: https>. I don't want to use a package manager, but am aware that this would solve the problem easily.

I went through Unknown url type error in urllib2 and applied the different referenced patches to setup.py, set the SSL variable in Modules/Setup and Modules/Setup.dist to the installation prefix which I passed to ./config --prefix of OpenSSL 1.1.1 and set LD_LIBRARY_PATH to the OpenSSL installation prefix.

I suspect that one such a minimal system more things have to be fixed than suggested in the other answer. Why is there no --with-ssl switch for configure?

The build script is

apt-get update && apt-get install --yes wget xz-utils make gcc patch
wget http://www.cpan.org/src/5.0/perl-5.26.1.tar.gz && tar xf perl-5.26.1.tar.gz && cd perl-5.26.1 && ./configure.gnu && make -j16 && make install && cd ..
wget https://ftp.gnu.org/pub/gnu/gettext/gettext-0.19.8.1.tar.xz && tar xf gettext-0.19.8.1.tar.xz && cd gettext-0.19.8.1 && ./configure && make -j16 && make install && cd ..
wget https://www.zlib.net/zlib-1.2.11.tar.gz && tar xf zlib-1.2.11.tar.gz && cd zlib-1.2.11 && ./configure && make -j16 && make install && cd ..
wget https://www.kernel.org/pub/software/scm/git/git-2.13.3.tar.gz && tar xf git-2.13.3.tar.gz && cd git-2.13.3 && ./configure --with-perl=/usr/local/bin/perl && make -j16 && make install && cd ..
            # need `--with-perl` passed to `configure` because the `/usr/local/bin/perl` isn't picked up regardless where it's found in PATH, see https://stackoverflow.com/questions/48931247/cant-locate-extutils-makemaker-pm-in-inc-during-git-build for details
wget https://www.openssl.org/source/openssl-1.1.1-pre1.tar.gz && tar xf openssl-1.1.1-pre1.tar.gz && cd openssl-1.1.1-pre1 && ./config --prefix=/usr/local/openssl && make -j16 && make install && cd ..
            # need to install into explicit prefix different from `/usr/local` in order to make OpenSSL get picked up by Python's patched `configure` (specifying `OPENSSL_ROOT=/usr/local` doesn't work)
wget https://www.python.org/ftp/python/2.7.14/Python-2.7.14.tgz && tar xf Python-2.7.14.tgz && cd Python-2.7.14 && wget https://gist.githubusercontent.com/rkitover/2d9e5baff1f1cc4f2618dee53083bd35/raw/7f33fcf5470a9f1013ac6ae7bb168368a98fe5a0/python-2.7.14-custom-static-openssl.patch && patch -p1 <python-2.7.14-custom-static-openssl.patch && env OPENSSL_ROOT=/usr/local/openssl ./configure --disable-shared && make -j16 && make install && cd ..
git clone git://git.gnome.org/jhbuild && cd jhbuild && ./autogen.sh --prefix=/usr/local && make && make install && cd ..
env JHBUILD_RUN_AS_ROOT= jhbuild bootstrap

I chose jhbuild bootstrap as test command because that's the desired outcome, feel free to suggest a better one.

I created a SSCCE at https://gitlab.com/krichter/jhbuild-docker-build which allows to run the build script in GitLab CI and an example run including a full build log can be investigated at https://gitlab.com/krichter/jhbuild-docker-build/-/jobs/53948513.

回答1:

I can get it to fail with <urlopen error [SSL: NO_CIPHERS_AVAILABLE] no ciphers available (_ssl.c:661)> (which I deal with at FIXME) after

  • adding --prefix=/usr/local/ --openssldir=/usr/local to config of OpenSSL,
  • applying https://gist.githubusercontent.com/rkitover/2d9e5baff1f1cc4f2618dee53083bd35/raw/7f33fcf5470a9f1013ac6ae7bb168368a98fe5a0/python-2.7.14-custom-static-openssl.patch as explained in the referenced question above and
  • patching Modules/Setup.dist and Modules/Setup with

    --- Modules/Setup.dist  2018-02-23 01:12:46.537415323 +0100
    +++ Modules/Setup.dist.d    2018-02-23 01:44:45.235092890 +0100
    @@ -215,10 +215,10 @@
    
     # Socket module helper for SSL support; you must comment out the other
     # socket line above, and possibly edit the SSL variable:
    -#SSL=/usr/local/ssl
    -#_ssl _ssl.c \
    -#  -DUSE_SSL -I$(SSL)/include -I$(SSL)/include/openssl \
    -#  -L$(SSL)/lib -lssl -lcrypto
    +SSL=/usr/local
    +_ssl _ssl.c \
    +   -DUSE_SSL -I$(SSL)/include -I$(SSL)/include/openssl \
    +   -L$(SSL)/lib -lssl -lcrypto
    
     # The crypt module is now disabled by default because it breaks builds
     # on many systems (where -lcrypt is needed), e.g. Linux (I believe).
    

    (patch -u -p0 <somename.patch)

A example build overcoming the initial error can be found at https://gitlab.com/krichter/jhbuild-docker-build/-/jobs/53978734.