Cryptacular is broken

Question

this weekend our docker image broke because it cannot be build anymore. While looking into the stats, I saw this line:

crypt_blowfish-1.2/crypt.h:17:23: fatal error: gnu-crypt.h: No such file or directory

In more detail:

  Running setup.py bdist_wheel for cryptacular: started
  Running setup.py bdist_wheel for cryptacular: finished with status 'error'
  Complete output from command /usr/local/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-sayd65k0/cryptacular/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/tmp5734bf55pip-wheel- --python-tag cp36:
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.linux-x86_64-3.6
  creating build/lib.linux-x86_64-3.6/cryptacular
  copying cryptacular/__init__.py -> build/lib.linux-x86_64-3.6/cryptacular
  creating build/lib.linux-x86_64-3.6/cryptacular/crypt
  copying cryptacular/crypt/test_crypt.py -> build/lib.linux-x86_64-3.6/cryptacular/crypt
  copying cryptacular/crypt/__init__.py -> build/lib.linux-x86_64-3.6/cryptacular/crypt
  creating build/lib.linux-x86_64-3.6/cryptacular/bcrypt
  copying cryptacular/bcrypt/test_bcrypt.py -> build/lib.linux-x86_64-3.6/cryptacular/bcrypt
  copying cryptacular/bcrypt/__init__.py -> build/lib.linux-x86_64-3.6/cryptacular/bcrypt
  creating build/lib.linux-x86_64-3.6/cryptacular/core
  copying cryptacular/core/test_core.py -> build/lib.linux-x86_64-3.6/cryptacular/core
  copying cryptacular/core/__init__.py -> build/lib.linux-x86_64-3.6/cryptacular/core
  creating build/lib.linux-x86_64-3.6/cryptacular/pbkdf2
  copying cryptacular/pbkdf2/test_pbkdf2.py -> build/lib.linux-x86_64-3.6/cryptacular/pbkdf2
  copying cryptacular/pbkdf2/__init__.py -> build/lib.linux-x86_64-3.6/cryptacular/pbkdf2
  running egg_info
  writing cryptacular.egg-info/PKG-INFO
  writing dependency_links to cryptacular.egg-info/dependency_links.txt
  writing namespace_packages to cryptacular.egg-info/namespace_packages.txt
  writing requirements to cryptacular.egg-info/requires.txt
  writing top-level names to cryptacular.egg-info/top_level.txt
  reading manifest file 'cryptacular.egg-info/SOURCES.txt'
  reading manifest template 'MANIFEST.in'
  writing manifest file 'cryptacular.egg-info/SOURCES.txt'
  copying cryptacular/bcrypt/_bcrypt.c -> build/lib.linux-x86_64-3.6/cryptacular/bcrypt
  running build_ext
  building 'cryptacular.bcrypt._bcrypt' extension
  creating build/temp.linux-x86_64-3.6
  creating build/temp.linux-x86_64-3.6/crypt_blowfish-1.2
  creating build/temp.linux-x86_64-3.6/cryptacular
  creating build/temp.linux-x86_64-3.6/cryptacular/bcrypt
  gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DNO_BF_ASM -Icrypt_blowfish-1.2/ -I/usr/local/include/python3.6m -c crypt_blowfish-1.2/crypt_blowfish.c -o build/temp.linux-x86_64-3.6/crypt_blowfish-1.2/crypt_blowfish.o
  gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DNO_BF_ASM -Icrypt_blowfish-1.2/ -I/usr/local/include/python3.6m -c crypt_blowfish-1.2/crypt_gensalt.c -o build/temp.linux-x86_64-3.6/crypt_blowfish-1.2/crypt_gensalt.o
  gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DNO_BF_ASM -Icrypt_blowfish-1.2/ -I/usr/local/include/python3.6m -c crypt_blowfish-1.2/wrapper.c -o build/temp.linux-x86_64-3.6/crypt_blowfish-1.2/wrapper.o
  gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DNO_BF_ASM -Icrypt_blowfish-1.2/ -I/usr/local/include/python3.6m -c cryptacular/bcrypt/_bcrypt.c -o build/temp.linux-x86_64-3.6/cryptacular/bcrypt/_bcrypt.o
  In file included from /usr/local/include/python3.6m/Python.h:39:0,
                   from cryptacular/bcrypt/_bcrypt.c:26:
  crypt_blowfish-1.2/crypt.h:17:23: fatal error: gnu-crypt.h: No such file or directory
   #include <gnu-crypt.h>
                         ^
  compilation terminated.
  error: command 'gcc' failed with exit status 1

  ----------------------------------------
  Failed building wheel for cryptacular
  Running setup.py clean for cryptacular

Our implementation works with the package cryptacular which uses bcrypt. None of the packages were updated last week and I dont know where to start.

As additional information, the error occurs in this block of our Dockerfile

RUN apt-get install -y build-essential libfontconfig && \
    pip install -q -U pip && \
    pip install -q -r requirements.txt && \
    apt-get remove -y --purge build-essential && \
    apt-get autoremove -y && \
    apt-get clean -y

where the requirements.txt uses cryptacular==1.4.1

Edit: We are working with the python:3.6-slim image

Answer 1

I was able to install using a virtualenv with python 2.7.14, installed via venv

(test) rodrigo@trujillo:/tmp/test $ python --version
Python 2.7.14

(test) rodrigo@trujillo:/tmp/test $ pip --version
pip 10.0.1 from /tmp/test/lib/python2.7/site-packages/pip (python 2.7)

(test) rodrigo@trujillo:/tmp/test $ pip install cryptacular
Collecting cryptacular
  Using cached https://files.pythonhosted.org/packages/73/bd/714b3fbfb3392d6b4e658638d9b74f77ce1072725209c08a6becd90836a7/cryptacular-1.4.1.tar.gz
Collecting pbkdf2 (from cryptacular)
Requirement already satisfied: setuptools in ./lib/python2.7/site-packages (from cryptacular) (39.2.0)
Building wheels for collected packages: cryptacular
  Running setup.py bdist_wheel for cryptacular ... done
  Stored in directory: /home/rodrigo/.cache/pip/wheels/05/cd/1f/64fb0fb06f2735e2ca898935061275465573db7f4f98fa1db4
Successfully built cryptacular
Installing collected packages: pbkdf2, cryptacular
Successfully installed cryptacular-1.4.1 pbkdf2-1.3

(test) rodrigo@trujillo:/tmp/test $ pip list
Package       Version
------------- -------
cryptacular   1.4.1  
pbkdf2        1.3    
pip           10.0.1 
pkg-resources 0.0.0  
setuptools    39.2.0 
wheel         0.31.1 

Answer 2

My problem solved by install cryptacular before install apex

Reference: https://github.com/NVIDIA/apex/issues/170#issuecomment-477321199

Answer 3

I had the same issue running python 3.6.6. With this one liner I was able to install cryptacular from the latest commit.

pipenv install -e hg+https://bitbucket.org/dholth/cryptacular@cb96fb3#egg=cryptacular

You will need to have mercurial installed

apt-get update && apt-get install -y mercurial

More info on this thread

Answer 4

We don't know the error exactly, but last weekend there was an security update in the base image. Therefore we pinned our python image to 3.6.4.