pip fails with AttributeError: 'module' ob

Question

I recently found that pip was completely broken for me. This is on Ubuntu bionic 18.04.

$ pip --version    
Traceback (most recent call last):                                           
  File "/usr/bin/pip", line 9, in <module>
from pip import main                                   
  File "/usr/lib/python2.7/dist-packages/pip/__init__.py", line 22, in <module>
from pip._vendor.requests.packages.urllib3.exceptions import DependencyWarning
  File "/usr/lib/python2.7/dist-packages/pip/_vendor/__init__.py", line 64, in <module>
vendored("cachecontrol")                      
  File "/usr/lib/python2.7/dist-packages/pip/_vendor/__init__.py", line 36, in vendored
__import__(modulename, globals(), locals(), level=0)
File "/usr/share/python-wheels/CacheControl-0.11.7-py2.py3-none-any.whl/cachecontrol/__init__.py", line 9, in <module>
File "/usr/share/python-wheels/CacheControl-0.11.7-py2.py3-none-any.whl/cachecontrol/wrapper.py", line 1, in <module>
File "/usr/share/python-wheels/CacheControl-0.11.7-py2.py3-none-any.whl/cachecontrol/adapter.py", line 4, in <module>
File "/usr/share/python-wheels/requests-2.18.4-py2.py3-none-any.whl/requests/__init__.py", line 84, in <module>
File "/usr/share/python-wheels/urllib3-1.22-py2.py3-none-any.whl/urllib3/contrib/pyopenssl.py", line 46, in <module>
File "/usr/lib/python2.7/dist-packages/OpenSSL/__init__.py", line 8, in <module>
 from OpenSSL import crypto, SSL                                                
  File "/usr/lib/python2.7/dist-packages/OpenSSL/crypto.py", line 12, in <module>
from cryptography import x509                                  
  File "/usr/lib/python2.7/dist-packages/cryptography/x509/__init__.py", line 8, in <module>
from cryptography.x509.base import (                       
  File "/usr/lib/python2.7/dist-packages/cryptography/x509/base.py", line 16, in <module>
from cryptography.x509.extensions import Extension, ExtensionType
  File "/usr/lib/python2.7/dist-packages/cryptography/x509/extensions.py", line 18, in <module>
from cryptography.hazmat.primitives import constant_time, serialization
  File "/usr/lib/python2.7/dist-packages/cryptography/hazmat/primitives/constant_time.py", line 9, in <module>
from cryptography.hazmat.bindings._constant_time import lib
AttributeError: 'module' object has no attribute '_init_cffi_1_0_external_module'

I've seen other references but they don't match my situation, or don't have answers:

• Python Cryptography run time error on _init_cffi_1_0_external_module - Stack Overflow
• Bug #1512792 “Pip crashes with cryptography/cffi/openssl error” : Bugs : python-pip package : Ubuntu
• Bug #1484870 “Devstack not working in Fedora 22 (python-cffi too...” : Bugs : devstack

How can I fix this?

Answer 1

After looking at this some more, I discovered that the Fedora bug report above had information that really helped:

at some point, cffi 1.1.2 from pip is being overwritten with cffi 0.8.6 from the python-cffi package. As cryptography 1.1.2 is making a call to the cffi_1_0_external_module' it doesn't exists and thus fails.

I dug around, and found that I had an obsolete cffi version in ~/.local/lib/python2.7/site-packages/ (presumably from a pip install --user ... some years ago).

A general way to explore that and check versions is:

>>> import cffi
>>> cffi.__version__
'0.8.6'
>>> cffi
<module 'cffi' from '$HOME/.local/lib/python2.7/site-packages/cffi/__init__.pyc'>

Removing that directory fixed pip, but presumably could have interefered with something else I installed long ago.

I also hear that it can help in some related situations to use easy_install (since pip isn't working...) to upgrade cffi:

easy_install -U cffi

I'm still curious if there is a large lesson here about how to avoid this sort of thing in the future.

• Why is cffi both within pip and an external package?
• Are local installs dangerous, since they persist past OS and Python upgrades?
• Are there general packaging/dependency best practices that would avoid these sorts of package dependency problems?