I'm trying to deploy my app to Heroku. It is using pyOpenSSL, which requires cryptography, which requires libffi. I found a custom buildpack that includes libffi here: https://github.com/mfenniak/heroku-buildpack-python-libffi. However, cryptography cannot seem to find libffi even though it's on LD_LIBRARY_PATH:
$ heroku run bash
heroku> echo $LD_LIBRARY_PATH
/app/.heroku/vendor/lib:/app/vendor/libffi-3.0/lib
heroku> ls /app/vendor/libffi-3.0/lib
libffi-3.0.13 libffi.a libffi.la libffi.so libffi.so.6 libffi.so.6.0.1 pkgconfig
However, I'm not sure LD_LIBRARY_PATH is available during install process, but it is part of the compile step of the buildpack: https://github.com/mfenniak/heroku-buildpack-python-libffi/commit/6ce48d4fd6c55fc3dc462cf6300c17854732b6e2
In general, this buildpack works and I used it previously with bcrypt (https://pypi.python.org/pypi/bcrypt).
Here is the heroku deployment process which fails:
$ git push staging master
Fetching repository, done.
Counting objects: 20, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (11/11), done.
Writing objects: 100% (11/11), 1.71 KiB | 0 bytes/s, done.
Total 11 (delta 9), reused 0 (delta 0)
-----> Fetching custom git buildpack... done
-----> Python app detected
-----> No runtime.txt provided; assuming python-2.7.4.
-----> Using Python runtime (python-2.7.4)
-----> Noticed cffi. Bootstrapping libffi.
PKG_CONFIG_PATH=:/app/vendor/libffi-3.0/lib/pkgconfig/
libffi.pc is in-place at /app/vendor/libffi-3.0/vendor/lib/pkgconfig/libffi.pc, libffi-based builds should work!
-----> Installing dependencies using Pip (1.3.1)
Obtaining file:///app (from -r requirements.txt (line 1))
Running setup.py egg_info for package from file:///app
Downloading/unpacking pyOpenSSL==0.14 (from -r requirements.txt (line 76))
Running setup.py egg_info for package pyOpenSSL
no previously-included directories found matching 'doc/_build'
Downloading/unpacking cryptography==0.2.2 (from -r requirements.txt (line 77))
Running setup.py egg_info for package cryptography
Traceback (most recent call last):
File "<string>", line 16, in <module>
File "/tmp/pip-build-u45370/cryptography/setup.py", line 113, in <module>
"build": cffi_build,
File "/app/.heroku/python/lib/python2.7/distutils/core.py", line 152, in setup
dist.run_commands()
File "/app/.heroku/python/lib/python2.7/distutils/dist.py", line 953, in run_commands
self.run_command(cmd)
File "/app/.heroku/python/lib/python2.7/distutils/dist.py", line 972, in run_command
cmd_obj.run()
File "<string>", line 14, in replacement_run
File "/app/.heroku/python/lib/python2.7/site-packages/setuptools/command/egg_info.py", line 259, in find_sources
mm.run()
File "/app/.heroku/python/lib/python2.7/site-packages/setuptools/command/egg_info.py", line 325, in run
self.add_defaults()
File "/app/.heroku/python/lib/python2.7/site-packages/setuptools/command/egg_info.py", line 361, in add_defaults
sdist.add_defaults(self)
File "/app/.heroku/python/lib/python2.7/site-packages/setuptools/command/sdist.py", line 199, in add_defaults
build_py = self.get_finalized_command('build_py')
File "/app/.heroku/python/lib/python2.7/distutils/cmd.py", line 312, in get_finalized_command
cmd_obj.ensure_finalized()
File "/app/.heroku/python/lib/python2.7/distutils/cmd.py", line 109, in ensure_finalized
self.finalize_options()
File "/app/.heroku/python/lib/python2.7/site-packages/setuptools/command/build_py.py", line 73, in finalize_options
_build_py.finalize_options(self)
File "/app/.heroku/python/lib/python2.7/distutils/command/build_py.py", line 46, in finalize_options
('force', 'force'))
File "/app/.heroku/python/lib/python2.7/distutils/cmd.py", line 298, in set_undefined_options
src_cmd_obj.ensure_finalized()
File "/app/.heroku/python/lib/python2.7/distutils/cmd.py", line 109, in ensure_finalized
self.finalize_options()
File "/tmp/pip-build-u45370/cryptography/setup.py", line 52, in finalize_options
from cryptography.hazmat.primitives import constant_time, padding
File "cryptography/hazmat/primitives/constant_time.py", line 21, in <module>
_ffi = cffi.FFI()
File "/app/.heroku/python/lib/python2.7/site-packages/cffi/api.py", line 56, in __init__
import _cffi_backend as backend
ImportError: libffi.so.6: cannot open shared object file: No such file or directory
Complete output from command python setup.py egg_info:
running egg_info
creating pip-egg-info/cryptography.egg-info
writing requirements to pip-egg-info/cryptography.egg-info/requires.txt
writing pip-egg-info/cryptography.egg-info/PKG-INFO
writing top-level names to pip-egg-info/cryptography.egg-info/top_level.txt
writing dependency_links to pip-egg-info/cryptography.egg-info/dependency_links.txt
writing manifest file 'pip-egg-info/cryptography.egg-info/SOURCES.txt'
warning: manifest_maker: standard file '-c' not found
Traceback (most recent call last):
File "<string>", line 16, in <module>
File "/tmp/pip-build-u45370/cryptography/setup.py", line 113, in <module>
"build": cffi_build,
File "/app/.heroku/python/lib/python2.7/distutils/core.py", line 152, in setup
dist.run_commands()
File "/app/.heroku/python/lib/python2.7/distutils/dist.py", line 953, in run_commands
self.run_command(cmd)
File "/app/.heroku/python/lib/python2.7/distutils/dist.py", line 972, in run_command
cmd_obj.run()
File "<string>", line 14, in replacement_run
File "/app/.heroku/python/lib/python2.7/site-packages/setuptools/command/egg_info.py", line 259, in find_sources
mm.run()
File "/app/.heroku/python/lib/python2.7/site-packages/setuptools/command/egg_info.py", line 325, in run
self.add_defaults()
File "/app/.heroku/python/lib/python2.7/site-packages/setuptools/command/egg_info.py", line 361, in add_defaults
sdist.add_defaults(self)
File "/app/.heroku/python/lib/python2.7/site-packages/setuptools/command/sdist.py", line 199, in add_defaults
build_py = self.get_finalized_command('build_py')
File "/app/.heroku/python/lib/python2.7/distutils/cmd.py", line 312, in get_finalized_command
cmd_obj.ensure_finalized()
File "/app/.heroku/python/lib/python2.7/distutils/cmd.py", line 109, in ensure_finalized
self.finalize_options()
File "/app/.heroku/python/lib/python2.7/site-packages/setuptools/command/build_py.py", line 73, in finalize_options
_build_py.finalize_options(self)
File "/app/.heroku/python/lib/python2.7/distutils/command/build_py.py", line 46, in finalize_options
('force', 'force'))
File "/app/.heroku/python/lib/python2.7/distutils/cmd.py", line 298, in set_undefined_options
src_cmd_obj.ensure_finalized()
File "/app/.heroku/python/lib/python2.7/distutils/cmd.py", line 109, in ensure_finalized
self.finalize_options()
File "/tmp/pip-build-u45370/cryptography/setup.py", line 52, in finalize_options
from cryptography.hazmat.primitives import constant_time, padding
File "cryptography/hazmat/primitives/constant_time.py", line 21, in <module>
_ffi = cffi.FFI()
File "/app/.heroku/python/lib/python2.7/site-packages/cffi/api.py", line 56, in __init__
import _cffi_backend as backend
ImportError: libffi.so.6: cannot open shared object file: No such file or directory
----------------------------------------
Command python setup.py egg_info failed with error code 1 in /tmp/pip-build-u45370/cryptography
Storing complete log in /app/.pip/pip.log
! Push rejected, failed to compile Python app
To git@heroku.com:my-app-staging.git
! [remote rejected] master -> master (pre-receive hook declined)
error: failed to push some refs to 'git@heroku.com:my-app-staging.git'
Update
Strangely enough, if I ssh to a dyno heroku run bash and then pip install pyOpenSSL, then it succeeds. But this doesn't seem to help figuring out what's wrong with the deployment process.
