pip install dependency links

Question

I am using python version 2.7 and pip version is 1.5.6.

I want to install extra libraries from url like a git repo on setup.py is being installed.

I was putting extras in install_requires parameter in setup.py. This means, my library requires extra libraries and they must also be installed.

...
install_requires=[
    "Django",
    ....
],
...

But urls like git repos are not valid string in install_requires in setup.py. Assume that, I want to install a library from github. I have searched about that issue and I found something which I can put libraries such that in dependency_links in setup.py. But that still doesn't work. Here is my dependency links definition;

dependency_links=[
    "https://github.com/.../tarball/master/#egg=1.0.0",
    "https://github.com/.../tarball/master#egg=0.9.3",
], 

The links are valid. I can download them from a internet browser with these urls. These extra libraries are still not installed with my setting up. I also tried --process-dependency-links parameter to force pip. But result is same. I take no error when pipping.

After installation, I see no library in pip freeze result in dependency_links.

How can I make them to be downloaded with my setup.py installation?

Edited:

Here is my complete setup.py

from setuptools import setup

try:
    long_description = open('README.md').read()
except IOError:
    long_description = ''

setup(
    name='esef-sso',
    version='1.0.0.0',
    description='',
    url='https://github.com/egemsoft/esef-sso.git',
    keywords=["django", "egemsoft", "sso", "esefsso"],
    install_requires=[
        "Django",
        "webservices",
        "requests",
        "esef-auth==1.0.0.0",
        "django-simple-sso==0.9.3"
    ],
    dependency_links=[
        "https://github.com/egemsoft/esef-auth/tarball/master/#egg=1.0.0.0",
        "https://github.com/egemsoft/django-simple-sso/tarball/master#egg=0.9.3",
    ],

    packages=[
        'esef_sso_client',
        'esef_sso_client.models',
        'esef_sso_server',
        'esef_sso_server.models',
    ],
    include_package_data=True,
    zip_safe=False,
    platforms=['any'],
)

Edited 2:

Here is pip log;

Downloading/unpacking esef-auth==1.0.0.0 (from esef-sso==1.0.0.0)
  Getting page https://pypi.python.org/simple/esef-auth/
  Could not fetch URL https://pypi.python.org/simple/esef-auth/: 404 Client Error: Not Found
  Will skip URL https://pypi.python.org/simple/esef-auth/ when looking for download links for esef-auth==1.0.0.0 (from esef-sso==1.0.0.0)
  Getting page https://pypi.python.org/simple/
  URLs to search for versions for esef-auth==1.0.0.0 (from esef-sso==1.0.0.0):
  * https://pypi.python.org/simple/esef-auth/1.0.0.0
  * https://pypi.python.org/simple/esef-auth/
  Getting page https://pypi.python.org/simple/esef-auth/1.0.0.0
  Could not fetch URL https://pypi.python.org/simple/esef-auth/1.0.0.0: 404 Client Error: Not Found
  Will skip URL https://pypi.python.org/simple/esef-auth/1.0.0.0 when looking for download links for esef-auth==1.0.0.0 (from esef-sso==1.0.0.0)
  Getting page https://pypi.python.org/simple/esef-auth/
  Could not fetch URL https://pypi.python.org/simple/esef-auth/: 404 Client Error: Not Found
  Will skip URL https://pypi.python.org/simple/esef-auth/ when looking for download links for esef-auth==1.0.0.0 (from esef-sso==1.0.0.0)
  Could not find any downloads that satisfy the requirement esef-auth==1.0.0.0 (from esef-sso==1.0.0.0)
Cleaning up...
  Removing temporary dir /Users/ahmetdal/.virtualenvs/esef-sso-example/build...
No distributions at all found for esef-auth==1.0.0.0 (from esef-sso==1.0.0.0)
Exception information:
Traceback (most recent call last):
  File "/Users/ahmetdal/.virtualenvs/esef-sso-example/lib/python2.7/site-packages/pip/basecommand.py", line 122, in main
    status = self.run(options, args)
  File "/Users/ahmetdal/.virtualenvs/esef-sso-example/lib/python2.7/site-packages/pip/commands/install.py", line 278, in run
    requirement_set.prepare_files(finder, force_root_egg_info=self.bundle, bundle=self.bundle)
  File "/Users/ahmetdal/.virtualenvs/esef-sso-example/lib/python2.7/site-packages/pip/req.py", line 1177, in prepare_files
    url = finder.find_requirement(req_to_install, upgrade=self.upgrade)
  File "/Users/ahmetdal/.virtualenvs/esef-sso-example/lib/python2.7/site-packages/pip/index.py", line 277, in find_requirement
    raise DistributionNotFound('No distributions at all found for %s' % req)
DistributionNotFound: No distributions at all found for esef-auth==1.0.0.0 (from esef-sso==1.0.0.0)

It seems, it does not use the sources in dependency_links.

Answer 1

I faced a similar situation where I want to use shapely as one of my package dependency. Shapely, however, has a caveat that if you are using windows, you have to use the .whl file from http://www.lfd.uci.edu/~gohlke/pythonlibs/. Otherwise, you have to install a C compiler, which is something I don't want. I want the user to simply use pip install mypackage instead of installing a bunch of other stuffs.

And if you have the typical setup with dependency_links

setup(
  name = 'streettraffic',
  packages = find_packages(), # this must be the same as the name above
  version = '0.1',
  description = 'A random test lib',
  author = 'Costa Huang',
  author_email = '[e-mail]',
  install_requires=['Shapely==1.5.17'],
  dependency_links = ['http://www.lfd.uci.edu/~gohlke/pythonlibs/ru4fxw3r/Shapely-1.5.17-cp36-cp36m-win_amd64.whl']
)

and run python setup.py install, it is simply going to pick the shapely on Pypi and cause trouble on Windows installation. After hours of researching, I found this link Force setuptools to use dependency_links to install mysqlclient and basically use from setuptools.command.install import install as _install to manually install shapely.

from setuptools.command.install import install as _install
from setuptools import setup, find_packages
import pip

class install(_install):
  def run(self):
    _install.do_egg_install(self)

    # just go ahead and do it
    pip.main(['install', 'http://localhost:81/Shapely-1.5.17-cp36-cp36m-win_amd64.whl'])

setup(
  name = 'mypackage',
  packages = find_packages(), # this must be the same as the name above
  version = '0.1',
  description = 'A random test lib',
  author = 'Costa Huang',
  author_email = '[e-mail]',
  cmdclass={'install': install}
)

And the script works out nicely. Hope it helps.

Answer 2

You need to make sure you include the dependency in your install_requires too.

Here's an example setup.py

#!/usr/bin/env python
from setuptools import setup

setup(
    name='foo',
    version='0.0.1',
    install_requires=[
        'balog==0.0.7'
    ],
    dependency_links=[
        'https://github.com/balanced/balog/tarball/master#egg=balog-0.0.7'
    ]
)

Here's the issue with your example setup.py:

You're missing the egg name in the dependency links you setup.

You have

https://github.com/egemsoft/esef-auth/tarball/master/#egg=1.0.0.0

You need

https://github.com/egemsoft/esef-auth/tarball/master/#egg=esef-auth-1.0.0.0

Answer 3

Pip removed support for dependency_links a while back. The latest version of pip that supports dependency_links is 1.3.1, to install it

pip install pip==1.3.1

your dependency links should work at that point. Please note, that dependency_links were always the last resort for pip, ie. if a package with the same name exists on pypi it will be chosen over yours.

Note, https://github.com/pypa/pip/pull/1955 seems to start allowing dependency_links, pip kept it, but you might need to use some command line switches to use a newer version of pip.

EDIT: As of pip 7 ... they rethought dep links and have enabled them, even though they haven't removed the deprecation notice, from the discussions they seem to be here to stay. With pip>=7 here is how you can install things

pip install -e . --process-dependency-links --allow-all-external

Or add the following to a pip.conf, e.g. /etc/pip.conf

[install]
process-dependency-links = yes
allow-all-external = yes
trusted-host =
    bitbucket.org
    github.com

EDIT

A trick I have learnt is to bump up the version number to something really high to make sure that pip doesn't prefer the non dependency link version (if that is something you want). From the example above, make the dependency link look like:

"https://github.com/egemsoft/django-simple-sso/tarball/master#egg=999.0.0",

Also make sure the version either looks like the example or is the date version, any other versioning will make pip think its a dev version and wont install it.