Is there a way to list pip dependencies/requiremen

Question

Without going through with the installation, I want to quickly see all the packages that pip install would install.

Answer 1

IN CASE you have the packages already installed, this script can fetch all the dependencies from a requirements file by running the command pip show mentioned by @Sardathrion.

import commands

fil = open("requirements.txt")
for package_line in fil.readlines():
    if "==" in package_line:
        package = package_line.split("==")[0]
    elif "[" in package_line:
        package = package_line.split("[")[0]
    else:
        package = package_line

    output = commands.getoutput('pip show %s' % package)
    try:
        required = output.split("\n")[-1].split(":")[1]
    except Exception as e:
        required = ""
        print "error {} in package {}".format(e, package)

    if len(required) > 1:
        print package, "-- ****%s***" % required

Answer 2

The closest you can get with pip directly is by using the --no-install argument:

pip install --no-install <package>

For example, this is the output when installing celery:

Downloading/unpacking celery                                                                                   
  Downloading celery-2.5.5.tar.gz (945Kb): 945Kb downloaded
  Running setup.py egg_info for package celery

    no previously-included directories found matching 'tests/*.pyc'
    no previously-included directories found matching 'docs/*.pyc'
    no previously-included directories found matching 'contrib/*.pyc'
    no previously-included directories found matching 'celery/*.pyc'
    no previously-included directories found matching 'examples/*.pyc'
    no previously-included directories found matching 'bin/*.pyc'
    no previously-included directories found matching 'docs/.build'
    no previously-included directories found matching 'docs/graffles'
    no previously-included directories found matching '.tox/*'
Downloading/unpacking anyjson>=0.3.1 (from celery)
  Downloading anyjson-0.3.3.tar.gz
  Running setup.py egg_info for package anyjson

Downloading/unpacking kombu>=2.1.8,<2.2.0 (from celery)
  Downloading kombu-2.1.8.tar.gz (273Kb): 273Kb downloaded
  Running setup.py egg_info for package kombu

Downloading/unpacking python-dateutil>=1.5,<2.0 (from celery)
  Downloading python-dateutil-1.5.tar.gz (233Kb): 233Kb downloaded
  Running setup.py egg_info for package python-dateutil

Downloading/unpacking amqplib>=1.0 (from kombu>=2.1.8,<2.2.0->celery)
  Downloading amqplib-1.0.2.tgz (58Kb): 58Kb downloaded
  Running setup.py egg_info for package amqplib

Successfully downloaded celery anyjson kombu python-dateutil amqplib

Admittedly, this does leave some cruft around in the form of temporary files, but it does accomplish the goal. If you're doing this with virtualenv (which you should be), the cleanup is as easy as removing the <virtualenv root>/build directory.

Answer 3

Another option is to use a helper script similar to this one which uses the pip.req.parse_requirements API to parse requirements.txt files and a distutils.core.setup replacement to parse setup.py files.

Answer 4

The command pip install <package> --download <path> should be used, as mentioned in comments by @radtek, since as of 7.0.0 (2015-05-21), --no-install is removed from pip. This will download the dependencies needed into <path>.

Answer 5

The accepted answer is no longer relevant for more current versions of pip and does not give an immediate answer without perusing multiple comments so I am providing an updated answer.

This was tested with pip versions 8.1.2, 9.0.1 and 10.0.1.

To get the output without cluttering your current directory on Linux use

pip download [package] -d /tmp --no-binary :all:

-d tells pip the directory that download should put files in.

Better, just use this script with the argument being the package name to get only the dependencies as output:

#!/bin/sh

PACKAGE=$1
pip download $PACKAGE -d /tmp --no-binary :all: \
| grep Collecting \
| cut -d' ' -f2 \
| grep -Ev "$PACKAGE(~|=|\!|>|<|$)"

Also available here.

Answer 6

The answer by @Jmills is stellar. It has a bug in the negative matching which causes some dependencies to be missed. In order to ensure that a package is not marked as a dependency of itself, he included the line grep -v $PACKAGE, which also negatively matches any dependency with the original package name as a sub-string, so jupyter_core is not listed as a dependency of jupyter, for example.

For my use case, I found it useful to have an implementation in python code instead of a shell script. I haven't included the original bug, though anyone is free to add it back in if they would like. I've borrowed an stdout capturing context manager to hopefully make the dependency gathering more intuitive.

from cStringIO import StringIO
import sys
import pip

class Capturing(list):
    def __enter__(self):
        self._stdout = sys.stdout
        sys.stdout = self._stringio = StringIO()
        return self
    def __exit__(self, *args):
        self.extend(self._stringio.getvalue().splitlines())
        del self._stringio    # free up some memory
        sys.stdout = self._stdout

def get_dependencies(module_name):
    with Capturing() as out:
        pip.main(['download', module_name, '-d', '/tmp', '--no-binary', ':all:'])
    return [line.split(' ')[1] for line in out if 'Collecting' == line[:10]][1:]

In case you don't need the version numbers, those are easy enough to filter out.

import re

def module_name(module_name_with_version):
    return re.match('[^!<>=]*', module_name_with_version).group()