Reference requirements.txt for the install_require

Question

I have a requirements.txt file that I'm using with Travis-CI. It seems silly to duplicate the requirements in both requirements.txt and setup.py, so I was hoping to pass a file handle to the install_requires kwarg in setuptools.setup.

Is this possible? If so, how should I go about doing it?

Here is my requirements.txt file:

guessit>=0.5.2
tvdb_api>=1.8.2
hachoir-metadata>=1.3.3
hachoir-core>=1.3.3
hachoir-parser>=1.3.4

Answer 1

Requirements files use an expanded pip format, which is only useful if you need to complement your setup.py with stronger constraints, for example specifying the exact urls some of the dependencies must come from, or the output of pip freeze to freeze the entire package set to known-working versions. If you don't need the extra constraints, use only a setup.py. If you feel like you really need to ship a requirements.txt anyway, you can make it a single line:

.

It will be valid and refer exactly to the contents of the setup.py that is in the same directory.

Answer 2

from pip.req import parse_requirements did not work for me and I think it's for the blank lines in my requirements.txt, but this function does work

def parse_requirements(requirements):
    with open(requirements) as f:
        return [l.strip('\n') for l in f if l.strip('\n') and not l.startswith('#')]

reqs = parse_requirements(<requirements_path>)

setup(
    ...
    install_requires=reqs,
    ...
)

Answer 3

Most of the other answers above don't work with the current version of pip's API. Here is the correct* way to do it with the current version of pip (6.0.8 at the time of writing, also worked in 7.1.2. You can check your version with pip -V).

from pip.req import parse_requirements
from pip.download import PipSession

install_reqs = parse_requirements(<requirements_path>, session=PipSession())

reqs = [str(ir.req) for ir in install_reqs]

setup(
    ...
    install_requires=reqs
    ....
)

* Correct, in that it is the way to use parse_requirements with the current pip. It still probably isn't the best way to do it, since, as posters above said, pip doesn't really maintain an API.

Answer 4

Yet another parse_requirements hack that also parses environment markers into extras_require:

from collections import defaultdict
from pip.req import parse_requirements

requirements = []
extras = defaultdict(list)
for r in parse_requirements('requirements.txt', session='hack'):
    if r.markers:
        extras[':' + str(r.markers)].append(str(r.req))
    else:
        requirements.append(str(r.req))

setup(
    ...,
    install_requires=requirements,
    extras_require=extras
)

It should support both sdist and binary dists.

As stated by others, parse_requirements has several shortcomings, so this is not what you should do on public projects, but it may suffice for internal/personal projects.

Answer 5

It can't take a file handle. The install_requires argument can only be a string or a list of strings.

You can, of course, read your file in the setup script and pass it as a list of strings to install_requires.

import os
from setuptools import setup

with open('requirements.txt') as f:
    required = f.read().splitlines()

setup(...
install_requires=required,
...)

Answer 6

Here is a complete hack (tested with pip 9.0.1) based on Romain's answer that parses requirements.txt and filters it according to current environment markers:

from pip.req import parse_requirements

requirements = []
for r in parse_requirements('requirements.txt', session='hack'):
    # check markers, such as
    #
    #     rope_py3k    ; python_version >= '3.0'
    #
    if r.match_markers():
        requirements.append(str(r.req))

print(requirements)