Check if my Python has all required packages

Question

I have a requirements.txt file with a list of packages that are required for my virtual environment. Is it possible to find out whether all the packages mentioned in the file are present. If some packages are missing, how to find out which are the missing packages?

Answer 1

You can create a virtualenv with access to the system site packages and test check whether the package (or another dependencies) are installed or not. This way the packages are not really installed (if you just want to check). An example using virtualenv wrapper would be:

$ cat requirements.txt 
requests
simplejson

$ mkvirtualenv --system-site-packages test
Running virtualenv with interpreter /usr/bin/python2
New python executable in test/bin/python2
Also creating executable in test/bin/python
Installing setuptools, pip...done.

$ pip install -r requirements.txt
Downloading/unpacking requests (from -r requirements.txt (line 1))
  Downloading requests-2.10.0-py2.py3-none-any.whl (506kB): 506kB downloaded
Requirement already satisfied (use --upgrade to upgrade): simplejson in /usr/lib/python2.7/dist-packages (from -r requirements.txt (line 2))
Installing collected packages: requests
Successfully installed requests
Cleaning up...

$ pip install -r requirements.txt
Requirement already satisfied (use --upgrade to upgrade): requests in /home/yucer/.virtualenvs/test/lib/python2.7/site-packages (from -r requirements.txt (line 1))
Requirement already satisfied (use --upgrade to upgrade): simplejson in /usr/lib/python2.7/dist-packages (from -r requirements.txt (line 2))
Cleaning up...

$ deactivate

$ rmvirtualenv test
Removing test...

Answer 2

The pythonic way of doing it is via the pkg_resources API. The requirements are written in a format understood by setuptools. E.g:

Werkzeug>=0.6.1
Flask
Django>=1.3

The example code:

import pkg_resources
from pkg_resources import DistributionNotFound, VersionConflict

# dependencies can be any iterable with strings, 
# e.g. file line-by-line iterator
dependencies = [
  'Werkzeug>=0.6.1',
  'Flask>=0.9',
]

# here, if a dependency is not met, a DistributionNotFound or VersionConflict
# exception is thrown. 
pkg_resources.require(dependencies)

Answer 3

You can run pip freeze to see what you have installed and compare it to your requirements.txt file.

If you want to install missing modules you can run pip install -r requirements.txt and that will install any missing modules and tell you at the end which ones were missing and installed.

Answer 4

If requirements.txt is like :

django
oursql
sys
notexistingmodule

Then the following script will tell you which modules are missing :

#!/usr/bin/python3
fname = 'requirements.txt'
with open(fname, 'r', encoding='utf-8') as fhd:
    for line in fhd:
        try:
            exec("import " + line)
        except:
            print("[ERROR] Missing module:", line)

This would print :

[ERROR] Missing module: notexistingmodule

Answer 5

Based on the answer by Zaur, assuming you indeed use a requirements.txt file, you may want a unit test, perhaps in tests/test_requirements.py, that confirms the availability of packages.

This approach uses a subtest to independently confirm each requirement. This is useful so that all failures are documented.

from pathlib import Path
import unittest

import pkg_resources


class TestRequirements(unittest.TestCase):
    def test_requirements(self):
        """Recursively confirm that requirements are available."""
        # Ref: https://stackoverflow.com/a/45474387/
        requirements = (Path(__file__).parents[1] / 'requirements.in').read_text().strip().split('\n')
        requirements = [r.strip() for r in requirements]
        requirements = [r for r in sorted(requirements) if r and not r.startswith('#')]
        for requirement in requirements:
            with self.subTest(requirement=requirement):
                pkg_resources.require(requirement)