pip installs package twice

问题:

Unfortunately I can't reproduce it, but we have seen it several times:

pip installs one packages twice.

If you uninstall the first, the second gets visible and can get uninstalled, too.

My question: How can I check with python if a package is installed twice?

Background: I want to write a test which checks this (devOp)

Update

• Packages are installed in a virtualenv.
• The two packages have different versions.
• This is not a duplicate of solutions which solve this by hand. I search a solution to detect this with python code. How to resolve this is not part if my question.

Update 2

The command pip freeze outputs the package only once:

pip freeze | grep -i south
South==0.8.1

But in the virtual-env it exists twice:

find lib -name top_level.txt |xargs cat | grep -i south
south
south

ls lib/python2.7/site-packages/| grep -i south
south
South-0.8.1-py2.7.egg
South-0.8.4-py2.7.egg-info

回答1:

This should work:

def count_installs(pkg_name):
    import imp, sys
    n = 0
    for location in sys.path:
        try:
            imp.find_module(pkg_name, [location])
        except ImportError: pass
        else: n += 1
    return n

e.g.

>>> count_installs("numpy")
2
>>> count_installs("numpyd")
0
>>> count_installs("sympy")
1

回答2:

I use this method to check if a package is installed twice:

def test_pip_python_packages_installed_twice(self):
    # https://stackoverflow.com/a/23941861/633961
    pkg_name_to_locations=defaultdict(set)
    for dist in pkg_resources.working_set:
        for pkg_name in dist._get_metadata('top_level.txt'):
            for location in sys.path:
                try:
                    importutils.does_module_exist_at_given_path(pkg_name, [location])
                except ImportError:
                    continue
                if location.startswith('/usr'):
                    # ignore packages from "root" level.
                    continue
                pkg_name_to_locations[pkg_name].add(location)

    errors=dict()
    for pkg_name, locations in sorted(pkg_name_to_locations.items()):
        if pkg_name in ['_markerlib', 'pkg_resources', 'site', 'easy_install', 'setuptools', 'pip']:
            continue
        if len(locations)==1:
            continue
        errors[pkg_name]=locations
    self.assertFalse(errors, 
                     'Some modules are installed twice:\n%s' % '\n'.join(['%s: %s' % (key, value) for key, value in sorted(errors.items())]))

importutils

def does_module_exist_at_given_path(module_name, path):
    '''
    imp.find_module() does not find zipped eggs.
    Needed for check: check if a package is installed twice.
    '''
    for path_item in path:
        result=None
        try:
            result=imp.find_module(module_name, [path_item])
        except ImportError:
            pass

        if result:
            return bool(result)
        if not os.path.isfile(path_item):
            continue
        # could be a zipped egg
        module=zipimport.zipimporter(path_item).find_module(module_name)
        if module:
            return bool(module)
    raise ImportError(module_name)

Related: imp.find_module() which supports zipped eggs

回答3:

South-0.8.1-py2.7.egg is a zip archive with South source code, South-0.8.4-py2.7.egg-info is a directory with metadata files for South library.

.egg-info (for libraries built from .tar.gz) or .dist-info (for libs installed from wheels .whl) are present for every library installed by pip.

.egg archive is created if the library marked as zip_safe in metadata (setup(zip_safe=True) in setup.py). Otherwise pip makes a directory with extracted python source files.

Very old versions of setuptools was able to install several versions of the same library and mark one of them as active, but mentioned functionality was dropped for years ago if I recall correctly.