Canonical replacement for --use-mirrors

Question

PyPI can be unreliable. I've had an unfortunate number of Travis-CI builds fail because pip fails to install one of my requirements (lxml is the most notorious offender).

Various online resources recommend the --use-mirrors flag, which has solved the issue for me thus far. However, --use-mirrors is deprecated for a number of good reasons.

Unfortunately, as mentioned in the link, one of the primary reasons for the removal of the flag is that the new CDN backed PyPI shouldn't have the same issues. It does. I still have issues with my builds, and I still can't reliably install packages with pip unless I use --use-mirrors.

The release notes for release 1.5 on 2014-01-01 recommend using one of the flags -i, -index-url, or --extra-index-url. Which is great, except... We run into some of the same issues that --use-mirrors had, namely that these mirrors can't necessarily be trusted.

The PyPI mirrors list has actually been removed, leaving us with some unofficial mirrors. Thus I'm left with a choice: keep using --use-mirrors and hope that one of the issues above is fixed before it is removed, or pick a mirror and hope it works and is trustworthy.

Is there a widely accepted and trusted mirror? Or a widely accepted and trusted alternative? Basically, how should I handle this problem?

Answer 1

Frankly, I have never faced the issue that you are describing - so I do not know what to do to resolve issues with the public pypi index.

However, as a general practice; I can recommend the following which is what we use when deploying (as the systems we deploy to do not have access to the Internet):

1. Create a local pypi mirror, and publish your packages there. You can do this in many ways. The simple approach with basket or you can do what we did and create your own pypi mirror (see: How to roll my own pypi? for some suggestions).

2. Use wheel. This is what we are migrating to because the install process is super simple and does not require reliance on other servers.

I know having a global pypi index is a great convenience but as part of a deployment build-chain I would use it as a backup; for one it is on a network I do not control (hence it may be unreachable or unreliable); and more importantly my systems may not need access to the Internet during the build process.