How to get “python -m venv” to directly install la

Question

As part of the compilation step for a new python version, I fetch and run get-pip.py, to have the latest pip installed next to the python executable:

$ /opt/python/3.7.0/bin/python --version
Python 3.7.0
$ /opt/python/3.7.0/bin/pip --version
pip 18.0 from /opt/python/3.7.0/lib/python3.7/site-packages/pip (python 3.7)

I have 25 such versions under /opt/python, although I mostly use the five latest versions of each major.minor version that is not EOL. To setup an invorment I used to run virtualenv or my virtualenvutils with the -p /opt/python/X.Y.Z/bin/python option to get a virtual environment with a specific version.

With Python 3.7 this gives the imp module deprecation warning:

$ virtualenv -p /opt/python/3.7.0/bin/python /tmp/py37virtualenv
Running virtualenv with interpreter /opt/python/3.7.0/bin/python
Using base prefix '/opt/python/3.7.0'
/opt/util/virtualenvutils/lib/python3.6/site-packages/virtualenv.py:1041: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
  import imp
New python executable in /tmp/py37virtualenv/bin/python
Installing setuptools, pip, wheel...done.

I have little hope this will be solved in virtualenv, as this has had a PendingDeprecationWarning at least since 2014 (as can be seen from the output in this question)

While investigating replacing virtualenv with python -m venv in virtualenvutils, I first created a new venv based virtual environment by hand:

$ /opt/python/3.7.0/bin/python -m venv /tmp/py37venv
$ /tmp/py37venv/bin/pip --version
pip 10.0.1 from /tmp/py37venv/lib/python3.7/site-packages/pip (python 3.7)

That has an old pip version! If you use it, you'll get:

You are using pip version 10.0.1, however version 18.0 is available.
You should consider upgrading via the 'pip install --upgrade pip' command

In the virtual environment created with virtualenv you immediately get the latest version:

$ /tmp/py37virtualenv/bin/pip --version
pip 18.0 from /tmp/py37virtualenv/lib/python3.7/site-packages/pip (python 3.7)

I can run a post-creation step:

/tmp/py37venv/bin/pip install -U --disable-pip-version-check pip 

which will take extra time. And if there was a some security update for pip, this would imply running the non-secure version to get a secure version, an ideal point of attack.

From virtualenvutils it is trivial to do the multiple steps to create a pip-less virtualenv and then add pip using get-pip.py. From the command-line this is not so simple:

$ /opt/python/3.7.0/bin/python -m venv --without-pip /tmp/py37venvnopip
$ /tmp/py37venvnopip/bin/python -c "from  urllib.request import urlopen; response = urlopen('https://bootstrap.pypa.io/get-pip'); open('/tmp/tmp_get_pip.py', 'w').write(response.read())"
$ /opt/python/3.7.0/bin/python /tmp/tmp_get_pip.py
......
$ /opt/python/3.7.0/bin/pip --version

pip 18.0 from /opt/python/3.7.0/lib/python3.7/site-packages/pip (python 3.7)

What is causing /opt/python/3.7.0/bin/python -m venv to take that old pip version? Is that the version available when 3.7.0 was released?

How can I update my install under /opt/python/3.7.0 in some way so that using /opt/python/3.7.0/bin/python -m venv creates a virtualenv with the latest pip version without reverting to scripts, aliases or using multiple commands? Having the latest pip installed under /opt/python/3.7.0 obviously is not enough.

There are two bundled wheels:

/opt/python/3.7.0/lib/python3.7/ensurepip/_bundled/setuptools-39.0.1-py2.py3-none-any.whl
/opt/python/3.7.0/lib/python3.7/ensurepip/_bundled/pip-10.0.1-py2.py3-none-any.whl

I suspect I need to update those. Is there a better way than updating those by hand? Some option for /some/python -m venv would be nice.

(And running /some/python -m ensurepip --upgrade doesn't do the trick)

---

Running the deprecated /opt/python/3.7.0/bin/pyvenv has the same old pip version problem.

Answer 1

It is an expected behavior. python -m venv calls python -m ensurepip to install pip and This answer shows that ensurepip would only install the bundled version even with --upgrade option. There isn't any official option to update the bundled pip and setuptools.

Well I have also no good idea to fix this problem as it just is the designed behavior. I would like to give two suggestions:

1. Use pipenv. It is really good! And it will be the next-generation official package manager in the future(Although there is a big problem related to current Pypi's structure. In short, a package manager can only decide the dependencies with downloading the whole package. This gives a huge difficulty to building dependencies graph.).

2. Implement your custom EnvBuilder, actually there is an official example about this. And in the example, it also use get-pip.py to install the latest pip.