Upgrade python in a virtualenv

Question

Is there a way to upgrade the version of python used in a virtualenv (e.g. if a bugfix release comes out)?

I could pip freeze --local > requirements.txt, then remove the directory and pip install -r requirements.txt, but this requires a lot of reinstallation of large libraries, for instance, numpy, which I use a lot.

I can see this is an advantage when upgrading from, e.g., 2.6 -> 2.7, but what about 2.7.x -> 2.7.y?

Answer 1

I wasn't able to create a new virtualenv on top of the old one. But there are tools in pip which make it much faster to re-install requirements into a brand new venv. Pip can build each of the items in your requirements.txt into a wheel package, and store that in a local cache. When you create a new venv and run pip install in it, pip will automatically use the prebuilt wheels if it finds them. Wheels install much faster than running setup.py for each module.

My ~/.pip/pip.conf looks like this:

[global]
download-cache = /Users/me/.pip/download-cache
find-links =
/Users/me/.pip/wheels/

[wheel]
wheel-dir = /Users/me/.pip/wheels

I install wheel (pip install wheel), then run pip wheel -r requirements.txt. This stores the built wheels in the wheel-dir in my pip.conf.

From then on, any time I pip install any of these requirements, it installs them from the wheels, which is pretty quick.

Answer 2

For everyone with the problem

Error: Command '['/Users/me/Sites/site/venv3/bin/python3', '-Im', 'ensurepip', '--upgrade', '--default-pip']' returned non-zero exit status 1.

You have to install python3.6-venv

 sudo apt-get install python3.6-venv

Answer 3

How to upgrade the Python version for an existing virtualenvwrapper project and keep the same name

I'm adding an answer for anyone using Doug Hellmann's excellent virtualenvwrapper specifically since the existing answers didn't do it for me.

Some context:

• I work on some projects that are Python 2 and some that are Python 3; while I'd love to use python3 -m venv, it doesn't support Python 2 environments
• When I start a new project, I use mkproject which creates the virtual environment, creates an empty project directory, and cds into it
• I want to continue using virtualenvwrapper's workon command to activate any project irrespective of Python version

Directions:

Let's say your existing project is named foo and is currently running Python 2 (mkproject -p python2 foo), though the commands are the same whether upgrading from 2.x to 3.x, 3.6.0 to 3.6.1, etc. I'm also assuming you're currently inside the activated virtual environment.

1. Deactivate and remove the old virtual environment:

$ deactivate
$ rmvirtualenv foo

Note that if you've added any custom commands to the hooks (e.g., bin/postactivate) you'd need to save those before removing the environment.

2. Stash the real project in a temp directory:

$ cd ..
$ mv foo foo-tmp

3. Create the new virtual environment (and project dir) and activate:

$ mkproject -p python3 foo

4. Replace the empty generated project dir with real project, change back into project dir:

$ cd ..
$ mv -f foo-tmp foo
$ cdproject

5. Re-install dependencies, confirm new Python version, etc:

$ pip install -r requirements.txt
$ python --version

If this is a common use case, I'll consider opening a PR to add something like $ upgradevirtualenv / $ upgradeproject to virtualenvwrapper.

Answer 4

I just want to clarify, because some of the answers refer to venv and others refer to virtualenv.

Use of the -p or --python flag is supported on virtualenv, but not on venv. If you have more than one Python version and you want to specify which one to create the venv with, do it on the command line, like this:

malikarumi@Tetuoan2:~/Projects$ python3.6 -m venv {path to pre-existing dir you want venv in}

You can of course upgrade with venv as others have pointed out, but that assumes you have already upgraded the Python that was used to create that venv in the first place. You can't upgrade to a Python version you don't already have on your system somewhere, so make sure to get the version you want, first, then make all the venvs you want from it.

Answer 5

On OS X or macOS using Homebrew to install and upgrade Python3 I had to delete symbolic links before python -m venv --upgrade ENV_DIR would work.

I saved the following in upgrade_python3.sh so I would remember how months from now when I need to do it again:

brew upgrade python3
find ~/.virtualenvs/ -type l -delete
find ~/.virtualenvs/ -type d -mindepth 1 -maxdepth 1 -exec python3 -m venv --upgrade "{}" \;

UPDATE: while this seemed to work well at first, when I ran py.test it gave an error. In the end I just re-created the environment from a requirements file.

Answer 6

Did you see this? If I haven't misunderstand that answer, you may try to create a new virtualenv on top of the old one. You just need to know which python is going to use your virtualenv (you will need to see your virtualenv version).

If your virtualenv is installed with the same python version of the old one and upgrading your virtualenv package is not an option, you may want to read this in order to install a virtualenv with the python version you want.

EDIT

I've tested this approach (the one that create a new virtualenv on top of the old one) and it worked fine for me. I think you may have some problems if you change from python 2.6 to 2.7 or 2.7 to 3.x but if you just upgrade inside the same version (staying at 2.7 as you want) you shouldn't have any problem, as all the packages are held in the same folders for both python versions (2.7.x and 2.7.y packages are inside your_env/lib/python2.7/).

If you change your virtualenv python version, you will need to install all your packages again for that version (or just link the packages you need into the new version packages folder, i.e: your_env/lib/python_newversion/site-packages)