Reinstall virtualenv with tox when requirements.tx

Question

Previously I was manually using a Makefile that looked something like this:

.PHONY: all
all: tests

.PHONY: tests
tests: py_env
    bash -c 'source py_env/bin/activate && py.test tests'

py_env: requirements_dev.txt setup.py
    rm -rf py_env
    virtualenv py_env
    bash -c 'source py_env/bin/activate && pip install -r requirements_dev.txt'

This had the nice side-effect that if I changed requirements_dev.txt or setup.py, it would rebuild my virtualenv. But feels a bit clunky.

I'd like to use tox to do a similar thing. I understand tox has a --recreate option, but I'd rather call that only when I need to.

My new setup is something like this:

# Makefile
.PHONY: all
all: tests

.PHONY: tests
tests:
    tox

and

# tox.ini
[tox]
project = my_project
envlist = py26,py27

[testenv]
install_command = pip install --use-wheel {opts} {packages}
deps = -rrequirements_dev.txt
commands =
    py.test {posargs:tests}

An ideal solution would use just things in tox, however an acceptable solution would involve the Makefile and the --recreate flag.

Answer 1

There seems to be an open issue in tox for just this problem.

https://github.com/tox-dev/tox/issues/149 (click and add your comment and vote, making the authors aware about how common the issue is)

We'll need to either submit a patch or work around it. Workaround that come to mind:

1. List dependencies directly in the tox.ini. Use your build system to ensure that the tox.ini stays in sync with the requirements.txt.
2. Add a rule to your Makefile that does a tox --recreate whenever the requirements.txt changes.

Workaround 2 seems most straightforward.

Answer 2

Here's the Makefile workaround I ended up going with:

REBUILD_FLAG =

.PHONY: all
all: tests

.PHONY: tests
tests: .venv.touch
    tox $(REBUILD_FLAG)

.venv.touch: setup.py requirements.txt requirements_dev.txt
    $(eval REBUILD_FLAG := --recreate)
    touch .venv.touch

Example:

$ make tests
touch .venv.touch
tox --recreate
[[ SNIP ]]
$ make tests
tox 
[[ SNIP ]]
$ touch requirements.txt
$ make tests
touch .venv.touch
tox --recreate
[[ SNIP ]]

Answer 3

Determined to solve this, I've written a tox plugin to accomplish this: https://github.com/asottile/tox-pip-extensions

The plugin hooks into virtualenv creation and uses venv-update to keep dependencies in sync.

The usage is pretty straightforward:

• install tox-pip-extensions alongside tox (in my setup, I have a virtualenv at ~/venv with tox and tox-pip-extensions installed, and then symlinked ~/venv/bin/tox -> ~/bin/tox)

• enable the extension as follows:

  [tox]
  tox_pip_extensions_ext_venv_update = true
  # the rest of your tox.ini file as normal...