I am switching a project that currently uses pipenv
to poetry
as a test to see what the differences are. The project is a simple, redistributable, Django app. It supports Python 3.6-8, and Django 2.2 and 3.0. I have a tox.ini
file that covers all combinations of Python and Django thus:
[tox]
envlist = py{36,37,38}-django{22,30}
[testenv]
whitelist_externals = poetry
skip_install = true
deps =
django22: Django==2.2
django30: Django==3.0
commands =
poetry install -vvv
poetry run pytest --cov=my_app tests/
poetry run coverage report -m
The problem that I am having (which does not exist in the pipenv
world) is that the poetry install
statement will always overwrite whatever is in the deps
section with whatever is in the poetry.lock
file (which will be auto-generated if it does not exist). This means that the test matrix will never test against Django 2.2 - as each tox virtualenv gets Django 3.0 installed by default.
I don't understand how this is supposed to work - should installing dependencies using poetry
respect the existing environment into which it is being installed, or not?
So - my question is - how do I set up a multi-version tox (or travis) test matrix, with poetry as the dependency manager?
My pyproject.toml
defines Python / Django versions as:
[tool.poetry.dependencies]
python = "^3.6"
django = "^2.2 || ^3.0"
The generated poetry.lock
file (not committed) has this Django version information:
[[package]]
category = "main"
description = "A high-level Python Web framework that encourages rapid development and clean, pragmatic design."
name = "django"
optional = false
python-versions = ">=3.6"
version = "3.0"
UPDATE: include clean tox output
This is the result when I delete the lock file, and recreate the tox environment. As you can see, tox installs Django==2.2
as a dependency in the virtualenv, but poetry
then updates this to 3.0
when it installs.
I need a solution that runs the poetry install, respecting existing package installs. i.e. if pyproject.toml
states Django = "^2.2 || ^3.0"
, and 2.2 is already installed, then pin to that version - don't attempt to upgrade.
my-app$ tox -r -e py36-django22
py36-django22 recreate: .tox/py36-django22
py36-django22 installdeps: Django==2.2
py36-django22 installed: Django==2.2,my-app==0.1.0,pytz==2019.3,sqlparse==0.3.0
py36-django22 run-test: commands[0] | poetry install -vvv
Using virtualenv: .tox/py36-django22
Updating dependencies
Resolving dependencies...
1: derived: django (^2.2 || ^3.0)
...
PyPI: 10 packages found for django >=2.2,<4.0
...
1: Version solving took 3.330 seconds.
1: Tried 1 solutions.
Writing lock file
Package operations: 52 installs, 1 update, 0 removals, 3 skipped
- ...
- Updating django (2.2 -> 3.0)
- ...
UPDATE 2
Following instructions from sinoroc below - I have updated the tox file to remove skip_dist
and include isolated_build
. This works, sort of. tox builds the package, and installs it - but only the non-dev version, which does not include pytest
, coverage
and a host of linting tools that I'd like to include at a later point. i.e. the tools I want to run through tox are specified as dev-dependencies in poetry. There is a solution here, to include all of these inside the tox file - but that seems self-defeating - as then I have poetry and tox both declaring dependencies.
[tool.poetry.dependencies]
python = "^3.6"
django = "^2.2 || ^3.0"
[tool.poetry.dev-dependencies]
pytest = "^3.0"
pytest-cov = "^2.8"
pytest-django = "^3.7"
coverage = "^4.5"
pylint = "^2.4"
pylint-django = "^2.0"
flake8 = "^3.7"
flake8-bandit = "^2.1"
flake8-docstrings = "^1.5"
isort = "^4.3"
mypy = "^0.750.0"
pre-commit = "^1.20"
black = "=19.3b0"
UPDATE 3: solution
[tox]
isolated_build = True
envlist = lint, mypy, py{36,37,38}-django{22,30}
[travis]
python =
3.6: lint, mypy, py36
3.7: lint, mypy, py37
3.8: lint, mypy, py38
[testenv]
deps =
pytest
pytest-cov
pytest-django
coverage
django22: Django==2.2
django30: Django==3.0
commands =
django-admin --version
pytest --cov=my_app tests/
[testenv:lint]
deps =
pylint
pylint-django
flake8
flake8-bandit
flake8-docstrings
isort
black
commands =
isort --recursive my_app
black my_app
pylint my_app
flake8 my_app
[testenv:mypy]
deps =
mypy
commands =
mypy my_app