Poetry: How to run tox and poetry together against multiple dependency versions

[hugorodgerbrown]

• [x] I have searched the issues of this repo and believe that this is not a duplicate.
• [x] I have searched the documentation and believe that my question is not covered.

Issue

I have a question around the use of poetry and tox, when testing a library against a matrix of supported dependencies. My current example is a Django app, which I would like to test against Django 2.2. and 3.0. The tox.ini config is as below (taken from the docs - https://python-poetry.org/docs/faq/#is-tox-supported):

[tox]
envlist = py{36,37,38}-django{22,30}

[testenv]
deps =
    django22: Django==2.2
    django30: Django==3.0

whitelist_externals = poetry
skip_install = true

commands =
    poetry install -vvv
    poetry run django-admin --version
    poetry run pytest tests/

My pyproject.toml file lists the Django dependency as:

[tool.poetry.dependencies]
python = "^3.6"
django = "^2.2 || ^3.0"

When running tox, the version number output for all test runs is 3.0. It looks like the lock file is created on the first test run, and then re-used for the rest, and that the lock file is always the latest version of Django (3.0), even when 2.2 is already installed.

I'm a bit stuck at this point, as without being able to test against a matrix of versions I can't progress. Any help gratefully received. (I am also happy to add a docs PR once I've worked out the solution, as I can't be the only person with this issue?)

---

Update: adding output from a test run that shows a.) Django 2.2 being installed by tox, and then b.) poetry overwriting it.

my-app$ tox -r -e py36-django22
py36-django22 recreate: .tox/py36-django22
py36-django22 installdeps: Django==2.2
py36-django22 installed: Django==2.2,my-app==0.1.0,pytz==2019.3,sqlparse==0.3.0
py36-django22 run-test: commands[0] | poetry install -vvv
Using virtualenv: .tox/py36-django22
Updating dependencies
Resolving dependencies...
   1: derived: django (^2.2 || ^3.0)
   ...
PyPI: 10 packages found for django >=2.2,<4.0
   ...
   1: Version solving took 3.330 seconds.
   1: Tried 1 solutions.

Writing lock file

Package operations: 52 installs, 1 update, 0 removals, 3 skipped

  - ...
  - Updating django (2.2 -> 3.0)
  - ...

[hugorodgerbrown]

Cross-posted to StackOverflow - https://stackoverflow.com/q/59377071/45698

[Mojeer0]

Same issue, I'm to test against django 1.11 up to 3.0. But poetry always override the specified deps. One way to solve this would be to allow specifying a version to poetry install as long as it doesn't collide with the poetry.toml
In tox we could then use install_command = poetry install {packages}

[sinoroc]

I made a suggestion to hopefully solve this kind of issues on the _StackOverflow_ question as well as on the issue #1941.

Since _tox_ already takes care of installing the project and its dependencies in virtual environments, there is no need to get _poetry_ involved. Except to build the _sdist_, which is exactly what isolated_build does.
Now the next issue is that _tox_ doesn't know about _poetry_'s dev-dependencies. A pattern I have seen commonly used to remedy this in other contexts, is to list such dependencies as an _extra_.

So it could look like the following, note how the _extra_ test replaces the dev-dependencies and how the commands do not mention _poetry_ at all:

tox.ini

[tox]
envlist = py{36,37,38}-django{22,30}
isolated_build = True

[testenv]
# ...
deps =
    django22: Django==2.2
    django30: Django==3.0
extras =
    test
commands =
    pytest

pyproject.toml

[tool.poetry]
# ...

[tool.poetry.dependencies]
python = "^3.6"
django = "^2.2 || ^3.0"
#
pytest = { version = "^5.2", optional = true }

[tool.poetry.extras]
test = ["pytest"]

[build-system]
# ...

On the other hand, the drawback is that to get the development dependencies in the normal _poetry_ virtual environment an extra call to poetry install --extras 'test' is required.

[wesleykendall]

I got around this issue by exporting all dependencies, removing Django, and then using pip to install them:

[tox]
isolated_build = true
envlist = py{36,37,38}-django{22,30}

[testenv]
deps =
    django22: Django>=2.2,<2.3
    django30: Django>=3.0,<3.1
whitelist_externals =
  poetry
  bash
skip_install = true
commands =
    bash -c 'poetry export --dev --without-hashes -f requirements.txt | grep -v "^[dD]jango==" > .requirements.txt'
    poetry run pip install --no-deps -r .requirements.txt
    poetry run django-admin --version
    poetry run pytest dir/

There may be a nicer way to run commands with pipe in tox.ini other than bash -c, but this works for me for just running tests. No changes to my pyproject.toml either.

[sinoroc]

Would you mind expanding on the advantages of your solution?

Anyway... Unless I am missing something, I believe this could probably be simplified:

• isolated_build shouldn't be necessary, since the project is not buil by _tox_, because of skip_install;
• poetry run shouldn't be necessary, since _tox_ already does virtual environment isolation and _pip_ installs the dependencies.

Maybe something like that:

[tox]
envlist = py{36,37,38}-django{22,30}

[testenv]
deps =
    django22: Django>=2.2,<2.3
    django30: Django>=3.0,<3.1
whitelist_externals =
  bash
skip_install = true
commands_pre =
    bash -c \'poetry export --dev --without-hashes -f requirements.txt | grep -v "^[dD]jango==" > .requirements.txt\'
    python -m pip install --no-deps -r .requirements.txt
    python -m pip install .
    python -m django --version
commands =
    python -m pytest dir/

Probably the _bash_ part can be simplified as well.

[wesleykendall]

@sinoroc the main advantage is no need to change pyproject.toml. This is a work-around for those that don't want to maintain an "extras" part of pyproject.toml just for testing requirements, not to mention having to do poetry install --extras 'test' as mentioned previously. Of course one needs to keep in mind that pip install is not going to do conflict checking or anything else like poetry does. For my situation this doesn't matter since I have other checks in my continuous integration for this.

And yea, there are likely some ways to make the original code nicer. I just posted what was working for me after hacking around at it. You can also condense it more if you want to avoid storing a ".requirements.txt" file:

bash -c 'poetry export --dev --without-hashes -f requirements.txt | grep -v "^[dD]jango==" | poetry run pip install --no-deps -r /dev/stdin'
poetry run pytest ...

I leave poetry run prefixes in my examples because I have other parts of my continuous integration process after tox that use poetry for venv management. But yea you don't have to use poetry and can use normal pip

[sinoroc]

I see, but at this point, we are skipping so many of the advantages of _tox_, it might be worth questioning why use it at all. Oh by the way, you might want to add skipsdist = true as well.

But sure, no discussion about it, each project has its own specific needs that call for its own specific workflow. It's good we have the flexibility to adapt to each needs.

To me, the key element that pops out of this discussion, is that there is no straightforward way for _tox_ to get the list of _dev_ dependencies from _poetry_. And that is what is preventing a clean integration between these two tools.

[wesleykendall]

@sinoroc my suggestion is for people that don't want to use the previous suggestions. It doesn't skip many advantages of tox and is not that different than the original solutions. One could also say that the problem is also that poetry does not allow one to override dependencies in a poetry install, which is effectively what I'm doing by dynamically ignoring the Django poetry tries to install.

There's been quite a bit of noise since my previous suggestion. So, if you are coming to this thread and want a way to use tox in a poetry project without hacking your pyproject.toml, you can do this:

[tox]
isolated_build = true
envlist = py{36,37,38}-django{22,30}

[testenv]
deps =
    django22: Django>=2.2,<2.3
    django30: Django>=3.0,<3.1
whitelist_externals =
  poetry
  bash
skip_install = true
commands =
    bash -c 'poetry export --dev --without-hashes -f requirements.txt | grep -v "^[dD]jango==" | poetry run pip install --no-deps -r /dev/stdin'
    # An example command to show that it is the right Django version
    poetry run django-admin --version
    poetry run pytest dir/

[webartifex]

@wesleykendall
I am in a similar situation. Question: what happens if someone clones my project with your solution but does not have poetry installed? IMO that should break then, right?

So, if I want my pyproject.toml and tox.ini as generic as possible, there is no way to avoid specifying the dev dependencies twice?

Proposed way forward: Ask the tox dev's to include an option such that tox also installs the pyproject.toml dev dependencies?

[jlumbroso]

@wesleykendall Your solution is the most elegant one I've seen so far IMHO.

Here's how I would strive for tighter uncoupling right now:

• Have the dependency export happen on commit. I.e., on commit, export dependencies from Poetry as done here with GitHub Actions. This generates both files requirements.txt and requirements-dev.txt.
• Instead of:
  shell bash -c 'poetry export --dev --without-hashes -f requirements.txt | grep -v "^[dD]jango==" | poetry run pip install --no-deps -r /dev/stdin'
  one can then to:
  shell bash -c 'cat requirements-dev.txt | grep -v "^[dD]jango==" | pip install --no-deps -r /dev/stdin'

@webartifex The above tweak should address your issue since it no longer requires poetry when cloning/testing the package.

Other issues re: tox/poetry: https://github.com/python-poetry/poetry/issues/848, https://github.com/python-poetry/poetry/issues/1745, https://github.com/python-poetry/poetry/issues/1941.

[sinoroc]

I published on _PyPI_ a plugin for _Tox_ to instruct it to install _Poetry_'s development dependencies in the test environments: tox-poetry-dev-dependencies. It's just a proof of concept. I didn't test much, only with a very simple project.

[johnthagen]

Edit: I've been using tox-poetry-installer and as of 0.5.0 it has been working very well for our team.

It would be helpful if there was some official docs at https://python-poetry.org/docs/faq/#is-tox-supported had some discussion about the issues around pyproject.toml dev-dependencies and tox.

So far, the best solution I've found that doesn't break tox's isolation by whitelisting poetry/bash/etc is to use @sinoroc's tox-poetry-dev-dependencies plugin. This allows using tox in a normal, isolated way without having to modify pyproject.toml. There are a few open issues (https://github.com/sinoroc/tox-poetry-dev-dependencies/issues/48), but as poetry-core progresses, these should be solvable.

[sinoroc]

I think a good course of action, would be for people here (i.e. people who have experience with tox and poetry), to make some clear condensed suggestion of what they would like to see in the documentation/FAQ.

• keep it short
• keep it non opinionated
• keep it general enough that anyone can use it without being surprising / counter-intuitive
• keep it simple for beginners
• make it meaningful so that it is clear what task the poetry+tox combination achieves (test against multiple Python versions and/or test against multiple versions of a dependency)
• avoid suggesting 3rd party tool (I do not think tox plugins should be in the FAQ)

Someone already made a pull request: https://github.com/python-poetry/poetry/pull/2416

Maybe people should go have a look at it, comment on it, make suggestions, etc.

[jlumbroso]

Belatedly, thank you @sinoroc, your package takes care of this need very nicely! 🙏🏻