tox is unable to use a range of python interpreters

[ssbarnea]

tox is unable to use a range of python interpreters

If basepython=python3 and the projects supports python3.5+, it will pick python3.4 if that is the default interepreter in the developer machine. If you set it to specific version, like python3.5 it will fail to run on systems that do only have python3.6 installed.

Use case scenario

We have Sample project which supports py27,py35,py36,py37 and its default envlist mentions all of them.

The project is also configured with ignore_missing_interpreters in order to allow contributors to run tests locally with their own python versions (unlikely that they will have all possible versions on python). CI is obviously configured to perform testing with all possible interpreters so is not really affected by the implicit envlist value.

We have two developers, John and Marry. John happens to use an operating system where python3 points to python3.4 (CentOS 7). He also installed additional newer interpreters which can be found by tox in PATH), like pythion3.5

Marry has a newer Fedora 30 system where python3 points to python3.6. She does not have any other interpreter installed.

We need to run tox -e lint environment using pyton3.5 or newer because flake8 requirement mention it. It does not matter if lint runs with py35, py36 or even py37 any of them being supported.

Currently there is no way to make tox run correctly for both developpers, even i both of them are using interpreters supported by that python project.

[gaborbernat]

So here's what I can envision here. What if we make basepython a list, so now you can say basepython = python3.7, python3.6, python3.5, python3.4. The first present wins. This only would work on non pyxy format envs. Furthermore, we can also add a convenience flag called basepython_is_latest_py = True (by default False) that sets basepython to the latest available pyxy factor available and listed inside envlist.

Opinions @asottile, @ssbarnea?

[ssbarnea]

@gaborbernat Having basepython a list sounds like a very good idea. It is a very easy to understand feature.

[asottile]

I wonder if it would be better to implement this in virtualenv

something like virtualenv -p '>=3.6' and have it auto discovery based on the "normal" ways (py for windows, and assume pythonX.X on posix?)

one issue with making basepython a list is backward compatibility where old tox expects a string -- I suspect the error message there would be rather unhelpful

[gaborbernat]

Maybe we can migrate to that later. Initially, I would like to battle test things within tox. No backwards compatibility issue for tox as python3.6'.split(',') works just great giving a list with a single element; however, this would be breaking for plugins 🤔 not sure how big of a deal this would be though 🤔

[asottile]

it's more that new configs + old versions of tox would give inscrutable error messages:

$ tox
python create: /tmp/t/.tox/python
ERROR: InterpreterNotFound: python3.6, python3.7
___________________________________ summary ____________________________________
ERROR:  python: InterpreterNotFound: python3.6, python3.7

[gaborbernat]

Easily fix-able via a good minversion though 👍

[astrojuanlu]

Easily fix-able via a good minversion though +1

There are use cases for maxversion as well. For example: I want an environment to use python3 as basepython, but I want it to be <3.9 because my project dependencies don't have yet wheels for Python 3.9.

[asottile]

@astrojuanlu minversion is tox's version -- I think you're a bit confused :)

[brechtm]

I ended up here via #1565 because I wanted to force using a particular interpreter when executing tox envs on GitHub Actions. I was able to sidestep the issue by adding a factor to my TOXENV variable based on the interpreter version configured for a particular job. This simply sets/modifies TOXENV based on the interpreter version. For example, when no TOXENV is set, it sets it to _pypy3_ when the _pypy-3.6_ or _pypy-3.7_ interpreter is selected for a job. If TOXENV is set to _regression_, it overwrites it with _regression-pypy3_.

Here's an example of my 'regression' env being executed on several interpreters. The TOXENV envvar is set/modified using an extra step in my workflow which calls a Python script.

EDIT: Oops, this was not executing my 'regression' testenv but rather the default one! Thankfully, An Opinionated tox.ini File seems to offer a workaround.

[asottile]

@brechtm you probably don't need to do that -- tox defaults to the python it's running in so as long as you actions/python and then install tox it'll reuse that same python for your unversioned envs

[brechtm]

@asottile I install tox (+ plugins and their dependencis) using pip3.9 (current Python) to avoid any issues specific to the interpreter I want to test against. For example, I'm using tox-poetry-installer which depends on poetry. The latter won't run on Python 3.10, but now I can still test my project against it.

[ssbarnea]

The problem is real and is becoming more visible with time. We clearly need an option like python range constraint for unpinned environments. With all @gaborbernat efforts focused on tox4 I think we will have to wait a bit more. Funny enough pre-commit suffers from the same limitation which makes some use specific cases PITA, forcing repo owner to use only a specific python version - very bad for platform portability of tests, as is almost impossible to find a pythonX.Y common between the major distros. Yeah, hardcode devs find a way (like pyenv) but for average contributor, it is a PITA.

Yes,for some

[gaborbernat]

tox 4 already contains this feature 👍🏻 As such I'll close this as done, and we can report bugs in new tickets.