Tox: Tox envlist selects incorrect python versions

[labrys]

Minimal reproducible example and detailed description

If a Python version is not available for a defined environment in the envlist tox selects the default Python for some environments and fails with InterpreterNotFound for others. This should be consistent and fail for all versions of python that aren't available.

Prerequisites:

• Must have at least one version of Python installed (Python 3.7.0 in this example)
• Must __NOT__ have at least one of the affected Python environments installed
• Must __NOT__ have at least one of the unaffected Python environments installed
• Must have tox installed in the current environment

Reproduce:

Create the following tox.ini in the current directory:

[tox]
envlist =
;    Affected Python Environments (not comprehensive)
;    The following will not fail if they are unavailable
    py25
    py33
    py40
;    Unaffected Python environments (not comprehensive)
;    The expected "InterpreterNotFound" error will be displayed if they are unavailable
    py34
    py35

skipsdist = true

[testenv]
commands =
    python --version

Check current Python version

In this example Python 3.7.0 is the system default, your version may differ

>python --version
Python 3.7.0

Run test and review output

Only one success and one failure shown for brevity:

>python -m tox
python -m tox
py25 create: E:\src\toxin\.tox\py25
py25 runtests: PYTHONHASHSEED='50'
py25 runtests: commands[0] | python --version
Python 3.7.0
...
py34 create: E:\src\toxin\.tox\py34
ERROR: InterpreterNotFound: python3.4

Python 2.5 is not installed but an environment for Python 2.5 was created (py25 create: E:\src\toxin\.tox\py25) but is using the current system default Python 3.7.0

Python 3.4 is not installed and the expected error is displayed (ERROR: InterpreterNotFound: python3.4)

OS and pip list output

>python -c "import platform; print(platform.platform())"
Windows-10-10.0.17134-SP0

>py -0
Installed Pythons found by py Launcher for Windows
 -3.7-32 *
 -3.6-32
 -2.7-32

>pip list
packaging (17.1)
pip (9.0.1)
pluggy (0.6.0)
py (1.5.4)
pyparsing (2.2.0)
setuptools (28.8.0)
six (1.11.0)
tox (3.1.0)
virtualenv (16.0.0)

[obestwalter]

Hi @labrys, thanks for the detailed report. I can reproduce this with 3.0 on Linux.

[labrys]

@obestwalter While this is being looked into, would it also be possible to add a py3 environment which selects any available Python 3 version?

[gaborbernat]

@labrys it already does see https://tox.readthedocs.io/en/latest/example/basic.html?highlight=py2#a-simple-tox-ini-default-environments py3 is already supported. With the latest release we just made it work on Windows too http://tox.readthedocs.io/en/latest/changelog.html#features

Basic usage — tox 3.1.2.dev6 documentation

Changelog history — tox 3.1.2.dev6 documentation

[gaborbernat]

We've added Windows support with our lates release 3.1.0. See https://github.com/tox-dev/tox/issues/856 and https://tox.readthedocs.io/en/latest/changelog.html#features. Did you try with that?

[labrys]

@gaborbernat I double checked and I wasn't using the dev version of tox in my test when I checked py3. I'll check it later tonight.

[gaborbernat]

My bad, not the dev version, however, you'll need https://tox.readthedocs.io/en/3.1.1/changelog.html aka 3.1.1.

Changelog history — tox 3.1.1 documentation

[labrys]

Just updated to 3.1.1 and it looks like it works as advertised, the py3 environment is building as we speak.

[asottile]

914 should address the others

[gaborbernat]

Seems to me all issues here have been addressed.