tox4: materialising a set_env may depend on set_env

[nicoddemus]

Hi,

Just tried the new 4.0.0a2 of tox on pytest's repository (congrats on the rewrite!).

I tried py36-xdist environment, however the command executed was just pytest, rather than the expected pytest -n auto.

λ tox4 -e py36-xdist
py36-xdist: install_deps> .tox4\py36-xdist\Scripts\python.exe -I -m pip install -r d:\projects\pytest\requirements-879of98y.txt
.package: install_requires> .tox4\.package\Scripts\python.exe -I -m pip install setuptools-scm[toml]>=3.4 wheel setuptools>=42.0
.package: get_requires_for_build_sdist> python d:\projects\pytest\.env36\lib\site-packages\tox\util\pep517\backend.py True setuptools.build_meta
.package: install_build_requires> .tox4\.package\Scripts\python.exe -I -m pip install setuptools-scm>=3.4 setuptools>=>=42.0
.package: prepare_metadata_for_build_wheel> python d:\projects\pytest\.env36\lib\site-packages\tox\util\pep517\backend.py True setuptools.build_meta
py36-xdist: install_package_deps> .tox4\py36-xdist\Scripts\python.exe -I -m pip install pluggy<1.0.0a1,>=0.12 mock colorama iniconfig argcomplete hypothesis>=3.56 requests py>=1.8.2 attrs>=19.2.0 toml packaging atomicwrites>=1.0 nose xmlschema importlib-metadata>=0.12
.package: build_sdist> python d:\projects\pytest\.env36\lib\site-packages\tox\util\pep517\backend.py True setuptools.build_meta
py36-xdist: install_package> .tox4\py36-xdist\Scripts\python.exe -I -m pip install --no-deps --force-reinstall d:\projects\pytest\.tox4\.package\dist\pytest-6.2.2.dev10+g32b645fd9.tar.gz
py36-xdist: commands[0]> pytest

On tox==3.21.0, I get:

λ tox -e py36-xdist
.package recreate: d:\projects\pytest\.tox\.package
.package installdeps: setuptools>=42.0, setuptools-scm[toml]>=3.4, wheel
py36-xdist create: d:\projects\pytest\.tox\py36-xdist
py36-xdist installdeps: pygments>=2.7.2, pytest-xdist>=2.1.0, -e.
py36-xdist inst: d:\projects\pytest\.tox\.tmp\package\1\pytest-6.3.0.dev88+gf6529fd3a.tar.gz
py36-xdist installed: apipkg==1.5,argcomplete==1.12.2,atomicwrites==1.4.0,attrs==20.3.0,certifi==2020.12.5,chardet==4.0.0,colorama==0.4.4,elementpath==2.1.1,execnet==1.7.1,hypothesis==5.49.0,idna==2.10,importlib-metadata==3.3.0,iniconfig==1.1.1,mock==4.0.3,nose==1.3.7,packaging==20.8,pluggy==0.13.1,py==1.10.0,Pygments==2.7.3,pyparsing==2.4.7,pytest @ file:///D:/projects/pytest/.tox/.tmp/package/1/pytest-6.3.0.dev88%2Bgf6529fd3a.tar.gz,pytest-forked==1.3.0,pytest-xdist==2.2.0,requests==2.25.1,sortedcontainers==2.3.0,toml==0.10.2,typing-extensions==3.7.4.3,urllib3==1.26.2,xmlschema==1.4.1,zipp==3.4.0
py36-xdist run-test-pre: PYTHONHASHSEED='895'
py36-xdist run-test: commands[0] | pytest -n auto

Other than that it seems to work fine.

I used https://github.com/pytest-dev/pytest/commit/f6529fd3a14dc99939a372352652b443eaf38962.

Let me know if I can provide more information.

[gaborbernat]

Thanks for reporting. There's some fancy circular env-var dependency in here https://github.com/pytest-dev/pytest/blob/f6529fd3a14dc99939a372352652b443eaf38962/tox.ini#L21-L42, probably a bug in our env-var/pos-args manifestation. Will look into it 👏

[gaborbernat]

I think this is the part due to https://github.com/tox-dev/tox/issues/1779, in short: the set_env only influences what arguments you pass into the command, but the {env:} replacement only works on the caller shells environment variables, not the new ones defined within set_env. This will need to change somehow.

[nicoddemus]

Thanks!

This will need to change somehow.

Do you mean in our tox.ini or in tox?

[gaborbernat]

Within tox 4 ☺️

[gaborbernat]

It so seems I've not accounted for the possibility of set_env depending on set_env 🤦; this will require some fairly major rework within the configuration system. This was exactly the kind of bugs I've hoped to discover via early alpha testers. Thanks!

PS. #1779 is part of the reason, but things go deeper.

[gaborbernat]

Resolved via https://github.com/tox-dev/tox/pull/1784

[gaborbernat]

Released now via https://tox.readthedocs.io/en/rewrite/changelog.html#v4-0-0a3-2021-01-13

[nicoddemus]

Thanks for the ping @gaborbernat, I can confirm that it now works as expected.