Tox: 2.8.0: {[testenv]setenv} substitution broken

[yogi-sagar]

While installing tempest as part of devstack install, below commnd blows up with an error -
tox -r --notest -efull

Error -

2017-09-01 18:22:20.661 | ++ lib/tempest:install_tempest:612          :   pushd /opt/stack/new/tempest
2017-09-01 18:22:20.661 | ~/tempest ~/devstack
2017-09-01 18:22:20.662 | ++ lib/tempest:install_tempest:613          :   tox -r --notest -efull
2017-09-01 18:22:21.720 | Traceback (most recent call last):
2017-09-01 18:22:21.720 |   File "/usr/local/bin/tox", line 11, in <module>
2017-09-01 18:22:21.720 |     sys.exit(cmdline())
2017-09-01 18:22:21.720 |   File "/usr/local/lib/python2.7/dist-packages/tox/session.py", line 38, in main
2017-09-01 18:22:21.720 |     config = prepare(args)
2017-09-01 18:22:21.720 |   File "/usr/local/lib/python2.7/dist-packages/tox/session.py", line 26, in prepare
2017-09-01 18:22:21.720 |     config = parseconfig(args)
2017-09-01 18:22:21.720 |   File "/usr/local/lib/python2.7/dist-packages/tox/config.py", line 242, in parseconfig
2017-09-01 18:22:21.720 |     parseini(config, inipath)
2017-09-01 18:22:21.720 |   File "/usr/local/lib/python2.7/dist-packages/tox/config.py", line 796, in __init__
2017-09-01 18:22:21.721 |     replace=name in config.envlist)
2017-09-01 18:22:21.721 |   File "/usr/local/lib/python2.7/dist-packages/tox/config.py", line 827, in make_envconfig
2017-09-01 18:22:21.721 |     res = meth(env_attr.name, env_attr.default, replace=replace)
2017-09-01 18:22:21.721 |   File "/usr/local/lib/python2.7/dist-packages/tox/config.py", line 964, in getdict_setenv
2017-09-01 18:22:21.721 |     definitions = self._getdict(value, default=default, sep=sep)
2017-09-01 18:22:21.721 |   File "/usr/local/lib/python2.7/dist-packages/tox/config.py", line 975, in _getdict
2017-09-01 18:22:21.721 |     name, rest = line.split('=', 1)
2017-09-01 18:22:21.721 | ValueError: need more than 1 value to unpack
2017-09-01 18:22:21.733 | + lib/tempest:install_tempest:1            :   exit_trap

Logs link - http://logs.openstack.netapp.com/ci-logs/logs/65/499765/5/upstream-check/cinder-cDOT-FCP/3566c47/logs/devstacklog.txt.gz

[obestwalter]

Thanks @yogi-sagar - I will have a look right away.

[adamrothman]

@obestwalter This broke my builds as well (looks like the same error):

$ make local-test
ORBIT_AWS__SERVICES__DYNAMODB__ENDPOINT_URL=http://192.168.99.100:8000 \
    ORBIT_SERVICES__NEO4J__HOST=192.168.99.100 \
    ORBIT_SERVICES__REDIS__HOST=192.168.99.100 \
    AWS_ACCESS_KEY_ID=[REDACTED] AWS_SECRET_ACCESS_KEY=[REDACTED] \
    tox --recreate
Traceback (most recent call last):
  File "/usr/local/bin/tox", line 11, in <module>
    sys.exit(cmdline())
  File "/usr/local/lib/python3.6/site-packages/tox/session.py", line 38, in main
    config = prepare(args)
  File "/usr/local/lib/python3.6/site-packages/tox/session.py", line 26, in prepare
    config = parseconfig(args)
  File "/usr/local/lib/python3.6/site-packages/tox/config.py", line 242, in parseconfig
    parseini(config, inipath)
  File "/usr/local/lib/python3.6/site-packages/tox/config.py", line 796, in __init__
    replace=name in config.envlist)
  File "/usr/local/lib/python3.6/site-packages/tox/config.py", line 827, in make_envconfig
    res = meth(env_attr.name, env_attr.default, replace=replace)
  File "/usr/local/lib/python3.6/site-packages/tox/config.py", line 964, in getdict_setenv
    definitions = self._getdict(value, default=default, sep=sep)
  File "/usr/local/lib/python3.6/site-packages/tox/config.py", line 975, in _getdict
    name, rest = line.split('=', 1)
ValueError: not enough values to unpack (expected 2, got 1)
make: *** [local-test] Error 1

[adamrothman]

Here's my tox.ini, if it helps:

[tox]
envlist = py36


[testenv]
deps =
    -rrequirements.txt
    flake8
    mypy
    pyflakes>=1.6.0
    pytest
    pytest-asyncio
passenv = ASYNC_TEST_TIMEOUT AWS_ACCESS_KEY_ID AWS_DEFAULT_REGION AWS_SECRET_ACCESS_KEY ORBIT_AWS__SERVICES__DYNAMODB__ENDPOINT_URL ORBIT_SERVICES__NEO4J__HOST ORBIT_SERVICES__REDIS__HOST
setenv =
    ORBIT_ENV = test
    ASYNC_TEST_TIMEOUT = 10
    AWS_DEFAULT_REGION = us-west-2
commands =
    flake8 orbit tests
    mypy --ignore-missing-imports orbit tests
    pytest {posargs} tests/unit
    pytest {posargs} tests/integration


[testenv:docker]
deps =
    flake8
    mypy
    pyflakes>=1.6.0
    pytest
    pytest-asyncio
sitepackages = True


[testenv:verbose]
setenv =
    {[testenv]setenv}
    ORBIT_AWS__SERVICES__DYNAMODB__ENDPOINT_URL = http://192.168.99.100:8000
    ORBIT_SERVICES__NEO4J__HOST = 192.168.99.100
    ORBIT_SERVICES__REDIS__HOST = 192.168.99.100
commands =
    pytest -s -vv {posargs}


[testenv:lint]
commands =
    flake8 orbit tests
    mypy --ignore-missing-imports orbit tests


[flake8]
ignore = E501

[obestwalter]

{[testenv]setenv} breaks with the new changes looks like we don't have a test for this substitution behaviour yet. I will create a reproducer and see if we can fix that with a quick bug fix soon. Please stick with 2.7 until then.

[cboylan]

https://github.com/tox-dev/tox/pull/521#discussion-diff-125055520R791 is the source of the bug. Basically if you run tox -efoo then config.envlist == ['foo'] but tox will still attempt to parse envs bar and baz and foobar. But none of those will get replacements done beacuse those names are not in ['foo'].

[cboylan]

Digging in more I see two potential "fixes" for the original issue https://github.com/tox-dev/tox/issues/515. The first one is more of a workaround. Just set a default on all env vars (granted you may want to fail in some cases rather than relying on a default so maybe this isn't so great). The other is always replace internal tox key:value pairs and treat the replace flag as an indication of whether or not you should replace external key:value pairs. Or maybe just add another flag for external vs internal.

But I am not familiar with this code base so will defer to others.

[obestwalter]

minimal reproducer for this is:

[tox]
envlist = spam

[testenv]
setenv = DONTCARE = 0

[testenv:eggs]
setenv = {[testenv]setenv}

[obestwalter]

@cboylan - if you want to try for a fix, have a go - reproducer is in place - it's getting late here on this side of the planet, so I might have a look tomorrow :)

[cboylan]

This isn't a complete fix. The original tempest tox.ini now fails on missing boolean values when run against the commit above. I figured I'd throw something up before my day ends though.

[obestwalter]

Let's keep this open until the fix is out ...

[obestwalter]

I pushed a test package with the completed fix to devpi if somebody wants to test this:

pip install --index https://devpi.net/obestwalter/dev/+simple "tox==2.8.1.dev8+gda3a86a"

I tried already with tempest and that works again.

[obestwalter]

changed the implementation

pip install --index https://devpi.net/obestwalter/dev/+simple "tox==2.8.1.dev11+g64e9012"

[obestwalter]

Last dev version before release (I promise):

pip install --index https://devpi.net/obestwalter/dev/+simple "tox==2.8.1.dev13+g35e0dcc"

[obestwalter]

I think I will merge the fix and release a bugfix update today. If someone could review first it would be nice, but I am pretty confident that the fix does not make things worse ...

[obestwalter]

Fixed in 2.8.1

[adamrothman]

Thanks @obestwalter, I'll try 2.8.1 out tomorrow and let you know how it goes.