Tox: setup.cfg parsed differently on Jenkins

[rsokl]

This bug can be reproduced using this trivial repo: https://github.com/rsokl/tox-bug

Context

• Using a setup.cfg file instead of tox.ini
• Running tox on Jenkins (i.e. JENKINS_URL or HUDSON_URL are environment variables are set)

Problem

The documentation specifies that the setup.cfg file should contain a [tox:tox] section. However, on Jenkins, the file is parsed to look for a [tox] section instead.

Example

Consider the simple setup.cfg

# contents of setup.cfg
[tox:tox]  # does not work on Jenkins; [tox] does work                          
envlist = py37
toxworkdir = build/tox

The toxworkdir is set as-expected when tox is run locally (off-Jenkins):

>>> tox
GLOB sdist-make: /home/rsokl/tox_dummy/setup.py
py37 create: /home/rsokl/tox_dummy/build/tox/py37

Running this "on Jenkins", the [tox:tox] section is not found and toxworkdir fails to get set:

>>> env JENKINS_URL="" tox
GLOB sdist-make: /home/ry26099/tox_dummy/setup.py
python create: /home/ry26099/tox_dummy/.tox/python

Now we can modify the setup.cfg to contain a [tox] section, and find inverted behavior

# adapted setup.cfg
[tox]  # works on Jenkins; does not work locally                          
envlist = py37
toxworkdir = build/tox

Running locally, no tox section is found, which is to be expected since we deviate from the documented [tox:tox] form (note that the toxworkdir is not set):

>>> tox
GLOB sdist-make: /home/ry26099/tox_dummy/setup.py
python create: /home/ry26099/tox_dummy/.tox/python

But Jenkins does see this tox section (note that the toxworkdir is properly set):

>>> env JENKINS_URL="" tox
GLOB sdist-make: /home/ry26099/tox_dummy/setup.py
py37 create: /home/ry26099/tox_dummy/build/tox/py37

[asottile]

that is weird, it's probably due to the custom behaviour documented here

[daneah]

@asottile The documentation seems to state that [tox:jenkins] can override global configuration, implying that if you don't override anything you'll still get the global configuration. But when using setup.cfg, global configuration does not appear to be honored. I was able to confirm this by moving my (not working) [tox:tox] section from setup.cfg to a [tox] section in a tox.ini file and see it start working again. A workaround is to specify [tox:tox:jenkins] in setup.cfg, but this requires duplicating some config which is non-ideal (unless I'm missing an easy way to say "copy config from _____"?)

Hopefully this helps exemplify the issue:

# Running locally with setup.cfg:
#
# [tox:tox]
# envlist = py37

$ docker run -it --rm -v $(pwd):/code quay.io/python-devs/ci-image sh -c 'python3.7 -m pip install --user tox && cd /code && python3.7 -m tox -v -a'
using tox.ini: /code/tox.ini (pid 12)
using tox-3.14.1 from /home/runner/.local/lib/python3.7/site-packages/tox/__init__.py (pid 12)
default environments:
py37 -> [no description]

# Simulating Jenkins (by adding `-e JENKINS_URL=...`) with setup.cfg:
#
# [tox:tox]
# envlist = py37

$ docker run -it --rm -e JENKINS_URL='http://iam.jenkins' -v $(pwd):/code quay.io/python-devs/ci-image sh -c 'python3.7 -m pip install --user tox && cd /code && python3.7 -m tox -v -a'
using tox.ini: /code/tox.ini (pid 40)
using tox-3.14.1 from /home/runner/.local/lib/python3.7/site-packages/tox/__init__.py (pid 40)
additional environments:
python -> [no description]

# Simulating Jenkins after moving config to tox.ini:
#
# [tox]
# envlist = py37
#
# OR duplicating in setup.cfg:
#
# [tox:tox:jenkins]
# envlist = py37

$ docker run -it --rm -e JENKINS_URL='http://iam.jenkins' -v $(pwd):/code quay.io/python-devs/ci-image sh -c 'python3.7 -m pip install --user tox && cd /code && python3.7 -m tox -v -a'
using tox.ini: /code/tox.ini (pid 40)
using tox-3.14.1 from /home/runner/.local/lib/python3.7/site-packages/tox/__init__.py (pid 40)
additional environments:
py37 -> [no description]

[asottile]

oh yeah to be clear I'm pretty sure there's a bug here :) was hopefully giving a contributor a pointer to look for in the code

[daneah]

The TL;DR from my comment for the moment is that duplicating [tox:tox] to [tox:tox:jenkins] appears to be a sufficient workaround 😄

[gaborbernat]

Released as 3.14.2 - see https://tox.readthedocs.io/en/latest/changelog.html#v3-14-2-2019-12-02

Changelog history — tox 3.14.2.dev9 documentation

[daneah]

Thanks for the update @gaborbernat!