Tox: Documentation how per-platform commands work needs improvement

[keymone]

python version: 3.6
python sys.platform: darwin
tox version: 3.5.3

tox.ini content:

[tox]
envlist=py36
skipsdist=True

[testenv]
platform= macos: darwin
          linux: linux

commands=
  macos: echo macos
  linux: echo linux

tox invocation:

~/dev/tox-test % tox
py36 create: ~/dev/tox-test/.tox/py36
py36 run-test-pre: PYTHONHASHSEED='3999618367'
________________________________________________________________________________________ summary ________________________________________________________________________________________
  py36: commands succeeded
  congratulations :)

[obestwalter]

Hi @keymone. I see where this is confusing.

The docs say:

A testenv can define a new platform setting as a regular expression. If a non-empty expression is defined and does not match against the sys.platform string the test environment will be skipped.

That's not really much to work with, so we need to improve that.

As you can see in your invocation the environment created is called py36. What isn't really clear from the docs is that tox checks the name of the testenv against the platform and as there is nothing that fits it will simply not run any of the commands.

If you run tox against your ini like this you can see how it works:

$ tox -e macos,linux

linux installed: play==0.0.0
linux runtests: PYTHONHASHSEED='1353739769'
linux runtests: commands[0] | echo linux
WARNING:test command found but not installed in testenv
  cmd: /usr/bin/echo
  env: /home/ob/do/play/.tox/linux
Maybe you forgot to specify a dependency? See also the whitelist_externals envconfig setting.
linux
_______________________________________ summary _______________________________________
SKIPPED:  macos: platform mismatch
  linux: commands succeeded
  congratulations :)

As I am running on linux it uses that one and skips the macos env.

The example has an envlist set that assures this when running without a qualifier but it also obscures how this actually works, so definitely a documentation bug.

[FirePing32]

@keymone Can I work on this ?

[gaborbernat]

Please go ahead.

[FirePing32]

@obestwalter @gaborbernat So just running tox will give the same invocation as given by @keymone without running any commands but running tox -e macOS, linux will run commands but gives an error along with it.

[FirePing32]

@obestwalter @gaborbernat Please see #1753