Tox: skip missing interpreters should be a toggle-able flag

[gaborbernat]

It makes sense to set skip missing interpreters inside the tox file to ensure that users locally don't get failures for interpreters they don't have. In this cases, it should be still possible to override that and force not skipping missing interpreters in the CI environment for example.

[anthrotype]

Yeah, I like the idea of being able to set skip_missing_interpreters to false, but only when running in a CI environment. Maybe some command-line option or environment variable could be used to override the default setting in the tox.ini.

[gaborbernat]

Yeah, that was my thought too :+1: I might go for the command line version as that's more explicit I believe.

[obestwalter]

I think I am missing the point here. As far as I understand, I can call tox --skip-missing-interpreters already to get that effect and I can also configure it as a global option in tox.ini: http://tox.readthedocs.io/en/latest/config.html#confval-skip_missing_interpreters=BOOL - if this behaviour needs to be made more malleable/powerful, I would like to understand first what else is needed and by whom.

tox configuration specification — tox 3.1.2.dev5 documentation

[gaborbernat]

@obestwalter the fact is --skip-missing-interpreters is often what people want, so they set it more often than not to true via the config (we do this ourselves). The problem is once this is set to true inside the config there's no way at the moment to unset it at invocation side.

A simple use case of this is that if you set skip-missing-interpreters to true inside the config and then miss-configure your CI (so that a given Python is not available) the CI will succeed even though the tests could not be run (because you'll just fail).

I think the root problem here is that skip missing interpreters should really default to true. And then have a flag of fail-on-missing-interpreters that the people can use in the CI env. However, this would break our backward compatibility so what I envision is skip-missing-interpreters[=1] so that inside the CI people can do skip-missing-interpreters=0 to force failure if a given Python is missing.

[obestwalter]

Now I understand - thanks. I like your solution also. Keeps the default, but would make the config setting overridable by the commandline then.