Tox: basepython + shared envdir + default environments = confusion

[obestwalter]

This came up in #425:

tox.ini

[tox]
envlist = py36,py27
[testenv]
basepython=python3.6
envdir = {toxworkdir}/sharedenvdir
commands = python --version

Leads to the py27 environment also be run with python3.6 which is very likely not intended by the user and a mistake in the configuration.

This might lead to unnecessary confusion and I wonder if we should catch that and throw an appropriate error, when we encounter configurations like that?

[asottile]

A big +1 on this one, even without envdir this is pretty confusing and a problem I've seen a lot

[tox]
envlist = py27,py36
[testenv]
basepython=python3.6
commands = python --version

Surprise! no longer testing against python2.7

I wrote a tool at my last employment which scanned all of the repositories and checked for pyXX environments that had a mismatched interpreter version, perhaps something of the sort could be baked into tox itself? (if it sees for instance a py27 environment but the interpreter endswith python3.6 it could error?). Unfortunately the tox-specific tool I wrote is lost to closed source :( (though I've rewritten the multi-repository scanner bit)

[alvations]

We're having the same problem. Somehow we can find the Python version on the terminal but tox can't https://stackoverflow.com/questions/48274339/where-did-tox-look-when-searching-for-basepython. And we've no idea where exactly is tox searching.

Is it possible to pass in information to tell tox to specifically look for the Python binaries at a specific location?

[gaborbernat]

@alvations tox uses the PATH environment variable, so try to set it to where to look for :+1:

[asottile]

@alvations seems this is probably too complicated to solve over stack overflow comment thread :) mind shooting me an email and we can try and work out iteratively to debug this (or if I can get ssh access -- I can probably find the bug directly). My initial suspect is the eval + inspect:
https://github.com/tox-dev/tox/blob/336f4f6bd8b53223f940fc5cfc43b1bbd78d4699/tox/interpreters.py#L61
https://github.com/tox-dev/tox/blob/336f4f6bd8b53223f940fc5cfc43b1bbd78d4699/tox/interpreters.py#L81

(we can most likely replace this with json since py25 is long since dead -- but I want to confirm that's the problem)

If this seems ok, shoot me an email: [email protected]

[alvations]

@asottile Thanks for extending the help! Sadly, the project owner (@stevenbird) isn't me so I don't have the rights to grant ssh access.

I've tried almost all combinations of cloudbees' jenkins and tox just isn't detecting the python3.4.

Is there additional information that I could provide that could help you to patch tox to find the interpreter?

Maybe one solution is to isolate one tox run per environment and force every environment to run an alias python=/scratch/.../bin/python3 and use basepython=3

[asottile]

I was mostly going to trace through some stuff and see why it failed. tox -vvv might give more information as well?

Also is it _just_ python3.4 or are other versions having issues? I wonder if there's something strange with how that one is configured...

[alvations]

Okay, it seems to work when I reconfigure my matrix. Previously the matrix is on different tox testenv.

Now I've

• removed the original tox matrix on cloudbees
• set a user-defined matrix on the Python versions and
• then make name changes on the tox.ini (https://github.com/nltk/nltk/pull/1940/files) and
• use py${PYTHON_VERSION}-jenkins in the TOXENV pattern when configuring the Tox builder

Now this seems to work:
https://nltk.ci.cloudbees.com/job/nltk-cloudbees/PYTHON_VERSION=3.5.0,jdk=jdk8latestOnlineInstall/19/console

All the injected or pre- Tox Builder shell variables are not reachable by the TOXENV pattern. So I had to resort to global variables set on the matrix configuration and not in injected enviroment or pre-Tox shell.

So the question is who's throwing away these variables? Is it tox that's not reading the $PATH correctly? Or has cloudbee thrown away and recreated a new environment before Tox builder? The latter will make little sense then anything that's ran before Tox builder is pretty much useless.

Is there a way for tox to run in debug mode and print $PATH in the environment it's running in?

[alvations]

Something else unholy is going on though

After isolating the Python with user defined matrix,

• Python3.5 tests is successful https://nltk.ci.cloudbees.com/job/nltk-cloudbees/PYTHON_VERSION=3.5.0,jdk=jdk8latestOnlineInstall/22/console
• Python3.4 went nutty https://nltk.ci.cloudbees.com/job/nltk-cloudbees/PYTHON_VERSION=3.4.3,jdk=jdk8latestOnlineInstall/lastBuild/console

Python3.5 and python2.7 seem to work since they come with the distro's default python2 and python3.

[alvations]

@asottile seems like it's the 127 char thing... I've tried to shorten our variables and removed the workspace name length and now it runs! https://nltk.ci.cloudbees.com/job/nltk-cloudbees/71/ =(

This must be one big odd bug...

Is there some way to bypass this workspace name length thingy?

[asottile]

Unfortunately not, the 127 character limitation is a limit on shebang length in the os itself.

We _could_ work around the OS and do all shebang resolution ourselves. Here's some prior art from a project of mine: https://github.com/pre-commit/pre-commit/blob/98ec74dcabd8279f99b072e4232c5042ce16924b/pre_commit/parse_shebang.py#L57-L73

[stephenfin]

I'm assuming this is resolved now, @gaborbernat. Not sure why it didn't link from my commit message.

[gaborbernat]

You're correct.