Pyenv-virtualenv: Tox can't find python3.x executable in virtualenv

[gangefors]

After creating a virtualenv with:

pyenv virtualenv 3.5.3 myenv

I'm not able to run tox with this ini file:

[tox]
envlist = py35

[testenv]
commands =
    pytest {posargs: --pep8 --pylint}

deps = .[test]

The following error appears:

ERROR: InvocationError: Failed to get version_info for python3.5: b"pyenv: python3.5: command not found\n\nThe `python3.5' command exists in these Python versions:\n  3.5.3\n\n"

After checking my virtualenv it's clear that it doesn't contain a python3.5 symlink.

(myenv) ➜  ~ ll /home/stefanga/.pyenv/versions/myenv/bin/      
total 32K
-rw-r--r-- 1 stefanga axusers 2.2K Mar  3 13:06 activate
-rw-r--r-- 1 stefanga axusers 1.3K Mar  3 13:06 activate.csh
-rw-r--r-- 1 stefanga axusers 2.4K Mar  3 13:06 activate.fish
-rwxr-xr-x 1 stefanga axusers  276 Mar  3 13:06 easy_install
-rwxr-xr-x 1 stefanga axusers  276 Mar  3 13:06 easy_install-3.5
-rwxr-xr-x 1 stefanga axusers  248 Mar  3 13:06 pip
-rwxr-xr-x 1 stefanga axusers  248 Mar  3 13:06 pip3
-rwxr-xr-x 1 stefanga axusers  248 Mar  3 13:06 pip3.5
lrwxrwxrwx 1 stefanga axusers   47 Mar  3 13:06 python -> /home/stefanga/.pyenv/versions/3.5.3/bin/python
lrwxrwxrwx 1 stefanga axusers    6 Mar  3 13:06 python3 -> python

After creating the python3.5 -> python symlink manually tox executed without any errors.

Shouldn't all executables (python, pythonx and python x.y) be available in the virtualenv?

---

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

[blueyed]

See https://github.com/yyuu/pyenv/issues/92#issuecomment-145319528.

[gangefors]

Thanks! But should I really have to use another plugin to get the expected behaviour?

To me it just feels like there should be a pythonX.Y symlink in the virtualenv's bin folder after creating a new virtualenv. Since the executables are present in the system's python and also pyenv installed python versions I'd expect to find similar symlinks in the virtualenvs as well.

But if you disagree, feel free to close this issue.

[eddie-dunn]

I'm with @gangefors on this. Is there a reason that said symlinks can't be created automatically by pyenv?

[yyuu]

If you think so, just write something by yourself.

[gangefors]

To anyone that finds this thread and experience the same issue and don't want to use yet another plugin to solve it.

I've found a workaround to this issue.

By specifying exactly which python binary you want to use you can get the expected behaviour.

pyenv virtualenv -p python3.5 3.5.3 myenv

will create a virtualenv with the following contents in ./bin

ll ~/.pyenv/versions/myenv/bin 
total 12M
-rw-r--r-- 1 stefanga axusers 2.1K Mar  7 14:51 activate
-rw-r--r-- 1 stefanga axusers 1.1K Mar  7 14:51 activate.csh
-rw-r--r-- 1 stefanga axusers 2.2K Mar  7 14:51 activate.fish
-rw-r--r-- 1 stefanga axusers 1.2K Mar  7 14:51 activate_this.py
-rwxr-xr-x 1 stefanga axusers  277 Mar  7 14:51 easy_install
-rwxr-xr-x 1 stefanga axusers  277 Mar  7 14:51 easy_install-3.5
-rwxr-xr-x 1 stefanga axusers  249 Mar  7 14:51 pip
-rwxr-xr-x 1 stefanga axusers  249 Mar  7 14:51 pip3
-rwxr-xr-x 1 stefanga axusers  249 Mar  7 14:51 pip3.5
lrwxrwxrwx 1 stefanga axusers    9 Mar  7 14:51 python -> python3.5
lrwxrwxrwx 1 stefanga axusers    9 Mar  7 14:51 python3 -> python3.5
-rwxr-xr-x 1 stefanga axusers  12M Mar  7 14:51 python3.5
-rwxr-xr-x 1 stefanga axusers 2.4K Mar  7 14:51 python-config
-rwxr-xr-x 1 stefanga axusers  256 Mar  7 14:51 wheel

The plugin (probably the python venv module) now creates a virtualenv with python3.5 binary as the primary binary and creates symlinks to this. The result is that I've got my pythonX.Y binary in the virtualenv and tox is happy again.

[anentropic]

I have same problem:

$ pyenv virtualenv 3.6.2 myenv
$ pyenv activate myenv
$ tox -e py36-dj111
py36-dj111 create: /App/.tox/py36-dj111
ERROR: Error creating virtualenv. Note that some special characters (e.g. ':' and unicode symbols) in paths are not supported by virtualenv. Error details: InvocationError('Failed to get version_info for python3.6: b"pyenv: python3.6: command not found\\n\\nThe `python3.6\' command exists in these Python versions:\\n  3.6.2\\n\\n"',)
_______________________________________________________ summary _______________________________________________________
ERROR:   py36-dj111: Error creating virtualenv. Note that some special characters (e.g. ':' and unicode symbols) in paths are not supported by virtualenv. Error details: InvocationError('Failed to get version_info for python3.6: b"pyenv: python3.6: command not found\\n\\nThe `python3.6\' command exists in these Python versions:\\n  3.6.2\\n\\n"',)

I could not get tox-pyenv plugin to solve this issue (it seems to be intended to solve a slightly different problem?)

However @gangefors suggestion above does work

If I delete my virtualenv and recreate it with pyenv virtualenv -p python3.6 myenv then tox will work

So it seems to be down to pyenv-virtualenv not creating the python3.6 symlink. I note that pip3.6 and easy_install-3.6 links _are_ created.

[eddie-dunn]

@yyuu I would like to solve this, preferrably in the same place as where the python and python3 symlinks are created. I couldn't find where this is, though.

Could you point me in the right direction?

[gangefors]

Possibly related to #206

[madhurabhogate]

@gangefors : The solution you suggested (pyenv virtualenv -p python3.5 3.5.3 myenv) works.
But in my tox.ini I want to verify for multiple python versions say py35, py36. By creating the the virtualenv in above manner; tox runs successfully for py35 version but fails for py36.

ERROR: py36: Error creating virtualenv. Note that some special characters (e.g. ':' and unicode symbols) in paths are not supported by virtualenv. Error details: InvocationError('Failed to get version_info for python3.6: b"pyenv: python3.6: command not found\n\nThe `python3.6\' command exists in these Python versions:\n 3.6.0\n 3.6.1\n\n"',)

How do we overcome this issues ? Can you suggest please ?

[gangefors]

@madhurabhogate I would suggest that you create different venvs for different python versions. Then you can use them at the same time with pyenv <shell|local> <py35venv> <py36venv>. The python binaries from both venvs should be available in your path for tox to find.

# Do the following in your shell
$ pyenv virtualenv -p python3.5 3.5.4 py35
$ pyenv virtualenv -p python3.6 3.6.3 py36
$ pyenv shell py36 py35

# Then you should have two active environments you can use
$ pyenv versions
  system
  3.5.4
  3.5.4/envs/py35
  3.6.3
  3.6.3/envs/py36
* py35 (set by PYENV_VERSION environment variable)
* py36 (set by PYENV_VERSION environment variable)

# Now you should be able to run tox for both py35 and py36 envlists

[madhurabhogate]

@gangefors : Thank you. An example would be really helpful :)

[hhstore]

you can try:

pyenv shell 3.5.3

ref: https://github.com/pyenv/pyenv/issues/856

[bsolomon1124]

Note that there is a small typo in @gangefors 's example (at least for pyenv 1.2.7) with a flag:

$ brew install pyenv-virtualenv
$ pyenv virtualenv -p python3.5 3.5.6 py35
$ pyenv virtualenv -p python3.6 3.6.6 py36
$ pyenv virtualenv -p python3.7 3.7.0 py37
$ pyenv shell py35 py36 py37

If using with tox, order of cmds seems to be a tiny bit finnicky and very particular:

$ pyenv shell py35 py36 py37
$ python3.7 -m venv venv
$ source ./venv/bin/activate
$ tox

[GhostofGoes]

The issue for me was simply a case of not having the Pythons on the pyenv path. Fixed by adding them to the path:

pyenv local 3.7.2 3.6.8 3.5.6 pypy3.5-6.0.0 pypy2.7-6.0.0

[mark0978]

I did pyenv local 3.4.9 3.5.6 3.6.8 but that did NOT fix the shims. For that I had to use rm (to remove the old shim) and ln -s to create the new shim

[cicorias]

i was able to fix this in a docker build here
https://github.com/Microsoft/agogosml/blob/7a41e4d136534c9b8caf93dc1a87cb52a62aa845/agogosml_cli/Dockerfile

The "gist" of it all is below:

RUN curl https://pyenv.run | bash
#see https://www.loganasherjones.com/2018/03/setting-up-your-python-environment/

RUN pyenv install 3.5.6
RUN pyenv install 3.6.8
RUN pyenv install 3.7.2
RUN pip install pytest tox
RUN pip install tox-globinterpreter

WORKDIR /usr/src/agogosml_cli

RUN echo "3.5.6" >> .python-version
RUN echo "3.6.8" >> .python-version
RUN echo "3.7.2" >> .python-version