Tox: fails to run on Windows

[jaraco]

Running on Windows 10 (Fall Creators update), Python 3.6.4, and tox 2.9.1, I get three errors:

PS C:\Users\jaraco\m\jaraco.windows> py -m tox
python create: C:\Users\jaraco\m\jaraco.windows\.tox\python
ERROR: invocation failed (exit code 100), logfile: C:\Users\jaraco\m\jaraco.windows\.tox\python\log\python-0.log
ERROR: actionid: python
msg: getenv
cmdargs: ['C:\\Program Files\\Python36\\python.exe', '-m', 'virtualenv', '--python', 'C:\\Program Files\\Python36\\python.exe', 'python']

Already using interpreter C:\Program Files\Python36\python.exe
Using base prefix 'C:\\Program Files\\Python36'
New python executable in C:\Users\jaraco\m\jaraco.windows\.tox\python\Scripts\python.exe
ERROR: The executable C:\Users\jaraco\m\jaraco.windows\.tox\python\Scripts\python.exe is not functioning
ERROR: It thinks sys.prefix is 'c:\\users\\jaraco\\m\\jaraco.windows\\.tox' (should be 'c:\\users\\jaraco\\m\\jaraco.windows\\.tox\\python')
ERROR: virtualenv is not compatible with this system or executable
Note: some Windows users have reported this error when they installed Python for "Only this user" or have multiple versions of Python installed. Copying the appropriate PythonXX.dll to the virtualenv Scripts/ directory may fix this problem.

ERROR: Error creating virtualenv. Note that some special characters (e.g. ':' and unicode symbols) in paths are not supported by virtualenv. Error details: InvocationError('C:\\Program Files\\Python36\\python.exe -m virtualenv --python C:\\Program Files\\Python36\\python.exe python (see C:\\Users\\jaraco\\m\\jaraco.windows\\.tox\\python\\log\\python-0.log)', 100)
______________________________________________________ summary _______________________________________________________
ERROR:   python: Error creating virtualenv. Note that some special characters (e.g. ':' and unicode symbols) in paths are not supported by virtualenv. Error details: InvocationError('C:\\Program Files\\Python36\\python.exe -m virtualenv --python C:\\Program Files\\Python36\\python.exe python (see C:\\Users\\jaraco\\m\\jaraco.windows\\.tox\\python\\log\\python-0.log)', 100)

I've not installed Python for "Only this user". I do have one other Python (2.7.14) installed and running py -m tox -e py27 does run successfully.

[jaraco]

It seems the issue is readily reproduced with just virtualenv, so the issue is likely upstream:

PS C:\Users\jaraco\m\jaraco.windows> py -m virtualenv .tox/new
Using base prefix 'C:\\Program Files\\Python36'
New python executable in C:\Users\jaraco\m\jaraco.windows\.tox\new\Scripts\python.exe
ERROR: The executable C:\Users\jaraco\m\jaraco.windows\.tox\new\Scripts\python.exe is not functioning
ERROR: It thinks sys.prefix is 'c:\\users\\jaraco\\m\\jaraco.windows' (should be 'c:\\users\\jaraco\\m\\jaraco.windows\\.tox\\new')
ERROR: virtualenv is not compatible with this system or executable
Note: some Windows users have reported this error when they installed Python for "Only this user" or have multiple versions of Python installed. Copying the appropriate PythonXX.dll to the virtualenv Scripts/ directory may fix this problem.

[jaraco]

Upstream issue is pypa/virtualenv#1116

[jaraco]

I'm pretty sure there's nothing to be done by tox, so closing.

[obestwalter]

@jaraco thanks for reporting this. IIUC this is still a problem in tox though until the upstream issue at https://github.com/pypa/virtualenv/issues/1116 is fixed though, so I'll reopen this so we can track it.

[TheFriendlyCoder]

NOTE: This does not appear to be a virtualenv bug. I can reproduce the problem reliably on my Windows 10 PC. I have 2 python interpreters installed: v2.7.14 and v3.6.5 (64bit). Both interpreters are installed under c:\Program Files (as should be done on any recent version of Windows). I only have tox and virtualenv installed on my python 3.6.5 install so there shouldn't be any conflicts, and I have tox v2.9.1 and virtualenv v15.2.0. Also, I can reliably create virtual environments for either python2 or python3 installs without using tox (ie: virtualenv -p "c:\Program Files\Python2.7\python.exe" .\venv) and I can successfully install Python packages and run the Python interpreter in those manually created virtual environments.... so again, the problem only appears to occur when creating virtual environments inside tox.

Oh, and I looked at the linked defect in virtualenv and for some reason I am unable to reproduce the problem. I thought at first the difference might be that the reporter was apparently using Powershell for their runtime environment, but I've confirmed that I get the exact same behavior I just described whether running the operations under a DOS shell or under Powershell, so I'm not sure what the discrepancy would be. Regardless it seems to be unrelated to this defect.

[jaraco]

@TheFriendlyCoder Can you replicate the issue if you invoke virtualenv with py -m virtualenv or python -m virtualenv?

[TheFriendlyCoder]

I've tested with the virtualenv file system alias / executable directly, as well as with the py.exe launcher and the python.exe executable - all with the same results. Virtualenv can be run and successfully creates virtual environments when run directly, but tox still fails.

[obestwalter]

I would be tempted to close this one as I can't reproduce it and there is no follow up on https://github.com/pypa/virtualenv/issues/1116.

[jaraco]

I'm no longer able to replicate the issue reported upstream. I used to install my Windows Pythons in C:\Program Files\Python37 (or similar), but these days I've given up on trying to be proper about security and all that and just use Chocolatey to install Python on Windows, which installs it in C:\Python37. I suspect the space in the file path is what is tripping up both Virtualenv and Tox. I suspect that TheFriendlyCoder didn't experience the issue in virtualenv directly because he was passing the full path to the executable in his call to create the virtualenv.

I tried installing Python 3.6 using the python.org installer, which installs to C:\Program Files\Python36 by default, but even in that environment, I'm unable to replicate the issue. Creating virtualenvs with either virtualenv itself or with tox succeeds now without error for me.

[obestwalter]

Thanks for checking that @jaraco - I think the state of Python on Windows is generally considerably better than a few years ago.

[Laurentiu-Andronache]

Please re-open this, I have the same issue with Python 3.7, I've reinstalled it like 10 times in different directories, for all users or just for the current one, using py or python, using --system-site-packages or not, etc. I have no idea wtf I'm doing anyway

maybe because I've symlinked to tox.exe from the virtual environment created specifically for it with mklink.exe?

"py -m venv .tox/new" works for me

[Laurentiu-Andronache]

It doesn't work even if I don't use the symlink created, and instead activate the .toxbase environment before running tox in my new project dir...

[Laurentiu-Andronache]

Error log here: https://pastebin.com/r4CE36is

[Laurentiu-Andronache]

If I activate toxbase environment and try py -m virtualenv .tox/new, I'm getting the same error

[Laurentiu-Andronache]

Omg, I just discovered that if I install the tox-venv plugin, it works. removed bad language towards virtualenv

[gaborbernat]

Yeah it's a virtualenv bug, please mind your language though.