Pipenv: Can't find the python path with python3

[klreeher]

Issue Description:

When I attempt to call pipenv shell, it does not create the virtual env and fails with an error.

Pretty sure this is related to closed issue 3363.

Steps to replicate

I've got a fresh install of Windows 10, with the windows Ubuntu utility.

The python version on my Windows install is 3.7.3. The python version on Ubuntu is 3.6.8, and there is no alias for python --

kate@orion:/mnt/c/Users/kate$ python --version

Command 'python' not found, but can be installed with:

sudo apt install python3
sudo apt install python
sudo apt install python-minimal

You also have python3 installed, you can run 'python3' instead.

I'm going to go attempt to fix this by adjusting my PATH variable, but I figured it was worth mentioning as this is my experience with freshly installed, out of the box a) windows, b) the linux on windows supported experience c) python on windows.

pipenv --support output from the bash shell (ubuntu)

$ pipenv --support

Pipenv version: '2018.11.26'

Pipenv location: '/home/kate/.local/lib/python3.6/site-packages/pipenv'

Python location: '/usr/bin/python3'

Python installations found:

['Traceback (most recent call last):\n', '  File "/home/kate/.local/lib/python3.6/site-packages/pipenv/vendor/vistir/misc.py", line 160, in _create_subprocess\n    combine_stderr=combine_stderr)\n', '  File "/home/kate/.local/lib/python3.6/site-packages/pipenv/vendor/vistir/misc.py", line 134, in _spawn_subprocess\n    return subprocess.Popen(cmd, **options)\n', '  File "/usr/lib/python3.6/subprocess.py", line 729, in __init__\n    restore_signals, start_new_session)\n', '  File "/usr/lib/python3.6/subprocess.py", line 1364, in _execute_child\n    raise child_exception_type(errno_num, err_msg, err_filename)\n', "OSError: [Errno 8] Exec format error: '/mnt/c/Users/kate/AppData/Local/Microsoft/WindowsApps/python.exe'\n", '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', '  File "/home/kate/.local/lib/python3.6/site-packages/pipenv/vendor/vistir/contextmanagers.py", line 150, in spinner\n    yield _spinner\n', '  File "/home/kate/.local/lib/python3.6/site-packages/pipenv/vendor/vistir/misc.py", line 314, in run\n    write_to_stdout=True\n', '  File "/home/kate/.local/lib/python3.6/site-packages/pipenv/vendor/vistir/misc.py", line 162, in _create_subprocess\n    sys.stderr.write("Error %s while executing command %s", exc, " ".join(cmd._parts))\n', 'TypeError: write() takes exactly one argument (3 given)\n']
Traceback (most recent call last):
  File "/home/kate/.local/lib/python3.6/site-packages/pipenv/vendor/vistir/misc.py", line 160, in _create_subprocess
    combine_stderr=combine_stderr)
  File "/home/kate/.local/lib/python3.6/site-packages/pipenv/vendor/vistir/misc.py", line 134, in _spawn_subprocess
    return subprocess.Popen(cmd, **options)
  File "/usr/lib/python3.6/subprocess.py", line 729, in __init__
    restore_signals, start_new_session)
  File "/usr/lib/python3.6/subprocess.py", line 1364, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
OSError: [Errno 8] Exec format error: '/mnt/c/Users/kate/AppData/Local/Microsoft/WindowsApps/python.exe'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/kate/.local/bin/pipenv", line 11, in <module>
    sys.exit(cli())
  File "/home/kate/.local/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/home/kate/.local/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/home/kate/.local/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 1114, in invoke
    return Command.invoke(self, ctx)
  File "/home/kate/.local/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/kate/.local/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/home/kate/.local/lib/python3.6/site-packages/pipenv/vendor/click/decorators.py", line 64, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
  File "/home/kate/.local/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/home/kate/.local/lib/python3.6/site-packages/pipenv/vendor/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/kate/.local/lib/python3.6/site-packages/pipenv/cli/command.py", line 146, in cli
    get_pipenv_diagnostics()
  File "/home/kate/.local/lib/python3.6/site-packages/pipenv/help.py", line 33, in get_pipenv_diagnostics
    python_paths = finder.find_all_python_versions()
  File "/home/kate/.local/lib/python3.6/site-packages/pipenv/vendor/pythonfinder/pythonfinder.py", line 136, in find_all_python_versions
    major=major, minor=minor, patch=patch, pre=pre, dev=dev, arch=arch, name=name
  File "/home/kate/.local/lib/python3.6/site-packages/pipenv/vendor/pythonfinder/models/path.py", line 328, in find_all_python_versions
    values = list(self.get_pythons(sub_finder))
  File "/home/kate/.local/lib/python3.6/site-packages/pipenv/vendor/pythonfinder/models/path.py", line 279, in get_pythons
    reverse=True
  File "/home/kate/.local/lib/python3.6/site-packages/pipenv/vendor/pythonfinder/models/path.py", line 277, in <genexpr>
    (p for p in self._filter_paths(finder) if p.is_python),
  File "/home/kate/.local/lib/python3.6/site-packages/pipenv/vendor/pythonfinder/models/path.py", line 264, in <genexpr>
    pth for pth in unnest(finder(p) for p in self.path_entries if p is not None)
  File "/home/kate/.local/lib/python3.6/site-packages/pipenv/vendor/pythonfinder/utils.py", line 251, in unnest
    for el in target:
  File "/home/kate/.local/lib/python3.6/site-packages/pipenv/vendor/pythonfinder/models/path.py", line 264, in <genexpr>
    pth for pth in unnest(finder(p) for p in self.path_entries if p is not None)
  File "/home/kate/.local/lib/python3.6/site-packages/pipenv/vendor/pythonfinder/models/mixins.py", line 76, in find_all_python_versions
    path_filter = filter(None, (sub_finder(p) for p in self.children.values()))
  File "/home/kate/.local/lib/python3.6/site-packages/pipenv/vendor/cached_property.py", line 35, in __get__
    value = obj.__dict__[self.func.__name__] = self.func(obj)
  File "/home/kate/.local/lib/python3.6/site-packages/pipenv/vendor/pythonfinder/models/path.py", line 501, in children
    for child_key, child_val in self._gen_children():
  File "/home/kate/.local/lib/python3.6/site-packages/pipenv/vendor/pythonfinder/models/path.py", line 493, in _gen_children
    entry = PathEntry.create(path=child, **pass_args)
  File "/home/kate/.local/lib/python3.6/site-packages/pipenv/vendor/pythonfinder/models/path.py", line 575, in create
    _new = cls(**creation_args)
  File "<attrs generated init c86539da5486ad599344bbea207791652ee09597>", line 16, in __init__
  File "/home/kate/.local/lib/python3.6/site-packages/pipenv/vendor/pythonfinder/models/path.py", line 518, in get_py_version
    py_version = PythonVersion.from_path(path=self, name=self.name)
  File "/home/kate/.local/lib/python3.6/site-packages/pipenv/vendor/pythonfinder/models/python.py", line 395, in from_path
    py_version = get_python_version(path.path.absolute().as_posix())
  File "/home/kate/.local/lib/python3.6/site-packages/pipenv/vendor/pythonfinder/utils.py", line 68, in get_python_version
    combine_stderr=False, write_to_stdout=False)
  File "/home/kate/.local/lib/python3.6/site-packages/pipenv/vendor/vistir/misc.py", line 314, in run
    write_to_stdout=True
  File "/home/kate/.local/lib/python3.6/site-packages/pipenv/vendor/vistir/misc.py", line 162, in _create_subprocess
    sys.stderr.write("Error %s while executing command %s", exc, " ".join(cmd._parts))
TypeError: write() takes exactly one argument (3 given)

[klreeher]

Adding an alias in my bash profile for python = python3 worked to allow correct pipenv shell creation, but I'm still getting the write error with pipenv --support.

[frostming]

Related: #3488

[thernstig]

I will add this comment here as well, taken from https://github.com/pypa/pipenv/issues/3488

In the Windows 10, May 2019 update (19.03) python.exe and python3.exe are found by default, but they do not point to a real Python executable. I.e. a user have these without having installed Python in Windows. Read this announcement https://devblogs.microsoft.com/python/python-in-the-windows-10-may-2019-update/ and more specifically this quote:

Finally, with the May 2019 Windows Update, we are completing the picture. While Python continues to remain completely independent from the operating system, every install of Windows will include python and python3 commands that take you directly to the Python store page.

I get this failure even though I have only installed python into the WSL distribution, and my project resides in the /home/username/ dir.

11:56:55 computer1 ~/code/scrapy (master)$ which python
/usr/bin/python
11:57:01 computer1 ~/code/scrapy (master)$ which python3
/usr/bin/python3
11:57:04 computer1 ~/code/scrapy (master)$ printenv PATH
/home/username/code/scrapy/.venv/bin:/home/username/.pyenv/shims:/home/username/.pyenv/bin:/home/username/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/mnt/c/Program Files/WindowsApps/CanonicalGroupLimited.UbuntuonWindows_1804.2019.521.0_x64__79rhkp1fndgsc:/mnt/c/WINDOWS/system32:/mnt/c/WINDOWS:/mnt/c/WINDOWS/System32/Wbem:/mnt/c/WINDOWS/System32/WindowsPowerShell/v1.0/:/mnt/c/WINDOWS/System32/OpenSSH/:/mnt/c/Users/username/AppData/Local/Microsoft/WindowsApps:/snap/bin

This completely breaks working in WSL and automatic installing like with pyenv, or using direnv with layout_pipenv. And the user on the 2019 May update has not even installed in Windows.

Should the issue be opened in https://github.com/sarugaku/pythonfinder instead?

[thernstig]

@frostming Do you know if there was an issue opened in pythonfinder? I could not find one.

[Flimm]

The fix in #3853 has not been released on Pypi. The latest release on Pypi is a year old. (I wish GitHub issues were kept open until the fix was released.)

[brainwane]

The fix is now available in a prerelease; see #3369 to track when the maintainers will publish the next release.