Pipenv: 2018.11.14 pipenv commands fail with pyenv

[torte]

Issue description

We use pyenv to support multiple Python versions on one machine.
When using any command with pipenv (version 2018.11.14) it fails with the below traceback.

Expected result

pipenv should execute the commands without errors.

Actual result

$ pipenv install
Traceback (most recent call last):
  File "/usr/local/bin/pipenv", line 11, in <module>
    sys.exit(cli())
  File "/usr/local/lib/python3.5/dist-packages/pipenv/vendor/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.5/dist-packages/pipenv/vendor/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.5/dist-packages/pipenv/vendor/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.5/dist-packages/pipenv/vendor/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.5/dist-packages/pipenv/vendor/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.5/dist-packages/pipenv/vendor/click/decorators.py", line 64, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
  File "/usr/local/lib/python3.5/dist-packages/pipenv/vendor/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.5/dist-packages/pipenv/vendor/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/usr/local/lib/python3.5/dist-packages/pipenv/cli/command.py", line 249, in install
    editable_packages=state.installstate.editables,
  File "/usr/local/lib/python3.5/dist-packages/pipenv/core.py", line 1725, in do_install
    pypi_mirror=pypi_mirror,
  File "/usr/local/lib/python3.5/dist-packages/pipenv/core.py", line 569, in ensure_project
    pypi_mirror=pypi_mirror,
  File "/usr/local/lib/python3.5/dist-packages/pipenv/core.py", line 489, in ensure_virtualenv
    python = ensure_python(three=three, python=python)
  File "/usr/local/lib/python3.5/dist-packages/pipenv/core.py", line 397, in ensure_python
    path_to_python = find_a_system_python(python)
  File "/usr/local/lib/python3.5/dist-packages/pipenv/core.py", line 360, in find_a_system_python
    python_entry = finder.find_python_version(line)
  File "/usr/local/lib/python3.5/dist-packages/pipenv/vendor/pythonfinder/pythonfinder.py", line 113, in find_python_version
    return self.system_path.find_python_version(
  File "/usr/local/lib/python3.5/dist-packages/pipenv/vendor/pythonfinder/pythonfinder.py", line 54, in system_path
    ignore_unsupported=self.ignore_unsupported,
  File "/usr/local/lib/python3.5/dist-packages/pipenv/vendor/pythonfinder/models/path.py", line 451, in create
    ignore_unsupported=ignore_unsupported,
  File "<attrs generated init 75e45f144e3d5510d54dd5fca6730b98fa0220a9>", line 38, in __init__
  File "/usr/local/lib/python3.5/dist-packages/pipenv/vendor/pythonfinder/models/path.py", line 116, in __attrs_post_init__
    self._setup_pyenv()
  File "/usr/local/lib/python3.5/dist-packages/pipenv/vendor/pythonfinder/models/path.py", line 196, in _setup_pyenv
    version_glob_path="versions/*", ignore_unsupported=self.ignore_unsupported
  File "/usr/local/lib/python3.5/dist-packages/pipenv/vendor/pythonfinder/models/python.py", line 154, in create
    sort_function=sort_function, version_glob_path=version_glob_path)
  File "<attrs generated init 66db6832296eebaad0ce8186c2c66d03343025d4>", line 17, in __init__
  File "/usr/local/lib/python3.5/dist-packages/pipenv/vendor/pythonfinder/models/python.py", line 98, in get_versions
    for p in self.get_version_order():
  File "/usr/local/lib/python3.5/dist-packages/pipenv/vendor/pythonfinder/models/python.py", line 70, in get_version_order
    version_order = [versions[v] for v in parse_pyenv_version_order()]
TypeError: 'NoneType' object is not iterable

Steps to replicate

Install pyenv as described in the README on https://github.com/pyenv/pyenv. Install a Python version of your choice using pyenv, e.g. pyenv install 3.7.0. When in your project create a Pipfile and add/change the following section:

[requires]
python_version = "3.7.0"

Now run any pipenv command in your project, .e.g pipenv install or pipenv shell.

---

<details><summary>$ pipenv --support</summary>

Pipenv version: `'2018.11.14'`

Pipenv location: `'/usr/local/lib/python3.5/dist-packages/pipenv'`

Python location: `'/usr/bin/python3'`

Python installations found:

Traceback (most recent call last):
  File "/usr/local/bin/pipenv", line 11, in <module>
    sys.exit(cli())
  File "/usr/local/lib/python3.5/dist-packages/pipenv/vendor/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.5/dist-packages/pipenv/vendor/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.5/dist-packages/pipenv/vendor/click/core.py", line 1114, in invoke
    return Command.invoke(self, ctx)
  File "/usr/local/lib/python3.5/dist-packages/pipenv/vendor/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.5/dist-packages/pipenv/vendor/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.5/dist-packages/pipenv/vendor/click/decorators.py", line 64, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
  File "/usr/local/lib/python3.5/dist-packages/pipenv/vendor/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.5/dist-packages/pipenv/vendor/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/usr/local/lib/python3.5/dist-packages/pipenv/cli/command.py", line 142, in cli
    get_pipenv_diagnostics()
  File "/usr/local/lib/python3.5/dist-packages/pipenv/help.py", line 33, in get_pipenv_diagnostics
    python_paths = finder.find_all_python_versions()
  File "/usr/local/lib/python3.5/dist-packages/pipenv/vendor/pythonfinder/pythonfinder.py", line 122, in find_all_python_versions
    python_version_dict = getattr(self.system_path, "python_version_dict")
  File "/usr/local/lib/python3.5/dist-packages/pipenv/vendor/pythonfinder/pythonfinder.py", line 54, in system_path
    ignore_unsupported=self.ignore_unsupported,
  File "/usr/local/lib/python3.5/dist-packages/pipenv/vendor/pythonfinder/models/path.py", line 451, in create
    ignore_unsupported=ignore_unsupported,
  File "<attrs generated init 75e45f144e3d5510d54dd5fca6730b98fa0220a9>", line 38, in __init__
  File "/usr/local/lib/python3.5/dist-packages/pipenv/vendor/pythonfinder/models/path.py", line 116, in __attrs_post_init__
    self._setup_pyenv()
  File "/usr/local/lib/python3.5/dist-packages/pipenv/vendor/pythonfinder/models/path.py", line 196, in _setup_pyenv
    version_glob_path="versions/*", ignore_unsupported=self.ignore_unsupported
  File "/usr/local/lib/python3.5/dist-packages/pipenv/vendor/pythonfinder/models/python.py", line 154, in create
    sort_function=sort_function, version_glob_path=version_glob_path)
  File "<attrs generated init e41401380504971a8c7664b563d716c9ae61830a>", line 17, in __init__
  File "/usr/local/lib/python3.5/dist-packages/pipenv/vendor/pythonfinder/models/python.py", line 98, in get_versions
    for p in self.get_version_order():
  File "/usr/local/lib/python3.5/dist-packages/pipenv/vendor/pythonfinder/models/python.py", line 70, in get_version_order
    version_order = [versions[v] for v in parse_pyenv_version_order()]
TypeError: 'NoneType' object is not iterable

Thanks for looking into it.

[techalchemy]

omg, unreal. Pyenv is causing me nothing but headaches

[techalchemy]

Workaround: set a global pyenv version with pyenv global <x.y.z>

[xsteadfastx]

Workaround: set a global pyenv version with pyenv global <x.y.z>

does not help at all.

here is my output:

$ pipenv lock                                                                                                                                                    
Traceback (most recent call last):
  File "/home/preuss/.local/bin/pipenv", line 11, in <module>
    sys.exit(cli())
  File "/home/preuss/.local/venvs/pipenv/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/home/preuss/.local/venvs/pipenv/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/home/preuss/.local/venvs/pipenv/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/preuss/.local/venvs/pipenv/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/preuss/.local/venvs/pipenv/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/home/preuss/.local/venvs/pipenv/lib/python3.6/site-packages/pipenv/vendor/click/decorators.py", line 64, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
  File "/home/preuss/.local/venvs/pipenv/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/home/preuss/.local/venvs/pipenv/lib/python3.6/site-packages/pipenv/vendor/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/preuss/.local/venvs/pipenv/lib/python3.6/site-packages/pipenv/cli/command.py", line 312, in lock
    ensure_project(three=state.three, python=state.python, pypi_mirror=state.pypi_mirror)
  File "/home/preuss/.local/venvs/pipenv/lib/python3.6/site-packages/pipenv/core.py", line 569, in ensure_project
    pypi_mirror=pypi_mirror,
  File "/home/preuss/.local/venvs/pipenv/lib/python3.6/site-packages/pipenv/core.py", line 489, in ensure_virtualenv
    python = ensure_python(three=three, python=python)
  File "/home/preuss/.local/venvs/pipenv/lib/python3.6/site-packages/pipenv/core.py", line 397, in ensure_python
    path_to_python = find_a_system_python(python)
  File "/home/preuss/.local/venvs/pipenv/lib/python3.6/site-packages/pipenv/core.py", line 360, in find_a_system_python
    python_entry = finder.find_python_version(line)
  File "/home/preuss/.local/venvs/pipenv/lib/python3.6/site-packages/pipenv/vendor/pythonfinder/pythonfinder.py", line 113, in find_python_version
    return self.system_path.find_python_version(
  File "/home/preuss/.local/venvs/pipenv/lib/python3.6/site-packages/pipenv/vendor/pythonfinder/pythonfinder.py", line 54, in system_path
    ignore_unsupported=self.ignore_unsupported,
  File "/home/preuss/.local/venvs/pipenv/lib/python3.6/site-packages/pipenv/vendor/pythonfinder/models/path.py", line 451, in create
    ignore_unsupported=ignore_unsupported,
  File "<attrs generated init 75e45f144e3d5510d54dd5fca6730b98fa0220a9>", line 38, in __init__
  File "/home/preuss/.local/venvs/pipenv/lib/python3.6/site-packages/pipenv/vendor/pythonfinder/models/path.py", line 116, in __attrs_post_init__
    self._setup_pyenv()
  File "/home/preuss/.local/venvs/pipenv/lib/python3.6/site-packages/pipenv/vendor/pythonfinder/models/path.py", line 196, in _setup_pyenv
    version_glob_path="versions/*", ignore_unsupported=self.ignore_unsupported
  File "/home/preuss/.local/venvs/pipenv/lib/python3.6/site-packages/pipenv/vendor/pythonfinder/models/python.py", line 154, in create
    sort_function=sort_function, version_glob_path=version_glob_path)
  File "<attrs generated init f5325a80b4da1c6cc0a1f4fb3fb509bfb5c0bbdc>", line 17, in __init__
  File "/home/preuss/.local/venvs/pipenv/lib/python3.6/site-packages/pipenv/vendor/pythonfinder/models/python.py", line 98, in get_versions
    for p in self.get_version_order():
  File "/home/preuss/.local/venvs/pipenv/lib/python3.6/site-packages/pipenv/vendor/pythonfinder/models/python.py", line 70, in get_version_order
    version_order = [versions[v] for v in parse_pyenv_version_order()]
  File "/home/preuss/.local/venvs/pipenv/lib/python3.6/site-packages/pipenv/vendor/pythonfinder/models/python.py", line 70, in <listcomp>
    version_order = [versions[v] for v in parse_pyenv_version_order()]
KeyError: 'system'

[jxltom]

Hi @xsteadfastx Or you can delete system in your pyenv's version order file $(pyenv root)/version

[xsteadfastx]

Hi @xsteadfastx Or you can delete system in your pyenv's version order file $(pyenv root)/version

thanks this worked as a workaround :+1:

[hartror]

Same issue running pipenv install.

Ubuntu 12.04 (don't ask)
Python 2.7.3
pip 9.0.1
pipenv 2018.11.14
pyenv v1.2.2 -> v1.2.8

Downgrading to pipenv 2018.7.1 fixed the issue (was our last known working version).

It appears the issue is parse_pyenv_version_order is returning None as the $PYENV_ROOT/version file doesn't exist. this file doesn't exist on any of the pyenv installs I have at hand.

[techalchemy]

Yes we are aware. If you run the command as described it will exist with valid data and you will have a workaround. We have a fix already set, and a workaround is provided above

[clintonb]

This issue is not fixed. I am unable to run builds on Travis: https://travis-ci.org/clintonb/cookiecutter-django/builds/456875078. They fail when I try to run pipenv install.

[jxltom]

@clintonb It is because it is fixed in master branch, but not released yet.

[clintonb]

I am running with the version from master. Looking at the PR that closed
this issue, all that was done was the addition of an if statement. Given
that the issue lies with a function returning None instead of an

iterable, that fix won’t work.

Clinton

[jxltom]

@clintonb It is fixed in https://github.com/pypa/pipenv/pull/3254

[clintonb]

Thanks @jxltom and @techalchemy! I have confirmed pipenv install works.

[nfplatzke]

This is still failing for me on a fresh install.

[techalchemy]

can you clarify "fresh install" -- we haven't released a fix yet, it is only fixed on the master branch.

[nfplatzke]

I installed using the get-pipenv.py checked into master. Will this not install the latest version?

[techalchemy]

ah I'm sure that installs the pypi version, I'll be cutting a release shortly

[sunbearc22]

I had used this command

pip3 install --prefix ~/.local -e git+https://github.com/pypa/pipenv.git#egg=pipenv

to installed the fixed master found in github to my ~/.local directory. It worked on Ubuntu 16.04. Thanks @techalchemy.

[crd]

@techalchemy I realize it's a holiday weekend and am not pressuring you -- I just wanted to ask if you ran into any trouble cutting a release or if you just didn't get around to it. Thanks!

[techalchemy]

Release is up