Pipenv: Unable to create virtualenv with Pipenv on a pyenv installed Python

[alexrecarey]

I expect something is wrong in my environment with this issue. Any pointers as to how to troubleshoot it are appreciated.

Issue description

On a new install of Pipenv using Homebrew, I cannot create a new environment when targetting a pyenv installed version of python.

alex@NerdPro ~/code/hack/pipenv-test                                                                                                           [16:38:08] 
> $ pipenv --python 3.6                                                                                                                                  
Creating a virtualenv for this project...
Pipfile: /Users/alex/code/hack/pipenv-test/Pipfile
Using /Users/alex/.pyenv/versions/3.6.6/bin/python3.6m (3.6.6) to create virtualenv...
⠋Running virtualenv with interpreter /Users/alex/.pyenv/versions/3.6.6/bin/python3.6m
Using base prefix '/Users/alex/.pyenv/versions/3.6.6/Python.framework/Versions/3.6'
Traceback (most recent call last):
  File "/usr/local/Cellar/pipenv/2018.7.1/libexec/lib/python3.7/site-packages/virtualenv.py", line 2343, in <module>
    main()
  File "/usr/local/Cellar/pipenv/2018.7.1/libexec/lib/python3.7/site-packages/virtualenv.py", line 712, in main
    symlink=options.symlink)
  File "/usr/local/Cellar/pipenv/2018.7.1/libexec/lib/python3.7/site-packages/virtualenv.py", line 927, in create_environment
    site_packages=site_packages, clear=clear, symlink=symlink))
  File "/usr/local/Cellar/pipenv/2018.7.1/libexec/lib/python3.7/site-packages/virtualenv.py", line 1134, in install_python
    copy_required_modules(home_dir, symlink)
  File "/usr/local/Cellar/pipenv/2018.7.1/libexec/lib/python3.7/site-packages/virtualenv.py", line 1064, in copy_required_modules
    dst_filename = change_prefix(filename, dst_prefix)
  File "/usr/local/Cellar/pipenv/2018.7.1/libexec/lib/python3.7/site-packages/virtualenv.py", line 1038, in change_prefix
    (filename, prefixes)
AssertionError: Filename /Users/alex/.pyenv/versions/3.6.6/lib/python3.6/lib-dynload/zlib.cpython-36m-darwin.so does not start with any of these prefixes: ['/Users/alex/.pyenv/versions/3.6.6/Python.framework/Versions/3.6/Extras/lib/python', '/Users/alex/.pyenv/versions/3.6.6/Python.framework/Versions/3.6', '/Users/alex/.pyenv/versions/3.6.6/Python.framework/Versions/3.6', '/Users/alex/Library/Python/3.6/lib/python/site-packages', '/Users/alex/.local/lib/python/3.6/site-packages', '/Users/alex/Library/Python/3.6/site-packages', '/Library/Python/3.6/site-packages']

Virtualenv location: 
Traceback (most recent call last):
  File "/usr/local/Cellar/pipenv/2018.7.1/libexec/bin/pipenv", line 11, in <module>
    load_entry_point('pipenv==2018.7.1', 'console_scripts', 'pipenv')()
  File "/usr/local/Cellar/pipenv/2018.7.1/libexec/lib/python3.7/site-packages/pipenv/vendor/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/Cellar/pipenv/2018.7.1/libexec/lib/python3.7/site-packages/pipenv/vendor/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/usr/local/Cellar/pipenv/2018.7.1/libexec/lib/python3.7/site-packages/pipenv/vendor/click/core.py", line 1043, in invoke
    return Command.invoke(self, ctx)
  File "/usr/local/Cellar/pipenv/2018.7.1/libexec/lib/python3.7/site-packages/pipenv/vendor/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/Cellar/pipenv/2018.7.1/libexec/lib/python3.7/site-packages/pipenv/vendor/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/Cellar/pipenv/2018.7.1/libexec/lib/python3.7/site-packages/pipenv/vendor/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/usr/local/Cellar/pipenv/2018.7.1/libexec/lib/python3.7/site-packages/pipenv/cli.py", line 294, in cli
    three=three, python=python, warn=True, site_packages=site_packages, pypi_mirror=pypi_mirror
  File "/usr/local/Cellar/pipenv/2018.7.1/libexec/lib/python3.7/site-packages/pipenv/core.py", line 639, in ensure_project
    ensure_pipfile(validate=validate, skip_requirements=skip_requirements, system=system)
  File "/usr/local/Cellar/pipenv/2018.7.1/libexec/lib/python3.7/site-packages/pipenv/core.py", line 244, in ensure_pipfile
    python = which('python') if not (USING_DEFAULT_PYTHON or system) else None
  File "/usr/local/Cellar/pipenv/2018.7.1/libexec/lib/python3.7/site-packages/pipenv/core.py", line 115, in which
    p = os.path.join(location, 'bin', command)
  File "/usr/local/Cellar/pipenv/2018.7.1/libexec/bin/../lib/python3.7/posixpath.py", line 80, in join
    a = os.fspath(a)
TypeError: expected str, bytes or os.PathLike object, not NoneType

Expected result

I expected to get a virtual environment based on python 3.6.6.

Actual result

When possible, provide the verbose output (--verbose), especially for locking and dependencies resolving issues.

Steps to replicate

On an OSX machine:

brew install python
brew install pyenv
brew install pipenv
pyenv install 3.6.6
pipenv --python 3.6

---

$ pipenv --support

Pipenv version: '2018.7.1'

Pipenv location: '/usr/local/Cellar/pipenv/2018.7.1/libexec/lib/python3.7/site-packages/pipenv'

Python location: '/usr/local/Cellar/pipenv/2018.7.1/libexec/bin/python3.7'

Other Python installations in PATH:

• 2.7: /Users/alex/.pyenv/shims/python2.7
• 2.7: /Users/alex/.pyenv/shims/python2.7
• 2.7: /usr/local/bin/python2.7
• 2.7: /usr/bin/python2.7
• 3.6: /Users/alex/.pyenv/shims/python3.6
• 3.7: /usr/local/bin/python3.7m

• 3.7: /usr/local/bin/python3.7

• 2.7.15: /Users/alex/.pyenv/shims/python

• 2.7.15: /usr/local/bin/python
• 2.7.10: /usr/bin/python
• 2.7.15: /Users/alex/.pyenv/shims/python2
• 2.7.15: /usr/local/bin/python2
• 3.7.0: /Users/alex/.pyenv/shims/python3
• 3.7.0: /usr/local/bin/python3

PEP 508 Information:

{'implementation_name': 'cpython',
 'implementation_version': '3.7.0',
 'os_name': 'posix',
 'platform_machine': 'x86_64',
 'platform_python_implementation': 'CPython',
 'platform_release': '17.7.0',
 'platform_system': 'Darwin',
 'platform_version': 'Darwin Kernel Version 17.7.0: Thu Jun 21 22:53:14 PDT '
                     '2018; root:xnu-4570.71.2~1/RELEASE_X86_64',
 'python_full_version': '3.7.0',
 'python_version': '3.7',
 'sys_platform': 'darwin'}

System environment variables:

• PATH
• TERM_PROGRAM
• ANDROID_HOME
• NVM_CD_FLAGS
• TERM
• SHELL
• TMPDIR
• Apple_PubSub_Socket_Render
• TERM_PROGRAM_VERSION
• TERM_SESSION_ID
• JENV_FORCEJAVAHOME
• LC_ALL
• ZSH
• NVM_DIR
• USER
• SSH_AUTH_SOCK
• PYENV_VIRTUALENV_INIT
• PAGER
• LSCOLORS
• _
• JENV_LOADED
• PWD
• JAVA_HOME
• LANG
• XPC_FLAGS
• JENV_FORCEJDKHOME
• XPC_SERVICE_NAME
• JDK_HOME
• PYENV_SHELL
• HOME
• SHLVL
• LESS
• LOGNAME
• LC_CTYPE
• SECURITYSESSIONID
• __CF_USER_TEXT_ENCODING
• PYTHONDONTWRITEBYTECODE
• PIP_PYTHON_PATH

Pipenv–specific environment variables:

Debug–specific environment variables:

• PATH: /usr/local/Cellar/pipenv/2018.7.1/libexec/tools:/Users/alex/.jenv/shims:/Users/alex/.jenv/bin:/usr/local/Cellar/pyenv-virtualenv/1.1.3/shims:/Users/alex/.pyenv/shims:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Users/alex/bin:/Users/alex/Library/Android/sdk/tools:/Users/alex/Library/Android/sdk/tools/bin:/Users/alex/Library/Android/sdk/platform-tools
• SHELL: /usr/local/bin/zsh
• LANG: en_US.UTF-8
• PWD: /Users/alex/code/hack/pipenv-test

---

Contents of Pipfile ('/Users/alex/code/hack/pipenv-test/Pipfile'):

[dgrant]

brew install python

This will install a new "Brew" python replacing your default MacOS system one. A good idea I guess... although why do you need to do that if you're going the pyenv route (which is the better route IMO):

Seems like an additional python you can do away with to make things simpler.

brew install pipenv

This pipenv is going inside your Brew python. Instead, install pyenv first, install a python with pyenv and then install pipenv using it.

[dgrant]

Remove anything in homebrew to do with python. Install only pyenv with brew, or install pyenv manually, using the instructions on their Github page. Then install Pythons using pyenv.

[uranusjr]

I cannot reproduce this. Reading the logs, it seems that the pyenv-installed Python is reporting wrong things about itself. Specifically, it reports itself as a framework build, while in reality it is not. This makes virtualenv confused and eventually results in the error.

I am inclined to think this is not caused by a bug in Pipenv (actually virtualenv), but a faulty environment configuration.

Can you do ls /Users/alex/.pyenv/versions/3.6.6 and show the result? Did you build the Python with --enable-framework set? Does removing and installing 3.6.6 with pyenv again help?

[alexrecarey]

Thanks for the feedback! I fixed it reinstalling 3.6.6, but for posterity, here are the steps:

I did install 3.6.6 with the --enable-framework flag. These are the contents of my /Users/alex/.pyenv/versions/3.6.6 folder:

$ ls /users/alex/.pyenv/versions/3.6.6
Python.framework bin              bin.orig         envs             include          lib              share

Like suggested, I reinstalled 3.6.6:

pyenv uninstall 3.6.6
env PYTHON_CONFIGURE_OPTS="--enable-framework" pyenv install 3.6.6
pipenv --python 3.6

Fixed! Thank you! 👍