pipenv does not use default python version under pyenv

Created on 12 Jun 2020 · 10Comments · Source: pypa/pipenv

Issue description

The pipenv documentation states this:

If you don’t specify a Python version on the command–line, either the [requires] python_full_version or python_version will be selected automatically, falling back to whatever your system’s default python installation is, at time of execution.

However with 2020.06.02 I'm seeing pipenv choose the most recent pyenv installed python rather than the one that's current the default when I run pipenv install.

Expected result

Python version is 3.7.7 as set by .python-version via pyenv. I've verified that pipenv is installed and running from 3.7.7. I start a new project with pipenv install and I expect to see a virtual environment created with Python version 3.7.7.

artlogic@wardenclyffe:~/Code/random/pipenv-again$ python -V
Python 3.7.7
artlogic@wardenclyffe:~/Code/random/pipenv-again$ pipenv --support
<details><summary>$ pipenv --support</summary>

Pipenv version: `'2020.6.2'`

Pipenv location: `'/home/artlogic/.pyenv/root/versions/3.7.7/lib/python3.7/site-packages/pipenv'`

Python location: `'/home/artlogic/.pyenv/root/versions/3.7.7/bin/python3.7'`

Actual result

pipenv uses a different pyenv installed python to create the venv:

artlogic@wardenclyffe:~/Code/random/pipenv-again$ pipenv install -v requests
Using python: None
Path to python: /home/artlogic/.pyenv/root/versions/3.8.2/bin/python3
Creating a virtualenv for this project…
Pipfile: /home/artlogic/Code/random/pipenv-again/Pipfile
Using /home/artlogic/.pyenv/root/versions/3.8.2/bin/python3 (3.8.2) to create virtualenv…

Steps to replicate

Install two versions of python using pyenv (3.7.7 and 3.8.3).
Make a project directory and type: pyenv local 3.7.7 (verify you're now using 3.7.7)
Install pipenv: pip install pipenv
Install a dependency: pipenv install requests
Watch the virtualenv get created with 3.8.3.

$ pipenv --support

Pipenv version: '2020.6.2'

Pipenv location: '/home/artlogic/.pyenv/root/versions/3.7.7/lib/python3.7/site-packages/pipenv'

Python location: '/home/artlogic/.pyenv/root/versions/3.7.7/bin/python3.7'

Python installations found:

3.8.2: /home/artlogic/.pyenv/root/versions/3.8.2/bin/python3
3.7.7: /home/artlogic/.pyenv/root/versions/3.7.7/bin/python3
3.7.7: /home/artlogic/.pyenv/root/versions/3.7.7/bin/python3.7m
3.7.7: /home/artlogic/.pyenv/root/versions/3.7.7/bin/python3.7
3.7.7: /home/artlogic/.pyenv/root/versions/3.7.7/bin/python3
3.7.5: /home/artlogic/.pyenv/root/versions/3.7.5/bin/python3
3.7.4: /home/artlogic/.pyenv/root/versions/3.7.4/bin/python3
3.7.3: /home/artlogic/.pyenv/root/versions/3.7.3/bin/python3
3.7.2: /home/artlogic/.pyenv/root/versions/3.7.2/bin/python3
3.7.0: /home/artlogic/.pyenv/root/versions/3.7.0/bin/python3
3.6.3: /home/artlogic/.pyenv/root/versions/3.6.3/bin/python3
3.6.1: /home/artlogic/.pyenv/root/versions/3.6.1/bin/python3
3.5.4: /home/artlogic/.pyenv/root/versions/3.5.4/bin/python3
3.5.3: /usr/bin/python3
3.5.3: /usr/bin/python3.5
3.5.3: /usr/bin/python3.5m
2.7.15: /home/artlogic/.pyenv/root/versions/2.7.15/bin/python2
2.7.14: /home/artlogic/.pyenv/root/versions/2.7.14/bin/python2
2.7.13: /usr/bin/python2
2.7.13: /usr/bin/python2.7

PEP 508 Information:

{'implementation_name': 'cpython',
 'implementation_version': '3.7.7',
 'os_name': 'posix',
 'platform_machine': 'x86_64',
 'platform_python_implementation': 'CPython',
 'platform_release': '4.9.0-12-amd64',
 'platform_system': 'Linux',
 'platform_version': '#1 SMP Debian 4.9.210-1 (2020-01-20)',
 'python_full_version': '3.7.7',
 'python_version': '3.7',
 'sys_platform': 'linux'}

System environment variables:

CLUTTER_IM_MODULE
PIPENV_VENV_IN_PROJECT
NVM_DIR
LS_COLORS
XDG_MENU_PREFIX
LANG
GDM_LANG
DISPLAY
PYENV_ROOT
OLDPWD
COLORTERM
NVM_CD_FLAGS
USERNAME
PYENV_VIRTUALENV_INIT
PYENV_HOOK_PATH
XDG_VTNR
SSH_AUTH_SOCK
XDG_SESSION_ID
USER
PYENV_DIR
DESKTOP_SESSION
RBENV_SHELL
QT4_IM_MODULE
QT_QPA_PLATFORMTHEME
PWD
HOME
JOURNAL_STREAM
SSH_AGENT_PID
PYENV_VERSION
QT_ACCESSIBILITY
XDG_SESSION_TYPE
XDG_DATA_DIRS
XDG_SESSION_DESKTOP
GJS_DEBUG_OUTPUT
GTK_MODULES
WINDOWPATH
VTE_VERSION
TERM
SHELL
RBENV_ROOT
QT_IM_MODULE
XMODIFIERS
NVM_BIN
XDG_CURRENT_DESKTOP
GPG_AGENT_INFO
QT_LINUX_ACCESSIBILITY_ALWAYS_ON
XDG_SEAT
SHLVL
PYENV_SHELL
WINDOWID
GDMSESSION
GNOME_DESKTOP_SESSION_ID
LOGNAME
DBUS_SESSION_BUS_ADDRESS
XDG_RUNTIME_DIR
XAUTHORITY
PATH
NVM_INC
GJS_DEBUG_TOPICS
SESSION_MANAGER
GTK_IM_MODULE
PIP_DISABLE_PIP_VERSION_CHECK
PYTHONDONTWRITEBYTECODE
PIP_SHIMS_BASE_MODULE
PIP_PYTHON_PATH
PYTHONFINDER_IGNORE_UNSUPPORTED

Pipenv–specific environment variables:

PIPENV_VENV_IN_PROJECT: 1

Debug–specific environment variables:

PATH: /home/artlogic/.pyenv/root/versions/3.7.7/bin:/home/artlogic/.pyenv/root/libexec:/home/artlogic/.pyenv/root/plugins/python-build/bin:/home/artlogic/.rbenv/root/shims:/home/artlogic/.rbenv/root/bin:/home/artlogic/.rbenv/ruby-build/bin:/home/artlogic/.pyenv/pyenv-virtualenv/shims:/home/artlogic/.pyenv/root/shims:/home/artlogic/.pyenv/root/bin:/home/artlogic/.pyenv/pyenv-virtualenv/bin:/home/artlogic/.dotfiles/config/.nvm/versions/node/v12.16.3/bin:/home/artlogic/.cargo/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
SHELL: /bin/bash
LANG: en_US.UTF-8
PWD: /home/artlogic/Code/random/pipenv-again

Type

Source

artlogic

👍7

Most helpful comment

Yes, Pipenv uses a different finding approach that the highest version goes first. You can anyway pass the python version by pipenv install --python=3.7.

Please change this behaviour. It is very unexpected. When you specify --python 3.8 it can happen on MacOS, that it uses 3.8.4 from homebrew instead of the 3.8.3 from pyenv. Pyenv can lag a bit, but homebrew changes the default python version and removes the previous one without any warning, which is why many of us use pyenv, so they have control over the versions installed and know when to update virtualenvs at their own pace.

It also goes against the notion of python3.8 -m <module> where one expects the module to use the python version of that it is installed in. It technically does, but it's unexpected that it switches versions for virtual env selection.

Last but not least, pinning my Pipfile to a version gets trickled to source repositories, which then trickles to CI/CD pipefiles and complicates that workflow, when a CI tests against multiple python versions. One normally would use pipenv --deploy, but that won't work if Pipfile is pinned against a specific version.

vylmen on 21 Jul 2020

👍7

All 10 comments

I am also seeing this behavior. It seems pipenv is using my system python (3.8.x) instead of my pyenv default python (3.6.x).

ms-lolo on 13 Jun 2020

Yes, Pipenv uses a different finding approach that the highest version goes first. You can anyway pass the python version by pipenv install --python=3.7.

frostming on 15 Jun 2020

Yes, Pipenv uses a different finding approach that the highest version goes first. You can anyway pass the python version by pipenv install --python=3.7.

Are you saying this was an intentional change and the documentation is out of date?

artlogic on 15 Jun 2020

Yes, it is not accurate

frostming on 15 Jun 2020

Would you accept a PR to update the docs for the time being?

This does feel like a bit of a UX regression to me. I found it very unexpected that pipenv chose a python that wasn't my current default to make the virtualenv. If you wouldn't mind, what's the rationale behind this change?

artlogic on 15 Jun 2020

👍4

Yes, Pipenv uses a different finding approach that the highest version goes first. You can anyway pass the python version by pipenv install --python=3.7.

vylmen on 21 Jul 2020

👍7

Last but not least, pinning my Pipfile to a version gets trickled to source repositories, which then trickles to CI/CD pipefiles and complicates that workflow, when a CI tests against multiple python versions. One normally would use pipenv --deploy, but that won't work if Pipfile is pinned against a specific version.

This seems to be no longer true, since --python option only writes to Pipfile when it creates one, not when one already exists. But to add to confusion, it writes python_version = "3.8" when using --python 3.8.3, so pipenv --rm && pipenv install will now select a different version :(

vylmen on 21 Jul 2020

It would be great if the docs were updated to reflect how pipenv install picks a "default python" to remove the confusion :)

In my case pipenv picked the python version installed by homebrew at /usr/local/bin/python3 which is not the default python on my system when I call which python.

Since I use pyenv to set / manage the default python on my system, I found a relatively painless workaround by adding the below in my .zprofile:

export PIPENV_DEFAULT_PYTHON_VERSION=$(pyenv which python)