pipenv uninstall --all-dev uninstalls shared default dependencies

Created on 2 May 2019 · 6Comments · Source: pypa/pipenv

Issue description

pipenv uninstall --all-dev uninstalls shared default dependencies

Expected result

When running pipenv uninstall --all-dev I'd expect all dev dependencies and their dependencies to be removed, _except_ those shared by default dependencies. Doing so breaks the application and forces me to run pipenv install.

Actual result

At least on one scenario that I observed, pipenv uninstall --all-dev will actually remove dev dependencies that are shared by default dependencies.

Steps to replicate

I'm using this sample application of mine to reproduce the behavior: https://github.com/fbcbarbosa/flask-chuck-norris

You'll notice that click is shared by black (a dev dependency) and Flask (a "default" dependency. When running pipenv uninstall --all-dev, click is also removed, breaking Flask.

$ pipenv install
...
$ pipenv graph
black==19.3b0
  - appdirs [required: Any, installed: 1.4.3]
  - attrs [required: >=18.1.0, installed: 19.1.0]
  - click [required: >=6.5, installed: 7.0]
  - toml [required: >=0.9.4, installed: 0.10.0]
Flask==1.0.2
  - click [required: >=5.1, installed: 7.0]
  - itsdangerous [required: >=0.24, installed: 1.1.0]
  - Jinja2 [required: >=2.10, installed: 2.10.1]
    - MarkupSafe [required: >=0.23, installed: 1.1.1]
  - Werkzeug [required: >=0.14, installed: 0.15.2]
pyjokes==0.5.0
pytest==4.4.1
  - atomicwrites [required: >=1.0, installed: 1.3.0]
  - attrs [required: >=17.4.0, installed: 19.1.0]
  - more-itertools [required: >=4.0.0, installed: 7.0.0]
  - pluggy [required: >=0.9, installed: 0.9.0]
  - py [required: >=1.5.0, installed: 1.8.0]
  - setuptools [required: Any, installed: 41.0.1]
  - six [required: >=1.10.0, installed: 1.12.0]
$ pipenv uninstall --all-dev
...
Removing black from Pipfile…
Uninstalling click…
Uninstalling Click-7.0:
  Successfully uninstalled Click-7.0

No package click to remove from Pipfile.
...
$ pipenv graph # graph says click is installed, but it isn't
Flask==1.0.2
  - click [required: >=5.1, installed: 7.0]
  - itsdangerous [required: >=0.24, installed: 1.1.0]
  - Jinja2 [required: >=2.10, installed: 2.10.1]
    - MarkupSafe [required: >=0.23, installed: 1.1.1]
  - Werkzeug [required: >=0.14, installed: 0.15.2]
pyjokes==0.5.0
$ pipenv shell
Launching subshell in virtual environment…
...
$ pip freeze
Flask==1.0.2
itsdangerous==1.1.0
Jinja2==2.10.1
MarkupSafe==1.1.1
pyjokes==0.5.0
Werkzeug==0.15.2

Running the application will throw an error, because click is not installed.

To tell the truth, the documentation is not clear on the expected behavior:

--all-dev             Un-install all package from [dev-packages].

Still, I find it unlikely that this is working as intended. Since in any reasonably sized project, dev dependencies and default dependencies will share some common dependencies, running pipenv uninstall --all-dev would always need to be followed by pipenv install.

My use case is that I want to run pipenv as part of my CI, and after running my tests I'd like to prune all dev dependencies before shipping my application - specifically, I'll be copying the pipenv environment inside a docker image.

To give another example, I'd expect it to work like npm prune --production.

$ pipenv --support

Pipenv version: '2018.11.26'

Pipenv location: '/usr/lib/python3.7/site-packages/pipenv'

Python location: '/usr/bin/python3'

Python installations found:

3.7.3: /home/walker/Workspace/src/github.com/fbcbarbosa/flask-chuck/.venv/bin/python3
3.7.3: /usr/bin/python3.7
3.7.3: /usr/bin/python3.7m
2.7.15: /usr/bin/python2

PEP 508 Information:

{'implementation_name': 'cpython',
 'implementation_version': '3.7.3',
 'os_name': 'posix',
 'platform_machine': 'x86_64',
 'platform_python_implementation': 'CPython',
 'platform_release': '5.0.9-200.fc29.x86_64',
 'platform_system': 'Linux',
 'platform_version': '#1 SMP Mon Apr 22 00:55:30 UTC 2019',
 'python_full_version': '3.7.3',
 'python_version': '3.7',
 'sys_platform': 'linux'}

System environment variables:

PIPENV_VENV_IN_PROJECT
NVM_DIR
LS_COLORS
TERMINATOR_UUID
XDG_MENU_PREFIX
MODULES_RUN_QUARANTINE
LANG
GDM_LANG
HISTCONTROL
LESS
DISPLAY
HOSTNAME
OLDPWD
SDKMAN_CANDIDATES_API
KUBECONFIG
EDITOR
COLORTERM
NVM_CD_FLAGS
FPATH
DESKTOP_AUTOSTART_ID
USERNAME
JAVA_HOME
KDEDIRS
XDG_VTNR
GIO_LAUNCHED_DESKTOP_FILE_PID
ZSH
SSH_AUTH_SOCK
XDG_SESSION_ID
MODULES_CMD
USER
ENV
PAGER
LSCOLORS
DESKTOP_SESSION
GOPATH
GRADLE_HOME
PWD
HOME
KOPS_CLUSTER_NAME
SSH_AGENT_PID
XDG_SESSION_TYPE
BASH_ENV
XDG_DATA_DIRS
TERMINATOR_DBUS_NAME
XDG_SESSION_DESKTOP
LOADEDMODULES
SDKMAN_DIR
TERMINATOR_DBUS_PATH
MAIL
ZSH_CUSTOM
WINDOWPATH
SHELL
VTE_VERSION
TERM
QT_IM_MODULE
XMODIFIERS
NVM_BIN
SDKMAN_CANDIDATES_DIR
XDG_CURRENT_DESKTOP
GIO_LAUNCHED_DESKTOP_FILE
KOPS_STATE_STORE
XDG_SEAT
SHLVL
MANPATH
MODULEPATH
GDMSESSION
LOGNAME
DBUS_SESSION_BUS_ADDRESS
XDG_RUNTIME_DIR
XAUTHORITY
MODULEPATH_modshare
GOBIN
PATH
SDKMAN_VERSION
DATASTORE_TYPE
MODULESHOME
HISTSIZE
SDKMAN_PLATFORM
SESSION_MANAGER
LESSOPEN
BASH_FUNC_module%%
BASH_FUNC__module_raw%%
BASH_FUNC_switchml%%
_
APPLICATION_INSIGHTS_NO_DIAGNOSTIC_CHANNEL
TERM_PROGRAM
TERM_PROGRAM_VERSION
PIP_DISABLE_PIP_VERSION_CHECK
PYTHONDONTWRITEBYTECODE
PIP_PYTHON_PATH
PIPENV_ACTIVE
LC_CTYPE
VIRTUAL_ENV
PS1
PIP_SHIMS_BASE_MODULE
PYTHONFINDER_IGNORE_UNSUPPORTED

Pipenv–specific environment variables:

PIPENV_VENV_IN_PROJECT: true
PIPENV_ACTIVE: 1

Debug–specific environment variables:

PATH: /home/walker/Workspace/src/github.com/fbcbarbosa/flask-chuck/.venv/bin:/home/walker/.nvm/versions/node/v12.1.0/bin:/home/walker/.sdkman/candidates/java/current/bin:/home/walker/.sdkman/candidates/gradle/current/bin:/usr/share/Modules/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/home/walker/bin:/var/lib/snapd/snap/bin:/home/walker/Workspace/bin:/home/walker/go/bin:/home/walker/Workspace/bin:/home/walker/go/bin:/home/walker/Workspace/bin:/home/walker/go/bin
SHELL: /bin/zsh
EDITOR: vim
LANG: en_US.UTF-8
PWD: /home/walker/Workspace/src/github.com/fbcbarbosa/flask-chuck
VIRTUAL_ENV: /home/walker/Workspace/src/github.com/fbcbarbosa/flask-chuck/.venv

Contents of Pipfile ('/home/walker/Workspace/src/github.com/fbcbarbosa/flask-chuck/Pipfile'):

[[source]]
name = "pypi"
url = "https://pypi.org/simple"
verify_ssl = true

[dev-packages]

[packages]
flask = "==1.0.2"
pyjokes = "==0.5.0"

[requires]
python_version = "3.7.3"

Contents of Pipfile.lock ('/home/walker/Workspace/src/github.com/fbcbarbosa/flask-chuck/Pipfile.lock'):

{
    "_meta": {
        "hash": {
            "sha256": "6a10557b3e2690522705a2875ca320334b9903a13e8bf0af2fb0273707afdbf0"
        },
        "pipfile-spec": 6,
        "requires": {
            "python_version": "3.7.3"
        },
        "sources": [
            {
                "name": "pypi",
                "url": "https://pypi.org/simple",
                "verify_ssl": true
            }
        ]
    },
    "default": {
        "click": {
            "hashes": [
                "sha256:2335065e6395b9e67ca716de5f7526736bfa6ceead690adf616d925bdc622b13",
                "sha256:5b94b49521f6456670fdb30cd82a4eca9412788a93fa6dd6df72c94d5a8ff2d7"
            ],
            "version": "==7.0"
        },
        "flask": {
            "hashes": [
                "sha256:2271c0070dbcb5275fad4a82e29f23ab92682dc45f9dfbc22c02ba9b9322ce48",
                "sha256:a080b744b7e345ccfcbc77954861cb05b3c63786e93f2b3875e0913d44b43f05"
            ],
            "index": "pypi",
            "version": "==1.0.2"
        },
        "itsdangerous": {
            "hashes": [
                "sha256:321b033d07f2a4136d3ec762eac9f16a10ccd60f53c0c91af90217ace7ba1f19",
                "sha256:b12271b2047cb23eeb98c8b5622e2e5c5e9abd9784a153e9d8ef9cb4dd09d749"
            ],
            "version": "==1.1.0"
        },
        "jinja2": {
            "hashes": [
                "sha256:065c4f02ebe7f7cf559e49ee5a95fb800a9e4528727aec6f24402a5374c65013",
                "sha256:14dd6caf1527abb21f08f86c784eac40853ba93edb79552aa1e4b8aef1b61c7b"
            ],
            "version": "==2.10.1"
        },
        "markupsafe": {
            "hashes": [
                "sha256:00bc623926325b26bb9605ae9eae8a215691f33cae5df11ca5424f06f2d1f473",
                "sha256:09027a7803a62ca78792ad89403b1b7a73a01c8cb65909cd876f7fcebd79b161",
                "sha256:09c4b7f37d6c648cb13f9230d847adf22f8171b1ccc4d5682398e77f40309235",
                "sha256:1027c282dad077d0bae18be6794e6b6b8c91d58ed8a8d89a89d59693b9131db5",
                "sha256:24982cc2533820871eba85ba648cd53d8623687ff11cbb805be4ff7b4c971aff",
                "sha256:29872e92839765e546828bb7754a68c418d927cd064fd4708fab9fe9c8bb116b",
                "sha256:43a55c2930bbc139570ac2452adf3d70cdbb3cfe5912c71cdce1c2c6bbd9c5d1",
                "sha256:46c99d2de99945ec5cb54f23c8cd5689f6d7177305ebff350a58ce5f8de1669e",
                "sha256:500d4957e52ddc3351cabf489e79c91c17f6e0899158447047588650b5e69183",
                "sha256:535f6fc4d397c1563d08b88e485c3496cf5784e927af890fb3c3aac7f933ec66",
                "sha256:62fe6c95e3ec8a7fad637b7f3d372c15ec1caa01ab47926cfdf7a75b40e0eac1",
                "sha256:6dd73240d2af64df90aa7c4e7481e23825ea70af4b4922f8ede5b9e35f78a3b1",
                "sha256:717ba8fe3ae9cc0006d7c451f0bb265ee07739daf76355d06366154ee68d221e",
                "sha256:79855e1c5b8da654cf486b830bd42c06e8780cea587384cf6545b7d9ac013a0b",
                "sha256:7c1699dfe0cf8ff607dbdcc1e9b9af1755371f92a68f706051cc8c37d447c905",
                "sha256:88e5fcfb52ee7b911e8bb6d6aa2fd21fbecc674eadd44118a9cc3863f938e735",
                "sha256:8defac2f2ccd6805ebf65f5eeb132adcf2ab57aa11fdf4c0dd5169a004710e7d",
                "sha256:98c7086708b163d425c67c7a91bad6e466bb99d797aa64f965e9d25c12111a5e",
                "sha256:9add70b36c5666a2ed02b43b335fe19002ee5235efd4b8a89bfcf9005bebac0d",
                "sha256:9bf40443012702a1d2070043cb6291650a0841ece432556f784f004937f0f32c",
                "sha256:ade5e387d2ad0d7ebf59146cc00c8044acbd863725f887353a10df825fc8ae21",
                "sha256:b00c1de48212e4cc9603895652c5c410df699856a2853135b3967591e4beebc2",
                "sha256:b1282f8c00509d99fef04d8ba936b156d419be841854fe901d8ae224c59f0be5",
                "sha256:b2051432115498d3562c084a49bba65d97cf251f5a331c64a12ee7e04dacc51b",
                "sha256:ba59edeaa2fc6114428f1637ffff42da1e311e29382d81b339c1817d37ec93c6",
                "sha256:c8716a48d94b06bb3b2524c2b77e055fb313aeb4ea620c8dd03a105574ba704f",
                "sha256:cd5df75523866410809ca100dc9681e301e3c27567cf498077e8551b6d20e42f",
                "sha256:e249096428b3ae81b08327a63a485ad0878de3fb939049038579ac0ef61e17e7"
            ],
            "version": "==1.1.1"
        },
        "pyjokes": {
            "hashes": [
                "sha256:01b90474d5c889c21cae88f0d5fc8db1334b2891a16df75cbf9a0886bfdea653",
                "sha256:745299c99b7a4015251b4876ecd59dd62803f458a9f0540d036f0f6627dcc67e"
            ],
            "index": "pypi",
            "version": "==0.5.0"
        },
        "werkzeug": {
            "hashes": [
                "sha256:0a73e8bb2ff2feecfc5d56e6f458f5b99290ef34f565ffb2665801ff7de6af7a",
                "sha256:7fad9770a8778f9576693f0cc29c7dcc36964df916b83734f4431c0e612a7fbc"
            ],
            "version": "==0.15.2"
        }
    },
    "develop": {}
}

Type good first issue help wanted

Source

fernandrone

Most helpful comment

Yeah I'd definitely merge a PR which improves the behavior but I think we'd need an accompanying addition to /peeps describing what exactly we expect here, I know I haven't spent too much time considering this because we aren't looking to add too many knobs so e just made it as simple as possible -- just remove the things

it does seem a bit obvious that we shouldn't remove the things that are also in the other section though...

techalchemy on 20 May 2019

👍2

All 6 comments

Also, if this is confirmed to be an issue, I'd happily open a PR and contribute!

This is the first _sane_ Python package management workflow I've found, and the first one that _almost_ enables me to purge dev dependencies from the environment (_almost_ due to this bug... at least it's not that bad, because I can always run pipenv install again and it will _only_ reinstall missing shared dependencies).

fernandrone on 2 May 2019

Currently pipenv just simply removes all packages in dev but not smart to detect the shared dependencies of default and dev packages. Personally I feels what you are proposing is quite reasonable.

jxltom on 9 May 2019

👍2

it does seem a bit obvious that we shouldn't remove the things that are also in the other section though...

techalchemy on 20 May 2019

👍2

Bump—This is still an issue. Not too difficult to fix when it happens, but annoying and definitely unexpected.

cur33 on 5 Feb 2020

May I draw your attention to this behaviour:

pipenv install deprecation         # note: this will install pyparsing since its a dependency of deprecation
pipenv install --dev pyparsing 
pipenv uninstall --all-dev         # note: this will uninstall pyparsing even if it is in the [packages] section of the Pipfile!!!
pipenv sync                        # note: this will reinstall pyparsing

So to sumarize the problem from my view:

What not works: pipenv uninstall --all-dev. This removes things that are still needed 👎
What works: pipenv --rm && pipenv sync. This realy slow but rock solid 👍
What I'm looking for is something such as 'pipenv clean --keep-non-dev-only' or pipenv uninstall --all-dev-only. This would by rock solid and fast 👍 👍