Pipenv: Pipfile and Pipfile.lock are not distributable

[l0b0]

Issue description

A colleague tested out a dependency for our project, and I have been pulling their changes and syncing my pipenv accordingly. However, after they uninstalled a package and I synced using the new Pipfile and Pipfile.lock my pipenv still has the package.

Steps to replicate

On a Docker instance based on the official ubuntu:18.04 image:

$ pipenv --support
Error: no such option: --support
$ cat > Pipfile <<EOF
> [[source]]
> url = "https://pypi.org/simple"
> verify_ssl = true
> name = "pypi"
> 
> [packages]
> pyinotify = "*"
> EOF
$ pipenv lock
Creating a virtualenv for this project...
Pipfile: /src/app/foo/Pipfile
Using /usr/bin/python3 (3.6.5) to create virtualenv...
⠋Already using interpreter /usr/bin/python3
Using base prefix '/usr'
New python executable in /src/app/foo/.venv/bin/python3
Also creating executable in /src/app/foo/.venv/bin/python
Installing setuptools, pip, wheel...done.

Virtualenv location: /src/app/foo/.venv
Locking [dev-packages] dependencies...
Locking [packages] dependencies...
Updated Pipfile.lock (2088fe)!
$ pipenv sync
Installing dependencies from Pipfile.lock (2088fe)...
  🐍   ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 1/1 — 00:00:02
All dependencies are now up-to-date!
$ sed -i -e '/pyinotify/d' Pipfile
$ pipenv lock
Locking [dev-packages] dependencies...
Locking [packages] dependencies...
Updated Pipfile.lock (3cbc06)!
$ cat Pipfile*
[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[packages]
{
    "_meta": {
        "hash": {
            "sha256": "ebffa69a1fa192d1cef7cb42ad79231ca976565c5ce371a70160b3048d3cbc06"
        },
        "pipfile-spec": 6,
        "requires": {},
        "sources": [
            {
                "name": "pypi",
                "url": "https://pypi.org/simple",
                "verify_ssl": true
            }
        ]
    },
    "default": {},
    "develop": {}
}
$ pipenv sync
Installing dependencies from Pipfile.lock (3cbc06)...
  🐍   ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 0/0 — 00:00:00
All dependencies are now up-to-date!
$ pipenv graph
pyinotify==0.9.6

Expected result

pyinotify should no longer be installed.

Actual result

pyinotify is still installed.

[uranusjr]

sync does not uninstall packages, only install them. You need pipenv clean.

This particular design decision is made so a local virtual environment can slightly diverge from the Pipfile (and lock file), to e.g. test a package before adding it to the dependency tree.

Side note for fellow maintainers: Maybe we can add a --clean flag to sync to run clean after sync automatically? Or, maybe this should be the default behaviour (since the verb sync indeed can be interpreted to include deletion by default), and a --no-clean flag disables cleaning? @techalchemy @erinxocon It’d be awesome @ncoghlan if you could provide some insights on this issue.

[l0b0]

But pipenv clean does not have a --dev flag. How do I tell it whether it should uninstall things which are only in dev-packages? Consider a situation where a production dependency is changed to a dev dependency. Now pipenv clean won't remove the package from a production environment.

[uranusjr]

clean in general should be smart enough to know whether it should remove dev-packages or not. Do tell us if you ever found an edge case though!

[ncoghlan]

Given the existing behaviour, and the number of other things being worked on at the moment, it almost certainly isn't worth the hassle of updating pipenv sync to also imply pipenv clean, especially as we can assume that there are already folks out there relying on the current behaviour.

The main user-centric rationale for doing things the way pipenv currently does them is that when using source control, the current pipenv sync can be undone by reverting to a previous version of the lock file, and then running the sync operation again. By contrast, when pipenv clean removes packages from the current virtual environment, that's it - if you want to add them back, you have to read the list of what was uninstalled, and add them back individually.