Pipenv: install's `--keep-outdated` does not seem to be respected

[seansfkelley]

(Closed as duplicate, see issue at #966 and PR at #3304)

When I have outdated dependencies, installing a new package with the --keep-outdated flag still updates dependencies that have no overlap with my newly-requested dependency. Similar issues (but pre---keep-oudated, I think?): #966, #1219, #1409.

Describe your environment

1. OS X 10.13.2
2. Python version: 3.6.4
3. Pipenv version: 11.0.2

Expected result

Packages that have no overlap in their dependencies with my current set of dependencies should not change my current set of dependencies when --keep-outdated is specified.

Actual result

My existing packages are updated as if I had not specified the flag.

Steps to replicate

> mkdir foo && cd foo
> pipenv shell
Creating a virtualenv for this project…
Using base prefix '/usr/local/Cellar/python3/3.6.4_2/Frameworks/Python.framework/Versions/3.6'
New python executable in /Users/seankelley/.virtualenvs/foo-9CvoEbl5/bin/python3.6
Also creating executable in /Users/seankelley/.virtualenvs/foo-9CvoEbl5/bin/python
Installing setuptools, pip, wheel...done.

Virtualenv location: /Users/seankelley/.virtualenvs/foo-9CvoEbl5
Creating a Pipfile for this project…
Spawning environment shell (/usr/local/bin/zsh). Use 'exit' to leave.
. /Users/seankelley/.virtualenvs/foo-9CvoEbl5/bin/activate
> pipenv install boto3==1.6.2
Installing boto3==1.6.2…                                                                                                                                                                  
Collecting boto3==1.6.2                                                                                                                                                                   
  Using cached boto3-1.6.2-py2.py3-none-any.whl                                                                                                                                           
Collecting jmespath<1.0.0,>=0.7.1 (from boto3==1.6.2)                                                                                                                                     
  Using cached jmespath-0.9.3-py2.py3-none-any.whl                                                                                                                                        
Collecting botocore<1.10.0,>=1.9.2 (from boto3==1.6.2)                                                                                                                                    
  Using cached botocore-1.9.3-py2.py3-none-any.whl                                                                                                                                        
Collecting s3transfer<0.2.0,>=0.1.10 (from boto3==1.6.2)
  Using cached s3transfer-0.1.13-py2.py3-none-any.whl
Collecting docutils>=0.10 (from botocore<1.10.0,>=1.9.2->boto3==1.6.2)
  Using cached docutils-0.14-py3-none-any.whl
Collecting python-dateutil<3.0.0,>=2.1 (from botocore<1.10.0,>=1.9.2->boto3==1.6.2)
  Using cached python_dateutil-2.6.1-py2.py3-none-any.whl
Collecting six>=1.5 (from python-dateutil<3.0.0,>=2.1->botocore<1.10.0,>=1.9.2->boto3==1.6.2)
  Using cached six-1.11.0-py2.py3-none-any.whl
Installing collected packages: jmespath, docutils, six, python-dateutil, botocore, s3transfer, boto3
Successfully installed boto3-1.6.2 botocore-1.9.3 docutils-0.14 jmespath-0.9.3 python-dateutil-2.6.1 s3transfer-0.1.13 six-1.11.0

Adding boto3==1.6.2 to Pipfile's [packages]…
Pipfile.lock not found, creating…
Locking [dev-packages] dependencies…
Locking [packages] dependencies…
Updated Pipfile.lock (1faca3)!
Installing dependencies from Pipfile.lock (1faca3)…
  🐍   ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 7/7 — 00:00



... and here I edit the Pipfile, replacing boto3's `==1.6.2` specification with `*` in order to recreate the environment where I first discovered this issue ...



> pipenv install --keep-outdated coverage         
Installing coverage…
Collecting coverage
  Using cached coverage-4.5.1-cp36-cp36m-macosx_10_12_x86_64.whl
Installing collected packages: coverage
Successfully installed coverage-4.5.1

Adding coverage to Pipfile's [packages]…
Pipfile.lock (1faca3) out of date, updating to (63af11)…
Locking [dev-packages] dependencies…
Locking [packages] dependencies…
Updated Pipfile.lock (63af11)!
Installing dependencies from Pipfile.lock (63af11)…
  🐍   ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 8/8 — 0

Pipfile pre-coverage, pre-editing

[[source]]

url = "https://pypi.python.org/simple"
verify_ssl = true
name = "pypi"


[packages]

"boto3" = "==1.6.2"


[dev-packages]



[requires]

python_version = "3.6"

Pipfile pre-coverage, post-editing

[[source]]

url = "https://pypi.python.org/simple"
verify_ssl = true
name = "pypi"


[packages]

"boto3" = "*"


[dev-packages]



[requires]

python_version = "3.6"

Pipfile.lock pre-coverage

{
    "_meta": {
        "hash": {
            "sha256": "8d7dcd28948fa28543f1ad7b866b5c37c583afe5a78a010ab3213e9bec1faca3"
        },
        "pipfile-spec": 6,
        "requires": {
            "python_version": "3.6"
        },
        "sources": [
            {
                "name": "pypi",
                "url": "https://pypi.python.org/simple",
                "verify_ssl": true
            }
        ]
    },
    "default": {
        "boto3": {
            "hashes": [
                "sha256:0537aa3c840b2a6c9cdc53704b1ca27c19f1850e16056db1f6241917a5eee648",
                "sha256:1576a72dd1a6435b29b41d541411fcabcef2777de9abebc50d01046ca24447f9"
            ],
            "version": "==1.6.2"
        },
        "botocore": {
            "hashes": [
                "sha256:8a8c59f4c7b4e3521d82aee4b8cba55e80de44617649c2320823505bfebbcaa4",
                "sha256:d5b892ab86cd3e8d6cb570dd5275bf1c600cbbf9f07a40a22bcdd9023c0e844f"
            ],
            "version": "==1.9.3"
        },
        "docutils": {
            "hashes": [
                "sha256:02aec4bd92ab067f6ff27a38a38a41173bf01bed8f89157768c1573f53e474a6",
                "sha256:51e64ef2ebfb29cae1faa133b3710143496eca21c530f3f71424d77687764274",
                "sha256:7a4bd47eaf6596e1295ecb11361139febe29b084a87bf005bf899f9a42edc3c6"
            ],
            "version": "==0.14"
        },
        "jmespath": {
            "hashes": [
                "sha256:6a81d4c9aa62caf061cb517b4d9ad1dd300374cd4706997aff9cd6aedd61fc64",
                "sha256:f11b4461f425740a1d908e9a3f7365c3d2e569f6ca68a2ff8bc5bcd9676edd63"
            ],
            "version": "==0.9.3"
        },
        "python-dateutil": {
            "hashes": [
                "sha256:891c38b2a02f5bb1be3e4793866c8df49c7d19baabf9c1bad62547e0b4866aca",
                "sha256:95511bae634d69bc7329ba55e646499a842bc4ec342ad54a8cdb65645a0aad3c"
            ],
            "version": "==2.6.1"
        },
        "s3transfer": {
            "hashes": [
                "sha256:90dc18e028989c609146e241ea153250be451e05ecc0c2832565231dacdf59c1",
                "sha256:c7a9ec356982d5e9ab2d4b46391a7d6a950e2b04c472419f5fdec70cc0ada72f"
            ],
            "version": "==0.1.13"
        },
        "six": {
            "hashes": [
                "sha256:70e8a77beed4562e7f14fe23a786b54f6296e34344c23bc42f07b15018ff98e9",
                "sha256:832dc0e10feb1aa2c68dcc57dbb658f1c7e65b9b61af69048abc87a2db00a0eb"
            ],
            "version": "==1.11.0"
        }
    },
    "develop": {}
}

Pipfile post-coverage

[[source]]

url = "https://pypi.python.org/simple"
verify_ssl = true
name = "pypi"


[packages]

"boto3" = "*"
coverage = "*"


[dev-packages]



[requires]

python_version = "3.6"


[pipenv]

keep_outdated = true

Pipfile.lock post-coverage

{
    "_meta": {
        "hash": {
            "sha256": "68b45659b8e63d966f0ad9bd96550a54c0b87ef67baef31d132042e90d63af11"
        },
        "pipfile-spec": 6,
        "requires": {
            "python_version": "3.6"
        },
        "sources": [
            {
                "name": "pypi",
                "url": "https://pypi.python.org/simple",
                "verify_ssl": true
            }
        ]
    },
    "default": {
        "boto3": {
            "hashes": [
                "sha256:cf5768f1b9f26d5f82369ef97e11eee0010d10f8723b081c61ff17a1cd199bea",
                "sha256:d817c21abcb41f390cec28e444d180db7d40ec2786ff5f54813af52e53598362"
            ],
            "version": "==1.6.3"
        },
        "botocore": {
            "hashes": [
                "sha256:8a8c59f4c7b4e3521d82aee4b8cba55e80de44617649c2320823505bfebbcaa4",
                "sha256:d5b892ab86cd3e8d6cb570dd5275bf1c600cbbf9f07a40a22bcdd9023c0e844f"
            ],
            "version": "==1.9.3"
        },
        "coverage": {
            "hashes": [
                "sha256:03481e81d558d30d230bc12999e3edffe392d244349a90f4ef9b88425fac74ba",
                "sha256:0b136648de27201056c1869a6c0d4e23f464750fd9a9ba9750b8336a244429ed",
                "sha256:198626739a79b09fa0a2f06e083ffd12eb55449b5f8bfdbeed1df4910b2ca640",
                "sha256:28b2191e7283f4f3568962e373b47ef7f0392993bb6660d079c62bd50fe9d162",
                "sha256:2eb564bbf7816a9d68dd3369a510be3327f1c618d2357fa6b1216994c2e3d508",
                "sha256:337ded681dd2ef9ca04ef5d93cfc87e52e09db2594c296b4a0a3662cb1b41249",
                "sha256:3a2184c6d797a125dca8367878d3b9a178b6fdd05fdc2d35d758c3006a1cd694",
                "sha256:3c79a6f7b95751cdebcd9037e4d06f8d5a9b60e4ed0cd231342aa8ad7124882a",
                "sha256:3d72c20bd105022d29b14a7d628462ebdc61de2f303322c0212a054352f3b287",
                "sha256:3eb42bf89a6be7deb64116dd1cc4b08171734d721e7a7e57ad64cc4ef29ed2f1",
                "sha256:4635a184d0bbe537aa185a34193898eee409332a8ccb27eea36f262566585000",
                "sha256:56e448f051a201c5ebbaa86a5efd0ca90d327204d8b059ab25ad0f35fbfd79f1",
                "sha256:5a13ea7911ff5e1796b6d5e4fbbf6952381a611209b736d48e675c2756f3f74e",
                "sha256:69bf008a06b76619d3c3f3b1983f5145c75a305a0fea513aca094cae5c40a8f5",
                "sha256:6bc583dc18d5979dc0f6cec26a8603129de0304d5ae1f17e57a12834e7235062",
                "sha256:701cd6093d63e6b8ad7009d8a92425428bc4d6e7ab8d75efbb665c806c1d79ba",
                "sha256:7608a3dd5d73cb06c531b8925e0ef8d3de31fed2544a7de6c63960a1e73ea4bc",
                "sha256:76ecd006d1d8f739430ec50cc872889af1f9c1b6b8f48e29941814b09b0fd3cc",
                "sha256:7aa36d2b844a3e4a4b356708d79fd2c260281a7390d678a10b91ca595ddc9e99",
                "sha256:7d3f553904b0c5c016d1dad058a7554c7ac4c91a789fca496e7d8347ad040653",
                "sha256:7e1fe19bd6dce69d9fd159d8e4a80a8f52101380d5d3a4d374b6d3eae0e5de9c",
                "sha256:8c3cb8c35ec4d9506979b4cf90ee9918bc2e49f84189d9bf5c36c0c1119c6558",
                "sha256:9d6dd10d49e01571bf6e147d3b505141ffc093a06756c60b053a859cb2128b1f",
                "sha256:be6cfcd8053d13f5f5eeb284aa8a814220c3da1b0078fa859011c7fffd86dab9",
                "sha256:c1bb572fab8208c400adaf06a8133ac0712179a334c09224fb11393e920abcdd",
                "sha256:de4418dadaa1c01d497e539210cb6baa015965526ff5afc078c57ca69160108d",
                "sha256:e05cb4d9aad6233d67e0541caa7e511fa4047ed7750ec2510d466e806e0255d6",
                "sha256:f3f501f345f24383c0000395b26b726e46758b71393267aeae0bd36f8b3ade80"
            ],
            "version": "==4.5.1"
        },
        "docutils": {
            "hashes": [
                "sha256:02aec4bd92ab067f6ff27a38a38a41173bf01bed8f89157768c1573f53e474a6",
                "sha256:51e64ef2ebfb29cae1faa133b3710143496eca21c530f3f71424d77687764274",
                "sha256:7a4bd47eaf6596e1295ecb11361139febe29b084a87bf005bf899f9a42edc3c6"
            ],
            "version": "==0.14"
        },
        "jmespath": {
            "hashes": [
                "sha256:6a81d4c9aa62caf061cb517b4d9ad1dd300374cd4706997aff9cd6aedd61fc64",
                "sha256:f11b4461f425740a1d908e9a3f7365c3d2e569f6ca68a2ff8bc5bcd9676edd63"
            ],
            "version": "==0.9.3"
        },
        "python-dateutil": {
            "hashes": [
                "sha256:891c38b2a02f5bb1be3e4793866c8df49c7d19baabf9c1bad62547e0b4866aca",
                "sha256:95511bae634d69bc7329ba55e646499a842bc4ec342ad54a8cdb65645a0aad3c"
            ],
            "version": "==2.6.1"
        },
        "s3transfer": {
            "hashes": [
                "sha256:90dc18e028989c609146e241ea153250be451e05ecc0c2832565231dacdf59c1",
                "sha256:c7a9ec356982d5e9ab2d4b46391a7d6a950e2b04c472419f5fdec70cc0ada72f"
            ],
            "version": "==0.1.13"
        },
        "six": {
            "hashes": [
                "sha256:70e8a77beed4562e7f14fe23a786b54f6296e34344c23bc42f07b15018ff98e9",
                "sha256:832dc0e10feb1aa2c68dcc57dbb658f1c7e65b9b61af69048abc87a2db00a0eb"
            ],
            "version": "==1.11.0"
        }
    },
    "develop": {}
}

[kennethreitz42]

the lockfile will always get re-generated if it is out-of-date.

[seansfkelley]

@kennethreitz Can you clarify what the intent of --keep-outdated is? The only documentation for it is "Keep out–dated dependencies from being updated in Pipfile.lock.", which seems pretty clear about _not_ changing those parts of the lockfile.

My goal here is to find a method to install a new dependency without updating the world, which is what it looked like this flag was for based on the discussion at #1255.

[kennethreitz42]

use pipenv update depname

[seansfkelley]

That doesn't work. If I do the same repro steps as above except substituting pipenv update --keep-outdated coverage or pipenv update coverage in place of pipenv install --keep-outdated coverage, the result is the same: boto3 and some of its dependencies are still updated.

[seansfkelley]

Ping @kennethreitz, what is the expected behavior here? The flag doesn't seem to do anything and the docs don't say much.

[swordsp]

Have tried pipenv install --selective-upgrade <packages> and it seems still update the whole Pipfile.lock .

[jacebrowning]

This is still an issue -- can we reopen?

I just converted my git links from https:// to ssh:// (still pinned to the same ref = <sha>) then ran $ pipenv install --keep-outdated, but all of my unpinned dependences got updated in Pipfile.lock.

IMO, --keep-outdated needs to be the default (and work as described) to avoid inadvertently pulling in a security venerability. We can't expect every project to have the resources to test every new version of every dependency.

[techalchemy]

@jacebrowning the specific issue with VCS repos is tracked in #2180, #1690, #1611, #2096 and has a PR open (#2209)

[techalchemy]

and we have another issue open tracking --keep-outdated with a separate pr

[seansfkelley]

@techalchemy which issue/PR is that? Nothing open I see now seems appropriate.

[matthijskooijman]

I'm running into the same issue, install --keep-outdated updating unrelated packages. None of my packages are installed from git, everything is just fetched from pypi (just like with @seansfkelley AFAICS), so none of the issues linked by @techalchemy seem to apply here.

[seansfkelley]

For those following along at home, it seems like this flag is indeed legitimately broken. From https://github.com/pypa/pipenv/issues/966#issuecomment-416784883:

--selective-upgrade is supposed to selectively upgrade only things that are dependencies of the new package, while --keep-outdated would hold back anything that satisfied the dependencies required by a new package. Slightly different, but I am fairly sure neither works correctly right now.

This issue should probably be re-opened to reflect reality.

[Nabellaleen]

Also looking for solution for more selective upgrade, not doing the .lock stuff by hand after each update :/
I think this issue should be re-opened, --selective-upgrade and --keep-outdated don't work yet

[henadzit]

Why is this closed? The issue is still present... It isn't possible to update one package without updating all packages. If you have lots of dependencies, you'll wait a few minutes just to see that pipenv upgraded packages and introduced a conflict.

[techalchemy]

966 #3304

[ncoghlan]

This issue ranks pretty highly on Google searches for "pipenv keep-outdated", so I've added the references to the open issues to the original post (otherwise folks may think the issue has been closed because it is working as expected, when it has actually been closed as a duplicate of an existing issue & PR).

[john-shine]

kennethreitz is an bigoted man, hope some cool guys take over this project