Pipenv: Document the "upgrade my dependancies" workflow

[jacobian]

As far as I can tell, the documentation doesn't seem to lay out a proscribed workflow for upgrading dependencies. I think this is a fairly common use-case, something like this:

As a user of pipenv, I've got a project with a Pipfile and a Pipfile.lock. It's been some time, and I'd like to know what's changed upstream, and to upgrade my dependencies (and test them as they go).

I'm happy to contribute this documentation myself, and I think I've figured out a good workflow. But I'm opening this issue to make sure I'm on the write track before I write something.

Here's what I think the upgrade workflow should look like:

1. Find out what's changed upstream: pipenv run pip list --outdated
2. Upgrade packages, two options:
   a. Want to upgrade everything? Just do pipenv install.
   b. Want to upgrade packages one-at-a-time? pipenv install --selective-upgrade <pkg> for each outdated package.
3. Run your tests, fix issues. If there are problems, pin versions in Pipfile.
4. Commit your upgraded Pipfile.lock (and perhaps Pipfile).

Does this sound right?

Something I don't understand about --selective-upgrade -- why does it modify Pipfile to add keep_outdated?

[kennethreitz42]

@jacobian sounds like we have some work to do, regarding your last point there.

[kennethreitz42]

That workflow sounds solid.

[kennethreitz42]

I feel a pain point in $ pipenv run pip list --outdated. I'd like to add a command that does that for you.

[jacobian]

I was wondering about that.

How about pipenv graph --outdated? Unlike pip list --outdated, it could show where in the dep tree the outdated thing lives. I think this is important information: I probably want to upgrade top-level dependencies (i.e. the things that I depend on), but unless there's a security issue or a serious bug, I probably _don't_ want to bother upgrading my secondary dependencies until I do a top-level upgrade.

[xsteadfastx]

pipenv install doesnt upgrade all packages here in 10.1.0. so whats the workflow? so docs would be great :)

[kennethreitz42]

that sounds like a great suggestion.

[ZhukovGreen]

Previously, when I was updating all packages, I was doing the following pipenv update --dev.
Now, rm Pipfile.lock and pipenv install. pipenv update disappeared.

A good idea may be to keep a backward compatibility and aware user about the new workflow

[zyv]

Hi there!

Just upgraded to 10.1.0 and found out that update vanished. In the past, I would run pipenv update --dry-run to find out what packaged needed updating and then pipenv update to actually update them. I didn't quite gather from this issue why update got removed and what the recommended workflow is right now, but if I understand the last posts correctly, the alternative currently is to run

1. pipenv run pip list --outdated
2. rm Pipfile.lock && pipenv install or is pipenv install --skip-lock the way to go?

Unfortunately, this isn't equivalent to pipenv update --dry-run, because pip list --outdated doesn't consider the restrictions on package versions in the Pipfile, so, for example, it reports Django 2.0.2 as being available, whereas I pinned it to the latest LTS in the Pipfile.

Can we maybe get a --dry-run mode for pipenv install --skip-lock that would ignore the lockfile and tell what packages it is going to update?!

Also, I'm wondering why one option is called --skip-lock and the opposite of it (the default?!) is called --ignore-pipfile... I would put them next to each other in the docs and I guess one could reconsider the naming, e.g. could --ignore-pipfile && --ignore-lock or --skip-lock && --skip-pipfile or --ignore-pipfile and --update (or --upgrade) have been better names?

Thanks!

[TimSimpsonR]

Also, I'm wondering why one option is called --skip-lock and the opposite of it (the default?!) is called --ignore-pipfile

Actually, I believe the default for pipenv install is --skip-lock. In my experience pipenv install will change the lock file if it detects an old dependency, as will pipenv sync. The only way to get it to use the lock file without modifying it is pipenv install --ignore-pipfile (note: I'm using pipenv version 10.1.0)

I am having a hard time understanding why keeping the update command wasn't clearer than the current situation. I also believed -maybe wrongly- that the whole point of pipenv was it stuck to using the lockfile in most situations while making it obvious how to update it (pun intended), but now it seems like getting it to not touch the lockfile takes a lot of effort.

[ghost]

Should pipenv install upgrade everything to latest with version="*"? I have seen that it is not doing this. I am using another source that points to an internal pypi.

[kakulukia]

please bring back update

[rhymes]

What about two new commands:

• pipenv outdated
• pipenv update (and pipenv update --dev)

?

[kennethreitz42]

update makes no sense when we have sync.

There needs to be an outdated command, and it'll likely be a flag to an existing command.

I'd like to keep the list of commands as short as possible, you see.

[techalchemy]

So can we get a plan together? I like the idea in the initial suggestion if adding —outdated to pipenv graph.

@kennethreitz to avoid confusion should we toss an update command in which only contains a warning to switch to sync until 10.2 or 11?

[kennethreitz42]

no, it's too late. switch already happened. It was across a major version for a reason.

[kennethreitz42]

We could add a vanity 'update' (which isn't listed in help), which prints out what to run now.

[kennethreitz42]

e.g. pipenv lock, pipenv sync

[techalchemy]

Yeah that was my thinking. And exit with an error code so it still technically fails

[rgant]

Best I can tell the purpose of sync isn't even documented. I have no idea why I would want to sync. I don't know what it does. I want to install and upgrade my dependencies. Seems like that should be a first order use case to cover somehow.

@kennethreitz

update makes no sense when we have sync.

What do you mean by that? How are they equivalent or related?

[kennethreitz42]

sync replaced update.

[kennethreitz42]

@techalchemy should we add 'update' which runs lock then sync?

[rgant]

Replaced it how?

Ok, obviously this is in flux and you guys are busy sorting it out.

I don't think I'm ready for pipenv 10.1, I'll roll back and wait for there to be some specific documentation for how I figure out if there are updates I can apply and how to do that.

[techalchemy]

I am becoming persuaded of this use case

[kennethreitz42]

this is why we have version numbers :)

[techalchemy]

That might be more intuitive... if I was coming at this as a brand new user who had no background in pip tools or anything I would probably just try to run pipenv update

[kennethreitz42]

https://github.com/pypa/pipenv/commit/1d021cd3ffabb718c3ea43dc5b64e3567fec2e8c

[kakulukia]

Oh, up until now i thought that lock is locking my installed modules at their installed versions and writes the lock file, so i can keep those .. obviously i totally misunderstood that feature. Am i alone with that assumption?

[greysteil]

Might make sense to stop displaying lock as a user option in the docs / help output - feels more useful (and designed for) automated dependency update tools?

[kennethreitz42]

Okay so in master, update is restored.

New workflow

[kennethreitz42]

Providing a package name to $ pipenv update (e.g. $ pipenv update requests) actually runs:

$ pipenv install --selective-upgrade requests.

[kennethreitz42]

Released as v0.10.1.

[kennethreitz42]

@jacobian that should help with your workflow docs.

[kennethreitz42]

Recommended Workflow

• Find out what's changed upstream: $ pipenv update --outdated
• Upgrade packages, two options:
  a. Want to upgrade everything? Just do $ pipenv update.
  b. Want to upgrade packages one-at-a-time? $ pipenv update <pkg> for each outdated package.

Run your tests, fix issues. If there are problems, pin versions in Pipfile.

[kennethreitz42]

added this to the docs :)

[vovkd]

What if i need install exactly locked dependecies, without updating Pipfile.lock

How to disable this behavior? Because i dont want automatically update all dependencies. The same is with installing with pipenv new package pipenv inistall some_package -- it will update all packages listed in Pipfile.lock. Do i need to specify package versions in Pipfile to do this?

[rhymes]

@vovkd try with pipenv install --keep-outdated

[vovkd]

@rhymes For now the only solution i've found is to explicitly specify package version when installing it. Than Pipfile will contain dependecies with specified versions so when installing some new packages or doing a clean install of all packages listed in Pipfile pipenv will install right versions, not most recent package versions.

[rhymes]

@vovkd yeah, my option didn't work. I like pipenv but I'm a little disappointed about this behavior. I'm sure it's very complicated to handle dependecies in general, hopefully in the future this will get better.