Pip: What flag will users use to call the new resolver?

[ei8fdb]

What is the user story

As a pip user I need a way to use the optional new dependency resolver so that I can install my pip packages.

Additional context
Right now, in order to use the new alpha dependency resolver a pip user needs to use the flag:

--unstable-feature=resolver

to call the new resolver, instead of calling the old resolver. The old resolver will be maintained for a period of time, but ultimately it will go away.

How will the pip resolver behaviour change over time

1. the default will initially be the old resolver.
2. then it will change to be the new resolver
3. finally the old resolver will go away
4. the new resolver will be the only option

What we need to find out?

1. Can the current flag continue to be used until the old resolver goes away?
2. If the answer to 1 is no, what should the new flag be?

How can we do this out?
Research:

• how other pip commands have been rolled out
• how other CLI tools roll-out new internal functionality that replaces old
• what user research is needed?
• what design work is needed?

What we need to do this design or research

• talk with maintainers
• talk with pip users

Are there possible solutions?
The goal is we want to choose an "answer that won't infuriate our users because they keep having to change scripts, or because the approach we choose otherwise imposes on their workflow in unnecessary ways."

Best practices would be to maintain consistency in the pip CLI. Must discuss more.

[brainwane]

We're discussing this in Zulip as well.

[brainwane]

TL;DR: we're going to use a new flag: --resolver=. We're figuring out now what will go on the other side of that equal sign.

I appreciate the work @ei8fdb has done to lay out this question and problem, and the work he and @pradyunsg and @pfmoore have done in Zulip to figure out what we will do.

Paul suggested:

1. We call the new flag --resolver=backtracking. The old resolver is --resolver=legacy.
   
   
   
   1. We introduce --resolver as part of the "main rollout", which we do in June. The default will be legacy at that point.
   
   
   2. We immediately remove --unstable-feature=resolver at that point, with no deprecation, as no-one should be using it for anything except manual testing anyway. We leave the machinery for --unstable-feature in place, but we drop the resolver option (so leaving us as having no actual unstable features).
   
   
   3. At some to-be-decided point, we switch the default for --resolver from legacy to backtracking. We warn for a period of one release before making that change.
   
   
   4. At a second to-be-decided point, we remove the legacy resolver code and the legacy option for --resolver. We again warn for one release before doing this.
   
   

There's quite a lot of work to implement this (test suite changes to handle the new flags, the actual flag code, etc). So we can't leave it too long to get this agreed.

I approve that general approach and in particular I approve the --resolver= flag, and the approach of replacing --unstable-feature= with --resolver= WITHOUT a one-release deprecation period.

So we can get started on that right away.

But I do want to take another day to refine the words we use on the right-hand side of that equal sign, the labels for different resolvers.

@ei8fdb refined this by suggesting, in the interest of using plain language that users are more likely to be familiar with:

Instead of the new resolver being -resolver=backtracking make it -resolver=beta
Instead of the old resolver being --resolver=legacy make it --resolver=old

@pradyunsg noted:

IMO, if a user is using the beta of the new implementation, and they upgrade to a newer version of pip where it's considered stable (or the default), they shouldn't need to change their CI. In other words, we shouldn't create extra work for early adopters once the feature goes stable. I'm not sure how that would work with an =beta approach.

I understand what Pradyun is saying. It's a good reason not to use the word "beta" here. But "backtracking" is a word that does not carry useful meaning for most people who use pip. So I'm thinking about a replacement word. Something plain-language that describes an important attribute of the resolver without only talking about newness.

I will sleep on this and have more suggestions in the morning. I am open to suggestions, but I want to avoid people talking back-and-forth and bikeshedding each other's suggestions; in the next 24 hours, if you have a suggestion, you can make it in this thread, but and you can make one reply to someone else's suggestion, but try not to reply again after that.

[pradyunsg]

"v2", "2020-rewrite" are two ideas.

I know I've brought up "new" in the past, but it'd likely be better to avoid that, so that we don't end up with a new new resolver at some point in the future. :)

[sbidoul]

legacy|backtracking is fine with me. Have you considered legacy|resolvelib?

[pfmoore]

Have you considered legacy|resolvelib?

I considered suggesting that, but there was talk about replacing resolvelib with PubGrub at some point, and I wasn't sure if that would need a new option, or would be considered "transparent". (And PubGrub is an even worse term, in terms of being obscure jargon...)

[nlhkabu]

I'm not sure that we're not just creating a whole lot more work for ourselves here?

My 2c:

• I think we should keep using --unstable-feature=resolver until the resolver is ready for prod.
• When it is ready, we should make the new resolver the default, and make the old resolver available under --deprecated-feature=resolver
• I am strongly against referencing the technical implementation in the command. the terms resolvelib and pubgrub mean absolutely nothing to the majority of pip users.

My thinking:

1. We have just done a big program of community outreach asking people to test the new resolver with --unstable-feature=resolver. If we decide to get rid of this command, we will have to do another round of communication. Inevitably, some people will not realise that this command is no longer available and become frustrated.

2. I have absolutely no issue with keeping the word "unstable" in the command. IMO, if it is not ready for use in production, then it is, by definition, unstable. We should not be embarrassed by this, or let it detract from the great work the team is doing.

3. I assume that the --unstable-feature flag was introduced as a process to deal with rolling out any new feature in future. By creating a new --resolver flag, we are undermining that process.

4. By introducing a --deprecated-feature flag we can create a universal process for deprecating features, that can be used again in future

Pros of this plan:

• The clearest path for pip users, given our previous community outreach
• The least amount of work for the team
• Reusable for future features

Cons:

• Not specific to the resolver
• Does not allow us to communicate different levels of readiness: the resolver is either unstable or ready for prod, there is no in-between.

[ei8fdb]

Does not allow us to communicate different levels of readiness: the resolver is either unstable or ready for prod, there is no in-between.

To communicate these different levels of readiness, I've seen the following communications working well:

• clearly written release-note
• public communications (in this case to lists/twitter/podcasts)

If a user never sees these communications - when a user updates pip, they get automatically the newest version of the feature.

If they don't upgrade - there is no difference anyway, they're still on the same version.

(For future features this strategy could also be used for communication of alpha/beta releases)

[uranusjr]

"v2", "2020-rewrite" are two ideas.

--resolver=2020 seems to read OK as well.

[brainwane]

I thought I could make a decision to make things easier for everyone, but @nlhkabu has persuaded me that I was wrong.

My current thinking is that we should go with the plan @nlhkabu has laid out, using

--unstable-feature=resolver

and then

--deprecated-feature=resolver

and I think we should make sure to get a final decision on this in tomorrow's meeting.

Heads-up to @dstufft @xavfernandez @cjerdonek @chrahunt and @dholth -- I'd appreciate if you could look at this discussion and share your opinion. However, we have a team meeting scheduled for about 14.5 hours from now, 9am-10am Eastern time, and I'd like to have this question decided by the end of that meeting so that everyone can implement it. So if you want to join in that meeting please message me and I'll get you the info to join it.

[uranusjr]

What would the transition look like when we flip the switch and declare the new resolver “production ready”? Does --unstable-feature=resolver immediately goes away, or would there be a transition period where both --unstable-feature=resolver and --deprecated-feature=resolver are available? (During which the former flag would do nothing, and print a warning saying this flag is going away.)

I’m thinking in the shoes of a pip user who need to switch between multiple Python environments (and therefore multiple pip installations, potentially of different versions), and is already using the new resolver before it’s “production ready”. It would be very frustrating if they need to remember which version of pip they are on when they do pip install. Otherwise they may either supply --unstable-feature=resolver and get an error (because that flag went away), or errornously use the legacy resolver because they thought their pip is new enough when it’s not.

[xavfernandez]

Like Tzu-ping, I'm wondering what --unstable-feature=resolver --deprecated-feature=resolver would actually mean and how the different pip versions will/should behave (print a warning ? error out ?)

[ei8fdb]

Like Tzu-ping, I'm wondering what --unstable-feature=resolver --deprecated-feature=resolver would actually mean and how the different pip versions will/should behave (print a warning ? error out ?)

Can you explain "would actually mean and how the different pip versions will/should behave" a bit more? I don't understand.

[brainwane]

Here is my attempt at elaborating on that question:

what --unstable-feature=resolver --deprecated-feature=resolver would actually mean

What behavior would we implement for the --unstable-feature=resolver flag? When a user uses it, what should they expect? When we write documentation and help text for it, what (roughly) would go in that?

What behavior would we implement for the --deprecated-feature=resolver flag? When a user uses it, what should they expect? When we write docs and help text for it, what (roughly) would go in that?

how the different pip versions will/should behave (print a warning ? error out ?)

We're about to release a beta, then 20.2, then (later) 20.3, and so on. For each of the versions of pip that we release, if someone uses one of those flags, for each flag, when should pip allow usage but print a warning (such as "this feature is deprecated and will be removed in the next release")? When should pip "error out," which means giving an error message saying "This feature was removed"? When should pip simply allow usage without printing any particular warning or error?

I think https://github.com/pypa/pip/issues/8371#issuecomment-637425353 addresses some of these questions.

[georgiamoon]

I see some discussion in this thread around a version name, what if we leaned into that idea?

[pfmoore]

Also, with this approach we'd need to have a check that people don't say --deprecated-feature=resolver and --unstable-feature=resolver in the same invocation of pip (and yet we also need to leave the option open that we could have another unstable feature on the go, so --unstable-feature=foo --deprecated-feature=resolver is valid).

The problem with over-general options like --unstable-feature is that they can result in lots more combinations of cases to consider than we'd get with single-purpose options.

[sbidoul]

I realize we have at least 3 features that we want to deprecate and where we need to give user some control during a transition period. It might be interesting to have a uniform mechanism to cover these with a single feature flag mechanism, to avoid the proliferation of specific options with a short lifetime. These features are

• the legacy/new resolver
• out-of-tree builds that we want to deprecate in favor of in place build or build via sdist (#7555)
• install via setup.py install that we want to deprecate in favor of building a wheel and installing from it (#8102 and specifically https://github.com/pypa/pip/issues/8102#issuecomment-640214691)

[sbidoul]

So in the spirit of generic feature flags, this could look like this:

• --disable-feature=legacy_resolver (same effect as --unstable-feature=resolver with a warning that the new resolver is not production ready),
• --enable-feature=legacy_resolver (in a transition period when the new resolver is the default, with a warning that the legacy resolver is deprecated)
• --disable-feature=setup_py_install (or --enable-feature=always_install_via_wheel?)
• --disable-feature=out_of_tree_builds

Using both enable and disable for the same feature is an error.

When a feature is completely removed, and the feature is enabled/disabled by the user, a warning says the flag has no effect.

[brainwane]

We need to finalize this decision in the next few days. There's some discussion in Zulip as well.

[brainwane]

@sbidoul @xavfernandez @ei8fdb @dstufft @cjerdonek @chrahunt please take a look at this (complicated) issue as well as the discussion on Zulip. Resolving this question is a major blocker to finalizing documentation (announcements, a testing guide, etc.) and releasing a beta.

[nlhkabu]

Updating this thread to reflect the latest discussion on Zulip in anticipation for our team meeting this afternoon.

Current consensus:

1. We will use a 2 flag system:
   
   
   
   • 1 to access a new/in development feature
   
   
   • 1 to access a feature that will soon be disabled
   
   
2. The name of the flags are still to be agreed
3. We will remove the --unstable-feature flag

We also need to make a decision on what we will call the new and old resolvers, because _resolver_ does not adequately describe the difference between the two.

[nlhkabu]

As per our Zulip chat, we're going to go with --use-feature and --use-deprecated.

For the new resolver, we'll use --use-feature=2020-resolver and --use-deprecated=legacy-resolver. The rollout will be:

Step 1:

• pip install X - uses old resolver. Raises warning for cases where the new resolver might behave differently
• pip install X --use-feature=2020-resolver - turns on new resolver
• pip install X --unstable-feature=resolver - errors. Users instructed to use pip install X --use-feature=2020-resolver instead.

Step 2:

• pip install X --use-deprecated=legacy-resolver - uses old resolver. Raises warning that old resolver will be deprecated on <date>
• pip install X - uses new resolver
• pip install X --use-feature=2020-resolver - works but warns users that flag is no longer necessary

Step 3:

• pip install X --use-deprecated=legacy-resolver - errors
• pip install X - uses new resolver
• pip install X --use-feature=2020-resolver - works but warns users that flag is no longer necessary. Can deprecate in future if we decide to develop an even newer resolver.

Next steps:

1. @nlhkabu to make a documentation PR to document this process (next to deprecation policy)
2. @pradyunsg to action Step 1

[pradyunsg]

• Raises warning for cases where the new resolver might behave differently

This becomes a blocker task for the July release then. I'll file an issue -- shouldn't be too much work. :)

[nlhkabu]

Closing in favour of https://github.com/pypa/pip/issues/8512 and https://github.com/pypa/pip/issues/8513. Please reopen if needed.