Pip: Oh look! It's April again! (Release 20.1)

[pradyunsg]

As per our Release Cadence, it's release month!

Looking at the changes since 20.0.2, we do have quite a few changes and ~350 commits at this time. That seems to mean that there's more than enough work on master to justify a release. :)

---

We're in our beta period! We welcome users to provide us with feedback on the beta release. pip's beta release can be installed by running:

python -m pip install --upgrade --pre pip

[pradyunsg]

Following on from https://github.com/pypa/pip/issues/7531#issuecomment-569881977, I'm happy to be the release manager for this release as well.

I promise to clean my build directory this time -- there's automation that will force me to anyway. :)

[atugushev]

Hello @pradyunsg! Any ETA?

[brainwane]

@atugushev There are some discussions maintainers probably need to have over the next few days -- I think we'll probably have an estimated date after that, but I would be surprised if we could get the release out in the first half of the month. Are you waiting for anything in particular?

[pradyunsg]

In terms of a timeline for this release, as of right now, I'm thinking of making a beta release on Monday, 20th April. This would give us the opportunity to iron out any kinks in our beta process prior to needing to use it for the resolver rollout in our next release cycle.

I'm still undecided on the length of the beta, but it'll definitely be at least 7 days.

@pypa/pip-committers Feel free to keep merging PRs as usual till April 13th (coming Monday). After April 13th, please let me know about PRs you'd like to merge before merging PRs. :)

[deveshks]

Hi @pradyunsg

Is the directive mentioned in the comment above, also applicable for folks who don't have explicit merge access as well?

[pradyunsg]

No. After April 13th, I'd like to be in-the-loop of, and significantly reduce the number of, PRs that myself and other committers merge until after the release is done.

[deveshks]

Thanks @pradyunsg for the response.

So If I understand correctly, April 13-20th is pre-beta, April 20th is the beta, and depending on the length of beta, which is atleast a week as per one your comments above, we will have a release date towards the end of April, and the above stated policy will be applicable till then.

[pfmoore]

@pradyunsg All of the new resolver work so far has gone in with "trivial" labels, as we intended to add a changelog entry covering the whole thing. Should we add that changelog for the beta?

[pradyunsg]

Should we add that changelog for the beta?

Yes! #8041.

[pradyunsg]

@pypa/pip-committers After EoD April 13th (basically after today) for you, if you plan on merging any other PRs, please add them to the 20.1 milestone and let me know before you do the merge. Thanks!

[pradyunsg]

Update: Right now, we're on track for a pip 20.1 beta 1 on April 20th.

---

Nearly all the relevant issues for pip 20.1 have either been merged or deferred to later releases. The only remaining issues are:

• NEWS entry for pip's next-gen resolver's alpha that's going out with this release.
  
  
  
  • needs to be written by some subset of {@brainwane, @pfmoore, @uranusjr, @pradyunsg}.
  
  
• Updating vendored libraries.
  
  
  
  • awaiting reviews from humans that are not-me (#8056)
  
  
• (non-blocker) Changing behavior around installed packages, in pip's next-gen resolver
  
  
  
  • basically ready to merge (#8014)
  
  

[brainwane]

So, the release notes will be a short human-readable summary of what's new in pip 20.1, going out to distutils-sig and similar mailing lists. It will link to the full CHANGELOG (each individual bugfix and feature NEWS entry included in the release) and to a wiki page about "here's what's up with the new resolver and how to test it". The CHANGELOG we will generate automatically. The wiki page, I am working on.

For the release notes: I would love help gathering the list of major new features and bugfixes. I know of:

• A new pip cache command
• in-place builds
• multi-threading in pip list --outdated
• An alpha version of pip's next generation resolver

What else should we include?

[uranusjr]

Not sure if it is universally counted as major, but I’m particularly excited that pip freeze can now correctly output the source direct URL the user installed from, by implementing PEP 610. More description is available in the pull request #7612.

[pradyunsg]

@brainwane I looked at our "CHANGELOG" for this release, and I don't think there's any other major changes (other than @uranusjr's suggestion).

---

I just did a dry run of our release process locally and hit #8091. I also got a chance to look at the resulting NEWS file, and based on that I've filed a few PRs now (#8089, #8093, #8088, #8092) related to our current NEWS entries.

I'd appreciate reviews on #8093, which moves the bulk of the NEWS entry for "in-place builds" into the relevant documentation section.

Looks like the last thing we're blocked on, are the finer details of communication around the new resolver. :)

[McSinyx]

multi-threading in pip list --outdated

I've was playing around with pip list yesterday and forgot to note it here, GH-7962 effectively added multi-threading to pip list --uptodate too.

[pradyunsg]

I just spotted that we never removed --skip-requirements-regex despite it being scheduled for this release cycle. :)

Filing a PR for that now -- #8098.

[sudopriestmx]

Hi! I noticed the issue opened where the new resolver alpha release is mentioned and that you are looking for windows testing.
Do you have a recommended workflow to cover most of the scenarios you are looking to test? For example, a project that has a lot of dependencies that I can test install and see what happens?

[pradyunsg]

With #8088 and #8099 in place, we're all set for this beta!

Started release engineering in #8100.

[brainwane]

@sudopriestmx thanks for the note! working on it. Please watch #8099 for edits as we improve those instructions.

[pradyunsg]

Made pip 20.1b1 release!

• Merged #8100.
• Had to update our release automation, because we introduced a regression in #7654, that I hadn't spotted in my test run earlier yesterday/today. :(
• Built distributions.
• Pushed 20.1b1 tag to GitHub.
• Uploaded built distributions to PyPI.

New thing: get-pip.py has not been updated, since we don't want people getting this pip beta right now.

[pradyunsg]

Release announcement emails + posts are sent:

• distutils-sig: https://mail.python.org/archives/list/[email protected]/thread/5OEXP4XQQHFLA67Z6TXFZRKABV5TRNPL/ (RIP GMail's formatting)
• https://discuss.python.org/t/announcement-pip-20-1b1-beta-release/3960

Now, to file a PR fixing that release automation regression. :)

[brainwane]

I've now further publicized the beta in several venues:

• #988
• pypa-dev
• @ThePyPA on Twitter (retweeted by ThePSF)
• chat.zulip.org
• python-list
• https://github.com/napari/napari/issues/631
• devpi's list
• the Carpentries discuss list
• wheel-builders
• #python IRC channel

Some other venues I am aware of, but am deliberately not publicizing the beta in yet. In particular:

• python-announce: am waiting another day before announcing there, to check for any big bugs that the existing beta testers will find.
• blog.python.org: similar situation to python-announce.
• python-dev mailing list: similar situation to python-announce.
• pypi-announce: I will announce the 20.1 release there, but we have promised that this will be a very low-traffic list, so I'm not announcing the beta there.
• Language-specific communities (such as Portugese- and Spanish-speaking contacts) and general groups of volunteers who are not necessarily particularly skilled at using Python (such as the psf-volunteers list): will reach out to them when the beta of the new resolver is ready for more robust testing (probably next month), and not now.

Pycoder's Weekly has already mentioned the new beta and I'm about to ping a few other newsletters:

• Import Python
• Python Weekly

And I've reached out to a few podcasts about our work in general, and if they get back to us super fast then they might cover the beta (but later coverage will still be useful):

• Podcast.__init__
• Test & Code
• Talk Python To Me
• The Changelog
• The Rabbit Hole

When I make a list like this, a frequent reply is "you should also publicize it in [place]". I shall preemptively reply: You are also free to do this, and I hope you will. If you do, please mention it here so we can add it to our future "places to notify" lists.

[pradyunsg]

Thanks @brainwane! ^>^

@pypa/pip-committers Let's avoid merging PRs into master until the main release is out. I think merging documentation PRs is fine but let's avoid code changes that aren't bug fixes for changes released as part of the beta.

In terms of a timeline for the main release, I'm aiming for Tuesday (28th April) -- if we've handled all our bug reports/bug fixes, I'll cut the release on that day. It's possible that we defer the release if there's a good reason to.

[brainwane]

I have now:

• announced to python-dev
• announced to python-announce
• notified Piwheels https://github.com/piwheels/piwheels/issues/213
• asked @ewdurbin to copy the above announcement to blog.python.org

[brainwane]

@yuvalreches and @ori-yitzhaki of JFrog Artifactory: Here's the announcement of the beta release of a new version of pip, pip 20.1b1. Could you try testing it out with JFrog Artifactory to check whether stuff breaks? If so, please fix any bugs on your end, and then please let us know of any bugs you have found in pip. We're hoping to release pip 20.1.0 around Tuesday 28 April. Thanks!

[pradyunsg]

Alrighty. We've had folks report regressions, and also point out that a bunch of stuff in the beta is an improvement. :)

I think this was a fairly decent beta. Tomorrow is release day, and the milestone is clear. @pypa/pip-committers if any release blockers are reported, please file an issue, add it to the milestone and label it as a release blocker.

[pradyunsg]

8157 is a blocker, that we realized/figured out about just now. I'll merge it once CI gives me an OK, and then start the release engineering. :)

[pradyunsg]

Made the release! pip 20.1 is out in the wild now.

• Uploaded to PyPI.
• Tag pushed to GitHub.
• Updated get-pip.py.

[sudopriestmx]

Is it normal that when I click the release notes link on both Github and PyPI it takes me to the 20.0.2 release notes? That’s what stable news is according to this link

[pradyunsg]

@sudopriestmx That's a caching issue I think. I just did a curl -X PURGE to force the documentation and PyPI to update. It looks fine to me now. Could you confirm?

[sudopriestmx]

@pradyunsg works perfectly now 👌 thanks for the hard work from the pip team for this release 🙌

[brainwane]

Thank you for publishing the release, @pradyunsg!

Now announced:

• on Discourse
• on distutils-sig
• on blog.python.org
• on pypi-announce
• on NumFOCUS mailing list
• [feel free to edit this comment to add other places we have announced it, other repo maintainers/triagers]

For longer-term thoughts on how we announce beta releases and how useful they are to gather bug reports, please go to #7628.

[pradyunsg]

@brainwane I've reopened this issue, since we don't close the "Release YY.M" issues until after any immediate bugfix releases (like #7531, https://github.com/pypa/pip/issues/6993 etc).

I'd like to keep that for this release as well, until we're sure that there's no significant regressions introduced. :)

[pradyunsg]

@pypa/pip-committers master is open for regular development now -- if we need to make bugfix releases, I'll do the "proper" bugfix procedure, and cherry-pick the relevant commits to make a release using those.

[pradyunsg]

Looking forward at pip 20.1.1, we have a couple of changes we might want to revert. We did not get significant user feedback on them during the beta (😞) -- a lot of users (and experts) took notice and flagged issues after the main release was made containing those changes.

For now, there's still a bunch of discussion happening in #7555, about how we want to deal with in-place builds and there's agreement that we'd want to remove the parallel code introduced.

[pradyunsg]

Okay! Looks like we're all set for a 20.1.1. I'll cherry pick the changes, and file the corresponding release PR for it.

I'll try to get it out in the next 24 hours, or on Monday (to avoid a Friday/weekend release).

[pradyunsg]

I was down with a pretty bad headache yesterday, so, the release PR (#8264) was filed just now. Expect a release in the coming hours. :)

[pradyunsg]

Made the release! pip 20.1.1 has been published!

• Uploaded to PyPI.
• Tag pushed to GitHub.
• Updated get-pip.py.

[pradyunsg]

Looks like there's no more new bug reports for that are related to changes introduced in pip 20.1.1 now.

With that, I'm gonna call it a wrap on pip 20.1 release cycle. Onward to pip 20.2! Thanks everyone who's contributed to this release! ^>^