Core-workflow: Delete old branches?

[zware]

We currently have useless branches for 2.0, 2.1, 2.2, 2.3, 2.4, 2.5, 2.6, 3.0, 3.1, and 3.2. I suggest tagging the HEAD of each branch (x.x-EOL, perhaps?) and then deleting the branch. Having those branches makes the branch list unwieldy in the GitHub UI, and also (but less so) in command line usage. Keeping those branches might also make unfamiliar contributors think that they are valid targets for changes.

[warsaw]

I agree that it may not make sense to carry branches for EOL'd versions. Why not use a tag namespace such as CLOSED/2.1 or EOL/2.1?

[brettcannon]

I'm +1 on their removal and +1 on the idea of namespacing the tags (I like EOL/2.1).

[ned-deily]

Are those branches all really useless? In the hg world, the changesets for maintenance releases are only in the branches, like say the tag v3.2.4.

[brettcannon]

Yep, they are really useless (unless my understanding of git is wrong which is entirely possible 😁). Since git views the whole world -- including branches and tags -- as just bookmarks on a single, massive DAG, there is no special connection to any one branch. E.g. the v3.2.4 tag stands on its own in the DAG.

And Ned inadvertently brings up a good point. 😉 Do we really need an EOL-specific tag? Can't we simply tag the end of the branches as the final release version for that branch? E.g. couldn't we just tag the 2.1 branch as v2.1.3 and not have to explicitly call out it's final state as EOL in the tag itself?

[zware]

The "EOL" is nicely explicit. I've done a trial run on my fork. Note that it looks like we have more tags than GitHub wants to handle in its UI, but the tags are there.

[orsenthil]

+1 to the deleting.

And yeah, I think, just having the branch as the final release tag should be good enough. Agree with Brett.

@zware , how does being explicit with EOL- helps us ? The useful information is the lineage of releases. which are captured in 2.1.0, 2.1.1 and ends finally ending with a number, say 2.1.3.

[vadmium]

Strong +1 for deleting the legacy-trunk branch. I understand this was the label for commits imported into Mercurial from the Subversion "trunk" branch. This branch continued as the 2.7 maintenance branch. It does not contain anything substantial compared to the 2.7 branch, only Mercurial metadata changes that IMO are unnecessary for Git: https://github.com/python/cpython/compare/2.7...legacy-trunk.

A disadvantage of renaming or removing the other branches is it makes it harder to search through repository history. You would have to look up what the last release number was, or remember whether to add the EOL annotation or not.

Comparisons of the other branches mentioned:

• https://github.com/python/cpython/compare/v2.0.1...2.0 (Only Mercurial metadata)
• https://github.com/python/cpython/compare/v2.1.3...2.1 (Total 70 unreleased commits)
• https://github.com/python/cpython/compare/v2.2.3...2.2 (77 commits)
• https://github.com/python/cpython/compare/v2.3.7...2.3 (Only metadata)
• https://github.com/python/cpython/compare/v2.4.6...2.4 (Couple of commits that were reverted)
• https://github.com/python/cpython/compare/v2.5.6...2.5 (Only metadata)
• https://github.com/python/cpython/compare/v2.6.9...2.6 (Only metadata)
• https://github.com/python/cpython/compare/v3.0.1...3.0 (400 commits)
• https://github.com/python/cpython/compare/v3.1.5...3.1 (10000?! Something funny going on.)
• https://github.com/python/cpython/compare/v3.2.6...3.2 (20 commits)

It is not seem clear if 3.2 is finished or not; PEP 392 says 3.2.7 was scheduled in Feb 2016. Also see https://mail.python.org/pipermail/python-dev/2016-June/145161.html. On the other hand: https://github.com/python/devguide/pull/33; people documented it as EOL with the argument it was still easy to get at the branch.

[zware]

@vadmium pretty much answered for me why the EOL tags are useful, but I'll reiterate: they show where the branch actually ended, which may or may not have been the last official release (case in point: 2.1). Or if we don't care about the history of the branch after the last official release, I'm fine with not bothering with the EOL tags, but I'd still like the old branches to go away :).

[ezio-melotti]

FWIW bpo might generate links to older branches (e.g. https://github.com/python/cpython/blob/2.4/LICENSE#L57) and if the branch is deleted, I assume this won't work anymore.
I can make bpo point to master, but the line numbers will be off if the code changed in the meanwhile.

[larryhastings]

Git metadata is so malleable, if we delete the branches, then later decide we want them for some reason, we can just recreate them, right?  And it'll be indistinguishable from having never deleted them in the first place.  So I say go ahead and delete them.

[ncoghlan]

If deleting or renaming the old branches is the only way to hide them from the PR creation dropdown, then +0 for deleting them and +1 for replacing them with an EOL/X.Y branch tag that sorts after the active numeric branches.

One perk of the latter approach is that it provides a nice programmatic way for release managers to say "That's it, we're done with this branch" without people subsequently asking "Hey, what happened to the X.Y branch?"

[brettcannon]

It sounds like people in general are okay with dropping the branches, although it will break some links out there (includings bugs.python.org). People also seem to like the EOL tags. Am I missing anything?

[dstufft]

Git metadata is so malleable, if we delete the branches, then later decide we want them for some reason, we can just recreate them, right? And it'll be indistinguishable from having never deleted them in the first place. So I say go ahead and delete them.

This is correct as long as we record somewhere the revision they were pointing to (like via a tag). The only gotcha here is if there are any commits that no branch or tag are pointing to, then they will eventually get garbage collected. So if we want to make sure we can restore them in the future, we should tag them.

[zware]

In the interest of keeping the bpo links working, perhaps we should replace the old branches with tags of the same name. This doesn't preclude also tagging the same commit as EOL/2.0, etc.

[ncoghlan]

I like the idea of branch->tag conversions, since the two aspects I'm interested in are:

• make the active branch UI easier to use
• don't lose historical metadata

If the branch "end-of-life" policy is "Note the X.Y branch commit hash, delete the X.Y branch, then tag the commit as X.Y and EOL/X.Y", then we keep a clear list of active branches, and also keep a clear record of where the old branches ended.

[larryhastings]

Zachary's idea of creating tags with the branch names seems like a
winner to me. This will also keep the revisions alive, so they won't
get garbage collected as dstufft points out. Seems like we have a
winner to me!

I don't feel a strong urge to have the "EOL-" tags, but I
think they're harmless, so I wouldn't fight about 'em either way.

[methane]

+1 to Zachary's idea.

On Tue, Feb 28, 2017 at 2:37 PM, larryhastings [e-mail]
wrote:

>

Zachary's idea of creating tags with the branch names seems like a
winner to me. This will also keep the revisions alive, so they won't
get garbage collected as dstufft points out. Seems like we have a
winner to me!

I don't feel a strong urge to have the "EOL-" tags, but I
think they're harmless, so I wouldn't fight about 'em either way.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/python/core-workflow/issues/31#issuecomment-282947368,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAMLqMqhYqwh58FSrnGL7ukK4brdeij6ks5rg7KrgaJpZM4MCJqG
.

[dstufft]

This seems confusing to me. We're going to have a tag '2.0' and a tag 'v2.0.0' or so that mean different things. Seems like a recipe for confusion.

Sent from my iPhone

On Feb 27, 2017, at 9:53 PM, Zachary Ware [e-mail] wrote:

In the interest of keeping the bpo links working, perhaps we should replace the old branches with tags of the same name. This doesn't preclude also tagging the same commit as EOL/2.0, etc.

[larryhastings]

I don't think it will lead to much confusion, simply because hardly
anyone will look at them. Remember, we're talking about 2.6 and
previous, and 3.0 through 3.2 (I think). I don' think they're of much
interest these days.

Let me summarize. Ignoring the "EOL"-prefixed ideas for now, here are
the options for what we could do with the legacy branches.

A. Keep them forever. This clutters up our branch namespace with junk
nobody's using, which in turn makes the GitHub UI a bit clumsy, but is
otherwise harmless.
B. Delete them and replace them with nothing. You can no longer ask git
for "2.2". This breaks legacy links from bpo, etc.
C. Delete them and replace them with tags using the same name. dstufft
suggests this might be confusing to somebody someday.

I prefer C, I could live with A, I think B is a bad idea.

[gpshead]

Has anyone asked Github for the ability to curate which branches show up in the UI?

[ncoghlan]

We haven't, but it's also one where if I was them I'd say "No", with the suggestion "Just convert them to tags instead".

We'd need to write our policy down somewhere (presumably in either the developer guide or the release management PEP) and be explicit that the vX.Y.Z and vX.Y tags refer to the actual releases, while X.Y and EOL/X.Y refer to the end state of closed branches, but I think git already gives us all the tools we need to tidy things up a bit.

[dstufft]

We could also make a second repository, cypthon-archive which holds all of our legacy branches. If someone needs them for some reason they can add it as a git remote and otherwise largely act as if they are one repo.

These branches will get garbage collected from the primary repo, but that is ok because all the commits will exist in the cypthon-archive repository.

Using this technique we could even delete old tags from the primary repository (and make the tag interface more useful for us) and archive the old tags.

The primary downside is it will break links for archived branches and tags.

Sent from my iPhone

On Feb 28, 2017, at 2:50 AM, larryhastings [e-mail] wrote:

Let me summarize. Ignoring the "EOL"-prefixed ideas for now, here are
the options for what we could do with the legacy branches

[dstufft]

Thinking about that more, if we made sure to make the cpython-archive repo a github fork, we would even still retain the ability to do a git diff via the github ui (with an extra two clicks).

[ned-deily]

"This seems confusing to me. We're going to have a tag '2.0' and a tag 'v2.0.0' or so that mean different things. Seems like a recipe for confusion."

That shouldn't be a problem as this is essentially what we have had in hg for years:

hg update -C 3.6   # checkout the head of the 3.6 branch
hg update -C v3.6.0  # checkout the snapshot of the 3.6.0 release

The only difference will be that 3.6 is now a git tag instead of an hg branch.
BTW, I've been persuaded about deleting the branches and going with Larry's option C.

[dstufft]

I don't think hg listed tags and branches together in one namespace though did it? Certainly git doesn't. Yes you can check out either because there is some heuristics to figure out which you wanted in a situation where it is ambiguous (such as checkout), but whenever you're looking at a list of them they are separated.

[ned-deily]

I guess I don't understand what your concern is. We, as a project, already have a convention for the use of the combined namespace of branches and tags: vm.n.o names denotes release snapshots, n.m names denote branches. That's not changing here and we're not going to allow arbitrary branch or tag names in the master cpython repo. It hasn't been a problem up to now and I don't see why it would be a problem going forward.

[ned-deily]

And, just to be clear, our policy has been that the release tags (vm.n.o) are immutable (release managers go to some lengths to not push them until the release is published) and that both tags and new branches can only be pushed to the canonical repo by admin types, e.g. release managers. We need to ensure that remains true.

[ned-deily]

Side note re immutable tags: perhaps release managers should use the --gpg-sign option with their release keys to sign release tags. I haven't tried it yet but I plan to for v3.6.1.

[brettcannon]

So it sounds like everyone now agrees with deleting the branches. So that just leaves agreeing on a tag name. Vote by reacting to the following comments ...

[brettcannon]

2.6

[brettcannon]

EOL/2.6

[dstufft]

So it sounds like everyone now agrees with deleting the branches. So that just leaves agreeing on a tag name. Vote by reacting to the following comments ...

I'd still just prefer moving the old branches (and possibly old tags) to an archive repository and keep the primary repository focused on ongoing development.

[warsaw]

I'm not sure why we need two repositories. Git can handle everything we've proposed in a single repository, so why complicate things? JMHO.

[brettcannon]

@dstufft couldn't we try it this way to start and then move to the archival repo if we chose to? I just want to minimize the amount of work during release time by not having to ask RMs to remember to sync to another repo to store the history.

[dstufft]

I'm not sure why we need two repositories. Git can handle everything we've proposed in a single repository, so why complicate things? JMHO.

Because not all the tooling works great with everything in a single repository. Take a look at the tag switcher in Github. This isn't really a deficiency in Github per say either, we have 309 tags so trying to represent that in a UI is going to be difficult no matter what you do. Limiting ourselves to just 2.7+ and 3.3+ pulls that down to just 110 tags.

[dstufft]

I just want to minimize the amount of work during release time by not having to ask RMs to remember to sync to another repo to store the history.

A separate repository does not need to involve any additional effort beyond the tagging approach. You wouldn't need to even have the current branches exist in the archival repository at all and it could have a simple default branch that just explains this repository is for archival purposes at all. Then the RMs never have to interact with it during releasing, the only time someone needs to interact with it is when we EOL a release series, in which case we push all of the tags for that release series to that repository, the branch for it, and then delete both from the primary repository.

[dstufft]

Forgot to say:

couldn't we try it this way to start and then move to the archival repo if we chose to?

Yes, of course we can. Git is fairly malleable in this way so going in either direction is not a permanent change. I just think that a separate repo solves more problems. The tag switcher is a bigger problem is my use of Github + CPython than the branch dialog is. If the branch dialog is unwieldy with 17 branches, the tag dialog is even worse at 309 (which is so many that the dialog is actually broken, rather than just being unwieldy).

[ned-deily]

OTOH, how often are people going to use tags from the GitHub GUI? They will be immutable and the branches they refer to will be read-only so there shouldn't be any PR's submitted against them. In other words, I don't think it's a big deal if the GitHub is suboptimal for tags in our use case.

[dstufft]

I don't know about other people, but I use the tag dropdown way more than I use the branch dropdown.

[ned-deily]

With cpython? Why?

[dstufft]

A few scenarios:

I find a bug in some software I'm writing that started occurring around CPython v3.6.1, but didn't occur in v3.6.0. I go to Github, and use the tag dropdown and the compare screen to generate a diff between the two versions.

I find a bug in some software that I've traced back to stemming from inside of some code in CPython. I know that I have CPython v3.5.2 so I pull that project up on Github, and hit the tag dropdown to load that specific version so I can locate the exact line that is being indicated in my traceback and look at the code around it.

I think the only time I ever use the branch dropdown is when making a backported PR, and if we get a bot that does those for us, then I can't think of a time I'd actually use the branch dropdown at all except maybe to check what version of pip is in the next to be released version of some branch.

[dstufft]

I'd guess maybe a big difference here is for me, Github (or whatever tool that project hosts their site on) is my primary tool for reading code other than the project I am _currently_ working on. Since I'm not working directly on that project while I'm using it in that capacity, unreleased changes are rarely interesting to me, but pulling up the code as it was released for a specific version is very interesting.

[ncoghlan]

Big -1 from me for multiple repos, as it breaks links, and creates more work for release managers.

Regardless of what happens with the GitHub UI dropdowns (which seem to only show the most recent N tags regardless of how many there are in the repo), it's also always relatively straightforward to jump to a particular tag of interest by clicking through to a particular source URL on master (or one of the other main branches), and then editing the URL bar to include the tag name at the appropriate spot:

• https://github.com/python/cpython/tree/master/Doc
• https://github.com/python/cpython/tree/v2.6.5/Doc

A separate repo also provides a worse initial UX as people go "I can't find the v2.6.5 tag, what happened to it?" and would have to be told "Oh, the EOL tags are only in the cpython-archive repo".

[dstufft]

Big -1 from me for multiple repos, as it breaks links, and creates more work for release managers.

How does it create more work for release managers and how does it break links anymore than renaming the reference to EOL/X.Y?

[ncoghlan]

We're not renaming the reference: we're either using the X.Y tag, or we're using both X.Y and EOL/X.Y. The "EOL/X.Y only" option dropped out of the running a while back precisely because it breaks branch-specific links (like those in the online documentation).

The extra work for release managers is the fact that instead of doing all their work on the one repo, they now need to keep track of "Do steps A, B, and C on the cpython repo, and steps X, Y, and Z on the cpython-archive repo". That extra work needs a pay-off to justify it, and I'm not seeing one.

[dstufft]

We're not renaming the reference: we're either using the X.Y tag, or we're using both X.Y and EOL/X.Y. The "EOL/X.Y only" option dropped out of the running a while back precisely because it breaks branch-specific links (like those in the online documentation).

I don't think that's true? At least you seem to be the only person who has mentioned having both of them, and @brettcannon's voting choices were "2.6 or EOL/2.6, not 2.6 or EOL/2.6 and 2.6. Of course I could have misread Brett's message but I don't think so.

The extra work for release managers is the fact that instead of doing all their work on the one repo, they now need to keep track of "Do steps A, B, and C on the cpython repo, and steps X, Y, and Z on the cpython-archive repo". That extra work needs a pay-off to justify it, and I'm not seeing one.

Uh, you literally only have to interact with the other repo once, when "closing" down an old release series. At that point you just push the old branch + tags to that remote before deleting them from the origin remote. Neither approach requires any effort from a RM during any release, only when a branch gets closed down.

[ncoghlan]

I voted for both of Brett's tag options, because I think we should have both, and it doesn't really cost anything to have both.

And since RM's are the ones that declare their branches EOL and manage any associated bookkeeping, we're agreeing that multiple archives will make those steps more complicated.

It also makes future historical analysis of CPython development more complicated, since changes that only applied to EOL maintenance branches may have been garbage collected from the main repository, while the archive repository may be behind the latest development commits. If there's only one repository, then we don't have any problems with historical commits being garbage collected, and nor do we have any potential problems with keeping the archive up to date.

git is good at preserving history and making it readily available for anyone that wants to look at it. Making that history harder for people to find doesn't make any sense.

[dstufft]

It takes two commands to have all of the history available to you in a single repository. Multiple repositories is not meaningfully harder or more complicated to deal with.

[ncoghlan]

For you, maybe. We have experienced core contributors that are currently unable to push their own changes because they don't yet understand how to use the GitHub PR workflow.

[dstufft]

$ git remote add archive [email protected]:python/cpython-archive.git
$ git fetch archive

Now you have all of the branches and tags available locally again.

[brettcannon]

I did propose the tag options as "one or the other" because that was my thinking, but if I misread this thread and people want 2.6 no matter what then I don't really see a point in having two tags.

[ncoghlan]

My rationale for proposing having both tags was that the EOL/X.Y variant makes it straightforward to programmatically check whether a branch is still active or not, but I just realised that we get that anyway just by virtue of the label being a tag rather than a branch, so we'd have:

git branch --list ?.? # All active branches
git tag --list ?.? # All EOL former branches
git tag --list v* # All release tags

GitHub doesn't let you remove emoji reactions so I can't change my "both" vote, but with that realisation, I think the only tags we need are the link-preserving X.Y ones (so the source links in the docs for EOL branches don't break).

[brettcannon]

It looks like only @warsaw voted exclusively for EOL/2.6 at this point.

[larryhastings]

if I misread this thread and people want 2.6 no matter what then I don't really see a point in having two tags.

If we don't have something (a tag or branch) called "2.6" we break links from bpo.

[brettcannon]

@larryhastings right, so I say just do 2.6 and skip the EOL tag.

[ezio-melotti]

bpo links can be fixed if necessary, as long as the branch is there. If it's called 2.6 it's less work for me, even though I will anyway have to fix the links for <2.6 if those branches are missing.

[ncoghlan]

It's not just bpo, it's source links in the documentation, mailing list archives, etc. (This is the same reason I want the PSF to keep hg.python.org around indefinitely, albeit in read-only mode)

With tag vs branch serving as an indicator of EOL status, and the plain X.Y tags preserving links regardless of their origin, I think we actually have a solid practical argument in favour of that approach, rather than it purely being a question of aesthetics.

[serhiy-storchaka]

At https://github.com/python/cpython/branches I see that some branches are marked as "active" and others as "stale". Thus GitHub distinguish them. Can it be configured to output only active branches in drop-down menu?

[brettcannon]

@serhiy-storchaka doesn't look like it.

[brettcannon]

Where did this leave off? Can we add the appropriate tags -- we seemed to settle on e.g. 2.6 -- and then delete the relevant branches? Or was there something else that needed to be settled?

[zware]

As far as I can tell, we're at the point of just doing the branch-to-tag replacement.

[brettcannon]

So what steps should be followed? I assume I will need to unprotect the branches from 3.2 - 3.0 and 2.6 - 2.0 to allow for the branch deletion. What git commands are necessary? We will also want to document these steps in PEP 101.

[dstufft]

Assuming you have an origin named upstream that points to python/cpython:

$ git fetch upstream
$ git tag 2.6 upstream/2.6
$ git push upstream refs/tags/2.6
$ git push upstream --delete 2.6

[dstufft]

I haven't tested ^, you can also delete the branch from the GitHub UI.

[ned-deily]

OK, I have now created and pushed signed tags for the following end-of-life branches and then deleted those branches in python/cpython:

2.0
2.1
2.2
2.3
2.4
2.5
2.6
3.0
3.1
3.2
legacy-trunk

That leaves the following still active branches:

2.7
3.3
3.4
3.5
3.6
master

For simple checkouts, the results should be the same before and after, e.g.:
git checkout 2.6

So, I think we are done here. Thanks, all!

[brettcannon]

🍾

Could you include the instructions on how to do this in the issue, Ned, so we have it written down somewhere? Or will you add them to PEP 101?

[ned-deily]

It's basically as Donald suggested but I was planning to add the steps to an EOL section of PEP 101 since it seems like it would make most sense for the tagging and removal of the branch to be done by the release manager as part of the final steps of moving a release cycle to end-of-life.