Binderhub: Migrate ALL providers to expecting main branch, not just GitHub, and announce the change

[sgibson91]

The work done in https://github.com/jupyterhub/binderhub/pull/1172 means that binderhub now looks for the main branch if a ref isn't provided, rather than master. This is probably a change we should roll out across all providers but I, at least, am unfamiliar with where these other communities are in this process and this change will be breaking previously established behaviour, possibly disrupting a lot of users. Therefore, I think we should have some form of "announcement" (blog post, twitter thread...) ready to go when we do make this change, just so users aren't blind-sided and they can find information on how to fix their links.

_Note: This change does not affect gists over which the user has no control of the ref naming._

TODOs

• [ ] Draft a blog post/twitter thread to announce the change
• [ ] Change the behaviour of the placeholder variable added in PR #1172

placeholder = "main";
if (provider === "gist") {
    placeholder = "master";
}

[minrk]

I think what might be even more appropriate, if more complex, is to use a special not-valid ref e.g. $default which tells us to use the git repo's default branch. It's not common to specify the branch name when using the default branch for things like git clone, and we are seeing the consequences of Binder's API diverging from this standard practice. I think it's worthwhile to investigate what would be involved in that (i.e. is there a good git and/or github call to resolve the default branch?) before hardcoding "main" as default everywhere. It may not be feasible, in which case this change is probably right, but I think we can prevent the inevitable confusion by letting the branch actually be unspecified.

[sgibson91]

Yes, I agree that that is the ideal resolution to this!

[sgibson91]

Would it need to be resolved before or after the repo is cloned?

_Edited to add:_ We could probably use the GET request in this code block if it needed to be resolved before https://github.com/jupyterhub/binderhub/issues/1145#issuecomment-682438661

[minrk]

Would it need to be resolved before or after the repo is cloned?

That's what I was aiming at with "is there a good git and/or github call to resolve the default branch?" GitHub definitely has an API for it, but I wonder if we can use the git remote API to make it work for all the git providers without needing a new implementation for every provider. That's always been a mysterious API to me, though.

[minrk]

Looks like it's possible, if we parse this output:

$ git ls-remote --symref https://github.com/minrk/jupyter-activity-data HEAD
ref: refs/heads/main    HEAD
97275c467fc137d7a3947c8f9b43635c69a4ea8f    HEAD
$ git ls-remote --symref https://github.com/jupyterhub/jupyterhub HEAD
ref: refs/heads/master  HEAD
f92560fed024288ed834e8e7f8a26363d6117a7c    HEAD

[sgibson91]

That's very cool!

[betatim]

Take a look at #843 which was closed by #895. Maybe we can hoist this out of the Git repo provider and use it for GitHub and GitLab as well?

[minrk]

Yes, this is related to #895 which also uses ls-remote to resolve refs. The addition of --symref HEAD is what lets us find the branch name associated with HEAD.

But this is even easier than I thought! HEAD is already a valid ref, so it's working already and we should just make our default ref HEAD instead of a branch name, and our existing APIs already work:

https://mybinder.org/v2/gh/binderhub-ci-repos/requirements/HEAD

This is strictly better than resolving the default branch name and using that in the URL, because it allows creating a stable link that will work across default branch changes without changes to the link.

Need to test with all providers, but I think it'll work.

[minrk]

1186 switches the default ref to HEAD. Works everywhere without modification, except for gist, where we were resolving master -> latest ref ourselves.