Mybinder.org-deploy: Consider a process around dedicated nodes for large events

[choldgraf]

We recently have had two large events both create massive Binder usage all at once. In particular, SciPy had many tutorials launch several hundred Binder sessions, and the HEP conference is going to do the same around about 20 tutorials/talks.

In some cases where lots of binder launches will be attached to a large event (e.g., SciPy), maybe it would be beneficial to have a dedicated BinderHub for it. That way we can track resource usage a bit better, and also control some things like limits across all repositories on that hub.

But, what kind of process should we use around this? When do we decide a dedicated hub is needed, and who puts in the labor to put it up? In addition, especially for large conferences with budgets, how can we ask for some kind of resources in return? We should figure these kinds of things out if we want something like this.

[kratsg]

It would be cool to enable tagging/flagging (almost like an urchin tag) to make this a bit easier to deal with.

[betatim]

I'm not sure I like the idea of a dedicated BinderHub. Setting one up, operating it and tearing it down comes with overhead that is bigger than adding a few repos to our list of "more resources for this repo" list. It is worth doing for demos like we did at NeurIPS but for SciPy and PyHEP I liked the repo based workflow we have.

@kratsg cacn you explain what you mean and what an urchin tag is?

[minrk]

For metrics purposes, adding a label to expanded-capacity launches could be nice so it's easier to tease-out how much of our load is from those vs regular traffic.

[manics]

As a longer term goal, we could look at a one-click deployment button for GKE (or even provide a choice of cloud), like for Azure?
https://github.com/alan-turing-institute/binderhub-deploy#rocket-deploy-to-azure-button

This might encourage conferences to setup their own BinderHub. Though requires people to think about required resources more than a day before the workshop 😀

[matthewfeickert]

This might encourage conferences to setup their own BinderHub.

To my mind this sounds _awesome_.

Though requires people to think about required resources more than a day before the workshop :smile:

Wouldn't it be the opposite though? 🙃 If you have your conference has its own BinderHub and know the number of attendees then don't you just need to spin it up with the ability to scale to the max number of attendees? I'm potentially missing obvious things here.

[manics]

@matthewfeickert I meant the _BinderHub_ would need to be setup in advance 😀.

More seriously this means conference organisers need to be planning this as a core part of the conference infrastructure, which includes ensuring they have dedicated time to setup and test, that they've got a cloud account with funds or have sponsorship, etc.