Mybinder.org-deploy: Write a new credit request for Turing cluster

[sgibson91]

The Turing cluster has run out of money on its Azure subscription. This means that the subscription has been put into a read-only state and most attempts to communicate with the cluster will probably result in an error.

I can submit another request for credits from the Turing. Here is the first request so we can see what we're dealing with: https://hackmd.io/SxF25aeRRCS1ZptyShYxtg
And here's a template for the new request: https://hackmd.io/87gUhsl9QtOVT5pzhBSy-Q (I'd like to keep these separate if at all possible, for history/record-keeping, but feel free to copy-paste anything across)

Things that will need updating:

• [x] No longer need to create a subscription
• [x] The amount requested - more than $10k?
  
  
  
  • Some rough calculations bring us out at $15k
  
  
• [x] Do we want the new allocation to start ASAP or be more in-line with the financial year? (i.e. October start)
  
  
  
  • Start ASAP RE: https://github.com/jupyterhub/team-compass/issues/320#issuecomment-662318498
  
  
• [x] Some tweaks to Platform Justification to update it but I think can pretty much stay as is
• [x] I think the Research Justification section will be a nice place to discuss the impact having the Turing within the Federation has had, thereby adding weight to our argument that we should continue to fund it
• [x] Computational Requirements
  
  
  
  • A bigger k8s cluster - I think we were at 2 core nodes and 4 user nodes after the last quota bump
  
  
  • No longer require a Container Registry as we're using DockerHub
  
  
  • No longer require the 'burstable' VM for the bot as we're using DockerHub
  
  
• [x] List of people who require access can probably be removed if we're not adding new people (those already with access should not have it revoked). Also, I think new policy may require those to be granted access to have a Turing email account, so may involve more paperwork than the last time we submitted a request.
• [x] Costing breakdown should reflect the changes outlined in Computational Requirements
  
  
  
  • Generated using Azure Pricing Calculator
  
  

If anyone has any free time/brain-space to contribute, please do! ✨

[sgibson91]

I've made the starting date 1st September 2020 as I don't know when the next working group that allocates the sponsorship will be meeting. Given it's August/summer holidays soon, it may be a while.

[sgibson91]

Some back of the envelope calculations bring about a $15k budget from beginning Sept 2020 to end of Sept 2021

[sgibson91]

I found out that these reviews are now online and happen almost immediately after submission. So I've bumped the start date up to August 1 since there's only the research justification section left and I'm pretty confident we can be ready to submit before the end of this month.

[sgibson91]

From chat with @choldgraf: We should tweak this to get as much cash per allocation as reasonably possible and we can always adjust the percentage of traffic to the Turing cluster to reflect this. Projecting extra traffic <--> extra cash would be useful here. I think I'll try asking for $25k?

[sgibson91]

TuringBinderClusterUsageJune2020.pdf

Here's a June 2020 usage report if it's helpful

[sgibson91]

I've upped the request to $25k (from ASAP to 30th September 2021) and we can scale Turing's percentage as we see fit to spend/not-spend the credits. I'll submit at 5pm BST.

[sgibson91]

I found out that these reviews are now online and happen almost immediately after submission. So I've bumped the start date up to August 1

This was potentially a "bad idea". Not critically speaking, just extra admin overhead and worth bearing in mind for the next round. The research computing working group (RCWG) are uncomfortable allocating credits that span an "Azure year" (Oct -> Oct) and I think this is due to a lack of guarantee on how/when Microsoft will continue to provide this gift and also a lack of "behind the scenes" processes.

I got comments back from the RCWG which I think stem from a misunderstanding that we will _always_ be asking for credits for this cluster and that allocating credits should be thought of as a donation because it's the "right thing to do" rather than judging this on the usual "research output" rubric. I brainstormed with Kirstie on Friday around why this is internally/externally a Good Thing for the Turing to be involved in and I will write a short paragraph in response. I also aim to transform this latest request into a template so that the next credit request has less overhead.

[choldgraf]

@sgibson91 we should also brainstorm ways that the Binder project can signal to Turing that Turing is useful in the ways that Turing cares about, if that makes sense. From the perspective of organizations, Binder cannot be seen as charity - it can be part of the reason they give, but there also needs to be a strategic benefit to them somehow otherwise the champions at each institutions will have to keep fighting internal battles to keep the credits flowing

[sgibson91]

@choldgraf I've written a paragraph to address that at the bottom of this hackmd https://hackmd.io/87gUhsl9QtOVT5pzhBSy-Q I would love feedback (I'm having a brain fog kinda day so it's not my best work)

They're definitely keen on the proposal, I think I just buried the lead a bit on exactly what this contribution is and I think we're using a process designed for funding research output and this doesn't fit that cookie cutter (though I understand that it does).

[sgibson91]

The good news is that the credit request went through while I was on holiday! 🎉 Now #1485 just needs resolving and we'll be back in business :)