Conda-forge.github.io: tasks and discussion for new GPU CI queue

[beckermr]

This issue is to document tasks and todos for the new GPU CI queue.

To-Do Items:

• [ ] decide on software to provision the CI
  
  
  
  • best candidates from internal discussions are drone or azure
  
  
• [ ] develop a process to permission feedstocks on the CI
  
  
  
  • [ ] we will want an allow list of feedstocks that are allowed to use the queue
  
  
  • [ ] we will want to add a job to conda-forge/admin-requests to add feedstocks to this list and provision them with the proper keys/permissions to access the CI
  
  
• [ ] put in changes to smithy to allow separate build and test phases in the CI config files
• [ ] make sure the build phase does not tie up a GPU in the CI system
• [ ] put in monitoring for the load on the queues
  
  
  
  • we have existing tools that will be able to output the load in five-minute increments to the conda-forge status page
  
  
  • we may want more than this however
  
  
• [ ] establish and document a clear process of who to contact when things fail or break

cc @dharhas @jakirkham @kkraus14 @viniciusdc

@mariusvniekerk for azure stuff

I missed Ray and Kent on the github handles. Can someone ping them here?

[jakirkham]

cc @ocefpaf @mike-wendt @raydouglass @teoliphant

[jakirkham]

cc @h-vetinari (Axel)

[jakirkham]

Should add I also don't know Kent's or others' GH handles. So please cc others as needed. Thanks! 😄

[beckermr]

Ahhhh thanks! I could not find Axel's github handle. See this one too for the other azure stuff: https://github.com/conda-forge/conda-forge.github.io/issues/1273

[dharhas]

cc @Kllewelyn @rtiwariops from openteams.

[leofang]

Very exciting!

[h-vetinari]

Thanks for opening this @beckermr!

Linking my closing comment from #1062 for reference. TL;DR:

In short: it would be amazing if any of the involved people / companies could take this as impetus to chip in something as well. GPU computing is only ever going to get larger, and I believe that sharing some (comparatively low) CI infra to enable conda-forge to do the building & integration costs would provide huge bang-per-buck for the people & companies that are building & using such packages.

[dharhas]

Adding @leej3 from Quansight.