Runner: Add support for cross-repo runners

[myoung34]

Describe the enhancement

Having an agent _per_ repo is very tedious and frustrating.
It would be nice to have agents that can work across repositories to prevent potentially having hundreds of agents to maintain (with rotating tokens) across hundreds of repos

Code Snippet

n/a

Additional information

n/a

[kubasejdak]

Also, it is hard to synchronize them if they share the same test hardware. Cross-repo runners is a must. Like in GitLab.

[bryanmacfarlane]

Yes! We do have org level runners coming soon along with the ability to edit labels. The combination will allow for more elastic use of runners across your org. Much of the infra is already in place ...

[nodesocket]

Agree, being able to point multiple repos at a single action runner instance is a must.

[Penagwin]

@bryanmacfarlane Is there any chance this will add the ability to run checks on pull requests from forked repositories? Our team currently forks our private repo and merges in, so our pull requests don't trigger any actions.

I know it's for security, but we're managing that through the organization access.

[bryanmacfarlane]

@Penagwin - a big part of org level runners is the ability to manage (route, labels etc.) but also policy and control over things like forks. We would ❤️ your thoughts in this area.

[Penagwin]

I think it'd be great to have org level trusted users - who's pull requests always trigger any actions even from a forked repository, and have repo level trusted forks that are allowed to run checks on pull requests.

Another more nuclear option would be a setting (per-repo) to allow running actions on any pull request to a private repository. It's likely not a good idea to let public repos have that option - and you'll have to handle what happens if somebody changes their repo from private -> public.

Again, because the only members of our organization are employees we can assume any pull request into our repository is non-malicious.

We're excited for labels too as we would like to have a different runner per environment, that way some of our development environments can be in a different part of our network.

[dougparrish]

@bryanmacfarlane

Yes! We do have org level runners coming soon along with the ability to edit labels. The combination will allow for more elastic use of runners across your org. Much of the infra is already in place ...

I have a customer that's interested in org-level runners and offers to be a beta tester. Do you have an ETA for this feature, e.g. 2.21 or 2.22 or farther out? Thanks...

[nodesocket]

👍 also looking for an eta. We have 6 repos, and I really don't want to setup and manage six virtual machines for GitHub action runners.

[GABeech]

As a work-around, the runner config is done in such a way as to allow you to run multiple runners on one machine.

It's still a pain to have to setup individually for each repo, but what i'm doing is creating a folder for each repo actions-runner-repo and running the installation steps there. all the config is contained in that directory and the service is setup based on the directory name.

So, while it's a pain and not as ideal as registering the runner at the org or enterprise level. It's better than running individual machines per repo.

[nodesocket]

@GABeech oh ok, that workaround will be a good solution to keep a single server. Do you know if it's possible to target a particular self-hosted runner based on branch?

if branch === 'master' use runner-production
if branch === 'stage' use runner-stage

[Penagwin]

@nodesocket

There's no support yet for custom labels yet (!?) so that's not currently possible (We want to do the same thing).

The "closest" you can get is if each environment is on a different OS, and use the OS label, but that'd be a really, really stupid workaround.

[bryanmacfarlane]

@Penagwin - custom label support work is currently underway. Org level runner work has also started.

[myoung34]

@bryanmacfarlane The real blocker for us using labels and cross-runner here is also long-term tokens from #244 (we'd want our runners to be automated and that does not align with the current token set up in place).

Is it possible to know if this is going to be released around the same timelines?
Also: are these milestones public? I'd be ok signing an NDA to get more information as its a major production blocker for us to stay fully in Github and not half Github half Gitlab

[bryanmacfarlane]

The api to get a token has released so you can get it JIT programmatically (and not copy / paste out of the UI)

https://developer.github.com/v3/actions/self_hosted_runners/#create-a-registration-token

[myoung34]

thanks @bryanmacfarlane I missed that one!

[bryanmacfarlane]

@myoung34 Cool. Yeah it just released at the end of Jan 😄

[Penagwin]

@bryanmacfarlane Is there any news or ETA on allowing pull requests from forks to trigger the builder for a private repo? This has been a big road block for us in implementing the CI, as this means our tests aren't run until after they're already merged in. We can't raise any warnings or prevent merges.

[edenreich]

ping - any update or release date for cross-repo runners ?

[myoung34]

@bryanmacfarlane is this something i just spied in 2.168.0 ?

Update Runner Register GitHub API URL to Support Org-level Runner (#339 #345 #352)

[TingluoHuang]

@myoung34 those are the runner side changes, there are GitHub.com side changes still in developing. 😄

[myoung34]

@TingluoHuang your feature flag causes your profile to 502 😆 , yall pushing those updates to certain staff? 😜

[bryanmacfarlane]

@myoung34 - yeah, we'll roll the feature flag through staff and internal rings and then bigger rings etc as part of the rollout.

If you're eager to try it and provide feedback then 👍 - let me know what orgs you want to kick the tires on. No promises but I would ❤️ feedback.

[j3parker]

I'd be interested in trying it for the Brightspace org! My email is [REDACTED]

[nodefortytwo]

Would love to try it on getndazn, my email is rick.[email protected]

[Crashdoom]

@bryanmacfarlane Would also be interested in helping trying it out on the ConventionCatCorp org, email is [email protected], if possible 👍

[dentuzhik]

@bryanmacfarlane your favourite feedback givers from @bolteu are CRAVING this, since our orders of new Mac Minis got blocked by COVID-19 supply chain delays and we are consistently running out of minutes 🙏

denis.[email protected]
cc ainar.[email protected]
cc [email protected]

cc @Kerizer

[EricWF]

@bryanmacfarlane

I would love if you could enable this feature for the following organizations:

• efcs
• libcxx

My email is [email protected]

[bryanmacfarlane]

I'll post here as soon as it can be tried out. It's currently in the works.

[hasnat]

thanks for updates. very eager to try this, if you can include organisations on my account.

[davidkarlsen]

I'd want to test this.
david.[email protected] org: tietoevryfs

[wallentx]

Omgggg yas plz! I'm currently running 37 self-hosted runners in k8s.
email is [email protected]
org is bypasslane

[timharris777]

@bryanmacfarlane , we are waiting on this as well for large scale adoption of github actions. Would love to test with our cfacorp org. Thanks!

[wallentx]

The api to get a token has released so you can get it JIT programmatically (and not copy / paste out of the UI)

https://developer.github.com/v3/actions/self_hosted_runners/#create-a-registration-token

    "and not copy / paste out of the UI"

Or.. you know, create some crazy workaround that you finally get to work 3 days after you find out the API was released :sweat_smile:
https://gist.github.com/wallentx/84bc705128c2ab1aba4989ae6a82d83d

[myoung34]

@wallentx usage for the new way is here: https://github.com/myoung34/docker-github-actions-runner/blob/master/entrypoint.sh#L9

[wallentx]

@myoung34 Yessir, essentially what I'm doing currently :+1:

[Penagwin]

We would heavily benefit from this for the org █
If you need more information please let me know, I'm the developer responsible for our team's devops.

[bryanmacfarlane]

I sent mails to those I could. @Penagwin and @timharris777 send me a mail confirming the org url you want to try. My email is my handle and that is at github

[timharris777]

I sent mails to those I could. @Penagwin and @timharris777 send me a mail confirming the org url you want to try. My email is my handle and that is at github

Just sent an email. Thanks!

[anelson]

Org-level runner support is the only thing holding us back from adopting Github Actions for our internal CI. We'd love to get this going on the assuriosw org.

[davidkarlsen]

With org-level runners, should there be a corresponding org-level way of obtaining the runner-tokens - similar to https://developer.github.com/v3/actions/self_hosted_runners/#create-a-registration-token which is repo-level?

[bryanmacfarlane]

Yes, there will be an org level api to obtain a token

[BrettMayson]

I'd love to get this on the Dynulo organization for testing. Email is brett @ mayson.io

[bryanmacfarlane]

@anelson send me an email. It’s above

[vincentbrison]

Hi @bryanmacfarlane, thanks for the update! We at https://github.com/fabernovel would love to try this, as org level runners is the only feature missing for us to fully move to GitHub Action.

[tbm98]

I'm seeing

[bryanmacfarlane]

This shipped today. https://github.blog/changelog/2020-04-22-github-actions-organization-level-self-hosted-runners/

Thanks for all the feedback we've received so far. There's also future work for more improvements in this area but it's supported now.

[myoung34]

[tbm98]

@Penagwin - a big part of org level runners is the ability to manage (route, labels etc.) but also policy and control over things like forks. We would ❤️ your thoughts in this area.

Currently it still doesn't work with forked private repos.

[FabianKoestring]

@Penagwin - a big part of org level runners is the ability to manage (route, labels etc.) but also policy and control over things like forks. We would ❤️ your thoughts in this area.

Currently it still doesn't work with forked private repos.

I think that is another not less important problem!

[edenreich]

is it possible to have this feature also for private accounts (paid membership) e.g not only for organization ? I would like to have one runner for building my repos (cross repo runner) and I think it's unfortunate that this feature is only for organization :/

[edenreich]

In case someone have the same issue running multiple runners with private team account, I've found a walk around, still a bit hacky, but it works.

You can always run multiple systemd services, so by changing the runner name to something unique, it would create a service name with that specified name, not ideal having to run multiple systemd services, would be nice to have a way to add / authenticate repos to / with single running service on private team accounts (not only organization)

So for example instead of actions-runner default name you would give it a name actions-runner-[reponame] then just run the rest of the commands mentioned on github configurations docs and at the end install and start the service sudo ./svc.sh install and sudo ./svc.sh start