Runner: self-hosted organization runner does not pick up jobs

[igagis]

Describe the bug
The organization-wide self-hosted runner does not pick up jobs. Note, that repository self-hosted runners work ok.

To Reproduce
Steps to reproduce the behavior:

1. Register a new self-hosted runner to an organization
2. Push a new commit to a repository which belongs to the organization from step 1
3. Check actions jobs status. Jobs which have runs-on: [self-hosted] are not picked up, constantly showing "Starting your workflow run...". After some timeout those jobs fail with "This check failed".

Expected behavior
The jobs which have runs-on: [self-hosted] are picked up by the self-hosted runner.

Note, if on step 1 the runner is registered to a repository instead of organization, then it works.

Runner Version and Platform

Runner version: 2.274.2
Ubuntu-18.04

What's not working?

After workflow fails there are messages "No runner matching the specified labels was found: self-hosted." on the "This run" page.

Job Log Output

N/A

Runner and Worker's Diagnostic Logs

````
ivan@geiger:~/actions-runner/_diag$ cat Runner_20201127-224042-utc.log
[2020-11-27 22:40:42Z INFO HostContext] No proxy settings were found based on environmental variables (http_proxy/https_proxy/HTTP_PROXY/HTTPS_PROXY)
[2020-11-27 22:40:42Z INFO HostContext] Well known directory 'Bin': '/home/ivan/actions-runner/bin'
[2020-11-27 22:40:42Z INFO HostContext] Well known directory 'Root': '/home/ivan/actions-runner'
[2020-11-27 22:40:42Z INFO HostContext] Well known config file 'Credentials': '/home/ivan/actions-runner/.credentials'
[2020-11-27 22:40:42Z INFO Listener] Runner is built for Linux (X64) - linux-x64.
[2020-11-27 22:40:42Z INFO Listener] RuntimeInformation: Linux 5.3.0-1016-oracle #18~18.04.1-Ubuntu SMP Mon Apr 27 10:16:19 UTC 2020.
[2020-11-27 22:40:42Z INFO Listener] Version: 2.274.2
[2020-11-27 22:40:42Z INFO Listener] Commit: 9987a8d024a6075cd2357d07c5aec9785126d5a9
[2020-11-27 22:40:42Z INFO Listener] Culture:
[2020-11-27 22:40:42Z INFO Listener] UI Culture:
[2020-11-27 22:40:42Z INFO HostContext] Well known directory 'Bin': '/home/ivan/actions-runner/bin'
[2020-11-27 22:40:42Z INFO HostContext] Well known directory 'Root': '/home/ivan/actions-runner'
[2020-11-27 22:40:42Z INFO Listener] Validating directory permissions for: '/home/ivan/actions-runner'
[2020-11-27 22:40:42Z INFO CommandLineParser] Parse
[2020-11-27 22:40:42Z INFO CommandLineParser] Parsing 1 args
[2020-11-27 22:40:42Z INFO CommandLineParser] parsing argument
[2020-11-27 22:40:42Z INFO CommandLineParser] HasArgs: False
[2020-11-27 22:40:42Z INFO CommandLineParser] Adding Command: run
[2020-11-27 22:40:42Z INFO Listener] Arguments parsed
[2020-11-27 22:40:42Z INFO Runner] ExecuteCommand
[2020-11-27 22:40:42Z INFO ConfigurationStore] currentAssemblyLocation: /home/ivan/actions-runner/bin/Runner.Listener.dll
[2020-11-27 22:40:42Z INFO HostContext] Well known directory 'Bin': '/home/ivan/actions-runner/bin'
[2020-11-27 22:40:42Z INFO ConfigurationStore] binPath: /home/ivan/actions-runner/bin
[2020-11-27 22:40:42Z INFO HostContext] Well known directory 'Bin': '/home/ivan/actions-runner/bin'
[2020-11-27 22:40:42Z INFO HostContext] Well known directory 'Root': '/home/ivan/actions-runner'
[2020-11-27 22:40:42Z INFO ConfigurationStore] RootFolder: /home/ivan/actions-runner
[2020-11-27 22:40:42Z INFO HostContext] Well known directory 'Bin': '/home/ivan/actions-runner/bin'
[2020-11-27 22:40:42Z INFO HostContext] Well known directory 'Root': '/home/ivan/actions-runner'
[2020-11-27 22:40:42Z INFO HostContext] Well known config file 'Runner': '/home/ivan/actions-runner/.runner'
[2020-11-27 22:40:42Z INFO ConfigurationStore] ConfigFilePath: /home/ivan/actions-runner/.runner
[2020-11-27 22:40:42Z INFO HostContext] Well known directory 'Bin': '/home/ivan/actions-runner/bin'
[2020-11-27 22:40:42Z INFO HostContext] Well known directory 'Root': '/home/ivan/actions-runner'
[2020-11-27 22:40:42Z INFO HostContext] Well known config file 'Credentials': '/home/ivan/actions-runner/.credentials'
[2020-11-27 22:40:42Z INFO ConfigurationStore] CredFilePath: /home/ivan/actions-runner/.credentials
[2020-11-27 22:40:42Z INFO HostContext] Well known directory 'Bin': '/home/ivan/actions-runner/bin'
[2020-11-27 22:40:42Z INFO HostContext] Well known directory 'Root': '/home/ivan/actions-runner'
[2020-11-27 22:40:42Z INFO HostContext] Well known config file 'MigratedCredentials': '/home/ivan/actions-runner/.credentials_migrated'
[2020-11-27 22:40:42Z INFO ConfigurationStore] MigratedCredFilePath: /home/ivan/actions-runner/.credentials_migrated
[2020-11-27 22:40:42Z INFO HostContext] Well known directory 'Bin': '/home/ivan/actions-runner/bin'
[2020-11-27 22:40:42Z INFO HostContext] Well known directory 'Root': '/home/ivan/actions-runner'
[2020-11-27 22:40:42Z INFO HostContext] Well known config file 'Service': '/home/ivan/actions-runner/.service'
[2020-11-27 22:40:42Z INFO ConfigurationStore] ServiceConfigFilePath: /home/ivan/actions-runner/.service
[2020-11-27 22:40:42Z INFO CommandSettings] Flag 'help': 'False'
[2020-11-27 22:40:42Z INFO CommandSettings] Flag 'version': 'False'
[2020-11-27 22:40:42Z INFO CommandSettings] Flag 'commit': 'False'
[2020-11-27 22:40:42Z INFO CommandSettings] Command 'configure': 'False'
[2020-11-27 22:40:42Z INFO CommandSettings] Command 'remove': 'False'
[2020-11-27 22:40:42Z INFO CommandSettings] Command 'warmup': 'False'
[2020-11-27 22:40:42Z INFO ConfigurationManager] LoadSettings
[2020-11-27 22:40:42Z INFO ConfigurationStore] IsConfigured()
[2020-11-27 22:40:42Z INFO ConfigurationStore] IsConfigured: True
[2020-11-27 22:40:42Z INFO ConfigurationManager] Is configured: True
[2020-11-27 22:40:42Z INFO ConfigurationStore] Read setting file: 269 chars
[2020-11-27 22:40:42Z INFO ConfigurationManager] Settings Loaded
[2020-11-27 22:40:42Z INFO ConfigurationStore] IsServiceConfigured()
[2020-11-27 22:40:42Z INFO ConfigurationStore] IsServiceConfigured: False
[2020-11-27 22:40:42Z INFO CommandSettings] Command 'run': 'True'
[2020-11-27 22:40:42Z INFO ConfigurationStore] IsConfigured()
[2020-11-27 22:40:42Z INFO ConfigurationStore] IsConfigured: True
[2020-11-27 22:40:42Z INFO ConfigurationManager] Is configured: True
[2020-11-27 22:40:42Z INFO Runner] Could not parse the argument value '' for StartupType. Defaulting to Manual
[2020-11-27 22:40:42Z INFO Runner] Set runner startup type - Manual
[2020-11-27 22:40:42Z INFO CommandSettings] Flag 'once': 'False'
[2020-11-27 22:40:42Z INFO Runner] RunAsync
[2020-11-27 22:40:42Z INFO ConfigurationManager] LoadSettings
[2020-11-27 22:40:42Z INFO ConfigurationStore] IsConfigured()
[2020-11-27 22:40:42Z INFO ConfigurationStore] IsConfigured: True
[2020-11-27 22:40:42Z INFO ConfigurationManager] Is configured: True
[2020-11-27 22:40:42Z INFO ConfigurationManager] Settings Loaded
[2020-11-27 22:40:42Z INFO MessageListener] {
"AgentId": 62,
"AgentName": "geiger",
"PoolId": 1,
"PoolName": "Default",
"ServerUrl": "https://pipelines.actions.githubusercontent.com/eMc1GyGdYp1Pn3AIiLBt1AQy39VBhA0ak6WnGv2vWqR163Rx43",
"GitHubUrl": "https://github.com/cppfw",
"WorkFolder": "_work"
}
[2020-11-27 22:40:42Z INFO MessageListener] Loading Credentials
[2020-11-27 22:40:42Z INFO ConfigurationStore] HasCredentials()
[2020-11-27 22:40:42Z INFO ConfigurationStore] stored True
[2020-11-27 22:40:42Z INFO CredentialManager] GetCredentialProvider
[2020-11-27 22:40:42Z INFO CredentialManager] Creating type OAuth
[2020-11-27 22:40:42Z INFO CredentialManager] Creating credential type: OAuth
[2020-11-27 22:40:42Z INFO HostContext] Well known directory 'Bin': '/home/ivan/actions-runner/bin'
[2020-11-27 22:40:42Z INFO HostContext] Well known directory 'Root': '/home/ivan/actions-runner'
[2020-11-27 22:40:42Z INFO HostContext] Well known config file 'RSACredentials': '/home/ivan/actions-runner/.credentials_rsaparams'
[2020-11-27 22:40:42Z INFO RSAFileKeyManager] Loading RSA key parameters from file /home/ivan/actions-runner/.credentials_rsaparams
[2020-11-27 22:40:42Z INFO MessageListener] Attempt to create session.
[2020-11-27 22:40:42Z INFO MessageListener] Connecting to the Runner Server...
[2020-11-27 22:40:42Z INFO RunnerServer] Establish connection with 100 seconds timeout.
[2020-11-27 22:40:43Z INFO GitHubActionsService] Starting operation Location.GetConnectionData
[2020-11-27 22:40:43Z INFO RunnerServer] Establish connection with 60 seconds timeout.
[2020-11-27 22:40:43Z INFO GitHubActionsService] Starting operation Location.GetConnectionData
[2020-11-27 22:40:43Z INFO RunnerServer] Establish connection with 60 seconds timeout.
[2020-11-27 22:40:43Z INFO GitHubActionsService] Starting operation Location.GetConnectionData
[2020-11-27 22:40:44Z INFO GitHubActionsService] Finished operation Location.GetConnectionData
[2020-11-27 22:40:44Z INFO GitHubActionsService] Finished operation Location.GetConnectionData
[2020-11-27 22:40:44Z INFO GitHubActionsService] Finished operation Location.GetConnectionData
[2020-11-27 22:40:44Z INFO MessageListener] VssConnection created
[2020-11-27 22:40:44Z INFO Terminal] WRITE LINE:
[2020-11-27 22:40:44Z INFO Terminal] WRITE LINE:
[2020-11-27 22:40:44Z WARN GitHubActionsService] Authentication failed with status code 401.
WWW-Authenticate: Bearer
Strict-Transport-Security: max-age=2592000
X-TFS-ProcessId: 70fa918e-f5a5-4e0d-82b6-4722607071c4
ActivityId: 621134b9-ae5c-46e5-b2e7-e2d1049b425f
X-TFS-Session: 13947296-095b-42eb-aa57-eba4d3575a7b
X-VSS-E2EID: 8939e4f7-287c-4695-8b53-fae18f3e9259
X-VSS-SenderDeploymentId: a07ab14e-025a-39c3-8d53-788cd7ce207f
X-TFS-ServiceError: The+user+%27System%3aPublicAccess%3baaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa%27+is+not+authorized+to+access+this+resource.
X-VSS-S2STargetService: 0000005A-0000-8888-8000-000000000000/visualstudio.com
X-MSEdge-Ref: Ref A: CC099E07D0EB4B028AE159734D8C21E7 Ref B: FRAEDGE0913 Ref C: 2020-11-27T22:40:44Z
Date: Fri, 27 Nov 2020 22:40:43 GMT

[2020-11-27 22:40:44Z INFO RSAFileKeyManager] Loading RSA key parameters from file /home/ivan/actions-runner/.credentials_rsaparams
[2020-11-27 22:40:44Z INFO RSAFileKeyManager] Loading RSA key parameters from file /home/ivan/actions-runner/.credentials_rsaparams
[2020-11-27 22:40:44Z INFO GitHubActionsService] AAD Correlation ID for this token request: Unknown
[2020-11-27 22:40:45Z INFO MessageListener] Session created.
[2020-11-27 22:40:45Z INFO Terminal] WRITE LINE: 2020-11-27 22:40:45Z: Listening for Jobs
[2020-11-27 22:40:45Z INFO JobDispatcher] Set runner/worker IPC timeout to 30 seconds.
[2020-11-27 22:50:26Z INFO Terminal] WRITE LINE: Exiting...
[2020-11-27 22:50:26Z INFO ConfigurationStore] IsServiceConfigured()
[2020-11-27 22:50:26Z INFO ConfigurationStore] IsServiceConfigured: False
[2020-11-27 22:50:26Z INFO HostContext] Runner will be shutdown for UserCancelled
[2020-11-27 22:50:26Z WARN GitHubActionsService] GET request to https://pipelines.actions.githubusercontent.com/eMc1GyGdYp1Pn3AIiLBt1AQy39VBhA0ak6WnGv2vWqR163Rx43/_apis/distributedtask/pools/1/messages?sessionId=3b349ea5-34d0-4329-8077-81c7c7c6fb9a has been cancelled.
[2020-11-27 22:50:26Z INFO MessageListener] Get next message has been cancelled.
[2020-11-27 22:50:26Z INFO JobDispatcher] Shutting down JobDispatcher. Make sure all WorkerDispatcher has finished.
[2020-11-27 22:50:26Z INFO Listener] Runner execution been cancelled.
````

[tfenster]

Just to add: I am running into the same issue with runners on Windows

[dhoppe]

I changed my account to an organisation so that I do not have to configure the self-hosted runners for each repository individually.

At the beginning I also had the error message mentioned above. Are you sure it is a private repository? The self-hosted runners are only allowed for private repositories for security reasons.

The self-hosted runners are only displayed for private repositories.
Private repository:

https://github.com/dhoppeIT/ansible-role-demo/actions/runs/432991963

Public repository:

https://github.com/dhoppeIT/ansible-role-demo/actions/runs/432997701

I think the error message is a bit misleading. Instead of stating that no self-hosted runner was found with the label in question, it should be pointed out that self-hosted runners are only allowed for private repositories.

[igagis]

@dhoppe why do you think self-hosted renners are only allowed from private repos? Where is it stated?
As I wrote in the original bug description, in case I register my self-hoster runner to a single repo (not to the organization) then it does pick up the jobs. Aren't there security vulnerability in this case?
Yes, my repos are public, but I don't see reasons why single repo self-hosted runner works, but organization-wide one doesn't.

[dhoppe]

https://docs.github.com/en/free-pro-team@latest/actions/hosting-your-own-runners/about-self-hosted-runners#self-hosted-runner-security-with-public-repositories

Just take a look at my first screenshot.

All repositories, excluding public repositories

You could create a new group which also allows public repositories, but this requires a GitHub Enterprise account.

[igagis]

@dhoppe Thank you, that worked! I even was able to modify the Default group to allow self-hosted runners for public repos and I don't have Gihub Enterprise account, so it is not needed for that.

I think this issue can be closed then.