Virtual-environments: API rate limit exceeded for Mac OS hosted by GitHub itself

[evandrocoan]

Describe the bug
I have setup GitHub Actions for Windows, Linux and Mac OS. Both of them use the https://github.com/arduino/actions, and are working fine, except for the Mac OS machine, which keeps complaining: https://github.com/evandroforks/anki/runs/526371357?check_suite_focus=true

 Set up protoc0s
##[error]API rate limit exceeded for 199.7.166.17. (But here's the good news: Authenticated requests get a higher rate limit. Check out the documentation for more details.)
Run Arduino/actions/setup-protoc@master
  with:
    version: 3.x
    include-pre-releases: false
  env:
    pythonLocation: /Users/runner/hostedtoolcache/Python/3.7.6/x64
##[error]API rate limit exceeded for 199.7.166.17. (But here's the good news: Authenticated requests get a higher rate limit. Check out the documentation for more details.)

Area for Triage:

Question, Bug, or Feature?:
"Bug"

Virtual environments affected

• [x] macOS 10.15
• [ ] Ubuntu 16.04 LTS
• [ ] Ubuntu 18.04 LTS
• [ ] Windows Server 2016 R2
• [ ] Windows Server 2019

Expected behavior
GitHub Actions hosted by GitHub itself, should not be complaning about API rate limits, as it is GitHub own server/machine which is originating the GitHub Request for another GitHub Service (GitHub Actions).

Actual behavior
API rate limit exceeded.
https://github.com/evandroforks/anki/runs/526371357?check_suite_focus=true

[maxim-lobanov]

Hello,
The issue comes from the way how GitHub counts requests for rate limit. For unauthorized requests, it limits by IP. All macOS VMs have the same IP address because of infrastructure.

GitHub complains about rate limit regardless of the source of request because GA workflow can be used to spam API too.
The most promising solution is improving those setup tasks to use authorized API requests instead of anonymous ones.
@alepauly , do you have any thoughts about it?

[fenollp]

I'd like to emphasize that windows and ubuntu runners do not get caught by rate limiting (probably because there are many more of these machines).
This makes it pretty surprising to the user (lessens the user experience).

Isn't it possible for all the runners to use the credentials of the user that pushed commits or created the job?

Do you have a link that helps setup explicit authentication in workflows, so this can be used as a workaround?

[alepauly]

@evandrocoan, @fenollp: I'm following up with the infrastructure team to figure out solutions to this. I'll update as soon as I get some answers.

[evandrocoan]

Isn't it possible for all the runners to use the credentials of the user that pushed commits or created the job?

This seems to be the best to solution to avoid the spammers problem.

I tried adding this: (from here https://help.github.com/en/actions/configuring-and-managing-workflows/authenticating-with-the-github_token)

      - name: Set up protoc
        uses: Arduino/actions/setup-protoc@master
        with:
          repo-token: ${{ secrets.GITHUB_TOKEN }}

But the build still failing: https://github.com/evandroforks/anki/runs/528131718?check_suite_focus=true#step:4:8 ##[error]API rate limit exceeded for 199.7.166.17. (But here's the good news: Authenticated requests get a higher rate limit. Check out the documentation for more details.)

I also, notice a Linux Build failing right now: https://github.com/evandroforks/anki/runs/528131695?check_suite_focus=true#step:4:7 ##[error]API rate limit exceeded for 13.90.156.177. (But here's the good news: Authenticated requests get a higher rate limit. Check out the documentation for more details.)

[alepauly]

@evandrocoan Just a quick not that it doesn't look like the setup-protoc Action you are using supports specifying the token. If you were able to do that, then you will certainly be able to get around the current issue. I'm still following with infrastructure to see if there is something we can do but I think the better bet is to perhaps contribute to that Action so it allow the API request to be authenticated. Auth'd requests have a higher limit than anonymous so that should solve the problem.

[evandrocoan]

Thanks for point it out. And as a side note, looks like this think of API rate limit is going to be a more common problem when GitHub Actions get more used. On that case, our Actions Workflow files would have to look like this for every action:

name: Windows Tests

on: [push, pull_request]

jobs:
  test:
    runs-on: windows-latest
    steps:
      - uses: actions/checkout@v2
        with:
          repo-token: ${{ secrets.GITHUB_TOKEN }}

      - name: Cache pacman
        uses: actions/cache@v1
        with:
          repo-token: ${{ secrets.GITHUB_TOKEN }}
          path: C:\Program Files\Git
          key: ${{ runner.os }}-pacman-${{ hashFiles('**/windows_checks.yml') }}

      - name: Set up python
        uses: actions/setup-python@v1
        with:
          repo-token: ${{ secrets.GITHUB_TOKEN }}
          python-version: 3.7

      - name: Set up protoc
        uses: Arduino/actions/setup-protoc@master
        with:
          repo-token: ${{ secrets.GITHUB_TOKEN }}

      - name: Set up node
        uses: actions/setup-node@v1
        with:
          repo-token: ${{ secrets.GITHUB_TOKEN }}
          node-version: 12

      - name: Upload python wheels
        uses: actions/upload-artifact@v1
        with:
          repo-token: ${{ secrets.GITHUB_TOKEN }}
          name: windows_python_wheels
          path: dist

If you pay attention, every third part action would require to have an with: repo-token: ..., which just seems redundant. As of now, the action which I was getting in trouble with was Arduino/actions, but tomorrow it can be any other action.

So, for a consistent Workflow (which always work and do not fail for due third part tools), all actions should have this with: repo-token: ... thing. Then, instead of the user (me) keep adding that redundant thing of with: repo-token: ..., the GitHub Actions Environment could inject this with: repo-token: ... into my recipes automatically. Therefore, all existent actions nowadays (as http://github.com/Arduino/actions) would have to be updated to start accepting this with: repo-token: ... thing.

[fenollp]

In my case the 403s I get from github are from plain curls to github.com. See these logs on macOS.

I agree that this issue is bound to come up more often as GAs usage grows. Let's see what infra people have to say :)

[maxim-lobanov]

@evandrocoan , Actually, All these actions (except Arduino) doesn't use GitHub API so they are not affected by rate limit issue. The setup-protoc is the single one that use it.
In general, I agree, all actions that invokes GitHub API directly should have repo-token parameter but as I know not many actions really invoke GitHub API.

As for your case, I think I have found the root cause of your issue:
You use legacy version of setup-protoc (uses: Arduino/actions/setup-protoc@master) task from deprecated repository: https://github.com/arduino/actions/tree/master/setup-protoc.
The new version is available as uses: arduino/[email protected] and it supports repo-token parameter.
Looks like owner of action doesn't update README properly so readme points to the old way to install task :(
If you click Use latest version button, the correct way to install task will be showed.

Just checked in my repo and

works perfect for me. Could you please try it from your side?

[evandrocoan]

Thanks @maxim-lobanov! It is working 100% now! https://github.com/evandroforks/anki/runs/531198664?check_suite_focus=true (the build is just failing due Mac OS related issues I am solving right now).

[evandrocoan]

Is it safe to expose repo-token to third part GitHub Actions?

On this other topic I found this: https://github.community/t5/GitHub-Actions/Github-Actions-Token-Scope/td-p/39811

 GitHub provides a token that you can use to authenticate on behalf of GitHub Actions, it automatically creates a GITHUB_TOKEN secret to use in your workflow.
The permissions of GITHUB_TOKEN are limited to the repository that contains your workflow. If you need a token that requires permissions that aren't available in the GITHUB_TOKEN, you can create a personal access token and set it as a secret in your repository.

So, this means that GitHub Actions which have access to the repo-token have only limited scope and can only operate over the repository where the token is coming from? Does it mean they can erase my code and its history if they fell like? So it should not be safe to pass that token and for security, should I make a fork of GitHub Actions requiring the repo-token? (after reviewing its code to assure they are not doing anything nasty)

@dae have forked the repository arduino/setup-protoc because neither he or me know enough about GitHub's actions architecture, to know whether it's safe to be exposing the GitHub token to third party actions (on this case arduino/setup-protoc.

Should he keep the fork he did or it is safe to pass the repo-token to the arduino/setup-protoc action?

[maxim-lobanov]

@evandrocoan ,
Sorry, I am not super-familiar with the security side of using community actions so could be wrong, but as I know, GITHUB_TOKEN doesn't allow to change repo history, push to master, or something similar. It provides permissions like contributor role (create new branch / pull-request)

Also, good practice is pointing to particular version of Action like arduino/[email protected] instead of arduino/setup-protoc@master because usually code base under particular tag is not updated and your pipeline won't stop work unexpectedly due to new update in Action. If you point master branch, all new changes will be consumed automatically.

Also, as a possible solution, you can generate own PAT manually, put it to the secrets and use instead of System GITHUB_TOKEN. In this case, you will be able to configure scope of your PAT and restrict it fully (to overcome API Rate limit, you can disable absolutely all permissions for token).
Drawback of this approach is that your own secret PAT won't be available in builds that are triggered on pull requests from forks.

[maxim-lobanov]

@evandrocoan , I am planning to close the issue. Please let me know if you have any other questions or have any concerns