Cli: Error: API Request rate limit surpassed for application

[mispy]

I'm running into this problem:

✔ Finished hashing 10849 files
- CDN diffing files...
✔ CDN requesting 2218 files
- Uploading 2218 files
 ›   Error: API Request rate limit surpassed for application

That was the first deploy today, so it's not an issue of deploying too frequently. It seems that the CLI tool doesn't limit itself to match the API rate limits?

Environment:

owid@terra3:~$ /mnt/volume-nyc1-01/owid/mispytest-grapher-code//node_modules/.bin/netlify --version
netlify-cli/2.6.0 linux-x64 node-v10.14.2

[fool]

Thanks for the report, @mispy . This is not unique to you, and our developers will be taking a look as soon as they can

[fool]

developing a list of customers to inform once we have a fix:

• https://app.intercom.io/a/apps/q245f50x/inbox/inbox/1479299/conversations/20320282682
• https://app.intercom.io/a/apps/q245f50x/inbox/inbox/search/conversations/20233263008
• https://app.intercom.io/a/apps/q245f50x/inbox/inbox/search/conversations/20148633045
• https://app.intercom.io/a/apps/q245f50x/inbox/inbox/unassigned/conversations/20171691765

[c0b41]

I have same problem, that's why i moved firebase for now

[fool]

Just an update - we have identified the problem and have a proposed solution that we'll implement soon. For the meantime, you can successfully deploy with our old (still functional and supported) CLI netlifyctl: https://github.com/netlify/netlifyctl

We'll report here when we have a fix.

[niklasravnsborg]

I also have the same problem:

Hope there is a fix soon.

[bcomnes]

Working on a fix for this now.

[bcomnes]

Issue is solved in master, expect a release tomorrow morning.

[bcomnes]

🎉[email protected]🎉

Please give that version and shot and let me know how it goes. Sorry it took so long to get out, I was traveling this last week.

[njam]

I have the same problem, with version 2.6.2:

$ npm install -g netlify-cli
/usr/local/bin/ntl -> /usr/local/lib/node_modules/netlify-cli/bin/run
/usr/local/bin/netlify -> /usr/local/lib/node_modules/netlify-cli/bin/run
+ [email protected]
added 292 packages from 167 contributors in 13.138s

$ netlify deploy --site=$NETLIFY_SITE --auth=$NETLIFY_AUTH_TOKEN --dir=dist/ --prod
Deploy path: /builds/denkmal/denkmal-website/dist
Deploying to live site URL...
- Hashing files...
✔ Finished hashing 748 files
- CDN diffing files...
✔ CDN requesting 718 files
- Uploading 721 files
 ›   Error: API Request rate limit surpassed for application

Second run succeeded with 202 files uploaded. So I guess the first run uploaded ~500 files before it errored.

[thomasjsn]

Same problem as before here as well; able to upload about 500 files at the time;

Deploy path: /home/hebron/uctrl-hugo/public
Deploying to live site URL...
✔ Finished hashing 3432 files
✔ CDN requesting 1855 files
🕑  (795/1856) Uploading img/parts/sized/ir-pir-sensor-qerk04-48.jpeg... ›   Error: Uploaded file /img/parts/sized/kapton-tape-gold-qerxy4-96.jpeg did not match deploy
➜  uctrl-hugo git:(master) ✗ netlify deploy --dir=public --prod
JSONHTTPError: Too Many Requests
    at NetlifyAPI.getCurrentUser (/usr/lib/node_modules/netlify-cli/node_modules/netlify/src/open-api/index.js:152:15)
➜  uctrl-hugo git:(master) ✗ netlify deploy --dir=public --prodDeploy path: /home/hebron/uctrl-hugo/public
Deploying to live site URL...
✔ Finished hashing 3432 files
✔ CDN requesting 1074 files
* (433/1075) Uploading img/parts/sized/ac-adapter-zgvz5q-96.jpeg... ›   Error: Uploaded file /img/parts/sized/air-pollution-sensor-r4dn8g-96.jpeg did not match deploy
- (433/1075) Uploading img/parts/sized/ac-adapter-zgvz5q-96.jpeg...%                                         
➜  uctrl-hugo git:(master) ✗ netlify deploy --dir=public --prod
JSONHTTPError: Too Many Requests
    at NetlifyAPI.getCurrentUser (/usr/lib/node_modules/netlify-cli/node_modules/netlify/src/open-api/index.js:152:15)

  netlify-cli/2.6.2 linux-x64 node-v11.6.0

[RajaBellebon]

Same problem using 2.6.3

 $ netlify deploy --dir dist/ -a $NETLIFY_TOKEN -s $NETLIFY_SITE_ID --prod
  Deploy path: /opt/atlassian/pipelines/agent/build/website/dist
  Deploying to live site url...
  - Hashing files...
  ✔ Finished hashing 3105 files
  - CDN diffing files...
  ✔ CDN requesting 3075 files
  - Uploading 3075 files
  ›   Error: API Request rate limit surpassed for application

[bcomnes]

Reopening. @thomasjsn that problem was fixed in 2.6.3 although it still sounds like there is an open issue. I will investigate.

[bcomnes]

Testing with 10000 files.

[bcomnes]

I think I identified the issue. I'll work to get a fix out tomorrow.

[bcomnes]

Possible fix here, but I'm finding there are some unhandled / unexpected API responses that need more attention.
https://github.com/netlify/js-client/pull/33

More tomorrow.

[bcomnes]

Ok, releasing https://github.com/netlify/cli/issues/206 with a fix once it's green.

[bcomnes]

Give [email protected] a shot. I tested it with 10000+ file uploads. Rate limiting kicks in around every 500 uploads, with a 60 second delay (controlled by the API). This is kind of tedious, and I will be discussing it with our API team.

[RajaBellebon]

@bcomnes just a question - why do we have to use node 8.10 >= ? any particular reason?

[bcomnes]

Oclif requires it: https://github.com/oclif/oclif/blob/master/package.json#L50

[bcomnes]

Use https://github.com/nodesource/distributions, homebrew or https://github.com/creationix/nvm to get modern Node.js.

[bcomnes]

Rate limit trigger was significantly increased, you should rarely see rate limits now.