Discord.js: Getting ratelimited

[Koyamie]

Please describe the problem you are having in as much detail as possible:
I am getting ratelimited by discord for an unknown reason
As said here it should never happen but I do not know why this is happening for me
https://github.com/discordjs/discord.js/blob/6be5051f9296fe9a10549f2f9a11c18776b1b617/src/rest/RequestHandler.js#L149

For more information, it's happening frequently at 00:00 GMT

I am also having a lot of 500 errors when this is happening.

Include a reproducible code sample here, if possible:

429 hit on route /gateway/bot

Code : 500
AbortError: The user aborted a request.
at RequestHandler.execute (/home/koyamie/Koya/node_modules/discord.js/src/rest/RequestHandler.js:106:9)
at runMicrotasks (<anonymous>)
at processTicksAndRejections (internal/process/task_queues.js:93:5)

Further details:

• discord.js version: lastest master
• Node.js version: 12.13.0
• Operating system: debian 9
• Priority this issue should have – please be realistic and elaborate if possible:

• [x] I have also tested the issue on latest master, commit hash:

[Koyamie]

Hello, kinda high issue, happening everyday since I updated the repo (was on a september commit)

[Fyko]

Is it possible you hit the max of 1000 logins per day? Have you tried listening on the debug event?

[Koyamie]

Is it possible you hit the max of 1000 logins per day? Have you tried listening on the debug event?

Nope, when I am hitting 1k logins a day its different, I can't login for 24 hours and debugging tells how many time I have to wait, here I am getting a 429 "Too many requests" and debugging only shows this :
"429 hit on route /gateway/bot"

[monbrey]

If you need help with discord.js installation or usage, please go to the discord.js Discord server instead: https://discord.gg/bRCvFy9
This issue tracker is only for bug reports and enhancement suggestions.

Since this is happening at exactly 0:00 it would seem there's some scheduled task running at this time causing it. We can investigate further on Discord.

[Koyamie]

If you need help with discord.js installation or usage, please go to the discord.js Discord server instead: https://discord.gg/bRCvFy9
This issue tracker is only for bug reports and enhancement suggestions.

Since this is happening at exactly 0:00 it would seem there's some scheduled task running at this time causing it. We can investigate further on Discord.

Yeah but, as the line is saying "This should never happen" I guess this is related to the lib.

[DevYukine]

i could imagine that this might be related to multiple Shards/Clients fetching the /bot/gateway endpoint at the same time and since they dont know of each other they do too many requests and hit the ratelimit, are you sure you dont fetch the /bot/gateway endpoint on your own? because d.js should only ever fetch that on startup and i can't imagine that your bot restarts everytime on 0:00 GMT

[Koyamie]

i could imagine that this might be related to multiple Shards/Clients fetching the /bot/gateway endpoint at the same time and since they dont know of each other they do too many requests and hit the ratelimit, are you sure you dont fetch the /bot/gateway endpoint on your own? because d.js should only ever fetch that on startup and i can't imagine that your bot restarts everytime on 0:00 GMT

I think too but I have no reason fetching this endpoint on my own.
Yup my bot isn't restarting everytime but at 0:00 GMT my game timestamps reset (like the tatsumaki daily and reputation) but there is no reason that it fetch this endpoint 🤔
This is weird

[DevYukine]

Okay i just looked in the src and saw that d.js actuall does call the /bot/gateway endpoint everytime it reconnects (this shouldn't be like that) and since you mentioned that you get these every day at 0:00 GMT this probaly means that your bot reconnects every day at that time (maybe related to your host?) sadly d.js does always fetch the session endpoint even when resuming a session but vlad actuall fixed that in #3393 so you could give this pr a try since it should fix your issue (he changed it so it only fetches /gateway/bot if no session_id id present).

Also you should probaly check why the bot reconnects everytime at 00:00 GMT, do you maybe change your ip or does your isp/host do that? i would recommend looking into that.

Even tho #3393 probaly fixes this issue i will open another one because the current implementation is very poor designed for bots using any ShardingManager and this needs to be changed since we currently do not handle ratelimits for /gateway/bot across shards.

[Koyamie]

Okay i just looked in the src and saw that d.js actuall does call the /bot/gateway endpoint everytime it reconnects (this shouldn't be like that) and since you mentioned that you get these every day at 0:00 GMT this probaly means that your bot reconnects every day at that time (maybe related to your host?) sadly d.js does always fetch the session endpoint even when resuming a session but vlad actuall fixed that in #3393 so you could give this pr a try since it should fix your issue (he changed it so it only fetches /gateway/bot if no session_id id present).

Also you should probaly check why the bot reconnects everytime at 00:00 GMT, do you maybe change your ip or does your isp/host do that? i would recommend looking into that.

Even tho #3393 probaly fixes this issue i will open another one because the current implementation is very poor designed for bots using any ShardingManager and this needs to be changed since we currently do not handle ratelimits for /gateway/bot across shards.

This is was I thought, I will give a try to this PR and check tonight if it is happening again and will update the issue. My bot is not reconnecting at 0:00 this is why I do not understand 😯

[DevYukine]

Okay i just looked in the src and saw that d.js actuall does call the /bot/gateway endpoint everytime it reconnects (this shouldn't be like that) and since you mentioned that you get these every day at 0:00 GMT this probaly means that your bot reconnects every day at that time (maybe related to your host?) sadly d.js does always fetch the session endpoint even when resuming a session but vlad actuall fixed that in #3393 so you could give this pr a try since it should fix your issue (he changed it so it only fetches /gateway/bot if no session_id id present).
Also you should probaly check why the bot reconnects everytime at 00:00 GMT, do you maybe change your ip or does your isp/host do that? i would recommend looking into that.
Even tho #3393 probaly fixes this issue i will open another one because the current implementation is very poor designed for bots using any ShardingManager and this needs to be changed since we currently do not handle ratelimits for /gateway/bot across shards.

This is was I thought, I will give a try to this PR and check tonight if it is happening again and will update the issue. My bot is not reconnecting at 0:00 this is why I do not understand 😯

Are you sure about this? because it very much looks like it (reconnecting does not only include re-identifying but also resuming a session) can you make sure and listen to the shardReconnecting event (e.g log it to the console or somehow notify yourself) and update this post once you tested it?

[Koyamie]

Update : I started debugging tonight to see what happen at 0:00 GMT, I found what in my bot was getting me ratelimited for 1h.

I was hitting 429 very very many times but with different enpoints, the most I saw was /channels/ID/messages this one should be fixed in my code but still hitting 429 even if it should never happen.

I havn't tried the PR because I got no time yesterday.

Anyway, I'll keep this issue open until we get a fix on master.

[iCrawl]

You are being very vague about this.

/channels/Id/messages, doesn't mean a lot. This needs more information on your part, maybe even some code samples of what your usage is and why you are hitting it.

It would be nice if this is getting a bit more context.

[DevYukine]

If you get ratelimited on other routes e.g /channels/ID/messages you might hit the Global Ratelimit and then hitting a 429 is normal but then there is still the question why you hit /gateway/bot aswell at that time 🤔

[Koyamie]

You are being very vague about this.

/channels/Id/messages, doesn't mean a lot. This needs more information on your part, maybe even some code samples of what your usage is and why you are hitting it.

It would be nice if this is getting a bit more context.

Yes very sorry will explain better as soon as I can

If you get ratelimited on other routes e.g /channels/ID/messages you might hit the Global Ratelimit and then hitting a 429 is normal but then there is still the question why you hit /gateway/bot aswell at that time 🤔

Maybe hitting it too much result to a gateway ban for 1 hour (Can't connect for 1 hour everytime it happens)

[Koyamie]

Hello, this is happening today but in different way, restarted my bot after making changes and now bot is having very difficulties to connect and I am hitting 429 way too much (I should never hit 429 remember)
Any help ?

Could be a possible API issue (https://status.discordapp.com/) but still, I should not hit 429

[Koyamie]

Hello, it happened again tonight with discord increased API response time. Any news for this ?

[Koyamie]

UP

[Nooc34]

Having the same issue here now.
429 hit on route /gateway/bot

Have 0 idea this has never happened before.

very frustrating

[vladfrangu]

If it happens once or a few times and you're sharded across multiple processes, it happens. If it's spammed, it's an issue.

d.js version, node.js version please!

[Nooc34]

Discord.js ^12.2.0
Node.js 11.x

It isnt spammed, just happens every minute or so.

[monbrey]

Discord.js ^12.2.0
Node.js 11.x

Discord.js v12 has a minimum Node version of v12 also. This doesn't line up

[Nooc34]

it worked for 3 months originally?

[SudhanPlayz]

Even me bot the problem. But i was just connected like 10 times

429 hit on route /gateway/bot
429 hit on route /gateway/bot
429 hit on route /gateway/bot
429 hit on route /gateway/bot
429 hit on route /gateway/bot
429 hit on route /gateway/bot
429 hit on route /gateway/bot
429 hit on route /gateway/bot
429 hit on route /gateway/bot
429 hit on route /gateway/bot
429 hit on route /gateway/bot
429 hit on route /gateway/bot
429 hit on route /gateway/bot
429 hit on route /gateway/bot
429 hit on route /gateway/bot
429 hit on route /gateway/bot
429 hit on route /gateway/bot
429 hit on route /gateway/bot
429 hit on route /gateway/bot

Node Engine: 12.x
Discord.js: ^12.2.0

[SudhanPlayz]

My project is not working on Any Bot Tokens Pls help everybots getting strucked at

Provided token: NzEwODE2MTI3MTA5OTU1NjA1.XsOUQw.***************************
Preparing to connect to the gateway...
429 hit on route /gateway/bot

I Don't what I want to do. also I just logging one time in my project

[Koyamie]

My project is not working on Any Bot Tokens Pls help everybots getting strucked at

Provided token: NzEwODE2MTI3MTA5OTU1NjA1.XsOUQw.***************************
Preparing to connect to the gateway...
429 hit on route /gateway/bot

I Don't what I want to do. also I just logging one time in my project

You are IP banned for 1 hour, just wait and retry.

[SudhanPlayz]

Thanks! I will try after 1 hour

[SudhanPlayz]

The ip was banned forever i think even it not worked now. I was using glitch.com to run my bot

[CatsMiaow]

• Node.js v12.18.2
• discord.js v12.3.1

Suddenly the bot did not respond without any error, and when I restarted the bot and tried to client login again, it did not login without any error.
When I called the Discord API directly from the server, I found out that Discord blocked it.

curl https://discord.com/api/v6

{"code": 0, "message": "You are being blocked from accessing our API temporarily due to exceeding our rate limits frequently. Please read our docs at https://discord.com/developers/docs/topics/rate-limits to prevent this moving forward."}

When I tried again after an hour, it would be nice if the bot was logged in but got an error saying it was blocked in Discord.

[Fyko]

I was running into this problem for... a very long time (dating back to mid march). Here are some sparse error logs. I reached out to a Discord staff member in the Discord Developers server and they said a bot on my network was consistently hitting rate limits and that led to a one hour CloudFlare ban.

In the end, I decided to migrate two of my larger bots (300 & 900 guilds) from my local machines ~ into the cloud ~ and that seemed to mitigate the problem entirely -- haven't ran into any bans since.

tldr: hoarding multiple bots on a single network should be avoided

[DjDeveloperr]

For me, it keeps spamming. It's a serious problem for me, bot goes down for an hour. And users shouting lol.
Seems to be a bug on Discord's end, since the retry after header is incorrect and so the lib keeps trying to reconnect.
This starts happening with a similar rate limit, on channels something route, it spams it and then gets IP banned. I'm sure discord.js is handling rate-limits correctly and I'm not doing any API Request outside lib.
In that channel rate limit too, even after the retry after time span ends, it's still getting rate limit headers, so probably Discord giving out wrong values for rate-limits?
Some info. I'm only hosting one bot on the VPS. My bot is in 330 ~ guilds, and not sharded of course. My bot usually hits limits on messages because of how it should actually work lol, but that shouldn't be an issue since Discord.js handles rate limits correctly. I'll look further to contact with Discord about this issue if this comment isn't getting noticed.. since people on the Discord.js server have been ignorant anyway.
This never happened to me before, just started happening since a week and it's like happening every 2-3rd day.
You might be curious how am I so sure about "incorrect rate limit headers", I had modified src/RequestHandler.js and added some more info to the log from the headers (and body too, showing that its _IP Banned_).
If you're curious what code I added, here it is:

    } else if (res.status === 429) {
      // A ratelimit was hit - this should never happen
      this.queue.unshift(item);
      let body = await res.text();
      this.manager.client.emit('debug', `429 hit on route ${item.request.route} - rate limit ${this.limit} - retry after ${this.retryAfter} - reset ${this.reset} - remainaing ${this.remaining} - global ${(this.manager.globalTimeout}\nResponse Body: ${body}`);
      await Util.delayFor(this.retryAfter);   
      return this.run();

About the logs I see in console from this rate limit:

Hope it helps to identify.. would appreciate if this issue is resolved..

Discord.js Version => 12.3.1
Node.js Version => 12.18.2

[Koyamie]

Happened to me today again too.
Usually it is mostly due to the sharding not having the same headers because of the large amounts of shards and the speed of the requests.

But today my logs are showing A SUPER HUGE spam of [RATELIMIT] GET /gateway/bot requests. Probably shards reconnecting but this went OVER 10k 429s in under 10 minutes which is very high.

[DjDeveloperr]

This bug is annoying, my bot just got rate limited again. I opened logs on time, and found that it was spamming some rate limit on messages. But due to the headers being incorrect, it had kept retrying and got IP banned at last, for an hour. I think this is a bug related to Discord API, no?

[msciotti]

Sorry to sort of butt in here—for the folks having this error, are you running your bot on multiple processes on the same machine? I'm unsure if DJS (or other libraries) support coordinating cross-process rate limits out of the box/without some config options enabled.

[DjDeveloperr]

Nope, my bot is on single process and not sharded.
It happened again (some minutes ago)...
Edit:
I just noticed that the RetryAfter header is in seconds when this rate-limit is hit! I got to know by observing the change in RetryAfter header, because the lib is treating it as given in milliseconds, while its coming in seconds from the API.
Still though, what is causing this rate-limit (or IP ban, we can say) exactly? Anyway, I think, my bot was hit suddenly on channel/:id/messages something route. (And yeah, my bot messages pretty fast sometimes because of its nature, but it shouldn't be a cause since the rate limits are handled correctly.)

[kyranet]

Note: it would be helpful to also include the version of discord.js you are using, e.g. v12.3.1 - this way we can check whether you are using old versions (specially v11) or latest one.

Also, you can still reproduce this by restarting your bot quickly after it logged in.

Sorry to sort of butt in here—for the folks having this error, are you running your bot on multiple processes on the same machine? I'm unsure if DJS (or other libraries) support coordinating cross-process rate limits out of the box/without some config options enabled.

discord.js doesn't have persistent nor coordinates cross-process rate limits, such thing would require an external service (i.e. Redis) to keep the state and help synchronizing all clients.

[DjDeveloperr]

I'd appreciate if this issue is given a priority. It's very disturbing \:v

Note: it would be helpful to also include the version of discord.js you are using, e.g. v12.3.1 - this way we can check whether you are using old versions (specially v11) or latest one.

I had mentioned before, but, again mentioning it,
Discord.js Version => 12.3.1
Node.js Version => 12.18.2

Also, I did a hacky work-around which at least doesn't spam after getting IP Banned. I noticed the RetryAfter header is in seconds when this ban is hit, I just modified it a bit again.
In src/RequestHandler.js

    } else if (res.status === 429) {
      // A ratelimit was hit - this should never happen
      this.queue.unshift(item);
      let body = await res.text();
      this.manager.client.emit('debug', `429 hit on route ${item.request.route} - rate limit ${this.limit} - retry after ${this.retryAfter} - reset ${this.reset} - remainaing ${this.remaining} - gt ${(this.manager||{globalTimeout:0}).globalTimeout}\nResponse Body: ${body}`);
      await Util.delayFor(this.retryAfter * (body.includes("You are being blocked") ? 1000 : 1));
      return this.run();

It would wait until the time is over correctly and not spam 429s.
Still not able to find out the cause of this.

[vladfrangu]

I just noticed that the RetryAfter header is in seconds when this rate-limit is hit! I got to know by observing the change in RetryAfter header, because the lib is treating it as given in milliseconds, while its coming in seconds from the API.

Unless you changed the REST api version to v8 yourself via client options, Discord returns the header as ms instead of seconds right now on the API version we support and use (v7)..

Now, I wonder if this is a Cloudflare ban instead of a Discord ban, @msciotti if you could enlighten us that'd be awesome (based on the JSON body response and the retry-after header being seconds, my gut says it is)! If this IS a Cloudflare ban, can we get a code that isn't 0 in the JSON body, if possible? 😅 Something like the current global ratelimit response would be amazing (or at least a different code so we can check by it and globally limit the entire library from doing further requests)

For @DjDeveloperr, since I noticed you modified the code, can you on 429's log the via header from res.headers? Discord replies will always have that header (usually with the content 1.1 google; Cloudflare bans don't have that).. We currently don't handle the difference between Clouflare and Discord bans, which may cause this issue

[DjDeveloperr]

I'm pretty late here - I had found the cause lol. It was because of webhooks, which sent "logs" so fast and their rate limits were not handled.

For @DjDeveloperr, since I noticed you modified the code, can you on 429's log the via header from res.headers? Discord replies will always have that header (usually with the content 1.1 google; Cloudflare bans don't have that).. We currently don't handle the difference between Clouflare and Discord bans, which may cause this issue

It doesn't happen anymore.

Unless you changed the REST api version to v8 yourself via client options, Discord returns the header as ms instead of seconds right now on the API version we support and use (v7)..

I didn't change, but I had to multiply the retryAfter with 1000 for avoiding the additional spam. :/

[vladfrangu]

I didn't change, but I had to multiply the retryAfter with 1000 for avoiding the additional spam. :/

That's most likely because the API ban comes from Cloudflare (or so I assume)... Hence why the multiply by 1000 worked; this is why I asked if you can log the Via header 😃

[Koyamie]

Yes, the API ban is coming from Cloudflare, we also call it a CF ban.

For my case, I will explain how it was happening:

My bot was hitting the 50req/s limit set by Discord, this was causing a Global Ratelimit. The shard that was receiving the ratelimit was correclty handling it but other ones could not, this is how I was getting massive 429's (with 320 shards means a lot) because they were still sending request while being ratelimited.

When this was happening it could solve itself after few seconds and was hitting around 300-400 429's but sometimes it was reaching the 10k limit in few seconds.

The last few weeks, it was happening 1 time a day and sometimes 2.

[DjDeveloperr]

I asked if you can log the Via header 😃

Sorry, but since I removed the webhook logs Rate limit didn't get hit since then, I assume its fixed now. It was a problem on my end.

[Koyamie]

I can confirm that using webhooks to logs creates high rate of 429's.

Im reaching around 100 ratelimits in past 10 minutes when I enable them.