Fastapi: [FEATURE] support for rate-limit

[nskalis]

Is your feature request related to a problem? Please describe.

I want to be able to rate-limit a specific endpoint/route by applying a rule: 1 req / 1 sec, both on global scope and on a per-host basis (Starlette already provides you with the client IP address),

because some endpoints, like authz for example, they can be more expensive than others especially when they communicate with other services.

Describe the solution you'd like

A FastAPI dependency, maybe, or a decorator, that can apply the rate-limit rule per route/endpoint.

Describe alternatives you've considered

This is supported in Flask: https://flask-limiter.readthedocs.io/en/stable/
Envoy proxy can rate-limit but requires a Redis instance.

PS. I just noticed a related issue/question about it https://github.com/tiangolo/fastapi/issues/181
Yes, the idea would be to be able to differentiate between types of users.

[wshayes]

Have you looked at Traefik for rate limiting? https://docs.traefik.io/v2.0/middlewares/ratelimit/

[nskalis]

yes, I noticed the other thread, you mentioning it, but it doesn't seem to support rate-limiting per route/endpoint, right ?

[nskalis]

ok, it supports rate-limiting per endpoint, but it is not strict, it is based on average.

Still, would you be interested in implementing that rather than rely on an infrastructure component for it ?

[wshayes]

Traefik is good enough for my needs at the moment and since it's already in my stack ... This does seem like a good plugin feature for FastAPI though.

[dmontagu]

For what it's worth, this probably could (should?) be built as an ASGI middleware and live entirely outside of FastAPI. Then it would also work with all ASGI servers. (For all I know, such a middleware already exists?)

(Traefik is also good enough for me though.)

There is some related discussion here: https://github.com/django/channels/issues/906

The conclusion:

I'm afraid I'm going to have to say "no" - rate limiting is a large field and there's lot of different approaches as to how you throttle and count requests, and where you persist and coordinate client data, and that in my opinion makes it unsuitable for putting into Channels itself (plus the fact that every piece of code we add increases our maintenance overhead on our small team).

As far as I can tell, there's nothing in Channels that would prevent this from being written as a third-party ASGI middleware, and so that's what I would recommend. If there's hooks missing in Channels that would be needed to make it work, then that's a different conversation.

[vitalik]

This is supported in Flask: https://flask-limiter.readthedocs.io/en/stable/

well it's not supported by Flask directly - this is an external library :)
My guess is that this library can be pretty easy changed to support fastapi as well at least the decorators.. - it's just 600 lines of code - so if you replace flask stuff with fastapi stuff - it should be pretty much ready to use

[tiangolo]

So, if you are doing IP based rate-limiting you will need to store that IP somewhere to be able to check if that IP has made more requests than allowed before a new request.

Saving that information in memory is probably a bad idea, as you probably would serve FastAPI with multiple workers, e.g. with Gunicorn using Uvicorn workers (or with Uvicorn starting the workers itself), each one of them would have its own memory and its own state for each IP.

There's no way to make it work by storing data in memory in a generalizable way that doesn't depend on running exactly one single worker.

So, I guess the best way to do it would be to store that in an external place, e.g. a Redis DB, with a counter and a time to live (I've done it with Couchbase). If you are doing that, you can also make it work differently for authenticated users, paid users, etc. Instead of just IP-based.

And as that would require integrating with an external storage system, that would be quite application-specific, depending on the storage system you use to cache already seen IPs. So it wouldn't fit as part of FastAPI itself.

[nskalis]

thank you all for the feedback provided. eventually, I used envoy + redis with all the benefits you mention above :)

[laurentS]

I've taken a stab at adapting flask-limiter to starlette and FastAPI. The result is at https://pypi.org/project/slowapi/ It doesn't have all the features of flask-limiter yet, but I'll work through them as I need them. It's a wrapper around the same limits library which does the actual job, so the foundations should be pretty solid (my own code is much less so...).

[abersheeran]

Maybe you can try https://github.com/abersheeran/asgi-ratelimit