Reqwest: Async Rate Limiting

[theduke]

A lot of APIs have rate limits, and manually implementing a rate limiter over and over again is quite annoying to me.

There was some previous discussion in #169 but that was focused on a sync context.

@seanmonstar would you be open to a built in rate limiter?
I imagine it working like this:

• keep a sliding list of past requests per domain which have rate limiting enabled ( token bucket or something simpler)
• if limit is reached, delay requests as necessary by just returning a delayed future

A simple implementation without prioritization could cause cascading delays though. A fix for that would be a request queue.

The complexity for this would not be too high and I would really love that functionality.

An alternative solution to this would be to have a async middleware feature with a pre-request hook that can return a future. That way a third party crate could supply the functionality easily, but async middleware is probably a larger topic. (remotely related discussion regarding sync hooks here: #155)

[seanmonstar]

We've been working on a general middleware stack, and have a form of rate-limiting here: https://github.com/tower-rs/tower/blob/master/tower-limit/src/rate/service.rs

It'd probably be useful to make adjustments there.

[edrevo]

@seanmonstar I've seen examples on how to integrate the hyper client and tower. Is it also possible to do that with reqwest and tower? Is there any example code or documentation that I can use as a hint to build that integration?

Thanks for all of your work with warp, hyper, reqwest & co. It's really great to use your libs!

[seanmonstar]

@edrevo the easiest way is to just use tower::service_fn:

let client = reqwest::Client::new(); // or use builder

let svc = tower::service_fn(move |req| {
    client.execute(req)
});

[ardeaf]

@seanmonstar Would you be able to provide a quick example on how to convert one of the reqwest examples into a rate-limited one?

ie, from the docs:

let client = reqwest::Client::new();
let res = client.post("http://httpbin.org/post")
    .body("the exact body that is sent")
    .send()
    .await?;

How would we refactor that into a rate limited request using tower-limit?

[Mathspy]

A bit late but here's an example of the above using tower if someone looks at this issue in the future

use reqwest::{Body, Method, Request, Url};
use std::time::Duration;
use tower::Service;
use tower::ServiceExt;

let client = reqwest::Client::new();
let mut svc = tower::ServiceBuilder::new()
    .rate_limit(100, Duration::new(10, 0)) // 100 requests every 10 seconds
    .service(tower::service_fn(move |req| client.execute(req)));

let mut req = Request::new(Method::POST, Url::parse("http://httpbin.org/post")?);
*req.body_mut() = Some(Body::from("the exact body that is sent"));

let res = svc.ready_and().await?.call(req).await?;

Definitely a bit more verbose but it works like a charm and the client and service needs to be setup only once anyway

[flexabyte]

I've not had any success getting this working by storing the service in a struct... I can't find a proper way to declare the type of the Service. If I do:

struct ServiceStruct {
    service: dyn tower::Service<Response=HttpResponse,Error=HttpError,Future=Pin<Box<Result<HttpResponse,HttpError>>>>,
... more fields ...
}

let mut svc = tower::ServiceBuilder::new()
    .rate_limit(1, Duration::new(0, 1600)) // 1 request every 1600ms
    .service(tower::service_fn(move |req| client.execute(req)));

ServiceStruct { service: svc, ... more fields }    

But the type definition for tower::Service in the struct is wrong, and I can't find any usable documentation for declaring the type... has anyone had success with this?

UPDATE:

I had the wrong end of the stick with this one, but I got there in the end. For anyone looking to do something similar, here is the adapted code for storing a rate limit service in a struct:

use reqwest::{Body, Method, Request, Url};
use std::time::Duration;
use tower::Service;
use tower::ServiceExt;
use tower_limit::rate::RateLimit;
use tower_util::ServiceFn;

struct ServiceStruct<T> {
    service: RateLimit<ServiceFn<T>>,
}

#[tokio::main]
async fn main() {
    let client = reqwest::Client::new();
    let service = tower::ServiceBuilder::new()
        .rate_limit(100, Duration::new(10, 0)) // 100 requests every 10 seconds
        .service(tower::service_fn(move |req| client.execute(req)));

    let mut service_struct = ServiceStruct{service};

    let mut req = Request::new(Method::POST, Url::parse("http://httpbin.org/post").unwrap());
    *req.body_mut() = Some(Body::from("the exact body that is sent"));

    let res = service_struct.service.ready_and().await.unwrap().call(req).await.unwrap();
    println!("res: {:?}", res);
}