Reqwest: API for setting an HTTP proxy for a Client

[Arnavion]

Along the line of hyper::Client::with_http_proxy

Useful for debugging without affecting other applications like an OS-level proxy setting would.

[ajdlinux]

As far as I can tell, reqwest doesn't implement the standard Unix convention of using the http_proxy and https_proxy environment variables to specify a proxy URL. This is a blocker for migrating my application from direct use of hyper, which manually implements this convention using with_http_proxy().

[Arnavion]

Well, at the very least if reqwest had something like with_http_proxy then the application using reqwest could set the proxy (eg via env_proxy like rustup does). Of course reqwest could also have a convenience method to read the env vars and set the proxy accordingly.

[ajdlinux]

Yes, that's what we currently do - we manually get the proxy URL from the environment and use the url crate to extract the necessary components for with_http_proxy(). (Thanks for the link to env_proxy, I might have to start using that!)

As such, an equivalent to with_http_proxy() would be really, really helpful.

[seanmonstar]

I've been working on this feature the past couple days, along with adding support to hyper for https-over-https-proxies. I'd like to take this opportunity to get the design right. Some things I'd like to consider for this feature:

• Common case is an HTTP proxy, make that easy
• Probably support common environment variables
• The design must make it easy to add in other protocols later, like SOCKS
• Some proxies require authorization, would be nice if a proxy in reqwest could be configured to pass the Proxy-Authorization header
• Some may want a list of proxies, such as 1 for HTTP and 1 for HTTPS
• curl offers the ability to make an exclude list for proxies, where any URL in the list skips using a proxy. Is this very useful, or sufficiently niche enough that people can just use 2 Clients?

---

I've spent most of my time getting https-over-https working, so the code I have so far doesn't handle most of the above considerations. At the moment, it only supports a single proxy, but making them is quite easy.

let http = Proxy::http("example.proxy.domain", 4321);
let https = Proxy::https("secure.proxy.example", 5432);

let proxy = Proxy::env(); // constructor name? allow passing a list of env vars to check?

// how to use them is not determined yet

// either
let client = reqwest::Client::proxied(http);
// or
client.proxy(https);

Maybe building a list is simply adding or anding proxies together? let list = http + https or http & https.

I'm not sure how you would clearly add an authorization to a proxy. Perhaps a mutator method, proxy.authorization(auth_string). Though it seems that this method and combining proxies such as above could lead to confusion...

[ajdlinux]

curl offers the ability to make an exclude list for proxies, where any URL in the list skips using a proxy. Is this very useful, or sufficiently niche enough that people can just use 2 Clients?

It's a bit niche, but I'm sure it would make life easier for people, particularly if it implements the no_proxy environment variable as per curl.

[Mazwak]

I'm evaluating rust, and I wanted to test reqwest but without proxy support it's not very usefull in my case.

All the use cases you listed are important. Exclude list are niche indeed, but they might as well be implemented to support PAC files.

I would add to your list :

• Support automatic proxy settings on Windows through WinINET or WinHTTP calls.
• Support PAC files. This is not easy as it's a javascript file, but would be tremendous.

[tshepang]

@seanmonstar would it be bad to add it as a method of Client:

let client = Client::new().proxy(url);

I imagine url will be IntoUrl, so I don't know how you'd do the user:pass thing... maybe:

let client = Client::new().proxy_url(url).proxy_auth(auth);

[tshepang]

@seanmonstar I also don't think Reqwest should read envvar... that's something that apps should handle, not libraries. I don't even think there should be a convenience API to read such. It should be easy enough to do this in those apps, once the API for using proxies is added.

[euclio]

Any movement on this? This is also blocking me from using reqwest instead of hyper.

[ajdlinux]

woot! 🎉🎉🎉

[ark-]

How would one use a proxy now then?

[sebasmagri]

@ark- you should look at the proxy tests for guidance.

[ark-]

@sebasmagri Thanks that does what I need