Requests: HTTPoxy migitation

[remram44]

https://httpoxy.org/

It is possible to set the HTTP_PROXY in CGI scripts by passing the Proxy header. If the script uses requests to download files, requests will happily use the attacker-supplied proxy to make requests.

This should be mitigated like it is in Perl (since 2001), Ruby, and libraries like curl.

I confirmed that HTTP_PROXY (in uppercase) is accepted as well as the conventional, lowercase http_proxy (requests 2.7.0)

[Lukasa]

We've been discussing this at length in IRC. We have a complex set of opinions here, but here they are:

1. In general, if you are running your Requests script in a location where your application allows writes to the environment, you should disable Requests' search of the environment. We have a flag for this: Session.trust_env. Setting it to False entirely and completely mitigates this risk.
2. CGI is an _extremely_ uncommon mode for running Python code. It is stunningly inefficient, and as far as I know essentially zero Python applications are developed using it.
3. Our search for proxies is actually done by the Python standard library. This means that the more efficient fix is in the Python standard library itself, which can mitigate the problem not just for Requests but for all the other clients in the Python standard library.

I am willing to consider the possibility of raising warnings when running Requests inside a CGI process with trust_env=True, and I'm even willing to consider the possibility of forcing trust_env to False in such a situation, but realistically for Python code the correct solution to this is simply to _not run your applications inside CGI_.

[remram44]

Makes sense to me. Avoiding HTTP_PROXY (uppercase) in a CGI context would probably be a good move, but if requests is not doing that directly there is probably no sense in you taking active measures. I myself only ever use wsgi. I'm gonna go ahead and close this.

[Lukasa]

@remram44 For what it's worth, I would _heartily_ support a patch to the stdlib's urllib.request module's getproxies method to implement this kind of checking. That seems like a much more productive place to put the patch. =) If you want to open a bug report for that, I'll happily chime in: I may even volunteer to write the patch myself!

[remram44]

I filed cpython-27568.