Requests: HTTP Proxy with prepared request (honouring env. var.)

[mxjeff]

Running the following it looks like the proxy env. var. is not honoured:

>>> # requests version 2.2.1 / python 3.4.0
>>> import requests
>>> import os
>>> os.environ['HTTP_PROXY'] = 'http://some.wrong.dns.example.org'
>>> req = requests.Request('GET', 'http://httpbin.org/get')
>>> r = req.prepare()
>>> s = requests.Session()
>>> s.send(r)
<Response [200]>

Is it the expected behaviour with the lower level interface ?
I was actually expecting request to transparently use HTTP_PROXY as it does with the main interface requests.get() or requests.request('GET',...).

Cheers

[Lukasa]

It's certainly the _coded_ behaviour: if you don't use something that goes through Session.request then you never get the environment variables.

However, I think this is unexpected. It doesn't seem like there's any reason we couldn't move the call to Session.merge_environment into Session.send, so I'm inclined to say we should do that. Unfortunately, it's a bit backward-incompatible IMO, so I'd want to sit on this until 2.9.0 or, more likely, 3.0.0. Thoughts @sigmavirus24?

[piotr-dobrogost]

OTOH such isolation from the environment could be desirable and the reason someone uses prepared requests in the first place.

[sigmavirus24]

So we explicitly made merge_environment_settings a public method which could be used by people taking advantage of the PreparedRequest flow. Perhaps we never quite documented it very well when we added it, but I'm not really for moving most of the logic into send because then someone's going to want something that does far less than send does and we will be wanting to move things back into Session.request. I think the existing behaviour is fine. We probably just need to document merge_environment_settings as a _potential_ part of the Prepared Request flow.

[Lukasa]

Sounds reasonable to me.

[mxjeff]

Alright, thanks for the clarification.

Then how should I use merge_environment_setting in order to merge only HTTP proxies var ?

Is this correct ?

>>> import requests
>>> import os
>>> os.environ['HTTP_PROXY'] = 'http://some.wrong.dns.example.org'
>>> req = requests.Request('GET', 'http://httpbin.org/get')
>>> r = req.prepare()
>>> s = requests.Session()
>>> settings = s.merge_environment_settings(req.url, {}, None, False, None)
>>> s.send(r, **settings)
# => should raise a gaierror because of wrong proxy in os.environ

@sigmavirus24 : I agree, the existing behavior is fine, It just lacks some precision in the documentation though.

[sigmavirus24]

@mxjeff right, that's why I said we probably need to document it better.

Also your code sample is correct.