Httpx: http2 not work when proxies apply to the AsyncClient
import asyncio
import httpx
proxies = {
"https": "http://127.0.0.1:12590",
"http": "http://127.0.0.1:12590"
}
async def main():
async with httpx.AsyncClient(http2=True, verify=False, proxies=proxies) as client:
response = await client.get('https://www.google.com/')
print(response.status_code)
print(response.http_version)
asyncio.run(main())
200
HTTP/1.1
import asyncio
import httpx
async def main():
async with httpx.AsyncClient(http2=True, verify=False) as client:
response = await client.get('https://www.google.com/')
print(response.status_code)
print(response.http_version)
asyncio.run(main())
200
HTTP/2
houoop
All 11 comments
Hi @houoop,
I went ahead and fixed the code formatting.
I'm wondering if the reason you don't get HTTP/2 back is due to verify=False. AFAIK HTTPS is required by _us_ for HTTP/2 to work (we don't support Upgrade: h2c yet, see https://github.com/encode/httpx/issues/503).
In general, what really helps is to be able to reproduce against a 100% local setup. Can you clarify what your localhost proxy is?
Eg I tried to spin up a local HTTP/2 + HTTPS setup using trustme-cli, hypercorn and mitmproxy. I'm running into certificate issues between the server and the proxy, but here's what I've got:
Click to expand
Dependencies:
$ python -m venv venv
$ . venv/bin/activate
$ pip install hypercorn starlette trustme-cli mitmproxy
Generate localhost certificates:
$ trustme-cli
$ cat server.key server.pem > cert.pem
ASGI app:
# app.py
from starlette.applications import Starlette
from starlette.routing import Route
from starlette.responses import PlainTextResponse
async def home(request):
return PlainTextResponse("Hello, world!")
app = Starlette(routes=[Route("/", home)])
Start hypercorn:
$ hypercorn app:app --keyfile server.key --certfile server.pem
Start the proxy:
$ mitmdump --cert "*=cert.pem"
HTTPX script:
import asyncio
import httpx
proxies = {"http": "http://localhost:8080", "https": "http://localhost:8080"}
async def main():
async with httpx.AsyncClient(
http2=True, verify="client.pem", proxies=proxies
) as client:
r = await client.get("https://localhost:8000")
print(r, r.http_version)
print(r.text)
asyncio.run(main())
HTTPX output:
<Response [502 Bad Gateway]> HTTP/1.1
<Response [502 Bad Gateway]> HTTP/1.1
<html>
<head>
<title>502 Bad Gateway</title>
</head>
<body>
<h1>502 Bad Gateway</h1>
<p>Certificate verification error for localhost: unable to get local issuer certificate (errno: 20, depth: 0)</p>
</body>
</html>
mitmproxy output:
127.0.0.1:61974: clientconnect
127.0.0.1:61974: CONNECT localhost:8000
<< Certificate verification error for localhost: unable to get local issuer certificate (errno: 20, depth: 0)
127.0.0.1:61974: clientdisconnect
florimondmanca
on 24 Mar 2020
HTTP/2 cleartext can be achieved in 2 ways:
- First the client sends an HTTP/1.1 request requesting to the server an upgrade to HTTP/2 by providing some headers like:
Connection: Upgrade, HTT2Settings,Upgrade: h2c,HTTP2Settings: .... If the server can speak HTTP/2 then it answers with a response status 101 (Switching Protocol), and then the actual response to the request in HTTP/2 protocol.
- Second is known as "with prior knowledge". The client does not send a first HTTP/1.1 request but directly send the request in HTTP/2 format (with frames...) assuming "with prior knowledge" that the server understands the HTTP/2 protocol. All of this without TLS.
Implementing the second option "with prior knowledge" would be very useful to query HTTP API without having to generate certificates (even self-signed).
laurentvaills
on 2 Apr 2020
I'm wondering if the reason you don't get HTTP/2 back is due to
verify=False. AFAIK HTTPS is required by _us_ for HTTP/2 to work (we don't supportUpgrade: h2cyet, see #503).
@florimondmanca I can verify that setting verify to anything, not just False, seems to break the HTTP/2 support. See example:
import httpx
import asyncio
import ssl
async def main():
ctx = ssl.SSLContext()
ctx.set_ciphers("ECDHE-ECDSA-AES128-GCM-SHA256")
async with httpx.AsyncClient(http2=True, verify=ctx) as client:
response = await client.get('https://www.google.com')
print(response.http_version)
asyncio.run(main())
Is there a better way to specify SSL cipher suites that does not break HTTP/2 support?
valiant1x
on 10 Apr 2020
im having these exact problem, using verify and http2 and proxy #905
is there any solution?
GnubiBORED
on 13 Apr 2020
@gnubibored No, there doesn鈥檛 seem to be an obvious solution yet; looks like something is simply not working on our side.
There is work being done in the HTTPCore repo to improve our proxy implementation and properly support HTTPS there, hopefully it will help us resolve this.
florimondmanca
on 13 Apr 2020
the issue is this wasnt happening when 4 months ago i was using httpx, so im gonna try using a previous release for see in what change works and in what no
GnubiBORED
on 13 Apr 2020
Thanks, would be useful to pinpoint at which release this started occurring, indeed. :-)
florimondmanca
on 13 Apr 2020
Someone found something about this? I been trying different versions but im not able to reproduce the issue, maybe i was wrong, in the ones i tested i get errors with ssl or i get http/1 when using proxy
GnubiBORED
on 19 Apr 2020
I just merged a change in HTTPCore dealing with this issue so this should be resolved from httpx 0.13.
yeraydiazdiaz
on 4 May 2020
@yeraydiazdiaz Thanks, I'm looking forward to the latest release.
houoop
on 4 May 2020
I just merged a change in HTTPCore dealing with this issue so this should be resolved from httpx 0.13.
I can confirm the issue is resolved in 0.13.0.dev1 release. HTTP/2 is preserved when specifying a custom verify SSL context.
valiant1x
on 10 May 2020
Related issues
x0day
路
4Comments
edocod1
路
3Comments
florimondmanca
路
4Comments
florimondmanca
路
3Comments
iwoloschin
路
4Comments
Most helpful comment
I can confirm the issue is resolved in 0.13.0.dev1 release. HTTP/2 is preserved when specifying a custom
verifySSL context.