Httpx: Combining efforts?

[njsmith]

Hey, I just discovered this. Looks pretty cool!

I don't know if you're aware, but we've been working on a similar project for a while now:

• https://github.com/urllib3/urllib3/issues/1323
• https://github.com/python-trio/urllib3/

Some key features:

• Based on urllib3, rather than a rewrite, to take advantage of their years of work finding edge-cases. Supports tricky stuff like proxies and early responses.
• Designed to support multiple sync and async backends (asyncio, trio, twisted, ...) within a single code base
• Python 2 support (not sure how important this is – we might end up dropping it – but there are some important projects like botocore and pip that want to support async but won't be able to drop python 2 for a few years yet)
• Working with the urllib3 devs to manage the transition: https://github.com/python-trio/urllib3/issues/84

I'd love to hear more about what you're planning, and compare notes. Could do it here, or do a video chat on Monday, or perhaps we could meet up at PyCon this week?

[tomchristie]

Yes mate! I'm not at PyCon sadly. We can either chat here or DM me on twitter and I'll ping you my skype username.

I've been doing some work on the adapters branch that's not yet made it back into master, but together with what's already here that'd give us...

• Easy to add trio support - all the asyncio specific bits factored into a single (small) place.
• HTTP/1.1 and HTTP/2 support.
• Sync and ASync variants.
• Composible approach to adding in behaviors for redirects, cookie persitence, environment modifiers (eg. .netrc), authentication.

We should chat.

[tomchristie]

We chatted. And it was ✨good✨.

I'm v. confident in the approach here - tho there will doubtless be tons of good experience and edge casing for us to bring across from urllib3. (Eg. decoding gnarliness, early responses, etc.)

Going to keep working on getting the API complete, fully test-covered, etc.etc. then can review what good stuff to pull across from urllib3's gnarly-real-world HTTP experience.

[njsmith]

I'd definitely suggest checking out the core HTTP logic and networking layer, which is where a lot of the attention has gone:

https://github.com/python-trio/urllib3/blob/bleach-spike/src/urllib3/_async/connection.py
https://github.com/python-trio/urllib3/tree/bleach-spike/src/urllib3/_backends

This handles proxy support, early responses, the backend API has been iterated on repeatedly to narrow it down to something workable, etc.