Httpx: Consider `URLLib3Transport` interface, since it's becoming public API.

Created on 21 May 2020  路  6Comments  路  Source: encode/httpx

Follow on from #963

There's a couple of things we could consider in the URLLib3Transport interface, now that it's becoming public API.

  • We don't really want to pass pool_limits, since it doesn't quite fit the parameters that urllib3 uses to configure it's PoolManager. We needed to pass it before, since we were using the class for our sync implementation, and needed the consistency between our async and sync cases, but now that we're providing it separately we should just mirror their native num_pools and maxsize.
  • With httpcore we provide different classes for the proxy and the plain-connection-pool transports. We could? choose to do the same with the urllib3 transport, rather than providing a single class with an optional proxy=... argument.

All 6 comments

Also we could just drop the URLLib3Dispatch compat shim, since it's only ever existed in the pre-releases.

And in any case, never made it to the public interface even then.

The only reason I'd be in favor of keeping URLLib3Transport is to make it easier for folks that use Requests to migrate over without having to deal with slight behavior differences right away.

I'm actually not _certain_ that's what people would do in practice - there are already lots of behavior differences that come from switching from _Requests_ anyway, so not sure if switching to our transport is a big leap.

Another argument in favor of dropping it before 1.0 lands would be reducing maintenance burden - now that the transport interface is public it would be pretty easy for someone to maintain this as a 3rd party component (esp given that we have a working implementation in 0.12.x for people to take inspiration from). But at the same time, it's not very high cost.

Edit: oh okay, I didn't get that you were talking about the Transport/Dispatch compatibility layer. Sure, let's drop it IMO. I guess what I wrote above^ is another discussion then. 馃槃

I'm 馃憤 on dropping URLLib3Transport completely tbh.

I might be missing something but it seems atm it feels like the remnants of our old sync flow rather than a core piece of HTTPX that users would reach for. It already has our lowest test coverage percentage, feels like it will eventually break for someone since it's now in the public API.

Okay, well let's drop the URLLib3Transport/URLLib3ProxyTransport convo from the 0.13 milestone so we can get this thing released, and keep the idea of URLLib3Transport as a third-party-package as an open option.

(Personally I probably think it's a pretty useful thing to have available to us, but I'm not absolute on it.)

Remaining points for 1.0 consideration...

  • Do we want URLLib3Transport/URLLib3ProxyTransport for consistency with httpcore interface, and urllib3's PoolManager/ProxyManager?
  • Do we want to drop URLLib3Transport into a third party package instead?

We can close this off now.

Once #1090 is addressed, the signatures are...

class URLLib3Transport(httpcore.SyncHTTPTransport):
    def __init__(
        self,
        *,
        verify: VerifyTypes = True,
        cert: CertTypes = None,
        trust_env: bool = None,
        pool_connections: int = 10,
        pool_maxsize: int = 10,
        pool_block: bool = False,
    ):

And...

class URLLib3ProxyTransport(URLLib3Transport):
    def __init__(
        self,
        *,
        proxy_url: str,
        proxy_headers: dict = None,
        verify: VerifyTypes = True,
        cert: CertTypes = None,
        trust_env: bool = None,
        pool_connections: int = 10,
        pool_maxsize: int = 10,
        pool_block: bool = False,
    ):
Was this page helpful?
0 / 5 - 0 ratings