Hyper: client: tunneling via CONNECT method example

[MOZGIII]

I'd like to use CONNECT method with HTTP (at least with 1.1, and, ideally with 2).

However, I don't think there's a way to unwrap the underlying stream from HTTP response.

Practically, I need an object that implements futures::io::AsyncRead and futures::io::AsyncWrite (from futures 0.3). Similar thing via tokio should work too.

[MOZGIII]

For anyone looking for a solution - I just made this for now: https://github.com/MOZGIII/http-proxy-client-async
It's not nearly as full-featured as hyper, but it does the job. It can be used with hyper's Connector impls or manually crafted async TCP/TLS streams.

[seanmonstar]

There's now an examples/http_proxy.rs file in master!

[MOZGIII]

Niice!

[MOZGIII]

That is actually a server example, while I'm looking for a client example. I.e. I want my app to connect to somewhere using HTTP protocol with CONNECT method. Not for other clients to connect to my app.

Please reopen.

[MOZGIII]

Oh, nvm, it's two in one.

[MOZGIII]

Ok, it's not. The code never establishes a connection to a third-party server via HTTP CONNECT proxy. It does implement tunneling for an HTTP proxy server. I need a client.

[seanmonstar]

Ah OK. As a starter, it'd be similar to this HTTP upgrades example, just setting CONNECT method instead of an upgrade: blah header.

[MOZGIII]

Thanks, that's what I was looking for!

[MOZGIII]

Except, the CONNECT method doesn't do a connection Upgrade, server just responds with HTTP/1.1 200 OK (with \r\n\r\n) and the tunnel is ready to go. How can I manually hijack the duplex TCP stream from the connection?

[MOZGIII]

If I recall correctly when I checked last time, the situation was the same: there was no way to get a raw stream from the body, and that was the blocker.

[seanmonstar]

I'm not sure I follow. Something like this should work:

async fn client_tunnel(addr: SocketAddr) -> Result<()> {
    let req = Request::builder()
        .uri(format!("{}", addr))
        .method("CONNECT")
        .body(Body::empty())
        .unwrap();

    let res = Client::new().request(req).await?;
    if res.status() != StatusCode::OK {
        panic!("Our server didn't CONNECT: {}", res.status());
    }

    match res.into_body().on_upgrade().await {
        Ok(upgraded) => {
            if let Err(e) = client_upgraded_io(upgraded).await {
                eprintln!("client foobar io error: {}", e)
            };
        }
        Err(e) => eprintln!("upgrade error: {}", e),
    }

    Ok(())
}

[MOZGIII]

Why would on_upgrade be invoked if the server doesn't send 101 Switching Protocols with the response, and I don't send any Upgrade header with the request?
Is it just that on_upgrade has a confusing name, but the implementation works correctly?

[seanmonstar]

It's the same fundamental mechanism, but the name "connect" is overloaded. It could also sound like a TCP connect.

[MOZGIII]

I'd name it into_stream() to avoid unnecessarily coupling to the Upgrade in the glossary.
As it is now, there's absolutely no way I would've guessed on_upgrade also works for CONNECT method, since CONNECT method doesn't rely on the upgrade mechanism, since CONNECT was in the HTTP long before we invented Upgrade and status code 101).

[MOZGIII]

Ok, maybe not into_... since that's a pattern and this fn returns a Future which doesn't fir the usual expectations, but still - it'd make way more sense if it was named after what it actually does.

[MOZGIII]

If not - at least the documentation has to properly explain that the internal implementation is not coupled with 101 status code and the Upgrade header, and will just provide the underlying stream either way.

I really like how Go does it - it's called Hijack() there. It's their own name, so people, in general, aren't familiar with it when they start using Go, but at least it doesn't cause this kind of confusion.