Runtime: HTTP connection primitives

[scalablecory]

Background and Motivation

In certain cases, when using MaxConnectionsPerServer, the current connection pooling logic in SocketsHttpHandler has some undesirable behaviors:

• MaxConnectionsPerServer doesn't work the way it sounds. Connections are partitioned into individual pools by some properties of the request, and each pool independently abides by this maximum connection limit. 4 pools means the actual limit is 4x larger than desired.
  
  
  
  • If you have strict limits, such as by firewall or Azure SNAT, this will not help you stay under them.
  
  
  • It is host-based, not IP-based, so you might end up well under your limits too.
  
  
• When MaxConnectionsPerServer is reached, a request waits in a queue until an in-flight request ends.

Proposed API

```c#
public abstract class HttpConnection : IDisposable, IAsyncDisposable
{
public ConnectionState State { get; }

// just calls ReserveStreamAsync() + SendWithReservationAsync()
public Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken);

// Used to reserve a stream on an HTTP/2 connection.
// Good for scenarios where if one connection is full, you will try another.
public abstract bool TryReserveStream(out HttpStreamReservation stream);
public abstract Task<HttpResponseMessage> SendWithReservationAsync(HttpStreamReservation stream, HttpRequestMessage request, CancellationToken cancellationToken);

// Used for when user wants to split out the "wait in queue" from the actual request activity.
public abstract Task<HttpStreamReservation> ReserveStreamAsync(CancellationToken cancellationToken);

public void Dispose();
protected abstract void Dispose(bool disposing);

public ValueTask DisposeAsync();
protected abstract ValueTask DisposeAsyncCore();

}

// The current connection state. Not a guarantee that the next Send() will succeed, but can be used as a quick pre-check.
public enum ConnectionState
{
Open, // can accept more requests.
Closing, // e.g. we have received GOAWAY from a HTTP/2 server, so no new requests should be made but existing reservations can continue.
Closed // e.g. we're doing Connection: close, so no new requests should be made.
}

public struct HttpStreamReservation
{
// no public members.
// wraps HTTP/2 Stream ID.
}

public class HttpSettings
{
public int MaxResponseHeadersLength { get; set; }
// ... and maybe some other things that exist on SocketsHttpHandler, like ResponseDrainTimeout
}

public sealed class Http1Connection : HttpConnection
{
public Http1Connection(IConnection connection, HttpSettings settings);
}

public sealed class Http2Connection : HttpConnection
{
public Http2Connection(IConnection connection, HttpSettings settings);
}

public class HttpRequestException
{
// internal member made public.
public RequestRetryType AllowRetry { get; }
}

// internal type made public.
public enum RequestRetryType
{
// The request must not be retried; this indicates we aren't certain the server hasn't processed the request.
NoRetry,

// The request failed on the current HTTP version, and the server requested it be retried on a lower version.
RetryOnLowerHttpVersion,

// The request failed due to e.g. server shutting down (GOAWAY) and should be retried on a new connection.
RetryOnNewConnection

}

```

Thoughts

• Some niceties of HttpClient, and even SocketsHttpHandler, would not be present.
  
  
  
  • Proxy resolution/establishment (IConnection should already be an established connection)
  
  
  • Versioning e.g. ALPN, H2C Upgrade, ALT-SVC upgrade.
  
  
  • Pooling.
  
  
  • Auto-redirect. (could be added, but may not be appropriate here)
  
  
  • Auto-decompress. (could be added. might be appropriate, but I think some other primitives to assist might be better)
  
  
• Some non-trivial but also not huge code reorganization needed e.g. negotiate auth and HTTP/2 stream ID allocation.

[msftbot[bot]]

Tagging subscribers to this area: @dotnet/ncl
Notify danmosemsft if you want to be subscribed.

[davidfowl]

Is IConnection related to https://github.com/dotnet/runtime/issues/1793? (say yes 😄)

[scalablecory]

Is IConnection related to #1793? (say yes 😄)

yes.

[maksimkim]

Few comments:

1. Should State and Dispose apis belong to underlying IConnection rather than HttpConnection?
2. Should HttpStreamReservation-based API belong to Http2Connection only? In case of http1 assuming client manages connections this api is hardly needed unless I'm missing smth.

[scalablecory]

1. Should State and Dispose apis belong to underlying IConnection rather than HttpConnection?

State is meant to communicate the protocol-level shutdown state, which is different from basic socket connectedness.

1. Should HttpStreamReservation-based API belong to Http2Connection only? In case of http1 assuming client manages connections this api is hardly needed unless I'm missing smth.

Yep, it's a no-op for HTTP/1. This just allows consistency in code calling these APIs. I tried it the way you suggested and wasn't happy with how it looked (especially when considering HTTP/3 coming down the road).

[bbepis]

Coming from a proxy development point of view, this would be a really welcome change.

Right now to implement a proxy for use by HttpClient, you're only able to either create an implementation based on IWebProxy (which is really unstable, not to mention performance issues) or create a custom HttpClientHandler implementation which does the HTTP 1.1 protocol handling from scratch. Essentially re-inventing the wheel, when these primitives already exist in the framework but are unable for public consumption.

Being able to spin up a new HttpConnection in a custom HttpClientHandler, and handing it a pre-initialized stream/connection would negate the need for a custom HTTP protocol implementation.

Since proxies tend to just be a wrapper for a TCP/UDP stream, the underlying HTTP traffic doesn't need to be changed and can just be passed straight through to the connection.
It'll also allow proxy implementations that only require an initial handshake (e.g. Socks5) and implementations that transform the entire stream (e.g. shadowsocks).

I do feel that if pooling ends up being included as part of this proposal, then the ability to specify an IConnection factory in the pool will be required as well. Otherwise the IConnections can not be set up in a required way

[VitaliyKurokhtin]

Does SendWithReservationAsync implicitly "return" connection to pool? Is there a scenario where client might want to send multiple requests over the same connection?

[scalablecory]

Does SendWithReservationAsync implicitly "return" connection to pool?

These APIs have no pooling at all; it would be the caller's responsibility to implement pooling if needed.

Is there a scenario where client might want to send multiple requests over the same connection?

Most HTTP traffic ends up sending multiple requests over the same connection. Either one-at-a time or (for HTTP/2) concurrently. This API allows for both.

[scalablecory]

Triage: we aren't confident there's enough time to perfect these APIs for .NET 5, and are investigating an experimental nuget package instead to help prove them out.

[JamesNK]

This feature would be useful for providing connectivity information in the gRPC client:

https://github.com/grpc/grpc/blob/master/doc/connectivity-semantics-and-api.md

[JamesNK]

Is there an event/callback to subscribe to connection state changes? That feature would enable creating a method like this:

Task WaitForStateChangedAsync(ChannelState lastObservedState, DateTime? deadline)

[JamesNK]

Versioning e.g. ALPN, H2C Upgrade, ALT-SVC upgrade.

What do you mean by versioning is not present? Can you give an example of a scenario limitation?

[scalablecory]

What do you mean by versioning is not present? Can you give an example of a scenario limitation?

These are constructed with, basically, a Stream. So it's up to you to connect a socket, establish SSL, check ALPN in the SSL handshake, etc. -- it requires a more in-depth knowledge of HTTP.

There's API to be figured out re: how you would perform an H2C upgrade or report out HTTP/2 ALT-SVC frames.