Runtime: WinHttpHandler perf optimization for ClientCertificates, ServerCertificateValidationCallback

[mark-cordell]

In a larger server app, there may be scenarios where different parts of the code need to use different client certificates on a per-request basis. Same with ServerCertificateValidationCallback. The native WinHttp library lets you efficiently pool connections across a session, and set client certificates and perform custom certificate validation on a per-request basis within that session, but the managed library forces you to share these settings across all requests in a session. People often use ServerCertificateValidationCallback as an instance delegate on some object that has a much shorter lifetime than the session.

It would be good if the managed library just let you set these on a per-request basis. It's really hard to get good perf otherwise.

[mark-cordell]

I created a pull request as an example of this could be fixed here:
dotnet/corefx#28842

[davidsh]

Thank you for opening this issue. I think this idea of having flexibility in the ServerCertificateValidationCallback on a per-request basis, but keeping connections pooled per handler is interesting.

We might want to consider doing this as part of the new SocketsHttpHandler, though. In addition, it might be best to view these API changes directly in HttpRequestMessage as well, instead of creating a derived class. SocketsHttpHandler is meant to be the new, platform-independent HTTP stack for .NET Core. It would be best to focus on that handler since it is planned to be the default HTTP handler for .NET Core 2.1.

cc: @geoffkizer @stephentoub @karelz

[stephentoub]

The native WinHttp library lets you efficiently pool connections across a session, and set client certificates and perform custom certificate validation on a per-request basis within that session

Can you elaborate on the behavior of this? What does it mean to pool connections used by requests with different validation callbacks, which are really about establishing the connection? Do you mean it uses different pools for each callback? And if so, since the main resource maintained by an instance of a SocketsHttpHandler is the connection pool, would it not be simpler to just have a different SocketsHttpHandler instance for each distinct callback, which is what the API already supports?

[mark-cordell]

@davidsh adding these fields to HttpRequestMessage would be great if that works.

@stephentoub when you create a new http request, you don't know if it's going to get one out of the pool or establish a new connection. I'm assuming here that consumers will typically want to the same validation callback logic for a given host/scheme/port destination, so they just always specify host/scheme/port + validation callback for each request, and everything works fine regardless of whether it establishes a new connection or gets one out of the pool. The problem comes when you want to make requests to multiple host/port/scheme combinations from one process, each with potentially its own validation callback logic. Now you have to maintain your own thread safe dictionary, which seems like an unnecessary overhead when the native winhttp library already takes care of that for you.

[stephentoub]

which seems like an unnecessary overhead when the native winhttp library already takes care of that for you.

For WinHTTP, maybe. But that's not a feature/complication of SocketsHttpHandler.

[karelz]

How common is the pattern to have special ClientCertificate / ServerCertificateValidationCallback per request?
Can you help me understand the scenario better? How often will the values of ClientCertificate / ServerCertificateValidationCallback change? Is it periodic update - providing smooth transition, or is it about multiplexing several values all the time?
How much perf wins does the WinHTTP optimization provide over creating new WinHTTP handler per each unique ClientCertificate/ServerCertificateValidationCallback setting?

To add new API, we should have good reason for it - user demand & value for users (i.e. perf wins).

[mark-cordell]

People who use client certificates will likely use different certificates when calling different hosts. Perhaps some will use different client certificates when calling different APIs on the same host. A large server app is likely making multiple kinds of requests to multiple hosts in parallel as part of its processing. Creating new WinHttpHandlers per unique ClientCertificate/ServerCertificateValidationCallback is not easily done because delegates are often tied to a specific object instance, so it's likely not easy to tell which ones are truly "unique".

People are likely to think that WinHttpHandler is a wrapper for the native WinHttp library. It would be disappointing to have a native library that gets its API right (client certificates and other properties can be set per request) and the same company (Microsoft) provides a managed wrapper for its own library and doesn't get the API right.

Asking people to come up with workarounds the limitations in the managed wrapper is going to be annoying for them, and it's easy to have errors with these kinds of implementations. The full .NET Framework version of HttpWebRequest even has issues with its locking implementation around connection management.

[blowdart]

Consider this like basic auth. You wouldn't share basic auth credentials across multiple servers, it'd be cached on a per server endpoint or set on a per request basic. Client certs would be the same. As would any type of authorization header.

[mjsabby]

@karelz This is basically for parity with an existing workload running just fine on HttpWebRequest.

It's for Bing.com's migration to .NET Core.

We use HttpWebRequest, which as well know is fatally flawed on .NET Core because it creates a new HttpClient per web request.

To remedy that, we decided to go with the WinHttpHandler and realized that this server validation callback is missing the per-request basis we need.

The native api does support the behavior we would like and it's somewhat unfortunate that the only reason why we can't access it from managed code is a missing API.

Is there any reason to not accept a PR here beyond the API-break?

This will be the 3rd assembly we fork because of a compat break or missing functionality.

[karelz]

HttpClient/HttpClientHandler programming model is different than HttpWebRequest/ServicePointManager.

• ServicePointManager is basically one giant static variable. Configuring ClientCertificates or ServerCertificateValidationCallback per server requires per-request settings. Which in implementation leads to mapping combination of endpoint & "interesting settings" to a pool of reusable connections.
  
  
  
  • Note that the "per-request" is implementation detail as it is the only way to express settings other than globally.
  
  
• HttpClientHandler basically represents a set of (transportation layer) settings and maintains a connection pool per endpoint.
  
  
  
  • It assumes developer will create new HttpClient/HttpClientHandler per each set of settings.
  
  

If there is need to customize ClientCertificates/ServerCertificateValidationCallback per endpoint, then it should be fairly straightforward by creating new instance per endpoint (or group of endpoints). Maybe even going as far as wrapping it in MultiplexingClientHandler which hides the fact there are multiple HttpClientHandler instances under the hood, each configured per specific endpoint. To address just ServerCertificateValidationCallback, one could create a MultiplexingServerCertificateValidationCallback which has per-endpoint setup.
Are there other scenarios? (e.g. using ClientCertificates to "impersonate users"?)

If some of the scenarios are common, we should think about ways to built them into HttpClient/HttpClientHandler (not just into WinHttpHandler).

Adding the APIs to WinHttpHandler, just because the underlying WinHTTP APIs happen to expose it that way does not seem right. The only potential advantage I can imagine is perf -- in which case we should see from a prototype perf comparison + convince ourselves such scenarios are common enough to add it to the API.

@mark-cordell WinHttpHandler is not managed wrapper for WinHTTP. It is HttpClientHandler programming model built on top of WinHTTP. The goal was never to do 1:1 mapping with WinHTTP. The idea is that WinHTTP is just an implementation detail.

@mjsabby HttpWebRequest is slow, compat-only API. If we see that beefing it up a bit (e.g. by sharing 1 static HttpClientHandler per "settings") is going to help people migrate, we can definitely consider it. It will never be 100% compatible for all corner-case scenarios (esp. not perf-wise).

To remedy that, we decided to go with the WinHttpHandler and realized that this server validation callback is missing the per-request basis we need.
The native api does support the behavior we would like and it's somewhat unfortunate that the only reason why we can't access it from managed code is a missing API.
Is there any reason to not accept a PR here beyond the API-break?

Can you please describe what is your scenario for "per-request" settings? Do you truly need them per-request, or just per-endpoint? Or are there other scenarios/patterns you use?
If there are good reasons to expose APIs in HttpClientHandler/WinHttpHandler, we can definitely consider changes. I would just rather do that when we understand the motivation scenarios (e.g. the "per-endpoint" settings mentioned above) and how common they are.
From long-term perspective, WinHttpHandler is just implementation detail. I would like us to think in general about HttpClientHandler (incl. SocketsHttpHandler) first rather than hyper-focus on WinHttpHandler.

[mjsabby]

@karelz ServerCertificateValidationCallback is a per-request thing.

I'm not going to argue with you on the history of how HttpWebRequest came to acquire this great property of having the ServerCertificateValidationCallback per-request. Maybe it was the design constraint of a single static variable that gave us this feature or maybe it was intentional.

The fact that WinHTTP also supports this makes me think it may have at least been a consideration.

Also, I understand you're saying it's a different API and different usage (HttpClient, etc.), but I'm pointing out that with these new APIs what was previously possible is no longer possible, i.e. you can't do the following:

For example foo.com:80/api?version=1&appid=5&partition=0 uses Scheme 1 of certificate validation, and foo.com:80/api?version=1&appid=6&partition=1 uses a different scheme. Both of these need to reuse the TCP connection for whatever reason. You may think it is only for performance, but there are also correctness reasons when you take into account data partitioning, sharding, etc. that requires machine affinity and the program is relying on the connection being the same.

On Desktop, HttpWebRequest will have a single TCP connection for this and be able to do different certificate validation and on .NET Core there is no way to do that.

WinHttpHandler is an implementation that we're using, so for us it's a hyper-focus :)

For all its warts and problems, HttpWebRequest solves at least one problem we have that other current solutions are unable to solve. SocketsHttpHandler is not out yet, as far as I can tell, but I would encourage us to have that capability in there as well.

[stephentoub]

HttpWebRequest will have a single TCP connection for this and be able to do different certificate validation

That's not true, is it? Won't there be different connections used?

[davidsh]

Won't there be different connections used?

Yes, this is not true. Different connections will be used.

On .NET Framework, HttpWebRequest creates separate ConnectionGroup objects (pools of TCP connections) even to the same endpoint (ServicePoint) depending on various critera. The criteria are based on authentication, validation callbacks, client certificates, etc.

Two HttpWebRequest's that use different ServerCertificateValidationCallbacks, for example, will end up having different ConnectionGroup's in any ServicePoint.

See:
https://github.com/Microsoft/referencesource/blob/b31308b03e8bd5bf779fb80fda71f31eb959fe0b/System/net/System/Net/HttpWebRequest.cs#L5366-L5407

[stephentoub]

Yes, this is not true.

Thanks for confirming, David.

SocketsHttpHandler is not out yet, as far as I can tell, but I would encourage us to have that capability in there as well.

From my perspective, this is not something we should be adding to SocketsHttpHandler. The pooling logic in the handler is currently simple, and thus fast. Adding in additional things that need to be checked/compared to determine what pool to use will end up being very much not pay-for-play, in that we'll end up incurring costs associated with pool maintenance and mapping even when that's not the common case. The fact that HttpWebRequest uses different requests for different delegates highlights that there's no savings to be had by having this in the handler implementation, at which point the caller may as well just have their own dictionary/mapping/whatever to the handler to be used; they can do that as if not more efficiently (given knowledge of the domain) than we can in the handler, and with minimal code.

[karelz]

This is not something we plan to do / take for WinHttpHandler, closing.
There is a workaround, which is applicable to all HttpClientHandlers. There is no evidence that WinHTTP can do anything more than that in lower layers anyway.