Runtime: Custom SNI (Server Name Indication) support in HttpClient

[alukyan]

I have a case where I need to establish HTTPS connection to an endpoint that may hosts several virtual HTTPS endpoints which name does not match the endpoint name and I need to use SNI to select a virtual server. https://en.wikipedia.org/wiki/Server_Name_Indication

Experiments with HttpClient show that Client Hello message contains the same name as the endpoint name and I cannot find a way to change it to a different virtual server name. Any suggestion on how to do this?

[davidsh]

Have you tried to add the "Host" request header manually which might be different from your DNS name you use in your http://servername.com uri?

[alukyan]

We tried setting Headers.Host to a different value, but when looking at the protocol exchange in WireShark we still see that Client Hello message contains the same URI that is passed in request.

SNI feature is really useful to talk to HTTPS service behind a proxy to connect to secure service inside private network.
And number of languages support it:

• Python - Connection.set_tlsext_host_name(name) - Specify the byte string to send as the server name in the client hello message.
• Golang -

type ClientHelloInfo struct {
        // ServerName indicates the name of the server requested by the client
        // in order to support virtual hosting. ServerName is only set if the
        // client is using SNI (see
        // http://tools.ietf.org/html/rfc4366#section-3.1).
        ServerName string

This TLS extension has been accepted as a standard quite some time ago (more than 10 years already), so I am wondering why .NET still does not support it.
This missing feature makes us terminate HTTPS at the proxy and then re-encrypt the traffic again - basically inserting our own "man-in-the-middle". Not good from security standpoint.

[geoffkizer]

I agree, we should support this.

@davidsh, are you saying that this should work? Is this just a bug?

@stephentoub FYI

[alukyan]

To clarify, .NET protocol exchange contains Client Hello message, but it is always set to the same URI as the request and HttpClient does not provide means to change it to custom value.

Current SNI implementation in .NET will work only if an endpoint has multiple public DNS records (classic website example), which is not always possible to do for REST API servers, especially with clusters where microservices are running behind application load balancers. Changing value in the Host header changes just a header, but with HTTPS all traffic, including headers is encrypted so this cannot be used for virtual host selection on a proxy unless HTTPS is terminated there.

Looking at implementation in Golang - they do not link the value set in Host header to the value set in Client Hello message, it can be configured independently

[davidsh]

There are no HttpClient nor HttpWebRequest APIs to specify an SNI name that is different from the DNS name. This is no support in either .NET Framework nor .NET Core for this in the HTTP stacks. This would API support as well as plumbing down to the transport layers (TLS/SSL) of the stacks in both Windows and *Nix.

And for SslStream class, we do not have SNI support either (although we have an active issue with requests for that feature).

[geoffkizer]

@alukyan, thanks for explaining in more detail.

[geoffkizer]

I do wonder whether we should be checking for a Host header and simply using this in the SNI message. It seems like this would make the common scenario work. I'm trying to imagine a scenario where you want the SNI hostname to be different than the Host header, and I'm not coming up with anything.

[Marcus-L]

This is needed for verifying domain names using TLS-SNI with Let's Encrypt (a free SSL certificate service). Per the draft (https://tools.ietf.org/html/draft-ietf-acme-acme-01#section-7.3), SNI entries with invalid (but known) hostnames are created on a web server, which are verified by a third party server.

Example, I want to verify that I own domain marcuslum.com, so:

1. I perform the steps to get a key from Lets Encrypt, then use this key to derive a dns name (i.e. "1234.5678.acme.invalid")
2. I create a self signed certificate with this domain as subject & subject alternate name
3. I install this certificate on my webserver and configure SNI so that the verifier can make a request to my server (via the IP address of marcuslum.com), but with SNI set to "1234.5678.acme.invalid"

As far as I can tell the only ways to do make the HTTPS call via HttpClient or HttpWebRequest in .NET are to:

1. temporarily write to the local hosts file (or to my dns server) to modify lookups for 1234.5678.acme.invalid to resolve to the same IP as marcuslum.com - requires local admin
2. hook into wsock32.dll's gethostbyname function as described here - not sure if this works, I didn't test it
3. use ServicePoint.BindIPEndPointDelegate with ServicePointManager as described here - but I think this only works if the SNI host has a valid DNS host entry

[davidsh]

We tried setting Headers.Host to a different value, but when looking at the protocol exchange in WireShark we still see that Client Hello message contains the same URI that is passed in request.

Now that we use a new HTTP stack (SocketsHttpHandler) as the default for .NET Core 2.1, SNI is now working. By default, the SNI will be the DNS name of the Uri used in the request. If a custom, "Host:" request header is added, then the SNI will use that name in the TLS/SSL handshake.

If there are further questions regarding this, please open up a new issue. Thx.

cc: @karelz