Runtime: Allow Disabling ClientWebSocketOptions Authorization Header Validations

[AustinMalpede]

Hello,

I'm running into some issues with the Authorization header that I'm attaching to the WebsocketConfiguration that I'm using. I'm essentially getting an error back that the header is malformed. Amazon SigV4 headers are not compliant with IETF standards, so there's a need to disable any sort of validation. It appears that in .NET, you can disable header validation for an HTTP Request [2]. I can't seem to find the same for the WebsocketConfiguration headers.

Here's an example of the header which I am trying to attach:

```c#
var webSocketConfiguration = new Action(options => {
options.SetRequestHeader("Authorization","AWS4-HMAC-SHA256 Credential= AKIAXXXXXXXXXXXYSZA /20190301/us-east-2/neptune-db/aws4_request, SignedHeaders=host;x-amz-date, Signature=b8155de54d9faab00000000000000000000000000a07e0d7dda49902e4d9202");
});

Attempting to initiate a websocket connection with the ClientWebSocketOptions shown above results in this error:

Unhandled Exception: System.Net.WebSockets.WebSocketException: Unable to connect to the remote server ---> System.FormatException: The format of value 'AWS4-HMAC-SHA256%20Credential=AKIAXXXXXXXXXXXYSZA/20190301/us-east-2/neptune-db/aws4_request,%20SignedHeaders=host;x-amz-date,%20Signature=b8155de54d9faab00000000000000000000000000a07e0d7dda49902e4d9202' is invalid.

at System.Net.Http.Headers.HttpHeaderParser.ParseValue(String value, Object storeValue, Int32& index)

at System.Net.Http.Headers.HttpHeaders.ParseAndAddValue(HeaderDescriptor descriptor, HeaderStoreItemInfo info, String value)

at System.Net.Http.Headers.HttpHeaders.Add(HeaderDescriptor descriptor, String value)

at System.Net.Http.Headers.HttpHeaders.Add(String name, String value)

at System.Net.WebSockets.WebSocketHandle.ConnectAsyncCore(Uri uri, CancellationToken cancellationToken, ClientWebSocketOptions options)
```

Is there any way a new ClientWebSocketOptions function could be exposed which does not enforce validating Authorization headers? I'm hoping for something akin to the HttpHeaders.TryAddWithoutValidation() function, but for websockets.

[rmkerr]

Thanks for bringing this up!

I think that it's probably not likely to meet the bar for new APIs. There are two reasons for that: (1) we haven't seen a strong demand for the change and (2) the fundamental issue here is not in .NET -- it's in the SigV4 headers.

(1) can always change if others run into this issue, and I think we can leave this issue open so that they can weight in.

[triggan]

The genesis of this issue appears to be changes made pre .NET 2.1. Before 2.1, WebSocketHandle.Managed.cs was not checking header formats. It was using StringBuilder() and just adding headers into the websocket upgrade request without any validation. Starting in 2.1, WebSocketHandle.Managed began checking header formats and doing validation. As people begin to upgrade to 2.1 and beyond, they'll continue to hit this issue.

My other issue here is that .NET gives you zero control over the upgrade HTTP request. Other programming languages/frameworks allow you to build your own HTTP request for upgrade to pass to a Websocket library. The current implementation of Websockets in .NET gives very little ability to customize this process to fit one's needs.

[davidsh]

@stephentoub

[theoneandonlyirvin]

@rmkerr I'm seeing the same issue. Even if the issue really lies with the SigV4 header, it seems that the ability to disable header verification should be consistent between typical HTTP/S requests and web sockets. This inconsistency alone should serve as justification for the capability being exposed for web sockets.

[AustinMalpede]

@stephentoub Thanks for taking care of this so quickly! Out of curiosity, is there any way that this fix could be backported to any 2.x LTS release?

Thanks,
Austin

[stephentoub]

Out of curiosity, is there any way that this fix could be backported to any 2.x LTS release?

@karelz will need to answer that. But generally the bar for what gets ported back to 2.x is really high (due to wanting to avoid risk that we could destabilize other things), and I somewhat doubt this would make it unless it started to impact a substantial number of developers.

[AustinMalpede]

@karelz The fix appears to be relatively benign. I would be very grateful if you would take it into consideration.

Thanks,
Austin

[tgourley01]

Wont there be a lot of users unable to use IAM auth on websockets?
(stuck on 2.0.x or waiting for 3.x?)
I vote for backport to 2.1.x to be picked up as LTS

[MatthewVines]

I'm one of those users. The sooner this is available the better.

[karelz]

Can you please summarize what kind of scenarios are blocked by this bug? How common are they?
(not sure what IAM auth is)
That will help us assess if we have enough justification for 2.1 backport. Thanks!

[AustinMalpede]

What is Gremlin.Net?

Gremlin is a graph traversal language and virtual machine developed by Apache TinkerPop of the Apache Software Foundation. Since it is open source it is maintained by nearly one hundred developers. It has multiple language variants, the one which we are using is Gremlin.Net.

What’s the issue?

.Net core 2.1 introduced a bug which is preventing us from deploying to production. Validations are now being performed against Authorization headers being attached to ClientWebsocketOptions. These validations fail when parsing AWS SigV4 Authorization headers. The graph database which we are using has IAM Authentication enabled which requires a valid AWS4-HMAC-SHA256 Authorization header be attached to the HTTP handshake, opening the websocket connection. If we had control over this HTTP request we would attach the Authorization header without performing validations. Sadly that is not an option in 2.1. We risk having our database exposed if Authentication is disabled so we’re determined to get this issue fixed as quickly as possible.

Is there a workaround?

Sadly, the answer to this is no. Stephen was quick to get this issue fixed in the upcoming .Net Core 3.0.0 release, however we are in standing water until then. That’s why we are pushing heavily for this fix to be backported to the latest 2.1.x LTS release.

Hopefully this answered any questions you have. Let me know if you need any more info.

Thanks,
Austin

[nberardi]

@karelz I would say that given the potential impact of this bug with no work arounds and the relative benign nature of this change according to the PR dotnet/corefx#35954. It would be great if you could back port this into .NET Core 2.1.

[karelz]

I agree that it looks like it may meet the bar for 2.1 servicing - reopening and marking for 2.1.x now. @wfurt can you please help with that? (PR + shiproom process)

Can someone fill me on how many production customers of Gremlin.NET are there? Any large companies?
Will all of them hit the problem? If not all, what percentage then?
(that kind of data will help us to get it through shiproom smoothly, thanks!)

[normj]

I can't help with the numbers but a bit more information for the need for this backport is in AWS .NET Core 2.1 is the latest version we support when it comes .NET serverless development. This is because Lambda only supports LTS version of a languages. So we expect a large usage of .NET Core 2.1 to continue for quite sometime and without this fix and there being no workaround the impression is given that .NET Core is a broken platform at least for the users of Gremelin.NET in a serverless environment.

[beebs-systap]

@karelz I am aware of at least one large multi-national company using Gremlin.Net and several large/medium sized companies. Without this change, there is not work around for customers to use Websockets with IAM Authentication.

@FlorianHockmann may be able to provide more detailed on the Gremlin.Net usage overall.

[FlorianHockmann]

Can someone fill me on how many production customers of Gremlin.NET are there? Any large companies?

I'm one of the maintainers of Apache TinkerPop and work mostly on Gremlin.Net which is the .NET variant of TinkerPop's Gremlin graph traversal language. Unfortunately we don't really have any insights into how many users Gremlin.Net has since anyone can just download and use it. The only source I have are the nuget.org statistics that show that the Gremlin.Net NuGet package has ~160k downloads with roughly 1,000 downloads per day (averaged over the last 6 weeks).

Will all of them hit the problem? If not all, what percentage then?

TinkerPop is the graph framework most graph databases support. This means that Gremlin.Net can be used to interact with databases like Amazon Neptune, JanusGraph, OrientDB, Azure Cosmos DB, or DSE Graph,
As far as I know, Amazon Neptune is the only TinkerPop enabled database that uses IAM Authentication. (Or can IAM Authentication also be used for other databases hosted on AWS?)
So not all Gremlin.Net users are affected, but I cannot say how many Gremlin.Net users are using it in combination with Amazon Neptune or on AWS in general.

That was of course specific to Gremlin.Net, but won't other WebSocket based application also be affected by this?

[karelz]

Thanks everyone for the information. The port was approved for May release (it was too late for April release :().
See details here: https://github.com/dotnet/corefx/pull/36130#issuecomment-474514580

@wfurt can you please create PR against release/2.2 as well? (I assume we need to port it manually) Thanks!

[stephentoub]

can you please create PR against release/2.2 as well? (I assume we need to port it manually)

Unless something's changed very recently, 2.1 flows into 2.2.

[karelz]

Ah, ok. If it is automatic, then we are good - @wtgodbe can you please confirm?

[wtgodbe]

The auto-merge from 2.1 -> 2.2 is still active, so we should be good to go

[karelz]

Great, thanks for confirmation @wtgodbe!

[AustinMalpede]

Thank you to everyone who helped make this happen. I really appreciate it!

[karelz]

Will ship as part of 2.1.11 release (see PR dotnet/corefx#36130)