Runtime: Reconsider allowing unsecure connections in .NET Core

[TroelsL]

I'm writing a C# wrapper around a very-much legacy SOAP/XML webservice.

The service uses a self-signed certificate that is expired. It also uses SSLv3, which, from what I can learn here and on the WCF teams github, is not allowed in corefx.

I fully understand the implications and reasoning behind this, but is there no way around this limitation? The company that made the device running the webservice stopped maintaining it years ago.

Will .NET Core not allow us to interact with such legacy devices? As I see it, being able to trust a self-signed certificate is no better than allowing SSL.

On a side note: My app works on Windows, but not on Ubuntu or Docker. Is that intentional?

[blowdart]

SSL 3 was considered insecure in 2014, and really really insecure, and should be taken out and buried in 2015. .NET Core deliberately excluded support for it, and SSL2. Given that the full .NET Framework can still allow teh use of such a deprecated protocol my only advice is to not use Core for this. It's not something we'd look at adding support for.

[TroelsL]

I have no strong objections to what you're writing, except that I'd love to be able to use .NET Core, being able to completely ignore certificate errors is permitted, and that I hate options being taken away from me.

But why does my app work on Windows though?

[blowdart]

What happens on linux? It's probably OpenSSL has even more strict default prohibitions on crypto.

[TroelsL]

I originally raised an issue on the WCF team github:

https://github.com/dotnet/wcf/issues/1749

My app is a svcutil (Connected Service) generated client that exposes a smart-home controller (web service) as a SignalR hub instead, solving both the SSL vulnerability (by wrapping it in HTTPS with TLS) as well as allowing client updates.

At the first call to the web service, I get a CurlException (full stack available in link):

Unhandled Exception: System.AggregateException: One or more errors occurred. (An error occurred
while sending the request.) ---> System.ServiceModel.CommunicationException: An error occurred
while sending the request. ---> System.Net.Http.HttpRequestException: An error occurred while
sending the request. ---> System.Net.Http.CurlException: SSL connect error at
System.Net.Http.CurlHandler.ThrowIfCURLEError(CURLcode error) at
System.Net.Http.CurlHandler.MultiAgent.FinishRequest(StrongToWeakReference'1 easyWrapper, CURLcode messageResult) --- End of inner exception stack trace ---

[blowdart]

@morganbr any ideas?

[stephentoub]

What happens on linux? It's probably OpenSSL has even more strict default prohibitions on crypto.

In .NET Core 1.0/1.1 CurlHandler was enforcing its own policy here, restricting this to TLS 1.x. In .NET Core 2.0+, CurlHandler just defers to the underlying system's defaults: https://github.com/dotnet/corefx/pull/13527
But libcurl versions since 7.39 also disable SSLv3 by default, so this won't change post that version.

But why does my app work on Windows though?

Probably due to https://github.com/dotnet/corefx/pull/13527#issuecomment-260741535?

[morganbr]

I think ServicePointManager lets you handle the self-signed, expired certificate. SSL3 is a tougher call. As @TroelsL points out, maybe explicitly opting into something insecure via ServicePointManager or other APIs should be OK, but I don't have insights into what that would take.

[mconnew]

@stephentoub, CurlHandler doesn't just defer to the underlying system defaults. See this commit that you made 😄 where SSL2 and SSL3 are removed from the list of defaults so that even if libcurl on the system allowed it, it still can't be used.
@blowdart, preventing users from very explicitly enabling SSL3 to communicate with an embedded device which there is no option to improve the security of is a bad idea. We are effectively doing a DOS on the device as we're denying the ability to communicate with it. If you are using SSL3, you might as well use plain unencrypted HTTP. We let users use unencrypted HTTP.

[stephentoub]

See this commit that you made

Yes, that's the PR I linked to in my comment. I forgot we kept that behavior via OpenSSL configuration. You can see a discussion of it in the linked issue.

[morganbr]

@mconnew, I wouldn't say there's a DOS attack! This is simply a question of whether .NET Core should support @TroelsL's legacy service.

@stephentoub, my security thoughts after a bit more research on this are that _at best_, we can only make SSL3 available via explicit opt-in as an insecure protocol. As far as precedent, Internet Explorer disables SSL3 by default, but users can opt in; however, Edge doesn't appear to allow it at all. The most difficult question is handling supportability -- the POODLE Attack and others are already known and more may come out over time. We'd be in a situation where we might feel obliged to service vulnerabilities that apps opted into.

I know this isn't quite a clear conclusion... to get to a 'yes', we'd need a clear plan on everything above (and be able to convince @blowdart and the rest of the security folks that it's a good idea).

[blowdart]

The cryptoboard were very happy with us disabling broken protocols. Given that there are other versions of .net that do support shooting yourself in the foot, I'm going to take serious convincing. And the introduction of a setting that makes it very clear, like .EnableBrokenCrypto()

[karelz]

Another angle to look at: How common is the scenario in the real world? How many customers have legacy services like @TroelsL?

[mconnew]

@blowdart, totally agree that if there is a change, calling it something very explicit like EnableBrokenCrypto() so there's no doubt that what is being done is at the users own risk. Maybe throw the word unsupported in there so there's no doubt about whether the scenario is supported or not. I.e. EnableUnsupportedBrokenCrypto().
Another possibility could be to create some mechanism to directly set curl options via the Dictionary HttpClientHandler.Properties. That way a customer could set the CURLOPT_SSLVERSION option and all we would need to do is get out of the way of the customers option (i.e. don't restrict the protocols passed to Interop.Ssl.SetProtocolOptions). This way we aren't enabling any broken crypto scenarios per se, just the ability to more directly control the curl options used, and if a customer chooses to use the mechanism to enable SSL3, that is then between them and libcurl and we're out of the loop.

[stephentoub]

Another possibility could be to create some mechanism to directly set curl options via the Dictionary HttpClientHandler.Properties

There's already the HttpClientHandler.SslProtocols property. Why wouldn't we just let that do what it's supposed to do? i.e. let the developer set it to something other than the default of None and we do our best to respect it.

[blowdart]

Because none was a compromise of a setting, because we can't rename it, and we need to be pushing people off broken crypto.

[stephentoub]

We're pushing people off broken crypto by disabling it by default. But if we're talking about adding another property that's NoReallyIWantTheseSslProtocols, then people will just learn to set that one instead, and we're not any more secure and have a worse API. If we're going to allow people to opt in by setting a property, we should just use the one we already have.

[blowdart]

Can we finally break backcompat and rename the None enum? :)

[morganbr]

@karelz the best data I can find is by looking at https://censys.io/blog/poodle , which found that as of October 2015, 0.12% of the top 1 Million sites only support SSLv3. Based on https://www.trustworthyinternet.org/ssl-pulse/ , support for SSLv3 on servers goes down a bit every month. Note that SSL3-only servers don't work with any popular browsers by default today.

[mconnew]

@morganbr, that's talking about HTTP use against a web server and not an appliance/device. A web server which can be reconfigured and modified is a whole different thing than a web service running on a device. It will often not be modifiable/configurable by the owner (as in can't change the code or modify the OS config) and when it goes out of support, there is no ability to "fix" it to use a more secure protocol. If we completely drop SSL3 with no way to override it, then we are turning those devices into expensive door stops.
The only difference between HTTP and HTTPS with SSL3 is one looks like it's secure if you don't know the entire picture. We still allow usage of HTTP, why not HTTPS with SSL3? If it's because SSL3 looks like it's secure, then find a way to make sure the customer knows it isn't. Require an environment variable called NETCORE_ENABLE_SSL3 to have the value "I am aware this provides no actual security and I have no other options", or add a property somewhere which delivers a similar kind of message. Completely disabling SSL3 while still letting HTTP be used just doesn't make sense to me. I understand having it disabled by default, but we should allow someone to turn it back on if they have a need.

[TroelsL]

Just to add another scenario I've encountered, this time professionally:

We are receiving data from several GSM-based devices (think IoT before that was a thing) that report status updates over HTTPS. The older versions of these use a firmware that does not support TLS. Some of these are located in hard-to-reach locations, so physically going there to update would be tremendously expensive. And while the data isn't sensitive, we do need it. Those devices needing a greater security level all use VPN and a private APN.

Currently, we are using .NET, but we would not be able to port the APIs these devices are calling to .NET Core. And with what you guys are doing right now, I'm very sure that eventually we will shift all new development to .NET Core.

So please, give us access to the gun that allows us to shoot ourselves in the foot. We'll sign the waiver.

[blowdart]

Yea, ok, convinced, with a suitable opt-in flag.

[morganbr]

Does anyone have a technical proposal on what this would look like? I think I could be convinced based on the specifics. My criteria would be:

1. A clear code or configuration change is required to enable SSLv3
2. SSLv3 doesn't get into any defaults. In particular, we need handling so that usage of SslProtocols.Default doesn't accidentally turn on SSLv3.
3. We need some idea of how to deal with the underlying OSes/libraries that may or may not support SSLv3.

[karelz]

Next step: We need formal API proposal. With sample. Motivation can be copied from above replies.

Anyone interested to grab it?

[los93sol]

Any updates on this? Just ran into this issue myself with a legacy device i need to communicate with that cannot be updated. I agree with other comments, let the SslProtocols property do its job and let me, as the developer, override it when i need it. I also agree with others, if youre taking the approach of lock it down then go ahead and drop HTTP support while you're at it you're at it...but seriously, please don't actually do that! Ironically, my application is designed to test security of devices and disabling this on me just makes me miss some of the data im collecting altogether...

[danmosemsft]

@los93sol the consensus above was to add API, and someone needs to write a proposal - you're welcome to give it a shot if you like.

[peturingi]

I found this issue when searching for a solution to this very problem. Turns out, I as well am trying to communicate with a legacy system.

[dan2dev]

We are migrating our whole platform to Dotnet and stepped in this issue. Do someone have a solution?

[creyke]

Please reconsider this. An explicit SSLv3 "opt-in" option is fundamental to enabling the development of new services with dependencies on old systems (often owned by other entities). This single issue is preventing our company from using only .NET Core, adding huge complexity to CI and CD when trying to support both .NET Core and, in this edge case, .NET Framework.

[danmosemsft]

@pmarcu

[stephentoub]

I still believe we already have the API, e.g. SslProtocols. We default to system defaults, and then if the dev overrides that by explicitly specifying something else, we respect that choice. We should just ensure we're doing that if we're not already.

[creyke]

Also, in the meantime, I still get the same error if I'm using plain old HTTP...

The service I want to consume is exposed over both HTTP and HTTPS... these work fine on my local box (64bit win10) over either HTTP or HTTPS and fails on 32bit Win10 in Azure Web Jobs (even though I've configured HTTP, not HTTPS). Can't therefore find any short term workaround :confused:

Works:

  OSDescription : Microsoft Windows 10.0.16299
  OSArchitecture : X64
  ProcessArchitecture : X64
  FrameworkDescription : .NET Core 4.6.26020.03

Fails:

  OSDescription : Microsoft Windows 10.0.14393 
  OSArchitecture : X64
  ProcessArchitecture : X86
  FrameworkDescription : .NET Core 4.6.26212.01

Error:

An error occurred while sending the request.
at System.Runtime.AsyncResult.End[TAsyncResult](IAsyncResult result)
at System.ServiceModel.Channels.ServiceChannel.SendAsyncResult.End(SendAsyncResult result)
at System.ServiceModel.Channels.ServiceChannel.EndCall(String action, Object[] outs, IAsyncResult result)
at System.ServiceModel.Channels.ServiceChannelProxy.TaskCreator.<>c__DisplayClass1_0.<CreateGenericTask>b__0(IAsyncResult asyncResult)

[karelz]

@creyke I don't understand your last comment about HTTP and same error. How is any error over HTTP related to support of SSLv3?
Also, where did you get .NET 4.6.26* versions from? That looks like full .NET Framework version, not .NET Core.

[creyke]

@karelz Those versions are what the RuntimeDescription.FrameworkDescription returns on .NET Core... there’s no .NET Framework in sight.

It appears when I define an endpoint over HTTP the WCF ServiceClient internally attempts to “upgrade” the protocol to HTTPS regardless... at least, I get the same error as I did over HTTPS. Extremely bizarre.

[karelz]

@creyke do you mean RuntimeInformation.FrameworkDescription?

Regarding WCF upgrade to HTTPS - it would be best to ask about it on wcf repo if there is a workaround / setting / something to avoid that. It seems rather remotely related to this issue (in the end-to-end sense, but not in the technical decisions here).

[karelz]

OK, I confirmed that .NET Core 2.0 returns .NET Core 4.6.26020.03 in RuntimeInformation.FrameworkDescription.

[mconnew]

@creyke, I can assure you that WCF does NOT change an http address into an https address under any circumstances for a client. Look at the inner exception, we always preserve the original exception thrown by any framework code such as HttpClientHandler.

[los93sol]

Is it possible that the unexpected change from http to https is caused by HSTS?

[karelz]

Looks like we are still strict on allowing only certain protocols:
https://github.com/dotnet/corefx/blob/1ae37add7583acebfa21bbc2074ab627a2f765a7/src/Common/src/System/Net/SecurityProtocol.cs#L20-L26

Not sure if SocketsHttpHandler enforces that as well, I see calls only from WinHttp and Curl handlers: https://github.com/dotnet/corefx/search?utf8=%E2%9C%93&q=ThrowOnNotAllowed&type=