Runtime: Support System.Security.Cryptography.Native OpenSSL 1.1.1 release

[bartonjs]

OpenSSL 1.1 has a new model of opaque structures with more accessor functions.

Anything previously obtained by an accessor function will likely be fine, but direct field accesses won't work.

For example, UpRefEvpPkey will need to change to EVP_PKEY_up_ref, but needs to do the current behavior on 1.0.x builds.

[tmds]

@bartonjs will this break the portable linux-x64 apps? Or will they work on systems with openssl 1.1?

[omajid]

OpenSSL 1.1 provides libssl.so.1.1. The shim currently looks for libssl.so.1.0.0 only.

[bartonjs]

@tmds Even the "portable" version is compiling against the 1.0.x ABI. libcrypto.so.1.0.0 and libssl.1.0.0 will still be required to be loadable (given the way things currently are. PRs can, of course, always change that)

[andreschort]

Any chance this makes it to the 2.0 Preview release?

[bartonjs]

@andreschort Nope, no one has been looking at it... we're still on 1.0.2.

[optlink]

As a work around, where can I specify the necessary cmake options to point at a different openssl location? For example, I have these options:

-DOPENSSL_INCLUDE_DIR=/usr/include/openssl-1.0 -DOPENSSL_SSL_LIBRARY=/usr/lib/openssl-1.0/libssl.so -DOPENSSL_CRYPTO_LIBRARY=/usr/lib/openssl-1.0/libcrypto.so

I've tried putting them as __CMakeExtraArgs in src/Native/build-native.sh but this seems to have no effect on where cmake looks.

[bartonjs]

The CMake instructions come from https://github.com/dotnet/corefx/blob/master/src/Native/Unix/System.Security.Cryptography.Native/CMakeLists.txt

I don't know that the -D options will help, because pkg-config is probably finding your 1.1 build, and I think pkg-config wins.

If you can list the package files for however you got OpenSSL 1.0.x on a system that has 1.1, look for something like openssl.pc, though it'll probably be a variant akin to openssl-10.pc. Your easiest workaround is to local edit line 15 to use whatever your 1.0 package name is.

Or comment out line 15 and set the variables it would have.

[optlink]

Thanks for the help, we've got it working by passing in some parameters to build-native.sh as seen here: https://aur.archlinux.org/packages/dotnet

[bartonjs]

@optlink Cool, glad to hear it. I clearly misremembered from the last time I had to try to figure out what find_packages did (while figuring out why it didn't do what I wanted it to). :smile:.

[sonix07]

+1 for openssl 1.1 native support

[TheBuzzSaw]

My Arch Linux dev environment just broke, and I'm pretty sure this is the cause.

[optlink]

@bartonjs Are there any other things I should be checking? It seems most dotnet commands involving network access fail without giving any useful error information. For example, dotnet restore runs and returns happily but doesn't actually do anything. I've tried linking to a version of libcurl that was built with openssl 1.0 but have had no luck with this.

[bartonjs]

@optlink If it's not failing catastrophically saying that it can't load System.Security.Cryptography.Native.OpenSsl, it shouldn't be an OpenSSL-related problem. I assume you're building all of the shims locally... do the unit tests pass?

corefx$ ./build.sh
...
corefx$ ./build-tests.sh
...

[optlink]

I can't run the unit tests:
Installing dotnet cli... Restoring BuildTools version 1.0.26-prerelease-01121-01... Failed to initialize CoreCLR, HRESULT: 0x80131500 ERROR: Could not restore build tools correctly. See '/home/optlink/dotnet/src/corefx-1.1.1/init-tools.log' for more details.1 Initializing BuildTools... /home/optlink/dotnet/src/corefx-1.1.1/init-tools.sh: line 121: /home/optlink/dotnet/src/corefx-1.1.1/packages/Microsoft.DotNet.BuildTools/1.0.26-prerelease-01121-01/lib/init-tools.sh: No such file or directory ERROR: An error occured when trying to initialize the tools. Please check '/home/optlink/dotnet/src/corefx-1.1.1/init-tools.log' for more details.1

[optlink]

We're attempting to work out the Arch Linux problems here: https://bbs.archlinux.org/viewtopic.php?pid=1708479

We've got openssl 1.0 linking but running anything network related from the CLI causes segfaults. Since curl links to openssl 1.1 and is also used by this project we are trying to link to a version of it built with openssl 1.0. We cannot get cmake to find and link the correct version of curl.

[bartonjs]

@optlink Ooooh, so not "network" (sockets / NetworkStream), but "HTTP". That's pretty different, in terms of what code runs.

The best way to bootstrap might be to build libcurl with some other provider (libcurl+nss, perhaps?). That'll prevent us from trying to talk to it, since we'll know the pointers are nonsense. (Well, because it'll respond to the OpenSSL callback option with CURLE_NOT_BUILT_IN).

Apparently we're going to have to plumb in sniffing on the TLS backend/version, so we can avoid this state.

[bartonjs]

I created dotnet/corefx#19306 for the libcurl+openssl11 problem.

[Kimi-Arthur]

Just a friendly ping. Will this get resolved for dotnet core 2.0 final release?

In the end, it makes more sense to support the latest version of base libraries like openssl (or work around them if possible)

Thanks for all the effort!

[bartonjs]

@Kimi-Arthur No, 2.0 hit feature-complete a while ago and this effectively isn't started.

• Some of the distros we're targeting still have 1.0.2 and don't have (and might not ever get) 1.1.0... and the distros that have 1.1.0 almost all have a compat package to bring down 1.0.2.
• While macOS has the ability to get OpenSSL 1.0.2, it's not super friendly, so there we did a different version of this: Switch to Security.framework.
• Not all Linux distros have the same #define values when they build OpenSSL, and so we used to have to build an interface library for every (major version of every) distro. We did work to be more flexible around that, and now it's easier to get .NET Core running on other distros.
• A whole lot of API was added to .NET Core that already exited in .NET Framework
• Windows users got the ability to load a PFX without writing the private key to disk (EphemeralKeySet)
• All platforms got the ability to create PKCS#10 Certificate Signing Requests (CertificateRequest)
• All platforms got the ability to programmatically create X.509 Public Key Certificates (via CertificateRequest)
• Areas other than cryptography also had busy schedules :smile:.

Since we didn't have a forcing function off of OpenSSL 1.0.2 the balancing act said that we could defer it while improving other areas. We're happy to take contributions to speed up the process.

[Drawaes]

How would the shim work in your mind for 1.1 and 1.02 at the same time, do you imagine it would check for 1.1 and if it doesn't find it fall back. Would this be done at the C++ shim level?

[bartonjs]

How would the shim work in your mind for 1.1 and 1.02 at the same time ... ?

I don't expect it would. I think 1.1 would be "I built locally / my distro built locally" and 1.0.2 would be used by "portable Linux", until we dropped 1.0.2 altogether.

[bartonjs]

To clarify a bit:

https://github.com/dotnet/corefx/blob/85e260e605f7e689ad1a11843ac2dc7d37a0eeb6/src/Native/Unix/System.Security.Cryptography.Native/openssl.cpp#L100-L108:

extern "C" ASN1_TIME* CryptoNative_GetX509NotBefore(X509* x509)
{
    if (x509 && x509->cert_info && x509->cert_info->validity)
    {
        return x509->cert_info->validity->notBefore;
    }

    return NULL;
}

=>

```cpp
extern "C" const ASN1_TIME* CryptoNative_GetX509NotBefore(X509* x509)
{

if OPENSSL_11 // or whatever

if (x509 != nullptr)
{
    return X509_get0_notBefore(x509);
}

else

if (x509 && x509->cert_info && x509->cert_info->validity)
{
    return x509->cert_info->validity->notBefore;
}

endif

return NULL;

}

[Drawaes]

Gotcha, if def 👍 :)

[Drawaes]

I am trying to add a function I need to the OpenSsl BIO, not sure I am getting what I need to do. I added

PER_FUNCTION_BLOCK(CRYPTO_set_ex_data, true) \

and
#define CRYPTO_set_data_ex CRYPTO_set_data_ex_ptr

but when I build I get

reinterpret_cast from 'void *' to 'decltype(CRYPTO_set_ex_data)' (aka 'int
  (CRYPTO_EX_DATA *, int, void *)') is not allowed
FOR_ALL_OPENSSL_FUNCTIONS

I am not sure I understand the procedure or am missing something to add a function?

[am11]

@Drawaes, I think we would need to patch https://github.com/dotnet/corefx/blob/9b4112d/src/Native/Unix/System.Security.Cryptography.Native/opensslshim.cpp#L46-L49 to use explicit / C-style cast:

-    // Get pointers to all the ICU functions that are needed
#define PER_FUNCTION_BLOCK(fn, isRequired) \
-    fn##_ptr = reinterpret_cast<decltype(fn)>(dlsym(libssl, #fn)); \

+    // Get pointers to all the OpenSSL functions that are needed
#define PER_FUNCTION_BLOCK(fn, isRequired) \
+   fn##_ptr = (decltype(fn)*)dlsym(libssl, #fn); \

(also line 11 has the ICU typo)

See http://winladen.tumblr.com/post/5244813131/staticcast-and-reinterpretcast-from-and-to-void and (shamelessly) dotnet/coreclr#12556 :)

[bartonjs]

@janvorli Do you know why the reinterpret_cast would be failing here? (and do you have an opinion on the shape of the fix?)

[janvorli]

@bartonjs let me take a quick look

[janvorli]

@bartonjs, @Drawaes - I have tried to add the function to the list of functions in opensslshim.h on my Ubuntu 14.04 as follows and it compiles fine:

--- a/src/Native/Unix/System.Security.Cryptography.Native/opensslshim.h
+++ b/src/Native/Unix/System.Security.Cryptography.Native/opensslshim.h
@@ -333,6 +333,7 @@ int EC_POINT_set_affine_coordinates_GF2m(const EC_GROUP *group, EC_POINT *p,
     PER_FUNCTION_BLOCK(X509_verify_cert, true) \
     PER_FUNCTION_BLOCK(X509_verify_cert_error_string, true) \
     PER_FUNCTION_BLOCK(X509_VERIFY_PARAM_set_time, true) \
+    PER_FUNCTION_BLOCK(CRYPTO_set_ex_data, true) \
     PER_FUNCTION_BLOCK(EC_GF2m_simple_method, false) \
     PER_FUNCTION_BLOCK(EC_GROUP_get_curve_GF2m, false) \
     PER_FUNCTION_BLOCK(EC_GROUP_set_curve_GF2m, false) \
@@ -623,6 +624,7 @@ FOR_ALL_OPENSSL_FUNCTIONS
 #define X509_verify_cert X509_verify_cert_ptr
 #define X509_verify_cert_error_string X509_verify_cert_error_string_ptr
 #define X509_VERIFY_PARAM_set_time X509_VERIFY_PARAM_set_time_ptr
+#define CRYPTO_set_data_ex CRYPTO_set_data_ex_ptr
 #define EC_GF2m_simple_method EC_GF2m_simple_method_ptr
 #define EC_GROUP_get_curve_GF2m EC_GROUP_get_curve_GF2m_ptr
 #define EC_GROUP_set_curve_GF2m EC_GROUP_set_curve_GF2m_ptr

I have tried to compile it with both clang 3.5 and clang 3.9. So I wonder what's different on your box.

[Drawaes]

Not sure it's a Ubuntu 14.04 clean vm running on windows 10. Just installed last night. Dotnet preview 2 installed (with dependencies) and then the dependencies required to build corefx from the website. Build.sh and build-native.sh both work fine until I add that function

[janvorli]

@Drawaes would you mind sharing the whole build log for the corefx src/build-native.sh using gist? Please delete the whole bin folder before that so that the log is complete.

[Drawaes]

Scratch that first one, I made a typo, I have typed that line a few times.. here is the "correct" broken one.

https://gist.github.com/Drawaes/7b2011e7b9eda9c6adea4e4fdac0c9b4

[Drawaes]

Moved to dotnet/corefx#22485

Seems the problem only occurs when running the build-native.sh from the root, not really a OpenSsl 1.1 problem :)

[mellinoe]

I don't expect it would. I think 1.1 would be "I built locally / my distro built locally" and 1.0.2 would be used by "portable Linux", until we dropped 1.0.2 altogether.

Any reason not to try to support both? I wouldn't be surprised if, in the future, the "old stable" distros and the "bleeding edge" distros support different versions, and we might want to support both of them with a portable binary.

[Drawaes]

Yeah 1.02 isn't going anywhere for a little bit

Version 1.1.0 will be supported until 2018-08-31.
Version 1.0.2 will be supported until 2019-12-31 (LTS).

So about 18 months of life left kicking in it... after that it should be dropped like a hot potato, no good reason to support it after that.

[Drawaes]

To be fair it's going to be a bit nasty, with the current "PAL" layer for openssl from my limited understanding it just straight maps functions from different Dlls, for 1.1 it is actual interface changes (for the better but still) I guess the mapping could be done in the same place with function pointers and to make the PAL more substantial, could also be a maintenance and testing nightmare to have all that code "hidden" in c/c++ behind the scenes.

[bartonjs]

@mellinoe

Any reason not to try to support both?

We can't compile for both. We'd have to make the build process be something like

• Build for 1.0.2 on system A
• Build for 1.1.0 on system B
• Merge the binaries, and make some kind of magic switching logic.

That would be even more magical than the portable assets currently are.

OpenSSL 1.1 has a massive number of breaking changes, don't let the fact that they only bumped the minor version fool you :smile:. The changes seem to largely be mechanical, but incompatible.

[Drawaes]

:) Imagine MS getting away with numbers like this for a "minor" version bump

63.4% | 760 new | 433 removed
https://abi-laboratory.pro/tracker/objects_report/openssl/1.0.2l/1.1.0/report.html

[mellinoe]

• Build for 1.0.2 on system A
• Build for 1.1.0 on system B
• Merge the binaries, and make some kind of magic switching logic.

@bartonjs I'm not suggesting it would be easy or trivial, but there doesn't need to be anything magical about it. We already do dynamic library and function loading -- we could extend it in various ways. We could also try something like build two native shims, and then select one or the other at runtime from managed code.

[bartonjs]

@mellinoe That sounds more magical than I'd like core security/crypto code to be. If business needs force us to do it we can; but until then I really want to keep it simple. It should always be clear what version is being loaded since our API exposes pointers out, and accepts pointers in. Once we have a heuristic it gets a lot easier for getting library mismatches leading to memory corruption.

[mellinoe]

Fair enough -- I guess I'm just not a fan of the "magical" description. From a simplified view, it would just involve two separate supported backends, which happen to both be usable from the same OS, and which happen to use different versions of a native lib. A bit hand-wavy, to be fair...

If business needs force us to do it we can; but until then I really want to keep it simple.

ifdef'ing seems like the right way to do it for the short-term. I brought this up because the cynic in me is already reading the complaints from folks whose Linux distros still only support 1.0 (which I guess still has official support for a little while).

[Kimi-Arthur]

Even if we/you don't do it magically, as long as there is a version supporting/using openssl 1.1.0, it would be great! Is it already there yet?

Distros like Arch can pass in different flags in their build config files (PKGBUILD) and choose the correct one for them. They already do that for a lot of packages I believe.

[ShalokShalom]

For when is OpenSSL 1.1 support planned?

[zero-master]

Everything got installed just fine until nuget mysteriously failed with these errors and I found this thread:

An error occurred while sending the request Error: SecureChannelFailure (The authentication or decryption has failed.) The authentication or decryption has failed. Error while sending TLS Alert (Fatal:InternalError): System.IO.IOException: Unable to write data to the transport connection: Cannot access a disposed object. Object name: 'System.Net.Sockets.Socket'.. ---> System.ObjectDisposedException: Cannot access a disposed object. Object name: 'System.Net.Sockets.Socket'.

[ShalokShalom]

@zero-master Which OpenSSL version do you use?

[zero-master]

@ShalokShalom

Here is the output of openssl version
OpenSSL 1.1.0g 2 Nov 2017

uname -a
Linux 4.13.12-1-ARCH dotnet/corefx#1 SMP PREEMPT Wed Nov 8 11:54:06 CET 2017 x86_64 GNU/Linux

[janvorli]

@zero-master there is also an openssl-1.0 package for ARCH Linux that you can install side by side with the openssl 1.1 (https://www.archlinux.org/packages/core/x86_64/openssl-1.0/). That should fix the issue for you.

[ghost]

Some folks have called it obnoxious, some are suggesting OpenSSL 1.1 should've been called OpenSSL 2.0, so generally not everyone is happy the way 1.0->1.1 transition went down: https://news.ycombinator.com/item?id=13284648. But this has become a fact of life that needs to be supported (and imo revisit the no-libressl decision at the same time as that one is saner).

While building CoreFX..

In the default package repository of Ubuntu 18.04 LTS, there is:

• curl and cmake that depend on libcurl4
• libcurl4-openssl-dev that depends on libssl1.1-dev
• libcurl-openssl1.0-dev that depends on libssl1.0-dev and libcurl3

Installing libcurl-openssl1.0-dev and libssl1.0-dev and removes cmake and curl.
Installing libcurl-openssl-dev installs libssl1.1 and removes libssl1.0.

To get it all working, I had to use cmake portable binary (https://cmake.org/files/v3.11/cmake-3.11.0-Linux-x86_64.tar.gz, extract and add to PATH) and wget instead of curl to satisfy openssl1.0 combination: libcurl-openssl1.0-dev, libcurl3, libssl1.0-dev.

[ShalokShalom]

This is the reason, why .Net does not build on my distribution since 2 years.

[bartonjs]

@kasper3 Ubuntu 18.04 had curl backed by OpenSSL 1.0 until fairly recently (when I checked the package index on around March 1st it still ended up at 1.0 for Bionic; their changelog says 21 Feb, so there must have been a delay in that version showing up on the feed).

Theoretically apt-get install --nodeps libssl1.0-dev libcurl-openssl1.0-dev would put you in a buildable state.

[ShalokShalom]

Such issues can get solved elegantly by Linuxbrew.

This is currently my favorite generic package manager for GNU/Linux and it lets you provide always the current release for all the distros.

They use so-called formulas, which you write in order to get it working are very descriptive and concise.

They install the packages into a separate directory, so nothing collides with the existing distro packages. :slightly_smiling_face:

There are a lot of such solutions: AppImages (works out of the box, if you ship it executable, without any client) and Habitat (which happens to ship to all three major platforms and is quite sophisticated since they include nearly all info about the package into it) as well as the more popular choices such as Flatpak and Snap and less well known stuff such as pkgsrc and nix, which are complicated.

I figured them all out and created a comparison sheet about it.

There are plenty of options, so is there imho no reason at all to let such issues hinder distros to build and to stay forced to 'ship' such outdated versions of CoreFX itself, since the majority of Linux distributions build slowly.

To be precise: Currently is the situation such, that you dont have the deployment to Linux in your hand, which can choose with one of the tools.

And to be precise once again: This is ontopic, since my distribution is on the current OpenSSL version since its release and as said by someone else:

There will always be the some distros, who are lacking behind and newer once who get hindered by that. Which is the reason why these tools exist. ^-^

Happy hacking

[joshfree]

Moving this to the 3.0 milestone as OpenSSL 1.1.1+ is required for TLS 1.3 support.

[omajid]

There's a PR here: https://github.com/dotnet/corefx/pull/30807

[joshfree]

https://tools.ietf.org/html/rfc8446

[omajid]

Can this issue be closed now?

[bartonjs]

Can this issue be closed now?

Yep. Thanks for pointing it out. ::whistles innocently::