Runtime: OpenSslCryptographicException initialising an X509Certificate2 on Ubuntu 14.04 or official docker image microsoft/aspnetcore:1.1.0

Created on 10 Feb 2017 · 16Comments · Source: dotnet/runtime

I have a Web API running in a docker container. The container is built from the provided 1.1.0 package:

  FROM microsoft/aspnetcore:1.1.0

and the Web API binaries are copied in. The API runs fine and returns data as expected until I turn on authentication, at which point it needs an X509SecurityKey to set the TokenValidationParameters.IssuerSigningKey value. It throws an exception when it attempts to initialise an X509Certificate2 from a string value:

    string certValue = certificate.Value;
    byte[] byteCert = Encoding.ASCII.GetBytes(certValue);
    return new X509Certificate2(byteCert);

throws an OpenSslCryptographicException:

    Unhandled Exception: System.Exception: Failed to extract the Token Signing certificate from the Federation metadata. ---> 
    Interop+Crypto+OpenSslCryptographicException: error:0D07803A:asn1 encoding routines:ASN1_ITEM_EX_D2I:nested asn1 error
       at Internal.Cryptography.Pal.CertificatePal.FromBlob(Byte[] rawData, String password, X509KeyStorageFlags keyStorageFlags)
       at System.Security.Cryptography.X509Certificates.X509Certificate..ctor(Byte[] data)
       at Mercury.Shared.Rest.Authentication.AdfsFederationMetadata.GetSigningCertificate()

The string value from which the X509Certificate2 is being initialised is:

    MIIC4jCCAcqgAwIBAgIQHWt3kGySgJxPtsalC0EoKzANBgkqhkiG9w0BAQsFADAtMSswKQYDVQQDEyJBREZTIFNpZ25pbmcgLSBzdHMuYWxsYW5ncmF5LmNvLnphMB4XDTE2MDkwNzA5MDQyM1oXDTE3MDkwNzA5MDQyM1owLTErMCkGA1UEAxMiQURGUyBTaWduaW5nIC0gc3RzLmFsbGFuZ3JheS5jby56YTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBANdq9BEuBRPsTdpngeFyXbfH5lBg5WyENQW0qz2FtDw3AvZhiPdFyvTPZIeZDc4vhg+gPuG8pCxhFa6hPqNIwnLSVuyhUi4/CtZrLghF2wVVcyriijvirzdVp2m56nO31NB5HXbSerTmey1gJsgumr+MiaM2CEI9z5ctwAp66jqM9jVv7kzqIwB33irSck+X97jUa9XVa0/0QPBdrSVUR0i4rmfZ9orRdTKC3IA13bD9duk2Kc9V7t8t/woo80Kbbb3ZseYk5N8AI+7RRw9+oSAm8zZQzBYkNkAMeI1mto1faXsm9Aea4HXbyCbvVOx/JGj5Ki7YK/BtzWAyCgRu0TkCAwEAATANBgkqhkiG9w0BAQsFAAOCAQEAd9rdJ1V9te8njMHiEuvr1DLVUBt2YdpZ8qGFC2rWDKeEW8iARrMfbtrlQovvE1radxoZ2gWR1AaaoEqzmLvF9fEAZ7xjR2P2UlT+ZgntfMn/c+3j7gWnfGNHIjosaYiyF72q4k6bgOx1TV8X08kD2AMBH27utXeeUQTZTd0bUWaWpr76NrDB95k4P6d0t5bkYsguiQjV+2t5/dSvrbTPVbXQmWGyC53IS2OI37AI2bIaeCCDkRHDoxu+L/DtgH8N60k2CLfa+pf0/cxQCR39p4Z+tquVKfYgJIsdZLD6bbrqK9VdpSR2vyUcDLMTGnO0tuDuzBd/xdhJ0GKbnBv3+g==

The same code runs with no problem on Windows, building a certificate from the same string.

Edit: Note that while I initially encountered this problem running a docker image, subsequesnt testing has show that it also occurs using Ubuntu 14.04 + .NET Core 1.1

area-System.Security bug os-linux os-mac-os-x tenet-compatibility

Source

Ettery

👍2

Most helpful comment

I had a suggestion on my StackOverflow question to decode the string from Base64 when converting to the byte array, and that has solved the problem, it would seem that the Windows code will detect and decode base64 where openssl does not. Might be worth doing that in the openssl wrapper to bring the behaviour in line.

I changed my code from

var byteCert = Encoding.ASCII.GetBytes(certString);

var byteCert = Convert.FromBase64String(certString);`

before calling new X509Certificate2(byteCert) and it now works on Windows, Ubuntu and Docker.

Doesn't explain why loading from a text file doesn't work, but I'm very happy to have a solution!

Ettery on 16 Feb 2017

👍4 🎉1

All 16 comments

@Ettery If you save that to a file, like so

and run openssl x509 -in thatfileyousaved -text on the same system does it work, or do you get the same error?

It parsed fine for me on Ubuntu 16.04 (OpenSSL 1.0.2g), and I was able to load it into a .NET process.

bartonjs on 10 Feb 2017

Yes, it works - I'm not sure exactly what to expect, but it gives the output below and no errors. As you can see, this is running under MINGW64 on Windows 10; I will try it on my Ubuntu server today and let you know if I have the same problem. Do you have any other guidance?

I tried taking the BEGIN and END markers out (which is all you've added) I get the following error, which is not the same as I get loading that cert through .NET:

peterwo@PETERWO-W7 MINGW64 /c/dev/gitlab/mercury.docker/1.1/debian/core-api (master)
$ openssl x509 -in adfscert4.crt -text
unable to load certificate
16300:error:0906D06C:PEM routines:PEM_read_bio:no start line:pem_lib.c:707:Expecting: TRUSTED CERTIFICATE

Output from processing the file as you described:

peterwo@PETERWO-W7 MINGW64 /c/dev/gitlab/mercury.docker/1.1/debian/core-api (master)
$ openssl x509 -in adfscert.crt -text
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            1d:6b:77:90:6c:92:80:9c:4f:b6:c6:a5:0b:41:28:2b
    Signature Algorithm: sha256WithRSAEncryption
        Issuer: CN=ADFS Signing - sts.allangray.co.za
        Validity
            Not Before: Sep  7 09:04:23 2016 GMT
            Not After : Sep  7 09:04:23 2017 GMT
        Subject: CN=ADFS Signing - sts.allangray.co.za
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                Public-Key: (2048 bit)
                Modulus:
                    00:d7:6a:f4:11:2e:05:13:ec:4d:da:67:81:e1:72:
                    5d:b7:c7:e6:50:60:e5:6c:84:35:05:b4:ab:3d:85:
                    b4:3c:37:02:f6:61:88:f7:45:ca:f4:cf:64:87:99:
                    0d:ce:2f:86:0f:a0:3e:e1:bc:a4:2c:61:15:ae:a1:
                    3e:a3:48:c2:72:d2:56:ec:a1:52:2e:3f:0a:d6:6b:
                    2e:08:45:db:05:55:73:2a:e2:8a:3b:e2:af:37:55:
                    a7:69:b9:ea:73:b7:d4:d0:79:1d:76:d2:7a:b4:e6:
                    7b:2d:60:26:c8:2e:9a:bf:8c:89:a3:36:08:42:3d:
                    cf:97:2d:c0:0a:7a:ea:3a:8c:f6:35:6f:ee:4c:ea:
                    23:00:77:de:2a:d2:72:4f:97:f7:b8:d4:6b:d5:d5:
                    6b:4f:f4:40:f0:5d:ad:25:54:47:48:b8:ae:67:d9:
                    f6:8a:d1:75:32:82:dc:80:35:dd:b0:fd:76:e9:36:
                    29:cf:55:ee:df:2d:ff:0a:28:f3:42:9b:6d:bd:d9:
                    b1:e6:24:e4:df:00:23:ee:d1:47:0f:7e:a1:20:26:
                    f3:36:50:cc:16:24:36:40:0c:78:8d:66:b6:8d:5f:
                    69:7b:26:f4:07:9a:e0:75:db:c8:26:ef:54:ec:7f:
                    24:68:f9:2a:2e:d8:2b:f0:6d:cd:60:32:0a:04:6e:
                    d1:39
                Exponent: 65537 (0x10001)
    Signature Algorithm: sha256WithRSAEncryption
         77:da:dd:27:55:7d:b5:ef:27:8c:c1:e2:12:eb:eb:d4:32:d5:
         50:1b:76:61:da:59:f2:a1:85:0b:6a:d6:0c:a7:84:5b:c8:80:
         46:b3:1f:6e:da:e5:42:8b:ef:13:5a:da:77:1a:19:da:05:91:
         d4:06:9a:a0:4a:b3:98:bb:c5:f5:f1:00:67:bc:63:47:63:f6:
         52:54:fe:66:09:ed:7c:c9:ff:73:ed:e3:ee:05:a7:7c:63:47:
         22:3a:2c:69:88:b2:17:bd:aa:e2:4e:9b:80:ec:75:4d:5f:17:
         d3:c9:03:d8:03:01:1f:6e:ee:b5:77:9e:51:04:d9:4d:dd:1b:
         51:66:96:a6:be:fa:36:b0:c1:f7:99:38:3f:a7:74:b7:96:e4:
         62:c8:2e:89:08:d5:fb:6b:79:fd:d4:af:ad:b4:cf:55:b5:d0:
         99:61:b2:0b:9d:c8:4b:63:88:df:b0:08:d9:b2:1a:78:20:83:
         91:11:c3:a3:1b:be:2f:f0:ed:80:7f:0d:eb:49:36:08:b7:da:
         fa:97:f4:fd:cc:50:09:1d:fd:a7:86:7e:b6:ab:95:29:f6:20:
         24:8b:1d:64:b0:fa:6d:ba:ea:2b:d5:5d:a5:24:76:bf:25:1c:
         0c:b3:13:1a:73:b4:b6:e0:ee:cc:17:7f:c5:d8:49:d0:62:9b:
         9c:1b:f7:fa
-----BEGIN CERTIFICATE-----
MIIC4jCCAcqgAwIBAgIQHWt3kGySgJxPtsalC0EoKzANBgkqhkiG9w0BAQsFADAt
MSswKQYDVQQDEyJBREZTIFNpZ25pbmcgLSBzdHMuYWxsYW5ncmF5LmNvLnphMB4X
DTE2MDkwNzA5MDQyM1oXDTE3MDkwNzA5MDQyM1owLTErMCkGA1UEAxMiQURGUyBT
aWduaW5nIC0gc3RzLmFsbGFuZ3JheS5jby56YTCCASIwDQYJKoZIhvcNAQEBBQAD
ggEPADCCAQoCggEBANdq9BEuBRPsTdpngeFyXbfH5lBg5WyENQW0qz2FtDw3AvZh
iPdFyvTPZIeZDc4vhg+gPuG8pCxhFa6hPqNIwnLSVuyhUi4/CtZrLghF2wVVcyri
ijvirzdVp2m56nO31NB5HXbSerTmey1gJsgumr+MiaM2CEI9z5ctwAp66jqM9jVv
7kzqIwB33irSck+X97jUa9XVa0/0QPBdrSVUR0i4rmfZ9orRdTKC3IA13bD9duk2
Kc9V7t8t/woo80Kbbb3ZseYk5N8AI+7RRw9+oSAm8zZQzBYkNkAMeI1mto1faXsm
9Aea4HXbyCbvVOx/JGj5Ki7YK/BtzWAyCgRu0TkCAwEAATANBgkqhkiG9w0BAQsF
AAOCAQEAd9rdJ1V9te8njMHiEuvr1DLVUBt2YdpZ8qGFC2rWDKeEW8iARrMfbtrl
QovvE1radxoZ2gWR1AaaoEqzmLvF9fEAZ7xjR2P2UlT+ZgntfMn/c+3j7gWnfGNH
IjosaYiyF72q4k6bgOx1TV8X08kD2AMBH27utXeeUQTZTd0bUWaWpr76NrDB95k4
P6d0t5bkYsguiQjV+2t5/dSvrbTPVbXQmWGyC53IS2OI37AI2bIaeCCDkRHDoxu+
L/DtgH8N60k2CLfa+pf0/cxQCR39p4Z+tquVKfYgJIsdZLD6bbrqK9VdpSR2vyUc
DLMTGnO0tuDuzBd/xdhJ0GKbnBv3+g==
-----END CERTIFICATE-----

Ettery on 13 Feb 2017

Just to confirm that I am seeing the same issue running the code above on Ubuntu 14.04, either inside a Docker container as described earlier, or directly on the OS after installing .NET Core 1.1 by following the official instructions. (dotnet-dev-1.0.0-preview2.1-003177 apt package for the record - name is very misleading but it appears to install .NET Core 1.1)

$ openssl x509 -in adfscert.crt -text

Is successful, giving the same result as above, while the code given in the original issue gives the error reported there, using the same string certificate value.

Since openssl appears to be working if reading the certificate from a file (if the header/footer are added), I am now looking for a work-around. I'm assuming that the corefx is wrapping OpenSSL for this functionality?

Ettery on 15 Feb 2017

I have now also attempted a work-around; As openssl seems happy to read the certificate from a text file (provided it has the headers and footers), I changed the code to write out a text file (with the head/foot) and attempt to import the certificate from file using :

File.WriteAllText(fileName, certText);
var coll = new X509Certificate2Collection();
coll.Import(fileName);

but unfortunately it throws the same error on Ubuntu 14.04 (again, it works on Windows):

Unhandled Exception: Mercury.Shared.Utils.Exceptions.ExtendedAggregateException: Failed to build an X509Certificate from the TokenSigningCertificateString: 
[
    MIIC1DCCAbygAwIBAgIQKm2tEiWCBpVPD//i6K0rRTANBgkqhkiG9w0BAQsFADAmMSQwIgYDVQQDExtBREZTIFNpZ25pbmcgLSBzdHMuZ3JheS5uZXQwHhcNMTYxMjE0MTcyODIwWhcNMTcxMjE0MTcyODIwWjAmMSQwIgYDVQQDExtBREZTIFNpZ25pbmcgLSBzdHMuZ3JheS5uZXQwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDQ2+rIldr1TcexvGodUzt+mVoZI88ZG0N8yBjz3Qdrh8x+qUcVD+rU2/N0dkHhjKzqso8qaASxBhuJfR/Sj07mj5NjRrRfY8K1ItqdWoVuVo2i55zuHULRhiRQt5Mz3ue2l4ir2gFOcCPBkvliEWb/XRaNFQfA3q7lN2KDlSJWBbwGlaoF5RqM8LVlCkiZ1DLa4oc0L1FAK/pSsE05ASIRL2Zi8qyTQ4KmYYGG4lV3nNX+4nIujTSUz8VMG/Qaibtap/kG3H//3yE5mK0LIjap9pK2W2CPMBAjzPlsT2igrx8fNNjgtLQAWDnz6xbP+F48Hv0/7Nj5MBvGM2PojHQfAgMBAAEwDQYJKoZIhvcNAQELBQADggEBACfczXarCH6GYM6FKFFI9q9bhBXnzgvs1mTdhSqzX1W/Dskvg+zHvnrpq3LMubhNQXYO8svR6qr/1co+6T9j6BCDbisTrXagpWFzrkOY4QmHZxUCZDR6vbppQ1fM8jxHFoK0crDa6jWvbOZJanx+7pusstL0Wka1rEMTUQgKDOzeJmqTniNTCqIB0PRIu9VvVBv0a91zMGKzzVSxWZHzDhqvKXM0+v/fTEVHsHLcxTu5uMfAv4lLnzcfqVEtLEvt0lmI1eI1Id+pmzMZAjfqLf9O4UCM24NAgueKA/zJxuK8XaVQUJ9C/DnmlHtNpP95E3zM/s5yAF+AnGdLSuyWvAg=
] 
written to certificate file: /tmp/tmp449IdG.tmp 
(error:0D07803A:asn1 encoding routines:ASN1_ITEM_EX_D2I:nested asn1 error) (First exception) ---> Interop+Crypto+OpenSslCryptographicException: 
 error:0D07803A:asn1 encoding routines:ASN1_ITEM_EX_D2I:nested asn1 error
   at Internal.Cryptography.Pal.StorePal.FromBio(SafeBioHandle bio, String password)
   at Internal.Cryptography.Pal.StorePal.FromFile(String fileName, String password, X509KeyStorageFlags keyStorageFlags)
   at System.Security.Cryptography.X509Certificates.X509Certificate2Collection.Import(String fileName, String password, X509KeyStorageFlags keyStorageFlags)

So this has become a showstopper which will prevent me from using Ubuntu or Docker for my WebAPI, since I need to load a certificate to initialise JwtBearerOptions for the:

app.UseJwtBearerAuthentication()

call.

How are other people managing to use JwtAuthentication on Linux? I can't be the only one using WebAPI authentication in Docker.

Ettery on 15 Feb 2017

Yes, CoreFx on Linux uses OpenSSL to know what a certificate is. The good news is that OpenSSL is being consistent, whether from a file or from bytes it's complaining that it doesn't like your certificate.

The bad news is that it's not being consistent between the commandline tool and our usage of the library. I've never heard of a case where it/we failed to load a legitimate certificate.

Unless the commandline tool is linked against a different version of the library this is just baffling. If you could catch it under gdb/lldb maybe you could at least determine how far in it is when it hits what it considers to be an error.

Do any certificates work on your environment?

bartonjs on 15 Feb 2017

I changed my code from

var byteCert = Encoding.ASCII.GetBytes(certString);

var byteCert = Convert.FromBase64String(certString);`

before calling new X509Certificate2(byteCert) and it now works on Windows, Ubuntu and Docker.

Doesn't explain why loading from a text file doesn't work, but I'm very happy to have a solution!

Ettery on 16 Feb 2017

👍4 🎉1

Oh, I totally missed that you used ASCII.GetBytes instead of FromBase64String. I guess Windows' PEM loader isn't a PEM-loader but a Base64-loader; and OpenSSL's checks the PEM armor.

Well, that'll certainly make for some interesting new tests.

bartonjs on 16 Feb 2017

Okay, so taking the static DER certificate MsCertificate data and converting it to Base64 (then back to bytes as ASCII) shows that Windows definitely reads the certificate, and Linux and macOS definitely don't.

I'm having trouble deciding where to draw the line here. On Windows and macOS we just pass the bytes down and the system does what the system does. On Linux it's a bit different, because we have to try both as DER and PEM (but there's no API for "as Base64 DER").

It'd have to be a fallback, because first-byte sniffing isn't sufficient. Apparently all of Windows, Linux/OpenSSL, and macOS are content reading

@"Monkey Business
-----BEGIN CERTIFICIATE-----
...
-----END CERTIFICATE----";

So it's not just "oh, the first byte's char-M, so this is clearly Base64".

I'm inclined to call this a "Windows convenience feature" for 2.0 to avoid the complexity that the fallback pathway would introduce.

bartonjs on 4 Apr 2017

@bartonjs We're hitting the same issue from new tests for the Azure IoT SDK.
The certificate is here: https://github.com/Azure/azure-iot-sdk-csharp/blob/master/provisioning/service/tests/Config/AttestationMechanismTests.cs#L22

According to @mamokarz this is working fine in our Java SDK on both Windows and Linux and may cause some confusion for our customers. Is there any documentation on supported certificate formats on each platform that we can point them to?

/cc @tameraw

CIPop on 21 Dec 2017

@CIPop nope. But the problem with the one you linked to is it's invalid PEM. A newline is required at the end of line 22's string, and before line 32's.

bartonjs on 21 Dec 2017

👍1

A newline is required at the end of line 22's string, and before line 32's.

Thanks @bartonjs! Still, it's odd that Windows works...

CIPop on 21 Dec 2017

Just means that OpenSSL and CAPI (and Security.framework) have different laxities in their PEM readers. The joys of calling system functions in the past is we don't really know what the edge cases are until people hit them and report them.

bartonjs on 21 Dec 2017

😄1 👍1

I'd like to put a vote towards this being resolved somewhere in the .Net stack before it his the underlying OS calls. We've just encountered this as we've been moving from Windows to Linux. Luckily we figured out the issue (and discovered this thread afterwards) so workaround wasn't difficult but a waste of time. It's easy enough to have just dealt with the decoding itself initially. .Net throwing an exception if we'd passed in raw bytes (not base 64 decoded cert) would have been good here. "Windows convenience feature" = "Unexpected behaviour of your application across platforms"

guytp on 22 Aug 2018

Having the same issue on:

dotnet --version

2.2.202

lsb_release -a

No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 18.04.2 LTS
Release:    18.04
Codename:   bionic

Exception:

Interop+Crypto+OpenSslCryptographicException: error:0D07803A:asn1 encoding routines:ASN1_ITEM_EX_D2I:nested asn1 error
   at Internal.Cryptography.Pal.CertificatePal.FromBlob(Byte[] rawData, SafePasswordHandle password, X509KeyStorageFlags keyStorageFlags)
   at System.Security.Cryptography.X509Certificates.X509Certificate..ctor(Byte[] data)
   at System.Security.Cryptography.X509Certificates.X509Certificate2..ctor(Byte[] rawData)

barbarosalp on 4 Apr 2019

This has worked:

var certificateWithoutHeaderAndFooter = certificateString
                        .Replace("\\n","")
                        .Replace("-----BEGIN CERTIFICATE-----", "") 
                        .Replace("-----END CERTIFICATE-----", "");
                    var certificateBytes = Convert.FromBase64String(certificateWithoutHeaderAndFooter);
                    var certificate = new X509Certificate2(certificateBase64Bytes);

barbarosalp on 4 Apr 2019

👍1

@barbarosalp That sounds like your contents aren't actually an X.509 certificate (content/header mismatch). I'm guessing that if you ran certificateBytes through X509Certificate2.GetCertContentType it'll say Pfx/Pkcs12, (If it actually says Cert then my fallback guess is certificate.RawData.Length is less than certificateBytes.Length)

bartonjs on 4 Apr 2019

Was this page helpful?

0 / 5 - 0 ratings