Runtime: HttpClient: SSL connect error with client certificate
Hi
I'm trying to do an https request using HttpClient
var clientHandler = new HttpClientHandler
{
ServerCertificateCustomValidationCallback = ServerCertificateCustomValidationCallback()
};
if (caData != null)
{
var clientCertificate = new X509Certificate2(Encoding.ASCII.GetBytes(caData.CertPem));
clientHandler.ClientCertificates.Add(clientCertificate);
}
using (var c = new HttpClient(clientHandler))
{
string requestUri = $"https://XXXXX";
HttpResponseMessage response = await c.GetAsync(requestUri);
}
My nginx is configuring to accept client certificate if given, but it doesn't do the verification ssl_verify_client optional_no_ca (it's done somewhere else)
nginx config:
server {
listen 444;
server_name localhost 127.0.0.1;
root /var/www/htdocs;
error_log /var/log/nginx/cws.error.log;
ssl on;
ssl_certificate /etc/ssl/XXXXX.crt;
ssl_certificate_key /etc/ssl/private/XXXXXX.key;
ssl_session_timeout 5m;
ssl_session_cache shared:SSL:1m;
ssl_protocols TLSv1.2;
ssl_ciphers HIGH:!aNULL:!MD5:!RC4;
ssl_prefer_server_ciphers on;
ssl_verify_client optional_no_ca;
}
When I use HttpClient with no client certificate: It works
When I use HttpClient with a client certificate, it failed with the following error
When I use curl (command line) both case works
Error while executing request ---> System.Net.Http.HttpRequestException: An error occurred while sending the request. ---> System.Net.Http.CurlException: SSL connect error
at System.Net.Http.CurlHandler.ThrowIfCURLEError(CURLcode error)
at System.Net.Http.CurlHandler.MultiAgent.FinishRequest(StrongToWeakReference`1 easyWrapper, CURLcode messageResult)
After looking into the code of corefx I finaly found more log with perfcollect
{\"workerId\":0, \"requestId\":0, \"memberName\":\".cctor\", \"message\":\"libcurl: 7.35.0 OpenSSL/1.0.1f CURL_VERSION_IPV6, CURL_VERSION_SSL, CURL_VERSION_LIBZ, CURL_VERSION_NTLM, CURL_VERSION_GSSNEGOTIATE, CURL_VERSION_ASYNCHDNS, CURL_VERSION_LARGEFILE, CURL_VERSION_IDN, CURL_VERSION_TLSAUTH_SRP, CURL_VERSION_NTLM_WB\"}
strings sampleTrace2.trace/lttngTrace/auto-20161020-194235/ust/uid/1001/64-bit/channel0_* | grep Curl
System.Collections.Generic.Dictionary`2[System.IntPtr,System.Net.Http.CurlHandler+MultiAgent+ActiveRequest]
System.Collections.Generic.Queue`1[System.Net.Http.CurlHandler+MultiAgent+IncomingRequest]
System.Collections.Generic.Queue`1[System.Net.Http.CurlHandler+MultiAgent+IncomingRequest]
System.Collections.Generic.Dictionary`2[System.IntPtr,System.Net.Http.CurlHandler+MultiAgent+ActiveRequest]
System.Collections.Generic.Dictionary`2[System.IntPtr,System.Net.Http.CurlHandler+MultiAgent+ActiveRequest]
System.Collections.Generic.Dictionary`2[System.IntPtr,System.Net.Http.CurlHandler+MultiAgent+ActiveRequest]
System.Collections.Generic.Dictionary`2[System.IntPtr,System.Net.Http.CurlHandler+MultiAgent+ActiveRequest]
{\"workerId\":379, \"requestId\":378, \"memberName\":\"FailRequest\", \"message\":\"Failing request: System.Net.Http.CurlException: SSL connect error\\n at System.Net.Http.CurlHandler.ThrowIfCURLEError(CURLcode error)\\n at System.Net.Http.CurlHandler.MultiAgent.FinishRequest(StrongToWeakReference`1 easyWrapper, CURLcode messageResult)\"}
System.Collections.Generic.Queue`1[System.Net.Http.CurlHandler+MultiAgent+IncomingRequest]
EmptyArray`1[System.Net.Http.CurlHandler+MultiAgent+IncomingRequest]
System.Collections.Generic.Dictionary`2[System.IntPtr,System.Net.Http.CurlHandler+MultiAgent+ActiveRequest]
System.Collections.Generic.Queue`1[System.Net.Http.CurlHandler+MultiAgent+IncomingRequest]
{\"workerId\":379, \"requestId\":378, \"memberName\":\"CurlDebugFunction\", \"message\":\"CURLINFO_TEXT: SSLv3, TLS handshake, Server key exchange (12):\"}
{\"workerId\":379, \"requestId\":378, \"memberName\":\"CurlDebugFunction\", \"message\":\"CURLINFO_SSL_DATA_IN: 333 bytes\"}
{\"workerId\":379, \"requestId\":378, \"memberName\":\"CurlDebugFunction\", \"message\":\"CURLINFO_TEXT: SSLv3, TLS handshake, Request CERT (13):\"}
{\"workerId\":379, \"requestId\":378, \"memberName\":\"CurlDebugFunction\", \"message\":\"CURLINFO_SSL_DATA_IN: 42 bytes\"}
{\"workerId\":379, \"requestId\":378, \"memberName\":\"CurlDebugFunction\", \"message\":\"CURLINFO_TEXT: SSLv3, TLS handshake, Server finished (14):\"}
{\"workerId\":379, \"requestId\":378, \"memberName\":\"CurlDebugFunction\", \"message\":\"CURLINFO_SSL_DATA_IN: 4 bytes\"}
System.Collections.Generic.Dictionary`2[System.IntPtr,System.Net.Http.CurlHandler+MultiAgent+ActiveRequest]
{\"objectA\":\"System.Net.Http.CurlHandler+CurlResponseMessage\", \"objectAHash\":40889321, \"objectB\":\"System.Net.Http.HttpRequestMessage\", \"objectBHash\":19650377}
{\"objectA\":\"System.Net.Http.StreamContent\", \"objectAHash\":34837044, \"objectB\":\"System.Net.Http.CurlHandler+CurlResponseStream\", \"objectBHash\":34424014}
{\"objectA\":\"System.Net.Http.CurlHandler+CurlResponseMessage\", \"objectAHash\":40889321, \"objectB\":\"System.Net.Http.StreamContent\", \"objectBHash\":34837044}
System.Collections.Generic.Queue`1[System.Net.Http.CurlHandler+MultiAgent+IncomingRequest]
System.Collections.Generic.Dictionary`2[System.IntPtr,System.Net.Http.CurlHandler+MultiAgent+ActiveRequest]
System.Collections.Generic.Dictionary`2[System.IntPtr,System.Net.Http.CurlHandler+MultiAgent+ActiveRequest]
{\"workerId\":379, \"requestId\":378, \"memberName\":\"CurlDebugFunction\", \"message\":\"CURLINFO_TEXT: Hostname was NOT found in DNS cache\"}
{\"workerId\":379, \"requestId\":378, \"memberName\":\"CurlDebugFunction\", \"message\":\"CURLINFO_TEXT: Trying 127.0.0.1...\"}
{\"workerId\":379, \"requestId\":378, \"memberName\":\"CurlDebugFunction\", \"message\":\"CURLINFO_TEXT: Connected to 127.0.0.1 (127.0.0.1) port 444 (#0)\"}
{\"workerId\":379, \"requestId\":378, \"memberName\":\"CurlDebugFunction\", \"message\":\"CURLINFO_TEXT: successfully set certificate verify locations:\"}
{\"workerId\":379, \"requestId\":378, \"memberName\":\"CurlDebugFunction\", \"message\":\"CURLINFO_TEXT: CAfile: none\\n CApath: /etc/ssl/certs\"}
{\"workerId\":379, \"requestId\":378, \"memberName\":\"CurlDebugFunction\", \"message\":\"CURLINFO_TEXT: SSLv3, TLS handshake, Client hello (1):\"}
{\"workerId\":379, \"requestId\":378, \"memberName\":\"CurlDebugFunction\", \"message\":\"CURLINFO_SSL_DATA_OUT: 268 bytes\"}
{\"workerId\":379, \"requestId\":378, \"memberName\":\"CurlDebugFunction\", \"message\":\"CURLINFO_TEXT: SSLv3, TLS handshake, Server hello (2):\"}
{\"workerId\":379, \"requestId\":378, \"memberName\":\"CurlDebugFunction\", \"message\":\"CURLINFO_SSL_DATA_IN: 94 bytes\"}
{\"workerId\":379, \"requestId\":378, \"memberName\":\"CurlDebugFunction\", \"message\":\"CURLINFO_TEXT: SSLv3, TLS handshake, CERT (11):\"}
{\"workerId\":379, \"requestId\":378, \"memberName\":\"CurlDebugFunction\", \"message\":\"CURLINFO_SSL_DATA_IN: 850 bytes\"}
{\"workerId\":379, \"requestId\":378, \"memberName\":\"CurlDebugFunction\", \"message\":\"CURLINFO_TEXT: SSLv3, TLS handshake, CERT (11):\"}
{\"workerId\":379, \"requestId\":378, \"memberName\":\"CurlDebugFunction\", \"message\":\"CURLINFO_SSL_DATA_OUT: 7 bytes\"}
{\"workerId\":379, \"requestId\":378, \"memberName\":\"CurlDebugFunction\", \"message\":\"CURLINFO_TEXT: SSLv3, TLS handshake, Client key exchange (16):\"}
{\"workerId\":379, \"requestId\":378, \"memberName\":\"CurlDebugFunction\", \"message\":\"CURLINFO_SSL_DATA_OUT: 70 bytes\"}
{\"workerId\":379, \"requestId\":378, \"memberName\":\"CurlDebugFunction\", \"message\":\"CURLINFO_TEXT: SSLv3, TLS change cipher, Client hello (1):\"}
{\"workerId\":379, \"requestId\":378, \"memberName\":\"CurlDebugFunction\", \"message\":\"CURLINFO_SSL_DATA_OUT: 1 bytes\"}
{\"workerId\":379, \"requestId\":378, \"memberName\":\"CurlDebugFunction\", \"message\":\"CURLINFO_TEXT: SSLv3, TLS handshake, Finished (20):\"}
{\"workerId\":379, \"requestId\":378, \"memberName\":\"CurlDebugFunction\", \"message\":\"CURLINFO_SSL_DATA_OUT: 16 bytes\"}
{\"workerId\":379, \"requestId\":378, \"memberName\":\"CurlDebugFunction\", \"message\":\"CURLINFO_TEXT: error:02001002:system library:fopen:No such file or directory\"}
{\"workerId\":379, \"requestId\":378, \"memberName\":\"CurlDebugFunction\", \"message\":\"CURLINFO_TEXT: Closing connection 0\"}
System.Collections.Generic.Dictionary`2[System.IntPtr,System.Net.Http.CurlHandler+MultiAgent+ActiveRequest]
System.Collections.Generic.Dictionary`2[System.IntPtr,System.Net.Http.CurlHandler+MultiAgent+ActiveRequest]
System.Collections.Generic.Dictionary`2[System.IntPtr,System.Net.Http.CurlHandler+MultiAgent+ActiveRequest]
{\"workerId\":379, \"requestId\":378, \"memberName\":\"FinishRequest\", \"message\":\"Curl result: 35\"}
System.Net.Http.CurlException
instance void [System.Net.Http] System.Net.Http.CurlHandler+MultiAgent::FinishRequest(class System.StrongToWeakReference`1<class System.Net.Http.CurlHandler/EasyRequest>,valuetype Interop/Http/CURLcode)
So
{\"workerId\":379, \"requestId\":378, \"memberName\":\"FinishRequest\", \"message\":\"Curl result: 35\"}
confirm that the "problem is in curl" due to curl result
35 SSL connect error. The SSL handshaking failed.
So I read other logs from curl lib, and I found an error: No such file or directory
{\"workerId\":379, \"requestId\":378, \"memberName\":\"CurlDebugFunction\", \"message\":\"CURLINFO_TEXT: error:02001002:system library:fopen:No such file or directory\"}
Then with strace I found which file was "not found"
[pid 32581] open("XXXXX/.dotnet/corefx/cryptography/crls/bd2b1cbf.crl", O_RDONLY) = -1 ENOENT (No such file or directory)
I tried to create it to see if it was really the "missing file" and then I got this error
error:0906D06C:PEM routines:PEM_read_bio:no start line
Now I don't know how to fix this error and make it work.
I don't know why curl is trying to load crl, and from where come the filename bd2b1cbf
Socolin
All 10 comments
cc: @stephentoub
davidsh
on 21 Oct 2016
@bartonjs, thoughts on this?
stephentoub
on 21 Oct 2016
~/.dotnet/corefx/cryptography/crls/ is the directory where we cache Certificate Revocation Lists (CRLs), the bd2b1cbf part is a hash of the subject name of the CA cert associated with the CRL (you can see what it is for any particular certificate with openssl x509 -in somecert.cer -subject_hash -noout).
You definitely shouldn't create the file on your own (just rm it if you haven't already).
For the original error:
Based on your data member names, your client cert is a public key only (since you call it PEM data, and there's not a PEM format which contains a private key), but using a client auth cert requires the private key be available. This could be OpenSSL's way of saying "but I don't have a private key available, how am I supposed to use this cert?".
bartonjs
on 21 Oct 2016
~/.dotnet/corefx/cryptography/crls/ is the directory where we cache Certificate Revocation Lists (CRLs), the bd2b1cbf part is a hash of the subject name of the CA cert associated with the CRL (you can see what it is for any particular certificate with openssl x509 -in somecert.cer -subject_hash -noout).
It seems to be different between
openssl x509 -in /tmp/aaa -subject_hash -noout
2238be1a
You definitely shouldn't create the file on your own (just rm it if you haven't already).
Already done it was just to confirm it was the good file
Based on your data member names, your client cert is a public key only (since you call it PEM data, and there's not a PEM format which contains a private key), but using a client auth cert requires the private key be available. This could be OpenSSL's way of saying "but I don't have a private key available, how am I supposed to use this cert?"
I use the same certificate with --cert my_file in curl's command line and it works
curl --cert my_file.pem --cacert ca.crt https://XXXXX/
Here my file (with some missing lines)
Bag Attributes
localKeyID: XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX
subject=/C=##/ST=#######/O=#######/OU=#######/CN=#######
issuer=/C=##/ST=#######/L=#######/O=#######/OU=#######/CN=#######/emailAddress=#######
-----BEGIN CERTIFICATE-----
MIIDajCCAlICAQEwDQYJKoZIhvcNAQELBQAwgZsxCzAJBgNVBAYTAkNBMQ8wDQYD
...
4GX6OsuZZIUVea/xnQWxhsI7+zdmo7MKgA48zqMN6iAsRoUmoTVIaL7xuCJbIcH+
BqNuST3XXYo4dFQazgI=
-----END CERTIFICATE-----
Bag Attributes: <No Attributes>
subject=/C=#######/ST=#######/L=#######/O=#######/OU=#######/CN=#######/emailAddress=#######
issuer=/C=#######/ST=#######/L=#######/O=#######/OU=#######/CN=#######/emailAddress=#######
-----BEGIN CERTIFICATE-----
MIIDtDCCApwCCQDChB8FRW09pDANBgkqhkiG9w0BAQsFADCBmzELMAkGA1UEBhMC
...
ot7z7sdqG6yor20JPJSyxQ5t7vFsLKM/aNf4RAaCO8WmWkubszPMag==
-----END CERTIFICATE-----
Bag Attributes
localKeyID: XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX
Key Attributes: <No Attributes>
-----BEGIN PRIVATE KEY-----
MIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQDMF4QIqR7KCdMg
...
3184dpGJzcOxj7NxfPEMFSM=
-----END PRIVATE KEY-----
As you can see the key is here:
-----BEGIN PRIVATE KEY-----
Socolin
on 21 Oct 2016
@Socolin Does my_file.pem look sort of like
-----BEGIN CERTIFICATE-----
Base64Message===
-----END CERTIFICATE-----
-----BEGIN PRIVATE KEY-----
Base64Message2==
-----END PRIVATE KEY-----
?
If so, that format is something which works from the commandline utilities, but not from the X509Certificate2 constructor. You'll need to convert it to a PFX. If the top segment goes to cert.cer and the bottom to cert.key then you can make a PFX with openssl pkcs12 -export -in cert.cer -inkey cert.key -out cert.pfx (You might be able to use -in my_file.pem-as-is, but I've never tried that).
bartonjs
on 21 Oct 2016
@Socolin Does my_file.pem look sort of like
There is one more section with the ca.cert included. This file is the one I use with curl with --cert option
subject=AAAAAAA
issuer=BBBBBB
-----BEGIN CERTIFICATE-----
My certficitate
-----END CERTIFICATE-----
subject=BBBBBB
issuer=BBBBBB
-----BEGIN CERTIFICATE-----
The ca.cert
-----END CERTIFICATE-----
-----BEGIN PRIVATE KEY-----
The private key
-----END PRIVATE KEY-----
With pkcs12 format it works.
So it seems the file format I used is not supported but I got no clear error, but It's ok for me.
Maybe improve error handling would be interessting ?
Thank you.
Socolin
on 21 Oct 2016
With pkcs12 format it works.
Excellent. Glad to hear it.
Maybe improve error handling would be interessting ?
I've opened dotnet/corefx#12962 to track adding a test for this case and ensuring that we have some vaguely useful error messages on both Windows and Linux. That'll take over from this issue.
bartonjs
on 24 Oct 2016
Hello, I would like to ask you a question, encountered the same problem during use, I use the pem + key synthesis of a p12 file, use the p12 file called special password to request apple api, the following error, hope your response
curl: (58) could not load PEM client certificate, LibreSSL error error:0906D06C:PEM routines:PEM_read_bio:no start line, (no key found, wrong pass phrase, or wrong file format?)
OldHorsse
on 5 Jan 2018
@bartonjs hope your response
OldHorsse
on 5 Jan 2018
I use the following method to synthesize p12 file
openssl pkcs12 -export -in
After I check the file again
Certificate bag
Bag Attributes:
subject = aaaaaaaaa
issuer = bbbbbb
----- BEGIN CERTIFICATE -----
----- END CERTIFICATE -----
PKCS7 Data
I also checked the pem file and key file
pem file as follows:
Certificate:
聽聽聽聽 Data:
key file is as follows:
Private-Key: (2048 bit)
modulus:
Now I do not know why an error occurred while requesting an api
聽 / usr / bin / curl --cert /{}.p12 --pass {} "www. **. com"
OldHorsse
on 5 Jan 2018
Related issues
jchannon
路
3Comments
sahithreddyk
路
3Comments
aggieben
路
3Comments
omariom
路
3Comments
matty-hall
路
3Comments
Most helpful comment
@Socolin Does
my_file.pemlook sort of like?
If so, that format is something which works from the commandline utilities, but not from the X509Certificate2 constructor. You'll need to convert it to a PFX. If the top segment goes to
cert.cerand the bottom tocert.keythen you can make a PFX withopenssl pkcs12 -export -in cert.cer -inkey cert.key -out cert.pfx(You might be able to use-in my_file.pem-as-is, but I've never tried that).