Go: crypto/x509: CentOS 7 "x509: failed to load system roots and no roots provided" issues.

[mckn]

Please answer these questions before submitting your issue. Thanks!

1. What version of Go are you using (go version)?
   1.6.2
2. What operating system and processor architecture are you using (go env)?
   I am running go 1.6.2 from the official docker container with CentOS7 as the hosting os.
3. What did you do?
   I was using the http.Client and I did a request to a https:// endpoint.
4. What did you expect to see?
   I was expecting to get a 200OK request. If I did the same request with curl or wget it worked like a charm so it wasnt any problems with the certs on the OS.
5. What did you see instead?
   When I was doing the request I got the following error: "x509: failed to load system roots and no roots provided"

Solution:
So since other tools worked with https on the host I knew that it was not any problems with the certificates. I checked where the ca bundle was located on the host system and compared it with the following file:

https://github.com/golang/go/blob/master/src/crypto/x509/root_linux.go

On CentOS 7 the certificate bundle is located at the following path:
/etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem

Then there are a link to that file on the following path:
/etc/ssl/ca-bundle.crt

Note that it is .crtinstead of .pemas in the root_linux.gofile.

So I tried to run my container to link the /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem to /etc/ssl/ca-bundle.pem and everything works. It was a work around to get my application to work but I would prefer it if we can add the /etc/ssl/ca-bundle.crt to the paths in root_linux.go.

Hope this is possible! Thanks!

[bradfitz]

Related: #14022 and https://golang.org/cl/20253 (see 4th comment from Krzysztof Malinowski).

[driusan]

I'm having a very similar problem under darwin/amd64 under both 1.6.2 and tip. I get an error "x509: certificate signed by unknown authority" for a certificate which is valid and works under chrome, safari, and curl. I've verified that the root authority being used by chrome is in the system keychain.

It seems like at a higher level than the _linux.go file it's somehow not parsing all the valid root authorities on the system.

[mckn]

@driusan have you validated your certificate with the openssl client?

openssl s_client -showcerts -connect your-uri

[driusan]

@mckn you're right, the openssl client also fails, while every other https client I've tried has no problem validating it. Googling around for the error from openssl ("Verify return code: 21 (unable to verify the first certificate)") is more helpful than the error from Go, as it leads to this:

http://stackoverflow.com/questions/7587851/openssl-unable-to-verify-the-first-certificate-for-experian-url

Which suggests the problem is when the intermediate certificate isn't included in the bundle returned by the server web browsers will dynamically retrieve it/verify it, while crypto/x509 (and hence net/http) and openssl just give up.

I'll take up the misconfiguration on the server I'm trying to access the sysadmins, but I still think this behaviour should be considered a bug in Go, since most developers will expect the certificate verification from net/http to validate in the same way as their web browser, curl, wget, etc.

(Sorry to have hijacked your issue.)

[gopherbot]

CL https://golang.org/cl/30375 mentions this issue.