Cargo: Cargo gives SSL error on Windows

[Tembocs]

Problem
I'm on Windows 10 x64 using Rust 1.36.
I'm trying to run cargo run get the following error message.

error: failed to download from `https://crates.io/api/v1/crates/assert_cmd/0.11.1/download`     

Caused by:
  [35] SSL connect error (schannel: next InitializeSecurityContext failed: 
Unknown error (0x80092013) - 
The revocation function was unable to check revocation because the revocation server was offline.)

Steps

1. Initialize a new Rust project
2. Run cargo run

Possible Solution(s)

Notes
Output of cargo version:
cargo 1.36.0 (c4fcfb725 2019-05-15)

I've tried the following without success.

1. Disable the Anti-virus
2. Disable firewall, and
3. I'm not behind a proxy
4. Reinstalled Git set it to use OpenSSL

This problem makes Rust unusable on Windows, which is the main development platform for me.

[ehuss]

There is a workaround at https://github.com/rust-lang/cargo/issues/7096#issuecomment-508831504

@alexcrichton It seems strange to have two reports for CRYPT_E_REVOCATION_OFFLINE in a row. Do you know exactly what it means? Does that error mean that the letsencrypt ocsp server is unreachable? Is the crates.io cloudfront configured to use OCSP stapling? I also see CRL servers in the certificate chain (crl.identrust.com for the intermediate authority), could it be an issue there? Or is this maybe something deeper about the user's certificate store?

[Tembocs]

@ehuss, the CARGO_HTTP_CHECK_REVOKE=false environmental variable works, thanks.

[alexcrichton]

@ehuss yeah I was wondering the same! I'm not actually sure what is offline when this comes out, although I wouldn't put it beyond an OCSP server somewhere that everyone basically uses (like letsencrypt) to be temporarily down or having load issues.

AFAIK crates.io is not configured with OCSP stapling (since I barely know what that is)...

We've seen this intermittently on CI in the past as well on AppVeyor a long time ago and ended up disabling it. I get the feeling that OSX/Linux don't actually do these revocation checks anyway, it's just Windows, so we may just want to always disable the feature here. I'd want to investigate though to make sure that we've got the same behavior on all three platforms.

[ehuss]

I've been experimenting with this a little bit. I'm able to reproduce by:

1. Block the OCSP servers on my router. That is, whatever your local IPs are for ocsp.int-x3.letsencrypt.org and isrg.trustid.ocsp.identrust.com.
2. Clear certificate cache on windows. (Not sure if this is necessary.)
   2a. certutil -urlcache * delete
   2b. certutil -setreg chain\ChainCacheResyncFiletime @now
3. Reboot
4. Delete something from cargo's cache and attempt to fetch it.

I've also verified that OCSP stapling is not enabled for crates.io. (cc @rust-lang/crates-io). Although the post at https://aws.amazon.com/blogs/aws/cloudfront-ssl-ciphers-session-ocsp-pfs/ implies that OCSP stapling is enabled by default for Cloudfront. I'm not really familiar with crates.io's infrastructure.

Blocking just the CRL server (crl.identrust.com) causes windows to pause about 10-20 seconds, it eventually succeeds (maybe windows cares more about ocsp than it does crl?).

I think one of the few things we can try is to figure out how to enable OCSP stapling on crates.io. However, that is bit out of my hands.

Blocking these servers doesn't seem to phase macos or linux. I tried deleting the cache on macos using crlrefresh, but I can't tell if it worked. It's possible I don't know how to flush the caches properly. It's also possible we aren't doing ocsp checking on those platforms.

[alexcrichton]

Currently crates.io is using Heroku natively and doesn't have a CDN in front of it (unlike static.crates.io). Does OCSP stapling fix this issue one way or another? If so I think that's a reasonable avenue to fix this, but otherwise I think we may want to just disable this by default on Windows since it's already not on by default on Linux/macOS (and I'm not sure if there's even a way to enable it there...)

[ehuss]

Oh, I'm not sure why I was thinking it had a CDN. I must have been confused between static.crates.io, and some earlier discussion about it.

According to this, heroku may not support it.

I'm pretty sure OCSP stapling would fix it. With the OCSP response included in the TLS handshake, there's no need to contact a separate service.

But since it's probably not an option, perhaps disabling it would be the best option. I'd like to get some kind of confirmation that linux and macos do not do ocsp checks. Certificate revocation has always been a sketchy part of certificate management.

[alexcrichton]

For Linux it's all OpenSSL, and according to this and other assorted scattered bits of info I don't think OpenSSL enables OCSP by default.

For macOS it's SecureTransport, and random blog posts assert OCSP checking is off-by-default (and another), but I can't really find anything more developer oriented indicating this. It appears that "Keychain Access" used to have an option to configure OCSP settings but from what I can tell the latest macOS (or at least my install of Mojave) doesn't have the same settings pane. Taking a look at the plist file mentioned in this blog post gives me "OCSPStyle: None" and "OCSPSufficientPerCert: YES". No idea what those mean...

All-in-all though I'm pretty certain that OCSP checking is not enabled by default on OpenSSL/SecureTransport, and we definitely have a lot of data showing that it is enabled on SChannel on Windows.