Home: NuGet cannot restore from HTTPS sources that have SSL certificate problems

[joelverhagen]

Idea

We should add a way for users to bypass certain HTTPS certificate errors. If the user has a source that does not have a valid certificate chain, they should still have some way of getting NuGet to interact with this source.

"Inspirations"

npm and yarn have this:

npm config set strict-ssl false
yarn config set strict-ssl false

Maven has some extra knobs and switches:

-Dmaven.wagon.http.ssl.allowall=true
-Dmaven.wagon.http.ssl.insecure=true
-Dmaven.wagon.http.ssl.ignore.validity.dates=true

PHP's Composer

{
    "config": {
        "disable-tls": true,
        "secure-http": false
    }
}

curl has it:

Specific Scenario

nuget.org server side runs functional tests against the staging slot of DEV, INT, and PROD that use nuget.exe. The DNS for this staging slot is some random blah.cloudapp.net. The certificate used is associated with the production DNS (e.g. *.nuget.org). This means nuget.exe fails SSL checks when running against this staging slot.

Proposal

Add a NuGet.config trustedCertificates section with standard <add> and <remove> tags. The value should be the SHA1 fingerprint (a.k.a. thumbprint) of the certificate. This allows developers to specifically opt-in into the tenuous state of SSL validation failures. The risk is mitigated since the developer must opt into it and even then only on a case-by-case basic.

Any certificate added to the trustedCertificates set is excluded from SSL validations -- I am thinking ALL validations exposed to use via the .NET API (untrusted root, name mistmatch, etc).

To improve discovery, we can add a log which writes out the fingerprint of the certificate that failed SSL validation at the error level.

We should also consider adding an environment variable to set this value. I believe npm has this and this sibling environment variable would match other NuGet.config keys, such as the location of the user packages directory.

Implementation

HttpClientHandler in System.Net.Http 4.3.0 (maybe earlier) has a way to set the certification validation check. This allows an app to configure the certificate validation on a per-request basis, instead of a per-AppDomain basis previously provided by ServicePointManager static properties.

With this approach, we should be able to implement the settings for all of our entry points: nuget.exe, .NET CLI, and (most tricky) Visual Studio.

The only possible problem is TFM compatibility with the System.Net.Http package. This requires investigation.

[scottbommarito]

Is there any danger for us to add an "allow all SSL certs" configuration much like the other package managers? I understand that it is a risky operation that somebody shouldn't do, but given that so many others do it, I could imagine somebody being frustrated that we don't allow the functionality.

[joelverhagen]

Good question. I would rather be reactive to that change based on community requirements. This feature request is already driving by an "external" requirement (server team). I think a broader option is unnecessary and even undesired for us. If a customer wants that broader SSL validation opt-out, they should speak up 😄.

[gavinbeatty]

For the problem of corporate firewalls which MITM traffic with their own CA, a solution other package managers/apps use is to add a config option to use a specific CA bundle, or respect the environment variables SSL_CERT_DIR or SSL_CERT_FILE. Examples:

• ~/.curlc: cacert = /path/to/ca
• ~/.wgetrc: ca_certificate=/path/to/ca
• ~/.pip/pip.conf:
  rc [global] cert = /path/to/ca
• rustup: respects SSL_CERT_FILE
• git: http.sslCAInfo

I think this suggestion might be orthogonal to yours, @joelverhagen. I don't know how common it is for certs to be broken -- in my case, I just want NuGet to recognize the unknown cert that is valid.

[joelverhagen]

@gavinbeatty wouldn't the Windows-y solution here be to install the corporate CA's certificate as a trusted root on that machine?

[KyleWiering]

@joelverhagen Correct, that is the way to remedy the issue of a self signed cert not being recognized. as @gavinbeatty suggests, most other utilities that connect through ssl provide an optional path to a cert bundle - as they do not know how to reference the Windows Certificate store.

• Newer composer uses PHP.ini openssl.cafile=path/to/cafile

The suggestion really is an approach to getting stuff working quickly in a local environment - and bringing nuget more in-line with other package managers. The downside to such ability, as @scottbommarito suggests, is that it can be used to turn off TLS inappropriately.

Not being able to temporarily ignore a cert signature is a frustration. Admittedly, I come from a more open source / composer / npm background - and internal proxy package hosts with self signed certs used congruously with the approved external package hosts is commonplace.

[pcewing]

Just a note, I came across this issue after trying to restore packages with NuGet on Ubuntu via the Rider IDE. It failed despite having trusted certificates installed via the ca-certificates package. Having the option to either bypass SSL or specify trusted certs via environment variables would be great!

[joelverhagen]

@pcewing, what would the ideal configuration be for you? I see three options so far:

1. Allow (trust) all SSL certificates. This seems like @scottbommarito's suggestion.

2. Allow a "cafile" option where you can point at a file containing one or more trusted certs. Presumably these are trusted _roots_ not just specific certificates. This seems to be @gavinbeatty and @KyleWiering's suggestion.

   1. Maybe supporting well known environment variables SSL_CERT_DIR and SSL_CERT_FILE.

3. Allow trusting of specific certificate fingerprints (thumbprints) via NuGet config. This does not imply trusting a root just trusting specific SSL certificates. This is my suggestion.

I think one or more of these could be supported at the same time. I want to get a sense of what covers different people's scenarios. Note although I am on the NuGet team, this feature is by no means a commitment -- we're in the brainstorming phase here 😄!

[pcewing]

@joelverhagen I think a combination of 1 with something like 2 or 3 would be great. Another package manager I frequently use for Erlang/Elixir is Hex, and they added support for a HEX_UNSAFE_HTTPS environment variable that can be set when SSL should be bypassed altogether.

Personally I think option 2 sounds better as setting environment variables is already a standard step when configuring build/CI machines and it's easy for an IT department to do globally on developer workstations. Option 3 seems more useful in the case where a repository specifically depends on trusted certs, so every developer who clones that repo automatically gets the Nuget.config pointing to them.

Either way, just having a mechanism would be nice. With .NET core being adopted and development becoming more common on other operating systems, it will probably become a more frequent request. And don't worry, I totally understand! :)

[gavinbeatty]

There are many solutions that might work:

• if the software looks in the Windows cert store, install the cert if you have permissions.
• if that doesn't work out (unfortunately it is quite common for non-Microsoft open source software to not look in the Windows cert store at all):
  
  
  
  • ignore all cert checks (option 1).
  
  
  • ignore specific cert checks (option 3).
  
  
  • specify an alternative cert (option 2).
  
  

I'm assuming NuGet does the right thing and looks in the Windows cert store, but I still slightly prefer option 2, the alternative cert file/dir (my suggestion).

• From memory, I think there are "personal" and "system" cert stores in Windows and it's not obvious which to choose.
• No permissions required.
• Actually checks certs.
• Everybody else is doing it. I think it's likely that people other than me have already been searching Stack Overflow for "how to specify ssl cert for nuget", etc., simply because that's the goto solution in other domains. It would be nice if the answer was a straightforward "we have that in nuget too".

[foresightyoue]

@joelverhagen I have the nuget problem by CA certificates as below,could it be fixed by engored the CA certificates?if so,how to set the nuget config with egnored the CA certificates?Thanks.

user@a1adaf3:/projects$ dotnet restore
log : Restoring packages for /projects/CSharpDemo/project.json...
error: Unable to load the service index for source https://api.nuget.org/v3/index.json.
error: An error occurred while sending the request.
error: Peer certificate cannot be authenticated with given CA certificates

[joelverhagen]

@hwx405562, today NuGet only supports using the built-in set of CA certificates. This behavior is defined by the implementation of .NET Core's HTTP stack. On Windows, this is the system's trusted CAs from the Windows certificate store. I am not sure about the specifics of macOS and Linux.

This issue is mainly focusing on designing a new feature. Could you file a new issue with your question, including OS version, .NET CLI version, repro steps, etc? We can focus on your particular issue on more depth with a seperate GitHub issue.
https://github.com/NuGet/Home/issues/new

[gavinbeatty]

It sounds like @pcewing and I would both be very pleased with environment variables, but @joelverhagen, what are your thoughts?

[anaselhajjaji]

In my side, I forked the nuget and modified the source code to accept all the certificates (it's for internal use in company).
https://github.com/anaselhajjaji/nuget/tree/db4c607f1b326074da583e9a13bae47932d963f4/nuget-binary

[zhili1208]

close as dupe of #5773

[emmellee]

@zhili1208, @joelverhagen, @rrelyea, could I persuade you and your nuget team members to take a renewed interest in support for ssl certificates? My organization wishes to create nuget packages for release to our internal feed provided by our TFS Server. The DoD relies heavily on the use of smartcards and CA certs and, like any DoD webserver, our TFS Server requires mutual ssl authentication. Unfortunately, we are unable to get past this and duplicate issue #5773. Any help your team can provide towards resolving this issue would be greatly appreciated. The TFS/VSTS teams have offered their resources towards resolving their piece of the puzzle (reference issue #5773). Thank you.

[tmds]

Maybe supporting well known environment variables SSL_CERT_DIR and SSL_CERT_FILE.

These variables are supported by .NET Core 2.1.

[alexei-matveev]

Some may be affected by Dotnet SDK using too specific SONAMEs for libssl.
I just had to troubleshoot somewhat similar problem.
Dotnet picks some old libss.so.1.0.0 from a random directory whereas most
executables refer to libssl.so.10 or libssl3.so on CentOS 7:

$ strace -e open dotnet restore 2>&1 | grep libssl
open("/lib64/tls/libssl.so.1.0.2", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/lib64/libssl.so.1.0.2", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/usr/lib64/tls/libssl.so.1.0.2", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/usr/lib64/libssl.so.1.0.2", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/usr/lib/nsr/lib64/libssl.so.1.0.0", O_RDONLY|O_CLOEXEC) = 79

Compare that to the output of

ldd /usr/bin/* | grep libssl

that is dominated by

    libssl3.so => /lib64/libssl3.so (0x00007f3584227000)
    libssl.so.10 => /lib64/libssl.so.10 (0x00007f78c8b66000)

This machine happens to have some legacy software installed that brings its own libssl.

[alexei-matveev]

With the newest version of SDK mine case is closed:

$ dotnet --info
.NET Core SDK (reflecting any global.json):
Version: 2.1.302
Commit: 9048955601

[awnugent]

Just in case this is helpful to others, I found that, in an Azure DevOps on-prem agent context, that simply disabling the Install NuGet task and relying on the nuget.exe installed with the ADO agent seemed to work.
I consider this a workaround and NOT a fix. MITM certs in sensitive, security conscious environments is a real concern and a real fix would be great. We actually have the MITM device certs in our chain and on the system in question, but apparently nuget doesn't respect their authority ;)
PLZ make better.

[tavisca-cchowdhary]

If you are having this error in a docker container then run

• docker system prune --volumes.

Then stop the docker app in your machine.
Restart your machine and running everything from start.

[cs101]

being able to specify SSL exceptions in an explicit way is valuable. especially in enterprises that use MITM certs like Zscaler.... I've being fighting this in my most recent two jobs. if Nuget doesn't provide a way to do it it's going to lead a a lot of risky work around anyway.

[BogusDude99]

Agreed, please give an option to bypass cert checks for the corporate MITM scenarios

[fsdrw08]

my current company also had a MITM security gate (bluecoat), same issue,
when run dotnet restore, error pops up,
The remote certificate is invalid according to the validation procedure.

Please provide a dotnet cli option to disable or bypass ssl validation... thanks