Berry: [Bug] 'unable to get local issuer certificate' - Config options for 'strict-ssl' and 'cafile' missing from Berry?

[jamador-usgs]

Describe the bug

Previously, there was an option to force SSL and set a CA file. Looking at the config options for Berry, this is no longer the case. We don't use a proxy to access the internet, but rather a signed certificate that acts as a MiTM for our traffic. I'm getting my team to just start to work with Yarn and I'd like us to migrate to the latest version. However, without the ability to set these options like with npm and yarn v1, i get the 'unable to get local issuer certificate' error and so I've been forced to uninstall Berry and go back to Yarn v1.21.1.

To Reproduce

Not sure how to reproduce this outside of our internal network. These options are just missing from Berry.

Screenshots
Running the npx @yarnpkg/doctor command from the migration guide gives me this:

Environment if relevant (please complete the following information):

• OS: Windows
• Node version 12.6.0
• Yarn version 2.0.0-rc.27

[arcanis]

So the main issue with that is the testing infrastructure. At the moment our mock server doesn't support certificates, so implementing that wouldn't be covered by tests and I'd very much want to avoid that as it seems the kind of thing very likely to regress given enough time.

If someone can spend some time to make the mock server understand https (I admit I'm not well enough versed in it to do it myself in a timely manner) and the resolver send them it would be helpful.

[eliasku]

@jamador-usgs in general you could use NODE_TLS_REJECT_UNAUTHORIZED=0 in front of yarn command, for example NODE_TLS_REJECT_UNAUTHORIZED=0 yarn plugin import interactive-tools, or you could just export variable to Node's ENV. I think it should work for the most of node-based cli packages

[ZickZakk]

@eliasku Thanks for the hint!
For windows users, simply add variable NODE_TLS_REJECT_UNAUTHORIZED with value 0 to your systems environment variables to make this work.

[yarnbot]

Hi! 👋

This issue looks stale, and doesn't feature the reproducible label - which implies that you didn't provide a working reproduction using Sherlock. As a result, it'll be closed in a few days unless a maintainer explicitly vouches for it or you edit your first post to include a formal reproduction (you can use the playground for that).

Note that we require Sherlock reproductions for long-lived issues (rather than standalone git repositories or similar) because we're a small team. Sherlock gives us the ability to check which bugs are still affecting the master branch at any given point, and decreases the amount of code we need to run on our own machines (thus leading to faster bug resolution faster). It helps us help you! 😃

If you absolutely cannot reproduce a bug on Sherlock (for example because it's a Windows-only issue), a maintainer will have to manually add the upholded label.

[unnKoel]

@ZickZakk seeting NODE_TLS_REJECT_UNAUTHORIZED with value 0 In my windows, It doesn't work. Are you sure you can?

[constantval]

Also hitting this issue trying to migrate our department projects to Yarn2. The workaround using NODE_TLS_REJECT_UNAUTHORIZED hasn't worked for us either.

[netizen]

@arcanis I am presently trying to introduce yarn 2 based monorepo in my workplace which is a banking company with security requirements. Non availability of cafile configuration in .yarnrc.yml will be the blocking factor for adoption.
What is the scope of work involved in adding the support for cafile in .yarnrc.yml?
If this can be done in less than a week, I would be happy to contribute.

[netizen]

So the main issue with that is the testing infrastructure. At the moment our mock server doesn't support certificates, so implementing that wouldn't be covered by tests and I'd very much want to avoid that as it seems the kind of thing very likely to regress given enough time.

If someone can spend some time to make the mock server understand https (I admit I'm not well enough versed in it to do it myself in a timely manner) and the resolver send them it would be helpful.

@arcanis I am ready to spend time to try and get the mock server understand https. I went through the Contributor Guidelines but unable to find relevant instructions for this kind of work. Please let me know how I can help.

[arcanis]

Hey @netizen, that would be really nice of you! Here are some pointers:

• The configuration code is here: Configuration.ts. This is where you'd add the new settings.
• The networking code is here: httpUtils. This is where you'd want to implement CA support.
• The tests are powered by a mock server defined here: tests.ts.
• And finally, you'd write the tests themselves into a new file inside our test folder.

Feel free to pop on our Discord if you have other questions, we'll be happy to answer (I'm myself AFK today, but other maintainers might be around).

[markerikson]

I can vouch that having this feature would also be critical for us as well, due to internal certificates.

And in fact I just tried setting NODE_TLS_REJECT_UNAUTHORIZED=0 on my Windows machine and running yarn install per the migration instructions, and that failed with the usual "local issuer certificate" error as mentioned above. I know that flag has worked with other packages and tools I've worked with, so not sure why it's not working here.

[arcanis]

Unfortunately it's still the same problem: I don't have any setup with internal certificates, and as far as I know none of the other contributors do, so any change we make would be completely untested. I'd really like to put this behind us, but it's exactly the kind of change that needs to be supported by someone experiencing the issue firsthand 😕

(That, or you can suggest a step-by-step very simple way to test, like perhaps a proxy tool or something - I can't fiddle with with my computers' global configuration for this)

[andreialecu]

I just opened a PR at #1934 which should address this.

You can test it before it gets merged with yarn set version from sources --branch 1934

Add to .yarnrc.yml:

• caFilePath: <path to CA .pem file) to supply a custom CA
• or enableStrictSsl: false to disable cert errors.

[jasonwilliams]

@netizen or @markerikson are you able to test the above? Sounds like they need people who had the issue to confirm it's fixed

[markerikson]

I'd love to, but the corporate machine I'm on atm is unfortunately rather locked down. When I tried yarn version from sources, it refused to let a builder.exe process run. So, doesn't look like I can try building this on my machine.

Can someone try generating a build that has both #1934 and #1920 in here? Those were the two issues I ran into the other day.

Alternately, I see that the PR checks for #1934 supposedly generated a build and "uploaded" it somewhere. Is that build accessible for download?

edit

Okay, I see that artifact is in fact attached to the job run. Lemme give that a shot.

[markerikson]

Good news! I pulled down the built artifact from #1934 , unzipped it, and replaced the yarn-berry.js from the earlier Yarn v2 install step with that version.

I then set yarn config set caFile $PATH/TO/OUR/CERT and yarn config set enableStrictSsl false. The build did have the lockfile changes from #1920 in it as well.

Yarn was successfully able to go pull all packages from our internal NexusRepository server - no SSL errors!

So, I'm going to say this is correctly working as intended.

The _bad_ news, on my end at least, is that something about the way Yarn v2 is trying to spawn post-install build scripts in the user temp folder is triggering the corporate process monitoring allowlist tool, which is blocking those processes from executing at all. So, no post-install scripts can build anything, which means there's a good chance I won't actually be able to use this on a corporate machine :( (side question: it looks like Yarn is maybe trying to copy Node itself to a temp folder and run it? Any pointers to how that behavior works and where it's implemented?)

[andreialecu]

In theory the build in #1934 should have the changes in #1920 already applied, since that was merged a few days prior.

[andreialecu]

I then set yarn config set caFile $PATH/TO/OUR/CERT and yarn config set enableStrictSsl false. The build did have the lockfile changes from #1920 in it as well.

Note that it should be caFilePath not caFile.

Also, you should use just one of the options. If you disable strict ssl, that takes precedence and the CA file doesn't really do anything. The flag will disable certificate validation.

[arcanis]

side question: it looks like Yarn is maybe trying to copy Node itself to a temp folder and run it? Any pointers to how that behavior works and where it's implemented?

To give a bit of context:

• Each time you run a script Yarn creates a directory where it puts all the binary the package is allowed access to (using symlinks).

• But because Windows symlink support is ... subpar at best, tools traditionally shim them using shellscripts (or rather, *.cmd scripts). Unfortunately it has one drawback: cmd.exe, the interpreter, asks confirmation when you press ^C, and it gets annoying real fast, especially if you have nested processes.

• @merceyz tried in this release a new approach where, instead of generating a *.cmd file, we instead generate a *.exe file. This way we control how the spawn is made, and the confirmation on ^C disappears.

• Unfortunately it seems to cause issues with setups like yours 🤔 I guess we'll have to provide a way to toggle it off, although it would be nice to feature-detect it ...

[markerikson]

@arcanis : yeah. Don't want to hijack the SSL thread for that discussion, but it would be really nice to have an alternate way to handle this. Our company has some process-monitoring software installed, and while it's not 100% all-encompassing, it frequently blocks "unknown" executables from executing at all. In this case, the .exe files in the temp folder are all getting killed before they can start, so the post-install steps all fail. (Which is a real shame, because it looks like the rest of the package installation process actually worked fine with the SSL changes, and I would really like to see what actual package dep issues exist after installing via v2.)

Hmm. does v2 have an --ignore-scripts option? Not immediately seeing one.

[merceyz]

Our company has some process-monitoring software installed, and while it's not 100% all-encompassing, it frequently blocks "unknown" executables from executing at all

Do you know what exactly causes it to block a file? Does it have to be signed?

Hmm. does v2 have an --ignore-scripts option? Not immediately seeing one.

It's a configuration https://yarnpkg.com/configuration/yarnrc#enableScripts

yarn config set enableScripts false

[arcanis]

Let's discuss about binjumper in https://github.com/yarnpkg/berry/issues/1938 👍

Thanks @andreialecu for implementing this much requested feature! 🎉

[jasonwilliams]

Note that it should be caFilePath not caFile.

@andreialecu how will people in future know this? Is it added to documentation somewhere?

[andreialecu]

The documentation will be updated before the next release.

If anyone finds this via Google, instructions can be found here: https://github.com/yarnpkg/berry/issues/798#issuecomment-704417635 🙂

[lennyburdette]

Thanks for this! I followed the steps in the comment above and was able to run yarn install with berry for the first time. :100:

I'm now trying to get my project to build and I may have discovered an issue related to this feature. I'm investigating typescript issues and I tried running:

yarn plugin import typescript
➤ YN0001: RequestError: unable to get local issuer certificate

If I remove the new caFilePath config value from .yarnrc.yml, the above command works correctly.

[andreialecu]

Is the CA cert only used for access to specific repositories, instead of the entire internet?

[lennyburdette]

Yes, the CA is for accessing our internal registry (an artifactory instance). I dunno if yarn should fetch plugins via the registry (which is a proxy as well as a private registry) or if it should have a way to use different CAs for difference hosts.

[markerikson]

In our case, the NexusRepository instance is a proxy for all queries to the public NPM registry. The intent is that the package manager is never going out to the public internet itself.

[andreialecu]

This is interesting.

I believe in yarn v1 you could only specify a global cafile, it was not configurable per repository. Correct?

The difference with yarn v2 is that it has additional functionality that uses the nework, such as the one for installing plugins. The PR applies the CA certificate to all network requests, including plugin downloads from github, as a side effect. This makes it work with MITM global proxies though.

Perhaps a way to scope the cafile based on hostnames might help? Similar to:

caFilePath:
  # either:
  "*": <file_path> # for all hosts
  # - or just:
  "npmjs.com": <file_path>

What approach would be best here for your use cases?

[andreialecu]

I opened #1946 which you can install via:
yarn set version from sources --branch 1946

It adds the ability to override the caFilePath per hostname. Example:

networkSettings:
  "github.com":
    caFilePath: <path>

# root caFilePath setting still works:
caFilePath: <path>

[lennyburdette]

The above steps worked for me.

I do wonder if npm/yarnv1 _added_ the CA to the build-in CAs akin to NODE_EXTRA_CA_CERTS. That would explain why they don't need per-host configs.

However, I'm totally fine with this approach. Would it be possible to inline the certs in the yaml? The old npm behavior supported this format:

ca[]="-----BEGIN CERTIFICATE-----\n ..."
ca[]="-----BEGIN CERTIFICATE-----\n ..."

[andreialecu]

Based on this issue: https://github.com/yarnpkg/yarn/issues/6578 it doesn't seem like v1 added the certs.

It's actually hard to add certs to the existing trust list, and much easier to replace it entirely. See https://stackoverflow.com/a/39972054/114732

Regarding inlining the certs, I guess if @arcanis thinks that's a good idea, it would be easy enough to add.

[lennyburdette]

Fair enough!

If you decide not to support inlining, can you expand the path when it includes ~? We'll probably update our company's dev environment setup scripts to place a .pem file in ~/.yarn/ or something.

[arcanis]

We already support environment variables in the Yarnrc, so you should be able to just use $HOME (cf the doc for the exact syntax).