Berry: [Bug] Yarn 2 Response code 404 (not found) when installing from jfrog

[DimitrK]

Describe the bug

During yarn install the following error appears

➤ YN0000: ┌ Resolution step
➤ YN0001: │ HTTPError: @acme/sdk@npm:1.18.2: Response code 404 (Not Found)
    at EventEmitter.l.on (/Users/dimitriskyriazopoulos/Development/ui/.yarn/releases/yarn-sources.js:24:328280)
    at process._tickCallback (internal/process/next_tick.js:68:7)
➤ YN0000: └ Completed in 2.48s
➤ YN0000: Failed with errors in 2.48s

To Reproduce

Create a Jfrog account
Navigate to library workspace
Set proper config credentials to .npmrc and point repository to jfrog
Use Yarn 1 to publish the package
Create another project with dependency the package published earlier
Switch to Yarn 2
Delete any yarn.lock if exists
Make a yarn install

Environment if relevant (please complete the following information):

• OS: OSX
• Node version 10.16.3
• Yarn version tested both 2.0.0-rc.29 & 2.0.0-rc.29.git.20200305.31ef68ca

Additional context

Some of company's in-house packages are being published and served by JFrog. The credentials and repository info are being found to .npmrc in the following fashion:

_.npmrc_

registry=https://acme.jfrog.io/acme/api/npm/npm/
_auth = ${ARTIFACTORY_AUTH}
email = ${ARTIFACTORY_USERNAME}
always-auth = true

Having a dependency on package.json that is being served from JFrog's artifactory (private repository)

_package.json_

"dependencies": {
    "@acme/sdk": "1.18.2",
}

Other approaches tried
Checking the documentation noticed some configuration options that could be set to .yarnrc.yml

Then I executed the following commands

yarn config set npmAlwaysAuth true
yarn config set npmRegistryServer https://acme.jfrog.io/acme/api/npm/npm/
yarn config set npmAuthToken $ARTIFACTORY_AUTH

The error message was:

➤ YN0000: ┌ Resolution step
➤ YN0041: │ webpack-cli@npm:3.3.0: Invalid authentication (as an unknown user)
➤ YN0000: └ Completed in 1.66s
➤ YN0000: Failed with errors in 1.66s

The same happened when removed npmAuthToken and executed

yarn config set npmAuthIdent $ARTIFACTORY_USERNAME:$ARTIFACTORY_AUTH

[Tirke]

I have the same problem. Can't find a working config for private artifactory access.

[deini]

I'm not using artifactory but this is the config I'm using for scoped packages, can you guys try something like this in your yarnrc.yml config?

npmRegistries:
  //api.bintray.com/npm/my-company/npm-private:
    npmAlwaysAuth: true
    npmAuthToken: <token>

npmScopes:
  my-company:
    npmRegistryServer: https://api.bintray.com/npm/my-company/npm-private
    npmPublishRegistry: https://api.bintray.com/npm/my-company/npm-private

Are you guys using artifactory as a proxy?

[Tirke]

@deini

I've used exactly that and I'm getting Invalid authentication (as an unknown user) on the first scoped package.

npmRegistries:
  //artifactory.tools.<any>.com/artifactory/api/npm/data-acquisition-da-npm-upload/:
    npmAlwaysAuth: true
    npmAuthIdent: "user:pass"

npmScopes:
  da:
    npmRegistryServer: https://artifactory.tools.<any>.com/artifactory/api/npm/data-acquisition-da-npm-upload/
    npmPublishRegistry: https://artifactory.tools.<any>.com/artifactory/api/npm/data-acquisition-da-npm-upload/

This is my current config in .npmrc, working with yarn v1.

@da:registry=https://artifactory.tools.<any>.com/artifactory/api/npm/data-acquisition-da-npm-upload/
//artifactory.tools.<any>.com/artifactory/api/npm/data-acquisition-da-npm-upload/:_password="password"
//artifactory.tools.<any>.com/artifactory/api/npm/data-acquisition-da-npm-upload/:username=username
//artifactory.tools.<any>.com/artifactory/api/npm/data-acquisition-da-npm-upload/:email=email
//artifactory.tools.<any>.com/artifactory/api/npm/data-acquisition-da-npm-upload/:always-auth=true

[piqueme]

I'm having the same issue as well.
I wonder if this is because Yarn 2 doesn't send along an email for identification, while JFrog requires it.

Comment about email not being sent: https://github.com/yarnpkg/berry/issues/930#issuecomment-584190605
JFrog NPM Repo docs (ctrl+f for "email"): https://www.jfrog.com/confluence/display/JFROG/npm+Registry

Would there be any way for Yarn 2 to send the email as well? Possibly via an extra config option?

[DimitrK]

@deini you are right, we are using artifactory as a proxy so we can't have per-scope configuration in our case.

As @piqueme mentioned already the missing email might be the issue since JFrog states clearly in docs the following

Your email address (npm publish will not work if your email is not specified in .npmrc)

[deini]

I had some time today to set up a free-trial with Artifactory and was able to reproduce what you guys are seeing. However, I was able to both publish and install from it. There are a couple of scenarios here and I'll try to address them all.

First, we need to clarify that all authentication configs must go in your repo's .yarnrc.yml or your global one (~/.yarnrc.yml). Yarn v2 doesn't read anything from .npmrc.

---

Starting with @DimitrK's scenario, Artifactory as a proxy (no per-scope config):
From the original issue description, I see that you tried doing

yarn config set npmAlwaysAuth true
yarn config set npmRegistryServer https://acme.jfrog.io/acme/api/npm/npm
yarn config set npmAuthToken $ARTIFACTORY_AUTH

This was really close! The issue here is that $ARTIFACTORY_AUTH is the basic auth and not the npmAuthToken. What you can do is:

yarn config set npmAlwaysAuth true
yarn config set npmRegistryServer https://acme.jfrog.io/acme/api/npm/npm
yarn npm login

This will ask for your username and password which will be exchanged for an npmAuthToken

Your repo's .yarnrc.yml will end up with the following config

npmAlwaysAuth: true
npmRegistryServer: "https://acme.jfrog.io/acme/api/npm/npm"

Then on your global config ~/.yarnrc.yml you can see the npmAuthToken is now set

npmRegistries:
  "https://acme.jfrog.io/acme/api/npm/npm":
    npmAuthToken: <TOKEN>

---

@Tirke's scenario, Scoped packages:

As @piqueme mentioned already the missing email might be the issue since JFrog states clearly in docs the following:
"Your email address (npm publish will not work if your email is not specified in .npmrc)"

This is probably the issue since like you guys already figured out, we are not sending the email. However, this section of the Artifactory docs is under the "Basic Auth" strategy (npmAuthIdent) which we strongly discourage.

We have a yarn npm login --scope <scope> however the scope has to be defined. The easiest way to do this is to manually add the scope to your .yarnrc.yml.

npmScopes:
  acme:
    npmRegistryServer: https://acme.jfrog.io/acme/api/npm/npm
    npmPublishRegistry: https://acme.jfrog.io/acme/api/npm/npm

Then you can do:
yarn npm login --scope acme

Which will ask for login/password, exchange it for an npmAuthToken and store it in ~/.yarnrc.yml.

Let me know if you guys have any more questions.

CC: @piqueme

[piqueme]

@deini Thanks so much for the detailed response! I was stuck on trying to use the "Basic Auth" strategy - you're right that if I switch to using tokens everything works out. This is a bit of change, but I guess it's not too interrupting for the security.

[deini]

Closing as I don't see anything actionable on our side.

[DimitrK]

Thanks @deini .

Although yarn npm login works for users who have simple login credentials, it won't work for business users with SAML SSO integrations to JFrog.

In such case you are being connected with the SSO provider when navigating through browser and your login is transparent. No passwords are issued and there is no access in setting a password within JFrog user profile.

A possible solution would be to add a browser login on yarn npm login spawned from CLI similar to what Heroku CLI does.

I will also open a ticket on them in order to get some feedback on that. I feel this limitation will stop many companies which are using JFrog + SSO switching to Yarn 2

[deini]

@DimitrK Good points, how is this handled by npm? Is there a way that you can get the npmAuthToken from Jfrog's UI?

Since I don't have access to a SAML SSO + Jfrog registry, I can't really try it out.

[DimitrK]

@deini Currently jFrog provides SSO users with an API key .

This is being handled right now as per documentation for Basic authentication .

In short, setting the registry and 3 more fields in .npmrc (example values):

registry=https://acme.jfrog.io/acme/api/npm/npm/
_auth =<API_KEY>
email = [email protected]
always-auth = true

[deini]

@DimitrK Is it really an API_KEY or your user:password base64 encoded? That's what I'm getting from the docs.

Can you try to decode it and use npm login and use that username/password?

[DimitrK]

@deini good news, it seems that API_KEY can be actually used as a password on its entirely. So doing yarn npm login and entering email as username and API_KEY as password works like a charm.

I also had to remove any node_modules folder for this yarn install to run properly although that should be irrelevant.

Thanks for your support on this.

[inomdzhon]

Hi there

I did the same commands

yarn config set npmAlwaysAuth true
yarn config set npmRegistryServer https://jfrog.tech/api/npm/frontend-local
yarn npm login

all done, I can do yarn npm publish too, but...

yarn add lodash

I got this

➤ YN0027: lodash@unknown can't be resolved to a satisfying range:

HTTPError: Response code 404 (Not Found)
    at o.<anonymous> (/Users/i.mirdzhamolov/projects/ui-kit/.yarn/releases/yarn-2.2.2.cjs:23:12912)
    at processTicksAndRejections (internal/process/task_queues.js:97:5)

➤ Errors happened when preparing the environment required to run this command.

When I delete npmRegistryServer from my local .yarnrc.yml – yarn add lodash works, but then doesn't work yarn npm publish...

How I can resolve this issue? 😞