Npm-check-updates: v3.1.13 either doesn't works with private registries

[yvele]

• [x] I have read the list of known issues before filing this issue.

• [x] I have searched for similiar issues before filing this issue.

• node version: 10.16.0 (also reproduced the bug with 8.10.0)

• npm version: 6.10.0 (also reproduced the bug with 6.9.0)
• npm-check-updates version: 3.1.13

---

Related issue https://github.com/tjunnone/npm-check-updates/issues/547 about the same issue with v3.1.12

v3.1.13 with #548 being merged was supposed to fix the bug but it looks like not to be working.

As a workaround I downgraded npm-check-updates to v3.1.11

Edit: Ho.. and thank you for that amazing tool that my team and myself are loving ❤️

[stoically]

We'd need some more information to help. Does it just time out? Any error? Do you have proxy / no_proxy configured, if so, how? How did you configure the private registry, scoped / unscoped?

[yvele]

I have a scoped private registry.
No timeout, no error, npm-check-updates acts like the private packages are ignored.

I don't use any proxy / no_proxy configuration (I use npm registry directly).

Also this is the content of my .npmrc

//registry.npmjs.org/:_authToken=${NPM_TOKEN}
access = restricted

# Configure how versions of packages are saved to package.json
save-prefix = '~'

As you see I'm using _authToken with an environment variable

[ioanlucut]

Having the same issue, but with a different symptom:

tes/lib/npm-check-updates.js:387
        throw err;
        ^

Error: Unable to authenticate, need: Basic realm="Artifactory Realm"
    at res.buffer.catch.then.body (/Users/ilucut/.nvm/versions/node/v10.15.0/lib/node_modules/npm-check-updates/node_modules/npm-registry-fetch/check-response.js:94:17)
    at process._tickCallback (internal/process/next_tick.js:68:7)

[stoically]

I see, pacote + libnpmconfig doesn't play with scoped registries or auth tokens yet.

Could you try npx npm-check-updates@stoically/npm-check-updates#registry-url-and-auth-token and see if that works for you?

[yvele]

@stoically nope the command is not working and is ignoring private packages 😞
Downgrading to v3.1.11 is definitely working

[stoically]

@yvele Could you try npx npm-check-updates@stoically/npm-check-updates#registry-url-and-auth-token-debug and see if it logs the correct registry url and token to the console?

[yvele]

npx npm-check-updates@stoically/npm-check-updates#registry-url-and-auth-token-debug
npx: installed 255 in 21.464s
Checking /Users/yvele/repo/package.json
[--------------------] 0/53 0%
https://registry.npmjs.org/ { token: '00000000-0000-0000-0000-000000000000', type: 'Bearer' }
https://registry.npmjs.org/ { token: '00000000-0000-0000-0000-000000000000', type: 'Bearer' }
https://registry.npmjs.org/ { token: '00000000-0000-0000-0000-000000000000', type: 'Bearer' }
...

Yes both the registry URL and the displayed token are the good one 👍

[stoically]

I see, thanks. Could you try https://github.com/tjunnone/npm-check-updates/issues/556#issuecomment-508704268 again? Should work now.

[yvele]

Yes it works! 👍

[inside]

Here's what I have, the 'secret' coming from the debug version is the correct token I have locally:

% npx npm-check-updates@stoically/npm-check-updates#registry-url-and-auth-token-debug
npx: installed 255 in 9.925s
Checking [path to package.json]
[--------------------] 0/108 0%https://my-private-registry-domain/path-to-packages/ { token: 'secret', type: 'Basic' }
etc...
[--------------------] 1/108 0[path to home directory]/.npm/_npx/32073/lib/node_modules/npm-check-updates/lib/npm-check-updates.js:387
        throw err;
        ^

Error: Unable to authenticate, need: Basic realm="MyRegistryName Realm"
    at res.buffer.catch.then.body [path to home directory]/.npm/_npx/32073/lib/node_modules/npm-check-updates/node_modules/npm-registry-fetch/check-response.js:94:17)
    at process._tickCallback (internal/process/next_tick.js:68:7)

% npx npm-check-updates@stoically/npm-check-updates#registry-url-and-auth-token
npx: installed 255 in 10.01s
Checking [path to package.json]
[--------------------] 1/108 0%[path to home directory]/.npm/_npx/32364/lib/node_modules/npm-check-updates/lib/npm-check-updates.js:387
        throw err;
        ^

Error: Unable to authenticate, need: Basic realm="MyRegistryName Realm"
    at res.buffer.catch.then.body ([path to home directory]/.npm/_npx/32364/lib/node_modules/npm-check-updates/node_modules/npm-registry-fetch/check-response.js:94:17)
    at process._tickCallback (internal/process/next_tick.js:68:7)

[stoically]

Is the token type reported from the debug version Basic?

Because it seems that underlying npm libs only send basic auth via username and password if no token is specified.

[rosskevin]

We are seeing this issue with scoped private packages from the npmjs registry.

I can also confirm that @stoically's branch has the working fix (from https://github.com/tjunnone/npm-check-updates/issues/556#issuecomment-508704268).

[sbmw]

I raised the original #547 issue about v3.1.12 not working with private registries. Your original fix in v3.1.13 worked for me, but ncu is no longer picking up updates from my private registry with v3.1.14 or v3.1.15.

As a guess, this may be as my .npmrc contains a "_password=" (obfuscated somehow) and not a "_authtoken=" field? If that's the case I'd be more than happy to revise my .npmrc, but it was created via "npm login" so I'm not sure what I should be doing differently to get an authtoken instead of a password?

[stoically]

Could you provide the full .npmrc configuration strings, as it's otherwise hard to determine what's going on? Thanks!

[sbmw]

Sure!

@myscope:registry=https://myregistry.net/
//myregistry.net/:_password=PASSWORD
//myregistry.net/:username=USER
//myregistry.net/:email=EMAIL
//myregistry.net/:always-auth=true

[sbmw]

That config is generated with the following command:

npm login --registry=https://rmyregistry.net/ --scope=@myscope --always-auth

[stoically]

Thanks, that was helpful. The fix should be in #560. Could you check npx npm-check-updates@stoically/npm-check-updates#basic-auth-token and see if it works for you?

Probably also covers @inside's issue.

[inside]

Is the token type reported from the debug version Basic?

Yes, Basic.

Because it seems that underlying npm libs only send basic auth via username and password if no token is specified.

I should fall into the case of _auth being present:

...
  } else if (shouldAuth && auth._auth) {
    headers.authorization = `Basic ${auth._auth}`
  }
...

Because my .npmrc is configured as:

registry=https://my-private-registry.domain/path/to/packages
_auth=${NPM_SECRET}
email=${USER}
always-auth=true

[inside]

npx npm-check-updates@stoically/npm-check-updates#basic-auth-token gives me:

...
Error: Unable to authenticate, need: Basic realm="Artifactory Realm"`
...

[stoically]

Oh well, seems like I simply missed the last else branch. #561 should have the fix. npx npm-check-updates@stoically/npm-check-updates#basic-legacy-auth-token to give it a spin.

[inside]

npx npm-check-updates@stoically/npm-check-updates#basic-legacy-auth-token works for me! :tada:

[sbmw]

I'm really sorry but the latest updates (and so v3.1.17) do not work for me. I've tested extensively and found out how to use _auth instead of _password with my registry (proget) but updates to private packages are just ignored.

My .npmrc now looks like the following:

@myscope:registry=https://myregistry.net/path/to/packages/
@myscope:registry=https://myregistry.net/path/to/packages/:_auth=TOKEN
@myscope:registry=https://myregistry.net/path/to/packages/:always-auth=true

Interestingly, if I set the _auth token to something deliberately incorrect then "npm install" etc will throw a 401 error as expected. However neither ncu -u (wth v 3.1.17) nor your two test branches throw any errors at all.

Edit: using a deliberately incorrect token with v3.1.13 throws a 401 as expected. It seems that the latest version is just ignoring scoped registries?

[stoically]

That specific form of putting the auth info into the scoped configuration is not supported by the underlying package registry-auth-token. So that would need the feature implemented by them. (it's also used by package-json)

If you configure it like described in your last https://github.com/tjunnone/npm-check-updates/issues/556#issuecomment-508989912 it should work.

[sbmw]

Ah I see.

Unfortunately, my original configuration as per https://github.com/tjunnone/npm-check-updates/issues/556#issuecomment-508989912 doesn't work. That's why I started playing around with _auth as an alternative.

Sorry that I wasn't able to test your fix branches sooner but I was travelling.

[stoically]

You could put a console.log(npmConfig); here and check whether registry, token, username, password or _auth are what you expect them to be.

[stoically]

The npmConfig is passed down to npm-registry-fetch (_auth is deprecated, btw)

[sbmw]

Console.log output for a package.json with 4 dependencies as below. Looks correct. The first package was private/scoped and the remaining packages were from npmjs.

The username and password are correct for the private registry (the only difference being that the password is base64 encoded in .npmrc but plain text in npmConfig - which seems correct).

The username and password for the private registry are passed to https://registry.npmjs.org/. That doesn't seem right, but it doesn't seem to matter.

If I use a deliberately incorrect _password in .npmrc there are no errors.

{
registry: 'https://myregistry.net/path/to/packages/',
username: 'USER',
password: 'PASSWORD'
}
{
registry: 'https://registry.npmjs.org/',
username: 'USER',
password: 'PASSWORD'
}
{
registry: 'https://registry.npmjs.org/',
username: 'USER',
password: 'PASSWORD'
}
{
registry: 'https://registry.npmjs.org/',
username: 'USER',
password: 'PASSWORD'
}

[stoically]

Yeah, auth info should not carry over to the next request, I'll provide a PR for that. But besides that, this looks correct to me. The only errors catched by ncu are 404s.

To see whether it's npm-registry-fetch not doing the right you could try await require('npm-registry-fetch').json('https://myregistry.net/path/to/packages/package', {username: 'plain', password: 'plain'})

[sbmw]

The out put to await require('npm-registry-fetch').json('https://myregistry.net/path/to/packages/package', {username: 'plain', password: 'plain'}) is:

{ db_name: 'registry', doc_count: 1, doc_del_count: 0 }

[stoically]

It seems that you gave it the URL to the registry, not to the package.

[sbmw]

Yep, sorry, my bad.

Changing the path to "https://registry/@scope/package/" throws a json parsing error as the call returns an HTML page: the Proget UI details screen for the package.

Basically, something like this example:

I suppose either a different URL format or an additional request header is required in order for the proget registry to return JSON rather than the UI. I'll do some more testing, but it was working with pacote in 3.1.13, so it's something in https://github.com/tjunnone/npm-check-updates/commit/1c47ea3cb1a4c9f1b20330150bc2ef1e439e02b5 that stopped things working for me.

[stoically]

That's interesting - in 3.1.13 no scoped registry urls or auth infos attached to registry urls were read and passed on to pacote at all. So you could try 3.1.13 and remove those scope/auth-configs and it should give you the same result.

[rwieruch]

In my case 3.1.13 worked fine, but upgrading gave me the bug https://github.com/tjunnone/npm-check-updates/issues/565.

[stoically]

How did you configure your registry url and auth info exactly? And does npx npm-check-updates@stoically/npm-check-updates#authinfo-cleanup2 work for you?

[sbmw]

@stoically sorry, haven't been able to reply for a few days.

ncu -v => 3.1.13

1) .npmrc as per https://github.com/tjunnone/npm-check-updates/issues/556#issuecomment-508989912:

ncu => picks up updates in npm and private packages

2) Remove scope declaration before registry=https... in .npmrc

ncu => only detects private package updates

3) Remove auth fields in .npmrc

ncu => 401 for private packages

4) Clear .npmrc

ncu => only discovers npm package updates

npx npm-check-updates@stoically/npm-check-updates#authinfo-cleanup2 throws a 401 for private packages, when my .npmrc is as per https://github.com/tjunnone/npm-check-updates/issues/556#issuecomment-508989912

[stoically]

Thanks for the follow-up. So I took another look at npm-registry-fetch, and realized it indeed does support scoped registries and auth infos, which is why 3.1.13 works for you just like that. I'm not exactly sure why the current code doesn't work as expected for you - but patching out the scope and auth info code seems nice anyway.

That left me with the question where the original issue from @yvele came from, and it turns out that libnpmconfig doesn't support env vars in configs - while registry-auth-token accounts for them - so that would need to be addressed - I'll put together a PR for libnpmconfig.

[sbmw]

Hey, thanks for all your hard work on this!

I just found something too that may or may not help you.

My testing on the later versions (e.g. ncu 3.1.18) run, find no package updates, but also throw no errors.

I was testing with a real (but simple) project, containing one private package dependency and a couple of npmjs.org dependencies.

However, when I strip out the global npm packages from package.json and leave only the private package, ncu DOES throw a 401 unauthorized error!

The output of

shows the correct registry, username and password in npmConfig

[sbmw]

Yes, with something like this:

ncu "falls back" (?) to the global registry and tries to get the private package from registry.npmjs.org, then throws an uncaught 404.

uses the correct registry, but fails with a 401 even though the credentials in npmConfig appear to be correct.

Strange! :)

[stoically]

Thanks for digging into this. I've opened PR #566 to get rid of the custom registry/auth handling. You could give npx npm-check-updates@stoically/npm-check-updates#libnpmconfig-env-vars a spin to see whether everything's OK (again).

That should also fix it for @rwieruch and @yvele.

[sbmw]

That works for me!