Npm-check-updates: `HTTPError: Response code 401 (Unauthorized)` on private registry

[inside]

• [x] I have read the list of known issues before filing this issue.

• [x] I have searched for similiar issues before filing this issue.

• node version: 10.15.0

• npm version: 6.5.0
• npm-check-updates version: 3.1.0

---

Hi, using a private registry with authentication needed, here's the output of my terminal with sensitive infos replaced:

% ncu
Checking [path to package.json]
[--------------------] 0/1 0%(node:1207) UnhandledPromiseRejectionWarning: HTTPError: Response code 401 (Unauthorized)
    at EventEmitter.emitter.on ([path to home directory]/.nvm/versions/node/v10.15.0/lib/node_modules/npm-check-updates/node_modules/got/source/as-promise.js:74:19)
    at process._tickCallback (internal/process/next_tick.js:68:7)

If I console.log(error.body) on line 78 of npm-check-updates/node_modules/got/source/as-promise.js, I get the reason of the 401 which is:

[ { status: 401,
       message: 'Failed to decode basic authentication token' } ] }

In my package's folder, I have a .npmrc which contains:

...
# https://docs.npmjs.com/files/npmrc
# > All npm config files are an ini-formatted list of key = value parameters. Environment variables can be replaced using ${VARIABLE_NAME}.
_auth=${MY_NPM_SECRET}
...

However, if I hard code my npm secret in the _auth key, ncu works fine.
I also want to mention that [email protected] works fine.

Can you tell me if this is a ncu or got bug? Or if this can be fixed by upgrading got?
Let me know if you need more informations, thank you!

[stoically]

ncu v3 uses package-json to retrieve package information, which in turn uses rc to parse the .npmrc file. rc however does not replace environment variables, which would be a special case for npm rc files. So this would need to get explicitly implemented in either package-json or rc (I'd say package-json). Here's how yarn did it. ncu v2 directly uses the unofficial npm api which parses the npmrc as expected.

[stoically]

Actually it's package-json's dependency registry-auth-token that uses rc. They handle env vars for :authToken and :_password configurations. The same logic would need to be applied in case of _auth.

[inside]

@stoically thank you! I'll suggest registry-auth-token's maintainer to handle the case of _auth.

[inside]

It turns out there already an open PR for the legacy auth token:
https://github.com/rexxars/registry-auth-token/pull/24

[inside]

https://www.npmjs.com/package/registry-auth-token has just been released from 3.3.2 to 3.4.0 with the above mentionned PR merged, thanks to @rexxars

[raineorshine]

Published ncu v3.1.1 to pick up [email protected]

[stoically]

@raineorshine Not sure, but I think since package-json has the range ^3.3.2 for registry-auth-token new ncu installs or upgrades automatically include 3.4.0.

[raineorshine]

@stoically Yes, but doesn't the package-lock need to be updated?

[stoically]

AFAIK the package-lock.json is only taken into account for npm installs inside a project directory, containing both the package.json and the -lock, but not for the dependencies being installed or global installs. So you're right, the -lock for ncu needs to be updated in order to include the new 3.4.0 for dev-purposes, but defining as direct dependency is probably not necessary. Sorry for nitpicking. 😟

[raineorshine]

Ah! Makes sense! :)

[inside]

Thanks to both of you, now I can have a beautiful output of what needs to be updated in my app. :heart_eyes:

[johannes-z]

@inside @stoically Sorry for necro, but I still have this issue with ncu v.3.1.2. Is there anything I have to do on my end? If I remove the packages that come from my private registry, ncu -u for example works again.

[stoically]

@johannes-z We would need a sample package.json with failing entries and information on how the registry is configured (.npmrc? env vars? scoped? auth method?) to reproduce.

[Rycochet]

Not really got the time to figure out exactly what's going on here, but the specific version that breaks things is 3.0.5, the previous patch works (and I'll not be updating until / unless this gets fixed).

I have two custom repos in the .npmrc - but no access tokens in it. They are stored instead in ~/.npmrc, which I'm assuming isn't read as it's no longer using npm (only looked at the commits quickly).

[stoically]

@Rycochet registry-auth-token uses rc to find and parse .npmrc files from multiple locations (including ~), so in theory it should behave like npm. Could you provide information on how exactly you configured the custom repos and the tokens, so this could potentially get reported to registry-auth-token?

[Rycochet]

My ~/.npmrc (same throughout the team):

save-prefix=~

@fortawesome:registry=https://npm.fontawesome.com/
//npm.fontawesome.com/:_authToken=<snipped>

@<private namespace>:registry=https://<private artifactory>/api/npm/npm/
//<private artifactory>/api/npm/npm/:always-auth=true
//<private artifactory>/api/npm/npm/:_authToken=<snipped>

Individual projects are a lot shorter and just say "this is namespaced" - project/.npmrc:

@fortawesome:registry=https://npm.fontawesome.com/
@<private namespace>:registry=https://<private artifactory>/api/npm/npm/

Not sure on the server that FortAwesome uses (FontAwesome pro license - the access is public hence listed here), but our company uses Artifactory Professional 6.8.3 rev 60803900 behind the company firewall (shouldn't make much of a difference). The /npm/npm on the end is deliberate - and ugly (and works lol)

[stoically]

Thanks. I've tested with that configuration, and in my case registry-auth-token returns the expected token for the scope. So this would need a bit more involved debugging to find what's actually the cause:

Clone ncu, npm install, add a console.log(packageUrl, authInfo); at line 43 in node_modules/package-json/index.js, execute the cloned ./bin/ncu from the project in question and see if you can spot whether the wrong registry or authInfo is used for the failing package?

[Rycochet]

@stoically It's not reporting which package is actually failing, but it is failing on 58/137 - so checked that and it's a normal npmjs one (the failing one is different every time, edit: the logged order is not the same as the install order apparently). The two namespaced entries are getting the correct tokens in there, so it's definitely reading the right files.

I tried deleting the entries in my package.json - and it's definitely the local Artifactory ones that are causing problems (@fortawesome is working correctly).

Basically, I'm at a loss here. The only thing I can think might be relevant is that Artifactory has issues with various things relating to npm, and it's doing something that breaks the new way of doing things, but works perfectly when npm itself is doing the checking (hence why it broke in 3.0.5)

edit2:

I've updated, and added an exclude to my ~/.ncurc.json file (because everything else is significantly better, and I know the exact versions available locally) -

{
    "reject": "/@<private namespace>\/.*/"
}

[stoically]

@Rycochet Interesting. So it probably comes down to some specific header being sent which Artifactory is checking, that is set by npm but not package-json/got.

I tried to find a difference, but

node_modules/got/request-as-eventemitter.js L234, console.log(options), running ncu

{ path: '/@scope%2Ftest',
  protocol: 'https:',
  hostname: 'registry.npmjs.org',
  hash: '',
  search: '',
  pathname: '/@scope%2Ftest',
  href: 'https://registry.npmjs.org/@scope%2Ftest',
  headers:
   { 'user-agent': 'got/9.6.0 (https://github.com/sindresorhus/got)',
     accept:
      'application/vnd.npm.install-v1+json; q=1.0, application/json; q=0.8, */*',
     authorization: 'Bearer xyz',
     'accept-encoding': 'gzip, deflate' },

npm/node_modules/node-fetch-npm/src/index.js L54, console.log(options), running npm install

{ protocol: 'https:',
  slashes: true,
  auth: null,
  host: 'registry.npmjs.org',
  port: null,
  hostname: 'registry.npmjs.org',
  hash: null,
  search: null,
  query: null,
  pathname: '/@scope%2ftest',
  path: '/@scope%2ftest',
  href: 'https://registry.npmjs.org/@scope%2ftest',
  method: 'GET',
  headers:
   [Object: null prototype] {
     connection: [ 'keep-alive' ],
     'user-agent': [ 'npm/6.4.1 node/v10.15.3 linux x64' ],
     'npm-in-ci': [ 'false' ],
     'npm-scope': [ '' ],
     'npm-session': [ 'xyz' ],
     referer: [ 'install' ],
     'pacote-req-type': [ 'packument' ],
     'pacote-pkg-id': [ 'registry:manifest' ],
     accept:
      [ 'application/vnd.npm.install-v1+json; q=1.0, application/json; q=0.8, */*' ],
     authorization: [ 'Bearer xyz' ],
     'accept-encoding': [ 'gzip,deflate' ] }

seem pretty similar. Some missing headers in case of got, but nothing that seems really relevant (and lowercase %2f, but that shouldn't make a difference either).

Maybe you could try to add those two logs in your environment and see if you can spot a difference that seems more relevant? (a package.json with only one artifactory dependency might be helpful) And as a last resort you could consider contacting artifactory and ask them why your requests are being rejected with correct authorization headers.

As a side-note, package-json is actually also used by npm through update-notifier.

[Rycochet]

Gah - found the problem, and it's partly me mis-reading the error message - I completely cleared my npm cache and did an npm ci - which failed with a certificate error. The same error as ncu - if I then add strict-ssl=false back into .nmprc then npm succeeds, where this fails.

RequestError: unable to verify the first certificate
    at ClientRequest.request.once.error (/usr/local/lib/node_modules/npm-check-updates/node_modules/got/source/request-as-event-emitter.js:178:14)
    at Object.onceWrapper (events.js:277:13)
    at ClientRequest.emit (events.js:194:15)
    at ClientRequest.origin.emit.args (/usr/local/lib/node_modules/npm-check-updates/node_modules/@szmarczak/http-timer/source/index.js:37:11)
    at TLSSocket.socketErrorListener (_http_client.js:392:9)
    at TLSSocket.emit (events.js:189:13)
    at emitErrorNT (internal/streams/destroy.js:82:8)
    at emitErrorAndCloseNT (internal/streams/destroy.js:50:3)
    at process._tickCallback (internal/process/next_tick.js:63:19)

I had thought this had been fixed, but apparently not. The certificate itself is a properly signed and valid wildcard certificate for the domain - and when I found the original post about strict-ssl I did see the bug report about it in Artifactory (but a quick search now doesn't find it, and can't dedicate more time to searching).

Feel free to pull this out into a separate issue if necessary, though got a feeling that this should be pushed upstream to get whichever package is relevant to read and follow that directive :-)

[stoically]

I've opened #528 to deal with that. Could you try npx npm-check-updates@stoically/npm-check-updates#ssl-support and see if that indeed fixes it for you?

[Rycochet]

Doesn't fix it sadly - timeout instead:

RequestError: connect ETIMEDOUT <redacted>:443
    at ClientRequest.request.once.error (/Users/rycochet/.npm/_npx/37411/lib/node_modules/npm-check-updates/node_modules/got/source/request-as-event-emitter.js:178:14)
    at Object.onceWrapper (events.js:277:13)
    at ClientRequest.emit (events.js:194:15)
    at ClientRequest.origin.emit.args (/Users/rycochet/.npm/_npx/37411/lib/node_modules/npm-check-updates/node_modules/@szmarczak/http-timer/source/index.js:37:11)
    at TLSSocket.socketErrorListener (_http_client.js:392:9)
    at TLSSocket.emit (events.js:189:13)
    at emitErrorNT (internal/streams/destroy.js:82:8)
    at emitErrorAndCloseNT (internal/streams/destroy.js:50:3)
    at process._tickCallback (internal/process/next_tick.js:63:19)

[stoically]

That's, uhm, interesting - a timeout seems completely unrelated. If I had to guess I'd say it has something to do with executing it through npx, though it works for me. I've tested with registry=https://expired.badssl.com/ and strict-ssl=false. Maybe, if you still have that cloned ncu laying around, could you try git fetch origin pull/528/head:ssl-support, git checkout ssl-support and execute the cloned ./bin/ncu again?

[Rycochet]

Ok, you can officially call me stupid (or distracted) - needed to work from home, and nothing else I've been up to today needed me to connect to the VPN... So I didn't... Erm...

That fixes it!!!

edit: Github needs a facepalm emoji!

[inside]

Hi,

For me it works with npm-check-updates@stoically/npm-check-updates#ssl-support but not with npm-check-updates@latest. Please see the output of my terminal. Did the bug come back?

% npx npm-check-updates@stoically/npm-check-updates#ssl-support
npx: installed 157 in 5.277s
Checking [path to package.json]
[====================] 108/108 100%

 @babel/core                               7.4.4  →   7.5.0
 @babel/plugin-proposal-class-properties   7.4.4  →   7.5.0
 @babel/plugin-transform-runtime           7.4.4  →   7.5.0
 @babel/preset-env                         7.4.4  →   7.5.0
 @babel/runtime                            7.4.4  →   7.5.0
 etc...

% npx npm-check-updates@latest
npx: installed 254 in 7.343s
Checking [path to package.json]
[--------------------] 1/108 0%[path to home directory]/.npm/_npx/26860/lib/node_modules/npm-check-updates/lib/npm-check-updates.js:387
        throw err;
        ^

Error: 403 Forbidden - GET https://my-private-registry-domain/path-to-packages/@babel%2fpreset-react
    at res.buffer.catch.then.body ([path to home directory]/.npm/_npx/26860/lib/node_modules/npm-check-updates/node_modules/npm-registry-fetch/check-response.js:104:15)
    at process._tickCallback (internal/process/next_tick.js:68:7)

[stoically]

Yeah, currently ncu doesn't support auths, see https://github.com/tjunnone/npm-check-updates/issues/556#issuecomment-508704268, it's in the pipeline to get fixed over in #557