Azure-pipelines-tasks: vsts-npm-auth returns wrong Basic Realm (visualstudio.com instead of azure.com)

[DynaSpan]

(Didn't know in which repo to create this issue so I chose this one)

Required Information

Entering this information will route you directly to the right team and expedite traction.

Question, Bug, or Feature?
Type: bug

Enter Task Name: vsts-npm-auth

Environment

Azure DevOps
npm v6.12.0
vsts-npm-auth v0.37.0
node v11.10.0
Windows 1903

Issue Description

npm install returns the error: Unable to authenticate, need: Basic realm="https://pkgsprodsu3weu.app.pkgs.visualstudio.com/" while our package URL is https://pkgs.dev.azure.com/[xxx]/_packaging/[xxx]/npm/registry/.

This does only happen on some projects; not all. We've migrated all .npmrc in the projects and the user's .npmrc to the dev.azure.com realm a couple of months ago.

vsts-npm-auth was run on all projects .npmrc and works for most projects. Forcing credential updates doesn't resolve it either, whether applied to the project's .npmrc or user's .npmrc.

````
vsts-npm-auth -C C:Users[xxx].npmrc -F

vsts-npm-auth v0.37.0.0

Getting new credentials for source:https://pkgs.dev.azure.com/[xxx]/_packaging/[xxx]/npm/registry/, scope:vso.packaging_write vso.drop_write
````

Troubleshooting

I've tried clearing all NPM caches and node_modules, updating NPM & vsts-npm-auth, regenerating the credentials, removing package-lock.json. But nothing works unfortunately. Updating the credentials returns the correct URL:

````
PS C:Users[xxx]AppDataRoamingnpm> vsts-npm-auth -C C:Users[xxx].npmrc

vsts-npm-auth v0.37.0.0

Already have credentials for https://pkgs.dev.azure.com/[xxx]/_packaging/[xxx]/npm/registry/.
PS C:Users[xxx]AppDataRoamingnpm> vsts-npm-auth -C C:Projects[xxx].npmrc

vsts-npm-auth v0.37.0.0

Already have credentials for https://pkgs.dev.azure.com/[xxx]/_packaging/[xxx]/npm/registry/.
````

Error logs

0 info it worked if it ends with ok
1 verbose cli [ 'C:\\Program Files\\nodejs\\node.exe',
1 verbose cli   'C:\\Users\\[xxx]\\AppData\\Roaming\\npm\\node_modules\\npm\\bin\\npm-cli.js',
1 verbose cli   'i' ]
2 info using [email protected]
3 info using [email protected]
4 verbose npm-session bc10c5eb62b72e40
5 silly install runPreinstallTopLevelLifecycles

[..]

38 silly pacote range manifest for @angular/cli@^8.3.9 fetched in 3036ms
39 http fetch GET 401 https://pkgs.dev.azure.com/[xxx]/_packaging/@[xxx]/npm/registry/@[xxx]%2fcore 100ms
40 silly fetchPackageMetaData error for @[xxx]/core@^0.1.1 Unable to authenticate, need: Basic realm="https://pkgsprodsu3weu.app.pkgs.visualstudio.com/"
41 http fetch GET 401 https://pkgs.dev.azure.com/[xxx]/_packaging/@[xxx]/npm/registry/@[xxx]%2fstyle 27ms
42 silly fetchPackageMetaData error for @[xxx]/style@^0.9.0 Unable to authenticate, need: Basic realm="https://pkgsprodsu3weu.app.pkgs.visualstudio.com/"
43 http fetch GET 200 https://registry.npmjs.org/@ngx-translate%2fcore 416ms
44 silly pacote range manifest for @ngx-translate/core@^11.0.1 fetched in 420ms

[..]

2285 silly pacote range manifest for @ngx-translate/http-loader@^4.0.0 fetched in 11ms
2286 silly resolveWithNewModule @ngx-translate/[email protected] checking installable status
2287 silly pacote range manifest for core-js@^3.3.2 fetched in 17ms
2288 silly resolveWithNewModule [email protected] checking installable status
2289 silly pacote range manifest for rxjs@~6.4.0 fetched in 18ms
2290 silly resolveWithNewModule [email protected] checking installable status
2291 silly pacote range manifest for tippy.js@^4.3.5 fetched in 14ms
2292 silly resolveWithNewModule [email protected] checking installable status
2293 silly pacote range manifest for tslib@^1.10.0 fetched in 15ms
2294 silly resolveWithNewModule [email protected] checking installable status
2295 silly pacote range manifest for zone.js@~0.9.1 fetched in 14ms
2296 silly resolveWithNewModule [email protected] checking installable status
2297 http fetch GET 401 https://pkgs.dev.azure.com/[xxx]/_packaging/@[xxx]/npm/registry/@[xxx]%2fstyle 97ms
2298 http fetch GET 401 https://pkgs.dev.azure.com/[xxx]/_packaging/@[xxx]/npm/registry/@[xxx]%2fcore 98ms
2299 silly fetchPackageMetaData error for @[xxx]/style@^0.9.0 Unable to authenticate, need: Basic realm="https://pkgsprodsu3weu.app.pkgs.visualstudio.com/"
2300 silly fetchPackageMetaData error for @[xxx]/core@^0.1.1 Unable to authenticate, need: Basic realm="https://pkgsprodsu3weu.app.pkgs.visualstudio.com/"
2301 timing stage:rollbackFailedOptional Completed in 1ms
2302 timing stage:runTopLevelLifecycles Completed in 23371ms
2303 silly saveTree [email protected]
2303 silly saveTree +-- @angular/[email protected]
2303 silly saveTree | `-- [email protected]
2303 silly saveTree +-- @angular/[email protected]
2303 silly saveTree +-- @angular/[email protected]
2303 silly saveTree +-- @angular/[email protected]
2303 silly saveTree +-- @angular/[email protected]
2303 silly saveTree +-- @angular/[email protected]
2303 silly saveTree +-- @angular/[email protected]
2303 silly saveTree +-- @angular/[email protected]
2303 silly saveTree +-- @aspnet/[email protected]
2303 silly saveTree +-- @fortawesome/[email protected]
2303 silly saveTree +-- @ngx-translate/[email protected]
2303 silly saveTree +-- @ngx-translate/[email protected]
2303 silly saveTree +-- [email protected]
2303 silly saveTree +-- [email protected]
2303 silly saveTree +-- [email protected]
2303 silly saveTree +-- [email protected]
2303 silly saveTree +-- [email protected]
2303 silly saveTree `-- [email protected]
2304 verbose stack Error: Unable to authenticate, need: Basic realm="https://pkgsprodsu3weu.app.pkgs.visualstudio.com/"
2304 verbose stack     at res.buffer.catch.then.body (C:\Users\[xxx]\AppData\Roaming\npm\node_modules\npm\node_modules\npm-registry-fetch\check-response.js:94:17)
2304 verbose stack     at processTicksAndRejections (internal/process/next_tick.js:81:5)
2305 verbose statusCode 401
2306 verbose pkgid @[xxx]/style@^0.9.0
2307 verbose cwd C:\my\project
2308 verbose Windows_NT 10.0.18362
2309 verbose argv "C:\\Program Files\\nodejs\\node.exe" "C:\\Users\\[xxx]\\AppData\\Roaming\\npm\\node_modules\\npm\\bin\\npm-cli.js" "i"
2310 verbose node v11.10.0
2311 verbose npm  v6.12.0
2312 error code E401
2313 error Unable to authenticate, need: Basic realm="https://pkgsprodsu3weu.app.pkgs.visualstudio.com/"
2314 verbose exit [ 1, true ]

[DynaSpan]

I've somehow managed to fix it by deleting the registry keys at HKEY_CURRENT_USER\SOFTWARE\Microsoft\VSCommon\14.0\ClientServices\TokenStorage\VisualStudio\VssApp and spamming the vsts-npm-auth tool till it finally worked. It still throws an exception but it finally generates a token and I can run npm install :).

````
PS C:ProjectsMy-Project> vsts-npm-auth -F -R -V Detailed -C C:ProjectsMy-Project.npmrc -T C:Users[xxx].npmrc

vsts-npm-auth v0.37.0.0

Parameters:
AuthenticationProviders=wia,federated
Config=C:ProjectsMy-Project.npmrc
Help=False
NonInteractive=False
TargetConfig=C:Users[xxx].npmrc
ExpirationMinutes=129600
ReadOnly=True
Force=True
Verbosity=Detailed
Creating npmrc file. Path: C:Users[xxx].npmrc
INI (resolved): @[xxx]:registry=https://pkgs.dev.azure.com/[xxx]/_packaging/[xxx]/npm/registry/
INI (resolved): always-auth=true
Probing https://pkgs.dev.azure.com/[xxx]/_packaging/[xxx]/npm/registry/
Probe response code: 401 Unauthorized
Credential type: Sps.
Has valid credentials: False.
Getting new credentials for source:https://pkgs.dev.azure.com/[xxx]/_packaging/[xxx]/npm/registry/, scope:vso.packaging
Getting authentication token from https://vssps.dev.azure.com/[xxx]/
Trying authentication provider Windows Integrated Authentication via Azure AD...
Authorization failure while retrieving a session token
Microsoft.VisualStudio.Services.Common.VssUnauthorizedException: VS30063: You are not authorized to access https://vssps.dev.azure.com.
at Microsoft.VisualStudio.Services.Common.VssHttpMessageHandler.d__17.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.VisualStudio.Services.Common.VssHttpRetryMessageHandler.d__3.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.d__46.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.d__431.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<SendAsync>d__271.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.d__261.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.VisualStudio.Services.DelegatedAuthorization.Client.DelegatedAuthorizationHttpClient.<CreateSessionToken>d__11.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.VisualStudio.Services.DelegatedAuthorization.Client.DelegatedAuthorizationHttpClient.<CreateSessionToken>d__8.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.VisualStudio.Services.WebApi.TaskExtensions.SyncResult[T](Task1 task)
at VSS.NuGet.Authentication.SpsAuthTokenProvider.GetAuthToken(Uri spsUri, TimeSpan sessionLength, Boolean allowInteractive, String scope, Boolean discardExistingCredentials)
Trying authentication provider Browser-based federated authentication...
SPS CreateSessionToken Activity ID: [xxx]
CreateSessionToken result: {"ClientId":"00000000-0000-0000-0000-000000000000","AccessId":"[xxx]","AuthorizationId":"[xxx]","HostAuthorizationId":"00000000-0000-0000-0000-000000000000","UserId":"[xxx]","ValidFrom":"2019-10-29T15:10:31.971952Z","ValidTo":"2020-01-27T15:20:18.9719678Z","DisplayName":"[xxx]","Scope":"vso.packaging","TargetAccounts":["[xxx]"],"Token":"[Redacted]","AlternateToken":null,"IsValid":true,"IsPublic":false,"PublicData":null,"Source":null}
Acquired JWT token valid from 29/10/2019 15:10:32 to 27/01/2020 15:20:19 (90.00:09:47 total, 89.23:59:39.9869983 remaining). Note that the server may reject the token before the end date due to revocation, etc.
````

[bastienJS]

I have the exact same error message(s) with the workaround he tried.. My user`s password had expired in azure devops thus the token in my npmrc file was invalid. I just generated a new PAT convert to base64 string and pasted it 2 times in the npmrc. Then npm install worked!

In my case there was no key in the registry set!

[edgarrs]

Please re-open it if this is still happening

[aucampia]

I was having the same error:

$ npm install
npm ERR! code E401
npm ERR! Unable to authenticate, need: Basic realm="https://pkgsprodsu3weu.app.pkgs.visualstudio.com/"

For me this was because my PAT on azure devops expired.

[webbiffy]

In my case the command vsts-npm-auth -config .npmrc generates PAT.
Howerver when running npm install I got an error "403 Forbidden - In most cases, you or one of your dependencies are requesting a package version that is forbidden by your security policy."
When I look at my PAT I can see there's read/write access in packaging, anyone having the same issue?

[JamesHough]

I had the same error message as the OP.

The only way that I could get it to work was to force the generation of a new token using the -f flag. I didn't delete any files or change the registry at all before running. If you are only downloading packages from your private npm on Azure DevOps, then the token can be read-only, you don't need a full access token. Read-only worked for me.

I used these switches to force a new read-only token to be generated on my account. This was the only way I could get the login dialog to appear so that I could provide the correct account to use with my company repo (not my personal login account).

vsts-npm-auth -config .npmrc -r -f -v normal