Home: `dotnet restore` fails with auth'd (VSTS) package source

[livarcocc]

_From @Mardoxx on September 11, 2017 15:17_

Steps to reproduce

1. Create new project and target netstandard (e.g. netcoreapp2.0, netstandard2.0)
2. Nuget.Config with custom package source e.g.

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <packageSources>
    <add key="NuGet" value="https://api.nuget.org/v3/index.json" />
    <add key="Common" value="https://mysource.pkgs.visualstudio.com/_packaging/Common/nuget/v3/index.json" />
  </packageSources>
</configuration>

1. dotnet restore
2. Restore fails

Expected behavior

:)

Actual behavior

:(

C:\Program Files\dotnet\sdk\2.1.0-preview1-006784\NuGet.targets(102,5): error : Failed to retrieve information about 'Microsoft.NETCore.App' from remote source 'https://mysource.pkgs.visualstudio.com/_packaging/3df2ef6b-68fa-412c-90fa-bcee06e30045/nuget/v3/flat2/microsoft.netcore.app/index.json'. [.csproj]
C:\Program Files\dotnet\sdk\2.1.0-preview1-006784\NuGet.targets(102,5): error :   Response status code does not indicate success: 401 (Unauthorized). [.csproj]

Restore packages works fine in VS2017.

Completely buggers up any dotnet ef commands too as these rely on the project building successfully!

Environment data

dotnet --info output:

.NET Command Line Tools (2.1.0-preview1-006784)

Product Information:
 Version:            2.1.0-preview1-006784
 Commit SHA-1 hash:  0b507e68fd

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.15063
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   C:\Program Files\dotnet\sdk\2.1.0-preview1-006784\


Microsoft .NET Core Shared Framework Host



  Version  : 2.0.0

  Build    : e8b8861ac7faf042c87a5c2f9f2d04c98b69f28d

---

Happens with net461 tfm too, so not really sure what the problem is here..

_Copied from original issue: dotnet/cli#7629_

[livarcocc]

_From @Mardoxx on September 11, 2017 17:45_

I think this is because I'm using a VSTS feed..

[livarcocc]

_From @Mardoxx on September 11, 2017 17:47_

Similar to https://github.com/dotnet/cli/issues/3977

[livarcocc]

_From @Mardoxx on September 11, 2017 17:52_

See also: https://www.visualstudio.com/en-us/docs/package/nuget/dotnet-exe

To use dotnet with VSTS NuGet feeds, you'll need to specify a Personal Access Token in plain text.
On Windows, NuGet 4 is functionally equivalent to dotnet, and we recommend using that instead.

[zhili1208]

@Mardoxx this is because you are using VSTS feed, nuget was trying to connect this feed but didn't have Personal Access Token

[Mardoxx]

Will it always need PAT?

On 12 Sep 2017 12:47 am, "Zhi Li" [e-mail] wrote:

@Mardoxx https://github.com/mardoxx this is because you are using VSTS
feed, nuget was trying to connect this feed but didn't have Personal Access
Token

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/NuGet/Home/issues/5874#issuecomment-328689909, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAOoMbQTc6LmYlLZScYzdirruIGNbNkKks5shcZ5gaJpZM4PTzBG
.

[zhili1208]

Yes, nuget gather dependency information from all enabled source, if the source is enabled, nuget will try to connect it. The workaround is disable VSTS feed or specify "--ignore-failed-sources"

[Mardoxx]

How does VS2017 restore packages without need for PAT? Must use a different
auth mechanism - why can't nuget do this?

Sorry - not too clued up on internals!

On 12 Sep 2017 6:59 pm, "Zhi Li" [e-mail] wrote:

Yes, nuget gather dependency information from all enabled source, if the
source is enabled, nuget will try to connect it. The workaround is disable
VSTS feed or specify "--ignore-failed-sources"

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/NuGet/Home/issues/5874#issuecomment-328933348, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAOoMWPXgaIkHhqtIPEGLKGBRGbxgav0ks5shsZjgaJpZM4PTzBG
.

[zhili1208]

it's actually the same, nuget sends two http requests to two sources(VSTS and NuGet.org here) for fetching the package information at the same time. If the VSTS return first, then you will get the 401 error because it can't find PAT, and if nuget.org return first and it find the package in this feed, then it will just use this one then ignore the VSTS one.

[Mardoxx]

Are you certain? I have my two feeds set up, the default nuget one and my
private VSTS one - the latter with packages that do not exist on nuget.org

• and it restores my packages just fine, never had to generate a PAT until
  I started playing with the dotnet command.

VS2017 restores all my packages - from both sources - without theneed of a PAT.

On 12 Sep 2017 10:31 pm, "Zhi Li" [e-mail] wrote:

it's actually the same, nuget sends two http requests to two sources(VSTS
and NuGet.org here) for fetching the package information at the same time.
If the VSTS return first, then you will get the 401 error because it can't
find PAT, and if nuget.org return first and it find the package in this
feed, then it will just use this one then ignore the VSTS one.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/NuGet/Home/issues/5874#issuecomment-328990241, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAOoMSqbq8x0-rBlLZsm1mFUDcwNUebFks5shvgkgaJpZM4PTzBG
.

[zhili1208]

I think that's because those packages are in your local cache, Could you try to delete the package folder at [username]\.nuget\packages and restore again ?

[Mardoxx]

The only way they could be in my local cache is if I had them installed
from my feed.... which is a VSTS feed!

Very strange.

Does VS2017 request a PAT for the feed under the current logged in VS user
and some how intercept and append it to the requests?

On 12 Sep 2017 11:39 pm, "Zhi Li" [e-mail] wrote:

I think that's because those packages are in your local cache, Could you
try to delete the package folder at [username].nuget\packages and restore
again ?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/NuGet/Home/issues/5874#issuecomment-329004269, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAOoMYlAg0M2pcCWroXy-T8G5nLVIzIHks5shwgmgaJpZM4PTzBG
.

[zhili1208]

nuget indeed reads credential from VS and try that credential with private feed, it might works with VSTS.

[rrelyea]

@Mardoxx - so to sum up, this sounds like the following:
1) if you are running on a CI with an authenticated feed, you need to configure that CI (via nuget.config or alternate ways) to have authentication information.
2) sometimes developers using VS will have authentication set up properly to work in VS, but that is via local state (passwords saved to windows password cache, etc...).

We understand that this scenario can cause confusion, and we are working on a documentation item to improve our support for CI/CD systems. This is covered by this issue: https://github.com/NuGet/Home/issues/5881

Going to close this one out, as it sounds like #5881 will help you too.

[sikemullivan]

I'm not sure what happened, but when I switched over to the yaml build's, I could no longer restore from my feeds.

Created build_nuget.config file with contents below:

<?xml version="1.0" encoding="utf-8"?> <configuration> <packageSources> <add key="nuget.org" value="https://api.nuget.org/v3/index.json" protocolVersion="3" /> <add key="FEEDNAME" value="https://MYPROJECT.pkgs.visualstudio.com/_packaging/FEEDNAME/nuget/v3/index.json" /> </packageSources> <packageSourceCredentials> <FEEDNAME> <add key="Username" value="vsts" /> <add key="ClearTextPassword" value="PERSONAL_ACCESS_TOKEN" /> </FEEDNAME> </packageSourceCredentials> </configuration>

Then added some bash to the yaml that hot swaps the token.

pool:
vmImage: 'Ubuntu 16.04'

variables:
buildConfiguration: 'Release'
projectName: 'MyProjectName'
codeCoverageThreshold: '10'

steps:

• script: |
  sed -i 's/PERSONAL_ACCESS_TOKEN/$(system.AccessToken)/g' build_nuget.config
  dotnet restore --configfile build_nuget.config
  dotnet build --configuration $(buildConfiguration)
  dotnet test $(projectName).Tests --configuration $(buildConfiguration) /p:CollectCoverage=true /p:CoverletOutputFormat=cobertura /p:Threshold=$(codeCoverageThreshold) /p:ThresholdType=method --logger trx
  dotnet pack $(projectName) --output $(build.artifactStagingDirectory) --no-build /p:Configuration=$(buildConfiguration) /p:PackageVersion=0.$(Build.BuildNumber) --verbosity Detailed
• task: PublishTestResults@2
  inputs:
  testRunner: VSTest
  testResultsFiles: '*/.trx'
• task: PublishCodeCoverageResults@1
  inputs:
  codeCoverageTool: 'cobertura'
  summaryFileLocation: '**/coverage.cobertura.xml'
  failIfCoverageEmpty: true
• task: DotNetCoreCLI@2
  inputs:
  command: 'push'
  nuGetFeedType: 'internal'
  packagesToPush: '$(build.artifactStagingDirectory)/*.nupkg'
  publishVstsFeed: 'MyOtherFeed'

[fleed]

Same problem as @sikemullivan ..
Everything works if run on a command line on the build agent.
The same commands don't run properly (end with a 401 unauthorised error) when run by Azure DevOps build pipeline.

[neman]

This is what helped me https://github.com/Microsoft/artifacts-credprovider