Azure-pipelines-tasks: NuGetCommand@2 not using the project name in the vstsfeed

[ThomasManson]

Type: Bug

Enter Task Name: NugetCommand

Environment

• Azure Pipelines
  Account name: VSOGD
  Team Project: BUMA-Productivity
  Build Definition: 2385
  Build Number: https://vsogd.visualstudio.com/BUMA-OperatorProductivity/_build/results?buildId=223552

• Agent
  Hosted

Issue Description

This is the task that is failing:
- task: NuGetCommand@2 displayName: 'Buma-dev Restore' inputs: command: restore restoreSolution: '$(Build.SourcesDirectory)/src/Buma.Op.$(Build.Repository.Name).sln' feedsToUse: select vstsFeed: '$(buma.feedName)' includeNuGetOrg: false restoreDirectory: $(Build.SourcesDirectory)/src/packages

The variable buma.feedname is set to the name of the feed that is set up in the same project scope.

When the task runs, it fails with: Response status code does not indicate success: 404 (Not Found - The feed with ID 'Buma.Op' doesn't exist.

This is because the task is using this uri: https://vsogd.pkgs.visualstudio.com/_packaging/Buma.Op/nuget/v3/index.json

However, when I check the settings for the feed, it should be using: https://vsogd.pkgs.visualstudio.com/BUMA-OperatorProductivity/_packaging/Buma.Op/nuget/v3/index.json

The project name (BUMA-OperatorProductivity) is missing from the uri.

Task logs

logs_223552.zip

Error logs

The error below shows the task is using the package source:
https://vsogd.pkgs.visualstudio.com/_packaging/Buma.Op/nuget/v3/index.json

The Feed in the Azure DevOps collection is:
https://vsogd.pkgs.visualstudio.com/BUMA-OperatorProductivity/_packaging/Buma.Op.Shared/nuget/v3/index.json

2019-11-17T21:13:34.8736557Z mono-sgen: URI: https://vsogd.pkgs.visualstudio.com/_packaging/Buma.Op/nuget/v3/index.json
2019-11-17T21:13:34.8736863Z mono-sgen: Is retry: False
2019-11-17T21:13:34.8737207Z mono-sgen: Matched prefix: https://vsogd.pkgs.visualstudio.com/
2019-11-17T21:13:34.8738528Z NuGet.Protocol.Core.Types.FatalProtocolException: Unable to load the service index for source https://vsogd.pkgs.visualstudio.com/_packaging/Buma.Op/nuget/v3/index.json. ---> System.Net.Http.HttpRequestException: Response status code does not indicate success: 404 (Not Found - The feed with ID 'Buma.Op' doesn't exist. (DevOps Activity ID: EC787ED7-E6E5-4BF9-88B0-205979379EE9)).
2019-11-17T21:13:34.8738943Z at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode () [0x00040] in <7ecf813f2d314058b05c6c092c47b77a>:0
2019-11-17T21:13:34.8739249Z at NuGet.Protocol.HttpSource+<>c__DisplayClass14_01[T].<GetAsync>b__0 (System.Threading.CancellationToken lockedToken) [0x00400] in <9bc69d145f5a4dbcbd11cfa298c5ba4f>:0 2019-11-17T21:13:34.8739573Z at NuGet.Common.ConcurrencyUtilities.ExecuteWithFileLockedAsync[T] (System.String filePath, System.Func2[T,TResult] action, System.Threading.CancellationToken token) [0x00231] in <9bc69d145f5a4dbcbd11cfa298c5ba4f>:0
2019-11-17T21:13:34.8739735Z at NuGet.Protocol.HttpSource.GetAsync[T] (NuGet.Protocol.HttpSourceCachedRequest request, System.Func2[T,TResult] processAsync, NuGet.Common.ILogger log, System.Threading.CancellationToken token) [0x000ea] in <9bc69d145f5a4dbcbd11cfa298c5ba4f>:0 2019-11-17T21:13:34.8739901Z at NuGet.Protocol.ServiceIndexResourceV3Provider.GetServiceIndexResourceV3 (NuGet.Protocol.Core.Types.SourceRepository source, System.DateTime utcNow, NuGet.Common.ILogger log, System.Threading.CancellationToken token) [0x001ff] in <9bc69d145f5a4dbcbd11cfa298c5ba4f>:0 2019-11-17T21:13:34.8740225Z --- End of inner exception stack trace --- 2019-11-17T21:13:34.8740584Z at NuGet.Protocol.ServiceIndexResourceV3Provider.GetServiceIndexResourceV3 (NuGet.Protocol.Core.Types.SourceRepository source, System.DateTime utcNow, NuGet.Common.ILogger log, System.Threading.CancellationToken token) [0x002b3] in <9bc69d145f5a4dbcbd11cfa298c5ba4f>:0 2019-11-17T21:13:34.8740717Z at NuGet.Protocol.ServiceIndexResourceV3Provider.TryCreate (NuGet.Protocol.Core.Types.SourceRepository source, System.Threading.CancellationToken token) [0x00213] in <9bc69d145f5a4dbcbd11cfa298c5ba4f>:0 2019-11-17T21:13:34.8741175Z at NuGet.Protocol.Core.Types.SourceRepository.GetResourceAsync[T] (System.Threading.CancellationToken token) [0x000b8] in <9bc69d145f5a4dbcbd11cfa298c5ba4f>:0 2019-11-17T21:13:34.8741378Z at NuGet.Protocol.Core.Types.SourceRepository.GetResourceAsync[T] () [0x00070] in <9bc69d145f5a4dbcbd11cfa298c5ba4f>:0 2019-11-17T21:13:34.8741528Z at NuGet.Protocol.HttpFileSystemBasedFindPackageByIdResourceProvider.TryCreate (NuGet.Protocol.Core.Types.SourceRepository sourceRepository, System.Threading.CancellationToken token) [0x0007c] in <9bc69d145f5a4dbcbd11cfa298c5ba4f>:0 2019-11-17T21:13:34.8741717Z at NuGet.Protocol.Core.Types.SourceRepository.GetResourceAsync[T] (System.Threading.CancellationToken token) [0x000b8] in <9bc69d145f5a4dbcbd11cfa298c5ba4f>:0 2019-11-17T21:13:34.8741840Z at NuGet.Protocol.Core.Types.SourceRepository.GetResourceAsync[T] () [0x00070] in <9bc69d145f5a4dbcbd11cfa298c5ba4f>:0 2019-11-17T21:13:34.8742168Z at NuGet.Commands.SourceRepositoryDependencyProvider.EnsureResource () [0x0007b] in <9bc69d145f5a4dbcbd11cfa298c5ba4f>:0 2019-11-17T21:13:34.8742503Z at NuGet.Commands.SourceRepositoryDependencyProvider.FindLibraryCoreAsync (NuGet.LibraryModel.LibraryRange libraryRange, NuGet.Frameworks.NuGet.Frameworks1137997.NuGetFramework targetFramework, NuGet.Protocol.Core.Types.SourceCacheContext cacheContext, NuGet.Common.ILogger logger, System.Threading.CancellationToken cancellationToken) [0x00088] in <9bc69d145f5a4dbcbd11cfa298c5ba4f>:0 2019-11-17T21:13:34.8742774Z at NuGet.Commands.SourceRepositoryDependencyProvider+<>c__DisplayClass19_0.<FindLibraryAsync>b__0 () [0x0008e] in <9bc69d145f5a4dbcbd11cfa298c5ba4f>:0 2019-11-17T21:13:34.8742911Z at NuGet.Commands.SourceRepositoryDependencyProvider.FindLibraryAsync (NuGet.LibraryModel.LibraryRange libraryRange, NuGet.Frameworks.NuGet.Frameworks1137997.NuGetFramework targetFramework, NuGet.Protocol.Core.Types.SourceCacheContext cacheContext, NuGet.Common.ILogger logger, System.Threading.CancellationToken cancellationToken) [0x00190] in <9bc69d145f5a4dbcbd11cfa298c5ba4f>:0 2019-11-17T21:13:34.8743069Z at NuGet.DependencyResolver.ResolverUtility+<>c__DisplayClass7_1.<FindLibraryFromSourcesAsync>b__0 () [0x0007b] in <9bc69d145f5a4dbcbd11cfa298c5ba4f>:0 2019-11-17T21:13:34.8743370Z at NuGet.DependencyResolver.ResolverUtility.FindLibraryFromSourcesAsync (NuGet.LibraryModel.LibraryRange libraryRange, System.Collections.Generic.IEnumerable1[T] providers, System.Func2[T,TResult] action) [0x00170] in <9bc69d145f5a4dbcbd11cfa298c5ba4f>:0 2019-11-17T21:13:34.8743511Z at NuGet.DependencyResolver.ResolverUtility.FindLibraryByVersionAsync (NuGet.LibraryModel.LibraryRange libraryRange, NuGet.Frameworks.NuGet.Frameworks1137997.NuGetFramework framework, System.Collections.Generic.IEnumerable1[T] providers, NuGet.Protocol.Core.Types.SourceCacheContext cacheContext, NuGet.Common.ILogger logger, System.Threading.CancellationToken token) [0x002ab] in <9bc69d145f5a4dbcbd11cfa298c5ba4f>:0
2019-11-17T21:13:34.8743683Z at NuGet.DependencyResolver.ResolverUtility.FindLibraryMatchAsync (NuGet.LibraryModel.LibraryRange libraryRange, NuGet.Frameworks.NuGet.Frameworks1137997.NuGetFramework framework, System.String runtimeIdentifier, NuGet.DependencyResolver.GraphEdge1[TItem] outerEdge, System.Collections.Generic.IEnumerable1[T] remoteProviders, System.Collections.Generic.IEnumerable1[T] localProviders, System.Collections.Generic.IEnumerable1[T] projectProviders, System.Collections.Generic.IDictionary2[TKey,TValue] lockFileLibraries, NuGet.Protocol.Core.Types.SourceCacheContext cacheContext, NuGet.Common.ILogger logger, System.Threading.CancellationToken cancellationToken) [0x005db] in <9bc69d145f5a4dbcbd11cfa298c5ba4f>:0 2019-11-17T21:13:34.8743850Z at NuGet.DependencyResolver.ResolverUtility.FindLibraryEntryAsync (NuGet.LibraryModel.LibraryRange libraryRange, NuGet.Frameworks.NuGet.Frameworks1137997.NuGetFramework framework, System.String runtimeIdentifier, NuGet.DependencyResolver.GraphEdge1[TItem] outerEdge, NuGet.DependencyResolver.RemoteWalkContext context, System.Threading.CancellationToken cancellationToken) [0x000e9] in <9bc69d145f5a4dbcbd11cfa298c5ba4f>:0
2019-11-17T21:13:34.8744090Z at NuGet.DependencyResolver.RemoteDependencyWalker.CreateGraphNode (NuGet.LibraryModel.LibraryRange libraryRange, NuGet.Frameworks.NuGet.Frameworks1137997.NuGetFramework framework, System.String runtimeName, NuGet.RuntimeModel.RuntimeGraph runtimeGraph, System.Func2[T,TResult] predicate, NuGet.DependencyResolver.GraphEdge1[TItem] outerEdge) [0x0022b] in <9bc69d145f5a4dbcbd11cfa298c5ba4f>:0
2019-11-17T21:13:34.8744322Z at NuGet.DependencyResolver.RemoteDependencyWalker.CreateGraphNode (NuGet.LibraryModel.LibraryRange libraryRange, NuGet.Frameworks.NuGet.Frameworks1137997.NuGetFramework framework, System.String runtimeName, NuGet.RuntimeModel.RuntimeGraph runtimeGraph, System.Func2[T,TResult] predicate, NuGet.DependencyResolver.GraphEdge1[TItem] outerEdge) [0x00595] in <9bc69d145f5a4dbcbd11cfa298c5ba4f>:0
2019-11-17T21:13:34.8744496Z at NuGet.Commands.ProjectRestoreCommand.WalkDependenciesAsync (NuGet.LibraryModel.LibraryRange projectRange, NuGet.Frameworks.NuGet.Frameworks1137997.NuGetFramework framework, System.String runtimeIdentifier, NuGet.RuntimeModel.RuntimeGraph runtimeGraph, NuGet.DependencyResolver.RemoteDependencyWalker walker, NuGet.DependencyResolver.RemoteWalkContext context, System.Threading.CancellationToken token) [0x000be] in <9bc69d145f5a4dbcbd11cfa298c5ba4f>:0
2019-11-17T21:13:34.8745016Z Unable to load the service index for source https://vsogd.pkgs.visualstudio.com/_packaging/Buma.Op/nuget/v3/index.json.
2019-11-17T21:13:34.8745078Z at NuGet.Commands.ProjectRestoreCommand.TryRestoreAsync (NuGet.LibraryModel.LibraryRange projectRange, System.Collections.Generic.IEnumerable1[T] frameworkRuntimePairs, NuGet.Repositories.NuGetv3LocalRepository userPackageFolder, System.Collections.Generic.IReadOnlyList1[T] fallbackPackageFolders, NuGet.DependencyResolver.RemoteDependencyWalker remoteWalker, NuGet.DependencyResolver.RemoteWalkContext context, System.Boolean forceRuntimeGraphCreation, System.Threading.CancellationToken token, NuGet.Common.TelemetryActivity telemetryActivity) [0x00194] in <9bc69d145f5a4dbcbd11cfa298c5ba4f>:0
2019-11-17T21:13:34.8745448Z at NuGet.Commands.RestoreCommand.ExecuteRestoreAsync (NuGet.Repositories.NuGetv3LocalRepository userPackageFolder, System.Collections.Generic.IReadOnlyList1[T] fallbackPackageFolders, NuGet.DependencyResolver.RemoteWalkContext context, System.Threading.CancellationToken token, NuGet.Common.TelemetryActivity telemetryActivity) [0x002f8] in <9bc69d145f5a4dbcbd11cfa298c5ba4f>:0 2019-11-17T21:13:34.8745984Z at NuGet.Commands.RestoreCommand.ExecuteAsync (System.Threading.CancellationToken token) [0x005f7] in <9bc69d145f5a4dbcbd11cfa298c5ba4f>:0 2019-11-17T21:13:34.8746053Z at NuGet.Commands.RestoreRunner.ExecuteAsync (NuGet.Commands.RestoreSummaryRequest summaryRequest, System.Threading.CancellationToken token) [0x000c9] in <9bc69d145f5a4dbcbd11cfa298c5ba4f>:0 2019-11-17T21:13:34.8746108Z at NuGet.Commands.RestoreRunner.ExecuteAndCommitAsync (NuGet.Commands.RestoreSummaryRequest summaryRequest, System.Threading.CancellationToken token) [0x00076] in <9bc69d145f5a4dbcbd11cfa298c5ba4f>:0 2019-11-17T21:13:34.8746165Z at NuGet.Commands.RestoreRunner.CompleteTaskAsync (System.Collections.Generic.List1[T] restoreTasks) [0x000d7] in <9bc69d145f5a4dbcbd11cfa298c5ba4f>:0
2019-11-17T21:13:34.8746207Z at NuGet.Commands.RestoreRunner.RunAsync (System.Collections.Generic.IEnumerable`1[T] restoreRequests, NuGet.Commands.RestoreArgs restoreContext, System.Threading.CancellationToken token) [0x00154] in <9bc69d145f5a4dbcbd11cfa298c5ba4f>:0
2019-11-17T21:13:34.8746253Z at NuGet.Commands.RestoreRunner.RunAsync (NuGet.Commands.RestoreArgs restoreContext, System.Threading.CancellationToken token) [0x000d9] in <9bc69d145f5a4dbcbd11cfa298c5ba4f>:0
2019-11-17T21:13:34.8746309Z at NuGet.Commands.RestoreRunner.RunAsync (NuGet.Commands.RestoreArgs restoreContext) [0x0006e] in <9bc69d145f5a4dbcbd11cfa298c5ba4f>:0
2019-11-17T21:13:34.8746510Z at NuGet.CommandLine.RestoreCommand.ExecuteCommandAsync () [0x00393] in <9bc69d145f5a4dbcbd11cfa298c5ba4f>:0
2019-11-17T21:13:35.0013788Z Response status code does not indicate success: 404 (Not Found - The feed with ID 'Buma.Op' doesn't exist. (DevOps Activity ID: EC787ED7-E6E5-4BF9-88B0-205979379EE9)).
2019-11-17T21:13:35.0069841Z ##[error]The nuget command failed with exit code(1) and error(Unable to load the service index for source https://vsogd.pkgs.visualstudio.com/_packaging/Buma.Op/nuget/v3/index.json.
Response status code does not indicate success: 404 (Not Found - The feed with ID 'Buma.Op' doesn't exist. (DevOps Activity ID: EC787ED7-E6E5-4BF9-88B0-205979379EE9)).)
2019-11-17T21:13:35.0079125Z ##[error]Packages failed to restore
2019-11-17T21:13:35.0080371Z ##[section]Finishing: Buma-dev Restore

[ThomasManson]

This appears to be an issue since project-scoped feeds have been introduced, and new feeds created through the UI can only be project scoped.
https://docs.microsoft.com/en-us/azure/devops/artifacts/feeds/project-scoped-feeds?view=azure-devops

[satbai]

I think this should be fixed now. Can you confirm so I can close the issue? Thanks!

[ImportSJC]

@satbai this bug still seems to be present for me. I have tried using the feed's name and ID and in both cases the URL generated is invalid, missing only the project name. do you documentation or a commit that indicates this has been fixed?

Also, does anyone have an alternative method of connecting to a project scoped feed that I can use as a workaround?

Thanks!

[satbai]

Sorry, there was a bug in the connect to feed page in the Artifacts UI. The connect to feed page showed an incorrect nuget URL for project scoped feeds. If you copied the feed URL for a project scoped feed during when the bug was out, you have to manually go back to the feed and copy the URL form the connect to feed page again. For a project scoped feed, the URL includes the project the feed is scoped to. https://docs.microsoft.com/en-us/azure/devops/artifacts/feeds/project-scoped-feeds?view=azure-devops#the-differences-between-an-organization-scoped-feed-and-a-project-scoped-feed

[ImportSJC]

@satbai Oh, I get what you are saying. Yes the connect feed page does now show the correct URL for project-scoped feeds. However the current version of the Nuget Command (NuGetCommand@2) still doesn't generate the correct URL when provided the name/ID of the feed in the VSTS input.

[satbai]

Ah I see, I shouldn't have assumed that you're using yaml. The task should have the code in place to create the correct project scoped URL based on the selection in the feed dropdown. Is the dropdown getting automatically populated with feed names for you? Is the feed in the same Azure DevOps organization as the pipeline? Can you check from the pipeline log which version of the NuGetCommand2 task you're running? Screen shot of your task setup may also be helpful. Thanks!

[fibann]

@satbai I am not sure about @ImportSJC but I am using yaml and can confirm that using the _unqualified_ feed name/GUID in vstsFeed/publishVstsFeed doesn't work for project-scoped feeds - the generated URL passed to nuget doesn't contain the project name even if the pipeline is in the same project as the feed.

I have tried adding the step from the GUI, the feed is shown in the dropdown and what it uses for vstsFeed is <project_guid>/<feed_guid>, which does indeed work (names instead of guids work fine too).

It seems to me that there is a lack of documentation here - the fact that it is necessary to prepend the project name/id is not mentioned in the NuGetCommand doc page nor in the "Connect to feed" page. All the examples there use unqualified feed names. The project-scoped feed doc page mentions adding the project name to the URL but makes no reference to NuGetCommand either.

[satbai]

Thanks for great feedback @fibann. I'm going to add some details about project-scoped feeds in the NuGetCommandV2 docs. In the Artifacts connect to feed page, for project scoped feeds, the Nuget URL shows the project in the URL automatically, and for collection scoped feeds the project doesn't exist in the URL. Please note that the _browser URL_ always includes a project, however, which might be confusing in some cases. We'll consider making a note about that in the docs as well.

[fibann]

In the Artifacts connect to feed page, for project scoped feeds, the Nuget URL shows the project in the URL automatically

Right. What's a bit confusing is that the "Publish packages" example in that page passes the unqualified feed name to dotnet/nuget.exe. I suppose (haven't tried) that if you set the correct feed URL in nuget.config that will work fine. The problem in my case (at least) was that I don't have a nuget.config with my project, but I am passing the feed to NuGetCommand@2 explicitly.

[satbai]

I can see how that can be confusing. If you were using nuget.config and updated the URL in the file, the push example from connect to feed would work because it refers to the key of the package source in nuget.config. In your case it wasn't obvious if you didn't use nuget.config. The NuGetCommandV2 docs should be updated soon with examples for project scoped feeds. The changes should help avoid issues like this.