Azure-pipelines-tasks: "dotnet tool install/update" not working with Azure Artifacts

[michael-x]

Note

Issues in this repo are for tracking bugs, feature requests and questions for the tasks in this repo

For a list:
https://github.com/Microsoft/azure-pipelines-tasks/tree/master/Tasks

If you have an issue or request for the Azure Pipelines service, use developer community instead:

https://developercommunity.visualstudio.com/spaces/21/index.html )

Required Information

Entering this information will route you directly to the right team and expedite traction.

Question, Bug, or Feature?
Type: Question or Feature request

Enter Task Name: DotNetCLI@2

list here (V# not needed):
https://github.com/Microsoft/azure-pipelines-tasks/tree/master/Tasks

Environment

• Server - Azure Pipelines
• Agent - Private: Windows Server

Issue Description

dotnet tool install (invoked as custom command via DotNetCLI@2) cannot install tool when NuGet points to Azure Artifacts.

We've got an Azure Artifacts feed for our packages that is configured to use the NuGet gallery as upstream source. We've created a NuGet.config file pointing to the Azure Artifacts feed only. Restoring packages using the restore command works, however trying to restore a tool via a custom command does not. Trying to retrieve information about the package results in a 401 (Unauthorized).

Looking at the implementation of DotNetCLI@2 it looks like there are special cases for dotnet pack, dotnet push and dotnet restore among other things handling implicit authentication. However there is not special treatment for dotnet tool install or dotnet tool update.

Feature Request: Handle dotnet tool install and dotnet tool update or make the authentication globally available to all dotnet commands.

Question: Is there a workaround for this scenario (other than not removing nuget.org via NuGet.config, which would be against the point of using Azure Artifacts with upsteram nuget.org)?

[hiyadav]

Hey @michael-x
Are you able to install the tool via "dotnet tool install" command with same nuget.config on a local setup using same sdk version?
If not, then it is most likely the dotnet sdk issue, as been discussed in the issue thread on dotnet cli repo: https://github.com/dotnet/cli/issues/9586 . According to the thread, the fix for this is in 3.0.100 preview and later.

Hope this helps

[michael-x]

@hiyadav, if I understand things correctly, the ticket that you referenced is about a situation where dotnet tool install doesn't work, because there are two feeds in NuGet.config and a feed that requires authentication (but is not authenticated) blocks installation, even if the tool comes from the other feed that does not require authentication.

This is more about the fact that DotNetCLI@2 does some magic for the "pack", "push" and "restore" commands to provide authentication for Azure Artifact feeds (and maybe others, don't know) that require authentication, but it doesn't do that for "tool install" or "tool update".

[oliverholliday]

That's the bug I'm experiencing @michael-x

2>C:\Program Files\dotnet\sdk\3.0.100-preview6-012264\NuGet.targets(123,5): error : Unable to load the service index for source https://myorg.pkgs.visualstudio.com/_packaging/dotnet-platform/nuget/v3/index.json. [C:\Users\Me\AppData\Local\Temp\lvpavr0a.4ul\restore.csproj]
2>C:\Program Files\dotnet\sdk\3.0.100-preview6-012264\NuGet.targets(123,5): error : Response status code does not indicate success: 401 (Unauthorized). [C:\Users\Me\AppData\Local\Temp\lvpavr0a.4ul\restore.csproj]
2>The tool package could not be restored.
2>Tool 'dotnet-ef' failed to install. This failure may have been caused by:
2>
2>* You are attempting to install a preview release and did not use the --version option to specify the version.
2>* A package by this name was found, but it was not a .NET Core tool.
2>* The required NuGet feed cannot be accessed, perhaps because of an Internet connection problem.
2>* You mistyped the name of the tool.
2>C:\dev\demo\source\Demo.Data\Demo.Data.csproj(31,9): error MSB3073: The command "dotnet tool install --local dotnet-ef --version 3.0.0-preview6.19304.10" exited with code 1.
2>Done building project "Demo.Data.csproj" -- FAILED.

[mulgurul]

Me to, so count me in on a fix:-)

[mulgurul]

PM> dotnet tool install --global dotnet-ef
C:\Program Files\dotnet\sdk\2.1.801\NuGet.targets(123,5): error : Unable to load the service index for source https://pkgs.dev.azure.com/notalib/_packaging/Dependencies/nuget/v3/index.json. [C:\Users\pmd\AppData\Local\Temp\rgjvalkm.rsi\restore.csproj]
C:\Program Files\dotnet\sdk\2.1.801\NuGet.targets(123,5): error :   Response status code does not indicate success: 401 (Unauthorized). [C:\Users\pmd\AppData\Local\Temp\rgjvalkm.rsi\restore.csproj]
dotnet : The tool package could not be restored.
At line:1 char:1
+ dotnet tool install --global dotnet-ef
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (The tool package could not be restored.:String) [], RemoteException
    + FullyQualifiedErrorId : NativeCommandError

Tool "dotnet-ef" failed to install. This failure may have been caused by:

* You are attempting to install a preview release and did not use the --version option to specify the version.
* A package by this name w
as found, but it was not a .NET Core tool.
* The required NuGet feed cannot be accessed, perhaps because of an Internet connection problem.
* You mistyped the name of the tool.

[hiyadav]

Hey,

We have added it to our backlog and will pick it up in coming sprints. I'll post about further update soon.

[mulgurul]

Is it possible to suggest a workaround?

Would be appreciated very much, thanks

---

Fra: himanshu yadav [e-mail]
Sendt: 26. september 2019 10:14:05
Til: microsoft/azure-pipelines-tasks
Cc: Peter Stjernholm Meldgaard; Comment
Emne: Re: [microsoft/azure-pipelines-tasks] "dotnet tool install/update" not working with Azure Artifacts (#10057)

Hey,

We have added it to our backlog and will pick it up in coming sprints. I'll post about further update soon.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHubhttps://github.com/microsoft/azure-pipelines-tasks/issues/10057?email_source=notifications&email_token=ABMIZXUVDA4NKPDMFV7RTN3QLRVM3A5CNFSM4HEKQ342YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD7UXG4A#issuecomment-535393136, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ABMIZXUKP4MIREYKTHAVHITQLRVM3ANCNFSM4HEKQ34Q.

[mulgurul]

Ok, just an update, because this problems expands as you search for a workaround.

It seems like the dotnet commands ignores the dropdown selections for "Package source" and "Default Project" in VS 2019 package manager console.

So it does not take these into account when executing a command like: dotnet tool install --global dotnet-ef
Trying to specify a source (-add-source) on the command line does neither help.

What I did then was to go to options->Nuget Pack..->Pack. sources and remove the entry pointing to our Azure repository. This helped and dotnet could now install the dotnet-ef tools.

Now trying to install a package as: dotnet add package Microsoft.EntityFrameworkCore.Design
result in: dotnet : Could not find any project in C:\Projects\Watermark\WatermarkSystem\Application\WatermarkSystem\.

So it does not take the "Default project selecting into account", ok so i CD to my project folder and repeat the command:

PM> dotnet add package Microsoft.EntityFrameworkCore.Design
  Writing C:\Users\pmd\AppData\Local\Temp\tmp2FC4.tmp
info : Adding PackageReference for package 'Microsoft.EntityFrameworkCore.Design' into project 'C:\Users\pmd\source\repos\TestEFCore\TestEFCore\TestEFCore.csproj'.
info : Restoring packages for C:\Users\pmd\source\repos\TestEFCore\TestEFCore\TestEFCore.csproj...
error: Unable to find package Microsoft.EntityFrameworkCore.Design. No packages exist with this id in source(s): Microsoft Visual Studio Offline Packages
error: Package 'Microsoft.EntityFrameworkCore.Design' is incompatible with 'all' frameworks in project 'C:\Users\pmd\source\repos\TestEFCore\TestEFCore\TestEFCore.csproj'.

Incompatible with what? This is an empty project with core 3.0 as target platform? How could it be incomp.?

[hiyadav]

To get unblocked you would need to prefill authentication in the nuget file for the feed that is to be used.
You can edit the nuget file using a scripting task like PowerShell or you can use NuGetAuthenticate tool for updating the nuget file with required authentication

[bishal-pdMSFT]

@elbatk is this something you should consider?

[rameshkanjinghat]

I too have the same issue. I have published a custom dotnet tool as nuget package to our AzureDevops artifacts. But when try to install the tool using dotnet tool install command I get unauthorized error.

[chklauser]

I have have the same issue, but with the dotnet tool restore command (to restore local tools specified in .config/dotnet-tools.json).

[vanderthiel]

Ok, for all that have this issue, I have a workaround so at least you can get your pipeline running.
I go one directory up and then install, so no nuget.config issues:

- task: Bash@3
  inputs:
    targetType: 'inline'
    script: |
      cd ..
      dotnet tool install -g dotnet-ef
      export PATH="$PATH:$HOME/.dotnet/tools/"
      cd $(build.sourcesdirectory)
      dotnet ef migrations script --idempotent --no-build --configuration $(buildConfiguration) -p <MyProject> -o $(build.artifactstagingdirectory)/migrations.sql

[VictorioBerra]

I am having this issue now and its frustrating. Why cant the dotnet tool restore work the exact same way as dotnet restore? I dont want to bake PATs into my Nuget.Conf, I don't want to include a hack task.

[bishal-pdMSFT]

@tejasd1990 would it be possible to use NugetAuthenticate task along with CLI task to solve this problem?

[VictorioBerra]

@bishal-pdMSFT

I tried this:

steps:

- task: NuGetAuthenticate@0

- task: DotNetCoreCLI@2
  inputs:
    command: 'custom'
    custom: 'tool'
    arguments: 'restore --interactive'

Got this:

C:Program Files\dotnet\sdk\3.1.202\NuGet.targets(124,5): error : Problem starting the plugin 'C:\Users\VssAdministrator.nuget\plugins\netcore\CredentialProvider.Microsoft\CredentialProvider.Microsoft.dll'. Plugin 'CredentialProvider.Microsoft' failed within 5.657 seconds with exit code -1. [C:\Users\VssAdministrator\AppData\Local\Temp\nx1ok0f3.nhzrestore.csproj]

This is with an Azure Hosted Build Agent (windows-latest).

[VictorioBerra]

This worked for me:

Add a global.json:

{
  "sdk": {
    "version": "3.1.300"
  }
}

steps:

- task: NuGetAuthenticate@0

- task: UseDotNet@2
  displayName: Install .NET Core
  inputs:
    useGlobalJson: true

- task: CmdLine@2
  displayName: 'Execute Cake Script'
  inputs:
    script: |
      dotnet tool restore --interactive

[henriksen]

Same problem, this was my workaround:

  - script: |
        dotnet tool restore  --add-source https://api.nuget.org/v3/index.json --ignore-failed-sources
    displayName: Restore .NET Tools

[VictorioBerra]

@henriksen --ignore-failed-sources clever, thanks!

I will say one downside is the tool isnt fetched through the Artifacts source so it wont be copied to artifacts forever.

[henriksen]

I will say one downside is the tool isnt fetched through the Artifacts source so it wont be copied to artifacts forever.

That's true, but for us that's not a huge problem since the tool we need is Entity Framework, and if that suddenly goes away, we're gonna have bigger problems 😁 YMMV

[andrewdmoreno]

Not sure if it was the right way to solve the problem, but I submitted a PR to add tool install as a command on the DotNetCoreCLI task with functionality similar to how the restore command words (I.E. selecting a feed to use). Worked like a charm when I tested so let's see what happens.