Azure-pipelines-tasks: DotNetCore currently does not support using an encrypted Api Key

[lostmsu]

Problem: can't push packages to nuget.org with dotnet nuget push task without exposing API key in build definition

Environment

• Server - VSTS Online
• Agent - Hosted 2017

Error logs

Using authentication information for the following URI: https://api.nuget.org/v3/index.json
Error: DotNetCore currently does not support using an encrypted Api Key.
Packages failed to publish

[bryanmacfarlane]

Can you detail what you mean by "exposing API key"? Something in a log? Something else?

[lostmsu]

@bryanmacfarlane everyone who can read build definition can read the API key.

Previously I used NuGet Publisher task, that allowed to pick a preconfigured connection, which would store credentials on the VSTS side. The new dotnet task + nuget push command also allows to pick the same connection, however it fails with the error above.

[bryanmacfarlane]

If you put the secret in a variable on the variables tab and click the lock, it puts it in a strong box encrypted and readers of the definition can't read it. It's set once. At run time the service will include it in the encrypted job message to the agent and the agent will only make it available to that task.

[lostmsu]

@bryanmacfarlane I guess it is a valid workaround.

The issue with failing build when configuring dotnet nuget push task for external feed using managed credentials still worth addressing.

[aldoms]

Using an ApiKey is currently not supported in dotnet because the required libraries for encrypting the key are not available, sorry for the inconvenience. You should be able to use a service endpoint configured with a username/password combination. If you can only use an ApiKey, I would suggest using the nuget 2.* task to push.

[infin8x]

Using the NuGet task is the correct workaround here, as @aldoms recommends. We're currently working
with the NuGet and dotnet teams (https://github.com/NuGet/Home/issues/6486) to improve the story for authenticated feeds and dotnet. As Aldo mentioned, some of the cryptography libraries that NuGet.exe uses on Windows don't have .NET Core (i.e. Mac/Linux compatible) equivalents yet, which is why this scenario doesn't work today.

[CodeSwimBikeRunner]

The issue with failing build when configuring dotnet nuget push task for external feed using managed credentials still worth addressing.

This does not work for us either.

[CodeSwimBikeRunner]

We must be able to do this through managed external feeds.

[astaerk]

I moved to the NuGet task like recommended by @aldoms and @alexmullans
This works, but dotnet nuget push also pushes symbols.
What is the workaround for publishing symbols? It seems like the NuGet task can't do it and the 'Index Sources & Publish Symbols' task only can publish to vsts or to a file share, but not to any official repo, right?

[infin8x]

The NuGet task can also publish symbols - you just need to remove the ;!$(Build.ArtifactStagingDirectory)/**/*.symbols.nupkg section of the default "Path to NuGet package(s) to publish" field, if it's still present.

[astaerk]

Ok, I already tried that, but I tried it with TFS. But now I realize this is a restriction of TFS. So you're saying this works with the official NuGet.org? Ok, I'll give that a try. Thanks!

[infin8x]

Closing this due to age and inactivity.

[SpyderHunter03]

Why did we close this ticket? The issue is still a valid issue that should get fixed. If you offer a "service" and the "service" doesn't work, then it is a bug and should be fixed one way or another. You can initially offer workarounds, but you should be actively fixing the issue and not just closing it.

[ScottyMac52]

If you put the secret in a variable on the variables tab and click the lock, it puts it in a strong box encrypted and readers of the definition can't read it. It's set once. At run time the service will include it in the encrypted job message to the agent and the agent will only make it available to that task.

That reply did not solve the issue he was having and now I am having it. It was unprofessional to close this ticket!

[infin8x]

@SpyderHunter03 @ScottyMac52 sorry for the closure, was in a triage pass trying to get a handle on our (substantial) bug count in this area. You're right that we should not have closed this.

Since we last updated the .NET Core task's NuGet functions (restore, pack, and nuget push), .NET Core has updated and added the ability to provide an API key on the command line and publish. If memory serves (it's been a while), this wasn't available then. I'll reopen this for tracking. We expect to make substantial updates to these tasks in Q2.

[marcwittke]

Well, this is biting me, too.

Pain started, when I wanted to push my nuget package with symbols. Learnt, that the preferred way is the snupkg format, so I was going down that alley. (Also learnt that myget does not support it yet, but that's another story)

First try, using the nuget task (version 2.*):

[command]/usr/bin/mono /home/agent/agent/~/agent/work/_tool/NuGet/4.1.0/x64/nuget.exe push /home/agent/agent/<bla>.snupkg -NonInteractive -Source https://api.nuget.org/v3/index.json -ApiKey *** -ConfigFile /home/agent/agent/work/_temp/Nuget/tempNuGet_4322.config -Verbosity Normal
400 (Snupkg upload failed. Please use latest NuGet clients (v 4.9.0 or above) and a V3 endpoint to push Symbol packages. For details, refer https://docs.microsoft.com/nuget/create-packages/symbol-packages-snupkg)

Okay, the nuget task is using an outdated nuget.exe. Well, there is dotnet nuget push as well. But:

 ##[error]Error: DotNetCore currently does not support using an encrypted Api Key.

😠 , well, let's do this as a custom task and write the API Key into the params... But:

[command]/home/agent/agent/work/_tool/dotnet/dotnet nuget push /home/agent/work/r1/a/<bla>.nupkg -s https://api.nuget.org/v3/index.json -k {{mykey}}
No executable found matching command "dotnet-nuget push"
##[error]Error: The process '/home/agent/agent/work/_tool/dotnet/dotnet' failed with exit code 1

WTF? copying and pasting the command into a shell works like a charm. So what's going on, at least the last one should work.

Sure, I could run a bash script, but this cuts the possibility to use the globbing feature 😢

update

putting a manual Nuget installer task and version 5.1.0 solved the first try

[lrushl]

WTF? copying and pasting the command into a shell works like a charm. So what's going on, at least the last one should work.

@marcwittke , in order to fix this please make sure your task looks like this

- task: DotNetCoreCLI@2
  displayName: Push Nuget Package
  inputs:
    command: custom
    custom: nuget
    arguments: >
      push $(Build.ArtifactStagingDirectory)/*.nupkg
      -s $(NuGetSourceServerUrl)
      -k $(NuGetSourceServerApiKey)

But yeah, still duplication is needed (nupkg/snupkg), unfortunately :(

[RehanSaeed]

It should also be noted that the allowPackageConflicts argument only exists on NuGetCommand. There seems to be no way with with DotNetCoreCLI to get it to publish several .nupkg's when one of them returns 409 Conflict because the package already exists.

[twsl]

Any ETA, when we can expect a proper solution to this issue?

[Postlagerkarte]

This issue is open now for 531 days, 12 hours, 10 minutes and 23 seconds and there is still no solution? At least throw us an explanation of why it takes so long?

[aasim]

Hi @Postlagerkarte - sorry for the delay, but I think we do have a solution to your original issue now.

We have recently released the NuGetAuthenticate task that can be used in a pipeline to authenticate with Azure Artifacts and/or other NuGet repositories. The design philosophy behind this task is that we would authenticate all the repositories you describe in this task, and get out of the way leaving you fully in control of running your nuget or dotnet commands in the pipeline as you see fit. You can read more about the NuGetAuthenticate task here. It also gives some examples on how to configure the task.

In your case, you can create a service connection to nuget.org and add the NugetAuthenticate task at the start of your pipeline with this service connection as input. After that you can add just a regular powershell task to run dotnet nuget push, and it should work as expected. Let me know if this solution doesn't work for you and we can fix any other underlying issues.

Aasim

[twsl]

This is not really a sastisfying solution to the issue. I expect the dotnet commands to work just like on my own machine. Having to use workarounds is quite annoying.

[MarkKoz]

@aasim That is not correct. The docs state that task also doesn't support service connections using API keys. We basically have to do this.

[Jin-JBA]

@aasim as mentioned by @MarkKoz, NuGetAuthenticate task does not support API key, I am getting the error below while using this task.

ApiKey service connections are not supported in this task. Instead, use -ApiKey (NuGet) or --api-key (dotnet) when invoking the tool itself. See the task documentation for more details.

[knight1219]

I can confirm that the NugetAuthenticate task doesn't support APIKeys still.

[winstonhenke]

So many hoops to jump through just to use a private NuGet feed. So I if I need to build a project that uses X private feeds I'll need to add X tasks like the one above(using restore instead of push)? That seems slow and bloated.

[babelshift]

2 years later and no progress on this. Is this on any roadmap to be addressed?

[edgarrs]

Take a look at this "how do I" answer:
https://docs.microsoft.com/en-us/azure/devops/pipelines/tasks/package/nuget-authenticate?view=azure-devops#how-do-i-configure-a-nuget-package-source-that-uses-apikey-nuget-api-keys-such-as-nugetorg

[ScottyMac52]

So, you post an article from 8/2019 and then simply close the issue? That's
really poor customer service!

On Mon, Jun 22, 2020, 9:24 AM Edgar Ruiz [e-mail] wrote:

Closed #7160
https://github.com/microsoft/azure-pipelines-tasks/issues/7160.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/microsoft/azure-pipelines-tasks/issues/7160#event-3469210523,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ADKNOFHDI2ZLICDEDJYO4UDRX6AVHANCNFSM4E65YI2Q
.

[babelshift]

Are we to assume that this will never be addressed now that the issue is
closed?

EDIT: Sorry, didn't mean to reply to @ScottyMac52.

[ScottyMac52]

That certainly was my reasonable assumption.

On Mon, Jun 22, 2020, 2:31 PM Justin Skiles [e-mail]
wrote:

Are we to assume that this will never be addressed now that the issue is
closed?

On Mon, Jun 22, 2020, 12:46 PM Scott McIntosh [e-mail]
wrote:

So, you post an article from 8/2019 and then simply close the issue?
That's
really poor customer service!

On Mon, Jun 22, 2020, 9:24 AM Edgar Ruiz [e-mail]
wrote:

Closed #7160
https://github.com/microsoft/azure-pipelines-tasks/issues/7160.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<

https://github.com/microsoft/azure-pipelines-tasks/issues/7160#event-3469210523

,
or unsubscribe
<

https://github.com/notifications/unsubscribe-auth/ADKNOFHDI2ZLICDEDJYO4UDRX6AVHANCNFSM4E65YI2Q
>

.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<
https://github.com/microsoft/azure-pipelines-tasks/issues/7160#issuecomment-647640906
,
or unsubscribe
<
https://github.com/notifications/unsubscribe-auth/AAUY4X4ZS4HGGEVQSAOCG6LRX6DHHANCNFSM4E65YI2Q

.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/microsoft/azure-pipelines-tasks/issues/7160#issuecomment-647779796,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ADKNOFDV7P4KGQLA6TNR7U3RX7ES7ANCNFSM4E65YI2Q
.

[twsl]

The nuget solution has been posted over and over. Closing this by saying "ignore this bug, use another method" is a really shitty decission.

[lostmsu]

@edgarrs this workaround is the only one that actually works. But it can't be used for CI easily, as you have to know the package version in advance to find the nupkg file. Another problem is that user interface for creating a .NET Core CLI push task permits you to select a service connection, that was established with an API key, which will not work when pipeline actually runs.

The proper fix would be to:

1. In the .NET Core CLI push task UI show an error when unsupported connection is picked.
2. Add the docs link you provided to the UI.
3. Add a field to enter arguments to .NET Core CLI push task so that I could type --api-key $(NuGetOrgKey) there like in the build task:
   

[woeterman94]

Again, this issue is closed but the bug is still here?