Azure-pipelines-tasks: Restoring packages from Azure Artifacts feeds when using Powershell

[johnkors]

Environment

• Azure Pipelines
• Azure Artifacts
• dev.azure.com/icenorge/iceno

Issue Description

I want to build a .NET project in a Azure build pipeline, where the project is referencing a NuGet that is in a secured Azure Artifacts feed. I want to do this using powershell (using Cake build, and not the task way.

My azure-pipelines.yaml:

pool:
  vmImage: 'VS2017-Win2016'
steps:
- powershell: ./build.ps1

How can I get the build user executing the dotnet restore in the build.ps1 to authenticate to a secured Azure Artifacts feed when running in a Powershell script? As mentioned, I don't want to use the restore tasks.

I have set up a Service Connection to the Azure Artifacts NuGet feed and named it IceNorgeAzureFeedServiceConnection, but I'm only seeing 401s during the dotnet restore step of my pipeline build.

Ideally, I'd like something similar to the setup of the NuGetCommand@2 task, or for the build user to _just work_ OOB and have access to all my feeds within the same org/project.

steps:
powershell: ./build.ps1
  inputs:
    externalFeedCredentials: IceNorgeAzureFeedServiceConnection

The feed is in the same org + project as the pipeline build.

[infin8x]

Right now, you'd need to manually construct a nuget.config and put it alongside your build.ps1 so dotnet discovers it as part of restore. Specifically, you'd need the packageSources and packageSourceCredentials sections, using the cleartextpassword variant. The password would be the build's access token.

We're working on a better experience here, thanks for the feedback.

[johnkors]

Ok, thanks @alexmullans ! I got it working. For anyone else looking into it, I ended up doing:

azure-pipelines.yml:

pool:
  vmImage: 'VS2017-Win2016'
steps:
- powershell: ./build.ps1
  env:
      AZUREDEVOPS_PAT: $(System.AccessToken)

build.cake:
Before running dotnet restore, I call this method to setup nuget manually:

private void AddAzureArtifactsAsAuthenticatedNuGetSource(){
        var source = new {
            Name = "IceNorgeAzureFeed",
            Source = "https://pkgs.dev.azure.com/icenorge/_packaging/IceNorgeAzureFeed/nuget/v3/index.json",
            ApiUserName = "VSTS", //dont-care value
            ApiKey = EnvironmentVariable("AZUREDEVOPS_PAT") 
        };

        if (NuGetHasSource(source.Source))
        {
            NuGetRemoveSource(source.Name, source.Source);     
        }
        NuGetAddSource(
            source.Name,
            source.Source,
            new NuGetSourcesSettings {
                UserName = source.ApiUserName,
                Password = source.ApiKey,
                IsSensitiveSource = false,
                Verbosity = NuGetVerbosity.Detailed
            }
        );
        NuGetSetApiKey(source.ApiKey, source.Source);
}

[infin8x]

Thanks for sharing! Glad you got it working.

[davidroberts63]

Thank you @johnkors I ran into this exact situation yesterday. Also, would you be interested in a custom Az Pipelines task that would do this?A sort of NuGetConfigureSources task?

@alexmullans is there somewhere I can vote on the enhancement you spoke of?

[infin8x]

@davidroberts63 we're just about to move from UserVoice over to a Developer Community-powered suggestion site, so the public voting system is a bit in flux at the moment. I've made a note of your ask in our internal tally though.

[johnkors]

@davidroberts63 hm, not sure I'd care for it, to be honest.

I prefer my builds to be ignorant of _where they execute_ (Travis CI, TeamCity, AppVeyor, ++), so I put as little as I possibly can into any yml file or build configuration any of these provide. I want my builds to be easy to move between CI systems. So I try to:

1) script everything
2) minimize the use of vendor specific features (here: azure tasks)
3) build in docker containers

[infin8x]

@johnkors makes sense. We should have a range of options for you:

1. For complete portability, you can manually construct a nuget.config as I described. The only thing you'll need to change across CI systems is the variable that contains the token that has access to your private feed.
2. We'll provide a task that automatically constructs a nuget.config for your Azure Artifacts feed and/or any external feeds you provide credentials for.

To your original suggestion that "for the build user to just work OOB and have access to all my feeds within the same org/project", 2 things:

1. By default, the build identity has access to all feeds in the same organization, unless you change the defaults on creation or remove Project Collection Build Service from your feed's permissions. But,
2. We don't automatically create a nuget.config (.npmrc, etc.) that tells every build about all those feeds because we don't know which feeds any particular build may rely on. For option 2 above, we'll still need you to tell us about the feeds you depend on (either by providing a nuget.config that we add credentials to or by listing the feeds you want and allowing us to construct a config file on your behalf).

[johnkors]

👍 Frankly, I'm quite OK with manually constructing/adding the secure nuget feed during the build. I would have to do the same on any other public CI service as well, and as you say — the same applies to npm(rc) and friends as well. I find it's worth the extra mile to be decoupled from the vendor specific stuff.

What would be nice, is a clear documentation page on the subject.

Using Cake/Fake/Make/Rake or any other scripted build instead of the Azure Tasks? Here are some gotcha's [..]

For me, the gotcha was that I had to inject the variable System.Accesstoken explicitly when using powershell. I expected the access token just to be available for my build without any mods to the yml. 😬

[davidroberts63]

@johnkors same idea for me as well. I'd rather have the CI tool kick off a singular script that does all the things, mainly so I can also reproduce the build locally as much as possible. Also, (fairly unrelated) John, when using Cake if you also use ReportUnit watch out, it comes with nuget.exe which confuses Cake as to which nuget.exe and creds to use.

@alexmullans the second option is great, I have a number of people where I'm at that would likely end up using it. On a sidenote (let me know where to direct this) any way to set a build variable project collection wide or at the agent level (which would also exist in the build container)?

[infin8x]

@johnkors I'm working on a doc and some feature work around using authenticated resources in build, I'll make sure it covers the "I'll do it myself, where's the token" case. Thanks for the suggestion.

@davidroberts63 I've asked someone from the Pipelines team for their thoughts on your last question. IIRC, this is in the works, but they should respond back with more details.