Azure-pipelines-tasks: Nuget restore uses temporary config that can change path of packages folder

[Sethcran]

Environment

• Server - TFS on-premises Version 16.122.27102.1
• Agent - Private Windows Server 2016 Agent 2.122.1

Issue Description

• Task: Nuget v2.0 running Restore command against a solution.

I am specifying a nuget.config in the repository root:

inside my nuget.config is the following lines:

This was originally to share a single packages folder for all solutions in the repository (there are a lot of solutions in the repository that are not included in this build, but are part of other builds).

Build fails with errors due to packages not being found. Packages are referenced in projects in a relative form: ....packages...

I believe these lines from the logs are the reason why:
-23T21:47:02.6777422Z Saving NuGet.config to a temporary config file.
2018-03-23T21:47:02.7245991Z [command]D:websites1_work_toolNuGet4.0.0x64nuget.exe restore D:websites1_work2sDownloadMyFormDMFWeb.sln -Verbosity Detailed -NonInteractive -ConfigFile D:websites1_work2NugettempNuGet_142743.config

Navigating to this directory shows a packages folder here.
Copying the nuget file to a temporary directory breaks relative packages folders defined in nuget.config

This does not occur on V1 of the Nuget task.

[jeffabailey]

I'm running into this issue as well, any workaround?

[Sethcran]

Workaround for me was just to downgrade to version 1.

[tibitoth]

This issue also occurred to me. I cannot build projects with old packages.config where the csprojs reference packages from ..packages folder.

[ColinDabritzViewpoint]

This issue is affecting our builds, although we easily work around it by specifying the destination directory in the restore step as $(Build.SourcesDirectory)/packages

The lack of clear documentation around the "$" root of Nuget.config location syntax made it a little more complicated to track down.

Our project has multiple solutions one folder deep, all using the same nuget.config file at the root of the proejct.

It specifies the repositoryPath as:

  <config>
    <add key="repositoryPath" value="$/../packages" />
  </config>

based on this stack overflow comment here:
https://stackoverflow.com/questions/18376313/setting-up-a-common-nuget-packages-folder-for-all-solutions-when-some-projects-a

Quoting from 'Vermis' October 8, 2013 solution:

For the repositoryPath setting, you can specify an absolute path or relative path (recommended) using the $ token. The $ token is based on where the NuGet.Config is located (The $ token is actually relative to one level below the location of the NuGet.Config). So, if I have \Solutions\NuGet.Config and I want \Solutions\Packages I would need to specify $\..\Packages as the value.

I would expect this $ syntax to be documented here:
https://docs.microsoft.com/en-us/nuget/reference/nuget-config-file
but I was unable to find it.

From the logs it appears that the step constructs a new temporary config file including other step settings and changes, but the fact that the temporary file is stored in a different location causes the $ config relative path to be incorrect. It should be relative to the source config file path. Perhaps the path could be literalized as part of creating the second temporary config?

For us, the workaround is easy enough. We are using version 2 of the Nuget step. Under "Advanced" settings, our Destination Directory is overridden to be $(Build.SourcesDirectory)/packages which is the same folder as we intend.

This is not a critical issue for us, but it would be nice to clean up a little down the road.

Thank you all for your work in the VSTS ecosystem! It's been a great environment for getting work done, and it is developing rapidly.

[Identifier]

Same problem here. Switching the NuGet Restore Task to v1.* instead of v2.* fixed it like @Sethcran said.

[Identifier]

@bryanmacfarlane, it looks like keithrob might no longer be with the company. Can you assign this to someone?

[satbai]

The v2 NuGet Restore task now uses the user provided nuget.config directly without creating a temporary one when using a NuGet version greater than 4.8.1. Please change the NuGet version that's used for this task by adding a new NuGet Tool Installer task before the NuGet Restore task.

This fix is deployed to almost everywhere so you can give it a try, but there is one more deployment ring to go and that's scheduled for tomorrow, so if it doesn't work for you yet please give it a day or so and try again.

[ransagy]

I've just come across this (fixing a whole bunch of old projects with packages.config, circa nuget 2.7 msbuild-auto-restore era junk. yay for ifix nugetrestore tool!) and i have ~200 pipelines to fix which is.. a bit painful, in the least.

Is there any plan to update the tasks/whatever images used to have newer nuget built in?

EDIT: @satbai I've just tried it with a NuGet Tool Installer task set to 4.9.2 and still the same - repositoryPath is ignored, a temporary config file was created. How can i know if i'm on the deployed group or not?

[zjrunner]

We can't update the default version of nuget on the machine because many people took a dependency on the quirks of the one on the images (3.3 I believe), particularly around how it deals with config discovery, repository path, and relative paths. We attempted to do an update a couple years ago and it caused enough pain we had to roll it back and build the whole ToolInstaller flow based on that mistake.

For people that want to roll forward, they should be able to use the tool installer in a way that it helps them stay up to date as an opt-in flow. That puts the customer in control of risk and with a simple way to opt in or workaround issues that come up with specific versions (changing the image is not a valid path in these one-off cases generally)

[ransagy]

Thanks for the explanation. I guess I'll automate adding the Nuget Tool Installer task to all those pipelines somehow, maybe via the REST API. I'll wait for it to work properly on one though, When this is deployed across the board.

[ransagy]

now that I'm reading @satbai 's comment again i just want to be sure i understand it right - what exactly is the user provided nuget.config ? The one in .nuget\nuget.config? Configured somewhere else in the task/pipeline? I hope the former.

FWIW my NuGet task says its v2.146.0. What version should i expect for the fix to this to be deployed on?

[satbai]

Hi @ransagy, by user provided nuget.config I mean the file you can add in the NuGet 2.* Restore task (see the image below). Does this work in your scenario? NuGet task version 146.0 contains the fix.

[ransagy]

@satbai You're right; I misunderstood, This does seem to be deployed/working as intended now. I, For some reason, Assumed selecting my feed there will just add the source in NuGet rather than write a new config. Thanks!