Renovate: Error updating NuGet lock file when credentials are defined for a feed

[lazyboy1]

What Renovate type, platform and version are you using?

Self-hosted v23.84.6 running in Bitbucket Pipelines.

Describe the bug

Renovate always fails to update lock file when a private feed requiring credentials is defined in nuget.config. The problem is pretty obvious from the logs - the command dotnet nuget update source is run with empty string as the value of username and password, but this command is not supported and throws an error.

Relevant debug logs

_Note: I redacted some of the data._

DEBUG: dotnet command (repository=**redacted**, branch=**redacted**)
       "cmd": [
         "dotnet nuget update source **redacted** --username **redacted** --password **redacted** --store-password-in-clear-text",
         "dotnet restore **redacted** --force-evaluate",
         "dotnet nuget update source **redacted** --username '' --password '' --store-password-in-clear-text"
       ]
DEBUG: Executing command (repository=**redacted**, branch=**redacted**)
       "command": "dotnet nuget update source **redacted** --username **redacted** --password **redacted** --store-password-in-clear-text"
DEBUG: exec completed (repository=**redacted** , branch=**redacted** )
       "cmd": "dotnet nuget update source **redacted** --username **redacted** --password **redacted** --store-password-in-clear-text",
       "durationMs": 442,
       "stdout": "Package source \"**redacted**\" was successfully updated.\n",
       "stderr": ""
DEBUG: Executing command (repository=**redacted** , branch=**redacted** )
       "command": "dotnet restore **redacted**  --force-evaluate"
DEBUG: exec completed (repository=**redacted** , branch=**redacted**)
       "cmd": "dotnet restore **redacted** --force-evaluate",
       "durationMs": 2422,
       "stdout": "  Determining projects to restore...\n  Restored **redacted** (in 947 ms).\n",
       "stderr": ""
DEBUG: Executing command (repository=**redacted**, branch=**redacted**)
       "command": "dotnet nuget update source **redacted** --username '' --password '' --store-password-in-clear-text"
DEBUG: Failed to generate lock file (repository=**redacted** , branch=**redacted** )
       "err": {
         "killed": false,
         "code": 1,
         "signal": null,
         "cmd": "dotnet nuget update source **redacted** --username '' --password '' --store-password-in-clear-text",
         "stdout": "",
         "stderr": "System.ArgumentException: Value cannot be null or whitespace. (Parameter 'token')\n   at Microsoft.DotNet.Cli.CommandLine.OptionError..ctor(String message, String token, AppliedOption option) in /_/src/source/CommandLine/OptionError.cs:line 21\n   at Microsoft.DotNet.Cli.CommandLine.Parser.UnrecognizedArg(String arg) in /_/src/source/CommandLine/Parser.cs:line 170\n   at System.Linq.Enumerable.SelectListIterator`2.MoveNext()\n   at System.Collections.Generic.List`1.InsertRange(Int32 index, IEnumerable`1 collection)\n   at System.Collections.Generic.List`1.AddRange(IEnumerable`1 collection)\n   at Microsoft.DotNet.Cli.CommandLine.Parser.Parse(IReadOnlyCollection`1 rawArgs, Boolean isProgressive) in /_/src/source/CommandLine/Parser.cs:line 111\n   at Microsoft.DotNet.Cli.CommandLine.Parser.Parse(String[] args) in /_/src/source/CommandLine/Parser.cs:line 31\n   at Microsoft.DotNet.Cli.ParserExtensions.ParseFrom(Parser parser, String context, String[] args)\n   at Microsoft.DotNet.Cli.Program.ProcessArgs(String[] args, ITelemetry telemetryClient)\n   at Microsoft.DotNet.Cli.Program.Main(String[] args)\n",
         "message": "Command failed: dotnet nuget update source **redacted** --username '' --password '' --store-password-in-clear-text\nSystem.ArgumentException: Value cannot be null or whitespace. (Parameter 'token')\n   at Microsoft.DotNet.Cli.CommandLine.OptionError..ctor(String message, String token, AppliedOption option) in /_/src/source/CommandLine/OptionError.cs:line 21\n   at Microsoft.DotNet.Cli.CommandLine.Parser.UnrecognizedArg(String arg) in /_/src/source/CommandLine/Parser.cs:line 170\n   at System.Linq.Enumerable.SelectListIterator`2.MoveNext()\n   at System.Collections.Generic.List`1.InsertRange(Int32 index, IEnumerable`1 collection)\n   at System.Collections.Generic.List`1.AddRange(IEnumerable`1 collection)\n   at Microsoft.DotNet.Cli.CommandLine.Parser.Parse(IReadOnlyCollection`1 rawArgs, Boolean isProgressive) in /_/src/source/CommandLine/Parser.cs:line 111\n   at Microsoft.DotNet.Cli.CommandLine.Parser.Parse(String[] args) in /_/src/source/CommandLine/Parser.cs:line 31\n   at Microsoft.DotNet.Cli.ParserExtensions.ParseFrom(Parser parser, String context, String[] args)\n   at Microsoft.DotNet.Cli.Program.ProcessArgs(String[] args, ITelemetry telemetryClient)\n   at Microsoft.DotNet.Cli.Program.Main(String[] args)\n",
         "stack": "Error: Command failed: dotnet nuget update source **redacted** --username '' --password '' --store-password-in-clear-text\nSystem.ArgumentException: Value cannot be null or whitespace. (Parameter 'token')\n   at Microsoft.DotNet.Cli.CommandLine.OptionError..ctor(String message, String token, AppliedOption option) in /_/src/source/CommandLine/OptionError.cs:line 21\n   at Microsoft.DotNet.Cli.CommandLine.Parser.UnrecognizedArg(String arg) in /_/src/source/CommandLine/Parser.cs:line 170\n   at System.Linq.Enumerable.SelectListIterator`2.MoveNext()\n   at System.Collections.Generic.List`1.InsertRange(Int32 index, IEnumerable`1 collection)\n   at System.Collections.Generic.List`1.AddRange(IEnumerable`1 collection)\n   at Microsoft.DotNet.Cli.CommandLine.Parser.Parse(IReadOnlyCollection`1 rawArgs, Boolean isProgressive) in /_/src/source/CommandLine/Parser.cs:line 111\n   at Microsoft.DotNet.Cli.CommandLine.Parser.Parse(String[] args) in /_/src/source/CommandLine/Parser.cs:line 31\n   at Microsoft.DotNet.Cli.ParserExtensions.ParseFrom(Parser parser, String context, String[] args)\n   at Microsoft.DotNet.Cli.Program.ProcessArgs(String[] args, ITelemetry telemetryClient)\n   at Microsoft.DotNet.Cli.Program.Main(String[] args)\n\n    at ChildProcess.exithandler (child_process.js:308:12)\n    at ChildProcess.emit (events.js:314:20)\n    at ChildProcess.EventEmitter.emit (domain.js:483:12)\n    at maybeClose (internal/child_process.js:1021:16)\n    at Process.ChildProcess._handle.onexit (internal/child_process.js:286:5)"
       }

To Reproduce

1. Create a .NET Core project and reference an outdated package.

2. Add a nuget.config file defining a feed that requires username and password:

   <configuration>
    <packageSources>
      <clear />
      <add key="nuget.org"
         value="https://api.nuget.org/v3/index.json" />
      <add key="PrivateFeed"
         value="my-private-feed-url" />
    </packageSources>
   </configuration>
   

3. Set credentials in Renovate hostRules config:

   {
    "hostRules": [
      {
        "timeout": 60000,
        "baseUrl": "my-private-feed-url",
        "hostType": "nuget",
        "username": "the-username",
        "password": "the-password"
      }
    ]
   }
   

I could not provide an example repo because I can't share credentials to a private feed...

Additional context

NuGet lock file support was added in #7375.

From a quick search in code, it seems the command with empty user and pass is run by lib/manager/nuget/artifacts.ts:37.

[rarkins]

Is it possible that the first command - with username/password correct - works but then the last command that attempts to clear them and uses '' fails?

Cc @fgreinacher can you comment?

[viceice]

Maybe we should use dotnet nuget remove source instead of setting empty?

[fgreinacher]

Oops, I'm not sure how I missed that problem 🤔

Maybe we should use dotnet nuget remove source instead of setting empty?

That unfortunately won't work, because it removes the entry from the NuGet.config, possibly breaking follow-up actions.

It seems that --username needs to be set while --password can be empty. So what about changing the command to dotnet nuget update source ${registry.name} --username ${username} --password '' --store-password-in-clear-text?

[rarkins]

Definitely no way to do it with environment variables?

[viceice]

Definitely no way to do it with environment variables?

Nope, but we can add the target config file with --configfile. So we can point that to ~/.config/NuGet/NuGet.Config.
So we add and remove the source there or simply delete the file after run

[viceice]

https://docs.microsoft.com/en-us/nuget/consume-packages/configuring-nuget-behavior

[lazyboy1]

It seems that --username needs to be set while --password can be empty. So what about changing the command to dotnet nuget update source ${registry.name} --username ${username} --password '' --store-password-in-clear-text?

Password can't be empty, the same error is thrown. It is possible to run instead: dotnet nuget update source ${registry.name} --username 'dummy' --password 'dummy' --store-password-in-clear-text

[fgreinacher]

It seems that --username needs to be set while --password can be empty. So what about changing the command to dotnet nuget update source ${registry.name} --username ${username} --password '' --store-password-in-clear-text?

Password can't be empty, the same error is thrown. It is possible to run instead: dotnet nuget update source ${registry.name} --username 'dummy' --password 'dummy' --store-password-in-clear-text

Really weird, for me it works:

I'm on Windows 10 2004 and in this case .NET SDK 3.1.404.

[lazyboy1]

It does work on Windows but not on Linux. I am trying these commands in docker image mcr.microsoft.com/dotnet/sdk:3.1:

[fgreinacher]

It does work on Windows but not on Linux.

Oh, that's really weird, thanks for digging deeper!

Nope, but we can add the target config file with --configfile. So we can point that to ~/.config/NuGet/NuGet.Config.
So we add and remove the source there or simply delete the file after run

That could work, yes.

I am also wondering whether we could just not reset the credentials. Are the containers re-used across repos?

[rarkins]

The containers aren't reused when in binarySource=docker

[fgreinacher]

Nice, so let's just get rid of the reset logic? Thinking more, this might cause unexpected situations when people are using Renovate in non-docker mode (e.g. when using the fat image) across multiple repos. Let's try to solve this like @viceice suggested.

[fgreinacher]

Others seems to have similar problems, see e.g. https://github.com/NuGet/Home/issues/6243 or https://github.com/NuGet/Home/issues/6045 and there is currently no proper solution.

I guess something like this would work:

mkdir $RANDOM_TMP_DIR
cd $RANDOM_TMP_DIR
dotnet new nugetconfig
cd -
dotnet nuget add source xyz --configfile $RANDOM_TMP_DIR/nuget.config
dotnet restore --configfile $RANDOM_TMP_DIR/nuget.config
rmdir $RANDOM_TMP_DIR

[renovate-release]

:tada: This issue has been resolved in version 23.84.9 :tada:

The release is available on:

• GitHub release
• 23.84.9

Your semantic-release bot :package::rocket:

[lazyboy1]

I am testing 23.84.9 which includes #7779. I am now getting a new error:

DEBUG: found NuGet.config (repository=**repo**, branch=**branch**)
       "nuGetConfigPath": "/tmp/renovate/repos/bitbucket/**repo**/nuget.config"
DEBUG: clearing registry URLs (repository=**repo**, branch=**branch**)
DEBUG: adding registry URL (repository=**repo**, branch=**branch**)
       "registryUrl": "https://api.nuget.org/v3/index.json"
DEBUG: adding registry URL (repository=**repo**, branch=**branch**)
       "registryUrl": "**my-private-nuget-source-url**"
DEBUG: dotnet command (repository=**repo**, branch=**branch**)
       "cmd": [
         "dotnet new nugetconfig --output /tmp/renovate/cache/others/nuget/d093376669c3dae5",
         "dotnet nuget add source https://api.nuget.org/v3/index.json --name nuget.org --configfile /tmp/renovate/cache/others/nuget/d093376669c3dae5/nuget.config",
         "dotnet nuget add source **private-source-url** --name **private-source-name** --configfile /tmp/renovate/cache/others/nuget/d093376669c3dae5/nuget.config --username pipelines --password **redacted** --store-password-in-clear-text",
         "dotnet restore **path-to-csproj** --force-evaluate --configfile /tmp/renovate/cache/others/nuget/d093376669c3dae5/nuget.config"
       ]
DEBUG: Executing command (repository=**repo**, branch=**branch**)
       "command": "dotnet new nugetconfig --output /tmp/renovate/cache/others/nuget/d093376669c3dae5"
DEBUG: exec completed (repository=**repo**, branch=**branch**)
       "cmd": "dotnet new nugetconfig --output /tmp/renovate/cache/others/nuget/d093376669c3dae5",
       "durationMs": 2221,
       "stdout": "Getting ready...\nThe template \"NuGet Config\" was created successfully.\n\nAn update for template pack Microsoft.DotNet.Common.ItemTemplates::3.1.10 is available.\n    install command: dotnet new -i Microsoft.DotNet.Common.ItemTemplates::5.0.0\n",
       "stderr": ""
DEBUG: Executing command (repository=**repo**, branch=**branch**)
       "command": "dotnet nuget add source https://api.nuget.org/v3/index.json --name nuget.org --configfile /tmp/renovate/cache/others/nuget/d093376669c3dae5/nuget.config"
DEBUG: Failed to generate lock file (repository=**repo**, branch=**branch**)
       "err": {
         "killed": false,
         "code": 1,
         "signal": null,
         "cmd": "dotnet nuget add source https://api.nuget.org/v3/index.json --name nuget.org --configfile /tmp/renovate/cache/others/nuget/d093376669c3dae5/nuget.config",
         "stdout": "error: The source specified has already been added to the list of available package sources. Provide a unique source.\n\n\nUsage: dotnet nuget add source [arguments] [options]\n\nArguments:\n  PackageSourcePath  Path to the package source.\n\nOptions:\n  -n|--name                       Name of the source.\n  -u|--username                   Username to be used when connecting to an authenticated source.\n  -p|--password                   Password to be used when connecting to an authenticated source.\n  --store-password-in-clear-text  Enables storing portable package source credentials by disabling password encryption.\n  --valid-authentication-types    Comma-separated list of valid authentication types for this source. Set this to basic if the server advertises NTLM or Negotiate and your credentials must be sent using the Basic mechanism, for instance when using a PAT with on-premises Azure DevOps Server. Other valid values include negotiate, kerberos, ntlm, and digest, but these values are unlikely to be useful.\n  --configfile                    The NuGet configuration file. If specified, only the settings from this file will be used. If not specified, the hierarchy of configuration files from the current directory will be used. For more information, see https://docs.microsoft.com/nuget/consume-packages/configuring-nuget-behavior.\n  -h|--help                       Show help information\n",
         "stderr": "",
         "message": "Command failed: dotnet nuget add source https://api.nuget.org/v3/index.json --name nuget.org --configfile /tmp/renovate/cache/others/nuget/d093376669c3dae5/nuget.config\n",
         "stack": "Error: Command failed: dotnet nuget add source https://api.nuget.org/v3/index.json --name nuget.org --configfile /tmp/renovate/cache/others/nuget/d093376669c3dae5/nuget.config\n\n    at ChildProcess.exithandler (child_process.js:308:12)\n    at ChildProcess.emit (events.js:314:20)\n    at ChildProcess.EventEmitter.emit (domain.js:483:12)\n    at maybeClose (internal/child_process.js:1021:16)\n    at Process.ChildProcess._handle.onexit (internal/child_process.js:286:5)"
       }

Turns out that dotnet new nugetconfig command creates a file that already includes the nuget.org primary source:

<packageSources>
  <!--To inherit the global NuGet package sources remove the <clear/> line below -->
  <clear />
  <add key="nuget" value="https://api.nuget.org/v3/index.json" />
</packageSources>

Because these "add" nodes are executed in an hierarchy (computer + user + local), it is considered best practice to always <clear /> before <add />, to avoid clashes between local and user/computer configs. For that reason, my local config file already includes https://api.nuget.org/v3/index.json as a source. The new code now tries to add all sources to the new temporary config file, and fails on source https://api.nuget.org/v3/index.json.

Maybe it would be better to copy the existing config file to the temporary location instead of creating a new one?

For reference, all our nuget.config files look like this:

<?xml version="1.0" encoding="utf-8"?>
<configuration>
    <packageSources>
        <clear />
        <add key="nuget.org"
             value="https://api.nuget.org/v3/index.json" />
        <add key="private-feed-name"
             value="private-feed-url" />
    </packageSources>
</configuration>

[fgreinacher]

Thanks for detailed update @lazyboy1. I think we're making progress now. I'll take care of removing the remaining problem.

Maybe it would be better to copy the existing config file to the temporary location instead of creating a new one?

I'd really like to start with a fresh config to ensure that we only use the NuGet.config settings we actively support.

[renovate-release]

:tada: This issue has been resolved in version 23.90.1 :tada:

The release is available on:

• GitHub release
• 23.90.1

Your semantic-release bot :package::rocket: