Core: ResolvePackageAssets error NETSDK1064: Package XYZ, version 5.1.2 was not found.

[jzabroski]

ResolvePackageAssets error NETSDK1064: Package XYZ, version 5.1.2 was not found.

_@nguerrera Posting this here in case anyone else runs into this issue, and would like some suggestions on how to troubleshoot it_.

Under certain conditions, a build pipeline may not be repeatable due to ResolvePackageAssets step internal to dotnet.exe clean failing:

C:\Program Files\dotnet\sdk\3.1.101\Sdks\Microsoft.NET.Sdk\targets\Microsoft.PackageDependencyResolution.targets(234, 5): error NETSDK1064: Package Autofac, version 5.1.2 was not found. It might have been deleted since NuGet restore. Otherwise, NuGet restore might have only partially completed, which might have been due to maximum path length restrictions.

Example pipeline:

1. dotnet.exe clean Tools.sln --configuration Release
2. NuGet.CommandLine.4.9.2toolsNuGet.exe restore Tools.sln
3. dotnet.exe rebuild Tools.sln --configuration Release

General

TeamCity was configured to use NuGet.CommandLine.4.9.2. Upgraded to NuGet.CommandLine.5.4.0
TeamCity version was 2019.1.1 (build 66192). Upgraded to TeamCity server version is 2019.2.2 (build 71923).
Visual Studio Build Tools 2019 was 16.4.3. Upgraded to Visual Studio Build Tools 2019 16.4.5

Was using Microsoft (R) Build Engine version 16.4.0+e901037fe for .NET Framework

Possible Ideas on What Causes This Issue

• Running build tasks as SYSTEM\NT Authority may be problematic (searching online, this appears to be a common problem setting up Jenkins to build C# projects) for some reason
• If you have two packages with the same version anywhere in your nuget PackageReference dependency graph (I tested this, and this did not seem to be the root cause)
• If you are on an older version of NuGet
• If you are using MSBuild.exe or Visual Studio Build Tools 2019 instead of dotnet.exe to build a .NET SDK csproj formatted project file (I'm not sure why this would cause the problems)

[jzabroski]

Removing nuget.exe step altogether seems to fix the problem. That makes sense since dotnet build should do the restore. Issue was likely caused migrating from old csproj file format to newer csproj file format.

[nguerrera]

I think I’ve seen this before. When you restore with a system account, the nuget global packages folder under the user profile directory ends up being in a location that is subject to virtualization between x64 and x86 processes. Then if you restore with say an x86 nuget.exe and build with an x64 dotnet.exe, the build can’t find the packages that the prior restore claims it acquired because the path in project.assets.json doesn’t work in a process of different bitness!

[nguerrera]

Thank you for separating this out, I think it will be very helpful for others down the road.

[gregpakes]

We are hitting this issue at the moment in Azure Pipelines. It took us a few days to figure out what the issue was and honestly, we still don't know why it is occurring. Our solution is to point Nuget to restore packages to a folder that does not virtualize between x86 and x64.

I think at the very least Nuget should alter the error message to suggest this as a possible solution. It already suggests the max path length as a solution (which actually led us up the garden path for quite a while).

i am not sure which repo I should raise this in - which one do you think?

[minhhieugma]

I have the same issue. It causes me 1 day until I read your advice. I have to change to x64, before it was x86

[gregpakes]

Just to give some more information as to our specific issue. Our Azure private agent was running as Local System rather than a User. This meant that the centralised Nuget location for this user is in a folder that is subject to x64 and x86 bitness changes. So if the packages were restored using x64 Nuget but then your build used x86 MSBuild, the MSBuild would not see any packages there. We fixed it by changing the user that our agent runs under.

[minhhieugma]

@gregpakes What should we do in case we want to build the x86 project but need to keep the x64 agent since we can use this agent for building other x64 projects later? Maybe we need to have two separated x64 and x86 agents?

[gregpakes]

What user is your agent running as? Read my previous message.