Home: Race condition during restore for DotnetCliToolReference packages in the default parallel processing mode

[dtivel]

Could not find file 'C:\A\1\1\s\.test\work\09e15e0a\bc7be395\globalPackages\.tools\z\20.0.0\netcoreapp1.0\z.nuget.cache'.
System.IO.FileNotFoundException: Could not find file 'C:\A\1\1\s\.test\work\09e15e0a\bc7be395\globalPackages\.tools\z\20.0.0\netcoreapp1.0\z.nuget.cache'.
File name: 'C:\A\1\1\s\.test\work\09e15e0a\bc7be395\globalPackages\.tools\z\20.0.0\netcoreapp1.0\z.nuget.cache'
   at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
   at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost)
   at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share)
   at NuGet.Common.FileUtility.SafeRead[T](String filePath, Func`3 read)
   at NuGet.Commands.RestoreCommand.EvaluateCacheFile()
   at NuGet.Commands.RestoreCommand.<ExecuteAsync>d__33.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at NuGet.Commands.RestoreRunner.<ExecuteAsync>d__7.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at NuGet.Commands.RestoreRunner.<ExecuteAndCommitAsync>d__6.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at NuGet.Commands.RestoreRunner.<CompleteTaskAsync>d__9.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at NuGet.Commands.RestoreRunner.<RunAsync>d__2.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at NuGet.Commands.RestoreRunner.<RunAsync>d__0.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at NuGet.Commands.RestoreRunner.<RunAsync>d__1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at NuGet.CommandLine.RestoreCommand.<ExecuteCommandAsync>d__52.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at NuGet.CommandLine.Command.Execute()
   at NuGet.CommandLine.Program.MainCore(String workingDirectory, String[] args)


NuGet Version: 5.8.0.13082
Restoring NuGet packages for solution C:\A\1\1\s\.test\work\09e15e0a\bc7be395\solution\solution.sln.
MSBuild auto-detection: using msbuild version '16.7.0.37604' from 'C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\MSBuild\Current\bin'. Use option -MSBuildVersion to f

(truncated)

[kartheekp-ms]

Thanks for creating this issue. I am working on NuGet/Client.Engineering#67. I noticed the same test failure in my private build and I thought this failure was due to running tests in parallel.
//cc @zivkan

[nkolev92]

@kartheekp-ms Can we just disable this test and reword the issue in a way that calls out the actual bug?
My gut feeling is that while we have an actual, the impact of this one is infrequent enough to warrant adding additional synchronization.

DotnetCLIToolReference packages almost never have a missing version.

[kartheekp-ms]

AFAIK, RestoreNetCore_MultipleProjects_SameToolDifferentVersionsAsync is not a flaky test there is a race condition which I explained in this comment.

[nkolev92]

Thanks for taking care of this @kartheekp-ms!

I have clarified the issue title further and prioritized the issue based on it's impact.

[nkolev92]

Clearing out the metadata.
This is a restore problem that I don't think needs fixing because of the extremely low impact.
I don't expect non special tool scenarios to hit this.

[zivkan]

It was causing a large number of test failures and was very annoying 😛