Sdk: Builds of solutions with multiple inter-dependent .NET Core projects frequently fail with CS2012 "Cannot open" file errors

Created on 19 Nov 2016  路  21Comments  路  Source: dotnet/sdk

Steps to reproduce

With a solution with multiple .NET Core projects with many sharing project dependencies:

  • dotnet restore
  • dotnet build

Expected behavior

Solution builds.

Actual behavior

Solution sometimes builds, but also often fails with errors similar to:

CSC : error CS2012: Cannot open 'C:\a\w\1\s\src\Project\obj\Debug\netstandard1.4\Project.dll' for writing -- 'The process cannot access the file 'C:\a\w\1\s\src\Project\obj\Debug\netstandard1.4\Project.dll' because it is being used by another process.'

Sometimes there are warnings:

warning MSB3026: Could not copy "obj\Debug\netstandard1.4\Project.dll" to "bin\Debug\netstandard1.4\Project.dll". Beginning retry 1 in 1000ms. The process cannot access the file 'C:\a\w\1\s\src\Project\bin\Debug\netstandard1.4\Project.dll' because it is being used by another process. [C:\a\w\1\s\src\Project\Project.csproj]

Errors and warnings can occur for different projects.

Switching from dotnet build to msbuild /maxcpucount:1 avoids the problems.

The errors occur more frequently on build servers (Azure VMs, Standard F2S) than on laptop (i7-5600U, 16GB, SSD).

Our build script replaces version numbers in AssemblyInfo.cs files, so many projects are rebuilt on every build.

Executing dotnet build --no-incremental also seems to increase failure rate locally (where otherwise repeated builds may not build all projects).

Environment data

dotnet --info output:

.NET Command Line Tools (1.0.0-preview3-004056)

Product Information:
 Version:            1.0.0-preview3-004056
 Commit SHA-1 hash:  ccc4968bc3

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.14393
 OS Platform: Windows
 RID:         win10-x64
picture frankbuckley

Most helpful comment

Well it's really easy to reproduce with NET SDK 2.0.2

dotnet publish -o /home/user/publish_dir

Will publish everything from a solution into one folder in case argument of -o is an absolute directory. This is super convenient and a way to go in case you make sure projects are compatible. Though publish commands can fight for files of course.

picture sherlock1982 on 7 Nov 2017
👍3

All 21 comments

Are there any custom build scripts involved? Particularly direct MSbuild invocations? Is this a sln build or multiple parallel project builds?

@nguerrera @dsplaisted

TheRealPiotrP picture TheRealPiotrP on 20 Nov 2016

sln - no custom build scripts.

I can replicate pretty regularly on my laptop by simply running dotnet build in the directory with the sln file after changing source files - or by executing dotnet build --no-incremental. Running dotnet msbuild /m:1 does not error.

The build servers run VSTS agents that simply step through the following:

  • Run GitVersion and update AssemblyInfo.cs files
  • Execute dotnet restore in the root directory with the sln file
  • Execute dotnet build in the root directory with the sln file
  • Run unit tests using vstest.console.exe

On the build servers dotnet build fails with CS2012 pretty much every time.

frankbuckley picture frankbuckley on 20 Nov 2016

Just to confirm - I am still seeing this behaviour with RC2 - dotnet build a solution with inter-dependent projects - results in:

CSC : error CS2012: Cannot open 'C:\src\Project1\obj\Debug\netstandard1.4\Project1.dll' for writing -- 'The process cannot access the file 'C:\src\Project1\obj\Debug\netstandard1.4\Project1.dll' because it is being used by another process.' [C:\src\Project1\Project1.csproj]

and:

C:\Program Files\dotnet\sdk\1.0.0-preview4-004233\Microsoft.Common.CurrentVersion.targets(3190,5): error MSB3491: Could not write lines to file "obj\Release\netstandard1.4\CoreCompileInputs.cache". The process cannot access the file 'C:\src\Project3\obj\Release\netstandard1.4\CoreCompileInputs.cache' because it is being used by another process. [C:\src\Project3\Project3.csproj]

Again, running dotnet msbuild /m:1 remains a workaround.

PS C:\src> dotnet --info
.NET Command Line Tools (1.0.0-preview4-004233)

Product Information:
 Version:            1.0.0-preview4-004233
 Commit SHA-1 hash:  8cec61c6f7

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.14393
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   C:\Program Files\dotnet\sdk\1.0.0-preview4-004233
frankbuckley picture frankbuckley on 13 Dec 2016

One of my projects seems to be able to reliably replicate this on AppVeyor - dotnet build fails with error CS2012 yet dotnet msbuild /m:1 succeeds.

dotnet build: https://ci.appveyor.com/project/jkommer/gravatarhelper/build/4.0.0.29#L213
dotnet msbuild /m:1: https://ci.appveyor.com/project/jkommer/gravatarhelper/build/4.0.0.39

Source and CI setup is visible at: https://github.com/jkommer/GravatarHelper/tree/d353a9fdc3264aa5d9fab0a1cf511cdf0e1d7509

jkommer picture jkommer on 14 Jan 2017

@piotrpMSFT: Do you have any ideas about what might be happening here? Is it possible that this is an MSBuild issue?

DustinCampbell picture DustinCampbell on 18 Jan 2017

it's possible. I've nto seen a local repro yet...

TheRealPiotrP picture TheRealPiotrP on 18 Jan 2017

@piotrpMSFT @DustinCampbell Try this for a repro:

https://github.com/frankbuckley/dotnetbuildcs2012repro

I reliably get results like:

PS D:\frankdev\dotnetbuildcs2012repro> .\build.ps1
  Restoring packages for D:\frankdev\dotnetbuildcs2012repro\Library2\Library2.csproj...
  Restoring packages for D:\frankdev\dotnetbuildcs2012repro\Library6\Library6.csproj...
  Restoring packages for D:\frankdev\dotnetbuildcs2012repro\Library3\Library3.csproj...
  Restoring packages for D:\frankdev\dotnetbuildcs2012repro\Library5\Library5.csproj...
  Restoring packages for D:\frankdev\dotnetbuildcs2012repro\Library10\Library10.csproj...
  Restoring packages for D:\frankdev\dotnetbuildcs2012repro\Library11\Library12.csproj...
  Restoring packages for D:\frankdev\dotnetbuildcs2012repro\Library1\Library1.csproj...
  Restoring packages for D:\frankdev\dotnetbuildcs2012repro\Library4\Library4.csproj...
  Writing lock file to disk. Path: D:\frankdev\dotnetbuildcs2012repro\Library2\obj\project.assets.json
  Writing lock file to disk. Path: D:\frankdev\dotnetbuildcs2012repro\Library3\obj\project.assets.json
  Writing lock file to disk. Path: D:\frankdev\dotnetbuildcs2012repro\Library11\obj\project.assets.json
  Writing lock file to disk. Path: D:\frankdev\dotnetbuildcs2012repro\Library10\obj\project.assets.json
  Writing lock file to disk. Path: D:\frankdev\dotnetbuildcs2012repro\Library1\obj\project.assets.json
  Writing lock file to disk. Path: D:\frankdev\dotnetbuildcs2012repro\Library4\obj\project.assets.json
  Writing lock file to disk. Path: D:\frankdev\dotnetbuildcs2012repro\Library6\obj\project.assets.json
  Writing lock file to disk. Path: D:\frankdev\dotnetbuildcs2012repro\Library5\obj\project.assets.json
  Restore completed in 804.549ms for D:\frankdev\dotnetbuildcs2012repro\Library11\Library12.csproj.
  Restore completed in 804.5725ms for D:\frankdev\dotnetbuildcs2012repro\Library3\Library3.csproj.
  Restore completed in 804.5921ms for D:\frankdev\dotnetbuildcs2012repro\Library10\Library10.csproj.
  Restore completed in 804.5464ms for D:\frankdev\dotnetbuildcs2012repro\Library4\Library4.csproj.
  Restore completed in 804.5853ms for D:\frankdev\dotnetbuildcs2012repro\Library2\Library2.csproj.
  Restore completed in 804.5834ms for D:\frankdev\dotnetbuildcs2012repro\Library6\Library6.csproj.
  Restore completed in 804.5615ms for D:\frankdev\dotnetbuildcs2012repro\Library1\Library1.csproj.
  Restore completed in 804.7167ms for D:\frankdev\dotnetbuildcs2012repro\Library5\Library5.csproj.
  Restoring packages for D:\frankdev\dotnetbuildcs2012repro\Library9\Library9.csproj...
  Restoring packages for D:\frankdev\dotnetbuildcs2012repro\Library8\Library8.csproj...
  Restoring packages for D:\frankdev\dotnetbuildcs2012repro\Library7\Library7.csproj...
  Writing lock file to disk. Path: D:\frankdev\dotnetbuildcs2012repro\Library8\obj\project.assets.json
  Writing lock file to disk. Path: D:\frankdev\dotnetbuildcs2012repro\Library7\obj\project.assets.json
  Writing lock file to disk. Path: D:\frankdev\dotnetbuildcs2012repro\Library9\obj\project.assets.json
  Restore completed in 360.9902ms for D:\frankdev\dotnetbuildcs2012repro\Library8\Library8.csproj.
  Restore completed in 362.4869ms for D:\frankdev\dotnetbuildcs2012repro\Library7\Library7.csproj.
  Restore completed in 362.6074ms for D:\frankdev\dotnetbuildcs2012repro\Library9\Library9.csproj.

  NuGet Config files used:
      C:\Users\FrankBuckley\AppData\Roaming\NuGet\NuGet.Config
      C:\Program Files (x86)\NuGet\Config\Microsoft.VisualStudio.Offline.config

  Feeds used:
      https://api.nuget.org/v3/index.json
      C:\Program Files (x86)\Microsoft SDKs\NuGetPackages\
    Microsoft (R) Build Engine version 15.1.458.808
    Copyright (C) Microsoft Corporation. All rights reserved.

  Library1 -> D:\frankdev\dotnetbuildcs2012repro\Library1\bin\Debug\netstandard1.4\Library1.dll
  Library1 -> D:\frankdev\dotnetbuildcs2012repro\Library1\bin\Debug\netstandard1.4\Library1.dll
  Library8 -> D:\frankdev\dotnetbuildcs2012repro\Library8\bin\Debug\netstandard1.4\Library8.dll
  Library7 -> D:\frankdev\dotnetbuildcs2012repro\Library7\bin\Debug\netstandard1.4\Library7.dll
  Library12 -> D:\frankdev\dotnetbuildcs2012repro\Library11\bin\Debug\netstandard1.4\Library12.dll
  Library9 -> D:\frankdev\dotnetbuildcs2012repro\Library9\bin\Debug\netstandard1.4\Library9.dll
  Library10 -> D:\frankdev\dotnetbuildcs2012repro\Library10\bin\Debug\netstandard1.4\Library10.dll
  Library4 -> D:\frankdev\dotnetbuildcs2012repro\Library4\bin\Debug\netstandard1.4\Library4.dll
  Library4 -> D:\frankdev\dotnetbuildcs2012repro\Library4\bin\Debug\netstandard1.4\Library4.dll
  Library2 -> D:\frankdev\dotnetbuildcs2012repro\Library2\bin\Debug\netstandard1.4\Library2.dll
CSC : error CS2012: Cannot open 'D:\frankdev\dotnetbuildcs2012repro\Library\obj\Debug\netstandard1.4\Library6.dll' for writing -- 'The process cannot access the file 'D:\frankdev\dotnetbuildcs2012repro\Library6\obj\Debug\netstandard1.4\Library6.dll' because it is being used by another process.' [D:\frankdev\dotnetbuildcs2012repro\Library6\Library6.csproj]
  Library2 -> D:\frankdev\dotnetbuildcs2012repro\Library2\bin\Debug\netstandard1.4\Library2.dll
  Library6 -> D:\frankdev\dotnetbuildcs2012repro\Library6\bin\Debug\netstandard1.4\Library6.dll
C:\Program Files\dotnet\sdk\1.0.0-preview4-004233\Microsoft.Common.CurrentVersion.targets(3932,5): warning MSB3026: Could not copy "obj\Debug\netstandard1.4\Library3.dll" to "bin\Debug\netstandard1.4\Library3.dll". Beginning retry 1 in 1000ms. The process cannot access the file 'D:\frankdev\dotnetbuildcs2012repro\Library3\obj\Debug\netstandard1.4\Library3.dll' because it is being used by another process. [D:\frankdev\dotnetbuildcs2012repro\Library3\Library3.csproj]
  Library3 -> D:\frankdev\dotnetbuildcs2012repro\Library3\bin\Debug\netstandard1.4\Library3.dll
  Library3 -> D:\frankdev\dotnetbuildcs2012repro\Library3\bin\Debug\netstandard1.4\Library3.dll
C:\Program Files\dotnet\sdk\1.0.0-preview4-004233\Sdks\Microsoft.NET.Sdk\build\Microsoft.NET.Sdk.targets(82,5): error : Cannot find project info for 'D:\frankdev\dotnetbuildcs2012repro\Library1\Library1.csproj'. This can indicate a missing project     reference. [D:\frankdev\dotnetbuildcs2012repro\Library5\Library5.csproj]

Note that the solution compiles without errors in VS2017.

PS D:\frankdev\dotnetbuildcs2012repro> dotnet --info
.NET Command Line Tools (1.0.0-preview4-004233)

Product Information:
 Version:            1.0.0-preview4-004233
 Commit SHA-1 hash:  8cec61c6f7

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.14393
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   C:\Program Files\dotnet\sdk\1.0.0-preview4-004233
frankbuckley picture frankbuckley on 18 Jan 2017

@frankbuckley thanks for the repro. I just PR'd it with latest csproj file reductions to enable using latest CLI.

I did see a file-in-use issue, but it was nicely handled as a warning:

/dev/cli/dotnet/sdk/1.0.0-rc4-004536/MSBuild/Microsoft.Common.CurrentVersion.targets(3935,5):
warning MSB3026: Could not copy "obj/Debug/netstandard1.4/Library3.dll" to 
"bin/Debug/netstandard1.4/Library3.dll". Beginning retry 1 in 1000ms. 
The process cannot access the file '/dev/repro/r4786/dotnetbuildcs2012repro/Library3/obj/Debug/netstandard1.4/Library3.dll' because
it is being used by another process. [/dev/repro/r4786/dotnetbuildcs2012repro/Library3/Library3.csproj]

@frankbuckley @DustinCampbell this seems like a good experience to me. Great would be 'no file in use' but I don't believe that is achievable for 1.0.0.

Any concern with this being closed?

TheRealPiotrP picture TheRealPiotrP on 20 Jan 2017

I should add, after the warning [and retry] the build did flow through to completion without intervention.

TheRealPiotrP picture TheRealPiotrP on 20 Jan 2017

@piotrpMSFT @DustinCampbell - thanks for looking at this. Sorry to say I am still seeing errors.

I have just applied your PR, checked out, downloaded and installed 1.0.0-rc4-004578 and I am still seeing CS2012 errors:

https://github.com/frankbuckley/dotnetbuildcs2012repro/tree/master/Results

PS D:\frankdev\dotnetbuildcs2012repro> dotnet --info
.NET Command Line Tools (1.0.0-rc4-004578)

Product Information:
 Version:            1.0.0-rc4-004578
 Commit SHA-1 hash:  93a94e652b

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.14393
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   C:\Program Files\dotnet\sdk\1.0.0-rc4-004578

This is running on a quad core i7-6820HQ (system-info.txt) with SSDs.

frankbuckley picture frankbuckley on 20 Jan 2017

@piotrpMSFT @DustinCampbell - just for fun I have tried the following on 2 VMs:

VM1 is Azure Standard DS2 v2 (2 cores, 7 GB memory)

Results: https://raw.githubusercontent.com/frankbuckley/dotnetbuildcs2012repro/master/Results/output-vm1.txt

  • On first run - error MSB3491
  • Subsequent runs - build OK, no errors

VM2 is Azure Standard DS3 v2 (4 cores, 14 GB memory)

Results: https://raw.githubusercontent.com/frankbuckley/dotnetbuildcs2012repro/master/Results/output-vm2.txt

  • On first run - error CS2012 (then MSB4018)
  • Subsequent runs - "MSB3026: Could not copy", "CS2012: Cannot open"

Continuing to attempt on VM2 I started seeing error MSB3713 with an AssemblyInfo.cs locked by another process. I had a look at running processes and there was an orphaned dotnet.exe. I ended that process and got back to regular error CS2012s - log:

https://raw.githubusercontent.com/frankbuckley/dotnetbuildcs2012repro/master/Results/output-vm2-b.txt

Finally, I tried &dotnet msbuild DotNetBuildErrors.sln /m:2 on VM2, which built (though msbuild /m did not)- log:

https://github.com/frankbuckley/dotnetbuildcs2012repro/blob/master/Results/output-vm2-c.txt

Trying again on my laptop, I also get &dotnet msbuild DotNetBuildErrors.sln /m:2 working reliably, but not &dotnet msbuild DotNetBuildErrors.sln /m or &dotnet build at all.

/m:2 is better than where we were a couple of months ago (all our automated builds of VS2017 solutions are set to /m:1 otherwise they fail) - but it still seems there are problems here (1.0.0-rc4-004578).

When the next RC is available, I will move some real solutions to the latest CLI/MSBuild bits and see what happens.

frankbuckley picture frankbuckley on 20 Jan 2017

I am no longer seeing errors with 1.0.0-rc4-004771. Thanks!

frankbuckley picture frankbuckley on 2 Mar 2017

This doesn't seem to be fixed for me with 1.0.1 on linux

feliwir picture feliwir on 18 Mar 2017

It is not fixed on 1.0.4 on Linux as well

alex-fomin picture alex-fomin on 11 May 2017
👍1

@alex-fomin @feliwir Can you open a new issue with fresh repro steps. I suspect it's something different. Thanks.

nguerrera picture nguerrera on 11 May 2017

Well it's really easy to reproduce with NET SDK 2.0.2

dotnet publish -o /home/user/publish_dir

Will publish everything from a solution into one folder in case argument of -o is an absolute directory. This is super convenient and a way to go in case you make sure projects are compatible. Though publish commands can fight for files of course.

sherlock1982 picture sherlock1982 on 7 Nov 2017
👍3

this still occurs in 2.1.1, linux. fails sometimes when running the dotnet test, happens mostly on our builder machine but not only

benyitzhaki picture benyitzhaki on 24 Jun 2018
👍2

I've been seeing this issue as well consistently on our build servers, 2.1.5.

cjablonski76 picture cjablonski76 on 20 Nov 2018

@cjablonski76 this issue has been closed. If you have a consistent repro for this, could you please file a new issue with the repro steps or (even better), a repro repo that we can clone and reproduce the problem ourselves?

livarcocc picture livarcocc on 20 Nov 2018

So far I haven't been able to get a consistent repro of it, if I can figure it out I'll open an issue. Thanks!

cjablonski76 picture cjablonski76 on 21 Nov 2018

@livarcocc I just opened a new issue with a repository to reproduce the issue with: https://github.com/dotnet/cli/issues/10406

cjablonski76 picture cjablonski76 on 28 Nov 2018
Was this page helpful?
0 / 5 - 0 ratings

Related issues

gkhanna79 picture gkhanna79  路  3Comments
thomaslevesque picture thomaslevesque  路  3Comments
natemcmaster picture natemcmaster  路  3Comments
srayuws picture srayuws  路  3Comments
dasMulli picture dasMulli  路  3Comments