Home: project.assets.json does not work with shared intermediates

[davkean]

Files that generated into the intermediates, typically always include the project name in the name of the file:

Debug\ConsoleApp146.AssemblyInfo.cs
Debug\ConsoleApp146.csproj.FileListAbsolute.txt
Debug\ConsoleApp146.csprojResolveAssemblyReference.cache
Debug\ConsoleApp146.dll
Debug\ConsoleApp146.pdb
ConsoleApp146.1.0.0.nuspec
ConsoleApp146.csproj.nuget.g.props
ConsoleApp146.csproj.nuget.g.targets

The assets file does not:

project.assets.json

This is going to lead to clashes when projects share intermediate directories, which is pretty common in large projects such as Roslyn.

[rrelyea]

good issue. we think we can change without too much work...but likely too late for rtm.
/cc @jainaashish @emgarten

[emgarten]

Agreed, this would be a good change. We've been working towards making this file name/path extensible, but there a few places still expecting project.assets.json.

[davkean]

Yeah, let's fix it later - just make sure you coordinate with us.

[emgarten]

$(ProjectAssetsFile) will contain the full path that NuGet writes to. This of course won't exist if the project hasn't been restored yet, so it might be hard to rely on this.

[davkean]

Good to know, we should be respecting that: https://github.com/dotnet/roslyn-project-system/issues/1437

[davkean]

Make note - this also affects two projects in the same directory: https://github.com/dotnet/roslyn-project-system/issues/1528

[caleblloyd]

When I define a different IntermediateOutputPath, nuget files still always end up in $(ProjectDir)\obj\. I am following this MSDN blog post

Example:

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>netcoreapp1.1</TargetFramework>
    <IntermediateOutputPath>$(SolutionDir)\obj\$(Configuration)\$(MSBuildProjectName)\</IntermediateOutputPath>
    <OutputPath>$(SolutionDir)\bin\$(Configuration)\$(AssemblyName)\</OutputPath>
  </PropertyGroup>

All other files get moved correctly, except for these 3 which still appear in $(ProjectDir)\obj\:

App.csproj.nuget.g.props
App.csproj.nuget.g.targets
project.assets.json

[emgarten]

~@caleblloyd NuGet reads BaseIntermediateOutputPath~

update by @nkolev92

NuGet now respects MSBuildProjectExtensionsPath.

Please refer to microsoft/msbuild#1603 and NuGet/NuGet.Client#2131 and NuGet/NuGet.Client#2121.

Original comment: https://github.com/NuGet/Home/issues/4463#issuecomment-404863327

[davkean]

Given where the base intermediate path is set, you can't set it without expanding SDK into explicit imports of a props and targets.

[caleblloyd]

I am trying to run a Docker container that has code mounted into it while also developing on the host inside of an IDE. The IDE keeps running Dotnet Restore and conflicting with the Docker container's Dotnet Restore.

I have to do multiple mounts to exclude the bin and obj folders if they are in the default path, so I thought maybe I'd move them using BaseIntermediateOutputPath. But this causes Entity Framework Tools and dotnet test to not work properly.

Is there another solution to this besides for waiting for the ability to name project.assets.json different names on the Host and in Docker? I can run different build configurations on the Host and Docker (e.g. run dotnet run -c Debug on the Host and run dotnet run -c Docker on Docker) and right now the only conflict I'm getting is that Nuget keeps clobbering the other side

[davkean]

Can you expand on the issues you have with EF and dotnet test when you change BaseIntermediateOutputPath with a bug over on http://github.com/roslyn-project-system? I'll move them to the appropriate repros if it's a bug in those tools.

[caleblloyd]

Yes, I will do that in a few days. Thanks!

[jdasilva]

@davkean Does this mean that having a shared intermediate folder is supported once these issues are fixed?

[davkean]

@jdasilva These are two issues that I found scouting it - but I've not confirmed that it's fully supported yet. Sounds like @caleblloyd was running into issues that I'd like to capture

[jdasilva]

Thanks, I was thinking about a comment like this https://github.com/dotnet/sdk/issues/760#issuecomment-276259786. I've been using a common intermediate folder for a couple of projects since at least VS2013 and this still works under VS2017 with the old project system. I'd love to move to the new one, so I really meant "supported" in the sense of if something blocks this will it be considered a bug? You seem to be doing that already which is great and I just want to make sure I'm on the same page.

[davkean]

@jdasilva Sorry missed this comment (for some reason NuGet issues go to my personal email).

Yes we'd consider anything blocking multiple projects from building to a common intermediate as a bug - this is the reason that we prefix most things in obj with the project name. This issue is the last one that I know that is blocking building to a common intermediate folder (MSBuild fixed their last one)

[davkean]

Here's another issue that results as a problem of this - https://github.com/dotnet/project-system/issues/1935. Basically, building a new csproj in the same directory as the old csproj, breaks the build because the old csproj picks up the same assets file.

[emgarten]

@davkean when will https://github.com/dotnet/project-system/issues/1437 be fixed?

[nguerrera]

Possible issues in SDK too: dotnet/sdk#1438

[nkolev92]

Some related work:
https://github.com/NuGet/NuGet.Client/pull/2056

[nickrandolph]

Just got this issue where we have two csproj in same folder. One is multi-targetted (netstandard2.0, net461 etc), the other just net461. Can't resolve a reference to a net45 nuget package.

[davkean]

@emgarten If you folks are blocked on dotnet/project-system#1437 to fix this - I'll fix it immediately in 15.8.

[nkolev92]

@davkean
I'm not sure if that's the only "blocker", but fixing that would certainly go a long way.
//cc @rrelyea

[piksel]

For anyone else finding this and needs a workaround, save the following as Directory.Build.props in your project directory:
.xml <Project> <PropertyGroup> <MSBuildProjectExtensionsPath>obj\$(MSBuildProjectName)\</MSBuildProjectExtensionsPath> </PropertyGroup> </Project>
This will cause project.assets.json to be written to $(ProjectDir)\obj\ProjectFileWithNoExtension\project.assets.json. And works fine with multiple .csproj in the same directory (what I needed).

Comment on this gist if it's not related to the issue: https://gist.github.com/piksel/950646e301e0fd994465b26773170494

[lucasgl]

Any reason BaseIntermediateOutputPath cannot also override MSBuildProjectExtensionsPath?
(I would guess ordering...)

[nkolev92]

@lucasgl

NuGet now respects MSBuildProjectExtensionsPath.

Please refer to https://github.com/Microsoft/msbuild/issues/1603 and https://github.com/NuGet/NuGet.Client/pull/2131 and https://github.com/NuGet/NuGet.Client/pull/2121.

MSBuildProjectExtensionsPath defaults to BaseIntermediateOutputPath, the trick you need to set it early enough in the evaluation to avoid these problems. Doing in the Directory.Build.Props does just that.

[BrightLight]

Glad I found this. I ran into the same issue last week. Is there any ETA on when this will be fixed? Our build server calls msbuild with /p:BaseIntermediateOutputPath=<somefolder> (a "Temp" folder inside the working directory of the build job). All was fine as long as the solution only contained .Net framework targets. Now colleagues have added two(!) .Net Standard projects which breaks the build because, of course, there can only be one project.assets.json in this "Temp" folder.
Is there a temporary workaround available?

[piksel]

@BrightLight I posted a workaround

[jeffkl]

I'd like to revive this bug and work on a fix.

There is no way to guarantee that restore will result in a successful build unless the MSBuildProjectExtensionsPath is unique for each project. If you try to use a rooted path like obj\$(MSBuildProjectName) you could still have two projects in the tree with the same name. So renaming project.assets.json wouldn't help us there.

The only way to guarantee that the assets from a restore are unique per project is to have a unique path per project. So I suggest that we fail restore if multiple projects share the same directory. Otherwise you get non-deterministic restores since there's no guarantee what order the project.assets.json is written. Not to mention that you get into a state where restore succeeds but your build fails with compiler errors about missing types because package assets are wrong.

Does everyone agree that we should just block the ability for projects in restore to share MSBuildProjectExtensionsPath?

[SwooshyCueb]

With this solution, could two project files exist in the same directory if they set MSBuildProjectExtensionsPath to different locations?

[jeffkl]

@SwooshyCueb yes that would work. As long as the MSBuildProjectExtensionsPath is unique, NuGet would be restore just fine. However, MSBuild will give you an error if you build in that directory and don't specify a project to build.

[nguerrera]

I think a best effort should be made to make the path unique in common cases. Consider the trivial case of two projects in the same folder with default output paths. For ages, this simple case worked because everything put the project name in obj files, until assets file came along. So I think it is good to have the error as you suggest, but I still see value in being unique by default up to project name.

[nkolev92]

There are no technical blockers anymore afaik.

The concern would be the migration plan.
For better or worse newer NuGet versions are often used with older build tooling, so this change would potentially break that.

This needs to be carefully analyzed to avoid fall outs similar to what we had to fix with https://github.com/dotnet/sdk/pull/14517.