Sdk: Setting BaseIntermediateOutputPath breaks netstandard2.0 builds

[Zastai]

Setting BaseIntermediateOutputPath breaks netstandard2.0 builds

I have a solution with multiple projects.
These import a common properties file (as the first thing in the project file), which set BaseIntermediateOutputPath (along with IntermediateOutputPath and OutputPath) in order to have build artifacts outside the source tree.
This has so far worked fine.

However, with the 2.0 SDK, this breaks when targeting netstandard2.0; none of the types from the standard libraries are found.
As such, this seems related to #803; however, the same problem does not occur when targeting netcoreapp2.0 (which I would assume to have the same basic setup).

Steps to Reproduce

1. run: dotnet new classlib
2. delete the obj folder
3. edit project file, adding <BaseIntermediateOutputPath>foo\</BaseIntermediateOutputPath>
   _(any value works as long as it's not obj\)_
4. run: dotnet restore
   _this correctly creates the relevant files in foo\_
5. run: dotnet build
   _lots of CS0518 and CS0246 errors ensue_

Partial Workaround

To a certain extent this can be worked around by explicitly adding PackageReference entries for the needed standard library assemblies (e.g. adding System.Runtime 4.3.0 makes the basic classlib sample compile). Note that adding a reference to NETStandard.Library 2.0.0 does _not_ help; that results in a build warning plus the same compilation issues).
However, for some new APIs (e.g. SerializableAttribute and ExternalException) it does not seem so obvious to find which packages to reference (the API docs list assemblies, not packages; neither System.Runtime nor System.Runtime.InteropServices provides ExternalException, despite what the docs suggest).

Note that without #1486 I could probably just customize RestoreOutputPath instead (unless there are other files that are created via BaseIOP instead of IOP/ROP/OP).

[ricky-hartmann-zocdoc]

This doesn't break the build with a netcoreapp2.0 application, but the output result is unusable because it can't load assemblies that it thinks it should be able to. Same issue as this it seems like, and an example project I've gotten to work with using netcoreapp1.1 can be seen here. The master branch on this uses netcoreapp1.1 and it builds fine

[dasMulli]

It is generally unsafe to set BaseIntermediateOutputPath in the "main body" of a project.

There are two workarounds at the moment:

1. Create a Directory.Build.props file at project or solution level that define the variable:

<Project>
  <PropertyGroup>
    <BaseIntermediateOutputPath>$(MSBuildThisFileDirectory)obj\$(MSBuildProjectName)\</BaseIntermediateOutputPath>
  </PropertyGroup>
</Project>

This file will be automatically imported by all projects in the directory tree early enough to set the property (= imported in the Microsoft.Common.props file before it uses BaseIntermediateOutputPath).

1. Move from the <Project Sdk=".."> syntax to explicitly importing the Sdk's props and targets file and set the property early in the project file:

<Project>
  <PropertyGroup>
    <BaseIntermediateOutputPath>some\other\dir\</BaseIntermediateOutputPath>
  </PropertyGroup>
  <Import Project="Sdk.props" Sdk="Microsoft.NET.Sdk" />

  <!-- All your project's other content here -->

  <Import Project="Sdk.targets" Sdk="Microsoft.NET.Sdk" />
</Project>

[hartmannr76]

Option 2 worked great!

I'm going to try option 1 out later because that seems like it could be shared better across a solutions projects.

[Zastai]

If that Directory.Build.props indeed works at the solution level, that is a fairly clean workaround. Option 2 would also be OK, but seems more fragile (are future SDKs guaranteed to only need Sdk.props and Sdk.targets?).

Is there (usable) documentation on which variables MSBuild/CoreSDK/... use, and which one(s) are safe to set when? And which optional props/targets/tasks files are loaded (and when/whence)?

[davkean]

As @dasmulli mentioned, due to the places that we set the restore directory for NuGet.props/targets/assets, setting $(BaseIntermediateOutputPath) inside body of project should have never "properly" worked/respected in 1.1 - are you sure this worked?

We really need a topic on this in the docs. @rainersigwald do know how/where I would file this? Do I just file in the docs repo?

[rainersigwald]

@davkean we can either add it to https://docs.microsoft.com/en-us/visualstudio/msbuild/common-msbuild-project-properties or start a new page on docs. Usually easiest to file a bug on MSBuild since that's where those of us who could write the initial draft of the doc hang out, but a bug on docs should eventually percolate to us, too.

At the moment, we're without a doc writer so it might be a bit.

[Zastai]

Well, I needed APIs that weren't in netstandard1.6, so while I had switched to the new project format (for the multitargeting and built-in nuget pack), I was targeting the desktop framework only (net20/35/40/45). That worked just fine (it still created those restore files, but I suppose they don't actually get used unless target core/std).
Only when I added netstandard2.0 did things start to break.

Workaround 1 does for me, so I'm good.
(Note that that doc page also claims that setting IntermediateOutputPath means that BIOP has no effect; given that it's also the basis for RestorePath, that's not accurate.)

One thing I will say: docs on ALL properties for the various languages would be nice; I was recently bitten by the fact that I set an Instrument property, which apparently triggered the use of a (pretty much undocumented) C# compiler option. Would it not make sense to add namespaces, or at least use consistent prefixes? I would have been far less likely to set CscInstrument or Csc:Instrument. That could even serve as a basic form of documentation - e.g. any property starting with "Sdk:" is explicitly for internal sdk use only.

[davkean]

@Zastai Thanks for the feedback and ideas, agreed our documentation story/IntelliSense docs aren't great. It's something that I think about a lot, and have some other ideas to make it better (for example, by stop showing irrelevant stuff).

[dasMulli]

There's also https://github.com/dotnet/docs/issues/2642

[nguerrera]

We have #803 to detect the case of BaseIntermediatePath set too late and trigger an error or warning. We absolutely do need better documentation on this. I've added an explicit comment about it to dotnet/docs#2642. I'm closing this since the action items (docs + diagnostics) are tracked elsewhere.

[rraallvv]

I couldn't make option 2 work in @dasMulli's post for a Xamarin.Forms project on Visual Studio for Mac, for option 1 the configuration setting (Debug/Release) is not available.

Does anyone know of a way to get the configuration setting from Directory.Build.props? Thanks.

[dasMulli]

@rraallvv the Configuration property is either set by the project system loading a project or defaulted in the project file itself, so you'd need to add a default for Configuration:

  <PropertyGroup>
    <Configuration Condition="'$(Configuration)' == ''">Debug</Configuration>
    <BaseIntermediateOutputPath>$(MSBuildThisFileDirectory)obj\$(Configuration)\$(MSBuildProjectName)\</BaseIntermediateOutputPath>
  </PropertyGroup>

This default would be performed by the time Sdk.props is imported, but in case a Directory.Build.props is used or msbuild properties are placed above the sdk import, they won't see this default when setting properties.

[rraallvv]

@dasMulli thanks a lot.

[DoCode]

@nguerrera, @dasMulli, please reopen this issue!

[DoCode]

Because:

When I use a netcoreapp2.x with a _project reference_ to a netstandard2.x project and I set the BaseIntermediateOutputPath, then the build/publish failed with this error:

~\Microsoft.NET.Sdk\build\Microsoft.PackageDependencyResolution.targets(201,5): error :
Assets file '~\netcoreapp-netstandard-refs\src\_build\bin\obj\project.assets.json' doesn't have a target for '.NETStandard,Version=v2.0'.
Ensure that restore has run and that you have included 'netstandard2.0' in the TargetFrameworks for your project.
[~\netcoreapp-netstandard-refs\src\dotnetcore-lib\dotnetcore-lib.csproj]

Steps to reproduce:

1) Create class lib
dotnet new classlib --name dotnetcore-lib --output src/dotnetcore-lib
2) Create web app
dotnet new web --name aspnetcore --output src/aspnetcore
3) Add class lib as reference to web app
dotnet add .\src\aspnetcore\aspnetcore.csproj reference .\src\dotnetcore-lib\dotnetcore-lib.csproj
4) Restore and publish (without custom BaseIntermediateOutputPath) => that works 👍
dotnet restore .\src\dotnetcore-lib\dotnetcore-lib.csproj dotnet restore .\src\aspnetcore\aspnetcore.csproj dotnet publish .\src\dotnetcore-lib\dotnetcore-lib.csproj dotnet publish .\src\aspnetcore\aspnetcore.csproj
5) Restore and publish (with custom BaseIntermediateOutputPath) => that fails 👎
dotnet restore .\src\dotnetcore-lib\dotnetcore-lib.csproj /p:BaseIntermediateOutputPath=..\..\_build\bin\obj\dotnetcore-lib\ dotnet publish .\src\dotnetcore-lib\dotnetcore-lib.csproj /p:BaseIntermediateOutputPath=..\..\_build\bin\obj\dotnetcore-lib\ dotnet restore .\src\aspnetcore\aspnetcore.csproj /p:BaseIntermediateOutputPath=..\..\_build\bin\obj\aspnetcore\ dotnet publish .\src\aspnetcore\aspnetcore.csproj /p:BaseIntermediateOutputPath=..\..\_build\bin\obj\aspnetcore\

What the hell we make it wrong?

[davkean]

@DoCode Please file a new bug.