Msbuild: Setting BaseIntermediateOutputPath correctly in a SDK-based project is hard

[srivatsn]

NuGet restore drops the project.assets.json file to the $(BaseIntermediateOutpath). If a user customizes that by setting the property in a SDK-based project like so:

<Project Sdk="Microsoft.NET.Sdk">
   <PropertyGroup>
         <BaseIntermediateOutputPath>C:\blah</BaseIntermediateOutputPath>
   </PropertyGroup>
</Project>

Then the project.assets.json gets dropped to that folder correctly. However if there are nuget packages in this project that have taskstargets then the generated project.nuget.g.propstargets are imported by Microsoft.Common.Props. Therefore BaseIntermediateOutputPath needs to be defined before the import of the common props and for that they have to know to expand the SDK import:

<Project>
   <PropertyGroup>
         <BaseIntermediateOutputPath>C:\blah</BaseIntermediateOutputPath>
   </PropertyGroup>
   <Import Sdk="Microsoft.NET.Sdk" Project="Sdk.props" />
...
   <Import Sdk="Microsoft.NET.Sdk" Project="Sdk.targets" />
</Project>

I don't know what we can do to fix this but logging an issue here so that atleast this serves as documentation for people running into this issue.

[cdmihai]

An easier way is to use the new Sdk attribute for import elements:

<Project>
  <PropertyGroup>
    <BaseIntermediateOutputPath>C:\blah</BaseIntermediateOutputPath>
  </PropertyGroup>

  <Import Project="Sdk.props" Sdk="Microsoft.NET.Sdk/1.0.0" />
...
  <Import Project="Sdk.targets" Sdk="Microsoft.NET.Sdk/1.0.0" />
</Project>

See #1493

[srivatsn]

Has that syntax been implemented already?

[cdmihai]

According to this diff it was: https://github.com/Microsoft/msbuild/pull/1492/files#diff-325c8a74f9ae27c1b3f8870e9cb64678L2310

[rainersigwald]

For posterity: another option is using a Directory.Build.props to set this. See https://github.com/dotnet/sdk/issues/802.

[Ziflin]

Is there are "correct" solution for this? All I'm trying to do is move the bin\ and obj\ folders that .NET Core projects dump in the project folder into a Build\ directory at the solution level so they're separate from the source code. I added this:

    <PropertyGroup>
        <OutputPath>$(SolutionDir)\Build\$(ProjectName)\bin\$(Configuration)</OutputPath>
        <BaseIntermediateOutputPath>$(SolutionDir)\Build\$(ProjectName)\obj</BaseIntermediateOutputPath>
    </PropertyGroup>

to the top of our .csproj files, but it appears that $(ProjectName) is not set at this point? Is there some correct way to accomplish this as it's driving me crazy. :) Note that it appears that $(SolutionDir) works fine. It also looks like OutputPath always has the netcoreapp1.1 or netstandard1.6.1 folder appended as well.

[rainersigwald]

Both moving the imports as in https://github.com/Microsoft/msbuild/issues/1603#issuecomment-275271522 and using a Directory.Build.props as in https://github.com/Microsoft/msbuild/issues/1603#issuecomment-277726334 are "correct", @Ziflin. Chose whichever meets your needs better.

[Ziflin]

I don't have an imports section. This is just a simple .NET Core Console App and Lib solution. So the console app uses:

<Project Sdk="Microsoft.NET.Sdk">
    <PropertyGroup>
        <OutputType>Exe</OutputType>
        <TargetFramework>netcoreapp1.1</TargetFramework>
    </PropertyGroup>
</Project>

And I added the PropertyGroup in my first comment above this PropertyGroup. Is $(ProjectName) supposed to be valid or is there some other variable name I can use that is for the project's name? It's basically treating it like it's not set.

Sorry I'm not clear on how to use the Directory.Build.props from that other comment/issue..

[rainersigwald]

@Ziflin your two options are:

Create a file named Directory.Build.props in a folder above your projects (maybe next to your solution? you know your repo layout best) that sets the properties you want to set. It will be automatically included in any project below it in the directory structure.

Or change your project file from the implicit imports model to explicit imports, so that you can control order. These are exactly identical:

-<Project Sdk="Microsoft.NET.Sdk">
+<Project>
+ <Import Project="Sdk.props" Sdk="Microsoft.NET.Sdk" />
    <PropertyGroup>
        <OutputType>Exe</OutputType>
        <TargetFramework>netcoreapp1.1</TargetFramework>
    </PropertyGroup>
+ <Import Project="Sdk.targets" Sdk="Microsoft.NET.Sdk" />
</Project>

After you've made the implicit imports explicit, you can add or move things around them to affect relative order.

[Ziflin]

@rainersigwald Thanks for the help, but neither of those methods seem to have any effect on the fact that the $(ProjectName) variable is not set which makes it very difficult to do something like:

<BaseIntermediateOutputPath>$(SolutionDir)\Build\$(ProjectName)\obj</BaseIntermediateOutputPath>

And have it create a Build\MyProject\obj folder under the solution. If I hardcode a name for $(ProjectName):

<BaseIntermediateOutputPath>$(SolutionDir)\Build\MyProject\obj</BaseIntermediateOutputPath>

then it works as expected, but then I am unable to use a single Directory.Build.props file for all the projects in the solution. The $(SolutionDir) and $(Configuration) variables seem to work fine so I wasn't sure if I just had the name wrong for the project name variable or if it was a bug.

[rainersigwald]

Would $(MSBuildProjectName), which is a well-known property automatically populated by MSBuild based on the file name, work for you?

[Ziflin]

That works perfectly! Sorry for the confusion. I saw that $(SolutionDir) was working and assumed $(ProjectName) would as well.

[Nirmal4G]

From @srivatsn
BaseIntermediateOutputPath needs to be defined before the import of the common props and for that they have to know to expand the SDK import.

Even easier way is to generalize implicit imports placement.

<Project Sdk="Custom.Sdk">
  <PropertyGroup Evaluation="BeforeImplicitProps">
    <BaseIntermediateOutputPath>..\..\Build</BaseIntermediateOutputPath>
  </PropertyGroup>
...
  <PropertyGroup Evaluation="AfterImplicitTargets">
    <SomeImportantPropertyAfterTargets>Value!!!</SomeImportantPropertyAfterTargets>
  </PropertyGroup>
</Project>

would translate to this

<Project>
  <PropertyGroup>
    <BaseIntermediateOutputPath>..\..\Build</BaseIntermediateOutputPath>
  </PropertyGroup>

  <Import Project="Sdk.props" Sdk="Custom.Sdk/1.0.0" />
...
  <Import Project="Sdk.targets" Sdk="Custom.Sdk/1.0.0" />

  <PropertyGroup>
    <SomeImportantPropertyAfterTargets>Value!!!</SomeImportantPropertyAfterTargets>
  </PropertyGroup>
</Project>

Here, assume Custom Sdk uses common Sdk. This is helpful in creating custom .proj file with Sdk story to them like I can use any custom Sdk that uses common Sdk or itself. See Issue #1686

This is one way to fix the problem and It does it even before all the props and after all the targets, which would be useful for many debugging and logging scenarios.

[jeffkl]

@Nirmal4G Directory.Build.props is imported before BaseIntermediateOutputPath is set so I would recommend you just use that.

[Nirmal4G]

@jeffkl I know!

But I want to set some properties (within the Project file) even before all the implicit props and have some targets after all the implicit targets, something along those lines!

[KevinH-MS]

Where in the SDK props do we implicitly import .props from nuget packages? I wanted to look at it and see if I can come up with any other clever tricks for my Sdk props (trying to avoid Directory.Build.props for the time being), but I can't find the actual place where we look at the project assets and import them.

[Nirmal4G]

See the lines here, the comments will tell you everything!

https://github.com/microsoft/msbuild/blob/b38e4ceeaaec36c5237ae698041e9b9f18c84876/src/Tasks/Microsoft.Common.props#L22-L63

And in the targets...

https://github.com/microsoft/msbuild/blob/b38e4ceeaaec36c5237ae698041e9b9f18c84876/src/Tasks/Microsoft.Common.targets#L116-L147

That is how nuget and other package managers (paket, etc) import the restored assets!

[Nirmal4G]

You can modify MSBuildProjectExtensionsPath to generalize your assets output!

[dasMulli]

It's not just MSBuildProjectExtensionsPath though, as I wrote for https://stackoverflow.com/questions/45575280/msbuild-nuget-restoreoutputpath-how-to-make-it-work, there are more properties that need to work together:

• BaseIntermediateOutputPath - used to construct:
  
  
  
  • ProjectAssetsFile (SDK targets)
  
  
  • MSBuildProjectExtensionsPath if unset (MSBuild - Microsoft.Common.props)
  
  
  • RestoreOutputPath (NuGet targets)
  
  
• MSBuildProjectExtensionsPath - could theoretically be set to something different
• RestoreOutputPath - tells restore where to drop assets file and extension targets.

If those three don't point to the same directory -> 💩

The dangerous part is that those three properties are coming from different components so if you're not careful about which one is set where (=> set base..path early or set all), you won't have a good time.

[Nirmal4G]

I had the same experience, and that was the solution that saved me, thank you for that, but if you are using the latest tools you can get away with those problems.

But if you are using props/targets that are before the ProjectExtensions logic, It's better to shim up (_by detecting_ ImportProjectExtensionProps) those in Directory.Build.props/targets so that you can use them with your old toolsets ensuring forward compatibility.

[dsplaisted]

Thanks for the summary of the interrelated properties, @dasMulli.

MSBuildProjectExtensionsPath is the only one of these properties that is used in the common props, before the body of the project. So I think a good solution would be:

• Use MSBuildProjectExtensionsPath instead of BaseIntermediateOutputPath to construct the RestoreOutputPath. This would mean it would be OK to change BaseIntermediateOutputPath in the body of a project, and it would mean that the intermediate build output would go in the specified folder, but the NuGet assets, props, and targets files would continue to go in the default obj folder.
• Add an errror to MSBuild if the MSBuildProjectExtensionsPath is modified between when it's used in the common props and when it's used in the common targets. This would catch anyone who tries to override this property in the body of their project file.

[dasMulli]

but the NuGet assets, props, and targets files would continue to go in the default obj folder.

Maybe this needs to be checked with the original requirement of the build authors asking for this.. if the goal of setting any of these properties ist to make sure no obj folder (and then bin with similar modifications) is created at the project level, this may not help them at all.

An instance where I have seen this being used is having multiple csproj files in the same directory, which need to set all of the directories to a deeper level containing $(MSBuildProjectName) as well.

[Athari]

@dsplaisted I agree with @dasMulli — the only reason I'm changing $BaseIntermediateOutputPath and all other paths is to put all files generated by the build (a.k.a. not in version control) into one directory at the root of the solution.

A lot of .NET projects do this. I'm actually surprised that something as basic as puting all artifacts into one directory instead of dozens of directories all over the solution not only isn't the default, but requires so much jumping through the hoops.

[Nirmal4G]

I know backwards compat is important and I also know that generalization of output properties will benefit a lot of people like us who are using large projects with different structures.

We need that as @Athari says. So, there must be some way to do both!

[dsplaisted]

@dasMulli @Athari We have dotnet/sdk#867 for supporting a root output path where all generated files would go.

However, no matter what properties we offer, it's not possible for everything to work correctly if you set those properties in the body of your project (ie after the common .props files have been evaluated). That's because the common .props automatically import from $(MSBuildProjectExtensionsPath), which defaults to $(BaseIntermediateOutputPath), which defaults to obj. NuGet restore lays down .props and .targets files which need to be imported in this way. So the ideal thing to do is to override those properties before they are used, either via a Directory.Build.props file or before explicitly importing the SDK props and targets via the <Import Project="Sdk.props" Sdk="Microsoft.NET.Sdk" /> syntax.

What I'm proposing here is that if you don't set the properties in the "right" place, that we preserve correct build behavior (ie importing the .props and .targets from NuGet) over keeping absolutely everything out of your build folder, but building your project incorrectly.

Note that with my proposal if you set BaseIntermediateOutputPath in Directory.Build.props, then that value would flow through to the MSBuildProjectExtensionsPath and the RestoreOutputPath, so both the NuGet generated files as well as the intermediate build files would go under that folder.

[AmadeusW]

Just wanted to say that I chose .NET core for my project to have less boilerplate than in the full .NET framework. It's disappointing that I need to add Directory.Build.props to get the custom path to work.

It's also a shame that these properties work in certain conditions but not in others. It's a huge pain point for newcomers who expect a simple property to just work.

We're lucky to have

• patient and dedicated users who put up with this
• wonderful Microsoft engineers who readily help out on github

Having said that, such bad infrastructure

• prevents wider adoption of .NET Core - there is no way I would recommend it to anyone, despite being vested in success of this platform
• takes up time of engineers that need to address these issues

Sorry for polluting an engineering thread with this rant, but I want my voice to be heard. .NET Core has had issues like that since inception, and every time I start a new project or update VS I run into a new set of problems. Meanwhile, I don't see any progress happening in msbuild.

[dsplaisted]

@AmadeusW Thanks for the feedback. We're actively working on this issue. The fixed behavior will be that your build will succeed and it will use the folders you specified for the build output. The NuGet output will still go in the un-redirected obj folder in your project folder if you haven't applied one of the workarounds. This is because of technical limitations because some of the NuGet output (the generated .props file) needs to be used before the BaseIntermediateOutputPath property has been set.

[dasMulli]

Previously, one would have

<Project>
  <PropertyGroup>
    <BaseIntermediateOutputPath>C:\blah</BaseIntermediateOutputPath>
  </PropertyGroup>
  <Import Project="$(MSBuildExtensionsPath)\$(MSBuildToolsVersion)\Microsoft.Common.props" Condition="Exists('$(MSBuildExtensionsPath)\$(MSBuildToolsVersion)\Microsoft.Common.props')" />
  <!-- Project content. Lots and lots of project content. -->
  <Import Project="$(MSBuildToolsPath)\Microsoft.CSharp.targets" />
</Project>

to achieve the same without breaking potential extension points. Even with the new NuGet PackageReference, this has the same effects for all projects upgraded from packages.config, regardless of .net core or new project system.

So having the default case without any <Import> in magic places (misplaced property groups => dangerous) is definitely better.

[Athari]

@dasMulli I agree. If "magic" importing doesn't work (not working as intended is not a huge improvement over breaking build), then the feature should be dropped or at least not used by default, even though it looks nice in some cases. A lot of developers will try overriding BaseIntermediateOutputPath, so they will have to google for workarounds either way. And even if you know about the workaround, there're no easy way to convert "magic" properties to imports; you have to google and copypaste the code every single time. This is impractical.

[dsplaisted]

This was fixed with https://github.com/NuGet/NuGet.Client/pull/2131 and #3059

[gaziqbal]

To summarize, if I want to override my intermediate and output folders, it appears that following is needed in a Directory.Build.props file in my project or enclosing solution folder. Please call out if this is incorrect :)

<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
  <!-- Common properties -->
  <PropertyGroup>
    <!-- SolutionDir is not defined when building projects explicitly -->
    <SolutionDir Condition=" '$(SolutionDir)' == '' ">$([MSBuild]::GetDirectoryNameOfFileAbove($(MSBuildProjectDirectory), MySolution.sln))\</SolutionDir>
    <!-- Output paths -->
    <BaseIntermediateOutputPath>$(SolutionDir)bin\obj\$(Configuration)\$(MSBuildProjectName)\</BaseIntermediateOutputPath>
    <IntermediateOutputPath>$(SolutionDir)bin\obj\$(Configuration)\$(MSBuildProjectName)\</IntermediateOutputPath>
    <MSBuildProjectExtensionsPath>$(IntermediateOutputPath)\</MSBuildProjectExtensionsPath>
    <OutputPath>$(SolutionDir)bin\out\$(Configuration)\</OutputPath>
    <OutDir>$(OutputPath)</OutDir>
    <DocumentationFile>$(SolutionDir)bin\doc\$(Configuration)\$(MSBuildProjectName).xml</DocumentationFile>
  </PropertyGroup>
</Project>

[rolandh]

After reading this whole thread I cannot believe there is not a better solution. There should be a simple "ObjectPath" and "BinPath" in the project properties that can be filled out via the VS GUI.

It is such a simple request to move the intermediate directories. Instead we need a separate file? It has taken me an hour of trying things and reading this thread to figure out how to do such a "simple" task.

[dasMulli]

I feel your pain. Unfortunately that's the downside of making the common way super easy and clean - configuring things deep down get harder. mostly duet to history and the way things are layered.

[agaace]

"configuring things deep down get harder. mostly duet to history and the way things are layered"

Sorry but I don't consider changing the output directory for a project a "deep down" configuration.
"mostly duet to history and the way things are layered" is also not an acceptable excuse for me in a brand new framework. I mean, .NET has been around for nearly 2 decades, fine, it has acquired its quirks over time. But that's why .Core was not a version upgrade and an entirely new framework requiring migration - fine by me. And now you're brushing issues off by saying .Core has its "history baggage"? Sorry, not buying it.

Sorry for yet another rant. I just wanted to build a simple .Core project and been trying to figure this issue out since yesterday. Could have literally written hundreds of lines in the meantime but I had to spend all this time reading threads, jumping to links, reading documentation and trying to put it all together just to figure out how to configure my build. Turns out you either let Visual Studio generate everything for you, or you need a PhD in how MS Build works internally. There's nothing, null, nada, zero in between. OK, rant over.

[Nirmal4G]

@agaace You are absolutely right.

I myself from .NET framework days, and I still want many changes that are proposed here.

As long as there are docs documenting the migration for every new breaking change, I don't care about how many are there.

Forgive me, team, but what he said, I 💯 agree.