Sdk: When using custom $(BaseIntermediateOutputPath) the build still picks files from obj/

[mcartoixa]

I like to customize $(BaseIntermediateOutputPath) on my build servers to a centralized location so that I am very sure to start from fresh (just delete the folder and there you go).
On a development machine, Visual Studio generates files (e.g. AssemblyInfo.cs) in the default folder, (obj/, inside the project folder). When trying the build server script, it regenerates those files in the customized $(BaseIntermediateOutputPath) folder but it stills picks up the default ones that have been generated before for compilation...

My workaround is to add the following property on my projects in the build server script: DefaultItemExcludesInProjectFolder=bin/**;obj/**. My use case might be a fringe one, but I think these folders should be excluded by default, in a hardcoded way (cf. https://github.com/dotnet/sdk/blob/master/src/Tasks/Microsoft.NET.Build.Tasks/targets/Microsoft.NET.Sdk.DefaultItems.targets#L39).

Using .NET Core 3.0 preview 7.

[dasMulli]

Does your .gitignore exclude obj/ files? Are there files checked into source control?

If obj is not the base intermediate output path, i'd argue that the SDK should consider it for default items, especially if there are files checked into source control.
I don't think hardcoding a list of special directory and file names isn't a good practice. The web SDK does so for a few things where it is absolutely needed (node_modules) but that is specific to web-related tooling.

[mcartoixa]

@dasMulli In my case those files are excluded. But developers cannot test the server build because Visual Studio generates them and they conflict with the ones generated by the server build.

[livarcocc]

cc @dsplaisted to take a look.

It does feel somewhat corner case.

[dsplaisted]

Can you customize BaseIntermediateOutputPath for both the build servers as well as the development machine? That's what we do in this repo, for example, everything goes in the artifacts folder at the root of the repo.

[mcartoixa]

@dsplaisted As I said I already have a workaround for this. Customizing BaseIntermediateOutputPath by project is indeed another possibility, but:

• it requires editing the *.csproj manually, which I tend to do as little as possible.
• it requires changing the developers habits with Visual Studio (more documentation).

I would rather provide this property (and others) externally during _the build._ I always thought of BaseIntermediateOutputPath (and others like OutDir) as a promise of a customization point in the build. The promise was kept on .NET Framework (and I made extensive use of that). It seems to me that the promise is not kept on .NET Core (hence this issue).

Coming from .NET Framework, I have a lot of habits and assumptions that I may have to change. Please advise.

[mcartoixa]

BTW (this may be another issue) if I build a project with dependencies (instead of a solution, e.g. because of https://github.com/Microsoft/vstest/issues/411) it seems to me that the dependencies will not pick this property (and probably others): they will restore and build in obj\.

[dasMulli]

it requires editing the *.csproj manually, which I tend to do as little as possible.

This is what Directory.Build.props is now for - making it easy to apply configuration to multiple projects at once without editing a lot.

E.g. a file like this

<Project>
  <PropertyGroup>
    <BaseIntermediateOutputPath Condition="'$(BaseIntermediateOutputPath)' == ''">$(MSBuildThisFileDirectory)obj\$(MSBuildProjectName)\</BaseIntermediateOutputPath>
    <BaseOutputPath Condition="'$(BaseOutputPath)' == ''">$(MSBuildThisFileDirectory)bin\$(MSBuildProjectName)\</BaseOutputPath>
  </PropertyGroup>
</Project>

Would redirect the output of all the project files in subdirectories.

Changing the build infrastructure completely between "inner loop" and CI builds is possible, but I personally don't think it is a good idea to do so. If the problem arises from the devs running CI scripts locally, then I believe the devs should use a similar setup during development and have an understanding of their setup (>DevOps).
Your opinion on this may be different, I'd be interested to hear it.

[mcartoixa]

@dasMulli Thanks I will try that.
TL;DR I still see it as a workaround though. You used to be able to customize the build by providing those properties dynamically. This is no longer the case. I guess it is fine if it is on purpose.

As for the _devops_ thing, if we manage to agree on the principles (tough already in general) it may still translate differently for different teams. My experience has been that Visual Studio was an excellent IDE (still is), but that it tried to cram too many things into the projects: FxCop (yeah, I'm old), line counting, code coverage, live publication...

• What if I want to use an external tool in my build (e.g. OpenCover) ?
  
  
  
  • Don't forget to add references to every project that might need it (more documentation)...
  
  
  • Don't forget to restore the dependencies prior to opening the projects in Visual Studio.
  
  
  • The new _PackageReference_ system solves this problem. Still, don't forget to add the references in every project.
  
  
• What if my external tool does not integrate with MSBuild (e.g. CLOC or yaml2json)?
  
  
  
  • Create and maintain a custom MSBuild task, and see the previous point.
  
  
  • Use Exec in your build, but don't forget to download the tool prior to opening the projects in Visual Studio (or make them part of your repository).
  
  
• What if my external tool is only supposed to run once per build (e.g. ReportGenerator)?
  
  
  
  • Make a project the _special_ one (more documentation). The tool will only run when this project is built.
  
  
• What if I need a special build for using a tool (FxCop (again) used to need a build with the constant CODE_ANALYSIS set):
  
  
  
  • Add a specific configuration for this tool (or use the Debug one).
  
  

My take has been that there is a need for a coordinating script that covers all this: let Visual Studio handle edition, debugging and testing. Let an external script do the rest, like preparing an environment (download dependencies for instance). I love MSBuild, but some people prefer Cake, FAKE, psake: to each his own. Maybe if solutions where proper, editable MSBuild files this would have settled the case a long time ago...

My scripts are based on a set of conventions that make them reusable from repository to repository. If you're a new developer don't worry, you can use Visual Studio as is (_dev_) and even add new projects: it should work on your desktop and on the build server. Want to check how the build server will react to your latest changes? No problem, the build scripts are right there for you to try. And if you want to delve into them, which is encouraged (_ops_), well they are right there for you to see (and edit). Added benefit: build server configurations (AppVeyor, Travis, Azure Pipelines...) are very light and require very little maintenance.

As for the initial issue? Well if I managed to explain my process well enough you can see that it makes sense to me separate the outputs of your everyday work and the outputs of the build scripts. Your mileage may vary, and I'm certainly not saying everyone should work like this. But this has served me well over the years, and by applying the same principles (and conventions) to other technologies (ant, phing or gulp) it greatly eases the discovery of a new project.

Final thought: requiring people to edit XML files might turn them off. But it might force others to take an interest in MSBuild, which they wouldn't if they were not forced to. So, yeah...