Sdk: Build and publish should (optionally?) clean output folder

[nil4]

Steps to reproduce

In an empty folder named Test, run dotnet new. Replace the generated project.json file with:

{
  "version": "1.0.0-*",
  "buildOptions": { "emitEntryPoint": true  },
  "dependencies": {},
  "frameworks": { "net461": {} }
}

Run dotnet restore then dotnet build. The project builds; Test.exe and Test.pdb are created under bin\Debug\net461 and bin\Debug\net461\win7-x64.

In the Test folder, create an AppConfig.json file. The content is irrelevant; in a real application this file would simply need to be copied to the output folder or included in the published output.

1. The project is found to not follow the company naming guidelines.
   Rename the Test folder to Test2, then run dotnet build.
   Test2.exe and Test2.pdb are created in the output folders, but the obsolete Test.exe and Test.pdb files are still present.
2. Under buildOptions, add: "copyToOutput": "AppConfig.json".
   Run dotnet build and note AppConfig.json is copied to bin\Debug\net461\win7-x64.
   The config file is later renamed or removed. Remove the copyToOutput setting, or rename/remove the AppConfig.json file and run dotnet build again.
   Note AppConfig.json is still present in the output folder.
3. Add a new section to project.json: "publishOptions": { "include": "AppConfig.json" }.
   Run dotnet publish and note AppConfig.json is copied to bin\Debug\net461\win7-x64\publish.
   The config file is later renamed or removed. Remove the publishOptions setting, or rename/remove the AppConfig.json file and run dotnet publish again.
   Note AppConfig.json is still present in the publish folder.

Expected behavior

A flag can be passed to dotnet build and dotnet publish that removes any files or folders in the output which have not been generated by the current operation.

With the flag specified, the end result should be the same as if the output folder was deleted prior to invoking the build/publish, but without losing the benefits of incremental build.

Actual behavior

No such flag is available. To ensure a clean build/publish output, one needs to delete the bin folder or pass a unique --output path on each invocation, thus losing incremental build.

Environment data

dotnet --info output:

.NET Command Line Tools (1.0.0-rc2-002611)

Product Information:
 Version:     1.0.0-rc2-002611
 Commit Sha:  bf8f0edd89

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.10586
 OS Platform: Windows
 RID:         win10-x64

[meyerrj]

It would be helpful if the dotnet publish command had an argument such as --clean or --delete-existing. Such an option would be similar to the checkbox in Visual Studio's File Publish Options for "Delete all files prior to publish".

Having a dotnet clean command would be nice as well... The current alternatives appear to be manually deleting stale files from the bin/obj folders, or adding custom tasks for cleanups.

[jch254]

I agree this would be fantastic.

A somewhat clean way around this is to specify an output folder when using dotnet build/dotnet publish with the --output folder_name argument and utilising a prebuild/prepublish script in project.json that removes/cleans out folder_name.

[ArveSystad]

A flag to do cleaning would be great.

As of now, for CI purposes, one has to add separate steps to make sure this happens, or else you'll end up with a folder structure like PackageRootProject/OutputDir/OutputDir/OutputDir/OutputDir..... It seems to add another level of "OutputDir" for every time I run publish without deleting the output between runs.

[livarcocc]

Which version of the tooling are you using? project.json or msbuild?

If you are using CLI 1.0.0 or 1.0.1, you can run dotnet clean to clean your bin folder.

Also, on the new tooling, this Appending of OutputDir should not happen by default. Of course, since this is MSBuild, you can modify the properties to cause this, but this is something that you should fix in your project.

[ArveSystad]

Tested now on my computer at home, reproduces easily there.

Running

• VS2017 Professional (15.0.0+26228.10)
• CLI 1.0.0
• Windows 10 (10.0.14393)

Reproduces by simply creating a new WebAPI project from File-New-Project => ASP.NET Core Web Application (.NET Core).

From the project folder, I say dotnet publish --output OutputFolder a few times, and the nested OutputFolders appear.

dotnet clean cleans the bin folder, but that's not really where the problem lies. dotnet publish (so the published files end up in bin) works fine, no problem there. Seems like an issue when specifying the output folder.

[livarcocc]

Is your csproj a vanilla csproj? I just tried this and it did not repro for me. I ran dotnet publish --output outputdir multiple times and ended up with a single outputdir folder with the right contents inside of it. No nested outputdir folders.

[ArveSystad]

Yup, 100% vanilla from the New Project dialog. Then, before I do anything, i say dotnet publish --output outputdir twice, and I end up with /outputdir/outputdir/<files>

However, I do notice now, that the innermost outputdir only contains some config files etc, no compiled DLLs - they only live on the first, correct level.

[dasMulli]

I am able to reproduce it as well, but only when using the web sdk tough.

(macOS 10.12, 1.0.1 CLI)

1. dotnet new console
2. update to web sdk
3. create a foo.txt
4. update foo.txt for publishing using <None Update="foo.txt" CopyToPublishDirectory="Always"/>
5. dotnet publish --output dirA
6. dotnet publish --output dirB

results in:

Source with ./repro.sh:
foo.zip

[ArveSystad]

Yep, I've only been testing with web projects, so that might just be right. Tried creating an empty console app now, which worked as it should (no nested output folders),

[livarcocc]

@ArveSystad Can you file this issue under aspnet/websdk for them to take a look?

[ArveSystad]

Yep! Will do first thing tomorrow morning.

[urbanhusky]

I've had such an issue a while ago (project.json days) - the issue in my case was that I did not exclude the output directory from content or compilation.

So if you do not use an output directory below ./bin/ (as in ./bin/Release/publish), your output directory might get picked up during the compilation and ends up inside the output directory. So each layer of output/output will contain the previously compiled version.

I'm currently trying to get a clean output directory - or even a proper clean before I run dotnet publish...

[jhudsoncedaron]

Bug is still in existence. I discovered the output folder contains binaries that can't be built anymore.

[nathan-alden-hp]

This bug is horrendous because it violates the principle of least astonishment. A publish directory should always contain only the files required to run the app at the moment of compile and publish, and never anything else. At the very least, a switch should've been added when the publish command was added to support deleting the publish output directory first.

Additionally, dotnet clean only cleans the files from bin\Release\netcoreapp2.0, e.g., and not bin\Release\netcoreapp2.0\publish, which is the default location for publish output.

[slaneyrw]

The solution to this was to execute dotnet clean to purge the bin folder.... great IF your publish path is a child of your output. The issue is still present if your output folder is somewhere else.

Please re-open and fix

[StefanoVuerich]

I agree with @meyerrj , something like that is definitely needed.

[StillLearnin]

This is still a current issue. We just updated a Nuget package and all the tests succeeded but the publish folder still had the old version and so production failed. We fixed it by adding steps in the build process to delete the obj and bin folders but this should not be necessary.

[Nburnettt]

If this is not included during publish, could you please point out or file a canonical method for deleting the bin/ and obj/ directories?

Very frequently, packages changing versions produces the
"Duplicate 'System.Reflection.Assembly...'"
errors which require manual/ad-hoc deleting of the bin and obj directories. If publish can not handle these cases, it seems we should at least have a "dotnet clean --purge" or something of the sort that clears these files.

[dasMulli]

@Nburnettt this sounds more like you have nested project structures, mulitple csproj files in the same directory or something similar.. this can be fixed by adding an exclusion for other project paths (e.g. https://github.com/SabotageAndi/NewCSProj/blob/248fd105d2b560009de0dccf0d17f124f07b1716/MultipleProjecsInSameFolder/multiprojecttest_project2.csproj#L10)

Do note that publishing (dotnet publish) now has an incremental behavior implemented by https://github.com/dotnet/sdk/pull/3957 which should make its way into 5.0.0 previews.