Sdk: Publish should clean the output folder

[TheRealPiotrP]

Moving from https://github.com/dotnet/cli/issues/2142 on behalf of @billwert.

Verified this repros in Preview3 CLI.

---

If you publish into a folder that has binaries in it, the existing binaries aren't removed. For example, if you had a build system publishing to a specified folder with --output, were publishing stand alone, and switch to shared framework, your output folder will still have all the stand alone binaries in it.

[dsplaisted]

What's the expectation here? Should the publish operation logically delete all files from the publish folder before publishing? (The actual implementation should probably avoid deleting and recopying files that haven't changed.)

Or do files in the publish folder that weren't placed there by a previous publish operation need to be preserved? For example, if the app runs from the publish folder and saves some of its local data in files in that directory, do we need to avoid wiping those out?

Preserving files that were written to the publish folder by something else than the publish process would be a lot more complicated, but it is what MSBuild does for files in the output folder.

[billwert]

Good question. The original motivation for the issue was I was experimenting with switching between publishing stand alone and for shared FX. In that light deleting all the files seems fine.

I'm not sure that it's a reasonable expectation that files the app makes and leaves behind in the output directory, if it's run from there (in a debugging scenario, say) should persist. What would the expectation be when the clean verb is used?

For that matter, if clean is supported now perhaps this issue can go away.

[rainersigwald]

It's really hard to come up with a design for this that pleases everyone. There are many competing constraints:

• If I rename my program between builds, I don't want a stale copy of the old named program lying around.
• If I drop a reference, I don't want a stale copy of the reference in the output possibly gunking up my assembly loads.
• If I'm in the middle of an edit-build-debug cycle, I want things to be super fast.
• If I'm applying some custom state for the problem I'm debugging, I don't want every typo-fix build to blow that away.
• If I'm building to prepare for deployment (building an installer or NuGet package or Docker image or whatever), I want absolutely no files that aren't the result of the current source state.

Historically, this has been handled in MSBuild by a few things:

• Opt-in lists of "output files produced by the build" that can be persisted, diffed between builds, and used to clean up listed outputs of the previous build that weren't produced this time (IncrementalClean).
• Manually-placed files aren't cleaned up.
• "Official" builds are generally done as clean builds--in CI systems often by just deleting the whole enlistment + all intermediates and outputs and building.

This is . . . certainly not an elegant, principled design. But it does allow most of the scenarios to go forward.

[RichiCoder1]

Very interested in this. We have a build project that produces versioned script bundles (aurelia/webpack), and currently it'll keep placing every different bundle so we have n different versions of the bundle in publish output. Currently I just have a pre-publish target that nukes the publish output, but it'd be nice if this was built in.

[tjbarbour]

Would it be possible to add a --clean flag to accomplish this? Similar to what is mentioned here:

https://github.com/dotnet/cli/issues/2855#issuecomment-220387484

[gravidThoughts]

I have to second the idea by @tjbarbour .

I'm not understanding the complexity you are describing @rainersigwald . Keep the default behavior as it is, but allow someone to use the --clean flag with the understanding that it will delete everything in the output directory (of which there are several uses cases). I haven't looked at the code, but it is hard to imagine this wouldn't be straightforward.

@dsplaisted We use publish in our CI environment because we require a self-contained build. After publish, we build a nuget package. The expectation is I should have an easy way to clean the output directory completely. The flag would allow you to not break existing behavior. I see no drawbacks except the idea that adding flags to cli programs should not be done lightly .

[nguerrera]

I would consider this fixed by #3957.

It's not necessarily exactly what everyone asks for (they ask for different things), but I believe it strikes the right balance and follows what msbuild did for incremental clean of build dir.

Just some reasons not to wipe out the output directory:

1. You can have a workflow where you add extra files (say configuration, plugins) while debugging, then in your inner loop you have to keep recreating them.
2. A typo in the output directory can have disastrous consequences and lead to data loss. I once broke my whole machine this way.

[nguerrera]

cc @sfoslund

[sfoslund]

@dsplaisted @wli3 is there anything else to do here or was this fixed by #3957?

[dsplaisted]

@sfoslund we can close this