Conan: Enhancement: add `conan cleanup` command

[DmitrySokolov]

1. After a package was built the dir .conan/.../build (and possibly .conan/.../source) is unwanted. There should be a way to delete it.
2. Some binary tools and libs (Qt, for instance) write cache, config, etc. in their directories (.conan/.../package) during application build. There should be a way to delete all files that are not in the package manifest.
3. When using conan test_package a build dir and .pyc files are created in test_package dir. There should be a way to delete them.

[memsharded]

For 1. there is already conan remove "*" -s -b -f

For 2. that is a bit controversial. Packages shouldn't change their packages directories, otherwise the manifest will not match, at least in the current approach. Maybe this new point of view could be good: allow that, but provide "cleanup".

1. rm -rf test_package/build?

Overall, 1 is not very well-known, 2 could be interesting, and 3, quite a small gain, but a gain.

I think it can be considered, proposing for 0.23.

[lasote]

To understand it, for 2), How would you know what files you need to clean up?

[DmitrySokolov]

conanmanifest.txt lists all files in a package. So, any file (inside a package dir) that is not in that list should be deleted.

[memsharded]

A note: packages that provide a template file, then change it later with some local information at package_info() method. The hash changes, but is not a new file. I guess that those files should be left as they are.

[memsharded]

I am going to move this one to next 0.24. We want to release a RC for 0.23 asap, as there are a few important changes that we want to move forward, and this one is not so critical.

[memsharded]

We have been going back and forth with https://github.com/conan-io/conan/pull/2034 for a while, without being able to agree on the UI. The underlying issue is that this "clean" command, as it was being proposed, is nothing but an alias to:

$ conan remove * -b -s -f && rm -rf test_package/build

So I think this doesn't make sense on its own, but maybe better to define some command-alias system like git has, which allows users to configure commands at their will.

[cleeland]

Just to be clear, the resolution in this situation is to execute

conan remove * -b -s -f to clean the local cache?

[macdew]

I'd like to add a "me too" to this issue... After a package has been created, there should be no more need for source/build folders? I think Conan should clean these up unless something analogous like --keep-source etc is used? Keeping source/build folders around uses up a sizable chunk of diskspace.

This is particularly noticeable in our case because we tend to run Conan with --build=outdated and --update. Whenever people just make a small tweak to a recipe for one platform, this will trigger lots of rebuilds on all other platforms (which are now updated if the packager forgets to refresh all platforms). It would be nice to get recipe revisioning out of experimental status.

[maddanio]

+1
these files are just about never needed, so they should only be kept explicitly
what is the rationale for keeping them around?

[memsharded]

what is the rationale for keeping them around?

The sources are needed if you are building any other variant/configuration of the package, and some sources take some time to download. Users requested to keep the source folder by default

The build folder can also be reused, for example if a build_id() is defined, because you can create different package binary configurations from the same build (e.g. a multi-build that creates debug and release artifacts, but you still want to package them separately)

Also, users wanted to keep the build folder there for inspection, analyzing if something went wrong in the package step for example.

[maddanio]

but wont these multi builds happen at the same time? afterwards it can go, no? I guess debugging can be useful, but should it be default? it very very quickly fills up space...
also I dont hink i will ever remember conan remove * -b -s -f, but conan cleanup, yes :)

[memsharded]

but wont these multi builds happen at the same time? afterwards it can go, no? I guess debugging can be useful, but should it be default? it very very quickly fills up space...

No, you don't know in advance. Multiple conan create can create different packages, without building them again because both the source and the build folders haven't been removed. Conan cannot know it, it depends on the internals of the build. So this needs to be an explicit user action.

[boussaffawalid]

is there an easy way to delete all old packages but only keep latest version.
For example, I have a package A with these versions in the local cache: 1.0 1.1 1.2 ......1.9 2.0
I want to cleanup all old versions and only keep 2.0

[w3sip]

I would say that even the "delete latest version" approach isn't perfect. But, if the usage of each package in cache is tracked, can delete the "least recently used" ones. I would love to see a command that trims the size of cache to a certain limit, while deleting packages in LRU order.

[tkhyn]

Hi, as #2050 was closed, and as I agree with @maddanio that conan remove * -b -s -f is hard to remember (I have had to google my way here a few times already!), would it be possible to add the conan clean alias built-in?