Conan: [bug] `conan remove` does not delete packages with broken build

[WPMGPRoSToTeMa]

Environment Details (include every applicable attribute)

• Operating System+version: Windows 10 / WSL1 Ubuntu 18.04.2
• Compiler+version:
• Conan version: 1.24.0
• Python version: 3.7.5 (Windows 10) / 3.6.9 (WSL1 Ubuntu 18.04.2)

Steps to reproduce (Include if Applicable)

1. Generate package with a broken build (e.g. specify boost/1.72.0 with C++20 and VS2019).
2. Execute conan remove "*" -s -b -f to clear cache (or conan remove "boost/*" -s -b -f).
3. Broken builds are still present in .conan folder.

[uilianries]

Hi @WPMGPRoSToTeMa !

Conan did exactly what you asked. Let's analyze your command:

conan remove "boost/*" -s -b -f

-s tells to remove the source folder, in this case, %CONAN_USER_HOME%/.conan/data/boost/<any>/_/_/source

-b is for removing the build folder. As you didn't set any specific ID, it will remove all build folder. %CONAN_USER_HOME%/.conan/data/boost/<any>/_/_/build

-f Force, don't ask anything, just do it.

The package folder is there because you didn't ask, but you can add -p to remove all package folders. Or, you could remove all package data by running:

conan remove -f "boost/*"

That command removes all folders related to boost, including %CONAN_USER_HOME%/.conan/data/boost

If you want to see more details, when running that command, you set the logging level to 10:

set CONAN_LOGGING_LEVEL=10

So you will see something similar to:

 conan remove "openssl/*" -s -b -f
DEBUG :conan_api.py   [171]: INIT: Using config '.conan\conan.conf' [2020-04-21 10:07:59,916]
DEBUG :tracer.py      [157]: CONAN_API: remove(pattern=openssl/*,query=None,packages=None,builds=[],src=True,force=True,remote_name=None,outdated=False) [2020-04-21 10:07:59,918]
DEBUG :remover.py     [16]: REMOVE: folder .conan\data\openssl\1.1.1d\_\_\source [2020-04-21 10:07:59,925]
DEBUG :remover.py     [16]: REMOVE: folder .conan\data\openssl\1.1.1d\_\_\scm_source [2020-04-21 10:08:00,259]
DEBUG :remover.py     [16]: REMOVE: folder .conan\data\openssl\1.1.1d\_\_\build\3fb49604f9c2f729b85ba3115852006824e72cab [2020-04-21 10:08:00,260]
DEBUG :remover.py     [16]: REMOVE: folder .conan\data\openssl\1.1.1d\_\_\build [2020-04-21 10:08:00,672]

Reference: https://docs.conan.io/en/latest/reference/commands/misc/remove.html

[WPMGPRoSToTeMa]

@uilianries thank you! This helped to clean them. Now I want to delete only broken packages, leaving successful package untouched. I'm not sure I'm using the correct terminology, basically I want to clean all the garbage produced by the broken builds of boost package (that were produced by the incorrect boost options in conanfile.txt). Or maybe there is a way to just find IDs of broken packages at least? (so then I can pass them to remove command)

Also, is there a way to override the cache folder for specific conan install so I won't pollute my "global" cache?

[memsharded]

Hi @WPMGPRoSToTeMa

Just to make sure, "broken" packages are those that the build has failed for some reason, right? We use "broken" or "dirty" for things that were left in a dirty state after some failure: network connection while downloading, or an Exception in the build. If the build() method launched an exception, the package folder is marked as dirty, and that is automatically used for later removal, so a broken package is not actually used by consumers or uploaded to your server.

If after a failed build you want to force remove the package you have the full package id in the output, you can copy it from the output and run conan remove boost/1.72.0:package_id -f and that should remove that specific package. Is this what you are looking for?

Note that otherwise, a specific command to remove those dirty packages is not that necessary, as they will not be used anyway, they occupy a bit of space, but will eventually be wiped out when the cache is cleaned or that package is removed, and the correct binaries are always on the server, the cache should be mostly that, a cache, and can be cleaned regularly (actually in CI, a separate temporary folder is recommended for each separate job, the cache is not concurrent).

Also, is there a way to override the cache folder for specific conan install so I won't pollute my "global" cache?

Yes, you can define the CONAN_USER_HOME environment variable, and that will define a completely new cache for you for that environment. Take into account that the cache contains all configuration too: remotes, settings.yml, profiles, hooks... If you have some custom configuration, then you might be interested in the conan config install command, that will automatically set up this in your current cache.

[WPMGPRoSToTeMa]

@memsharded thank you for clarification.