Bazel: Cache management facilities

[cgils]

Please consider adding some way of setting a maximum cache size. At the moment our project's build caches grow unbounded leading to us discarding them periodically.

A facility with which to display cache information would be nice as well. Something like the output of ccache -s, for example:

  cache directory                     /home/user/.ccache
  cache hit (direct)                 34993
  cache hit (preprocessed)             153
  cache miss                         23479
  compile failed                         3
  ccache internal error                  1
  preprocessor error                     1
  cache file missing                     1
  unsupported source language          382
  no input file                       7322
  files in cache                     70445
  cache size                          16.4 Gbytes
  max cache size                      90.0 Gbytes

[dslomov]

@cgils, are you talking about project "cache" (it is not really a cache, it is just your output directory) or stuff that gets into ~/.cache/bazel?

[cgils]

Stuff that gets put into ~/.cache/bazel.

[glinscott]

I vote for this to be higher priority :). Bazel is commonly filling up hard drives on our project from the ~/.cache/bazel directory growing unbounded.

[kavehv]

+1

[jianga]

+1

[jiayiliu]

+1, esp. if the home dir is on NFS.

[lefromage]

+1

[burnzzz]

+1

[softprops]

+1

[llhe]

+1

[dahlstrom-g]

+1

[hlopko]

@buchgr

[buchgr]

@jin do you have an idea to which component to assign this issue?

[jin]

@buchgr team-Local-Exec sounds like a good fit.

[jmmv]

There are several kinds of data to be managed here:

• Bazel "installation files". This is https://github.com/bazelbuild/bazel/issues/2109 and they should be auto-purged. No question.

• Output trees. These are just the build artifacts of a given build. They are not a cache. Bazel discarding them automatically is a bad idea as we'd be throwing away people's data. bazel clean removes these on a workspace basis. (I think these files living under ~/.cache/ is also a mistake.)

• Fetch/remote cache (whatever it's called). bazel fetch can download a lot of stuff and this stuff is shared across workspaces. This is a cache and could be automatically cleaned. However, pruning stuff from here affects the ability to work offline, so I'd be wary of doing that too. It'd be pretty annoying to fetch everything before hopping into a plane, only to discover later that some random stuff was pruned and you cannot do anything any longer.

Separately, having a command to dump all files Bazel knows about, grouped by category/project and with a summary of their size, would be awesome. Tag each entry with an ID so you can tell bazel clean some-id and it'd be even better.

[edrumwri]

+1

[jmmv]

Per #2765, one more thing to consider if implementing a smarter clean command is bazel clean //target. Noting here because this would possibly conflict with the bazel clean some-id proposal above.

[ozio85]

+1

[jonatanj]

+1