Bazel: improve --experimental_local_disk_cache

[buchgr]

Many users like the --experimental_local_disk_cache option, but it needs some improvements.

• Replace the two options --experimental_local_disk_cache and --experimental_local_disk_cache_path with one --local_disk_cache=/path/to/disk/cache.
• Add a --local_disk_cache_size=SIZE option to specify an upper bound on the folder. A LRU cache is probably fine. We'll probably need to introduce a lockfile to make sure only one Bazel server uses the cache at a time.
• Document it. Probably https://docs.bazel.build/versions/master/remote-caching.html is a good place.

cc: @davido

[davido]

Actuall there are three options. The one you havn't mentioned: --experimental_strict_action_env. Without this options, the local action cache is useless, if the workspace location changed (clone project in two different locations). See also discussion on the #3042, where local action cache was introduced.

I agree with your suggestions. I can look into renaming the options and I agree that to lose "experimental" part is a good idea. I also started to look into adding upper bound size limit to local action cache. Surprisingly, Bazel doesn't have any means for now to do that (no LRU cache or similar).

[ittaiz]

Why lock it?
On Mon, 19 Mar 2018 at 13:04 David Ostrovsky [e-mail]
wrote:

Actuall there are three options. The one you havn't mentioned:
--experimental_strict_action_env. Without this options, the caches is
useless, if the workspace changed (clone project in two different
locations). See also discussion on the original issue where local action
cache was introduced.

I agree with your suggestions. I can look into renaming the options and I
agree that we lose experimental part is a good idea. I also started to look
into adding upper bound size limit to local action cache. Surprisingly,
Bazel doesn't have any means for now to do that (no LRU cache or similar).

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/bazelbuild/bazel/issues/4870#issuecomment-374175356,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABUIF4Sgargwlf86MAnobdRBr5OxKHqoks5tf5CkgaJpZM4Sv7KB
.

[buchgr]

@davido good point, but --experimental_strict_action_env is not specific to the local disk cache. We should certainly document it.

Yes, Bazel currently doesn't have that, but the remote module could just have it's own LRU cache that it keeps in memory while the server is running. That should be sufficient I think.

@ittaiz Not a strict requirement, I just imagine it to be difficult to keep the LRU cache in sync with multiple Bazel's deleting from it concurrently.

[buchgr]

@davido would you want to work on this? 😃

[davido]

@buchgr Yes, feel free to assign it to me.

[promiseofcake]

Should we also have it respect the no-cache options as the remote-cache has? exclude-specific-targets-from-using-the-remote-cache?

[RNabel]

Is there a timeline for this feature?

[buchgr]

@promiseofcake yes we should.

@RNabel we love contributions :-)

[RNabel]

@davido There are a few of your commits referenced here -- are you still working on this?

[davido]

@RNabel There is a pending change for review: https://bazel-review.googlesource.com/c/bazel/+/53810.

However, all this change does, is to consolidate already working local disk cache options. Gerrit Code Review projects is using this feature for many months now. See the documentation how to activate it (you can pick any paths):

https://github.com/GerritCodeReview/gerrit/blob/master/Documentation/dev-bazel.txt#L366-L389

Moreover, in very recent Bazel version, I added the ability to specify $HOME user directory for local action cache. So that when this CL is included in bazel release, we would be able to commit this configuration in the repository and thus activate local action cache per default (instead of providing documentation in our build tool chain how to activate the local action cache, that nobody reading anyway):

build --experimental_local_disk_cache_path=~/.gerritcodereview/bazel-cache/cas

[buchgr]

@davido's changes have just been merged ... yay! https://github.com/bazelbuild/bazel/commit/4ee7f114387187c8b578b7f41674b9e68593d20e

I ll open another issue for discussing garbage collection.

[buchgr]

5139