Bazel: "bazel --batch" and "bazel shutdown" seem to invalidate cached action_listeners

[rtsai]

Description of the problem / feature request:

Issuing any of "bazel shutdown" or "bazel --batch" seem to cause action listeners to be unnecessarily re-executed even though compiles are not re-executed.

Feature requests: what underlying problem are you trying to solve with this feature?

N/A

Bugs: what's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

A sample workspace is attached. Our real use-case is running clang-tidy as an action_listener, which is much more costly.

listener_example.tar.gz

What operating system are you running Bazel on?

Ubuntu-16.04

What's the output of bazel info release?

release 0.11.1

If bazel info release returns "development version" or "(@non-git)", tell us how you built Bazel.

N/A

What's the output of git remote get-url origin ; git rev-parse master ; git rev-parse HEAD ?

N/A

Have you found anything relevant by searching the web?

No.

Any other information, logs, or outputs that you want to share?

Sample workspace attached.

listener_example.tar.gz

[buchgr]

@dslomov seems skylark related? can you triage this issue please?

[rtsai]

Friendly ping. Is any initial diagnosis available?

[dslomov]

@c-parsons could you take a look?

[rtsai]

Friendly ping. Is any initial diagnosis available?

[c-parsons]

Sorry for the delay on a response here, it fell off my radar.
buchgr@: I'm not sure how this is a skylark issue.

Shutting down a server (invoking shutdown or using --batch) means you lose incremental build-analysis information. (However, you don't lose all information about the action cache, which is materialized).

So, when you rerun the build, bazel regenerates the full action graph, but then acknowledges that some of the actions should hit cache (their inputs haven't changed, and their outputs are still present in your build output directory, so they shouldn't be rerun). It seems, however, that the action cache for extra actions is perhaps consistently invalid, which means that these actions are rerun.

So this is a matter of determining why the actions registered by extra_action are not appropriately cached on disk.

That said, action_listener is marked experimental , and we're currently investigating a deprecation plan for this rule, so I highly doubt we'll be able to prioritize this.

[getim]

I did not check for changes that affect this but from some tests it looks like the issue has been fixed since version 3.4.0

[getim]

I don't know what the deprecation plan for extra_actions is right now but I did quickly check some more cases and it looks like this is only fixed in the simplest of cases:

• When listening for CppCompile actions and building a simple cc_library with only 1 source file, no headers and no dependencies, the extra_action is not rerun from Bazel 3.4.0 onwards.
• Just adding a header-file to the cc_library is enough to break it again and cause re-execution of the extra_action after a "bazel shutdown".

In addition to that, I have noticed that a re-execution can be forced by just touching a BUILD-file, not even requiring a "bazel shutdown".