Bazel: "bazel build" supporting a query expression in place of target patterns

[laszlocsomor]

Problem

It's a common scenario for CI systems to query for some tests, then call "bazel test" on them. The query results are often longer than the command line length limit, particularly on Windows. Test logs are also very hard to read when the build/test command lists a gazillion targets.

Feature Request

"bazel build" and "bazel query" could support specifying a query expression, rather than a target pattern. Bazel would first evaluate the query, then build/test the results.

For simplicity's sake, we could introduce a new flag, --target_query=<query-expr>, and prohibit specifying target patterns if this flag is already present.

Justification

The query results are often longer than what fits on the command line, especially on Windows.
Passing the query instead of its results would:

• avoid having to run the bazel client twice
• avoid the need to use recursive test_suites like //src:all_windows_tests (whose sole purpose is to work around the command line length limit)
• avoid the need for Bazel supporting flagfiles (https://github.com/bazelbuild/bazel/issues/3750)

[jlisee]

I have another use case that might increase interest. In some complex system you might have plugins and external configuration files that express relationships not directly encoded in BUILD files. They can be modeled by a query expression, or a suitably complex list of them. Right now I have to execute multiple bazel build commands with varying --build_tag_filters options in order to achieve the same goal.

One modification to this proposal would be allow the user to supply multiple --target_query=<query-expr> and the union of all result targets would be built.

[gregestren]

Joseph, could you sketch a simple example of what you mean?

I get your point. But this seems like a request that needs careful design consideration: what relationships should be in BUILD files vs. query vs. somewhere else? Whatever the answer, I think it's better for us to have principled reasons behind said answer vs. an incomprehensible mishmash.

We've faced similar issues when organizing Bazel's ever-increasing customization support. We have BUILD files, select(), new configuration schemes like constraint_setting, platforms and toolchains, CROSSTOOL, .bzl files, and user-definable flags. I'd like to remain clear as we can about which relationships should be expressed where, and which relationships Bazel shouldn't try to maintain at all.

Marking as P3 not because I think this doesn't matter, but because I suspect it'll take some sustained design effort to materialize. So for now let's let this collect use cases and user priority stories. We can upgrade the priority when that leads to a proper commitment to see this through.

[Meai1]

I think the idea here is that it would be nice to have a consistent syntax for finding targets or anything else. Why should the syntax be different between the build step and the query step when finding a build target is just a tiny subset that could be easily supported by the query syntax.

[mattofak]

A use case I just found was to build all the reverse dependencies of a package to see if I broke anything. A work around in that case is to (in bash)

bazel build `bazel query ...`

[gregestren]

I wonder if cquery could support this more naturally than pure query. The difference being that cquery accepts all the usual build command line flags, and its implementation essentially runs Bazel's analysis phase then quits before execution.

[bricerebsamen]

Because we needed something like that, one of my colleagues wrote a python script to go around this missing feature. In a nutshell it gets the result of the query and pass it to bazel build: subprocess(["bazel", "build"] + query_results(query)). So far we never run into issues with limitations on the number of arguments (not sure if that applies to subprocess).
It'd be great if it would be supported natively by bazel as suggest by the OP.