Bazel: Make the volatile (and therefore non-rebuilding) build status values accessible in a dict in ctx

[cgruber]

Description of the problem / feature request:

Currently, in order to get various things like build stamps or git rev-parse output into things, one must shell out to a shell-script which accesses and unpacks volatile-status.txt, depending on it as an input. While this does allow you to (say) generate a BuildStatus.java or build_status.h or build_status.xml file, it forces you out of starlark to do it, etiher into bash or custom python, which is pretty gross.

Feature requests: what underlying problem are you trying to solve with this feature?

Supply not only ctx.version_file but a ctx.volatile_status dict which is the parsed dictionary of that file. This can then be accessed in a rule that looks more like this:

_template = """<?xml version=\"1.0\" encoding=\"UTF-8\"?>
    <resources>
        <string name="git_sha">{git_sha}</string>
    </resources>
"""
def _impl(ctx, **kwargs):
    output = ctx.actions.declare_file("res/values/git_sha.xml")
    ctx.actions.write(output, _template.format(
        git_sha = _git_sha(ctx)
    ))
    return [DefaultInfo(files = depset([output])), OutputGroupInfo(all_files = depset([output]))]

git_sha_resource_xml = rule(implementation = _impl, output_to_genfiles = True)

While this example of usage isn't slam-dunk win over a shell script, you have to imagine it with four or five values, which end up being gross to implement in a script, but really clean in starlark.

Given that the file is generated in an orderly way, and the usage here is limited to a rule's ctx, it seems pretty hard to abuse, especially if it only supports the volatile statuses, which don't need to be tracked for changing (so depending on the string values of the contents won't require a rebuild anyway). Unsure of the underpinnings, but if stable_status is necessary too, perhaps just accessing that dict can create an "inputs" based dependency implicitly. Regardless, keeping it to just volatiles should avoid any issues with hermetic builds.

Bugs: what's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

N/A

What operating system are you running Bazel on?

mac os X 10.14

What's the output of bazel info release?

release 0.19.0

Have you found anything relevant by searching the web?

Lots of discussions which result in "use it in a genrule"

[benjaminp]

I don't see how you can implement this without defeating analysis caching. If this proposal was implemented, every stamped build would have to reanalyze the world just to see if changed volatile status affected the output of analysis.

It would also require executing the workspace status command before doing any analysis. Even though workspace status commands tend to be fast, it's not desirable to introduce more nonparallel components into the overall Bazel critical path.

[ittaiz]

I might be off here but you can already use variables from the status in
variables which support make substitutions, right? Why isn’t that a
performance problem
On Wed, 28 Nov 2018 at 9:44 Benjamin Peterson [e-mail]
wrote:

I don't see how you can implement this without defeating analysis caching.
If this proposal was implemented, every stamped build would have to
reanalyze the world just to see if changed volatile status affected the
output of analysis.

It would also require executing the workspace status command before doing
any analysis. Even though workspace status commands tend to be fast, it's
not desirable to introduce more nonparallel components into the overall
Bazel critical path.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/bazelbuild/bazel/issues/6786#issuecomment-442350395,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABUIF12o-VFmC93tYwcg4huU9ZpXyD8Dks5uzj7agaJpZM4Y2r_C
.

[ulfjack]

We could allow implementing actions in Starlark, but they'd still be executed in the execution phase, not the analysis phase.

[Globegitter]

@cgruber I have written a small set of rules that should simplify the reading of stamp values: https://github.com/ecosia/rules_stamp. See example usage: https://github.com/ecosia/rules_stamp/blob/master/examples/BUILD.bazel. Let me know if that makes things simpler.

[cgruber]

I'll check it out!