Pants: `genrule` support via isolated processes

[stuhood]

Most of the pieces are now in place to implement something like genrule using @rules, and there are consumers for whom we would immediately be able to get our foot in the door to clean up legacy orchestration code.

A simple initial strategy for this would be to have the rule produce the equivalent of the resources or files targets, but there are likely other products/consumers that would be useful as well.

We should also likely have support for treating a genrule as a test... ie, for the exit code of the rule to be used as pass/fail for ./pants test $target.

[mateor]

It would be nice to have artifacts that are going to dist be a Product. A new task could consume that Product, resolve the files from .pants.d and output them to the distdir.

Tasks could then add arbitrary file paths to that Product and be totally isolated from things like pants_distdir.

[captify-akraievoy]

Good time of the day, @stuhood, interested in general state of this - do we want it? should we want it? Are there some pants analogs/substitutes for genrule as of 2019? Has it been postponed in favour of migrating pants to python 3?

Please see:

• #457
• other analogous stacks supporting this: bazel buck please

[stuhood]

My feeling currently is that having a "bash API" (genrule) as well as a rule API is a thing that we would like to avoid if we can by making the rule API as simple as possible., and so (when we've had time) we've been working on making our extension API significantly better.

Our hope is that pants' new rule API is sufficiently easy to use that folks don't feel the need to write bash scripts to extend pants... if after having shipped that API that does not turn out to be the case, we'll revisit how to do shell-based extension in the context of the new API. See the rendered version of #7600 for the latest info on the new API, but keep in mind that it is still multiple months away from being what we suggest for end users.

In the meantime, we'd suggest that folks use v1 Tasks, documented here: https://www.pantsbuild.org/howto_plugin.html and here https://www.pantsbuild.org/howto_plugin.html

[stuhood]

At this point, I believe that my comment from last year is now stale. We don't need to use "lack of a genrule-alike" as a forcing function to improve the plugin API, because the plugin API is in pretty great shape (although it will likely not be stable until support for one more language is implemented). So I think that this ticket is something we should consider doing soon.