Pants: add `build` goal to support compiled languages

[tdyas]

Pants v2 no longer has a compile goal. To support developing plugins for compiled languages, Pants should have a build goal that plugins can hook using a Union type.

[Eric-Arellano]

Agreed. Thoughts on naming itbuild vs compile?

I'm also wondering how we might model Rust's check command, if that would be included in this new goal or something else.

[tdyas]

Agreed. Thoughts on naming itbuild vs compile?

My vote is for build. I viewcompile as a term limited to compiled languages. build is a general enough term to encompass compilation of languages, transpiling of languages like Typescript, and maybe "building" non-language targets for which package does not make sense.

[tdyas]

I'm also wondering how we might model Rust's check command, if that would be included in this new goal or something else.

Some ideas:

• A check goal that is only applicable to Rust
• (needs better naming) A --check option for the build goal
• Language-specific check option, e.g. --rust-check

[asherf]

Why add another goal ? Why not use the existing "package" goal?

The fact that the internal semantics (compiling, building, transpiling, etc...) are different for every language seem irrelevant.
What matters is the result , I.e. an executable and deployable file.

[tdyas]

Why add another goal ? Why not use the existing "package" goal?

The fact that the internal semantics (compiling, building, transpiling, etc...) are different for every language seem irrelevant.
What matters is the result , I.e. an executable and deployable file.

package makes sense for a go_binary target that generates binaries; it does not make sense for a go_package target because that go_package target does not generate any output external to Pants whereas go_binary will. I would rather Pants use a name that does not imply that there will be a build artifact for a target that does not generate them.

Bazel uses build for that goal name. Maybe package should just be merged into build?

To further your point, why even have separate goals at all, e.g. the test goal? We should just run package on python_test targets to run the tests.

EDIT: (Which would be an absurd result in my view, which furthers my point.)

[Eric-Arellano]

We considered build instead of package and rejected it for this exact reason: we wanted to make clear that package is to produce some external artifacts, rather than compiling code.

[stuhood]

I'll add another vote for a check goal, which makes no guarantee to expose the intermediate compile outputs (because that's expensive in the context of remoting and remote caching) even if they exist.

Things that would run under the goal:

• the typecheck goal would be deprecated in favor of check, so mypy would run in check
• the equivalent of Rust cargo check (which is effectively just typechecking with no codegeneration)
• the lightest form of compilation available for go (essentially, a "debug"/unoptimized compile) and ditto eventually for the JVM

The catchall "build" works for Bazel because it exposes _all_ intermediate outputs, and so there is no point differentiating "I just wanted to check whether something _can_ build" from "please build it and produce artifacts". I think that a better dichotomy for pants is check vs package... where the former checks that your code would compile while producing as few artifacts as possible, while the latter produces artifacts.

And I don't think that a build --check mode makes sense, because in --check mode (for Rust and Python, for example) there are no artifacts produced: having toggling the flag change whether it outputs any artifacts is a fundamental difference, and should be behind a separate goal, imo.

[Eric-Arellano]

Would we ever want to expose the intermediate compile outputs to end users through a goal like compile? Or that isn't very meaningful to expose, and you'd only expose via package?

[stuhood]

Would we ever want to expose the intermediate compile outputs to end users through a goal like compile? Or that isn't very meaningful to expose, and you'd only expose via package?

I think that we would want to prefer to never dump out "all of the intermediate artifacts of a build", which is what you might expect from a "compile" goal, because that very rarely makes sense. An exception is for IDE integrations, but that is very specific, and I think that we should treat it that way.

So yea, I think lean in on package exposing only the output of *_binary targets, archives, etc.

[tdyas]

I think that we would want to prefer to never dump out "all of the intermediate artifacts of a build", which is what you might expect from a "compile" goal, because that very rarely makes sense. An exception is for IDE integrations, but that is very specific, and I think that we should treat it that way.

I believe the answer depends on the language. For C/C++, for example, I have on occasion needed to compile individual files to assembly form with gcc's -S option. Rhetorically speaking, how should that be accomplished in Pants? As a separate "c-asm" goal or as an option to the build goal? (No need to solve this, just using it as an existential example of when a developer would want an intermediate output, or at least a form of it.)

Stepping back, it is probably useful to distinguish between the "ordinary" usage of a build and advanced usages. We should have a mechanism to not preclude advanced usages (whatever they may be). But we should first survey the build semantics of various languages, and then see what falls out.