Cabal: New `build-tool-depends` goals to complement `build-tools`

[hvr]

When you write build-tools: alex, Cabal checks that an executable of this name is installed. It is jointly the responsibility of the package manager, e.g., cabal-install, to arrange for an executable named alex to be installed.

In a recent commit (c0a4860202393882d2fd0f4de253c3af3a092fe2) we added a special case in cabal-install to handle "well known" build-tools like alex and happy, building and installing them automatically when they are requested in build-tools. However, there remains a more general problem of letting users ask for an executable, which is known to live in a particular package. build-tools as currently specified cannot really handle this, because it is specified to only specify executable names (not package names.) So it seems we really need a new field to give this information. Here is the proposal:

• We introduce a new tool-depends field, which consists of pkg:exename version-constraints specifications, e.g., tool-depends: gtk2hs-buildtools:gtk2hs2hs < 2.0. This means that we must build the gtk2hsc2hs executable from gtk2hs-buildtools before building this package. In principle cabal-install need only build the gtk2hsc2hs executable, although it's also permissible to just build all the executables in the package (if, e.g., the Setup script doesn't support per-component Cabal <https://github.com/ezyang/ghc-proposals/blob/master/proposals/0000-componentized-cabal.rst>_). If qualification is omitted it is assumed that you need ALL executables from the package.
• build-tools refers exclusively to non-packaged executables, e.g. provided by the system. However, for backwards compatibility, we introduce an internal mapping hard-coded into cabal-install which maps some build-tools fields to package names. The intended interpretation is that build-tools: happy > 0.4 actually elaborates to tool-depends: happy:happy > 0.4. This mapping will be configurable using the build-tool-packages field in a cabal-project and also modifiable by command line argument. (Names can be bikeshedded of course.)

Examples:

• tool-depends: mytoolpkg: cabal-install will install all executables defined by a package named mytoolpkg, and ensure they are available prior to building this package.
• tool-depends: mytoolpkg:mytool: cabal-install will install the executable mytool from mytoolpkg before building this package
• build-tools: happy: this is equivalent to writing tool-depends: happy:happy.
• build-tools: someprog: as before this does not effect solver behavior; there just simply needs to be a someprog in the path.

---

_Note: this was originally conceived by @ezyang as part of #220 (which has been addressed); however, since this represents an extension of the .cabal format and hasn't yet been implemented it's been split out into its own issue._

[ezyang]

I updated the description slightly, and removed the user-configurability of the mapping; I don't think we're going to do that.

[phadej]

Related comment: ATM specifying build-tools with unknown tool: no warning is printed, but compilation fails. That should be fixed as well. (try e.g. servant-aeson-specs with tests enabled)

[Ericson2314]

So for my work towards (the reduced goals of) https://github.com/haskell/cabal/issues/4047, I think the next step is basically desugaring build-tools into tool-depend. I'm a little fuzzy on the semantics of build-tools, so if somebody could outline them, that would be much appreciated. The end result should be I implement all of tool-depend but the parsing of the field itself (and at that point, I'll be happy to do that too).

On the actual code side, one can now grep for LegacyExeDependency---my plan is most of that code should instead use a new ExeDependency for this, and the desguaring all happen in once place.

[ezyang]

OK. Do you need a roadmap or are you enough in the brainspace to figure it out?

The current semantics of build-tools is this:

1. An entry in build-tools denotes an executable name, which we require to be in the PATH when the package is being built.
2. The exception to (1) is when the name in build-tools refers to a locally defined executable; for a non per-component build (traditional ./Setup configure), then it doesn't have to be in the PATH because we're going to build it ourselves. (In the per-component build case, internal deps are all external so they need to be in the PATH anyway).
3. In general, it is not known who provides the executable in question. The exceptions are when the executable name is internal (it's defined by the package being built), or when the executable name corresponds to one of a few hardcoded names (alex, happy, etc) in which case cabal-install has hardcoded the particular package to solve for in this case.

[Ericson2314]

OK to close with https://github.com/haskell/cabal/issues/4217 and https://github.com/haskell/cabal/issues/4217 picking up the rest?

[ezyang]

Yep by me.