Cabal: What's the correct way to build Cabal and alex?

[snoyberg]

Sorry for the somewhat vague question, but I'm not sure what the expected behavior is. With Cabal-2.0.0.2 and alex-3.2.1, we have this situation:

• Cabal depends on alex as a build tool
• alex has a custom Cabal setup script

Naively, this is a cyclic dependency. I can see a few ways of getting this both to build:

1. Rely on the Cabal that ships with GHC to build alex, and then build Cabal with that alex. From a reproducibility standpoint, that makes me nervous: if I want to pin my Cabal library version, I'll still end up building one package with a different version of the Cabal library.
2. Require that alex be present before building Cabal, which seems suboptimal.
3. Some kind of weird hack with capturing the generate files from alex?

I may simply be missing out on something obvious here.

[phadej]

If you build Cabal-2.0.0.2 with -f-parsec, then you shouldn't need alex.

Yet, in next major Cabal alex will be required. I'm not sure, maybe easiest solution would be to preprocess alex files: opinions @hvr, @dcoutts?

[snoyberg]

Oh, I didn't realize that it was gated behind the parsec flag. That's perfect, thank you! Though I _am_ interested in how this will be addressed once parsec is required.

[ezyang]

The proposed solution in https://github.com/haskell/cabal/pull/4654 is to commit a copy of the alex-generated files to the repo, allowing us to remove build-tools: alex entirely from the Cabla file.

I am a bit concerned about the effects of checking in automatically generated files on the development process; the obvious issues are (1) depending on what version of alex is used, the output could wobble unnecessarily, and (2) it makes merge conflicts a bit more painful.

I was under the impression that Cabal had some affordance for including generated files in sdists, so it wouldn't be necessary to commit them into the repo. See for example https://github.com/haskell/cabal/issues/2940 where @simonmar makes reference to this feature. However, @hvr is not at all sure how this works at all (maybe Cabal will still be willing to use an installed alex even if it's not build-tool'ed).

[hvr]

Fwiw, I consider what we do in #4654 a temporary workaround (i.e. I don't like the idea to check in generated artifacts into Git either). And yes, cabal may sometimes pick up an Alex on $PATH, but I consider that to be a bug that I intend to fix before 2.2 gets released; the way it works now is not principled.

And as for, "Cabal had some affordance for including generated files in sdists"; I want to get rid of that too; there's no good reason to support this in Cabal proper anymore. The 2 very special corner cases I know are better handled by using a custom release generation process, rather than burden lib:Cabal by it.

[ezyang]

@hvr, but since this was advertised as a feature, and because packages rely on it, any such fix would be gated by cabal-version, right?

[hvr]

@ezyang where exactly is it advertised as a feature? (and yes, it'd be gated via cabal-version where it's essential to not render packages already on Hackage invalid; but that still needs to be evaluated).

[ezyang]

Looking at https://cabal.readthedocs.io/en/latest/developing-packages.html#modules-and-preprocessors it is not specified that it is necessary to mention these hardcoded preprocessors in build-tools.

[hvr]

Fair enough, and neither did custom-setup/setup-depends exist; this is just another hole in the "no untracked dependencies" design principle of CABAL we're going to close... :-)

and btw, I'm revising .cabal files on Hackage to retroactively add those missing build-tools lines, because those packages fail to build on matrix.hho; fortunately, most packages do specify build-tools, so there's little to be done.

[phadej]

@hvr, checking in isn't required, then one just need to update HOWTOs. After all, we already have Cabal/Makefile...

[hvr]

@phadej sure, but this has implications for the GHC buildsystem too; so long-term I'd rather fix the problem at the root, i.e. have alex/happy fixed to be properly installable, rather than having to workaround this by a hack in lib:Cabal. I want to reduce the technical debt, rather than grow it.

[snoyberg]

I'm all in favor of more reproudcible build plans, but I'm not sure I understand the proposal here. Is the idea that Cabal will only elect to generate code from Alex input files if Cabal itself built and installed the alex executable?

[phadej]

@snoyberg I think that @hvr's plan is to make alex build-type: Simple, which will make things simpler for all