Graphql-engine: [RFC] Switch from `stack` to `cabal-install` for building Haskell code

[lexi-lambda]

What

I propose we switch away from using stack as our Haskell build tool to cabal-install (aka the cabal command-line executable).

Why

Why bother switching build tools? Generally, the reasons fall into two categories: reasons stack is insufficient and reasons cabal-install has gotten better.

Problems with stack

• First and foremost, the number one reason to switch away from stack is the way it handles profiling. I do not know how to properly profile Haskell code using stack, as it forcibly compiles all dependencies with -fprof-auto (commercialhaskell/stack#2853), which is simply untenable. To me, this is a dealbreaker: when I have needed to profile graphql-engine, I have needed to rebuild it with cabal-install anyway.

• My experience with stack 2.x has been poor. It uses a new caching system called pantry, which is similar in spirit to cabal-install’s Nix-style build caching. However, pantry’s caching model is worse—it does not properly account for different sets of build flags in dependencies (which is related to the above point on why -fprof-auto cannot be disabled)—and it relies on a global SQLite database that provides zero options from recovering from an invalid caching state.

• Also related to the above, there is no way to build library dependencies with different optimization levels, so it is not possible to reliably link against libraries compiled with -O2.

• Getting access to new versions of GHC and libraries with stack is slow. The use of Stackage LTSes means reliable builds, but it also means you’re locked into those packages unless you want to do a lot of constraint solving yourself. stack has also recently removed the stack solver escape hatch, leaving it with no constraint solver capabilities whatsoever.

• Various other minor infelicities, some of which I wrote about here.

Improvements in cabal-install

cabal-install’s support for Nix-style local builds (aka cabal new-build) combined with support for per-project configuration (in the form of cabal.project files) is a total game-changer. cabal-install now supports essentially all the flexibility provided by stack.yaml files and much more:

• cabal.project files support multi-project builds, as well as using packages directly from git repositories, just like stack.yaml files do.

• cabal.project files allow fine-grained control over the build options for every dependency in your project, and it actually caches them properly. If you modify the optimization or profiling settings for a dependency, cabal new-build will rebuild the minimal set of things necessary, caching the build results globally.

• cabal new-freeze allows pinning the results of build plan construction, just like package lockfiles in other ecosystems like npm, ensuring reliable builds without needing to rely on a Stackage snapshot.

• Improvements in infrastructure like Stackage, Hackage CI, and HEAD.hackage have made cabal-install build plans more consistent than ever. Error reporting for plan construction failure is still not ideal, but it is readable, and unlike stack, cabal-install allows manually weakening individual constraints of individual dependencies to allow fine-grained control over the constraint solver, if it really comes to that.

It is hard to overstate how pleasant I have found using cabal new-build to build Haskell projects recently compared to stack. One of the coolest features of cabal new-build is that it’s entirely configuration file driven, so you can create a cabal.project.local file that is .gitignored, and you can use that to control what cabal new-build will do on your local machine: you can change optimization levels, modify profiling options, tweak settings for building documentation, and more, all without needing to muck with any command-line flags. Furthermore, the available options are fairly well-documented, and I found getting started with cabal new-build to be pretty easy.

How

Even if you’re sold by my sales pitch for cabal-install, perhaps you’re worried that switching away from stack will require a ton of work and will screw up your workflow. Everyone’s workflow is a little different, so I can’t say for certain what your experience will be, but mine has been complete ease. I have already been regularly building graphql-engine with cabal-install on my local machine, and it’s so easy to do that it really hasn’t even involved any significant overhead despite the fact that everything in the project is set up for stack.

Here’s what changes:

1. We swap out the stack.yaml file for a cabal.project file. This is the cabal.project file I’ve been using:

   packages: .
   
   package *
     optimization: 2
   
   package graphql-engine
     ghc-options: -j
   
   source-repository-package
     type: git
     location: https://github.com/hasura/pg-client-hs.git
     tag: de5c023ed7d2f75a77972ff52b6e5ed19d010ca2
   
   source-repository-package
     type: git
     location: https://github.com/hasura/graphql-parser-hs.git
     tag: f3d9b645efd9adb143e2ad4c6b73bded1578a4e9
   
   source-repository-package
     type: git
     location: https://github.com/hasura/ci-info-hs.git
     tag: ad6df731584dc89b72a6e131687d37ef01714fe8
   

   This basically just works.

2. Instead of running stack build to build graphql-engine, you run cabal new-build. Instead of passing command-line options to cabal new-build, create a cabal.project.local file that specifies any changes to how the project should be built on your machine. For example, my cabal.project.local file usually looks like this:

   package *
     documentation: true
   
   package graphql-engine
     optimization: 0
     documentation: false
   

   As the options imply, this enables building docs for all my dependencies and building graphql-engine with -O0. When I want to build with profiling enabled, I change my cabal.project.local file to this:

   profiling: true
   
   package *
     documentation: true
     profiling-detail: none
   
   package graphql-engine
     profiling-detail: toplevel-functions
   

   This automatically rebuilds everything with profiling, and profiling-detail: none means GHC will not automatically insert any cost centers in my dependencies, while profiling-detail: toplevel-functions means it will add cost centers for all top-level functions in graphql-engine. It’s easy to adjust these settings per-package as necessary while doing performance debugging.

3. The one major downside to using cabal-install is that it does not manage GHC versions automatically, so you have to install GHC yourself. Fortunately, there are two tools to do this easily and reliably: ghcup and stack.

   Yes, that’s right: I have been using stack to manage my installed versions of GHC even without using it to actually build anything. I just run stack --resolver=ghc-<some_ghc_version> exec -- bash, and I get dropped into a shell with ghc in my path. If you want a lighter-weight solution, ghcup is also available, but I’ve found this to work totally fine.

Everything else should basically still work the same way it currently does. CI scripts and contributing guides will have to change, but the changes are very small.

When

I do not want to force a different workflow on anyone, so if nobody has any drastic objections to this plan of action, I will prepare a branch with the necessary changes to swap out stack for cabal-install. Anyone who wants to verify their workflow is not impacted by the change should take that time to try building everything on the branch, and after some period of time, I’ll merge it in.

If anyone does have any major objections or concerns, please voice them! There are probably ways we can make things work.

[0x777]

This has been on my mind too.

I do not know how to properly profile Haskell code using stack, as it forcibly compiles all dependencies with -fprof-auto

Yeah, I agree, this is super annoying, I just figured out how to live with it using profiteur/profiterole etc. Nice to see that cabal handles this better.

The use of Stackage LTSes means reliable builds

If I'm not wrong, we can use stackage snapshots with cabal? So we won't lose out any reproducibility that stack offers with cabal.

The last time I tried cabal new-build, one thing annoyed me, which is this workflow:

When I want to build with profiling enabled, I change my cabal.project.local file to ...

Instead, if we could specify these customisations in separate files say cabal.project.local.profiling, cabal.project.local.fast etc, I would just do

cabal new-build --project-file-local cabal.project.local.fast

for fast builds with -O0 and

cabal new-build --project-file-local cabal.project.local.profile

to enable profiling. I think this can be done with --project-file but that would mean cloning the cabal.project file and not just overriding some options. Am I missing any of the features that would enable such a workflow?

[lexi-lambda]

Instead, if we could specify these customisations in separate files say cabal.project.local.profiling, cabal.project.local.fast etc

I agree that it would be nice if cabal-install provided a way to specify a “cascade” of project files, so you could combine any number of different project files into a single configuration. Without that, however, it’s always possible to have multiple cabal.project.X files and copy them to cabal.project.local to effectively switch between different configurations. This is what I do in CI for eff, for example.

Personally, I actually kind of like the cabal.project.local workflow; I just have a few different sets of options that I comment in or out as needed, and I like being able to add various flags in there like -ddump-simpl or -dcore-lint as desired. I‘m certainly open to other approaches as well, though.

[lexi-lambda]

If I'm not wrong, we can use stackage snapshots with cabal? So we won't lose out any reproducibility that stack offers with cabal.

I think we can do this, but to be entirely honest I don’t see the point. When I’ve built graphql-engine with cabal-install, the solver has just worked—it hasn’t had any difficulty constructing a build plan—and we can use cabal new-freeze to lock ourselves to a particular build plan until we explicitly update something. This is how most other package ecosystems work, and I think it’s a superior workflow.

One of the nice things about the existence of Stackage is that we still benefit from it even if we do not consume it directly. Stackage effectively provides a kind of ecosystem-wide CI that helps to ensure packages are kept up to date and version bounds are properly maintained. This is good, as it makes it more likely that cabal-install will be able to construct proper build plans with minimal fuss (as it helps to ensure version bound information is kept up to date).

[0x777]

Without that, however, it’s always possible to have multiple cabal.project.X files and copy them to cabal.project.local to effectively switch between different configurations. This is what I do in CI for eff, for example

:+1: I guess the workflow that I want can be easily handled by a shell script.

I agree, with cabal new-freeze we shouldn't have any reproducibility issues. I also like that with stackage, we don't have to constantly pay attention to package updates, bump up the stackage version, run tests and we are good to go. But I guess that workflow can also be achieved with cabal's --allow-newer and check if the build succeeds and tests pass?

[lexi-lambda]

👍 I guess the workflow that I want can be easily handled by a shell script.

Depending on the details, it might also be very easy to just add to cabal-install directly via PR! I get the sense they’re very open to improvements, it’s just mostly a project maintained by volunteers, so some things that people want don’t get implemented until someone takes the initiative to do it themselves.

I also like that with stackage, we don't have to constantly pay attention to package updates, bump up the stackage version, run tests and we are good to go. But I guess that workflow can also be achieved with cabal's --allow-newer and check if the build succeeds and tests pass?

I think that all we have to do to update dependencies is either update the freeze file directly (which is easy—it just outputs an extra cabal.project.freeze file with a big constraints section) or just delete the whole thing and re-run cabal new-freeze --upgrade-dependencies. That will trigger the construction of a fresh build plan, which will prefer to use the latest versions of packages whenever possible.

We might need to add some version bounds to some of our dependencies if we ever end up with a build plan that gives us a newer version of a library than we can actually build against, but that’s pretty easy and provides value, anyway. And if we want to upgrade absolutely everything to the newest possible version, then yes, we can always use --allow-newer='graphql-engine:*' to ask cabal-install to ignore all our version bounds and to try to construct a new build plan from that.

[0x777]

I'll open an issue with cabal-install folks to see if they are interested in this. I have nothing more to add, but I think folks who are using intero might be affected. I'll let them pitch in. cc @rakeshkky @hgiasac @ecthiender @nizar-m.

[jberryman]

I was considering modifying my dev.sh script to use cabal-install directly per above. Any thoughts on that? Would you accept a PR that made that change and checked in a cabal.project file and cabal.project.freeze file that just reproduced our stackage package versions?

I also feel confident we could (maybe even more easily) maintain graphql-engine under cabal install by just manually upgrading and freezing dependencies.

But I also want a little more clarity on what that would look like: do we expect developers to always be building/testing/benchmarking against the frozen dependencies (I think that's a good idea)? How often do we plan on bumping them, and what is that process like (hopefully not too often or for no reason since that simply cause a lot of rebuilding for folks, and the process should maybe involve some benchmarks... not sure)?

[jberryman]

...and also:

• how should we think about versioning GHC versions?
• related: should we be more lax about versions of base, cabal and the special libraries that ship with GHC?

[lexi-lambda]

Would you accept a PR that made that change and checked in a cabal.project file and cabal.project.freeze file that just reproduced our stackage package versions?

I’ve been meaning to open a PR like this for a while, but I haven’t quite gotten around to it. Personally, I’d like to make the switch all in one go—I’d rather not have some things (e.g. CI, CONTRIBUTING.md, dev.sh) using stack and others using cabal-install.

But I also want a little more clarity on what that would look like: do we expect developers to always be building/testing/benchmarking against the frozen dependencies (I think that's a good idea)?

Yes, my intent was to always build using the frozen dependencies.

How often do we plan on bumping them, and what is that process like (hopefully not too often or for no reason since that simply cause a lot of rebuilding for folks, and the process should maybe involve some benchmarks... not sure)?

I don’t think there needs to be too much ceremony around this if someone wants to bump a particular dependency simply because they want a newer feature or something like that. As for bumping them because we want to pull in bugfixes and things like that, I’m not sure, but to be honest I’m not super worried about that, either—the ecosystem is usually pretty stable. We haven’t been bumping our LTS very often and that seems to have been fine.

• how should we think about versioning GHC versions?
• related: should we be more lax about versions of base, cabal and the special libraries that ship with GHC?

I think we should probably all build with the same GHC version, and we should probably pin base to a particular version to enforce that.

[jberryman]

@lexi-lambda

I’ve been meaning to open a PR like this for a while, but I haven’t quite gotten around to it. Personally, I’d like to make the switch all in one go—I’d rather not have some things (e.g. CI, CONTRIBUTING.md, dev.sh) using stack and others using cabal-install.

Yeah that makes sense. How about an initial PR with just cabal.project and cabal.project.freeze with comments to the effect of "you can use this to preview a cabal-install workflow"? That would be easy for me to open today (and maybe I will anyway just so the work is out there, even if it gets closed).

I don’t think there needs to be too much ceremony around this if someone wants to bump a particular dependency simply because they want a newer feature or something like that

Okay yeah I think you're right. It shouldn't disrupt normal local development cycle anyway.

@0x777 sort of obvious I guess but it might work well and follow a unixy convention to have many cabal.project.local.<mode> files and just symlink them to a gitignored cabal.project.local to enable them. But I agree it seems an obvious and good feature to be able to select from the command line or env var

[lexi-lambda]

Yeah that makes sense. How about an initial PR with just cabal.project and cabal.project.freeze with comments to the effect of "you can use this to preview a cabal-install workflow"? That would be easy for me to open today (and maybe I will anyway just so the work is out there, even if it gets closed).

Yeah, I think that makes sense. It could even serve as a collaborative branch if others want to help move it along so it doesn’t have to be entirely one person. I’d definitely be willing to pitch in a little time, and I can share my current cabal.project with you if you’d like (though I imagine you’ve already put one together on your own, so maybe it doesn’t matter).

[jberryman]

Cool feel free to comment or contribute to that branch ^ I think I'm just using what you posted above.

I don't feel like I have a great idea yet of what a good development workflow looks like using cabal-install for this project. I seem to be triggering a lot of rebuilds of dependencies as I tweak the local file, and sometimes graphql-engine isn't rebuilt when I change certain things (like profiling-detail) it seems. Quite possible I'm just doing something dumb.