Haskell-ide-engine: Explain how to install with cabal in README

[ivanperez-keera]

The README favours installation methods other than cabal. For reasons that at too long to discuss here, some of us do not like using stack.

While the readme states that cabal can be used to install haskell-ide-engine, this is only briefly mentioned, and no explicit instructions are given.

As it turns out, this package compiles just fine with the good-old cabal sandboxes, which are a perfectly good way of keeping things isolated.

Either with v2 or with v1, it would be great if the README contained specific instructions of how to install this with cabal.

[mpickering]

@ivanperez-keera I think the main reason for the complexity is that the installation script builds 3 different versions of hie, one for each of the last 3 GHC releases.

There is also the problem of the large number of submodules which are because certain dependencies have
not made it onto hackage in months/years. We are in the process of trying to decouple from these dependencies so hopefully some progress in this regard by the end of the summer.

But you are right that cabal new-build works fine to build one of the executables and this is how I do local development.

[Anrock]

@mpickering could you make a PR and document howto use cabal new-build in README?

[samuelpilz]

I can write some documentation about this. However, this is not a good idea. Before we had install.hs we had a new issue about some error that was due to an incorrect installation every day. Officially supporting a different installation method will bring these back.

[fendor]

The install.hs almost supports build with only cabal.
Especially with the incoming build simplifications.
To make it possible, you need to compile install.hs somehow. Ghc needs to know about the packages shake, directory, and soon extra.
The location to install hie to is defined by stack --local-bin.
you may replace, for yourself, stackLocalBin with the location that you want hie to be in.
then it should be possible to use ./install cabal-build to install only with cabal.

[mpickering]

I'm just talking about the fact you can run cabal new-build to build the project if you just want one executable. I'm not say the installation script should be used to use cabal.

[samuelpilz]

as far as I know it the steps to build hie (directly after a fresh clone) are the following:

1. git submodule sync && git submodule update --init
2. somehow get hoogle (>=5.0.17 to be safe) into the path and call hoogle generate
3. build hie with the build-system of your choice
4. install hie and the hie-wrapper into the path
5. repeat steps 3 and 4 of all required ghc-versions, renaming the hie binaries to hie-X depending on the ghc-version they were compiled with

Historically, a few more steps are included, (of which I am unsure if really needed)

1. call cabal install Cabal-X
2. install happy before calling <stack|cabal> install
3. call stack build --copy-compiler-tool

Still, there are some gotchas:

• On MacOs, this also does not work out-of-the-box (random icu issues)
• On Windows, cabal new-install does not work (random symlink issues, iirc)

Currently, the environment is a bit unstable

• hope that cabal-helper-wrapper can be found and that it can find the compiler it was compiled with; or wait for the new hie-bios
• hope that hlint's data-files can be found; or wait for fixes for this issue
• hope for a few more things... (hoogle-db, ghc-mod project-detection, ...)

As you can see, this is a non-trivial process!. We had a Makefile for that. But for Windows, we also had some *.ps file. These were incorrect and out-of-sync regularly and an effort has been made to replace these 2 with the install.hs, which builds in the library shake.

Pros:

• installs hie fairly reliably
• This is cross-platform
• is written in Haskell. :heart:
• is regularly tested for correctness (currently by me, and later by CI hopefully)
• users cannot forget steps
• users are informed of some known issues
• supports building and installing hie itself using cabal new-build

Cons:

• install.hs itself is currently not executable via cabal (from what I know)
• not suitable for local development

Hopefully, the build-system will get simpler in the next few months, after following projects have been finished:

• hie-bios replaces cabal-helper
• hlints data-files are dealt with in a reliable way
• cabal new-install works in windows in the future
• hie self-test on startup or during installation

adding "building with cabal new-install works just fine" to the readme is not correct

[ivanperez-keera]

Perhaps this question is naive, but shouldn't the necessary extra compilation steps (calling hoogle, using shake, etc.) be part of the Setup.hs file? Isn't this precisely what those are for?

[samuelpilz]

Perhaps, but I have no experience with that. Is it possible to initiate building hie with different compilers from there?

[ivanperez-keera]

No. The way to do something like that would be to change your PATH and re-run, or tell cabal which GHC to use (both of which seems reasonable to me).

[samuelpilz]

another non-trivial issue was the requirement that git submodule update --init has to be called before stack reads the stack.yaml for building hie, which contains references to the submodules.
The same issue is present with cabal: Directly at the start, the cabal.project is read which references submodules that are not initialized yet.

Also, can part of the code from the Setup.hs be run after the building? This is required for renaming the hie binaries in a way hie-wrapper expects them.

We had poor experiences (in my opinion) with requiring the users to perform extra manual steps.

[samuelpilz]

I firmly believe that a process like the install.hs is the correct way to install hie from source and setting up the correct environment. I will write all the unexpected requirements to building this project in this project's documentation in the next few days. I have already written something (https://github.com/haskell/haskell-ide-engine/blob/master/docs/Build.md), but this is neither good, nor up-to-date, nor complete.

However, I am open to suggestions on how something lie the install.hs can be executed from cabal. Moreover, I am also open to suggestions to a different build-system that satisfies all requirements (once documented).

[ivanperez-keera]

This repo uses submodules. The recommended way of cloning already does the update --init for the submodules, does it not?

I cannot find anything in install.hs renaming those files. How does hie-wrapper expect them to be named?

[ivanperez-keera]

Some things that the install.hs does, I would have expected to find in the cabal file, like dependencies on other haskell tools (e.g. happy, hoogle?).

I suspect, but I don't know for sure, that most "normal" users only have one version of ghc available, for which renaming the executables as executable-8.6.4 etc. and using symlinks should not be necessary.

Does the need to compile hie for different GHC versions arise from a need to have documentation for all those GHC versions only, or is hie actually linked in such a way that a version compiled with GHC 8.6 makes it useless if I switch to GHC 8.4 for a project?

I suspect, but I don't know, that the scripts could make more sense for people developing haskell-ide-engine, but not for people installing it just to use it. I also suspect that some files should be executed after installation, in a way similar to how you need to cabal update before using cabal install.

(EDIT: removed statement that I think is wrong.)

[samuelpilz]

I cannot find anything in install.hs renaming those files. How does hie-wrapper expect them to be named?

  copyFile' (localBin </> "hie" <.> exe)
            (localBin </> "hie-" ++ versionNumber <.> exe)
  copyFile' (localBin </> "hie" <.> exe)
            (localBin </> "hie-" ++ dropExtension versionNumber <.> exe)

My understanding is that, hie-8.6.4 does not work with ghc-8.4 projects due to compiler-internal breaking changes.

git clone ... --recursive works only for a fresh clone. After each pull, git submodule update is still required.
My opinion: relying on users to have to repository in a consistent state is not good.

The decision to introduce a one-stop-shop build and add the dependency to stack into the officially supported way of installing hie has been made several months ago now.
My opinion: having a consistent one-stop-shop build greatly improved the approachability of hie for new users, despite the fact that stack is now required

[ivanperez-keera]

My opinion: relying on users to have to repository in a consistent state is not good.

Unless users are going to regularly update and reinstall, this should not matter.

My opinion: having a consistent one-stop-shop build greatly improved the approachability of hie for new users, despite the fact that stack is now required.

I understand your position. But it also draws users away, and it is not necessary (since it installs without).

[samuelpilz]

We disagree indeed.

Unless users are going to regularly update and reinstall, this should not matter.

since hie is quite unstable and is in active development, this happens regularly.

... since it installs without [stack]

This is not true for most users (windows & MacOS) in most circumstances (need correct environment for hlint, hoogle, cabal-helper-wrapper, ...)

I AM in favor of a install-possibility without stack. However, this is not simple

[samuelpilz]

I vote to close this issue. Simply adding documentation to the readme is not a wise step.

I encourage the development of a build-system that is cabal-only that honors the requirements described in the documentation of the build system. However, I have not enough experience with cabal to do this.