Haskell-ide-engine: Support for cabal new-build / Per-project configuration of hie

[cfraz89]

Hi,

I'm attempting to rig a reflex project set up in this manner. Unfortunately hie balks due to it attempting to run cabal configure, which naturally fails as this project requires new-configure/new-build.

I would like to work on adding support for new-build, however I had a thought. Currently hie automatically decides whether to build using stack or cabal based on the presence of a stack.yaml file. My first thought was that this concept could be extended to new-style cabal projects by detecting the presence of a cabal.project file. However this may not be sufficient, as I believe the new-* system can be used even without this file.

Additionally, I personally have had issues even with the stack file autodetection in the past, in times where I was flip-flopping between using nix with cabal, and stack in the same project, and having to delete/rename the stack file to please hie was a pain.

So my thought was, it would be nice to have the build system selected by hie configurable/overridable per instance/project. I can think of two ways of going about this:

1. A command line arg to be added: --build-system [stack stackfile? | cabal .cabalfile? | cabal-new .cabalfile?]
2. Adding support for a per-project hie configuration file in the project root. If a (.?)hie.conf is detected. It reads a key such as build-system=stack and will act accordingly

Personally I like the idea of number two, as it will come in useful when working on multiple projects, whereas hie launch args are typically set up statically by the ide, at least currently.

Naturally, in the absence of any specified configuration, hie would use its current autodetection behaviour.

What do you think? If either option sounds reasonable, I can draft up a pr to implement support.

[alanz]

@cfraz89 I had a discussion with @DanielG on #ghc-mod IRC recently, and we discussed providing option 1.

But it is no just a case of choosing the build method to use, we need to work that support into cabal-helper and ghc-mod too.

If you want to jump in, we will be happy to point you in the right direction.

[alanz]

For the record

<alanz> I think having it limited to cabal >= 2 will help
<alanz> I regard myself as new to c-h. But maybe not anymore
<dxld> imo it doesn't make a difference
<alanz> I made a start, and at first blush it did not look too hard
<dxld> I guess we could cut support in c-h too but the problem areas aren't there
<dxld> its the sequencing of how project setup is handled in ghc-mod
<dxld> it just needs a good 'ol refactor
<dxld> thinking about it now I think that whole buissness is actually one of the more approachable areas in ghc-mod..
<dxld> *shrug*
<alanz> well, as I understand it, we need to notice that it is new-build, then be able to ask for the info we need from cabal (Same as before)
<alanz> and call "new-configure" somewhere along the way
<alanz> Id did bits of it, but can't find the branch on gh
<dxld> indeed
<alanz> just pushed it, branch new-build
<dxld> i don't see it
* willj has quit (Quit: http://www.kiwiirc.com/ - A hand crafted IRC client)
<dxld> never mind, got it
<alanz> it was just a start, ...
<alanz> And this is basically the meat of it, https://github.com/alanz/ghc-mod/commit/9cc749834aa6bacf57117f182cd282f2c5eaa556
<dxld> I meant I couldn't find the branch ;)
<alanz> I know, but I had literally just pushed it, so it was probably not showing up at first
<dxld> I think we should be keying off cabal.project, not dist-newstyle for now
<alanz> I think there was something in c-h too, but I may be wring
<alanz> wrong
<alanz> cradle detection needs to prioritise "dist-newstyle", then "dist", then ".stack-work"
<dxld> once new-build turns into regular build we'll have fun telling the two dist/ dirs apart :)
<dxld> we don't use .stack-work either
<dxld> just stack.yml
<alanz> Under some circumstances new-build creates a dist dir
<dxld> I know
<dxld> which makes it even more fun
<alanz> ok, basically stack last
<alanz> hence my proposed order
<dxld> I think what we ought to do is redesign the api around project finding
<alanz> unless overriden with a CLI arg
<dxld> out with this "priority" stuff and let the IDE/user decide what kind of project it is instead
<alanz> agree
<dxld> is there some way to handle that descision through lsp?
<alanz> I find myself sitting with cabal.project, and say 4 different stack files
<dxld> or will this just have to be some protocol extension magic
<alanz> I think it should go to the client
* willj (uid297006@gateway/web/irccloud.com/x-rmigfsgftfjnssgd) has joined
<dxld> yeah but how do we expose this setting to the client
<alanz> e.g. https://github.com/alanz/HaRe/tree/ghc-8.4
<dxld> (remember I know approximately nothing about lsp
<dxld> )
<alanz> Will have to give it some thought
<dxld> :D I don't think I've seen that logo yet, cute
<alanz> I think the fundamental thing is to make it possible to pass it in as an Option field
<alanz> An up and coming artist did it for me, one of my kids friends.
<dxld> if at all possible I'd like the ghc-mod-core api to be not so tethered to what ghc-mod used to do
<alanz> agree
<dxld> since this is a pretty fundamental part of the api I think it deserves more than just cramming it in Options
<alanz> personally I think it should strip down to providing the constext only, pretty much
<alanz> well, effectively you need to be able to say "switch to operating as new-build|stack.yaml|stack-8.2.2.yaml|etc"
<dxld> see the way I see it the flow ought to be: we look for possible project kinds, return a list to the lsp frotend, it dispalys a promt to the user (or hides it in settings somewhere) and then it tells us what it wants to use on session setup
<dxld> yeah
<alanz> that is more than what lsp can do
<dxld> but since not all project kinds are always going to be there I think we should support checking what is actually there
<alanz> At the moment, we do some scrabbling around to try and determine which version of hie to start up, to match the ghc used in the project
<dxld> and not just have the UI show lots of useless options
<alanz> that is the first problem
<dxld> I've been saying (for years now) we have to just handle that in a dispatcher
<dxld> the ide shouldn't have to even know about such problems
<alanz> So essentially it comes down to making something available to the client that can decide how to launch hie. Or which hie
<dxld> I don't think so
<alanz> well, if I am working on HaRe, I may want to work a bit with GHC 8.2.2, then switch to 8.4.4
<dxld> hie should just be a dispatcher that launches the right underlying exe automatiaclly
<dxld> or well with guidance of the ide/user's config
<alanz> yes, that is the concierge model
<alanz> there is an issue for it
<dxld> what I need to know is if that's possible with lsp and does that integrate with stock ides
<dxld> or do we need some haskell specific magic for that?
<alanz> And the "right" way means replacing a process with another, and keeping the messaging context in place
<dxld> sure
<dxld> that's perfectly doable
<alanz> the "cheap" way is to offer a probe process that can return the appropriate hie and params to use
<dxld> just buffer the messages until you manage to decide which instance to start and then replay :)
<dxld> (for instance)
<dxld> there's tons of ways to do it for sure
<alanz> its perfectly doable, but gets messy when you factor in all the different distributions etc
<dxld> true
<alanz> windows, nix, linux, ....
<dxld> but let's get back to the core issue, how to expose this in ghc-mod-core
<alanz> and within linux, ubuntu, debian, arch, fedora ...
<alanz> Well, that is why I suggested it coming in as a parameter of some kind. So it can be used to build one of these other things we just discussed
<dxld> isn't the whole point of lsp not having to have language specific hacks in each ide?
<alanz> and the param is either telling g-m-c to do what it always did by detecting, or forcing a particular config
<dxld> if we're passing random commandline options to hie doesn't that violate that?
<alanz> well, in practice each language has to have a tiny client shim which is used to decide which server to start
<alanz> See all the repos here: https://github.com/emacs-lsp/
<dxld> that shim running in the ide/editor?
<alanz> the main ones (lsp-mode, lsp-ui), and all the rest are language startups
<alanz> yes
<dxld> ah, ok
<dxld> so we could slap any version/project type negotiation in there, good.
<dxld> though that is kind of annoying ..
<alanz> And this is the key to the haskell one https://github.com/emacs-lsp/lsp-haskell/blob/master/lsp-haskell.el#L275
<alanz> yes
<alanz> And the vscode one has this massive shell script wrapper to try and work it out
<dxld> :)
<dxld> do you have a link to that?
<alanz> I would rather offer a thing something like "hie-chooser" that all clients could use
<alanz> yes, hold on
<dxld> by "the vscode one" you mean just the haskell hie thing for vscode, right?
<alanz> https://github.com/alanz/vscode-hie-server/blob/master/hie-wrapper.sh
<alanz> yes
<dxld> not like a shell script that figures out which lsp thing to start for each language
<dxld> ah, right
<alanz> but each language client in vscode does basically this
<alanz> So there is a tiny startup process, the rest is standard
<alanz> unless some extra commands are exposed, e.g. the HaRe refactoring ones for haskell
<dxld> re hie-chooser, yeah I was thinking some simple thing that a shell script could have a dialog with could work
<alanz> The hie wrapper ties in with the makefile in hie
<alanz> and needs to be updated for GHC 8.4
<dxld> I'll design the api in gmc to take full advantage of the project info we have then
<alanz> yes
<dxld> or actually factor that whole thing out into a really small library
<alanz> so split it in two, the chooser, which uses gmc (with or without c-h), and hie itself
<dxld> then we can build a tiny executable that just does project finding, that can be used to figure out which ghc version is in use and what project type
<alanz> and the chooser passes back params for gmc in hie
<alanz> yes
<dxld> I was thinking just a new library that implements finding all the project files
<dxld> like stack.yml, cabal.project, foo.cabal etc.
<dxld> and just exposes that
<dxld> I guess ghc-version stuff would have to be deferred actually
<dxld> because for some situations we'd need cabal-helper
<alanz> ok. to me we should put all the logic on one place, so we don't have to rewrite it for vscode, emacs, atom, ...
<dxld> though technically that project finder could also link against c-h
<alanz> yes
<dxld> yeah that's my thinking too
<alanz> And we must be able to pass in some a-priori stuff
<dxld> the seperate library is just so we don't have to link against GHC for this stuff
<dxld> like?
<alanz> like "I want to use stack-8.0.2.yaml" now
<dxld> oh yeah, sure
<alanz> but that goes as a param to the chooser
<dxld> I imagine something like --stack-yml=foo.yml
<dxld> and --cabal-project-file=...
<alanz> yes
<alanz> ad something to force straight old style cabal tooo
<dxld> of course
<dxld> hence --cabal-project-file
<dxld> --cabal-file would force the old style ;)
<alanz> ok
<dxld> ok I think I have a clearish idea of what needs doing now
<alanz> great

[cfraz89]

Hey alanz,

Been digging around old issues to figure out where to start here.
It seems like there was an effort to get build options directly into cabal-install 2771, which would have deprecated cabal-helper entirely. Maybe a good place to start would be to revive the PR and finish it off?

[alanz]

@cfraz89 That sounds like a good idea, eventually cabal-helper should go away.

Perhaps check with @DanielG too.

[newhoggy]

Has there been any progress on this issue?

[cfraz89]

@newhoggy I've been chipping away at baking all of cabal-helper's functionality into cabal, as well as supporting new-build in the process. Theres still a way to go, and unfortunately I've been stalled for the last 2 months as I've just moved overseas, and had to leave my computer behind. I'll be picking it back up once I purchase a new machine, however it'll still be a while before there's anything to show for it I think.

If you want to track progress, it's on the 'build-info' branch of https://github.com/cfraz89/cabal

[newhoggy]

Thanks for the update. Really appreciate it.

[newhoggy]

Hello, I was wondering how this is going?

[pshemass]

Hello, any update on that?

[Infinisil]

Check out https://github.com/haskell/haskell-ide-engine/pull/905 \o/

[alanz]

Ok, I just checked with @DanielG, and it seems that his update for cabal-helper / ghc-mod is going to take a bit longer, and depends on a future release of cabal.

So it is time to actually pay attention to this one.