Haskell-language-server: REPL Support

[alanz]

Haskell is legendary as a REPL driven language, via ghci. This is the last major piece missing from our IDE offering.

How will we do it?

This is my personal brain dump, specifically to seed a discussion on the topic. Nothing cast in concrete.
Note: I use "the server" to refer to the conglomerate of HLS, ghcide, hie-bios, etc.

• The REPL has to be part of the running server.
  
  
  
  • Given the immense memory usage cost we already incur, running a separate ghci session does not make sense
  
  
  • Also keeping things in sync becomes problematic
  
  
• It has to have unfettered access to its own IO
  
  
  
  • A user needs to be able to run code, which they have just written. This mostly does IO
  
  
  • A corollary of this is that running user code should not cause a lack of responsiveness in the server
  
  
• It has to have a well-defined, well-integrated way of integrating with an IDE.
  
  
  
  • We have come this far using LSP, ideally we should leverage existing protocols implemented in most IDEs
  
  

My high level view of a way of achieving all these goals is the following

• Use the Debug Adapter Protocol (DAP)
  
  
  
  • This already has first class support in VsCode and emacs, and coming to others as it picks up usage
  
  
• Expose a DAP TCP endpoint in the running server.
• DAP is just a message passing protocol, intending to talk to a target manager. This role is played by the server.
• Use the RunInTerminal reverse request to initiate the session. This opens up an inferior terminal in the client, which exposes the REPL
• Run something in this terminal which talks back to another endpoint in the server. This endpoint is used to run the read-eval-print loop for a ghci session
• The ghci session runs via remote-interpreter, against hooked stdio pipes, which are routed via the third endpoint to the terminal in the client.

Voila. The rest is detail.

Back of envelope sketch of the concept

See also

• Is bringing up a REPL within the scope of HLS? #202
• Extended Eval Plugin #438
• Use -fexternal-interpreter https://github.com/haskell/haskell-ide-engine/issues/1257
• https://github.com/alanz/erlang_ls/tree/dap-vscode

[Anrock]

Sounds nice, but I don't get what benefits this will provide vs. just running ghci in terminal? One I see is a proper debugging support which is awesome thing, but the issue here is called REPL not Debugger.

[alanz]

I don't get what benefits this will provide vs. just running ghci in terminal?

Doing this means you need a whole extra GHC session, with a potentially massive extra memory usage. For a small project this is fine, but it can be a problem.

And for me that fact that is has debugger support is a happy accident, I am not a strong believer in that stuff.

however, it does unlock cool things like https://github.com/hediet/vscode-debug-visualizer

[jneira]

I think a debugger could be attractive for developers from other languages which has it from the beginning.
But i've been an emacs user and to have the repl connected with the edited file and reload changes in one click was very handy.

[alanz]

to have the repl connected with the edited file and reload changes in one click was very handy
Does this mean you no longer do this? Or is it still good enough for you?

For me, the haskell inferior repl stopped working some time ago, so nowadays I run M-x shell, and launch a repl there, with cabal new-repl or whatever.

[1Computer1]

One thing I've always wanted was something like a scratch file, e.g. DrRacket but better. So you would type some valid GHCi lines in the scratch file, and it would watch the scratch file and your code for changes and run the scratch file in GHCi and you would see the output of each command. I don't really know how feasible this would be though.

[ndmitchell]

I think once we have basic REPL functionality, we can then go crazy with lots of more advanced forms. But getting a basic REPL enables a lot of other things.

@pepeiborra is the only person I've ever seen use the GHCi debugger effectively. But when he uses it, it is effective, and I think it's mostly a question of UI rather than anything fundamental.

[tittoassini]

The Eval plugin already comes rather close, you can write down a bunch of expressions and tests and get all of them executed with a single click.

One thing I've always wanted was something like a scratch file, e.g. DrRacket but better. So you would type some valid GHCi lines in the scratch file, and it would watch the scratch file and your code for changes and run the scratch file in GHCi and you would see the output of each command. I don't really know how feasible this would be though.

[tittoassini]

In fact, I am not sure that the traditional REPL interface is particularly effective.

When implementing a function, I find it way more useful to write a bunch of test in the functions' docs to explore and specify its behaviour and execute them in parallel after any change.

It's faster than keep retyping expressions in the REPL and has the huge advantage of leaving behind a set of permanent tests that are embedded in the function documentation. It's effectively a "docs for free" system.

The REPL model, based on the ephemeral evaluation of expressions that end up discarded and unrecorded, it's a massive waste of time.

What is really missing is the possibility of looking "into" complex functions to observe their behaviour, i.e. some intuitive form of debugging/logging.

Breaking down function in smaller functions can reduce but not completely substitute for an explicit and intuitive visualisation of the execution trail (at least for non-pure functions whose behaviour can only be properly observed "in vivo").

To conclude, I see way more potential in embedding evaluation in the source code/documentation itself than in having a separate evaluation environment. It's not a chance if 'notebook' environments that integrate textual documentation and 'persistent evaluation' like jupyter or mathematica are increasingly popular. They are not just a better way to develop code but also to use it and share its results.

[alanz]

@tittoassini I fully agree that having in vivo live code exploration is good, and leaves permanent artifacts at the point they are relevant.

But people use REPLs for other things too. And to some extent, having a first-class REPL solution makes the living doc model more tractable, in terms of expectations around edge cases.

[tittoassini]

@alanz No doubts REPL has its uses and it would be welcome.

Just saying that for most use cases the persistent-evaluation-in-place model, that systems like HLS make easily accessible, is a more efficient solution.

But there is no conflict.

In fact, the code required to implement both is pretty much the same. Just two different user interfaces to the same underlying evaluation engine.

[alanz]

A related GHC issue GHCi should support interpeting multiple packages/units with separate DynFlags
And there is another one I can never find, will keep looking.

[alanz]

I see vscode is proposing a Notebook extension too.

Although that might be a better fit for the eval plugin