Dhall-haskell: Multiline ability in repl

[thebritican]

Background

I've been using repl examples as documentation and :load, :save commands to allow readers to follow along with the docs. This has been great until we introduce deeply nested records like those seen in dhall-kubernetes.

Problem

You can see the following example is hard to follow in the single line repl. I've had to split up the k8s deployment record so readers can keep most of the record in view w/o scrolling

⊢ :let replicas = 1

⊢ :let spec = kubernetes.DeploymentSpec::{ template = kubernetes.PodTemplateSpec::{ metadata = kubernetes.ObjectMeta::{ name = "test-app-server" } }, replicas = Some replicas }

⊢ :let minimalDeployment = kubernetes.Deployment::{ metadata = kubernetes.ObjectMeta::{ name = "test" }, spec = Some spec }

It would be great if we could add a character to allow multi-line expressions

A potential solution

⊢ :let replicas = 1

⊢ :let spec = ~
      kubernetes.Deployment::{ ~
       , metadata = kubernetes.ObjectMeta::{ name = "test" } ~
       , spec = Some ~
          kubernetes.DeploymentSpec::{ ~
          ,  template = kubernetes.PodTemplateSpec::{ ~
             , metadata = kubernetes.ObjectMeta::{ name = "test-app-server" } } ~
             , replicas = Some replicas ~
          }

Perhaps :paste option could replace newlines pasted from your clipboard with such a character?

[sjakobi]

We should definitely support multi-line expressions in the REPL!

Admittedly I don't have much experience with REPLs apart from GHCi, where I'm not a huge fan of the multi-line support via :{/:}.

Are there existing REPLs that use a special end-line character like ~ to signal multi-line expressions?

Maybe we could look at what other similar languages do. PureScript, which also uses haskeline, has a :paste command. Can anyone comment on how well that works?

[Gabriel439]

This is possible, albeit a bit of work, as we would need to upstream repline support for interpreting lines read in this way. Alternatively, if we were using haskeline directly then this would be easier to to do

[sjakobi]

Ah, I thought we were already relying on haskeline for all our REPL needs, but apparently our usage is limited to autocompletion and catching exceptions, and we use repline too.

[thebritican]

Are there existing REPLs that use a special end-line character like ~ to signal multi-line expressions?

No, ~ was a sort-of silly suggestion. I'm only familiar with \ from bash and i'd guess it would be confusing since it's used to define lambdas already.

[Gabriel439]

I personally liked the :paste command idea

[basile-henry]

This issue looks interesting to me! :smile: Since I have already dipped my toes on the REPL part of the code-base, I'd love to tackle this over this ZuriHac weekend.

I like this :paste idea. Do we want to implement this just for the Dhall REPL or do you think this is something that is likely to be upstream-able to haskeline? See https://github.com/judah/haskeline/issues/69

Assuming we want a Dhall REPL only solution, I have a few extra notes/questions about this feature:

• Do we allow _"commands"_ in the multi-line paste (:let, :load, :save, ...)?
  I think we should otherwise it can only be used to evaluate a snippet of code which makes it not all that useful.
• If we do, which commands make sense to allow? I'm not sure :help makes much sense but at the same time it might be simpler to allow all the commands.
• We probably want to disallow nesting :paste commands. Unless there is a use case I am missing? :sweat_smile:
• Allow multiple commands in the same multi-line paste?
  We'll need to make sure they are properly executed sequentially since they might depend on one another.
  Should the output of each command be shown?
  What happens if an early commands fails to parse/other failure? Abort the other commands?
• Autocomplete support within multi-line (or will it _"Just Work"_?)
• Should it actually be named :paste or would something like :multiline make more sense?

I had a few extra ideas for different implementations. I think I still prefer the :paste version, but I'll share what my thoughts were anyway:

• Have another version of all the commands that would support multi-line :mlet, :msave and so on.
  The multiline would be _"committed"_ on double Enter (when encountering an empty line).
  But then this means no multiline support for just evaluating an expression.
• Have a :multiline command that switches to a multiline mode for the rest of the session (with a :nomultiline option to stop/or use :set/:unset if we want more modes in the future). I think this is similar to ghci's :set +m. This would also be committed on double Enter.
• Use Shift-Enter to go to a newline without committing the command (à la Slack). I think this would unfortunately not work with many TERM and would definitely need lower level access to the terminal than what repline/haskeline give us.

[sjakobi]

@basile-henry It would be great to see some progress on this issue! :)

Regarding multiline support for the various :-commands, would it make sense to support :paste as an "argument"?

So e.g. with :let, you'd type

⊢ :let x = :paste
<multiline
expression
here>

:type, :hash and :save could handle it similarly.

[basile-henry]

@sjakobi It's an interesting idea! I'm not sure how discoverable it would be though. I like the fact that you can change your mind about making something multi-line only at the end of the line.

Maybe we could have both a simple :paste command and a way to introduce a new line more "manually", maybe using a character such as \ (like Bash) as was suggested previously.
I think both solutions complete each other.

[sjakobi]

I'm not sure how discoverable it would be though.

Indeed.

Maybe we could have both a simple :paste command and a way to introduce a new line more "manually", maybe using a character such as \ (like Bash) as was suggested previously.
I think both solutions complete each other.

Sounds good to me. I somewhat agree with @thebritican that \ might cause confusion with lambdas though. How about %, $ or some other symbol that we don't use for syntax?

[Gabriel439]

I like to revisit the original requirement whenever the design is unclear.

The high-level requirement from @thebritican was to be able to author example tutorial material where the REPL commands span more than 80 columns.

Based on that requirement, we don't actually require the ability to paste more than one command at a time. Also, we wouldn't really need that anyway as the REPL already has a :load command for running a "script" of REPL commands.

Also, based on that requirement, we don't need this functionality to be dhall-aware or repline-aware. Just the ability to read input that spans more than 80 columns without typing in one very long line is enough, so the proper place for this seems like it would be haskeline.

[basile-henry]

we don't actually require the ability to paste more than one command at a time

Yes I understand that isn't the goal of this issue. My question about the multiple commands came from the way :paste as described in the Purescript issue works. It seems to only support evaluation as far as I can tell. In the Dhall REPL we use commands a lot more (since we use it for :let), so that means we would like to have multi-line support in commands as well as simple evaluations. Maybe the :paste way isn't appropriate then?

Just the ability to read input that spans more than 80 columns without typing in one very long line is enough

I agree that ideally haskeline would be able to hand us more than one line of text per input. But different applications might want to implement this in different ways (escaped newline, two newlines in row, special brackets, handle it differently depending on state). Maybe there is an API we could get in to haskeline that would make it easier to implement.

I know ghci uses haskeline, but it looks like it's rolling out its own multi-line as part of its own state:
https://gitlab.haskell.org/ghc/ghc/-/blob/master/ghc/GHCi/UI.hs#L1041-1047

If we can make haskeline more configurable then that would be great! But this feature will still need to be handled in Dhall REPL in one way or another.

[Gabriel439]

@basile-henry: Yeah, I agree that haskeline might not be able to support a general mechanism. However, since repline is intended to be more opinionated than haskeline maybe that's the more appropriate place to upstream such multi-line support

[basile-henry]

I agree that repline is probably the sweet spot. Low level enough to have make it easy to control line inputs, and opinionated enough that we might be able to get one type of multi line accepted.

Now what type of multi line flavour do you think would be best suited for the Dhall REPL?

[basile-henry]

\ might cause confusion with lambdas though

@sjakobi I think disallowing the start of a lambda as the last character of a line (in the REPL) is not too bad, I would struggle to read that as a lambda if it was allowed. But I am completely fine using another character others think \ would be confusing.

[Gabriel439]

@basile-henry: How about using a special character like ~ to begin a multi-line command and then ending it with a blank line?

[basile-henry]

Multi-line is now entirely implemented in repline and will be available with release 0.4 https://github.com/sdiehl/repline/pull/27

The flavor of multi-line is the same as the Purescript one linked to earlier in this issue. I ended up with this version as it makes two aspects I care about work well:

• Actually pasting text works (without requiring any post-processing).
• It can be made easily discoverable by mentioning it in the output of the :help command. An escape character might not be as discoverable.

I will open a PR using this new version as soon as it becomes available on Hackage. We can bike-shed the naming of the command then :sweat_smile:

Note: regarding multi-line command inputs, they will work, so we should be able to do something like this:

> :paste
-- Entering multi-line mode. Press <Ctrl-D> to finish.
| :let foo =
|   2 + 2
|
>

Multiple commands in a single paste won't be supported (this seems to also be the case in ghci's :{/:}).
Regarding autocomplete, I expect it'll work with no changes but I still need to confirm that.

[Gabriel439]

@basile-henry: Awesome work! I'm really looking forward to this 🙂