Cider: Have a configuration option, so that cider auto-erases the repl buffer when it becomes too large

[behrica]

The well known problem of emacs/cider's behavior on printing long or complex data structures in the repl
is very frustrating.

There are several efforts on going to solve this on the level of truncating the output automatically, in the nrepl for example.
The latest attempt is this:
https://github.com/nrepl/nrepl/issues/75

From my proper experiments, I think it is not enough to limit this it the nrepl allone.
(as a nrepl solution does not solve the problem, that the repl buffer grows and grows over time, and sometimes it fills too quickly for the user to react)

See the reasoning and a potiental, very simple, solution, based on automatically erasing the
repl buffer here:

https://github.com/nrepl/nrepl/issues/75#issuecomment-440999775

The proposed solution helps as well for #1115

[behrica]

see the patch to enable the auto-erase of the cider repl buffer here;
https://gist.github.com/behrica/387e5838d8f933f6eb239f27b17577fc

[behrica]

A good test case for the patch is the following:

1. enable cider pretty printing (important , as only in this case we get several insert events)
2. Evaluate the following in repl

(set! *print-length* false)
(range 1000000)

You will see in the Messages, that the buffer gets constantly erased and therefore stays in the limits.
Emacs is still unresponsive (and the evaluation cannot be interrupted in general, even though I noticed that some times C-c C-c does work) while it is printing,
but it will finish after some minutes and Emacs stays responsive.

The behavior of evaluating very long strings is far improved as well, in my view:

 (apply str (range 1000000))

Currently there is no way to prevent emacs blocking on this. (as there is no clojure variable such as print-length-string.
With the patch applied I can evaluate this on arbitrary long strings (until I reach OutOfMemory from the Jvm),
without crashing emacs.
Emacs might run for some minutes and be unresponsive in that time, but it will not crash or end in a state where no key can be pressed any more.
This is related to the large string problem discussed here #1115
So this patch would solve #1115 to a large extend.

[behrica]

I experimented as well with an other idea, which is to request user confirmation before inserting large content into the repl.
This goes as well in the line of thinking, that if I have a response of 1 MB, then I probably evaluated the wrong thing.

The function for this could then look like this:

(defun cider--checked-insert-before-markers (response-string buffer-size)

  (message "Currrent cider repl buffer size is: %s "(buffer-size))
  (message "Length of string to insert: %s " (length response-string))

  (let ((max-response-length 10000)
        (max-buferer-size 1000)
        (response-length (length response-string))
        )
    (if (> response-length max-response-length)
        (let ((choice (read-multiple-choice (format "Large response (%s). Continue ?" response-length)
                                            '((?y "yes")
                                              (?n "no")
                                              )
                                            )))
          (if (string-equal "yes" (second choice))
              (progn (if (> buffer-size max-buferer-size)
                         (progn (message "Buffer grew to large -> erase")

                            (let ((inhibit-read-only t))
                              (erase-buffer))
                            )      
                       )

                     (insert-before-markers response-string))

            )
          )
      (insert-before-markers response-string)

      )))

This works well, unless the response data structure is a "shortish vector" but each element is large.
Then the above would ask a lot of questions....

[behrica]

Maybe a good starting point into this direction would be to expose a new function:

(defun cider--checked-insert-before-markers (response-string buffer-size)
        (insert-before-markers response-string))))

which by default has the same behavior as current code.

But the user could implement his own logic easily to decide what to do in case of
"largish new input" and "large repl buffer".
This goes then in the direction of having a kind of ____insert reponse handler____

[bbatsov]

The suggestion is reasonable, but I think the default behaviour should be to just delete old parts of the REPL buffer not clear it completely. The most recent output is usually useful. You also need some handling for huge outputs/results to be directed straight to a dedicated popup buffer IMO. One big output and the REPL is dead even with auto-deletion. (or useless if you delete this before the user can see it)

[behrica]

In any case, this need to be very configurable, I think.
There should be no default, which deletes any output.

So maybe a good quick start could be to have this function as above,
'cider--checked-insert-before-markers'

which gets called on each insert into the repl.

Then we could easily implement and experiment with more advanced truncation behavior.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contribution and understanding!

[jumarko]

Any plans to implement this?
My emacs gets slow as hell if I keep REPL running with an application logging lots of stuff.

[cmal]

Same question.

[bbatsov]

It's relative easy to implement this (especially now that one can limit the maximum output a single evaluation can produce), I just never got to doing it. PRs welcome! :-)

[cursande]

Hi @bbatsov

Had some issues with this recently due to heavy logging, first time looking into cider code but I'd like to make a pass at implementing this :)

Reading the previous discussion, it seems like we can leave the limiting of individual eval output to cider-print-quota and just check the current buffer-size and whether it's larger than some custom byte threshold (nil/off by default). Am I missing anything?

I think the default behaviour should be to just delete old parts of the REPL buffer not clear it completely. The most recent output is usually useful

Looking at cider-repl-clear-buffer I'm not sure what the above would entail. Did you have specific points in mind? Or something like taking the current prompt and only keeping n many lines before it?

[bbatsov]

As the REPL buffer is just text we can simply check its size after each evaluation and trim the buffer's beginning by some predefined size. We also have to be careful to respect input boundaries not delete partially some input or its results.