Julia: Option to suppress displaying value of assignment in REPL

[jlumpe]

In Julia I guess assignment statements evaluate to the assigned value, so it makes sense from a language perspective that executing them in the REPL prints the result just as it would for any other statement. However I don't think I'm alone in that this is not desired a large proportion of the time (personally I would say at least 80-90% of the time I'm not interested in seeing the value). For some values it's not a big deal, but vectors take up one line per element when pretty printed so assigning even relatively small ones can take up most of my terminal height.

I just found out that adding a semicolon at the end suppresses printing, which definitely makes this less of an issue, but I think it would be very convenient to have a function or variable in the REPL module that lets you disable printing of assignments. In the case the user does want to see the value they just need to type the variable name again.

I see there is already a function ends_with_semicolon(line) that checks for the semicolon case and is usually used for the show argument to print_response():

https://github.com/JuliaLang/julia/blob/de48421024439b873c7616c607e3310a8f24c2d3/stdlib/REPL/src/REPL.jl#L1118

It seems like it shouldn't be too difficult to add another condition, if this sounds like a good feature.

[JeffBezanson]

I would say I want to see the value at least 99.9% of the time :)

[smallory]

For interactive analysis of data, this would be a really nice feature to be able to set. With a dataframe I am currently working with, I lose 15 minutes to wasted terminal I/O every time I forget the semicolon.

[StefanKarpinski]

If any large object displays the entire object in the REPL, then that's the real problem. All objects defined by Julia itself print truncated versions of themselves that don't take long to print.

[jlumpe]

When I submitted this issue I wasn't aware that ending a statement with a semicolon suppresses printing the value. The fraction of the time I'm interested in seeing the value is much closer to 50% than 99.99%, but that's often enough that adding a semicolon when needed is the quickest and easiest way to accomplish this.

In response to @smallory's issue, I think it could be of value to add a field to REPL.Options to truncate the output at some limit.

Currently, a hacky way of truncating output could be by modifying Base.active_repl (or passing a customization function to atreplinit in your startup.jl file). Assuming it is an instance of LineEditREPL, there is a specialdisplay field you can modify. It seems like results are printed with display(repl.specialdisplay, x) if it s not nothing (which it seems to be by default) or display(x) otherwise (see Base.REPR.print_response). You could set this to a custom AbstractDisplay implementation that truncates the output.

There is also repl.options.iocontext which defines the defaults for show when displaying to the REPL, but it already defaults to :limit => true so I'm not sure what else you can do with it.

[StefanKarpinski]

I think it could be of value to add a field to REPL.Options to truncate the output at some limit.

There's no need for an option because REPL output should _always_ be truncated. Any package that doesn't respect this should have issues filed against it.

[smallory]

Is this thread the best documentation to share for this requirement? My google-fu is to weak to find documentation of package expectations elsewhere that have this, and I'd like to be able to recommend a switch to Julia for EDA on large sets.

[StefanKarpinski]

I guess we haven't really documented this, although it's been in the ethos of the Julia REPL ecosystem from early on—we went to great pains to make sure that huge matrices are summarized and printed efficiently in the REPL. May I ask which data types are not following this?

[smallory]

DataFrame from DataFrames.

[jlumpe]

@StefanKarpinski I very much disagree that there is no need for it. It may be true that REPL output _should_ always be truncated, but third-party packages don't always follow this.

[StefanKarpinski]

We should document, but I'm not sure where. If anyone has any good ideas about where to document this, please either submit a PR or file an issue requesting that doc text be added.

[JeffBezanson]

DataFrame from DataFrames.

Huh? DataFrames are absolutely truncated by default in the REPL:

julia> DataFrame(Any[1:100 for i = 1:100])
100×100 DataFrame. Omitted printing of 90 columns
│ Row │ x1    │ x2    │ x3    │ x4    │ x5    │ x6    │ x7    │ x8    │ x9    │ x10   │
│     │ Int64 │ Int64 │ Int64 │ Int64 │ Int64 │ Int64 │ Int64 │ Int64 │ Int64 │ Int64 │
├─────┼───────┼───────┼───────┼───────┼───────┼───────┼───────┼───────┼───────┼───────┤
│ 1   │ 1     │ 1     │ 1     │ 1     │ 1     │ 1     │ 1     │ 1     │ 1     │ 1     │
│ 2   │ 2     │ 2     │ 2     │ 2     │ 2     │ 2     │ 2     │ 2     │ 2     │ 2     │
│ 3   │ 3     │ 3     │ 3     │ 3     │ 3     │ 3     │ 3     │ 3     │ 3     │ 3     │
│ 4   │ 4     │ 4     │ 4     │ 4     │ 4     │ 4     │ 4     │ 4     │ 4     │ 4     │
│ 5   │ 5     │ 5     │ 5     │ 5     │ 5     │ 5     │ 5     │ 5     │ 5     │ 5     │
│ 6   │ 6     │ 6     │ 6     │ 6     │ 6     │ 6     │ 6     │ 6     │ 6     │ 6     │
│ 7   │ 7     │ 7     │ 7     │ 7     │ 7     │ 7     │ 7     │ 7     │ 7     │ 7     │
│ 8   │ 8     │ 8     │ 8     │ 8     │ 8     │ 8     │ 8     │ 8     │ 8     │ 8     │
│ 9   │ 9     │ 9     │ 9     │ 9     │ 9     │ 9     │ 9     │ 9     │ 9     │ 9     │
⋮
│ 91  │ 91    │ 91    │ 91    │ 91    │ 91    │ 91    │ 91    │ 91    │ 91    │ 91    │
│ 92  │ 92    │ 92    │ 92    │ 92    │ 92    │ 92    │ 92    │ 92    │ 92    │ 92    │
│ 93  │ 93    │ 93    │ 93    │ 93    │ 93    │ 93    │ 93    │ 93    │ 93    │ 93    │
│ 94  │ 94    │ 94    │ 94    │ 94    │ 94    │ 94    │ 94    │ 94    │ 94    │ 94    │
│ 95  │ 95    │ 95    │ 95    │ 95    │ 95    │ 95    │ 95    │ 95    │ 95    │ 95    │
│ 96  │ 96    │ 96    │ 96    │ 96    │ 96    │ 96    │ 96    │ 96    │ 96    │ 96    │
│ 97  │ 97    │ 97    │ 97    │ 97    │ 97    │ 97    │ 97    │ 97    │ 97    │ 97    │
│ 98  │ 98    │ 98    │ 98    │ 98    │ 98    │ 98    │ 98    │ 98    │ 98    │ 98    │
│ 99  │ 99    │ 99    │ 99    │ 99    │ 99    │ 99    │ 99    │ 99    │ 99    │ 99    │
│ 100 │ 100   │ 100   │ 100   │ 100   │ 100   │ 100   │ 100   │ 100   │ 100   │ 100   │

[jlumpe]

Also, it's very easy to trigger large amounts of output accidentally while you are developing and testing your own types.

[smallory]

Thanks to Jeff's note, I poked around and found that this is terminal-dependent behaviour. If I use a fixed-size terminal with finite scrollback, I only get the first and last 13 lines. While I usually get over 12k lines dumped to the screen if I forget my semi-colon.

It does appear that the truncation code that DataFrames has needs a bug report.

And, when I'm in a terminal that is not correctly understood by a library I'm using, I'd like to turn off the default display of values.

[StefanKarpinski]

Julia provides a displaysize function which uses stty to determine the size of your display. For example, mine correctly determines that my terminal is currently 48 x 74:

julia> displaysize(stdout)
(48, 74)

Either that is not working correctly on your terminal or DataFrames is not using it to figure out how much of a DataFrame to display. What does that return in your setup?

[StefanKarpinski]

And, when I'm in a terminal that is not correctly understood by a library I'm using, I'd like to turn off the default display of values.

Rather than give you an option to mitigate the symptom of a bug, the right thing is to fix the bug. We clearly can't "turn off the default display of values" in the REPL because that's the whole point of a REPL. Without the display of values, it's just you typing a program incrementally (which you can do pretty easily if you want). The only thing that could be done is to suppress the printing of the result of assignment, but that's a hack because unlike Python where an assignment doesn't evaluate to anything, in Julia it does, and distinguishing between an assignment and not an assignment is not that clearcut: there are many expressions that do assignment but also do something else. That would also not fix the problem that examining a DataFrame when you're not doing assignment prints 12k lines of junk that you don't want to see.

[smallory]

Starting up a fresh session to check displaysize, getting:

julia> displaysize(stdout)
(24, 74)

So I tried loading my data set, and it's now busy dumping output... apparently a DataFrames bug.

And yeah, I get it about python's assignment hack. I pictured using the REPL more like IDL than python, where if I want to see something, I have to print it, and think about what parts I'm printing.

[StefanKarpinski]

apparently a DataFrames bug.

Ok, would be great if you could file an issue with DataFrames since they should be using displaysize to detect the size of your terminal when displaying.

[kzapfe]

Okey, it is not only in DataFrames this issue. Also HDF5 produces the same behaviour on certain terminals. I really do not know if its a problem of HDF5 or of the terminal, but if I open a big array of an HDF5 file and assign it to a variable, it prints the whole thing... and if it is a 4096 by 179000 points array, it is simpler to kill the process than to wait. By the way, I use the terminal embedded in a Emacs session... Anyone has experienced this behavior within emacs?

[StefanKarpinski]

I'm going to need some cooperation from the people posting here:

1. What does displaysize(stdout) show in your terminal? If it's something reasonable like (24, 74) or (20, 80) then this is not a Julia issue, it's an issue of packages not using displaysize to detect the output size and limit what they print. If it does show something unreasonable, then it's a Julia bug and we need to know that in order to fix it.

2. If it's not a Julia bug – i.e. displaysize(stdout) shows something reasonable – they you need to file issues with the relevant packages—namely DataFrames and HDF5. If you do so, please post links to those issues here so that we can track them and make sure they get fixed.

If you don't do these things, nothing is going to improve. I can't test on your computer and I'm not going to file issues about things that I didn't witness personally and can't reproduce on packages.

[kzapfe]

Okey, I think is something that may be with the Emacs and the displaysize and those packages. I will file things with them.

[ezaron]

Well, this is behavior that changed with a different version of Julia.

The emacs shell mode has well known limitations, but they did not
change. The cause for the change in behavior was internal to Julia. I
think the latest julia is expecting more from the shell than it used to,
but maybe it is readline or something else that is working a little
differently.

-Ed

On 7/9/20 4:31 PM, kzapfe wrote:
>

Okey, I think is something that may be with the Emacs and the
displaysize and those packages. I will file things with them.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/JuliaLang/julia/issues/31176#issuecomment-656400147,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABD4FGFI3NDKCL6D3MJ23J3R2ZHOPANCNFSM4G2ETQFQ.

[StefanKarpinski]

Folks. I explicitly asked for very detailed help to help you here, but no one has responded with that information. This cannot be done for you since only you have the system where this is a problem. If anyone wants this to improve they have to do what's asked in https://github.com/JuliaLang/julia/issues/31176#issuecomment-634176409 and report back.

[ezaron]

Thank you for coming back to this. Since this thread was closed, I had opened a new thread here:
https://github.com/JuliaLang/julia/issues/36492#event-3499936116

The displaysize and other information is mentioned in that new ticket, above, and it is all reasonable.

The TL;DR seems to be:

• Julia previously truncated array outputs within emacs shell-mode.
• In recent versions, Julia does not truncate output in emacs shell-mode.
• Shell-mode presents a dumb terminal to Julia, so it is not surprising that the REPL is not full-featured in shell-mode.
• Nonetheless, something changed on the Julia side, and latest Julia no longer truncates output within emacs shell-mode. (Different behavior obtains in other emacs terminal modes.)