Pluto.jl: Display DataFrame with more than 18 rows/columns

[lungben]

For larger DataFrames, are the first 18 rows and columns are displayed in Pluto.
Is there a way to increase this number?
I already tried the "standard methods" suggested by DataFrames.jl: setting the COLUMNS and LINES environment variables or using show, but without success.

using DataFrames

ENV["COLUMNS"]=1000
ENV["LINES"] = 50
# method that works in REPL / IJulia,  but seems to have no effect in Pluto
# https://juliadata.github.io/DataFrames.jl/stable/man/getting_started/#Installation

df = hcat([DataFrame(a=1:30, b=2:31) for _ in 1:10]..., makeunique=true)
# only 18 rows and 18 columns are displayed (the df size is 30x20)

show(df, allrows=true, allcols=true) # prints on the server side, but not in the notebook

Could you please help?

[fonsp]

That's odd - does IJulia display tables using unicode characters (like the REPL) or with an HTML table?

[lungben]

It is printed as HTML table.

screenshot

[fonsp]

I meant IJulia/Jupyter

[lungben]

In IJulia the table is also printed as HTML.

[Sh4pe]

I've added the PR #318 (a draft as of now) that would solve the problem. As of now, the environment variables LINES and COLUMNS would have to be set before importing Pluto however.

[fonsp]

Hey so another solution is to use a different IOContext for the text/html MIME that for the other MIMEs.

Changing the displaysize of the current IOContext will make html tables bigger, but will also make 2D arrays unreadable (because they use the same variable). But because Pluto has horizontal scrolling for HTML output, we can use a very large value by default - it will probably only affect DataFrames.jl.

The problem then is when you want to see even more rows/columns.

[lungben]

Sounds great!
I am usually setting ENV["COLUMNS"] = 1000 in my Jupyter notebooks, maybe this is a good starting point.

[fonsp]

But isn't 2D array printing also affected?

[non-Jedi]

Someone asked about this on zulip, so I thought it would be good to document the solution I offered here as well. You can use the BrowseTables package for this:

using BrowseTables
HTMLTable(df)

That will show the whole dataframe (or any other Tables.jl compatible source) as an html table (be careful with very large tables...).

Another solution would be the TableView package, but that doesn't currently work because Pluto isn't integrated with WebIO (#299).

[lungben]

BrowseTables does not work very well for me in Pluto because no horizontal scroll bars are shown, thus the output columns are truncated.

In my opinion the user experience would be best if the default display of DataFrames (or any Tables.jl interface) would display all columns (or at least with a very large cut-off, e.g. 100) and displays a scroll-bar if the table gets too broad.

When displaying a table, it is very rare that I do not want to see all columns (showing only the first e.g. 18 rows is usually fine for me). The need for an additional package or wrapper command for showing all columns of a DataFrame should be avoided imho.

[fonsp]

The need for an additional package or wrapper command for showing all columns of a DataFrame should be avoided imho.

Agreed!!

In the meantime, I came up with a more stable way to set the IOContext, and I added it to PlutoUI 0.6.8:

PlutoUI.WithIOContext(df, displaysize=(99999,99999))

read more here: https://github.com/fonsp/Pluto.jl/pull/318#issuecomment-715251330

[fonsp]

Perhaps I haven't written this down but the plan is that Pluto will have built-in support for Tables.jl (the lightweight abstract supertype of DataFrames etc), with lazy loading (#102), scrolling and more.

One of my big goals with Pluto is to be accessible to excel users

[lungben]

Thanks, sounds great!

[fonsp]

I have fixed #542, which means that we can now load in dependencies on the notebook process like Tables.jl

[fonsp]

I have started on building a Table viewer directly into Pluto:

https://github.com/fonsp/Pluto.jl/pull/646