Pluto.jl: WGLMakie support

[MikaelSlevinsky]

I'd like to be able to use Makie.jl inside Pluto.jl. Currently, that seems to cause an error. (Makie is an all-Julia plotting package that is separate from the Plots.jl conglomerate.)

EDIT: This code now runs in a Pluto.jl notebook.

Sample notebook (this is not a minimal failing example, but should help):

using FastTransforms, LinearAlgebra, WGLMakie, AbstractPlotting, JSServe, Random

```julia
import JSServe.DOM

```julia
function fillF!(Ft, F)
    m, n = size(F)
    FN = 0.0
    FS = 0.0
    @inbounds for j = 1:n
        FN += F[1, j]
        FS += F[m, j]
    end
    FN /= n
    FS /= n
    @inbounds for j = 1:n
        for i = 1:m
            Ft[i+1, j] = F[i, j]
        end
        Ft[1, j] = FN
        Ft[m+2, j] = FS
    end
    @inbounds for i = 1:m+2
        Ft[i, n+1] = Ft[i, 1]
    end
end

```julia
n = 256

```julia
begin
    U = zeros(n, 2n-1)
    Ft = zeros(Float64, size(U, 1)+2, size(U, 2)+1)

    P = plan_sph2fourier(U)
    PS = plan_sph_synthesis(U)

    θ = [0;(0.5:n-0.5)/n;1]
    φ = [(0:2n-2)*2/(2n-1);2]
    x = [cospi(φ)*sinpi(θ) for θ in θ, φ in φ]
    y = [sinpi(φ)*sinpi(θ) for θ in θ, φ in φ]
    z = [cospi(θ) for θ in θ, φ in φ]
end

```julia
function color(i)
Random.seed!(i)
fill!(U, 0.0)
ν = n÷4
U[1:ν, 1:2ν-1] = sphrandn(Float64, ν, 2ν-1)/ν
lmul!(P, U)
lmul!(PS, U)
fillF!(Ft, U)
return Ft
end

```julia
begin
    i = JSServe.Slider(1:100)
    clr = map(i) do ret
        return color(i.value.val)
    end
    JSServe.with_session() do s, r
        return DOM.div(i, i.value)
    end
end

```julia
begin
scene = Scene(resolution = (600, 400));
surf = surface!(scene, x, y, z, color = clr, colorrange = (-1.0, 1.0))[end];
update_cam!(scene, Vec3f0(2.5), Vec3f0(0), Vec3f0(0, 0, 1))
scene.center = false
scene
end

~~The output of the last code block issues the following failure *only because of `scene`*:~~
```julia
Failed to show value:

ArgumentError: collection must be non-empty

[email protected]:343[inlined]
|>(::Base.Iterators.Filter{Main.PlutoRunner.var"#17#18"{AbstractPlotting.Scene},Array{MIME,1}}, ::typeof(first))@operators.jl:854
#show_richest#16(::Bool, ::typeof(Main.PlutoRunner.show_richest), ::IOContext{Base.GenericIOBuffer{Array{UInt8,1}}}, ::Any)@PlutoRunner.jl:256
show_richest(::IOContext{Base.GenericIOBuffer{Array{UInt8,1}}}, ::Any)@PlutoRunner.jl:256
#sprint_withreturned#15(::IOContext{Base.PipeEndpoint}, ::Int64, ::typeof(Main.PlutoRunner.sprint_withreturned), ::Function, ::AbstractPlotting.Scene)@PlutoRunner.jl:244
(::Main.PlutoRunner.var"#kw##sprint_withreturned")(::NamedTuple{(:context,),Tuple{IOContext{Base.PipeEndpoint}}}, ::typeof(Main.PlutoRunner.sprint_withreturned), ::Function, ::AbstractPlotting.Scene)@none:0
format_output(::AbstractPlotting.Scene)@PlutoRunner.jl:189
fetch_formatted_result(::Base.UUID, ::Bool)@PlutoRunner.jl:32
top-level [email protected]:0

[fonsp]

It turns out that a Makie scene cannot be displayed as html, svg, png, jpg, bmp, gif nor plaintext, which is the list of MIME types that Pluto can render. So it looks like Makie is currently designed to only work with Jupyter, Juno, etc.

Julia has a very flexible display system, and Pluto executes script tags inside HTML output. Python+Jupyter doesn't have either, which is why many packages (including Julia packages that need to work with Jupyter) hook into specific IDE display systems. To stop this tradition, I don't want to allow packages to hook into Pluto's display system - Julia's built-in system should be good enough!

I'll look into Makie sometime, it should be able to use the HTML mime type 🙃

[karlwessel]

Maybe the WebGL based backend of Makie could be used then (I can't test Makie since my GPU is to old): https://github.com/JuliaPlots/WGLMakie.jl

[MikaelSlevinsky]

Thanks for the reply! I know next to nothing about plotting libraries, so I'll CC @SimonDanisch in case anything needs to be done on the other side regarding displays and display types.

[fonsp]

I think that it's better if I look at this myself and then talk with Simon 👍

[dpsanders]

I tried using WGLMakie. Something seems to appear but in the wrong place.

[dpsanders]

[dpsanders]

It would be fantastic if this worked!

[fonsp]

That's pretty close though!

[fonsp]

@dpsanders Is this how you got to that screenshot? It's not working for me :(

[dpsanders]

Yes, I think that's what I did...

[grero]

I got a simple 3D scatter plot with interaction to work on the latest Pluto version.

[fonsp]

Wow awesome, please share your notebook file! And your Pkg.status() terminal output?

[grero]

OK, here it goes.

### A Pluto.jl notebook ###
# v0.11.1

using Markdown
using InteractiveUtils

# ╔═╡ a871f600-d244-11ea-0dc6-356a2449b02e
let
    using WGLMakie
    using AbstractPlotting
    using MakieLayout
    using Random
    using LinearAlgebra
end

# ╔═╡ 5ece3dc4-d244-11ea-3191-4312c5215d3f
let
    using Pkg
    cd(joinpath(homedir(),"Documents","programming", "julia", "WGLMakieTests"))
    Pkg.activate(".")
end

# ╔═╡ cca11326-d244-11ea-221e-f185a1ae3b4e
let
    scene, layout = layoutscene(resolution=(300,400))
    lscene = layout[1,1] = LScene(scene, scenekw=(projection=cam3d!, raw=false))
    meshscatter!(lscene, rand(100), rand(100), rand(100), markersize=0.05)
    scene
end

# ╔═╡ Cell order:
# ╠═5ece3dc4-d244-11ea-3191-4312c5215d3f
# ╠═a871f600-d244-11ea-0dc6-356a2449b02e
# ╠═cca11326-d244-11ea-221e-f185a1ae3b4e

(WGLMakieTests) pkg> st
Project WGLMakieTests v0.1.0
Status `~/Documents/programming/julia/WGLMakieTests/Project.toml`
  [537997a7] AbstractPlotting v0.11.2
  [ad839575] Blink v0.12.3
  [7073ff75] IJulia v1.21.2
  [824d6782] JSServe v0.6.9
  [16fef848] LiveServer v0.5.1
  [ee78f7c6] Makie v0.11.0
  [dbd62bd0] MakieGallery v0.2.9
  [c3e4b0f8] Pluto v0.11.1
  [276b4fcb] WGLMakie v0.2.6
  [0f1e0344] WebIO v0.8.14

[fonsp]

Here's one that Simon sent me:

[MikaelSlevinsky]

I can confirm that WGLMakie works. Thanks!

It's a little slower than I would've thought given how fast Makie is. Executing scatter(rand(x), resolution=(500, 500)) takes about 13 ms for me (according to @time), but visually takes on the order of a second to load. It also appears to "refresh" the plotted cell output, which visually is jarring and negates the use of a bound variable.

Changing that line to the block begin scene = Scene(resolution = (500, 500)); scatter!(scene, rand(x), resolution=(500,500)) end helps a bit, but still nowhere near an update on the order of a blink of an eye.

[SimonDanisch]

Yeah, that is to be expected...
Makie is a state full plotting library, that isn't supposed to work in a reactive fashion.
Makie's speed comes from using Observable signals that only updates changed values.
E.g:

color = Observable(:red)
scene = scatter(1:4, color=color)
display(scene)
color[] = :green 

Will only sent 4 floats to the frontend, and will directly update those 4 floats on the gpu, making it super efficient.
On the other hand, Pluto does basically this:

color = Observable(:red)
on(color) do new_color
    # gets executed every time color changes
    display(scatter(1:4, color=new_color))
end

color[] = :green 

Which tears down the whole Makie scene, and creates it anew, sending lots of data to the frontend + executing lots of setup code.

There are two ways to workaround this. First of all, you can use JSServe.Slider, as in:

   sl = Slider(1:4)
end
begin 
    scatter(1:4, markersize=sl)
end

Which should make it possible, to update values inside the plot outside Pluto's interaction model - I haven't tried this yet, but I dont see why it should work like that!

On the other hand, I plan to offer a reactive API, that will work like this:

Scatter(rand(4))
end

Which then just creates lightweight plotting objects, that will get constructed over & over again when re-executed, but the display code will not be teared down & reused, and will receive just the values that have changed ;)

[MikaelSlevinsky]

@SimonDanisch, I appreciate that. I managed to get the markdown.jl example up and running (though it could use a minor release because the master markdown.jl link throws an error over a missing Styling). Basically, it would be nice if that reactivity were available from Pluto...

I tried using a JSServe.Slider in a Pluto notebook (if I understood option 1 correctly) but the Slider doesn't appear anywhere, even after constraining the resolution.

[SimonDanisch]

Ah forgot, that by default there is no show overload. You need to do this:

# ╔═╡ f023d66a-d73c-11ea-1480-cd03c63e6f3d
using WGLMakie, AbstractPlotting, JSServe, Colors

# ╔═╡ 0c6c78d6-d73d-11ea-368f-136f2e9b8b06
using JSServe.DOM

# ╔═╡ 12994f68-d73d-11ea-1b06-33d64355b63e
begin
    markersize = JSServe.Slider(LinRange(1, 20, 100))
    JSServe.with_session() do s, r
        return DOM.div(markersize, markersize.value)
    end
end

# ╔═╡ 891b327e-d73f-11ea-3f51-395f011e512f
begin
    hue_slider = JSServe.Slider(LinRange(1, 360, 100))
    color = map(hue_slider) do hue
        HSV(hue, 0.5, 0.5)
    end
    JSServe.with_session() do s, r
        return DOM.div(hue_slider, color)
    end
end

# ╔═╡ 9beb3696-d73d-11ea-0072-a58787d386d3
begin
    positions = rand(Point3f0, 10^6)
    scatter(positions, markersize=markersize, resolution=(500, 500), color=color)
end

And it's all buttery smooth:

There still needs to be a bit of work done, to make this more streamlined, but the steps to make the Pluto integration much nicer should be _fairly_ simple!

[MikaelSlevinsky]

Yep, managed to get the JSServe slider to work. Works great! Updated my script in the original post.

[ambiso]

Weirdly enough, zooming out in the 3D scatter plot only works in Chrome - in Firefox I can only zoom in.
Does someone have a workaround for that? The plots are a lot more responsive in Firefox. I've tried disabling all extensions but that didn't change anything.
I'm not sure if this is a bug in Pluto, Firefox or WGLMakie :smile:

[fonsp]

That's a WGLMakie buggy

[gszep]

I tried copying Simon's example with the same package version numbers and it appears to not be working anymore? I'm using chrome.

### A Pluto.jl notebook ###
# v0.11.1

using Markdown
using InteractiveUtils

# ╔═╡ 72a4f74e-41d8-11eb-1eba-213cda22ef14
begin
    using WGLMakie,Makie
    scatter(randn(55),resolution=(500,500))
end

# ╔═╡ Cell order:
# ╠═72a4f74e-41d8-11eb-1eba-213cda22ef14

Failed to show value:
ArgumentError: collection must be non-empty



md5-3d9e3287d1c321ae2043f743ec39cb0a



  [ee78f7c6] Makie v0.11.0
  [c3e4b0f8] Pluto v0.11.1
  [276b4fcb] WGLMakie v0.2.6

[gszep]

ah but using [537997a7] AbstractPlotting v0.12.9 directly works. So why does the above example work with using AbstractPlotting but not using Makie?