Makie.jl: More ease-of-use in the interaction API

[Datseris]

This issue requires the subplotting/layouting to be finalized first

As subplots become important, the interaction API that we currently have faces difficulties, see e.g. #238 . Here are some things that I believe will be helpful and will allow much more easier handling of interactive applications:

• [x] which_subscene : a function that returns which subplot the mouse cursor is on.
• [ ] to_subscene : a function that returns the coordinates of the mouse position within the _specific sub scene_ that we care about. This function should not trigger if the mouse is _not_ within the scene limits. This is basically already possible with something like:

to_screen(scene, mpos) = Point2f0(mpos) .- Point2f0(minimum(pixelarea(scene)[]))
mouseposition(scene) = to_world(scene, to_screen(scene, events(scene).mouseposition[]))

scplot = scatter(stuff)
pos = mouseposition(scplot)

• [ ] select_series : when clicking on a specific plot this function automatically returns the plotted series that you clicked on. E.g. you plot 10 timeseries and you click on one of them. This is returned as an index or I don't know how, so that it can be highligheted interactively and many other cool stuff. @SebastianM-C has already achieved this in his code, so please share it here :)

• [ ] Updating textsliders only when the mouse is "un-clicked". Some textsliders may represent parameters that produce an entire pipeline of simulations that may take a couple of seconds. In these cases it is not possible to produce 100s of simulations on the spot. Thus, it makes no sense to "triger" on slider move, but only on slider un-click (i.e. the final value).

• [ ] Selecting rectangle interactively. mouse = to_world(scene, Point2f0(mpos)) allows to get a single cliked point, but I would like to get the 2 corners of a selected rectangle (some how click, drag, unclick gives 2 points (the two corners)).

this post will be updated as we go along, and more things may be added

[SimonDanisch]

Updating textsliders only when the mouse is "un-clicked". Some textsliders may represent parameters that produce an entire pipeline of simulations that may take a couple of seconds. In these cases it is not possible to produce 100s of simulations on the spot. Thus, it makes no sense to "triger" on slider move, but only on slider un-click (i.e. the final value).

Observables.async_latest works pretty well for this already

[Datseris]

Okay, that is good to know! But for all of the things described here to be usable, they have to be documented as well, otherwise me and you and the developers will be the only people that can use them :)

[Datseris]

(I will be helping in documentation by the way, once things stabilize and I also actually know how things work)

[SebastianM-C]

@Datseris Regarding select_series: I'm not sure what would be the elegant / recommended way of doing this. What I did is quite brute force:

function get_series_idx(selected_plot, scene)
    plot_idx = findfirst(map(p->selected_plot === p, scene.plots))
    # println("scatter ", plot_idx)

    plot_idx
end

function change_α(series_alpha, idxs, α=0.005)
    foreach(i-> series_alpha[i][] = α, idxs)
end

function select_series(scene, selected_plot, scatter_α, hist_α, data, hist)
    series_idx = map(get_series_idx, selected_plot, scene)
    on(series_idx) do i
        if !isa(i, Nothing)
            scatter_α[i - 1][] = 1.
            change_α(scatter_α, setdiff(axes(scatter_α, 1), i - 1))
        else
            change_α(scatter_α, axes(scatter_α, 1), 1.)
        end
        return nothing
    end
end

This basically searches scene.plots for the plot that was clicked and changes the transparency of all the other plots to something small to highlight a series. The selected plot is found via

function setup_click(scene, idx=1)
    selection = Node{Any}(0)
    on(scene.events.mousebuttons) do buttons
        if ispressed(scene, Mouse.left) && AbstractPlotting.is_mouseinside(scene)
            plt, click_idx = Makie.mouse_selection(scene)
            selection[] = (plt, click_idx)[idx]
        end
    end
    return selection
end

selected_plot = setup_click(scatter_sc, 1)

The above function can be used to return the selected plot (with idx=1) or the index of the selected point within the clicked plot (with idx=2). The latter is useful when you have only one series and want the index of an element.
The other implementation detail (and the trickiest for me) is controlling the transparency.

data = Scene(resolution=(1000, 1000))
colormap = to_colormap(:viridis, size(sim, 1))
get_color(i) = AbstractPlotting.interpolated_getindex(colormap, colors[i], extrema(colors))
series_alpha = map(eachindex(sim)) do i
    simᵢ = sim[i]
    alpha = Node(1.0)
    if length(sim[i]) ≠ 0
        cmap = lift(α-> RGBAf0.(color.(fill(get_color(i), size(simᵢ, 1))), α), alpha)
        scatter!(data, [Point2f0(simᵢ[i, idxs[1]], simᵢ[i, idxs[2]]) for i in axes(simᵢ, 1)],
        colormap=cmap, color=fill(colors[i], size(simᵢ, 1)), markersize=ms)
    end
    alpha
end

The main idea is to have the alpha of the colors that we want to control as an observable vector. We need an observable for each series / color because they must be updated independently. If we want to map colors to values, then the above implementation is the solution that I've found (with @SimonDanisch 's help). If only colors are needed, then it's much easier to only use the color attribute.
For more details see https://github.com/SebastianM-C/Nuclear-surface-vibrations/blob/ed6838c761a612434839d6c6720d7130a5ffbe52/src/classical/visualizations.jl

I think that having easier access to alpha would make things easier, but I'm not sure what would this imply from an API design point of view.

[asinghvi17]

Something like which_subscene already exists in the form of hovered_scene.

[SebastianM-C]

I think that having some kind of "selection API" would help with this. As pointed out in #619 for example, there are already tools that allow this, but the results depend on the implementation detail of each type of plot.
I think that some of the most common operations would be to know

• what scene did I click / hover. The answer could be both as a scene object or as an index
• what element in the scene did I click / hover. An index would be most useful. See the barplot example in #619

If we can come to an agreement about this, I can try to make a PR.

[Datseris]

+1 and it also fits very well within the existing "clickable" select functionality of the functions select_point, select_rectangle and select_line we already have (they have documentation strings, but I don't know whether the functions are exposed in Makie's main docs. I think they should, I have tested them extensively with InteractiveChaos!)