Plotly.js: Rewrite WebGL code in regl

[etpinard]

The time has come to :hocho: the gl-vis project.

These multiple small modules are becoming harder and harder to maintain. A lot of helper methods are currently duplicated across several modules - which is adding a lot :hamburger: to our bundle size.

@mikolalysenko's new WebGL framework regl solves a lot of long-standing gl-vis issues such as multiple scopes per gl-context and improved memory management.

I placed this issue on the v2.0.0 milestone because it _might be_ a good opportunity to make the switch in-phase with bumping d3 to v4 but this is subject to change.

@dfcreative @monfera @bpostlethwaite

[dy]

More I get into gl-vis code more I realize that it is reasonable in terms of size/solutions.
Regl is awesome webgl API wrapper, but I would guess code of gl-vis components does not add much overhead compared to regl, they reuse lots of external packages. Also some packages have nothing to do with regl. For example, text-cache by itself adds 150kb of compressed code (~15%) to the final bundle, and regl is not going to help much with it.

[monfera]

I agree text rendering with WebGL has diverse compromises depending on the method, tesselation (vectorize-text), texture or SDF methods. I did some benchmarks and at least for the 2D plots I believe we can just overlay a gl context with a plain old 2d canvas layer. It's not difficult to keep their projection in sync.

In the 3D case, the applicability of 2D canvas for text is more of a challenge in that some of the text needs to be presented properly as part of the 3D space, i.e. possibly occluded by things in the 3D scene. Perhaps even for such cases, it's possible to do canvas text by analyzing whether a feature of interest (to which the text label belongs) is visible or not. Such analysis is possible because this is how picking works already. So even for 3D, if it's satisfactory to switch text labels on/off depending on the visibility of their anchors, text can be done with 2D canvas.

Similar to this consideration, there should ideally be more shared code between SVG and WebGL parts, for example, using the same exact contour model for both. Also, we can reuse some of the SVG elements for the accelerated substrates; e.g. why not render 2D plot axes, tooltips (as now), legends etc. in SVG, as there's no performance benefit to rendering in WebGL just for a few elements. Such unification alone can lead to other significant code size decrease and closer or perfect feature parity. WebGL layers can be used where truly needed, i.e. where the expected element count is at least in the thousands (e.g. lines, point markers, perhaps error bars - though I'd be more in favor of resampling resolutions because thousands of error bars are undecipherable anyway).

Also I'd expect some code collapse resulting from better sharing across things that are currently in different gl-vis repos that happen not to be factored out as modules. For example, much of the 2d projection logic is shared both in JavaScript and in GLSL.

[monfera]

P.S. gl-scatter2d-fancy currently also uses vectorize-text for calculating marker tessellation (markers are modeled as characters). Other approaches might be followed, e.g. something like this, except with much simpler shape generation instead of the interesting but expensive and limited 'superformula' thing: http://bl.ocks.org/monfera/85aa9627de1ae521d3ac5b26c9cd1c49

[monfera]

P.P.S. having said that, I agree that regl is also of non-trivial size, and that gl-vis has very nice properties, and even has certain benefits, for example, more direct control over what happens in WebGL. regl is a much higher level library and has its own set of autoconversions and heuristics in order to be permissive with diverse user input (typed vs untyped array; 2D arrays represented as array of arrays vs flat ndarray style etc.). One key benefit with regl is that, I think, it's more straightforward to maintain a set of renderers with it, whereas now there's a lot of juggling with each renderer as they're in separate repos (shared sentiment is that it's a bit overly granular for the plotly use case).

[dy]

I believe we can just overlay a gl context with a plain old 2d canvas layer.

Funny, that is exactly what I did in webgl plot-grid after consideration of _text-cache_ size and complexity of rendering lines in webgl, especially h/v lies, which are the worst case for antialiasing.
_Canvas2D_ turns out can simply be a texture for webgl, I suspect it is even faster than feeding data to GPU from js.

[monfera]

Nice! Also, good call on the gridlines, they come and go anyway when you zoom/pan, and you never need a lot of them at any given moment, otherwise they lose their ergonomics. I.e. the simplest thing is to draw them with Canvas dynamically according to the ever current visible domain. Re text, my recent experiment showed that 1000 words can be individually animated at high speed (60FPS in Chrome / Safari, 40 in FF). We'll probably never need 1000 words in one frame, esp. at 60FPS, even a dense full-page chart would have maybe a few hundred short texts (mostly, numbers), otherwise there's undesirable overlap (or reversing it, one would cull text labels to avoid overlap before trying to render too much text). For reference I took the 1000 words from https://hipsum.co/ :-)

(To clarify, I literally meant superimposing a Canvas layer, not even transferring it to a WebGL texture)

[dy]

For the notice https://github.com/dfcreative/regl-scatter2d

[etpinard]

scattergl v2 wishlist:

• plot 1e6 points in 1000ms
• fast updates (~ 100ms)
• reuse SVG cartesian axis/subplot logic and renderers
• share 1 gl context across all subplots on given gd (or even better 1 gl context per page)
• per-point support for marker size, color, symbol, opacity and line width, line color

[monfera]

Should we consider trellised, SPLOM and small multiple usages early in the design? The panels may or may not have identical projections or shared dimensions. IOW if it's designed with multi-panel use, it'll still work fine with a single panel, but the opposite is not necessarily true :-)

The more the panels, the more likely it is that some dimensions are shared, so it's useful to share model attributes across tiles to minimize GPU memory transfer time, already a serious latency component with singular panels. IOW it's a bit like parcoords which supports up to 64 dimensions, and this SPLOM concept shares dimension data, the code is 99% parcoords as is (ofc scattergl-fancy styling needs more attributes which reduces the available dimension count but eg. programs can be switched).

[dy]

@monfera are there best practices of sharing model attributes? I can think of providing dataBox/viewBox properties or sharing pre-created regl.buffer with data.

[monfera]

@dfcreative with parcoords I just put all dimensions (up to 64; client need was up to 50) into the attributes and the only switching is between the regular vertex shader and the pick vertex shader

• also, the dimension values were normalized to a domain of [0, 1] so that we minimize numerical precision issues with matrix operations bound to arise when matrix elements are of wildly different OOM, but its purpose was simply to let matrix based GPU crossfiltering work in WebGL

[etpinard]

Also @dfcreative we should make sure that all items in https://github.com/plotly/plotly.js/milestone/3 are :white_check_mark: in gl2d v2.

[alexcjohnson]

Along with this transition - or perhaps even better before it - we should find a way to test webgl plots in iOS - perhaps using https://www.browserstack.com/screenshots/api ? We need to make sure that bugs we've fixed before - such as #280, #1868 (any others that are iOS-specific?) - don't come back with this rewrite.

[dy]

regl-error2d, faster and w/o memory limitations. Should say working with regl is pure pleasure.

[etpinard]

scattergl was converted to regl in https://github.com/plotly/plotly.js/pull/2258

pointcloud should be next.

---

Dropping from the v2 milestone as this work can be in the v1.x series.

[sgentile]

please support one context for many charts as we are hitting limitations when using newest scattergl with many charts (1.33.x)

[longjohnhard]

Same issue, we are hitting this limit in Chrome and FF very fast. IE renders 30+ webgl charts easily. Hope this can be fixed soon. regards

[etpinard]

@sgentile @longjohnhard please subscribe to https://github.com/plotly/plotly.js/issues/2333 for the latest news on that front.

[vincex]

Hello guys, maybe i found an issue.
On a scattergl with multiple subplots if you show/hide a trace the gl context is not cleared before the redraw.
This issue seems realted to a check inside plot_api.js at line 212:

function drawFramework() {
...
...
    if(!fullLayout._glcanvas && fullLayout._has('gl')) {
            fullLayout._glcanvas = fullLayout._glcontainer.selectAll('.gl-canvas').data([{

the data for each gl-canvas is recreated in any case even if there is already a data with a regl reference.
This leads to the clearGlCanvases that does nothing if regl is not defined.

module.exports = function clearGlCanvases(gd) {
    var fullLayout = gd._fullLayout;

    if(fullLayout._glcanvas && fullLayout._glcanvas.size()) {
        fullLayout._glcanvas.each(function(d) {
            if(d.regl) d.regl.clear({color: true, depth: true});
        });
    }
};

I hope this helps.

[etpinard]

@vincex can you open a separate issue with a fully reproducible example. Thank you!

[vincex]

https://github.com/plotly/plotly.js/issues/2791

[jackparmer]

This issue has been tagged with NEEDS SPON$OR

A community PR for this feature would certainly be welcome, but our experience is deeper features like this are difficult to complete without the Plotly maintainers leading the effort.

Sponsorship range: $150k-$160k

What Sponsorship includes:

• Completion of this feature to the Sponsor's satisfaction, in a manner coherent with the rest of the Plotly.js library and API
• Tests for this feature
• Long-term support (continued support of this feature in the latest version of Plotly.js)
• Documentation at plotly.com/javascript
• Possibility of integrating this feature with Plotly Graphing Libraries (Python, R, F#, Julia, MATLAB, etc)
• Possibility of integrating this feature with Dash
• Feature announcement on community.plotly.com with shout out to Sponsor (or can remain anonymous)
• Gratification of advancing the world's most downloaded, interactive scientific graphing libraries (>50M downloads across supported languages)

Please include the link to this issue when contacting us to discuss.