Plotly.js: v2.0.0 wishlist

[etpinard]

_Leave this text box for planned improvements_

• [ ] upgrade d3 to v4.0.0 (now v5) - should be mostly a matter of updating the general update pattern.
• [ ] remove es-promise polyfill from bundle - and thus make IE users add their own Promise polyfill.
• [ ] drop jQuery event support
• [ ] Make components (e.g. Annotations, Shapes, RangeSlider, ...) register-able and remove them from the core bundle.
• [ ] revamp set of default colorscales
• [ ] make config arguments consistent https://github.com/plotly/plotly.js/issues/839
• [ ] remove config.plot3dPixelRatio map for backward compatibility
• [ ] Incorporate /remove all unofficial exposed methods (i.e Plotly.Plots, Plotly.Fx, Plotly.Snapshot, Plotly.PlotSchema, Plotly.Queue)
• [ ] drop event-based Plotly.Snapshot.toImage and merge src/snapshot in plot_api/to_image.js
• [ ] remove (some) components for lib/core.js - see https://github.com/plotly/plotly.js/pull/845
• [ ] drop Plotly.relayout handlers for the 'remove' and 'add' special values - see https://github.com/plotly/plotly.js/pull/1086
• [ ] for log axes, have layout express everything in data units instead of linearized units. This applies to range, tick0, and annotation and image positions.
• [ ] remove support for dates provided as epoch milliseconds
• [ ] remove the distinction between axis name and axis id (ie xaxis2 vs x2) probably by only using the name.
• [ ] rename current gl marker symbol 'cross' -> 'cross-thin' (i.e. revert https://github.com/plotly/plotly.js/pull/1482/commits/c0eb06599db481f2137ed97e79bab5cb2bc31351)

[etpinard]

On my list:

• Incorporate /remove all _unofficial_ exposed methods (i.e Plotly.Plots, Plotly.Fx, Plotly.Snapshot, Plotly.PlotSchema, Plotly.Queue)
• Use _proper_ state object instead of storing state in graph DOM element.
• Add OO API e.g :

var plot = Plotly.createPlot('graph', data, layout, config);

plot.restyle(/* */);
plot.relayout(/* */);
plot.resize();
// ...

• Drop jQuery event support
• Better, more-consistent events data
• Better, more-consistent config options

[mdtusz]

Ooooooh boy.

• use only d3 modules we need (fingers crossed)
• standardize interfaces across _all_ traces/plots
• svg/gl parity
• no DOM state (virtual dom + diffing would be awesomeeeee)
• discrete process/render modules.
• immutable _plot_ state (i.e. the returned myPlot contains methods, plotstate, and history, but it's current plotstate is always overwritten, and _old_ state is pushed onto a history list)
• es6 - it's definitely mature enough, and there's a lot of niceties that we could take advantage of
• agreed upon standard lib/prelude

_edit: April 14_

• logging utils that can be set to either verbose or silent (part of our prelude perhaps). Done

_edit: April 22_

• slimmed down interface. We could likely reduce the method count to 3-5, namely a constructor Plotly.plot, and instance methods Plot.update, Plot.export and maybe a few others that don't fit as an update.

_edit: April 26_

• tooling to verify public interfaces meet standards, e.g. checking that all attribute names are snake_case or camelCase (depending on what we choose)

_edit: May 3rd_

• all positioning in normalized units (margins specifically)

[monfera]

Besides agreeing with the above two lists, based on very limited work, take it with huge grain of salt:

1. D3 might be used in some places; e.g. category scales/axes could be done via the ordinal scale even if mapping is made to logical points (e.g. the index of the axis tick); sometimes two D3 scales are linked (range of one is domain of the other). Might not work out but D3 scales are pretty powerful, especially in the upcoming D3 4.0.
2. Smaller, and to the extent possible, pure functions, immutability as the default.
3. (long term) Thinking about reactive visualization, e.g. streaming in new data points into a plot; it would smoothly transition to a possibly enlarged x/y domain; similar behavior with controls. Maybe it's mostly covered, I haven't seen enough. The reason for mentioning it is, it chimes with pure functions and immutability in that it's easier to base dynamic behavior atop of less imperative code.
4. lodash/fp to replace typical, repeating code patterns, it's a painstakingly optimized library, it isn't just using slow [].map etc.
5. I saw that some Jasmine test require PlotlyInternal (rather than Plotly); maybe worth using the public API for all tests, but again, maybe PlotlyInternal is much like an API except some of it is not (yet) user-exposed.

[mdtusz]

To add to @monfera's testing comments - I'm personally not a fan of tests that just test the internal workings of code - they end up being quite brittle and often only ensure that code remains the way it was originally written. I'd love to see tests specific to plotting _only_ use the public API and inspect the returned object - if we no longer keep state in the DOM, this will be infinitely easier as well and rendering tests can be handled separately using mocks (and obviously, maintain unit tests for well specified functions) . As well - without the need for a real browser DOM, we'd be able to run tests (excluding image tests) headless!

[cpsievert]

Add OO API e.g :

This seems like a good place to discuss smooth transitions as well (requested in #142, and experimented with in https://github.com/ropensci/plotly/pull/547). It'd be awesome if plot worked like a d3 selection wrt transitions:

  .transition({duration: 100, ease: 'cubic'})
  .restyle('marker.color', 'red')

Also, in my mind, transitioning _positions_ (x/y) is the most important use case, so it'd be super useful if restyle() (or some other method) could also support this:

  .transition({duration: 100, ease: 'linear'})
  .restyle('scatter', {x: [1, 2], y: [1, 2]})

There are multiple ways to transition a path, but I think I'd almost always want to transition the transform.

[meglio]

• Please allow callbacks for labels, text, values etc, so that their number can be specified and a callback provided to retrieve values, labels, text, hover text etc by index (passed to callback as a parameter).
• Let us build hover text with a callback by index, and allow any HTML in the hover text

[cpsievert]

all positioning in normalized units (margins specifically)

+💯

[etpinard]

Remove our es6-promise polyfill and ask IE9, IE8, IE11 (ref) to use their own Promise polyfills.

[etpinard]

Revamp our set of default colorscales.

[monfera]

⬆️ there is palette overlap between plotly.js and some of its webgl renderers, with a different resolution. Slight restructuring would make room for one shared palette set definition at a finer than current resolution.

[etpinard]

Drop 'add' and 'remove' relayout values from adding and removing annotations and shapes.

Use e.g:

// to add a blank annotation:
Plotly.relayout(gd, 'annotations[2]', {});

// to remove a shape:
Plotly.relayout(gd, 'shapes[2]', null);

[etpinard]

Maybe we should drop Plotly.redraw in v2.0.0 ?

[rreusser]

@etpinard 👍 for removing unofficial exposed methods.

[alexcjohnson]

• Specify log axes with data values, not linearized values. ie range [0.1, 100] rather than [-1, 2]. For dates we're doing this in #1078 but I can't see a backward-compatible way to do it for logs. This also applies to annotation and image positions (but shapes already get it right!) and tick0. Perhaps in 2.0 we also remove the compatibility transform that #1078 introduces for dates that supports the old range-as-epoch-milliseconds format.
• Along the lines of "Drop 'add' and 'remove' relayout values" above, drop the built-in coordinate transforms when an annotation or shape has its axis references changed. This is really just for GUI editing so should reside in streambed.

[alexcjohnson]

• rename all the axis objects 'x', 'x2', 'y2' etc. instead of 'xaxis', 'xaxis2', 'yaxis2' so you can go straight from xref/yref to the axis object and we can get rid of Axes.id2name and name2id

In principle this allows us to also change log axis ranges to data values backward-compatibly, since you can't make a log axis without explicitly specifying the container. The only downside I see to this is attribute names are a bit overloaded: trace.x, annotation.x, scene.domain.x etc are data values or arrays of data values, but layout.x and scene.x are axis objects. Doesn't seem too confusing to me but it should be discussed.

[etpinard]

rename all the axis objects 'x', 'x2', 'y2' etc. instead of 'xaxis', 'xaxis2', 'yaxis2'

I'd vote for the opposite. Rename all axis ids 'xaxis', 'xaxis2', 'yaxis2', ...

[mdtusz]

I'd vote for not having "names" that require parsing at all - put them as indexed elements in a list.

[alexcjohnson]

I'd vote for the opposite. Rename all axis ids 'xaxis', 'xaxis2', 'yaxis2', ...

That could work. It gives us the same simplification, clears up the ambiguity, and is a smaller change for users to adapt to. Means we have to do something more clever to manage backward compatibility but maybe that's unavoidable.

I'd vote for not having "names" that require parsing at all - put them as indexed elements in a list.

Like layout.xaxes=[{...}, {...}, {...}]? two things bother me with that idea:

• If you delete an axis, all your references to later axes need to change
• It violates the idea of "simple things should be simple" - almost all plots have only one x axis, and making everyone put that axis into a list just because some other people will make multiple axes is annoying. And then scenes, ternaries, geos that can only have one axis of each type, either the structure is different or they also have that extra list cruft.

[etpinard]

Maybe we should drop support for colorscales in histogram traces. See https://github.com/plotly/plotly.js/issues/1495 and https://github.com/plotly/plotly.js/pull/1500

[finger563]

Regarding the removal of support for specifying date as epoch time in ms, is there no desire to support plots which specify time as epoch nanoseconds? Seeing as I have just such plots I'd be very interested to know :)

[alexcjohnson]

is there no desire to support plots which specify time as epoch nanoseconds?

The precision we have right now - 100 microseconds, one digit better than native dates support - is the best I found I was able to reliably handle using a single javascript number to specify the coordinate. I could imagine a system supporting arbitrary precision within a restricted range where the zero point of the axis linearization is allowed to float - this would be a fairly big project but could be done if someone was sufficiently interested in sponsoring it ;)

Note that this wouldn't imply any backward incompatibility, so need not be associated with v2

[etpinard]

We should take some time thinking about the config argument during the v2 push. For example,

• should Plotly.newPlot reset the context, like Plotly.purge does?
• should we allow config values to be updated via Plotly.udpate and/or should we add a granular Plotly.reconfig method?
• should we move the config values logic to a more supply-default-esque framwork as https://github.com/plotly/plotly.js/pull/1188 attempted?
• it would be nice to declare config options as we do for data and layout attributes and output them in the plot-schema - see #1188
• ...

[alexcjohnson]

• drop the undo/redo queues that we have built into restyle and relayout and our other API methods - we used these in our old workspace, but this is really not the right level to be managing this issue, as plots may be coupled to other application state (ie changes in data arrays) and using our queue would muck up that correspondence.

[alexcjohnson]

• drop the strange functionality relayout(gd, {'xaxis.reverse': true}) and change the handling of axis.autorange: 'reversed' from a weird disappearing setting to regular behavior where autorange: true always results in an increasing range and autorange: 'reversed' is always reversed. Would also require the doubleclick interactions to choose whether to set autorange to true or 'reversed' in order to behave the way they do today, but editing behavior would be more intuitive.

[alexcjohnson]

• drop errorbar array and arrayminus attributes in favor of arrayOk versions of value and valueminus

[alexcjohnson]

• drop errorbar traceref and tracerefminus - which are already unused within plotly.js but may still be used by the old workspace.

[alexcjohnson]

• are there any of the "special" restyle/relayout keys we need to keep or can we 🔪 all of them? In addition to axis.reverse mentioned above, there's orientationaxes, swapxy, and swapxyaxes here - seems like these should be dropped and just managed by editing apps.
• along with this, orientation has special handling to swap the x/y attributes. We should probably drop this special handling too, and just let the user/editor do the swapping they want.

from https://github.com/plotly/plotly.js/pull/1999#issuecomment-329318166

[etpinard]

• Rename (+ cleanLayout map) geo.projection.scale :arrow_right: geo.zoom (as in mapbox subplots)
• Rename (+ cleanData map) choropleth z :arrow_right: values (similar to pie traces)
• Make geo subplot respect the scrollZoom config option #143. Unfortunately. geo assumes scrollZoom: true, but the scrollZoom default is faise. done
• Replace PlotlyGeoAssets global with some field on Plotly. No need to have another global variables.

from https://github.com/plotly/plotly.js/pull/2030

[alexcjohnson]

from @etpinard above:

Make geo subplot respect the scrollZoom config option #143

and from #2041:

@etpinard - I think in v2 all subplot types should obey scrollZoom. Moreover, I think should add a corresponding layout attribute as people might want to turn on/off scroll interactions via a relayout call.

@rreusser - Agreed. Including gl3d. I actually rather dislike scrollZoom being a config parameter. I find embedplot, for one, suddenly has dramatic scroll interactions I never saw when developing the plot.

We definitely need saner management of scrollZoom. The current behavior grew accidentally but does have a sort of logic to it:

• scrollZoom was originally made a config option because you want to be able to avoid it in scrolling environments. For example a plotly plot in a blog post or news story: you scroll down quickly and suddenly discover that page scrolling has stopped and you're looking at a mostly blank plot, because you've accidentally zoomed way out before even looking at it. This is why plot.ly embeds have scrollZoom: false but the same plot displayed in a non-scrolling environment (such as plot.ly shareplot) have it enabled.
• This works OK for some subplot types (2D cartesian, ternary) that have other options for zooming (drag box and dragging axis ends). But 3D and maps only zoom with the scroll wheel, so it's a big hindrance to turn it off. So 3D we explicitly chose to exempt from scrollZoom: false, not sure if it was an explicit decision with maps or not but it fits that logic.

So what would "saner management" look like, given these constraints? One proposal (incorporating some of the above ideas):

• A layout attribute that defaults to false unless you have one of the subplot types that can only be zoomed this way, but that lets creators choose, and applies to all subplots of all types (per-subplot control seems confusing and unnecessary).
• A modebar button so viewers can enable/disable it too.
• Some pages (particularly plot.ly embed) will still want to actively suppress scroll zoom, but it's really only the initial scrollZoom setting they want to control - if the viewer has explicitly turned scrollzoom on they won't be surprised that it happens as they scroll past. So this wouldn't need to be a config parameter, these pages could just override the layout setting before the initial render.

[alexcjohnson]

To add some more detail to "better, more consistent event data":

• pie events were straight out of calcdata before. #2117 fixed a lot of that but left v and i in place for backward compatibility - remove them.
• sankey events are also still straight out of calcdata, so probably also aren't very standardized.
• there are almost certainly others. We need to develop a very clear spec of what event data should look like, independent of trace/subplot type, and make sure it's implemented consistently everywhere.

[szechyjs]

I would love to see autosizing fixed so that it works in flexbox or any dynamically sized containers. If a container's width/height is auto the plot size always defaults to 700x450px. This then requires the use of resize events and Plots.resize() to get things to behave. It would be nice to refactor the svg plotting to just be sized to 100% of the container instead of using explicit pixels values.

[etpinard]

Related to @szechyjs' https://github.com/plotly/plotly.js/issues/420#issuecomment-345755199: https://github.com/plotly/plotly.js/issues/106

[alexcjohnson]

• more visible and well-documented way to set logging level (than Plotly.setPlotConfig) - @etpinard suggests a top-level method for it like Plotly.setLogLevel

[alexcjohnson]

• remove layout.separators and only support locales (#2207)

[alexcjohnson]

• Let all traces - even those marked visible: false - increment default colors. Currently we have a kind of strange situation where traces that the user marks visible: false do NOT increment the default color (it's like they don't exist at all) but traces that plotly invalidates (eg for lacking data arrays) DO increment the default color. Seems kind of weird to hide and show traces and have that effect the color of other traces. See https://github.com/plotly/plotly.js/pull/2227#discussion_r159031147

[alexcjohnson]

• remove the plotly_selected event with undefined data that gets emitted when clicking on the plot area in a select mode - see #2241

[ancapdev]

On the topic of timestamps, in response to @alexcjohnson

The precision we have right now - 100 microseconds, one digit better than native dates support - is the best I found I was able to reliably handle using a single javascript number to specify the coordinate. I could imagine a system supporting arbitrary precision within a restricted range where the zero point of the axis linearization is allowed to float - this would be a fairly big project but could be done if someone was sufficiently interested in sponsoring it ;)

Is it possible to do something simpler for dateless timestamps? Time of day to nanosecond precision requires 47 bits and fits comfortably in double. It would be quite useful for financial time series on intraday tick data.

[alexcjohnson]

• change showlegend to legend.visible in line with most of our other containers. So the rule would be: if attributes are grouped in a container describing an object, the container should have a visible attribute inside, but visibility of features without their own containers continue to get show* attributes. (As mentioned in #2341)

[etpinard]

Suggestion: now that Plotly.react got implemented in https://github.com/plotly/plotly.js/pull/2341, perhaps we start thinking about removing :hocho: Plotly.plot, Plotly.redraw, Plotly.restyle, Plotly.relayout, Plotly.update and Plotly.moveTraces entirely from the library make Plotly.react the only official way to update an existing plotly graph? The restyle call signature has always been hard to grasp for new plotly.js users. As Plotly.react has the small call signature as Plotly.newPlot, this should be an improvement.

We could probably remove Plotly.extendTraces and Plotly.preprendTraces at some point too, but it might best to keep them and turn them into fast(er) :racehorse: routine optimize for streaming data.

Just a thought. Maybe some users _really_ like Plotly.restyle and Plotly.relayout ...

[etpinard]

• Add support for float64 typed arrays coordinates on date axes (in ms) - see commit https://github.com/plotly/plotly.js/pull/2388/commits/6dd2f69e1489982646a36fc488e9b23b5f5f315d off PR #2388

[etpinard]

• [ ] Make axis.automargin true by default - as proposed in https://github.com/plotly/plotly.js/pull/2243#issuecomment-356988681

[acats-dkolev]

Add TypeScript definitions types with every new Plotly.js release and link to NPM @types repo
https://www.npmjs.com/package/@types/plotly.js

[etpinard]

@acats-dkolev that's not a breaking change. You can follow https://github.com/plotly/plotly.js/issues/28 for the latest development info

[alexcjohnson]

• make parcoords tick label format default to the same as cartesian axes, rather than a fixed tickformat - see #2468

[bpostlethwaite]

Continuing from @etpinard comment https://github.com/plotly/plotly.js/issues/420#issuecomment-364454752

Instead of deprecating Plotly.plot rename Plotly.react to Plotly.plot. react may be a somewhat unfortunate name for a method that has the "whole plot specification" of Plotly.plot and the performance of Plotly.update. Also it is highly useful for those using JQuery, Angular, Vue etc.

[bytesnz]

My wishlist is around making the graphs more dynamic and responsive:

• Better settings for axis placement (see https://community.plot.ly/t/multiple-axis-smooshing/9621, https://jsfiddle.net/t2y3yaa3/ and https://github.com/plotly/plotly.js/pull/2243)
• autosizing and responsive resizing built in (mentioned above)

[etpinard]

Expanding on

Make components (e.g. Annotations, Shapes, RangeSlider, ...) register-able and remove them from the core bundle.

ErrorBars should also be made optional. To do so, we'll need to add another flavor of Registry.getComponentMethod that returns false if the component in question isn't registered.

[alexcjohnson]

• Change the default layout.hovermode from 'x' (compare) to 'closest'. In #778 we discussed setting restrictions on when cartesian plots could default to compare mode (we already separate out non-cartesian and horizontal traces here) but this is both difficult to do (requires digging into data arrays during supplyDefaults) and potentially error-prone, and can still be a source of confusion (see clickmode #1852, spikelines). Seems better just to default to 'closest' and make compare modes explicitly opt-in.

[alexcjohnson]

• Automatic positioning of legend and colorbar(s) so they don't overlap (see #2880)

[dmt0]

To add to the ancient discussion about axis names. Subplot naming for non-cartesian could be made more consistent. I.e. have scene and geo renamed to subplot like the rest. Perhaps gl3d to scene.

[alexcjohnson]

Proposal from @etpinard, came up during stacked area #2960:
Make 'fill' part of the mode flaglist, then the fill value just determines how to fill, not whether to fill.

• :hocho: fill: 'none'
• add 'fill' flag in scatter mode
• add that 'fill' flag the mode default whenever stackgroup is set
• coerce scatter fill (that maybe we should call fills similar to the markers and lines attribute containers) only if 'fill' is part of mode
• make tonexty the only valid fills value when stackgroup is set (tonextx for horizontal stacks). Lib.log something if some other value is set.

I might even go one step further, now that we have orientation (which is so far just used by the stackgroup, and if we did this it would still only have one value within a stack group but outside that it would be a per-trace attribute), we could 🔪 the 'x' and 'y' suffixes, determine that from orientation, and just let fill take values 'tozero', 'tonext', and 'toself'

[etpinard]

PR https://github.com/plotly/plotly.js/pull/2944 added layout.clickmode with default value 'event' which fires plotly_click event when clicking on data points. In v2, we could consider making 'select' or 'event+select' the default enabling click-to-select by default.

[alexcjohnson]

• Remove ALL auto* attributes: if a value is needed and not provided, that's enough to indicate that it should be determined automatically, just like all other defaults. This might be too broad a statement, but I think it's mostly the right goal. There are a few bits of functionality we will need to find a new home for, like autorange: 'reversed'
• And related to this but perhaps not exactly congruent, NO mutations to gd.data or gd.layout during Plotly.plot. This requirement has been hinted at elsewhere in this issue but I don't think it was explicitly stated.

3044 may serve as a model for this kind of behavior, using _private attributes to stash values determined during calc rather than supplyDefaults

[etpinard]

From https://github.com/plotly/plotly.js/issues/3248

• Stop emitting plotly_selected (with event data undefined) when clicking on graphs under dragmode 'select' or 'lasso.

https://github.com/plotly/plotly.js/blob/3a32ac00cf5e8619932f874960f11b4daf7ceb40/src/plots/cartesian/select.js#L258-L262

[jackparmer]

We're going to do this in (at least) 4 major releases next year instead of one massive major release. Will have a lot to celebrate by next year's holiday party!

Roughly:

1. [ ] v2.0 New styling defaults (colorway, reduce margin size, left-align title, etc) Jan-Feb 2019
2. [ ] v3.0 Separate traces and data March-June 2019
3. [ ] v4.0 @etpinard 's and AJ's lists of clean up items, plus OO API July - Sept 2019
   https://github.com/plotly/plotly.js/issues/420#issue-148094498
   https://github.com/plotly/plotly.js/issues/420#issuecomment-209507399
   (+ others in this issue, mostly related to events/callback support)
4. [ ] v5.0 Rewrite stack.gl 3d modules in regl, with 3d point selection and tweening transitions in mind Throughout 2019 with Q4 release target

Giving these items plenty of time so that the Chart Studio, Dash, and Plotly.py teams have time to adjust to each major change.

//cc @jonmmease @etpinard @alexcjohnson @nicolaskruchten @chriddyp @bpostlethwaite

[etpinard]

From https://github.com/plotly/plotly.js/issues/3450#issuecomment-455693468 (and cc https://github.com/plotly/plotly.js/pull/3553):

Agreed that the end state should be no text attributes that only ever appear on hover. But we should look at the traces that currently have no on-graph text and figure out which ones we can imagine adding it to in the future, before actually removing the text attribute (in a major bump).

[jasonleonhard]

Any plans to support react-native? I found a third party dev who build this: https://github.com/rynobax/react-native-plotly any thoughts?

[nite]

We're going to do this in (at least) 4 major releases next year instead of one massive major release. Will have a lot to celebrate by next year's holiday party!

Roughly:

1. [ ] v2.0 New styling defaults (colorway, reduce margin size, left-align title, etc) _Jan-Feb 2019_
2. [ ] v3.0 Separate traces and data _March-June 2019_
   3.

Hi - is this still active? Is there an updated roadmap/release date for 2.0.0? Is there RC/branch we could try out?

[jackparmer]

@nite sorry for the lack of update in this issue. We chose to prioritize new chart types (https://github.com/plotly/plotly.js/issues/2221) this year. We're still searching for a large company or consortium of companies to sponsor a v2 plotly.js API: https://plot.ly/products/consulting-and-oem/