Plotly.js: raster trace type: matplotlib imshow equivalent

[nicolaskruchten]

Could/should we have some trace type that would be equivalent to matplotlib's imshow() for showing full-color images? https://matplotlib.org/api/_as_gen/matplotlib.pyplot.imshow.html

basically heatmap but with r,g,b channels or something?

[nicolaskruchten]

cc @jonmmease @emmanuelle

[nicolaskruchten]

I feel like this would help Python adoption, as this comes up every once in a while.

[nicolaskruchten]

Thinking out loud here... imshow accepts either a single-channel vector, or RGB or RGBA, but we in principle could also accept HSV or other color-space specifications. This would allow us to do some cool combined-colormap type stuff where hue is mapped to one vector and lightness to another.

[etpinard]

related: https://github.com/plotly/plotly.js/issues/332

[nicolaskruchten]

this kind of thing https://en.wikipedia.org/wiki/Domain_coloring

[alexcjohnson]

I'm for it - I really like the idea of being able to specify 3 (or 2 or 4) 2D arrays and specifying whether they're RGB, HSV, HSL, or whatever. I had some instances in grad school of multiple independent 2D channels of data I wanted to overlay... never did find a mapping that was intelligible for 3 channels but for 2 channels either R+G, R+B, or L+H worked fairly well in my case.

[emmanuelle]

It would be great, yes!

[emmanuelle]

RGB channels for a start would be enough for 99% cases I think

[nicolaskruchten]

The third example on this page shows how a rastermapbox related trace type might work: http://datashader.org/topics/gerrymandering.html

[YoniChechik]

@nicolaskruchten it would be so useful to computer vision in python! hope to see it soon. Thanks!

[antoinerg]

it would be so useful to computer vision in python! hope to see it soon. Thanks!

@YoniChechik for your use cases are there additional features you would find valuable in this new trace type?

[YoniChechik]

@antoinerg , I think that a good line to follow is matplotlib.pyplot.imshow.
Key functions not to be missed IMHO are:

• labeling (data on hover)
• aspect ratio! it's very hard to set with bokeh the original image aspect ratio, or "equal" ratio, if needed.
• [0,0] origin: matlab and matplotlib both uses the origin as upper-left, which has since became standard.
• 2D and 3D (or 4D) input: 2D input should be like heatmap, with easy ability to change color limit min and max (default is data min and max). 3D/4D (RGB/A) input should be default to np.uint8 ([0,255]) or other types which will be in range [0,1].
• input array: must be able to take np.array and list of lists.

[nicolaskruchten]

Initial discussion with team:

• single input: z which is a "2-d array of 3- or 4-tuples"
• colorspace: "rgb" and "rgba" to start, eventually can be extended to e.g. "hsl" etc ... autodetect the 4 element to be alpha?
• colorspace bounds: 0-255 or 0-1 (independent setting for alpha? e.g. rgb in 0-255 and alpha in 0-1?)
• clipping and scaling all channels together, via zmin/zmax
• no colorbar
• similarly to funnel: for subplots that contain only/any image trace types, yaxis.autorange = 'reversed'
• in a future version we'll add selection to both image and heatmap
• in a future version we'll add interpolation
• in a future version we could fetch image data from a url, and/or provide it as base64

[nicolaskruchten]

Nice job @antoinerg ! Can you please create a new "image future directions" issue that captures all the stuff that didn't make it into this version?