Plotly.py: Showing RGBD images using graph_objects.Surface?

[kmader]

Not an issue exactly, but I am trying to show RGBD images (with Depth) using surface and the 'hack' I found to do it now was to convert the image into 8-bit GIF, use the 8-bit index as the value, color the image by value and use the gif 8-bit palette as the colorscale. Is there an easier way, or should I make a tutorial in case someone else wants to do the same thing?

from PIL import Image
img_as_8bit = lambda x: np.array(Image.fromarray(x).convert('P', palette='WEB', dither=None))
dum_img = Image.fromarray(np.ones((3,3,3), dtype='uint8')).convert('P', palette='WEB')
idx_to_color = np.array(dum_img.getpalette()).reshape((-1, 3))
colorscale=[[i/255.0, "rgb({}, {}, {})".format(*rgb)] for i, rgb in enumerate(idx_to_color)]
trace=go.Surface(
           z=depth_image,
        surfacecolor=img_as_8bit(rgb_image),
        cmin=0, 
        cmax=255,
        colorscale=colorscale
          )
fig = go.Figure(
    data=[trace],
    layout_title_text="3D Face Mesh"
)
fig

[nicolaskruchten]

This is pretty amazing, nicely done! We are, in fact, working on a way to make this much, much easier via a new image trace type. Here's the prototype in Javascript-land, and i'll be available in Python in 4ish weeks: https://codepen.io/antoinerg/pen/OJJPxJK

[nicolaskruchten]

Well, let me reel in my enthusiasm: the image trace type is 2-d not surface ;) Still, this is pretty cool, and we can maybe imagine generalizing surface like this someday.

[kmader]

Thanks, I'll get a public domain RGBD image and try to make a quick notebook out of it and make a PR to one of the examples

[kmader]

Here is a self-contained notebook using public data
https://www.kaggle.com/kmader/show-rgb-d-image-in-3d-plotly

[bsridatta]

Well, let me reel in my enthusiasm: the image trace type is 2-d not surface ;) Still, this is pretty cool, and we can maybe imagine generalizing surface like this someday.

@nicolaskruchten So can we have 2D RGB in a 3D plot say, alongside a scatter plot like in the picture below? I couldn't find a way to do so, I have tried the approach by kmader but looking for a better res image. Thanks