Plotly.js: mesh3d: Different lighting results with different hardware

[jackparmer]

Consider this codepen:

http://codepen.io/monfera/full/ggByRW

On my 12 in Macbook, I see this:

@etpinard 's Thinkpad and @monfera 's Macbook show this:

Another example - consider this codepen:

http://codepen.io/monfera/pen/XjxZjZ

Result on my 12 in Macbook:

Result on larger Macbooks and Thinkpad:

---

I tried this in Chrome, Safari, and Firefox with the same result.

Computer specs:

Chrome specs:

[rreusser]

Hmm… looks like possible flipped normals to me. I'd suggest maybe gl_FrontFacing isn't properly supported, but 2016 seems pretty recent to fail on that, and reading the stack overflow issue more closely… not sure that's it.

See: https://github.com/gl-vis/gl-surface3d/blob/master/shaders/fragment.glsl#L28-L30

http://stackoverflow.com/questions/24375171/is-there-a-reliable-alternative-to-gl-frontfacing-in-a-fragment-shader

[etpinard]

cc @dfcreative which have insight about this.

[monfera]

I don't have access to this machine but guessing is always fun:

• perhaps the gl floats are lowp - I think the standard is quite permissive about precision
• or we're using an extension that silently causes an issue

One thing to try: visiting a bunch of WebGL content on the web to see if those work properly.

[jackparmer]

Quick update: It doesn’t seem to matter what value I set for diffuse or specular. I can set them to 0.1 or 1 or 1000 and nothing changes.

[monfera]

Do these work fine? http://webglsamples.org/

[rreusser]

Gotchas like pow(-abs(x), 2.0) being inconsistent across platforms bit me a couple times, but I didn't happen to catch anything obvious with a quick glance.

[jackparmer]

Do these work fine? http://webglsamples.org/

Yeah, nothing looks "off" with those. I can also change the diffuse and specular settings here:

http://multivis.net/lecture/phong.html

[monfera]

Not that it should inherently make it fail, but mesh3d uses the Cook-Torrance model. E.g. here one can choose the reflection model in a dropdown: http://www.calebjohnston.com/webgl.html

[jackparmer]

Seems to work too - I see this?

[rreusser]

Oops, sorry. I had the wrong module.

Maybe………(though this is necessarily positive, isn't it?)? https://github.com/stackgl/glsl-specular-cook-torrance/blob/master/index.glsl#L26

[monfera]

One more thing - possibly relating to the above possibility (lower precision floats): I needed to tweak the EPSILON value in the normals calculation. The brain mesh is made up of _very_ small triangles. With the wrong EPSILON value, whole swaths of lighting just don't happen. This is my current best guess.

Way to test it: load some coarser mesh3d content from eg. plot.ly and see if _that_ works fine.

Also, my old commits made EPSILON configurable from within Plotly: https://github.com/mikolalysenko/normals/commit/a519d8a0fe38cb1db798e5e10a3d7f8cd96fa1b6#diff-f02129f250a223ce2d22f0b6e4fa7297

[jackparmer]

Hm, yeah, larger triangles _seem OK_:

https://plot.ly/~empet/13481/delaunay-triangulation-and-alpha-shapes-for-a-set-of-3d-points-1-delaunay-triang/

https://plot.ly/~RPlotBot/3008/moebius-band-triangulation/

(I found thesemesh3d plot.ly creations via: https://plot.ly/feed/?q=plottype:mesh3d)

[jackparmer]

Changing diffuse and specular still doesn't seem to have an effect though:

I can actually see visible changes with diffuse when ambient is turned down, I don't see any changes with specular.

[monfera]

I added the epsilon attribs to the codepen, with their respective default values: http://codepen.io/monfera/pen/XjxZjZ

Could you test if lower values make it work? E.g. 1e-9 instead of 1e-6, or 1e-15 instead of 1e-12. Or some much more ambitious change.

[jackparmer]

Epsilon attribs don't seem to effect anything on my machine - I still get the dark and foggy brain no matter how I set them:

[jackparmer]

Another example on a trisurf plot with less vertices:

https://plot.ly/~monfera/19

Robert sees this shiny helicopter:

I see this non-shiny midnight rider:

[jackparmer]

Here are some more brain images that might be useful in debugging. One shows specular reflection (is "shiny"). Another is an odd mix of shiny and unshiny.

Shiny:

Odd mix of shiny + unshiny:

Dark:

Brains.zip

[rreusser]

@jackparmer what does this look like for you?

https://rawgit.com/rreusser/057e5c31845192528cf62ba075540853/raw/index.html

It's just a simple sphere that uses some of the same basic lighting functions (cook-torrance and diffuse-lambert). It looks like this for me:

Source here

[jackparmer]

Looks a bit different, like this:

[rreusser]

Ah, sorry. I rotated it in the code at the last minute. That's a 👍 from me on being the same. Not that I thought it would be different, but just a quick sanity check.

[dy]

@jackparmer Ricky is right, gl_FrontFacing is broken here.

We can use that workaround, considering the 99% support of OES_standard_derivatives extension. There is even glsl-face-normal package for that purpose.
Here is the PR, which should fix that.

Also we may want to consider replacing gl_FrontFacing in all @gl-vis deps.

[etpinard]

From https://github.com/plotly/plotly.js/pull/1423#issuecomment-287411731 :

---

After digging into the behavior quite a bit with @alexcjohnson, here's what we conclude:

The current behavior mostly make sense. The really weird stuff is all in the light positioning.

• the magnitudes have to be really giant. Are they pixels or something?
• z exhibits really weird behavior and thresholds

Apart from how it does work, here's how we think it should work:

The light should be positioned effectively in clip coordinates with the addition of an aspect ratio. Conceptual test cases:

• a light position of [1, 0, 0] should be just on the right side of the screen (at the depth of the center, that is. the projection matrix makes this statement otherwise meaningless). It involves an (inverse?) view matrix somewhere in there, I think.
• a light position of [10, 0, 0] would be roughly speaking 10 half-screens worth off to the right (again, at the depth of/relative to the center point with respect to the alignment of the eye)
• a light position of [0, 1, 0] should be at the top of the screen. If the plot is not square, then one or the other of these will need to take into account an aspect ratio.
• a light position of [1, 1, 0] should be at 45 degrees NE regardless of the plot's aspect ratio. This constraint probably determines where the aspect ratio goes
• a light position of [0, 0, 1] should be more or less at the eye position, unless I'm wrong about scales. At the very least in the direction of the eye. Out of the screen instead of into because of the righthand rule.
• a light position of [0, 0, 10] should be way behind the camera
• If we can, it'd be great of length(light) > 1 implied the light tended not to fall inside the screen (I'm not 100% convinced this can be satisfied subject to the other constraints)

Note: scene aspect ratio and axis ranges seem to scale things entirely independently of the view, so I think they can be ignored. Open to feedback, but I think these constraints are enough to minimally fix the parts of the current behavior that are surprising without changing anything substantial. In talking with @alexcjohnson one conclusion is that existing behavior like the weird 1.00002 -> 1.00003 z threshold is best just fixed so that it's probably best not to preserve strict backward compatibility.

[jackparmer]

Hey @dfcreative I just ran into this issue again. Did https://github.com/gl-vis/gl-mesh3d/pull/8/ fix the dark mesh3d shading issue? I can bring in the laptop where this shows up if helpful.

[jackparmer]

One things that is weird, I don't see any shading issues in this gl-mesh3d example:
https://gl-vis.github.io/gl-isosurface3d/brainbrowser.html
I would expect to see the same issue as the plotly.js mesh3d brains.

[monfera]

Just a minor note, these last two images aren't too different; both seem to be primarily or exclusively lit by ambient light; it appears that the darker one simply has a lower light level. A high ambient level is usually not good because it "washes away" detail and reduces plasticity, as can be seen on the brainbrowser example. Neither seem to have eg. the specular component, compare to this:

In short, I think the dark brain above wants to be like what's seen in the codepen example, but misses components other than the (justly) subdued ambient component. I go by the look so I may well be wrong.

[jackparmer]

@monfera The issue is that the same mesh looks completely different depending on the viewer's computer. Here's another example: https://plot.ly/~empet/14759/. This mesh look completely different on my laptop versus my phone. These are the exact same plot (no lighting as been adjusted).

This is a huge bug IMO. Plotly meshes shouldn't look completely different depending on what computer you're using.

iPhone:

Laptop:

[dy]

We have that fixed in gl-mesh3d https://github.com/gl-vis/gl-mesh3d/pull/8, it takes a separate effort meeting the requirements here though

[jackparmer]

AFAICT it's a GPU issue. The MBP where Plotly.js meshes appear dark has an Intel 515 GPU, but the exact same MBP (same model, OS, and Chrome) with an Intel 5300 GPU displays meshes with "normal" lighting.

The Plotly Cloud image server also renders Plotly.js meshes overly dark, wonder what GPU it's using ATM:

[jackparmer]

Note: @nicolaskruchten also sees this with an _Intel Iris_ GPU on his MBP

[nicolaskruchten]

Here are my specs, for the record

[nicolaskruchten]

FWIW the first few links now look great on my machine :)

[nicolaskruchten]

(ah, but they look fine with 1.35.0 also... What's an updated test case I can try?)

[etpinard]

FWIW the first few links now look great on my machine :)

Strange. I haven't updated the CDN links yet.

[nicolaskruchten]

Sorry, too quick on the draw! I had initially seen these issues in a Dash app that showed a brain. Not sure what an easy-to-access test case is. Happy to check on anything pre/post if you like on my machine.

[etpinard]

https://cdn.plot.ly/plotly-1.36.0.min.js is now up!