Three.js: Rendering issue with normals and DoubleSided materials on some Adreno GPU series

[ignazkevenaar]

Description of the problem

Hi, I'm working with a small team on implementing a THREE scene with glTF and PBR/IBL.
It all works great except when testing on specific mobile devices we've come across a weird rendering issue.

It is easily reproducable (though only on some hardware!) by opening the MeshStandardMaterial example and setting the material.side to THREE.DoubleSide.

Steps to reproduce:

• Connect Android device using USB debugging and set up Chrome Remote Device Debugging
• Navigate to the MeshStandardMaterial example
• Switch to the viewer iframe in the console.
• Setting the material to doublesided using scene.children[1].children[0].material.side = THREE.DoubleSide , scene.children[1].children[0].material.needsUpdate = true

The models will appear as follows:

The gun model from the examples

The model that caused the initial problem

Is there anything we can do to provide more info? Anything blatantly obvious we've overlooked? Thank you in advance.

Some things of note

• Lighting the material using an envMap is required. Both LDR and HDR envMaps are affected.
• The affected devices (as far as we were able to test!) have a Qualcomm Adreno 500 series GPU
• When removing the normalMap, the material renders correctly (roughness, metallic) albeit without the normal map affecting the lighting ;)

Three.js version

• [ ] Dev
• [x] r101
• [ ] ...

Browser

• [ ] All of them
• [x] Chrome
• [x] Firefox
• [ ] Internet Explorer

OS

• [ ] All of them
• [ ] Windows
• [ ] macOS
• [ ] Linux
• [x] Android
• [ ] iOS

Hardware Requirements (graphics card, VR Device, ...)

Android Device with Qualcomm Snapdragon SOC with an Adreno 500 series GPU: We've tested on 505, 506 and 530

[Mugen87]

In general, certain mobile devices do not support highp but only mediump. This precision is not sufficient for lighting calculations in three.js materials right now. #14570 tracks this issue but a solution is hard to implement and test.

Can you please verify with one of your problematic devices how the following demo renders:

https://jsfiddle.net/15r8372z/show

Do you see a moire pattern on the geometries?

[ignazkevenaar]

Thanks for the reply.
The fiddle renders identically on both affected and non-affected devices. Here is a screenshot of the render of an affected device.

webglreport.com reports:
float/int precision: highp/highp

It's very interesting that the material renders 'correctly' as long as it has no normal map.

[Mugen87]

Um, okay. If the devices actually support highp, there might be still a problem related to the shader precision. We had once an issue where certain features related to structs did not work with highp but only with mediump, see #14137.

It's very interesting that the material renders 'correctly' as long as it has no normal map.

The lighting equations related to normal maps use derivate functions (dFdx and dFdy) which might be one possible source of the problem. Can you please test on one of your devices whether the following test is successfully executed or not?

https://www.khronos.org/registry/webgl/sdk/tests/conformance/extensions/oes-standard-derivatives.html?webglVersion=1&quiet=0&quick=1

[WestLangley]

@Mugen87 The OP is demonstrating DoubleSide is an issue here. We need to understand why DoubleSide causes a problem.

@donmccurdy I wonder if the reintroduction of tangent support can be used to solve problems with low-precision mobile by avoiding the use of screen-space derivatives.

[Mugen87]

@WestLangley I just try to find out if it's a precision related issue.

[WestLangley]

@Mugen87 That's cool. :-)

[donmccurdy]

@ignazkevenaar what tool was used to create the second model? Do you know if it can write a tangent attribute? The Blender 2.8 exporter can do this automatically, with an option on mesh export. If so, it may be worth checking against latest threejs dev branch.

[ignazkevenaar]

@Mugen87 The provided test passes

@donmccurdy The model was exported from Reallusion iClone Character Creator 3 as an FBX, then converted using FBX2glTF with flags --embed and --pbr-metallic-roughness. iClone CC doesn't give me the option to enable or disable exporting a tangent attribute. I've tried flagging FBX2glTF to explicitly keep the tangent attribute, but the exported FBX doesn't seem to have it to begin with as it is missing from the .gltf...

[donmccurdy]

Ok, thanks. FBX does support tangents, but I have no idea whether that particular tool would include them – most likely not, as you say.

In the meantime I don't have a way to test whether the tangent attribute will solve this particular issue, but I've filed a feature request (https://github.com/AnalyticalGraphicsInc/gltf-pipeline/issues/460) for an easier way to add MikktSpace tangents to a glTF file.

[Mugen87]

@ignazkevenaar On what Android version are the mentioned devices? We had once an issue that was introduced by a system upgrade which also upgrades the graphics driver.

Just to clarify things: Just using an environment map works, right? But adding the normal map is problematic? Do you also see the glitches with just normal map (so without an env map)?

[ignazkevenaar]

@Mugen87 The android versions are 7.1.2, 8.0.0 and '9'

We've done some more digging and I have good news and bad news...

My initial diagnosis of the problem was wrong. Good thing you asked!
I've looked again and we also see a difference with just the normal map, just not as severe.

That got me thinking, the highlights are where the shadows should be.
I've then tried to invert the z-axis - or blue channel - of the normal map (from 1 to 0) and view the scene again on a working device. Would you look at that!

Here is the kicker: It renders perfectly on "affected" devices now!
It seems that a select group (Adreno 500?) of devices interpret the z-axis as inverted.

[WestLangley]

I've then tried to invert the z-axis - or blue channel - of the normal map (from 1 to 0) and view the scene
It renders perfectly on "affected" devices now!
It seems that a select group (Adreno 500?) of devices interpret the z-axis as inverted.

It seems so, but you are not sure?

So the problem is device specific? And occurs with double side only?

[Mugen87]

It seems that a select group (Adreno 500?) of devices interpret the z-axis as inverted.

I would really like to know why this happens 🤔

[ignazkevenaar]

I've decided to do some more debugging and finally found out that the thing that causes the issue is gl_FrontFacing. It is used to flip the XY direction of the backface normal maps in normalmap_pars_fragment.glsl.js: mapN.xy *= ( float( gl_FrontFacing ) * 2.0 - 1.0 );
Commenting this line out results in inverted rear normal mapping (duh) but none of the "metal-like" problems on Adreno devices.

In fact - at least on my phone - using gl_FrontFacing at all causes issues, like so: bool unusedBool = gl_FrontFacing;. Without ever using or referencing that variable after its assignment.

I've looked at an alternative to gl_FrontFacing but due to my inexperience and the complexity of the meshphysical shaders, I wasn't able to get those to work.

[dghez]

Update here @WestLangley & @Mugen87

Same issue here on r114 - Oneplus 7 - GPU Adreno 640 - Chrome 80.0.39 - MeshStandardMaterial

Info:

• I'm exporting from blender and I'm experiencing the same issue, when I disabled the normalMap everything starts working again.

• I confirm that the bug comes from the doubleSide, if I set frontSide only everything works.

• I tried to export with tangents enabled

• I tried fixing with material.normalMapType = ObjectSpaceNormalMap, I get better result but far away from how it should be

Temporary solution:

BufferGeometryUtils.computeTangents(el.geometry)
el.material.vertexTangents = true

Hope this can help.

Here a screenshot with on the left - FrontFace, and on the right a DoubleSide

[WestLangley]

@dghez said

Same issue here

What is the issue, exactly? What are we supposed to observe in your screenshot?

Are you using MirroredRepeatWrapping?

[dghez]

Hey @WestLangley , sorry if I wasn't totally clear.

If you look at the screenshot you can see 2 armours, one is GOLD (that's how it should look) and one is SILVER, both with the same envMap and the same normalMap.
The only difference between them is that gold one has side: FrontSide and the silver one has side: DoubleSide, that is exactly the same issue as ignazkevenaar described.

Regarding the MirroredRepeatWrapping tbh I don't know, I didn't check.

[mrdoob]

Temporary solution:

BufferGeometryUtils.computeTangents(el.geometry)
el.material.vertexTangents = true

That may not be temporary...

[JordyvanDortmont]

@dghez We're currently using that solution as well in production specifically targeting Adreno GPUs.

var gpuHasFrontFacingDoubleSidedBug = false;
var debugInfo = renderer.getContext().getExtension('WEBGL_debug_renderer_info');
if (debugInfo !== null)
{
    var gpu = renderer.getContext().getParameter(debugInfo.UNMASKED_RENDERER_WEBGL);
    gpuHasFrontFacingDoubleSidedBug = gpu.match(/adreno.+(5|6)[0-9][0-9]/gi) !== null;
}

scene.traverse(function(node)
{
    if (node.material.side === THREE.DoubleSide && gpuHasFrontFacingDoubleSidedBug)
    {
        THREE.BufferGeometryUtils.computeTangents(node.geometry);
        node.material.vertexTangents = true;
        node.material.needsUpdate = true;
    }
}

Hope this helps!

[dghez]

@mrdoob and @JordyvanDortmont yeah, to me is fine to use that, no problem.
My post was more on "hey this is still happening, here are more info about the issue to help you solve it" and give a prod-ready solution to everyone will land in this issue.

Anyway, thanks both.

[arodic]

It appears this is still happening. I'm noticing this issue on double sided materials (Pixel 3 phone)

[Senglean]

@arodic I have some rendering issues on Snapdragon 845 and 855 too .. 865 seems to fix the issue