Babylon.js: Material SideOrientation/Normal flipped during glb export

[CadsoftBrianW]

Bugs

• Bug repro on playground:
  https://www.babylonjs-playground.com/#95MJI8#22
• Expected result: Material orientation is correct
• Current result: Material orientation is incorrect

Forum:
https://forum.babylonjs.com/t/material-sideorientation-normal-flipped-during-glb-export/2046

[deltakosh]

This one if for @drigax

[Drigax]

Looks like we currently only reverse the winding when exporting a mesh in a Left Hand coordinate system in Babylon with Clockwise triangle winding to GLTF, which is strictly Right Hand Coordinate System with Counterclockwise triangle winding.

That should explain why this only reproduces when we setup the scene with right hand coordinate system, as our materials are setup to have clockwise side orientation by default.

I believe that changing our exporter to reverse the triangle winding when we are either in a left hand coordinate system, or are working with a mesh with material indicating clockwise side orientation should fix our issue, so that we're always reversing the mesh winding when needed.

[Drigax]

After my first iteration, I realized that the coordinate system handedness shouldn't be relevant to how we handle reversing the mesh winding. Babylon applies an inverting transformation to imported GLTF files, and applies a similar transformation when exporting to GLTF in order to convert to a RightHand coordinate system.

However, we didn't previously account for this transform when exporting, so adding an additional check to only flip Clockwise->Counterclockwise when in a positive world space.

Making the following change appears to fix our problem.

EDIT 4/5/2019
After some intense debugging with @bghgary , we found that we don't need to check the world determinant's sign either. Removing the previous check that only rewound the meshes for Left hand coordinate systems was all that was required.