Three.js: [Feature request] Normal map in object space

[Julien-Arlandis]

Description of the feature

Object space is the local 3D space of the object. Up is always up and left is always left. It is a uniform space that you understand really well because that is what the world is really like. Tangent space, on the other hand, is best thought of as surface space. In tangent space, up is always away from the surface (in other words, along the normal).

Apparently, Three.js only supports tangent space for normal space textures, so it will be very fun to supports object space too.

[WestLangley]

Can you please explain your use case -- and need for this feature -- in more detail?

[Ben-Mack]

Yes, Object-space normal map is especially helpful in resolving normal map's seams problems on mesh with non-tilling normal map like game character or 3D scan model. Beside seams, Object-space normal map handle culvatures better, avoid smoothing problems from the low-poly vertex normals.

Please take a look at this post where the use case and pros/cons is explained very details: https://github.com/KhronosGroup/glTF/issues/1284

Cryengine doc also have section called Drawbacks of Tangent Space Lighting: http://docs.cryengine.com/display/SDKDOC4/Tangent+Space+Normal+Mapping

Quoted from a comment in BlenderArtists Tangent vs Object Space vs World Space

Object Space generally produces better results than Tangent Space. Tangent Space is still dependent on the original normals of the model, so if you’ve got weird smoothing, it’ll look weird too with Tangent Space. That’s why you have to add bevels for sharp angles. Object Space isn’t bothered by this at all, so it’ll be cleaner...

[Julien-Arlandis]

Thanks for your answer, I will illustrate my need by an example.
Compare the rendering of the same generated map with normal map in world space coordinates :
-with Sketchfab : https://sketchfab.com/models/387eb203b5fb4349b83fb83beb038e41
-with Three.js : http://pf-01.lab.parisdescartes.fr:1357/mapRender/?ref=savoie
We can see a discontinuity of faces when we use the wrong coordinate system. In my case it's very difficult to generate normal map in space tangent because of a specific geographic modelisation (2.5D).

[bhouston]

Object space normal maps are moderately common in 3D. It would be a useful feature for sure.

[WestLangley]

I thInk those are compelling arguments. I have this working on my fork. I'll file a PR soon.

[WestLangley]

Proposed API for object-space normal map support:

material.normalMap = myNormalMap;
material.objectSpaceNormalMap = true; // default false, implies a tangent-space normal map

Or

material.normalMap = myNormalMap;
material.normalMapType = THREE.ObjectSpaceNormalMapType; // default THREE.TangentSpaceNormalMapType

I prefer the second one, even though there are only two options in this case.

Other options,

material.normalMapType = THREE.ObjectSpaceType; // default THREE.TangentSpaceType

material.normalMapType = THREE.ObjectSpaceNormalMap; // default THREE.TangentSpaceNormalMap

/ping @mrdoob

[Julien-Arlandis]

Thanks for your propose.
I prefer the second option because it allows multiple type supports.

[bhouston]

@WestLangley I think the names should be:

THREE.ObjectSpace
THREE.TangentSpace

The reason is that there are other spaces that could be useful for specification elsewhere and then we do not have to have duplicate constants. Other spaces I can think of are:

THREE.WorldSpace
THREE.ViewSpace / Camera but without perspective transform
THREE.LightSpace
THREE.ScreenSpace
THREE.CustomSpace / you specify a matrix that defines the space.
THREE.TextureSpace / a UV space.

Most of these spaces are basically just matrix selection, except for screen space which includes a perspective transform. I do not think these should be implemented now, but if we follow this naming convention, we could.

I think that having space specifiers is sort of cool. I am sure there are at least a couple more than I am missing.

[WestLangley]

@bhouston

To clarify, I assume you are suggesting a different property name, too:

material.normalMap = myNormalMap;
material.normalMapSpace = THREE.ObjectSpace; // default THREE.TangentSpace

I think that having space specifiers is sort of cool.

I am not sure I see a use case for the others space types...

[WestLangley]

@Ben-Mack Is there an public-domain model we can use in an example?

[Ben-Mack]

@WestLangley I haven't found one, but you can generate it easily in xNormal by uncheck Tangent space in Normal map options

[bhouston]

I am not sure I see a use case for the others space types...

Often you want to apply a offset to an object and you want to specify in which space that offset is done: ViewSpace, ObjectSpace (local space) or WorldSpace. Or if you want to rotate an object, you may want to specify the coordinate system.

Basically these spaces can be used to show a transform widget in a specific coordinate system on an object and then you move it there. There is already some of this in the ThreeJS editor as well as the Clara.io editor. (In the ThreeJS editor remember there is a [ ] local check box that is switching between THREE.WorldSpace and THREE.ObjectSpace for the manipulators.)

Also I like being able to ask a node in the scene graph to give its its matrix in any of these spaces. If you have this type of accessor on the scene graph, implementing manipulators that are flexible in which coordiante system they use becomes easy.

node.getTransform( THREE.ViewSpace, camera );
node.getTransform( THREE.ObjectSpace );
node.getTransform( THREE.WorldSpace );
node.getTransform( THREE.CustomSpace, worldTransform );
node.getTransform( THREE.ParentSpace ); // the node above space.  Sometimes useful.

This is a very useful generalization that other engines do.

[bhouston]

@Ben-Mack can you create a simple example for @WestLangley ? It would be helpful as you know how to use xNormal.

[Ben-Mack]

@WestLangley I've attached zip file with object space normal map, diffuse map and OBJ file based on Ten24 Sample Scan
Object Space Normal map sample model.zip

[Julien-Arlandis]

Thanks for the model.
It's missing mtl file?
So, do you know how to tell the mtl file which geometry was used for the normal map?

[WestLangley]

@mlimper Is your glTF Nefertiti model with the two normal maps available in the public domain for use as a three.js example? It is reasonably-sized, which is a plus.

https://mlimper.github.io/misc/babylonjs-osn-tsn/

[mlimper]

I'll figure this out with the respective people who scanned the asset, will come back shortly

Happy to see object-space normals in Three.JS! :-)

[mlimper]

You can use the model under creative commons for non-commercial purposes (CC BY-NC), please acknowledge the lab that digitized it like this:

3D scan of a copy of the Nefertiti Bust, digitized by Fraunhofer IGD, Competence Center Cultural Heritage Digitization, http://www.cultlab3d.de/

Would be cool to see this as a three.js example!
Many thanks, and keep up the great work!

[WestLangley]

@mlimper Thank you!

@mrdoob We can create a separate object-space normal example using the Nefertiti model, or alternatively, we can use the model and its OS-normal map as part of the glTF example. Which would you prefer?

[donmccurdy]

OS normal maps are technically not "valid" glTF yet, even though it's certainly possible to make examples like this. So ideally this should be separate from the glTF examples for now.

^I'd also be OK with a note:

While glTF specifies tangent-space normals, this model has been modified to demonstrate object-space normals instead.