Three.js: Feature: Add transmissionFactor to Materials

[gkjohnson]

When rendering a transparent object in THREE the specular highlight fades along with the diffuse surface of the material. I'd expect it retain the same brightness considering specular is supposed to be more of a mirror like reflection. Here's a picture of a sphere with the Standard material in THREE:

metalness: 0, roughness: 0.3, opacity: 0.15

And here's what a similar sphere looks like in Unity, which is more what I'd expect:

metallic: 0, smoothness: 0.8, opacity: 11 / 255

As an aside I'm a bit surprised by the difference in darkness of the transparent objects between THREE and Unity. The THREE material has an opacity of 0.15 while the Unity material has an opacity of 0.043 (11 / 255) and yet they look the same. Switching Unity's color space from Linear to Gamma makes the results look more similar with the same with the same opacity. Is this a limitation of blending in the current WebGL APIs?

Thanks!

[WestLangley]

See #5810, #8245, #8255.

[gkjohnson]

Thanks! I figured this had been discussed previously but I couldn't find where -- it looks like https://github.com/mrdoob/three.js/pull/8276 is the last PR in the "premultiplied alpha" saga.

I may be missing something but is there a reason that it isn't enabled by default? From reading those issues it sounded like that's what was going to happen.

[gkjohnson]

Edit Disregard this comment I realized that I needed to set Material.needsUpdate to true in order for premultipliedAlpha to fully take effect.

[gkjohnson]

Sorry for jumping around a bit -- I think I understand a bit more of what's going on now and I see that setting premultipliedAlpha to true does make things look more correct. However it's still surprising to me that the opacity of the material affects the specular highlight instead of just the diffuse color. This means that even when premultipliedAlpha is true if the material has an opacity of 0 then nothing will render.

In Unity, though, the specular highlight is unaffected by the opacity. See a comparison here of a material with 0 and 100% opacity:

I see that the alpha multiplication happens as the very last step of the shader, meaning the specular (and every other effect) is already included in the color that is multiplied by the alpha. Instead shouldn't only the diffuse component be multiplied by the opacity? For example, the phong shader might look like this with no further alpha multiplication.

vec3 outgoingLight =
    reflectedLight.directDiffuse * diffuseColor.a +
    reflectedLight.indirectDiffuse * diffuseColor.a +
    reflectedLight.directSpecular + reflectedLight.indirectSpecular + totalEmissiveRadiance;

// ...

gl_FragColor = vec4( outgoingLight, diffuseColor.a );

Additionally it looks like the emissive value is also getting diluted by the materials opacity which doesn't feel right, either.

[mrdoob]

@bhouston do you have any thoughts?

[donmccurdy]

I don't have an immediate opinion on this, but will just throw in that glTF currently doesn't specify anything here, but probably should. (see https://github.com/KhronosGroup/glTF/issues/1457).

[bhouston]

In Unity, though, the specular highlight is unaffected by the opacity. See a comparison here of a material with 0 and 100% opacity:

Opacity should affect the specular highlight. Opacity is usually understood as a properly of the whole material, not of the base layer or the top layer of a layered material. Thus I believe Unity here is wrong.

[bhouston]

Additionally it looks like the emissive value is also getting diluted by the materials opacity which doesn't feel right, either.

This one I am not sure of. What do other tools do that are better designed than Unity? V-Ray, Arnold, Unreal Engine.

(I am a bit biased against Unity because Unity had some weird material behaviors from years ago which they never fix because of the need for backwards compatibility.)

[bhouston]

This means that even when premultipliedAlpha is true if the material has an opacity of 0 then nothing will render.

That is the correct behavior. A fully transparent material should not be seen. I am concerned that you have a very specific definition of opacity (opacity means opacity of diffuse layer only) that differs from how most others understand it.

[WestLangley]

@bhouston I do not yet know what the best model is, but check out https://www.babylonjs-playground.com/#19JGPR#13. I set glass.alpha = 0.1;

The shader appears to be increasing the material opacity by a quantity akin to the luminosity of the specular highlight. I don't know where that model comes from.

[bhouston]

I do not take game engines as good material models, they are usually hacked in. I would look at Disney's Physical Model from Burley, or V-Ray or Arnold -- the guys who specialize in rendering and correctness.

V-Ray defines opacity as "Opacity – Specifies how opaque or transparent the material is. A texture map can be assigned to this channel." https://docs.chaosgroup.com/display/VRAYRHINO/VRay+Material+%7C+VRayBRDF#VRayMaterial|VRayBRDF-Reflection

The reason is that if you want to fade out an object, you want to fade out all aspects of an object. To do that you modify opacity. In V-Ray if you want to make the base layer transparent you modify "refraction" or if you want light to leak into the diffuse layer you set "translucency."

In Arnold they use the term "transmission" and this refers to the base layer transparency: https://docs.arnoldrenderer.com/display/A5AFMUG/Transmission

Arnold discusses "Opacity" and "Transmission" in a way that I also think of it: "Opacity will also cut out the shape completely whereas Transmission will leave the reflections/speculars visible even on areas that are completely transparent."

https://docs.arnoldrenderer.com/display/A5AFMUG/Transmission+And+Opacity

So both V-Ray and Arnold view opacity as affecting both specular and diffuse layers and use other terms to refer to diffuse layer specific opacity (refraction for V-Ray, transmission for Arnold.)

[gkjohnson]

@bhouston

I am concerned that you have a very specific definition of opacity (opacity means opacity of diffuse layer only) that differs from how most others understand it.

That may be true and if that's the case I'm happy to be educated.

@WestLangley

The shader appears to be increasing the material opacity by a quantity akin to the luminosity of the specular highlight.

This is more or less what I expect -- whether or not it's correct is still what I'm trying to understand. Setting the alpha to 0 still results in a bright reflection.

@bhouston

So both V-Ray and Arnold view opacity as affecting both specular and diffuse layers and use other terms to refer to diffuse layer specific opacity (refraction for V-Ray, transmission for Arnold.)

So is this just a terminology issue and the lighting behavior I'm looking for (and that @WestLangley's image illustrates) is not necessarily incorrect? I'm not doubting the utility of a parameter that fades the whole material.

As another data point, Maya refers to this as "transparency" and indicates that it should _not_ affect the specular:

Note: If the material has specular highlights the transparency setting do not affect the highlights. So if you are trying to make an object disappear by animating the transparency attribute, you may also have to animate the specular highlight attributes.

And with the terminology in mind Unity never refers to the alpha field as "opacity". Instead it provides an option to make the material "transparent" or "fade". The "fade" model is what THREE seems to do now with premultipliedAlpha = true. "Transparent" does not seem achieveable unless I'm missing something.

So maybe this issue should be rephrased as "do THREE materials need a 'transmission' or 'transparency' parameter to enable bright specular highlights on transparent materials?"

Thanks for the references!

[mrdoob]

"Transparent" does not seem achieveable unless I'm missing something.

Isn't that what premultipliedAlpha = true produces?

[bhouston]

@gkjohnson wrote:

"Transparent" does not seem achieveable unless I'm missing something.

You are correct that the current ThreeJS material model does not allow one to make the base diffuse layer transparent independently of the specular layer. If we added transmission/transparency it would allow that.

@mrdoob wrote:

Isn't that what premultipliedAlpha = true produces?

It seems like it has that affect, but premultipliedAlpha only ensures that we apply opacity in the shader so that we can apply it before RGB are clamped. What happens in the GPU pipeline is that the RGBA values output from the shader are first clamped to the range of 0-1 by the hardware prior to applying the hardware blending ops (which in the case of non-premultiplied is the RGB are multiplied by A.) This hardware clamping of the RGBA outputs of the shader prior to the application of the blend ops can not be changed. In the case of non-premultipliedAlpha this artificially reduces the brightness of RGB when A is < 1. Basically if you do not premultiply, the brightest RGB can be is the value of A. This is incorrect.

So premultipliedAlpha just ensures that we do not get artificially clamped RGB range on bright transparent highlights. It does not allow for the base layer to be transparent/transmissive.

[bhouston]

So just to sum up, the solution is to add a "transmission" color to the material model to achieve this affect. I believe this is both the solution suggested by the best of bread renderers (Arnold + V-Ray) and it also is fully backwards compatible, unlike changing what opacity does.

[WestLangley]

I was typing this simultaneously, but I'll post it anyway... :-)

---

@mrdoob material.premultipliedAlpha = true indicates that the output of the shader should have the RGB channels premultiplied by the alpha channel.

To compensate, the bending function is modified so that normal alpha blending is still achieved. In other words, the rendered output should theoretically be the same as when material.premultipliedAlpha = false.

So, why do it?

The reason is the output of the shader is clamped to [ 0, 1 ]. So if it were not for the clamping, there would be no difference in the two approaches.

If the radiance of the hotspot is greater than 1, and the alpha is small, it is better to premultiply-before-clamping.

[mrdoob]

I see... Looks like I misunderstood the feature 😇

So, either we add transmission to MeshPhysicalMaterial or we wait to see what GLTF decides?

[donmccurdy]

Regardless of whether glTF alpha affects specular, its alpha is not meant to approximate physical transparency.

There's a fair chance physical transparency will come in a more advanced PBR extension in the future, and if so I would expect transmission to be included separately from alpha – Adobe's extension for their own web viewer is doing the same: https://github.com/KhronosGroup/glTF/blob/master/extensions/2.0/Vendor/ADOBE_materials_thin_transparency/README.md

I think adding transmission as a distinct thing from opacity, perhaps just on MeshPhysicalMaterial, would be reasonable.

[gkjohnson]

I agree that there's a lot more needed in order to properly model a transparent object with refraction and tinted transmitted light than can be achieved with just a single shader pass. But this should make roughly approximating something like clear glass a bit simpler.

So just to sum up, the solution is to add a "transmission" color to the material model to achieve this affect.

Should the transmission field be a color instead of just a float value? The ADOBE_materials_thin_transparency that @donmccurdy linked defines a transmissionFactor value and assumes the existing baseColor of the material defines how light is absorbed as it is transmitted.

[looeee]

I think adding transmission as a distinct thing from opacity, perhaps just on MeshPhysicalMaterial, would be reasonable.

Is there a reason why this wouldn't be added to MeshStandardMaterial as well?

[donmccurdy]

Is there a reason why this wouldn't be added to MeshStandardMaterial as well?

We have refractionRatio on both, so I guess I don't see why transmission couldn't go on both, sure.

Should the transmission field be a color instead of just a float value?

It's a float in Adobe's proposal and Blender's Principled BSDF node, so I'd lean that way, but are there other examples?

[bhouston]

I know that V-Ray transmission is a color but I am having a hard time justifying that flexibility. So let's go with "transmissionFactor" as a singular float that affects the opacity of the diffuse layer. This is a straight forward extension to our model and is fully backwards compatible and also compatible with the future direction of glTF. I really like it.

[bhouston]

Is there a reason why this wouldn't be added to MeshStandardMaterial as well?

It could easily go on both.

[bhouston]

If I was to design the ultimate transmission model for Three.JS it would be this:

• transmissionFactor float (what we are adding today, uses baseColor as the absorption color)
• transmissionIOR float (clear than the current "refractionRatio")
• transmissionScatter float
• transmissionDispersionAbbe float

[gkjohnson]

It looks like #17114 added this so I think the original intent of this issue has been addressed. Maybe this should stay open until #16996 is added, though?

[WestLangley]

In light of #17114, I think this issue can be closed.