Three.js: The shadow of a partially transparent object

[zhoushijie163]

Description of the problem

The shadow of a partially transparent object should be brighter than that of a completely opaque object, but they're same dark in threejs.

Three.js version

• [x] Dev
• [X] r83
• [ ] ...

Browser

• [x] All of them
• [ ] Chrome
• [X] Firefox
• [ ] Internet Explorer

OS

• [x] All of them
• [ ] Windows
• [X] Linux
• [ ] Android
• [ ] IOS

Hardware Requirements (graphics card, VR Device, ...)

graphics card is GTX 1080.

[mrdoob]

That's a limitation of the shadow technique we use. I'm not aware of (real time) shadow techniques that support this.

[mrdoob]

Does Unity or Unreal support this?

[zhoushijie163]

blend4web looks like support this.

[mrdoob]

Interesting... Do you have a test you can share?

[RemusMar]

The shadow of a partially transparent object should be brighter than that of a completely opaque object, but they're same dark in threejs.

This is not a viable option at runtime (significant processing power is required).
There are a few hacks and workarounds.
The best one seems to be the Unreal Engine one:

[tentone]

I may be wrong but it may be achievable using a auxiliar map to store translucency of the objects present in the shadow (depth) map. This approach would make shadows dependent on the material applied to the object, (this would allow to easly cast shadows of transparent textures).

[mrdoob]

How do you store into a depth map the information of 2 transparent objects on top of each other? 😁

[tentone]

Only an accumulated transparency of the objects is necessary, we can update these values ​​on the "translucency" map before discarding the fragments.

EDIT: I am only talking about translucency information for shadow casting, im not talking about multi-object transparency, that would require sorting or n-maps or similar to what i am suggesting above a weight system for order independent transparency.

[RemusMar]

I may be wrong but it may be achievable using a auxiliar map to store translucency of the objects present in the shadow (depth) map.

In the above Unreal Engine screenshot is used a transluicent material.
But as I said before, the runtime results (performances) are not very good.
A better approach is the Translucent Shadows (for static lights only):
https://docs.unrealengine.com/latest/INT/Engine/Rendering/Materials/HowTo/ColoredTransluscentShadows/

[MasterJames]

Yes Please, plus one to this Feature Request.
If you fade an object in and/or out it's only logical that the shadow also fades in realtime.
It is pretty obvious if not blaring to the user. It just looks bad to the point of unprofessionalism.

I guess it feels like it should be a given, step one for shadows. Also sounds like another (missing) optimizer feature [ FPS-monitoring to automatic disable performance bottlenecks on a per case situation. Like automatically setting shadow map density based on camera distance (via performance) would hopefully be possibly built-in (or an injected plugin) one day.]

[gkjohnson]

I had a recent interest in transparent shadows, as well. It looks like Unity uses a dither approach to rendering shadows for transparent objects, which obviously has some artifacts but doesn't look too bad when soft shadows are enabled:

This should work in real time and be performant, as well. Generally it might be useful to have dither transparency available as an option in all materials as an option for overcoming unstable transparent object sorting.

Unity _doesn't_ do this but it seems like you might be able to tint the shadows of transparent objects, as well, if you keep a color buffer around with the tint. But that's a much bigger change.

[gkjohnson]

I modified one of my test scenes to include transparent shadows:

https://gkjohnson.github.io/threejs-sandbox/screendoor-transparency/

There may be a different shadow filtering technique that can be used to further mitigate the dithering but I don't believe the same sampling functions that Unity is using are available in webgl.

[makc]

do you think it's possible to use discard-based shader when rendering
shadows, but normal shader for main camera?

[makc]

ah yes, the re is a check box for that

[gkjohnson]

For those interested I've added a quick approach to shadow tinting in my example. Of course there are transparency overlap issues but that's to be expected:

https://gkjohnson.github.io/threejs-sandbox/screendoor-transparency/

And I've found that setting the shadow radius to 1.4 helps mitigate the dither pattern I've used to be less noticeable (radius 1.0 vs 1.4):

It probably wouldn't be too hard to add transparency dithering to all shaders (including depth) so transparent shadows can be added through a customDepthMaterial if there's still interest in this:

var mesh = //...;
mesh.material = new MeshStandardMaterial( {
    color: 0xff0000,
    ditherTransparency: true,
    opacity: 0.5
} );
mesh.customDepthMaterial = new MeshDepthMaterial( {
    ditherTransparency: true,
    opacity: 0.5
} );

[mrdoob]

@gkjohnson how does the shader look like?

[gkjohnson]

It's pretty simple for the no color case. You compare the current fragments opacity against a threshold in some dither pattern in screen space and discard the fragment if it doesn't pass:

// This goes somewhere near the end of the shader after all
// calculations affecting opacity are done
if ( ditherPatternValue( mod( gl_FragCoord.xy, 4.0 ) ) > alpha ) {

    discard;

}

I'm using a DataTexture to pass the Bayer pattern into the shader but it looks like it can be procedurally generated, as well: https://www.shadertoy.com/view/Mlt3z8. My examples use the 4x4 matrix. Passing a texture into the shader affords some kind of customization or stylization of the dithering but I wouldn't think that needs to be built in.

[mrdoob]

Hmm, I don't know how to proceed here 😅

[gkjohnson]

@mrdoob Ha well I can figure the rest of it out and make a PR using the generated Bayer matrix if you're interested and think it can get merged. I just didn't want to put in the time if ultimately you thought it would be too complicated.

If it sounds good to you I'll make the changes when I get a chance!

[mrdoob]

Maybe this is something that could be added to WebGLShadowMap? So the API would be like this...?

renderer.shadowMap.enabled = true;
renderer.shadowMap.dithering = true;

[gkjohnson]

I see so that would make transparent shadows easier to enable "out of the box", right? Implementation-wise that would probably mean adding more material variants that render with the appropriate level of dithering to match the materials opacity?

I like making it simpler to turn on. How do you feel about adding renderer.shadowMap.dithering = true; in a separate PR? I think it's dependent on adding dither transparency to the other shaders (at least the depth shader) and would afford more flexibility, as well.

[mrdoob]

You're saying that you would like to enable this for some materials but not others?

[gkjohnson]

I think I'm mixing use cases in my mind here but I guess I see the opportunity to kill two birds with one stone (add general dither transparency to any material _and_ add lightened shadows for transparent objects).

Dithered transparency has utility beyond transparent shadows so I think it should be added to all materials if it's going to be added at all -- I've used it to overcome transparency sorting issues and in cases where you can't guarantee transparent object sorting (iterative rendering, compositing multiple buffers, etc). There are artifacts but they aren't so bad or can be hidden with different forms of AA.

If dither transparency is added to every material then transparent shadows come for free in the form of dithering on the DepthMaterial shader. Past that I could see adding an option into WebGLShadowMap to enable transparent shadows out of the box.

[mrdoob]

There are 2 ways to enable dither shadows; either adding it as a property in all materials, or adding it as a property in the WebGLShadowMap code. Both places are being checked when creating the final shader.

Adding it to WebGLShadowMap should be a few more lines. Adding it to materials is much more complicated things, if you start considering material reusage, serialisation, etc

The only reason I would add it to materials is because it's likely users will need two types of shadow types in the same render.

[gkjohnson]

I may not quite be seeing where the complexity comes in but naturally you're more familiar with those classes than I am.

In that case maybe it makes sense to add dithering to the MeshDepthMaterial and MeshDistanceMaterial so the WebGLShadowMap can use the parameter, first? So the WebGLShadowMap will create extra variants that cover 16 transparency possibilities which it will use if dithering = true. Is that right? And we can discuss the possibility of enabling dithered transparency across all materials in another issue?

[mrdoob]

If we add dithering to WebGLShadowMap then we only need to check in WebGLProgram to see what shader snippet needs to be added.

https://github.com/mrdoob/three.js/blob/dev/src/renderers/webgl/WebGLProgram.js#L376-L377

No need to modify the materials, the materials don't really know about shadows.

[gkjohnson]

@mrdoob I see the difference, now. So should I read this as you being against adding dithered transparency to materials or just that you see it and transparent shadows as two separate features?

[mrdoob]

Yeah... Even if they involve the same technique, I think they're separate issues.
I would focus first on the issue the OP reported.

[gkjohnson]

@mrdoob I'm realizing that you might have an alpha layer in your texture which should affect the dithered shadows. It looks like in the case of a displacement map it's expected that users specify a customDepthMaterial. Should that be the case for objects with transparency masks, as well?

[MasterJames]

Just wanted to say. I would be animating the alpha to fade in or out an object appearing. Having the shadow know without assistance what is physically correct would be ideal.
Thinking about depthMaps dither patterns etc. seems oddly disconnected (not that I am following what I would have to do which is the point because my expectation would be that it is correct on it's own). It should just contribute the alpha inverse of the light diminished by the strength of yup I guess the material.

[gkjohnson]

I came across another approach to rendering shadows for transparent objects here:

https://wickedengine.net/2018/01/18/easy-transparent-shadow-maps/

It requires a separate RGBA buffer (I bet you could get away with just RGB) to store the colors. It's assumed that transparent shadows are not cast on transparent objects but the result looks pretty nice. Here's a screenshot from the article:

[gkjohnson]

Looks like Babylon.js has just added support for transparent shadows using the same technique I proposed in #15999:

The Babylon.js announcement tweet.

And the Babylon.js feature PR (which references my sandbox implementation ❤️)

@mrdoob if there's still interest in this feature I can clean up that PR and get it ready for merging again.

[mrdoob]

@gkjohnson Somehow I missed #15999... 😕 Yes! Would be great if you can clean it up! 🙏

[gkjohnson]

@gkjohnson Somehow I missed #15999... 😕 Yes! Would be great if you can clean it up! 🙏

Just cleaned it up! Should be ready for review.