Three.js: Proposed: Order Independent Transparency

[bhouston]

Description of the problem

We (@Jozain and myself) are currently implementing out-of-order transparency for ThreeJS for a project of ours. We'd like to contribute it back. We are implementing the method of Morgan Mcguire, Weighted-Blended Order Independent Transparency: http://casual-effects.blogspot.ca/2014/03/weighted-blended-order-independent.html

We are helped a lot by @arose's example OIT code on which our stuff will be partially based, just our stuff will be designed into the core of Three.JS: https://github.com/mrdoob/three.js/issues/4814

Here is an proposed example of how to enable this mode:

var renderer = new THREE.WebGLRendererer( .... );

// add new "transparency" variable to renderer to select the transparency rendering mode
//renderer.transparency = THREE.PaintersTransparency; // the current ThreeJS method, the painter's algorithm
renderer.transparency = THREE.OrderIndependentTransperancy; // the new OIT method

This mode will be implemented by a new WebGLOrderIndependentTransparency class that will be responsible for managing the buffers. It will create two additional RenderTargets that will track the size of the current render buffer. The first will be the accumulation buffer which we will render all transparency objects to and then we will render the objects as well to some alpha product buffers - thus two separate renders. This will be followed by a "resolve" stage that renders the transparent objects over top of the existing beauty render of the opaque objects.

This workflow will work with base WebGL and work with multi-sample buffers as well. It can obviously be speed up using multiple render targets, but this shouldn't be that slow for non-huge numbers of transparent objects.

We are going to implement this now, just wanted to give a heads up as to our design so that if you have feedback on it, we can incorporate it now.

/ping @WestLangley @arose @spidersharma03

[tentone]

This would be really nice, since it seems like threejs will be moving for a new renderer for WebGL2 it would be a good chance to take a look at this and this (#9620)

[bhouston]

http://Clara.io will be staying on the current ThreeJS WebGLRenderer until WebGL 2 is available on more than 80% of devices - thus it needs to be available in Edge, Firefox and Safari before we can consider moving over to it.

[NeilGatenby]

Hi. You guys seen http://graphics.cs.williams.edu/papers/TransparencyTVCG17/? Same guy, more recent work? It's all about fog and dispersion and refraction and stuff which this issue is NOT about, but equation (5) is (I think) an improvement on equation 10 from the 2013 work that you guys are looking at.

https://www.youtube.com/watch?v=jWe5Ae22Ffs#t=8m59s, too

Hope this helps

Neil

[NeilGatenby]

So I got it working using blended transparency as per eqn(5) of http://jcgt.org/published/0002/02/09/. The @arose demo was a God-send! Thanks. Note, I've NOT implemented weighted blended transparency - for the reasons I outline here, for the interest of others:

(1) The weighting function is hard to tune. @arose has noted this, the paper itself notes this (mentioning at least SEVEN different functions), and unless all my scenes are very similar, that is a problem

(2) As noted in the 2013 JGT paper, the transparencies change as you get nearer to (further from) your scene! They say "depth-weighted results are surface order-independent but not translation-invariant along the depth axis". It's highly annoying when you moving through the scene!

(3) They justify the "weighted" part of their paper with lots of examples. Comparing their "blended" images to their "weighted-&-blended" images, one is hard pressed to spot the difference!!

Their (non-weighted) blended approach is better that Bavoil and Myers 2008's blending function, which is in turn better than Meshkin's 2007 original (see JCGT link, above, for specifics on those papers)

Hope this helps

Neil

[arose]

thanks @NeilGatenby, do you have an example online by any chance?

[NeilGatenby]

No

But am happy to sort that ... will post here when done

Still struggling to ...
(1) get the opaque render behind/amongst the transparent and
(2) initialise the revealage texture to 1.0 (or 1,1,1,1) ... I figure the accumulation texture is auto-zeroed, so less of an issue. Since my images (of just transparent objects) look good, I'm curious to see what will result when the revealage texture is properly initialised

N

[NeilGatenby]

https://github.com/mrdoob/three.js/compare/dev...NeilGatenby:oit is the diff between between my OIT and @arose's OIT ... hope that helps, all

@dunkyp deserves an honourable mention for any progress I made here - he helped me get started on this

[WestLangley]

@NeilGatenby NIce demo. IMHO, I'd set

transparentMaterial.depthWrite = false;

for a fair comparison with oit.

[NeilGatenby]

Thanks. Certainly improves that case. I guess things will go (more) awry when the transparent meshes are mixed up with opaque meshes that I want depth writing ON for?

[WestLangley]

Not necessarily. Opaque materials are rendered first by the renderer; transparent materials second. If the user sets material.depthWrite = false for the transparent materials, it can work fairly well, as you can see.

It can be problematic if half-semi-transparent/half-opaque textures are involved, however.

[NeilGatenby]

Interesting stuff. Thanks again. Sounds like it might work well for an "in between" quality ... when you don't want to wait for OIT or suffer the painter artefacts I'm seeing now.

Currently I can't get OIT working with opaque, anyway ... still struggling with the points I mentioned in my first post of today, viz... I want the depth buffer from my opaque pass to prevent both transparent passes from working where they needn't. Then I want to combine the colour buffer from that same opaque render with the 2 transparent textures I've created, for the final image.

Sounds easy in software but I'm struggling to do it with my limited three.js skills! ☺

[Ben-Mack]

@bhouston Can you share the code for Order Independent Transparency that you've implemented?

[bhouston]

The best code is actually @arose's code here: https://github.com/mrdoob/three.js/issues/4814

[trusktr]

Looks like @pailhead has also made some progress on order-independent-transparency here: https://discourse.threejs.org/t/depth-peel-and-transparency/5365 (or is that something else? It seems to solve some issues that I've seen with the order-dependent transparency)