Postprocessing: webgl2 and multisamplerendertarget

[drcmda]

now that threejs has switched officially to webgl2, could pp perhaps optimize with a flag or something so that we get native MSAA instead of having to use SMAA as an effect pass?

@looeee wrote down the basic steps here: https://discourse.threejs.org/t/advantages-disadvantages-of-using-webgl2/7717/7

const composer = new EffectComposer(renderer, {
  frameBufferType: HalfFloatType,
  webgl2: true // ?
})

[vanruesc]

MSAA support was added in [email protected]. Note that some effects don't work well with MSAA. See the wiki for details.

const context = renderer.getContext();
const maxMultisampling = context.getParameter(context.MAX_SAMPLES);

const composer = new EffectComposer(renderer, {
  multisampling: Math.min(8, maxMultisampling);
});

// Disable MSAA (creates new frame buffers).
composer.multisampling = 0;

[looeee]

Thanks for the clarification @vanruesc. I had expected to find a couple of passes where this causes problems so I'm not surprised. Is it just SSAO or have you found other passes that exhibit problems?

Have you tried using MSAA as a separate pass? In other words, proceed with a normal render target until you get to the point where you would do SMAA, then copy to a multi-sample target and immediately downsample. Hopefully, that would give you the performance gains of MSAA without the artifacts.

[vanruesc]

Is it just SSAO or have you found other passes that exhibit problems?

I believe it affects every effect that uses depth (SMAA with depth-based edge detection, DoF & SSAO). The OutlineEffect should be fine, but I haven't checked.

This article by Matt Pettineo explains MSAA in detail and says that "The coverage and occlusion tests are both performed at higher-than-normal resolution, which is typically 2x through 8x" and "For occlusion testing, the triangle depth is interpolated at each covered sample point and tested against the depth value in the z buffer. Since the depth test is performed for each subsample and not for each pixel, the size of the depth buffer must be augmented to store the additional depth values. In practice this means that the depth buffer will [be] N times the size of the non-MSAA case. So for 2xMSAA the depth buffer will be twice the size, for 4x it will be four times the size, and so on."

The size difference of the depth texture is probably what's causing the artifacts. This post by Phil Fortier has a section called "MSAA and SSAO together" that briefly mentions MSAA artifacts. Afaik, there's no obvious go-to solution for this other than maybe re-rendering depth into another render target.

Have you tried using MSAA as a separate pass?

It's been a few months since I've worked on this and I haven't taken any notes. However, I remember trying to blit/resolve the anitialiased image into another non-multisampled render target before running SSAO, but that didn't remove the artifacts.

[drcmda]

thanks for your explanation @vanruesc! for our project we opted to make MSAA the default and SMAA optional. i've tried with a bunch of examples and it looks pretty clean to me. i really had no idea you already had this implemented, this lib moves so fast im always amazed.

[looeee]

It's been a few months since I've worked on this and I haven't taken any notes.

OK, thanks for the info. I've been meaning to test this myself, but it's a little tricky since I don't think there's any three.js function to blit the multisampled render target meaning it has to be done with bare GL. Will report back here whenever I get around to properly checking this.

[vanruesc]

I don't think there's any three.js function to blit the multisampled render target meaning it has to be done with bare GL.

Yeah, I think you'd want to use blitFramebuffer for optimal performance but that's an experimental feature and there's no abstraction for it in three. I used a ShaderPass with a CopyMaterial to do the blitting.