Three.js: WebGLRenderer: Integrate PMREM

[Mugen87]

With the introduction of WebGLCubeMaps, the engine has now a mechanism to internally convert environment maps. I've quickly hacked in PMREMGenerator support today and it is actually possible to perform the conversion used in webgl_loader_gltf. It would love to see PMREM integrated in the renderer since it's a technical detail most users should not care about.

However, my implementation is a hack right now since it's not defined _when_ PMREMGenerator should do its work. The question is: How can WebGLCubeMaps detect textures that should be processed by PMREMGenerator?

[mrdoob]

I think @WestLangley was exploring this too.

[WestLangley]

I hacked in the same thing a few days ago as a proof-of-concept, and it worked beautifully - but not reliably, because I was not focused on that part.

Automatic PMREM conversion should only apply to equirectangular and cube environment maps for MeshStandardMaterial and MeshPhysicalMaterial. I created WebGLCubeUVMaps to do that.

IMO, CubeUVMaps should not be used for blurry scene.background because they are currently too blocky for that purpose. JMO.

[Mugen87]

@WestLangley Okay, I leave this topic up to you 👍 . And yeah depending on the code complexity a separate WebGLCubeUVMaps makes sense.

[WestLangley]

@Mugen87 This implementation details are a bit tedious (as we have discovered), so I likely will not be the one to file the PR. But I will help at the appropriate time in any other way I can. :-)

[Mugen87]

@WestLangley Okay^^. Have you already pushed your changes into a branch of your fork?

[WestLangley]

@Mugen87 Hehe. Sorry, no. The implementation details were so frustrating, I deleted it once I proved to myself the solution exists. :-)

As you pointed out, this feature is complicated by the fact that Basic, Lambert, and Phong do not support PMREM.

I think this is going to have to be part of a larger discussion involving what to do with Lambert and Phong...

[Mugen87]

I think I can make a PR ready as soon as #20439 goes in. I've implemented the feature request the way mentioned in https://github.com/mrdoob/three.js/issues/20115#issuecomment-675644813. Automatic PMREM conversion only applies to equirectangular and cube maps for PBR materials. Besides, Scene.background will not receive a cubeUV texture when using WebGLCubeMaps.get().

The consequence of this approach is that the background will not receive any blurriness anymore with below code:

new RGBELoader()
    .setDataType( THREE.UnsignedByteType )
    .setPath( 'textures/equirectangular/' )
    .load( 'royal_esplanade_1k.hdr', function ( texture ) {

        texture.mapping = THREE.EquirectangularReflectionMapping;

        scene.background = texture;
        scene.environment = texture;

        // load glTF model

    } );

With this setup, PBR materials will receive the PMREM generated env map in cubeUV format but the background will look very blocky (because it just displays the result of the equirect to cube map conversion). So the engine needs a separate mechanism to control the background's blurriness, see #18241. Until this is available, the internal PMREM conversion is not a full replacement for the current workflow.

Against this backdrop, I wonder if it makes more sense to solve #18241 first and then focus on PMREM integration. What do you guys think?

[WestLangley]

Yes, I have found that PMREM is not good for background blurriness. It is too blocky.

Personally, I would like to see Phong removed -- or moved to the examples somehow -- and perhaps replaced with MeshSpecGlossMaterial; the latter is full-featured; Phong is not.

I suppose Lambert will have to be retained as it supports Gouraud shading.

[Mugen87]

I don't think we have to remove materials in context of this issue.

The goal should be adding support for background blurriness (with cube maps) and automatic PMREM conversion for PBR materials.

In general I do not vote to remove build-in materials from the core since this would break a lot of user code. Instead I vote to reconsider the material support for WebGPURenderer.

[WestLangley]

@WestLangley said:

Personally, I would like to see Phong removed -- or moved to the examples somehow

I was reiterating _exactly_ what @mrdoob said to me offline. I happen to agree.

I prefer to see the library improved, and not held back by outdated illumination models. We have moved code to the examples before, and can do so again. Users will be just fine.

[Mugen87]

I suggest we open a new issue for what materials should be supported in the future. TBH, it's off-topic regarding this issue.

Users will be just fine.

Again, it seems you view on this topic (backwards compatibility) does not reflect how users _actually_ react on changes to the library. The project has to be way more carefully than before when things are removed. Only because it was done in the past, it does not mean it was the right way to do so.

[Mugen87]

Against this backdrop, I wonder if it makes more sense to solve #18241 first and then focus on PMREM integration.

I've changed my mind here. I think it's better if Scene.background also uses cubeUV until the project comes up with a solution for #18241. In this way, there will be no visual difference when the automatic conversion is used.

[WestLangley]

@mrdoob suggested moving Phong to the examples, and I was working on that in my fork, but given @Mugen87 is going to block it, it no longer makes sense for me to pursue it at this point.