Three.js: Unnecessary .initMaterial() calls if CubeCamera && [texture.encoding != output.Encoding || light-based-material]

[carstenschwede]

TLDR

If you render to a CubeCamera and

• have renderer.outputEncoding != renderTarget.texture.encoding, at each render, for each material a .acuireProgram() will be called, which is slow. (Example) (covered by #19056)

• have light-based materials, at each render for each material a new .initMaterial() will be called, which is slow due to .cloneUniforms()(Example) (covered by #13656, #14121 and now #19673)

Description of the problem

I noticed a severe performance regression when updating to latest THREE version (r117), causing my rendering times to increase from ~3ms to ~200ms. My scenes heavily rely on CubeCamera's, and I noticed an unusually high amount of material recompiles.

It appears that if you render a CubeCamera and you either

• have renderer.outputEncoding != renderTarget.texture.encoding or
• have a light based material in your scene

that lightsStateVersion property will flip/flop between the CubeCamera and main cameras renderStates, causing unnecessary calls to initMaterial. Even if the material does not need to be recompiled, the calculation of programCacheKey takes longer than usual, probably because it now involves cloning of the material's uniforms.

{
lightsStateVersion: 6,
materialProperties.lightsStateVersion: 5,
//^^^^^this causes the recompile^^^^^^

parameters.shaderID: "basic",
program: WebGLProgram {name: "MeshBasicMaterial", id: 81, getUniforms: ƒ, getAttributes: ƒ, destroy: ƒ, …},
program.cacheKey: "basic,highp,false,true,3000,false,false,3000,false,3000,false,,3000,false,false,3000,false,false,3000,false,false,false,false,false,false,false,false,false,false,false,false,false,0,false,,false,false,false,true,,false,,false,false,0,true,false,,8,4,false,0,0,0,0,0,0,0,0,false,1,0,false,0,false,false,0,0,false,false,false,3001,2,...",
programCacheKey: "basic,highp,false,true,3001,false,false,3000,false,3000,false,,3000,false,false,3000,false,false,3000,false,false,false,false,false,false,false,false,false,false,false,false,false,0,false,,false,false,false,true,,false,,false,false,0,true,false,,8,4,false,0,0,0,0,0,0,0,0,false,1,0,false,0,false,false,0,0,false,false,false,3001,2,...",
programChange: true
}

You can find an example here:

• TextureEncodings
• Light-based material

I didn't bisect the commit in question, but it appears to have happened between [r112,r115].

TextureEncoding
I don't expect all involved renderTargets to have the same encodings, but if that should be the case, I would recommend a corresponding warning message. On the other hand, IMHO the encoding of the renderTarget should not affect caching of the object's material at all? Although if it does, it might make sense to cache the material multiple times for each different encoding instead of flip/flopping between both encoding and causing a recompilation every time.

Thanks for investigating, please let me know if I can supply additional information.

Three.js version

• [x] Dev
• [x] r117
• [x] r116
• [?] ....
• [ ] r111

Browser

• [?] All of them
• [x] Chrome
• [x] Firefox
• [?] Internet Explorer

OS

• [?] All of them
• [?] Windows
• [x] macOS
• [?] Linux
• [x] Android
• [x] iOS

[carstenschwede]

Probably related to #19129.

[Mugen87]

Duplicate of #19056.

[carstenschwede]

@mugen87: Thanks, #19056 covers the recompiles due to different texture encodings. My second example here does not specify any texture encodings, yet it still invalidates light.state.version and calls .initMaterial() on every frame. Could you explain if this is also covered by #19056 or reopen if not?

[Mugen87]

Missed your second fiddle, sorry. It's a duplicate of #14121.

[carstenschwede]

Thanks!

[sciecode]

I don't expect all involved renderTargets to have the same encodings, but if that should be the case, I would recommend a corresponding warning message. On the other hand, IMHO the encoding of the renderTarget should not affect caching of the object's material at all? Although if it does, it might make sense to cache the material multiple times for each different encoding instead of flip/flopping between both encoding and causing a recompilation every time.

I've only now seen this issue, but I feel like it's important to address this comment.

I was aware my PR ( #19129 ) could cause problems of that matter. The PR fix was necessary because not caching the RTT encoding leads to materials not rendering correctly for the set encoding, which is a lot more problematic than it may sound like.

However, I do agree that there's a bit of a grey area, given that we also don't want force recompiles when flip-flopping encodings, tone mappings etc... This is especially important for the post processing pipeline.

I've noticed that @Mugen87's PR ( #19634 ) would be able to solve this problem, but unfortunately it also led to a big performance hit.

I think the other approach proposed by #15047 should be able to fix this, however the path to getting there requires some careful consideration of each program life-cycle and caching strategies. Given that we are smart about this, that should be the correct way of solving this problem. Which should be a priority, IMHO.

[Mugen87]

19673 is an important step for better performance and should also help to achieve #15047. However, @sunag needs to help in refactoring NodeMaterial otherwise it will block the PR. Or the project accepts a broken node material.