Godot: Mesh looks too glossy when using a roughness texture

[mysticfall]

Godot version:
master / 224d5371ff65a

OS/device including version:
_Manjaro Linux 17.1_

Issue description:
When I created a roughness texture using Substance Painter and used it in Godot, it makes the model look much too glossy compared to how it looked originally.

I don't expect it to look exactly the same between Substance and Godot, however, it feels like the texture must almost look all white to make it not to look too glossy.

I've attached a few screenshots for comparison:

This is how my character looks like in Substance Painter. I haven't started to paint rough/smooth areas, so the texture looks quite uniform like this:

However, when I apply it to a material in Godot, it looks glossy even when the roughness is set to the maximum value:

I also noticed that the shadow almost highlights glossy area as shown below. I'm not too sure if it's the same problem, so please let me know if I should separate it as another issue.

[jaran]

karroffel pointed out in discord that godot squares the roughness in it's final calculation and suggested the following fix. When we convert the material to a shader and then modify the roughness to equal the sqrt of the old value we get results much closer to substance painter.

Old roughness calculation in shader
ROUGHNESS = roughness_tex * roughness;
Modified to
ROUGHNESS = sqrt(roughness_tex * roughness);

screenshot from substance painter using default preview sphere project.

screenshot in godot without the sqrt fix. Very reflective

screenshot with the sqrt fix now looks very close to substance painter

shader params screenshot

Test project here: https://github.com/jaran/godot-substance-test

[tagcup]

How do they compare when the roughness uniform is set to 1?

[tagcup]

This may not be a matter of power.

Is it possible to see Substance's shader code? I wonder if they're using the alpha value from 2012 Burley notes (from section 5.6) rather than 2015 Burley notes (section 3.3). We use the latter for the reasons explained in Heitz paper.

[jaran]

I updated my comment with a screenshot of the params. Metallic and roughness are set to 1. Specular 0.5

[tagcup]

For isotropic case, the difference is essentially alpha = (0.5+roughness/2)^2 versus alpha = roughness^2.

[tagcup]

Thanks; if you're still seeing a difference at roughness = 1, that sqrt shouldn't be there.

Can you try setting alpha = (0.5 + roughness/2)^2 in the shader code (drivers/gles3/scene.glsl) and compare? That should at least tell us whether Substance is stuck with the 2012 Burley model or not.

[jaran]

@tagcup When I change the scene.glsl to

float alpha = pow((0.5 + roughness/2), 2);

and then undo the sqrt in the material shader and change it back to

ROUGHNESS = roughness_tex * roughness;

It goes back to being too reflective like the first example without the sqrt fix

without sqrt. to reflective.

with sqrt back to looking like substance painter

[tagcup]

I'm not sure what the exact problem is (there might be a missing scaling factor somewhere) but adding an ad-hoc sqrt isn't the solution (yes, roughness gets squared in the shader code, but that's what alpha is, you are supposed to square the roughness).

I thought it's possible to access Substance's shader code though, no? That should help pinpointing the problem.

[jaran]

I found a bunch of shader docs. I put what i think are the relevant ones here as i can't seem to find them hosted by substance anywhere. I do see they do alpha = roughness * roughness in a couple of places. Hope it helps.

https://www.dropbox.com/sh/31wq4i4bfi8g8zk/AACKSlDZUalolmgdy-maFkvDa?dl=0

[jaran]

I got a response from Allegorithmic. I put the fragment shaders they told me were relevant to this issue in the drop box (fs.glsl and pbr_ibl.glsl).

"We take the linear roughness value directly from the texture, and use "alpha is roughness squared". We never used Burley's 2012 remapping."

[tagcup]

Thanks! I'll give it a check during the weekend if someone else doesn't pinpoint the problem until then

[tagcup]

It'd be helpful if you could upload that Godot project for anyone trying to fix the issue

[jaran]

@tagcup sorry i just moved it to github from gitlab here

https://github.com/jaran/godot-substance-test

[karroffel]

Oh what just came to my mind, this might have two possible causes:

1. cubemap filtering
2. specular/diffuse light equations

---

1. Roughness of the environment reflection is implemented by taking the skybox and applying a blur to it. The rougher the surface the more blurred the sky needs to be. Maybe the association of roughness <-> blur behaves like blur = roughness ^ 2 or something similiar. That would cause applying a square root to the roughness to get the proper blur.

2. @mysticfall are there any lights in the scene? Because I noticed that the specular reflection on my GLES2 branch (using Phong) gets way too big with high roughness, that can cause things like skin to look way more shiny than it should be.

[tagcup]

I did some comparisons with substance using a metal sphere of varying roughness in steps of 0.1 with a different environment (no lights), and things don't match with sqrt in general either.

Here are the screenshots:

Clarification: upper image is from Godot, using current material shader without additional sqrt; lower is from substance painter with the same environment. Upper and lower don't match as-is at the same roughness. An image in the upper panel corresponding to square rooted roughness value from the lower panel doesn't match in general either.

[karroffel]

@tagcup that's pretty useful! That means it's most likely the blur applied to the environment map that does the blurring with different "speed" depending on the roughness than it should.

[tagcup]

@karroffel That could indeed be the case, because I couldn't spot any real differences in the material BRDF.

I guess one way to rule the environment map out would be to repeat the similar experiment with Unity vs Godot in an indoor scene, with an omni light and no background.

[karroffel]

I think what could be enough to fix this is change those lines

https://github.com/godotengine/godot/blob/5b7ebf4d04129d55cfaf346b0ce2de1fe8601579/drivers/gles3/rasterizer_storage_gles3.cpp#L1257

https://github.com/godotengine/godot/blob/5b7ebf4d04129d55cfaf346b0ce2de1fe8601579/drivers/gles3/rasterizer_storage_gles3.cpp#L1469

(one of them is not called from anywhere, I don't remember which one it is.)

Applying a pow(X, 2.0) on the roughness value should give the desired effects.

[fracteed]

Hey guys, I only recently discovered this thread, so thought I would put together a more definitive test scene between Substance Painter, Blender eevee and Godot. I have known for a while that Godot is a bit "off" with my textures from substance, but it can be a hard thing to quantify especially with various lighting conditions. Of course it is unrealistic to expect Godot to look exactly like other renderers, and it still does look pretty good.

As can be seen from this comparison, eevee is much closer to Substance than Godot. Godot is looking particularly off with the more blurred metallics as well as looking very blown out in general on the dielectrics (but that may be a saturation or levels issue?). The highlights on the tubing also don't look as rough as Painter/eevee.

They are all in orthographic mode and use the same studio hdr (from HDRI Haven). I have tried to rotate the hdri's in Substance and Blender to match the default HDRI in Godot. Really wish that hdri's could be rotated in Godot!...and yes I did file an issue for this long ago :)

I chose this hdri so that just the blurred highlight of the area light is showing, without too much extraneous detail. On the left are 4 grades of roughness (0, 0.1, 0.2, 0.3, 0.4) with a plastic dielectric material and on the right with metallic ones. The tubing has a 0.3 roughness.

I have attached a zip of the Substance Painter, Blender 2.8 and Godot files with all textures. Hopefully this may help isolate the issue!

roughness_test.zip

[fracteed]

@karroffel @tagcup @akien-mga is this something that is likely to be fixed for 3.1? Now that I am aware of it, I am second guessing everything I do in Substance-Godot. Not sure if @reduz knows about this issue?

[BastiaanOlij]

Just a really stupid remark, this may have been covered already. Godot when importing textures does an sRGB to linear conversion so the textures are in linear color space for rendering and lighting. After rendering it brings the final result back to sRGB. That is great for albedo textures that hold color information but it would change the balance of these types of textures.

Does disabling the sRGB setting on importing these textures make a difference?

[fracteed]

@BastiaanOlij , it was certainly worth trying and I had a go, changing this and other settings in the import tab, but to no avail. As can be seen from tagcup's image earlier, the roughness looks wrong as compared to Substance, even without any maps.

Have also attached another image from Godot 2.0.6, with the albedo color down at 50% grey from the default pure white. This is just to show the highlights better, as they look blown out on the dielectrics that I posted earlier. As can be seen here, the 0.1 and 0.2 roughness levels look almost identical, as with the 0.3 and 0.4 levels. If you compare this to the original Substance capture, which has a definite gradation between all levels. Even the Blender test doesn't fare so well for the dielectrics (the ones on the left).

[reduz]

By looking at this issue, I think what is clear is that roughness is not the issue, but that Substance uses a twice as bigger angle to do sampling from the cubemap. You can tell this by simple duplicating first half values in the screenshot posted, notice how they look similar now:

If this is standard, then Godot has to be changed to use the same angle.

[mysticfall]

Now it's finally fixed with #29182.