Armory: Limiting the amount of spot light on scene

[VeniGogniti]

SHORT DESCRIPTION:

In scene I have more than 4 spot light but when I press play only 4 of them are visible, others not. After switching off one of the lights, the next ones are switched on during play.
NearDay.zip

Armory: 06 win64
Operating system: win 7
Graphics card & driver: Radeon RX580

[indokan2001]

AFAIK it was intended to be maxed at 4. You can change the max size manually by changing the size of maxLights and maxLightsCluster in LightObject.hx, but it will slow down the rendering as the looping mechanism of lights is still need to be fixed.

[N8n5h]

Any ideas where to start looking to solve this one and fix the problem with lights, @luboslenco ?

[luboslenco]

IIRC the main pain is handling the shadow maps and all the light types. The cap is set to 16 lights total and 4 lights per cluster. The view is divided into 16x16x16 clusters.

Haxe part:
https://github.com/armory3d/iron/blob/master/Sources/iron/object/LightObject.hx#L30

Shader part:
https://github.com/armory3d/armory/blob/master/Shaders/std/clusters.glsl#L2

A good start could be to simplify uniforms passing:
https://github.com/armory3d/armory/blob/master/Shaders/std/light.glsl#L33

// Instead of
uniform mat4 LWVPSpot0;
uniform mat4 LWVPSpot1;
uniform mat4 LWVPSpot2;
uniform mat4 LWVPSpot3;

// Do
uniform mat4 LWVPSpot[maxLightsCluster];

Passing array of mat4 uniforms should nowadays work in Kha.

// Instead of
if (index == 0) {
    vec4 lPos = LWVPSpot0 * vec4(p + n * bias * 10, 1.0);
    direct *= shadowTest(shadowMapSpot[0], lPos.xyz / lPos.w, bias);
}
else if (index == 1) {
    vec4 lPos = LWVPSpot1 * vec4(p + n * bias * 10, 1.0);
    direct *= shadowTest(shadowMapSpot[1], lPos.xyz / lPos.w, bias);
}

// Do?
vec4 lPos = LWVPSpot[index] * vec4(p + n * bias * 10, 1.0);
direct *= shadowTest(shadowMapSpot[index], lPos.xyz / lPos.w, bias);

The problem is the above will cause trouble with WebGL. Alternative solution is to bake shadow maps into single atlas.

Here is where light matrices are bound:
https://github.com/armory3d/iron/blob/master/Sources/iron/object/Uniforms.hx#L785

Here is where shadow maps are bound:
https://github.com/armory3d/armory/blob/master/Sources/armory/renderpath/Inc.hx#L42

If that gets resolved we can investigate raising maxLights and maxLightsCluster.

[N8n5h]

I gave it a go and tried to simplify the uniform passing and here is my progress so far, I think the uniform part works in Krom at least, but I think I have a problem with index,
i.e., if I change this part from light.glsl:

if (index == 0) direct *= PCFCube(shadowMapPoint[0], ld, -l, bias, lightProj, n);
else if (index == 1) direct *= PCFCube(shadowMapPoint[1], ld, -l, bias, lightProj, n);
else if (index == 2) direct *= PCFCube(shadowMapPoint[2], ld, -l, bias, lightProj, n);
else if (index == 3) direct *= PCFCube(shadowMapPoint[3], ld, -l, bias, lightProj, n);

to simply

direct *= PCFCube(shadowMapPoint[index], ld, -l, bias, lightProj, n);

then shadows stop working in my simple scene with 4 point lights. I don't understand much of how index is calculated, so my question is: can index go outside the array range? I tried clamping it but still it didn't work.

int li = int(texelFetch(clustersData, ivec2(clusterI, i + 1), 0).r * 255);

The problem is the above will cause trouble with WebGL

Can you elaborate on this?

Alternative solution is to bake shadow maps into single atlas.

Have you eyed any implementation/presentation in particular?

@luboslenco

[luboslenco]

can index go outside the array range?

It should never get higher than maxLightsCluster - 1, and maxLightsCluster is defined to 4.

I am not sure the "name": "LWVPSpot[0]", "link": "_biasLightWorldViewProjectionMatrixSpot0" can work? It might need to be done via setFloats instead.

Example on how array of vectors is passed:
https://github.com/armory3d/armory/blob/master/Shaders/deferred_light/deferred_light.frag.glsl#L39
https://github.com/armory3d/armory/blob/master/Shaders/deferred_light/deferred_light.json#L32
https://github.com/armory3d/iron/blob/master/Sources/iron/object/Uniforms.hx#L640

The problem is the above will cause trouble with WebGL

Can you elaborate on this?

Dynamic array index was not allowed in WebGL. Maybe this is no longer the case for WebGL2?
https://stackoverflow.com/questions/30585265/what-can-i-use-as-an-array-index-in-glsl-in-webgl

Alternative solution is to bake shadow maps into single atlas.

Have you eyed any implementation/presentation in particular?

I recall reading about it in the presentation below (in the lighting section), unfortunately it's not very detailed.
http://advances.realtimerendering.com/s2016/Siggraph2016_idTech6.pptx
http://advances.realtimerendering.com/s2016/Siggraph2016_idTech6.pdf

[N8n5h]

can index go outside the array range?

It should never get higher than maxLightsCluster - 1, and maxLightsCluster is defined to 4.

Ok, maybe it has something to do with this https://stackoverflow.com/a/60110986

I am not sure the "name": "LWVPSpot[0]", "link": "_biasLightWorldViewProjectionMatrixSpot0" can work? It might need to be done via setFloats instead.

According to this and some of my tests it seems to be working, at least on Krom.
I've avoided setFloats because of having to write that array, I thought that maybe with the offset I could get past the extra loop when passing the values.

The problem is the above will cause trouble with WebGL

Can you elaborate on this?

Dynamic array index was not allowed in WebGL. Maybe this is no longer the case for WebGL2?
https://stackoverflow.com/questions/30585265/what-can-i-use-as-an-array-index-in-glsl-in-webgl

Ok, if the if/else if branching doesn't cause much performance difference than dynamic indexing, then it's possible to go in that route with the uniform setting with offset, right? What do you think?

I will also still be looking towards implementing shadowmap atlases, but it will take me some time since I'm still figuring out how that part of the rendering works in the engine.

[N8n5h]

Update: I got some free time again so I decided to continue with this.

I'm currently trying move the spot lights wvp matrices to a more "dynamic" solution so it's no longer constrained to just 4,

I am not sure the "name": "LWVPSpot[0]", "link": "_biasLightWorldViewProjectionMatrixSpot0" can work? It might need to be done via setFloats instead.

so I gave this a go to try to do it with a setFloats approach, and this is my progress so far https://github.com/N8n5h/armory/tree/light-fix-setfloats, https://github.com/N8n5h/iron/tree/light-fix-setfloats
I'm currently facing an issue that I'm not very sure why it's not working, but shadows stop working using the same logic but instead of an array of matrices bound via "LWVPSpot[0]" using a setfloats approach is not working. I managed to get renderDoc to work and I could check that the uniform is passed correctly and all the values are the same in both cases, so unless there is some typo that I didn't manage to catch, I'm not sure what's going on. Do you have any idea if setfloats uniforms are treated differently

This is my test file, I included renderDoc captures for both cases.
lights_tests.zip

If no solution rises up I will alternatively try to advance my original approach to make it more dynamic. The problem I face is that I'm not sure where the "dynamism" should be. Should the deferred_light.json file be parsed in python and then write dynamically a version with all the uniforms, or should it be done in iron... what do you think?

Also, another thing I was wondering... is it possible to make Zui draw the shadow map depth texture in gray scale or other than just plain red? It would help a lot with debugging to see what is going on in the texture :sweat_smile: ...

@luboslenco

[N8n5h]

Progress update: I managed to get RenderDoc to confirm that the atlasing solution for shadow maps I wrote is working

So now I'm trying to find a way to utilize this atlas data in the deferred shader...

I'm "following" along this tutorial which helped me writing some parts of the solution
https://catlikecoding.com/unity/tutorials/scriptable-render-pipeline/spotlight-shadows/

[N8n5h]

Progress update:

I managed to finally get the deferred/light/shadow shaders to work with an atlas instead of an array of shadowmaps for spot lights...

Now that I got that to work I will move to implement support for some of the trickier ones, directional lights (sun) which already has some sort of atlasing for cascading so I have to take that into account, and point lights, which have a lot of images per light so it may need to be approached differently https://www.gamedev.net/forums/topic/684019-point-light-shadows-in-shadow-atlas/

From there I still need to work

• in the current limitation of 4 light matrices to be sent to the deferred shader
• optimize the algorithm that I wrote for managing atlases which currently uses recursion in a lot of places.
• cover corner cases such as, what happens to the atlas if you remove lights?
• investigate how to make it work for forward

and then finally investigate the "in-fighting" between different lights shaders.