Terasology: Renderer: split shaders and nodes in conceptually atomic, simpler units

[emanuele3d]

A number of nodes execute shader programs that do a number of things at once.

For example:

• PrePostCompositeNode takes care of reflections, haze, volumetric fog (disabled), ambient occlusion and outline.
• InitialPostProcessingNode takes care of the bloom effect, chromatic aberration (not working), vignette and light shafts
• FinalPostProcessingNode adds film grain, depth-of-field and motion blur to the picture.

While perhaps performance-friendly this way of doing things 1) considerably reduces the debugging capability of tools examining the content of FBOs at any given point in the render graph 2) prevents the insertion of nodes _between_ these individual effects 3) in some cases generates unduly large shaders that are harder to debug and mantain.

This issue advocates for:

• the splitting of shaders and the associated nodes, toward more atomic shaders and nodes
• the development of generic, auxiliary nodes such as TintNode and CompositingNode

Notice that more atomic shaders and nodes, whenever possible, doesn't necessarily mean simpler shaders and nodes. For example, an atomic VignetteNode could provide currently unavailable functionality, such as the replacing or tinting of the vignette texture to signify different things affecting the player: damage, cold, heat, binoculars, underwater gogles and even dream-like states.

[eviltak]

The first step towards this goal in my opinion would be to allow user-defined modular shaders. The current post-processing shader and node design is a mess, with some nodes performing half of their processing in their own shader, feeding the intermediate result as an FBO into a sampler in the post processing composite node where it is sometimes processed further before finally being rendered to screen. Modular shaders would also help break down currently mushed-together nodes, like all the post-processing nodes. Instead of shader programs being hardcoded in a ShaderManager, each Node should be able to define its own shader.

[emanuele3d]

The first step towards this goal in my opinion would be to allow user-defined modular shaders.

Way too ambitious at this stage. It is a GSOC-level type of project and I'd want to see more details. The current primary goal of the renderer is to a) transform it into a Render Graph based architecture b) open its API. Then the idea is to go into shaderland and see what we can do to clean it up: we don't change it, we just clean it up. Then we can think about how to radically change it.

The current post-processing shader and node design is a mess

I can agree that shader management is a mess: we haven't touched it yet. If you want to work on the renderer you'll need to cope with the current node-based architecture and the concept of the render graph being worked on in PR #3088.

with some nodes performing half of their processing in their own shader, feeding the intermediate result as an FBO into a sampler in the post processing composite node where it is sometimes processed further before finally being rendered to screen.

From my perspective this is a perfectly normal, modular, rendering pipeline and it doesn't come even close to the complexity of the composites I have worked with in the film industry. If you wanted to criticize it (and it better be _constructive_ criticism) you might want to try again.

Instead of shader programs being hardcoded in a ShaderManager, each Node should be able to define its own shader.

Shaders aren't hardcoded in the ShaderManager, rather they are in .glsl files. But it is true that the ShaderManager currently has an hardcoded list of shaders which it processes before they become available to the nodes. It is certainly one of my goals to make sure that nodes can bring in their own shaders, we might be on the same page about this. We just haven't gotten to that point.

@eviltak: you reeeeallly want to have a more positive attitude if you want to work on the renderer. You might want to ask @vampcat for advice.

[eviltak]

If you wanted to criticize it (and it better be _constructive_ criticism) you might want to try again.

Sure, I'll take up the challenge. The ultimate goal of giving modules the ability to add their own nodes on the render pipeline hinges on everything being modular, which this issue slightly addresses by setting the goal of "atomizing" everything. I may not be as experienced as you in this regard, but I do know for sure that in it's _current_ state, the rendering pipeline is _far_ from modular.

Take the PrePostCompositeNode as an example. It processes the results of five different effects (all of which have their own separate nodes) by first loading intermediate output from their FBOs into samplers and then finally aggregating them together.

Now say I want to add another node to the pre-post processing stage implementing, say, SSDO. There is no way I can do that without modifying PrePostCompositeNode and its corresponding shader - if the SsdoNode was intended to be part of a module, I would not be able to do it at all without modifying engine code. To have the desired effect of simply plugging in and taking out nodes, this blurring of Node responsibility must be averted.

It's not like PrePostCompositeNode _must_ process the results of the five different effects, either. That responsibility can be handed over to their respective shaders, and all PrePostCompositeNode has to do is the last 6 lines in the shader. The workflow can go as follows:

• PrePostCompositeNode builds the opaque, transparent, depth, normal, and light buffers and passes them on to its neighbor in the RenderGraph, which would be (following the order in prePostComposite_frag.glsl) a (newly created, "atomized" node) VolumetricFogNode.
• VolumetricFogNode computes and updates the opaque and transparent color at each fragment and passes the updated buffers to its neighbor, a (newly created) LocalReflectionsNode, which again does the same and passes the buffers to its neighbor AmbientOcclusionNode, and so on.
• After the pre post composite pass is complete, all that is left to do is mix the opaque and transparent buffers into a new color buffer (essentially the last 6 lines of prePostComposite_frag.glsl).

Come to think of it, the issue does advocate something like the above, which is exactly what I meant by "modularizing" shaders, so I am not sure where the backlash comes from.

Of course, this problem may only be limited to the post-processing nodes, but I did specify:

The current _post-processing_ shader and node design is a mess

And so did your issue (indirectly).

---

@eviltak: you reeeeallly want to have a more positive attitude if you want to work on the renderer.

Not reeeeallly sure where you got the negativity from. The only negative thing in my comment was "the current _post-processing_ shader and node design is a mess" (emphasis again to prevent misconceptions, I was talking purely about the post processing node pipeline) which is just a more direct way of saying

While perhaps performance-friendly this way of doing things 1) considerably reduces the debugging capability of tools examining the content of FBOs at any given point in the render graph 2) prevents the insertion of nodes between these individual effects 3) in some cases generates unduly large shaders that are harder to debug and mantain.

I may not have all your years of experience to know that this is considered modular in the CG industry (if it is, I must admit I will be slightly disappointed), but I do know that it is something to be overcome in order to achieve the goals outlined in this issue.

[vampcat]

@eviltak I won't be commenting much on this, but you'll have to learn (probably the hard way) that just being decent at coding is not enough - learning how to communicate (not just string together some words in english, but rather communicate properly and effectively) is way more important. Before manu posted his message, I had half a mind to say something along the same lines, but I refrained - primarily because I was in your shoes a few weeks ago.

You are rude, inconsiderate and arrogant. The worst part? You don't even realize it. Why do I say it so bluntly? Because so was I, and I'm glad someone pointed it out.

[eviltak]

@vampcat Got it. I'm sorry if I overstepped boundaries @manu3d and @vampcat. I shall make conversations more genial and productive.

[emanuele3d]

The ultimate goal of giving modules the ability to add their own nodes on the render pipeline hinges on everything being modular, which this issue slightly addresses by setting the goal of "atomizing" everything.

Yes and no. A renderer API would, as a first step, allow modules to add nodes to the RenderGraph. This doesn't need modular shaders. Atomic nodes, concentrating on simple tasks such as tinting and compositing, would then create a finer modularity. So, I disagree that this issue "slightly" addresses the issue. It addresses it heads on by prescribing a concrete way forward:

• the splitting of shaders and the associated nodes, toward more atomic shaders and nodes
• the development of generic, auxiliary nodes such as TintNode and CompositingNode

Now say I want to add another node to the pre-post processing stage implementing, say, SSDO. There is no way I can do that without modifying PrePostCompositeNode and its corresponding shader - if the SsdoNode was intended to be part of a module, I would not be able to do it at all without modifying engine code. To have the desired effect of simply plugging in and taking out nodes, this blurring of Node responsibility must be averted.

This paragraph, and the other 20 lines that follow it, are precisely what I mean in: _"2) prevents the insertion of nodes between these individual effects"_. So, you are trying to crash trough an open door.

Not reeeeallly sure where you got the negativity from. The only negative thing in my comment was "the current post-processing shader and node design is a mess"

That is no small negative assertion. Never ever call somebody else's work a mess: you don't know what's behind it. Go back a couple of years and take look how the whole renderer looked like. You are ignorantly neglecting the progress that I, vampcat and tdgunes before him managed to do on top of begla's excellent prototyping work. We are still in the phase of transforming that monolithic (but highly functional) prototype into a production-ready, extensible part of the codebase. Calling any parts of it a mess is like arguing that a mechanic shop should be spotless clean during working hours.

Look, you started with the wrong foot, both here and in previous contributions, by being overly assertive. It's not the end of the world and we can all turn things around.

The golden rule always applies. Imagine you made something, it's far from finished and somebody, a complete stranger, wanted to make changes to it. He or she might have good ideas about it, he might even think along the same lines with you. But would you want to work with him if he is inconsiderate about the work you and others before you put into it? I don't think so, you'd just keep going your way until a better person comes along.

The kind of feedback you are giving is the kind of feedback you might be able to give to people you worked closely with, for years. With people you don't know, independently of the experience they might have, you want to add large amount of padding in your communication. For example:

• I see how this came to be this way. I wonder if it could be improved by...
• I'm unfamiliar with this code and I can't quite understand why this was done this way...
• I have a suggestion but I'm not sure how it fits in the bigger picture...
• I'd like to take care of this aspect by doing A, B and C, what do you guys think about it?
• etc...

Keep in mind the kind of contributions/interactions you'd like to receive/have from a complete stranger and you'll be fine. As you gain trust and experience it will get easier and more relaxed.

Now, take a big breath and put a smile on your face. We are here to have fun, learn and help each other.

[eviltak]

@manu3d Thank you, that's the best advice I have heard yet.

So to get my bearings straight, this issue focuses more on the node pipeline rather than the shaders, which will be the gradual next step after this issue is resolved.

[emanuele3d]

You are welcome @eviltak.

And yes, what you mention about the focus of this issue is fair.