Terasology: Rendering Engine Overhaul

[emanuele3d]

Note: the following description is a cut&paste of now closed issue #1741. We reopen it here to create an Epic, a container for a number of related issues.

Introduction

The current renderer is largely an hardcoded implementation. It features a number of features that can be toggled on/off, i.e. light shafts, or enabled to some pre-defined level, i.e. Depth of Field blur levels. But it is inflexible in the sense that new features must be added to the source code rather than, say, injected at runtime by a module.

Also, while the renderer might eventually support the concept of multiple cameras, each camera would have to render in pretty much the same visual style, dictated by the settings stored in the rendering config unless some kind of overriding machinery is added to enable/disable features separately from the rendering config. For example, CCTV-like cameras might not require the full range of visual effects currently available through the renderer, but might require some additional or alternative processing to provide their typical black&white, grainy/noisy look.

In this context I'd like to propose here the embryo of a new architecture for the renderer.

Rendering Tasks, Rendering Pipelines

The current renderer is mostly a collection of conceptually atomic rendering tasks. I.e, first the shadow map for the main light is rendered into a buffer. Then the scene reflected in the water surfaces is rendered into another buffer. Then the sky, the landscape and its inhabitants. Transparent objects are blended in and (skipping a number of steps for brevity) a number of post-processing effects are added eventually leading to (ta-dah!!!) the final image shown on screen or saved into a screenshot.

The key idea of the new architecture is to recognize that these tasks are organized into a Directed Acyclic Graph (DAG), by virtue of the fact that each task is potentially dependent on the output of other tasks being executed first and the fact that most tasks eventually provide the inputs for tasks that follow. This DAG is what is commonly referred to as the rendering pipeline, despite it not being quite a line but a number of branching and potentially overlapping paths.

In this context, adding new features should be as easy as adding tasks to the pipeline, or to better put it, nodes to the graph. Similarly, disabling features should be as easy as removing nodes from the graph. For example, at this stage OculusVR code is intertwined with the standard mono rendering mode. Enabling or disabling a given stereo mode (not just Oculus) would then be a matter of replacing a number of tasks in the pipeline with the stereo-only or mono-only version of them. All at runtime.

The new architecture should also allow for multiple pipelines to coexist.

• A CCTV monitor might display a grainy black&white image, while one nearby might display the same image in infrared. This would be handled by two separate rendering pipelines having world data and camera in common but applying different sets of rendering tasks to them.
• A portal nearby might trigger a whole pipeline to be attached to the primary pipeline (the one leading to the image on the user's screen) so that the world behind the portal is visible through it.

Advanced features

Also to keep in mind as potential, advanced, self-optimization features:

• automatic elimination of expensive, redundant state changes: some state changes in opengl are computationally expensive, i.e. binding or clearing a frame buffer. Ideally as soon as a node is added or removed from a rendering pipeline and well before the pipeline is executed, it should check for redundant state changes and eliminate all but the last one before a rendering. This would be done by wrapping expensive calls into their own rendering tasks.
• automatically estimate the fastest possible DAG configuration when alternatives are available: in some cases there is no unique DAG for the given set of rendering tasks. I.e. task B and C might need to be executed after task A, but executing task C first might reduce the number of expensive state changes and improve performance. Whenever a setting changes, a task is added or perhaps even when manually triggered by the user, the pipeline could do a few profiling runs with different DAG configurations to measure which one is the fastest. Frame rate would be expected to jitter during profiling, but it would happen only briefly, in the context of user-driven transitions, and would be probably acceptable.

[emanuele3d]

Since then

From the time the description in #1741 was written an important thing happened: our organization was accepted in the Google Summer of Code (GSOC) initiative.

This in turn had a number of consequences:

• A number of issues, including #1741, were listed as potential GSOC projects for students to evaluate.
• @tdgunes jumped aboard and eventually wrote a proposal GSOC project.
• @tdgunes and his proposal were accepted by our organization as one of three GSOC projects.
• @tdgunes implemented a large chunk of the architectural changes suggested above.

In particular the concept of nodes and tasks is now built in the rendering engine and all the _conceptual_ rendering steps are now wrapped into nodes which in turn trigger (or not!) rendering tasks. The rendering tasks are then executed every frame.

Also, one of the item described above is the automatic elimination of redundant OpenGL state changes, i.e. preventing the binding of the same frame buffer multiple times in a row, unnecessarily. This has been accomplished via classes implementing the StateChange interface.

To be done

Much remain to be done to complete this overhaul of the rendering engine. I will strive to add issues to this epic detailing what needs to be done. But let me list a few big items:

• The DAG is currently a node list. Nodes are currently arranged as a list, not as a graph. And they are not explicitly connected with each other: there is no explicit dependency between them even though, in fact, they are often dependent on each other, i.e. one node acting on the output of another.
• Modules can't interact with the system yet, i.e. to add or remove nodes: the functionality is not yet exposed through an API. And for proper use we'll need to provide the functionality to add and remove at runtime entire sub-graphs of pre-connected nodes, not just individual nodes.
• Many nodes still contain large chunks of legacy code, not taking advantage of the new architecture. This is an active area in this period as nodes are being refactored for this reason. Furthermore, this node-by-node refactoring also clarifies what each nodes does and requires, hinting to the way forward architecturally. For example, nodes in a DAG are connected via "edges". The various potential meanings of edges is emerging and becoming clearer as the nodes are being refactored.
• The way Materials/Shaders are handled hides a large amount of OpenGL state changes occurring in when a material is enabled. This makes it relatively difficult, for example, to find out which texture buffer is used in input by a particular node, in turn making dependencies between nodes difficult to untangle. Considerable changes in this area, and in particular in the ShaderParameters classes (which could be eliminated eventually) are foreseeable.

I'd like to conclude here this update. Stay tuned.

[spadaval]

@emanuele3d I'm considering redesigning this system for GSoC, and I have a few questions:

1. Do you think the render system could benefit from further abstraction/simplification? It looks complicated to me, but that's probably because I've never really worked with render systems before.
2. Is this system stable enough to survive another paradigm shift?
3. Is rendering in parallel something we're concerned about?

[emanuele3d]

@hybrideagle: Hi. Please note this issue is an "Epic". An Epic is a container for smaller issues, but you can see it that way only installing ZenHub: in GitHub you'll see it as a standard issue.

Regarding your questions:

1. I'd be happy to hear your ideas in this regard.
2. The system is in transition. But a good number of things are likely to be stable. It all depends on what you have mind when you write "paradigm shift". Ultimately, if you have an amazing idea that captures people's imagination, it might get implemented no matter what.
3. That's a bit broad. Define "rendering in parallel"? Because of course given the GPU's parallelism it could be argued we already do that.

[4Denthusiast]

This issue is a bit vague in what it describes, but it looks like it's now complete, with the extraction of the rendering graph to the CoreRendering module.