Godot: Multi-pass post-processing shaders

[seadra]

There are many effects that require multiple passes, each time with a different shader, either finally combining the outputs into a final one, or using the output of one pass as input to the next one.

A lot of 3D engines out there supports this in some form. Unity has the concept of Pass.

Right now, it is impossible to do multiple passes with godot, so we are restricted to "simple" stuff that can only involve a single pass.
Please add passes to godot!

[reduz]

Which kind of effects are you looking for ?

On Wed, Nov 5, 2014 at 1:47 PM, pikapikachu [email protected]
wrote:

There are many effects that require multiple passes, each time with a
different shader, either finally combining the outputs into a final one, or
using the output of one pass as input to the next one.

A lot of 3D engines out there supports this in some form. Unity has the
concept of Pass http://docs.unity3d.com/Manual/ShaderTut2.html.

Right now, it is impossible to do multiple passes with godot, so we are
restricted to "simple" stuff that can only involve a single pass.
Please add passes to godot!

—
Reply to this email directly or view it on GitHub
https://github.com/okamstudio/godot/issues/862.

[seadra]

On top of my mind, this or this.

[reduz]

for the first one, i don't really see how you need multipass, for the
second i don't either but i'm not quite sure i understand what it is.

What _is_ missing is post processing shaders, those will be available soon
and will support multiple passes.

On Wed, Nov 5, 2014 at 2:20 PM, pikapikachu [email protected]
wrote:

On top of my mind, this http://www.clicktorelease.com/code/npr-shading/
or this
https://www.reddit.com/r/computergraphics/comments/2l5s91/1bit_shader/.

—
Reply to this email directly or view it on GitHub
https://github.com/okamstudio/godot/issues/862#issuecomment-61845920.

[godotengine]

oh ok, yeah I understood the second one, but it is done using post
processing too, not really multipass rendering of objects.
multipass is planned for 1.1 (currently focusing on 1.0)

On Wed, Nov 5, 2014 at 2:27 PM, Juan Linietsky [email protected]
wrote:

for the first one, i don't really see how you need multipass, for the
second i don't either but i'm not quite sure i understand what it is.

What _is_ missing is post processing shaders, those will be available soon
and will support multiple passes.

On Wed, Nov 5, 2014 at 2:20 PM, pikapikachu [email protected]
wrote:

On top of my mind, this http://www.clicktorelease.com/code/npr-shading/

or this
https://www.reddit.com/r/computergraphics/comments/2l5s91/1bit_shader/.

—
Reply to this email directly or view it on GitHub
https://github.com/okamstudio/godot/issues/862#issuecomment-61845920.

—
Reply to this email directly or view it on GitHub
https://github.com/okamstudio/godot/issues/862#issuecomment-61847186.

_OkamStudio_

[seadra]

One pass paints surfaces to different colors based on normals and world positions, and from this output one pass does edge-detection based on this and draws the wireframe edges, one pass adds in lighting and textures.
After all the outputs form 1 and 2 are combined, a post-process shader does edge-detection and dithering.

So you think these steps (which are _not_ post-process, but passes in Unity) can be done in a single step? How?

[seadra]

And just slightly related but, is there a reason why we can't use the real CG or GLSL like in Unity?

With the current spec, is there any way of achieving things like 2D twisting? (and can we have sincos and half4 type please? :))

Or what about the double vision?
Since vertex shaders and fragment shaders are not functions in godot (why?), we can't dictate the inputs and outputs of the shaders freely. It is a problem for any input/output that is not readily given in the spec (VAR1 and VAR2 may alleviate the problem is certain cases, but not in general).
I know, this would be a compatibility-breaking change, but it's still pre-1.0, so please allow us to define shaders as functions freely!

[reduz]

Yes, but those (passes after the first one) are all post-processing passes
in a chain, not material passes. Multiple post-processing passes _will_ be
supported in 1.1.

The reason CG and GLSL are not supported is that Godot is not a low level
engine, the shader language aims to make common things easier rather than
give you absolute flexibility. The renderer does a lot of stuff behind your
back so you don't need to write shaders as complex.

2D shaders will be supported in 1.1

On Wed, Nov 5, 2014 at 2:46 PM, pikapikachu [email protected]
wrote:

And quite unrelated but, is there a reason why we can't use the real CG or
GLSL like in Unity?

With the current spec, is there any way of achieving things like 2D
twisting
http://http.developer.nvidia.com/CgTutorial/cg_tutorial_chapter03.html?

—
Reply to this email directly or view it on GitHub
https://github.com/okamstudio/godot/issues/862#issuecomment-61850152.

[seadra]

Yes, but those (passes after the first one) are all post-processing passes in a chain, not material passes. Multiple post-processing passes _will_ be supported in 1.1.

I'm not sure I'm following on this (what is a material pass? I believe material is a godot term).
I _know_ that they are ordinary passes in Unity. But let's say the same thing can be achieved using a post-process shader.

How can you add a pass that involves the objects texture and lighting using a post-process shader?

Or let's say you did the pass that is based on texture and lighting, how do you add a pass that involves edges based on object's normal vectors using post-process shader?

And not to mention that you have to combine the results of these two passes in a non-trivial way.

I can't see how it is possible.

And think in general terms. Unity has the concept of Pass for a reason.
That is because it really is impossible to achieve everything you can do using multiple passes by just using single step shader + post-processing.

The renderer does a lot of stuff behind your back so you don't need to write shaders as complex.

Fair enough for simple stuff. Not good if I want to do something that is not already there ---something like 2D twisting that is not even remotely complex. And sometimes, people _do_ need to write complex and specialized shaders for fancy effects.

Even if it's not the full GLSL, can't you at least let people define the vertex/fragment shader functions themselves and dictate the input/output?

[reduz]

Well, let me be a little clearer.

You _cant_ do edge detection in anything other than a post process, because
when you are drawing the actual geometry, you don't have access to the same
depth buffer you are writing to, and you can't draw beyond the bounds of
the object. In fact, the link you mentioned clearly specifies that it is
using a post process for this.

You can do this sort of fake lighting in a post-process fine because you
have access to position and normals, as Unity supports deferred rendering.

On Wed, Nov 5, 2014 at 3:41 PM, pikapikachu [email protected]
wrote:

Yes, but those (passes after the first one) are all post-processing passes
in a chain, not material passes. Multiple post-processing passes _will_
be supported in 1.1.

I'm not sure I'm following on this (what is a material pass?).
I _know_ that they are ordinary passes in Unity. But let's say the same
thing can be achieved using a post-process shader.

How can you add a pass that involves the objects texture and lighting
using a post-process shader?

Or let's say you did the pass that is based on texture and lighting, how
do you add a pass that involves edges that involves object's normal vectors
using post-process shader?

The renderer does a lot of stuff behind your back so you don't need to
write shaders as complex.
Fair enough for simple stuff. Not good if I want to do something that is
not already there ---something like 2D twisting that is not even remotely
complex

—
Reply to this email directly or view it on GitHub
https://github.com/okamstudio/godot/issues/862#issuecomment-61858827.

[seadra]

Sorry, it should be:

Or let's say you did the pass that is based on texture and lighting, how do you add a pass that involves edges surfaces based on object's normal vectors using post-process shader?

And again, the results of the 1st and 2nd parts are combined in a non-trivial manner (see below, the bottom for more).

You can do this sort of fake lighting in a post-process fine because you have access to position and normals, as Unity supports deferred rendering.

What about textures?

Here is what he says:

Pass 1: Surfaces

This is the same pass described before. Each face is colored based on the object position and the face normal. These values are amplified to keep adjacent normals from being too similar. The one addition since last time is the ability to manually dampen the influence of the normal; useful to prevent gradual smooth surfaces from generating lots of edges. For that, I use the the vertex color red channel, which is multiplied by the calculated normal before combining it with the object position to make the final face color.

Pass 2: Lighting+Textures
The Unity-calculated lighting gets put in the red channel, the green channel is currently unused (1.0), and the blue channel has the texture. This all works in single channels because the source images/calculations are grayscale. In order to keep the lighting and texture influences completely separate, the lighting is generated against a fully white albedo. This separation allows me to use different dither curves for lighting and texture later on in the combining stage. Also, since I have the vertex colors handy from the surfaces pass (which only uses the red channel), I also multiply the vertex color blue channel against the texture channel here. That let's me avoid actually creating textures for things that should just be a little bit darker overall, like the skylight windows.

In the final shader, I take these two outputs and combine them together. The surfaces buffer is used to generate the wireframe edges; the lighting+textures buffer is used for the rest. Because lighting and texture are in different channels, I can give them different dither curves now. I want the lighting to be sharp, with only 3 values: black, 50%, and white. The texture channel is overblown a bit to push it towards white, then it uses the full range of shades. You can see how sharp the lighting is while the texture has more varying dither patterns. To keep the edges visible in darkness, the lighting value determines which shade of line to use, black or white. The texture might be a little too blown out here, but I can tweak that later when more details get added.

So in the 2nd pass, it's not some fake lighting, it's the fully-fledged, Unity calculated lighting (which you do _not_ have access during post-process).

Also note that "The surfaces buffer is used to generate the wireframe edges; the lighting+textures buffer is used for the rest.", so you _do_ need the results of the 1st pass and 2nd pass _separately_ to do that kind of post-process.

[reduz]

yes but this is still done in post processing. you have to understand that
it is not technicaly possile to read from the same depth or color buffer
you are writing to, video hardware does not allow this. At much you can
only use blend modes.

On Wed, Nov 5, 2014 at 4:10 PM, pikapikachu [email protected]
wrote:

Sorry, I meant to say:

Or let's say you did the pass that is based on texture and lighting, how
do you add a pass that involves edges surfaces based on object's normal
vectors using post-process shader?

You can do this sort of fake lighting in a post-process fine because you
have access to position and normals, as Unity supports deferred rendering.

What about textures?

Here is what he says:

Pass 1: Surfaces

This is the same pass described before. Each face is colored based on the
object position and the face normal. These values are amplified to keep
adjacent normals from being too similar. The one addition since last time
is the ability to manually dampen the influence of the normal; useful to
prevent gradual smooth surfaces from generating lots of edges. For that, I
use the the vertex color red channel, which is multiplied by the calculated
normal before combining it with the object position to make the final face
color.

Pass 2: Lighting+Textures
The Unity-calculated lighting gets put in the red channel, the green
channel is currently unused (1.0), and the blue channel has the texture.
This all works in single channels because the source images/calculations
are grayscale. In order to keep the lighting and texture influences
completely separate, the lighting is generated against a fully white
albedo. This separation allows me to use different dither curves for
lighting and texture later on in the combining stage. Also, since I have
the vertex colors handy from the surfaces pass (which only uses the red
channel), I also multiply the vertex color blue channel against the texture
channel here. That let's me avoid actually creating textures for things
that should just be a little bit darker overall, like the skylight windows.

In the final shader, I take these two outputs and combine them together.
The surfaces buffer is used to generate the wireframe edges; the
lighting+textures buffer is used for the rest. Because lighting and texture
are in different channels, I can give them different dither curves now. I
want the lighting to be sharp, with only 3 values: black, 50%, and white.
The texture channel is overblown a bit to push it towards white, then it
uses the full range of shades. You can see how sharp the lighting is while
the texture has more varying dither patterns. To keep the edges visible in
darkness, the lighting value determines which shade of line to use, black
or white. The texture might be a little too blown out here, but I can tweak
that later when more details get added.

So in the 2nd pass, it's not a fake lighting, it's the fully-fledged,
Unity calculated lighting.

—
Reply to this email directly or view it on GitHub
https://github.com/okamstudio/godot/issues/862#issuecomment-61863394.

[seadra]

yes but this is still done in post processing.

Well, sure. My point is, post-process at the final step requires two inputs, which are the intermediate outputs of step 1 and 2, separately.

[reduz]

ahh alright, I understand what you mean. Yeah, this also will be added by
1.1 or 1.2

On Wed, Nov 5, 2014 at 4:35 PM, pikapikachu [email protected]
wrote:

yes but this is still done in post processing.
Well, sure. My point is, post-process requires two inputs from each step,
separately.

—
Reply to this email directly or view it on GitHub
https://github.com/okamstudio/godot/issues/862#issuecomment-61867395.

[seadra]

Whoaaaaaaa! Hurraaay!

[seadra]

OK, I know, now I'm stretching it, but I gotta try :)

Will we also be able to define our inputs and outputs to each shader?
Things like 2D twisting and double vision :) Or fish-eye camera. Oh, and half!

[reduz]

this stuff is more like a compositing pipeline and sometimes you need
custom g-buffers for it. The current renderer in godot targets GLES2, and
using g-gufers and other stuff is really difficult on it.
Eventually sometime next year the plan is to move towards GLES3 and leave
GLES2 for compatibility, so all those new features can be added.

On Wed, Nov 5, 2014 at 4:41 PM, pikapikachu [email protected]
wrote:

OK, I know, now I'm stretching it, but I gotta try :)

Will we also be able to define our inputs and outputs to each shader?
Things like 2D twisting and double vision :) Oh, and half4!

—
Reply to this email directly or view it on GitHub
https://github.com/okamstudio/godot/issues/862#issuecomment-61868334.

[seadra]

Golden :)

[seadra]

Oh and, will we get half and fixed types? :) Not very important for desktop games, but can become quite important for mobile games.

[akien-mga]

What's the status on this issue @reduz? I guess the work on 2D shaders for 1.1 (IIRC) somehow addresses this issue?

[akien-mga]

I'm not an expert on the topic, but I guess this PR more or less addresses this issue? #4084

[reduz]

Rendering engine rewrite for 3.0 will address these I think
On Apr 21, 2016 11:34 AM, "Rémi Verschelde" [email protected]
wrote:

I'm not an expert on the topic, but I guess this PR more or less addresses
this issue? #4084 https://github.com/godotengine/godot/pull/4084

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
https://github.com/godotengine/godot/issues/862#issuecomment-212945673

[seadra]

Any updates about post processing shaders? Planned for 3.1, maybe?

[reduz]

Yeah, layers / stencil / custom postprocessing has been pushed to 3.1

On Tue, Jan 16, 2018 at 1:10 AM, seadra [e-mail] wrote:

Any updates about post processing shaders? Planned for 3.1, maybe?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/godotengine/godot/issues/862#issuecomment-357848895,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AF-Z2_WuL7NXiSRLWLEACq2knruTo8Eyks5tLCE1gaJpZM4C3X_-
.

[seadra]

Still, the fact that it's on the roadmap is awesome! Thanks!

[Chaosus]

Well, Godot have shader passes now, post-processing is a subject of another issue(as well as other specific feature request). _Closed by bugsquad_

[seadra]

@Chaosus This issue is about multipass post-processing shaders, not multiplass material shaders. So can you please not close this issue?

[seadra]

Thanks!

[akien-mga]

Moving to the next milestone as 3.1 is now feature frozen.

[Anutrix]

Any updates?

[samdze]

Why can't we extend the issue to generic multi-pass shaders support like Unity does?

[akien-mga]

Feature and improvement proposals for the Godot Engine are now being discussed and reviewed in a dedicated Godot Improvement Proposals (GIP) (godotengine/godot-proposals) issue tracker. The GIP tracker has a detailed issue template designed so that proposals include all the relevant information to start a productive discussion and help the community assess the validity of the proposal for the engine.

The main (godotengine/godot) tracker is now solely dedicated to bug reports and Pull Requests, enabling contributors to have a better focus on bug fixing work. Therefore, we are now closing all older feature proposals on the main issue tracker.

If you are interested in this feature proposal, please open a new proposal on the GIP tracker following the given issue template (after checking that it doesn't exist already). Be sure to reference this closed issue if it includes any relevant discussion (which you are also encouraged to summarize in the new proposal). Thanks in advance!