Upbge: bgl_NormalTexture

[Jesusemora]

It should be easy to add a new layer for 2DFilters using the normal data of the scene, this would allow us to create Screen Space Reflection 2Dfilters. Right now the texture and normal data are sent to the shader to calculate lights, we just need to render the normal to a different layer during these light calculations.

The new layer would be called bgl_NormalTexture, and work in the same way as bgl_RenderedTexture and bgl_DepthTexture. it will also be a first step towards deferred shading.

Optionally, a bgl_RoughnessTexture and a roughness slider in the material tab would allow us to control the roughness of SSR in the filter, and maybe more (better SSAO?). bgl_RoughnessTexture would be float like bgl_DepthTexture.

An SSR 2DFilter will greatly improve the quality of bge graphics, by adding self reflection and refraction effects, all of this while mantaining forward rendering.

[panzergame]

Good proposal.

I'm looking for an implementation in blender shader (this one could be easy as it just a write to gl_FragData) and for custom shaders, for this one we certainly have to expose a bgl_Normal but the compatibility issue still remains for older shaders.

Also it would be interesting to have a global settings disabling normal attachment to the main FBO for performance.

Considering this issue #705, the normal map should be placed to the last available attachment to let the user using attachment from 0 to a defined maximum.

@lordloki : I will look for the design, let me know if you have some idea or if you want to manage this request.

[Jesusemora]

one more thing on bgl_roughness, we may want more information so it should be a vec2 or vec3 instead of float, and it could be in the material tab but also for textures.
we would need roughness but also reflection intensity for non reflective surfaces ,to manage dirt on mirror surfaces and to show non metalic surfaces like plastic.

[panzergame]

The roughness should be based on existing attributes or use a new one declared in material with maybe a texture influence ?

[Jesusemora]

A new roughness slider and a new mirror intensity slider in material tab, and if posible one in the texture tab.
I was thinking of rendering these to a layer to have more control, rendering roughness to the r channel and intensity to the g channel, combining the floats into an rgb texture.
if the layer has only roughness, the effect on SSR will affect all objects. mirror intensity is needed for surfaces in wich we don't want a reflection like plastic. if we don't get to have this layer, we could still use bgl_RenderedTexture and have the effect on everything, but it would be more stylized instead of realistic.
maybe call it something else like bgl_MirrorTexture. we could also have a third channel with metalic information if vec2 is not posible.

[adriansnetlis]

Mirror intensity is weird name and again would set more problems than solutions. Simply call it metallness. Do what everyone does if it looks right. Having unified workflow with almost every other piece of software in existance would be a good thing. You could also use the metallness value to control how much the specular reflection gets tinted and ideally also adjust the intensity of diffuse color to match the behaviours of metals.

[lordloki]

@panzergame I would like to finish deployment icon or hardware skinning first. After, It should be a good training for render part :-)

[panzergame]

I made a little UI proposal:

The items are called Render Attachment to remember the OpenGL framebuffer attachments. 8 attachments are available (later the first one will be reserved), to define one the user have to add using the + button and can remove one with - button.
The list shows the name of the attachment (only for user purpose) and its index.
Each attachment have two basic properties, the type used for default materials to recognize the data to output (normal, metalness or custom for no default write), the HDR to specify the data type char, half float or float. Then the custom attachment type use an extra arguments for the number of data it contains, between 1 and 4.

About the BGE design, I'm thinking about just leaving gl_FragData in custom shader and let the user manage it properly with the index he defined, for blender material an output node could be defined with a selected attachment.
From the filter side I'm just thinking of exposing an array bgl_RenderedTextures with all the attachments.

@Jesusemora : Does it looks on a correct path ?

[adriansnetlis]

This looks like a good idea.

A few more things that would be awesome:

• Attachment type "Filter" which lets you select a 2D filter that is rendered and then usable as attachment in another filter. Useful for multipass filters.
• Attachment types "Roughness" and "Emission" which would be similar to metallness function wise and data wise, but instead store roughness and emission values accordingly. Maybe a type "MRE" that is RGB color and contains all metallness, roughness and emission combined in one attachment. Useful when all those parameters are needed, reduces attachment count.

Also, for the filter side of attachments. A name bgl_Attachments would make more sense and be much shorter and thus faster to write.

[Jesusemora]

Does it looks on a correct path ?

I think it's great, personally I only care for functionality, something that works. custom layers seems to be the best way to do it.
the only problem I see is with assigning textures or values, all materials should output to the normal/metalness/custom layer. there should be an option to use a "standard" texture/materialValue instead of nodes.
Leaving the option in nodes would force all materials to be node materials, and that would complicate things.
i was thinking something like this:

this would render the textures to the layer, a similar option should be in the material tab to render the objects as a flat color for each attachment.

Attachment type "Filter" which lets you select a 2D filter that is rendered and then usable as attachment in another filter. Useful for multipass filters.

2D filters are applied in order, so you always get the result of the last filter in bgl_RenderedTexture. could you give an example of when what you suggest would be useful? i don't think is necesary.

Attachment types "Roughness" and "Emission" which would be similar to metallness function wise and data wise, but instead store roughness and emission values accordingly.

we are not talking about making deferred (yet), the only special layer is normal (color is obtained from bgl_RenderedTexture and depth from bgl_DepthTexture, wich we already have), the other layers like metalness, roughness, emission, etc, can just be floats (b/w). what is done with them is defined by the filter, we just need to know wich surfaces must be affected by reflections, wich must be blurry, wich must be shinny. we can do it with a simple mask, white areas of a layer will be affected by a shader, while the black areas will remain untouched:
gl_FragColor = mix(bgl_RenderedTexture(texcoor, 0), reflection, reflectivity)
things should remain as simple and modular as posible to maximize performance and facilitate the job of the coder. we may get deferred in the future, but we must go slowly, do the most important first to build something bigger over it.
edit:

Maybe a type "MRE" that is RGB color and contains all metallness, roughness and emission combined in one attachment. Useful when all those parameters are needed, reduces attachment count.

I think this is a good idea. It will be enough with just having a vec3 option for the attachment to render the texture as RGB instead of float (b/w). the rgb is just three floats and each channel can be accesed in glsl using x/y/z or r/g/b.

[panzergame]

Sorry i wasn't explicit enough: the type used for default materials to recognize the data to output, by that I mean to use a default technique in blender materials for the attachment type Normal and Metalness.
But as we can see the variety of output values is quite large, so the default mode for non-node materials can be limited.
In consideration I would like to know what output values are the most interesting ? i think we should not go up to 10 different values. Also they should not request extra computation from the shader internal values, so limiting to all values actually used by the shader, else the user might use nodes.

[Jesusemora]

In consideration I would like to know what output values are the most interesting ?

ok i looked at the unreal pbr to refresh my memory. unreal uses base color, metallic, specular, roughness, and emisive color in pbr. i think what adriansnetlis meant by roughness attachment was to use the specular roughness in the layer, u4 says that all materials have specular. this may be true for a pbr environment, but i think having a separate roughness from specular will give artists more freedom in what they want to make, you could always set it to the same values to make it pbr. what do you think?
base color should still be the rendering result, it has shadows and specular applied to it, as well as emission information. one point could be made about using emission information to create glow effects.
metallic affects how reflective a surface is, and roughness affects how blurry the reflection is.
so, we would want roughness, metallic and emission.

Sorry i wasn't explicit enough: the type used for default materials to recognize the data to output, by that I mean to use a default technique in blender materials for the attachment type Normal and Metalness.

i think we should not go up to 10 different values. Also they should not request extra computation from the shader internal values, so limiting to all values actually used by the shader, else the user might use nodes.

metallic is not used by blender right now, it will have to be added to the influence tab and the material tab, or use a different value like Ray Mirror, Ambient or Intensity.
we could get roughness from hardness but it feels buggy, the specular roughness applied to materials do not affect nodes or the material node (this might be a bug). we could also add a new roughness value just for this, or combine it with metallic and a third channel to use a single texture and just render it to the layer, and we could use a color picker in the material tab. this would make it a single RGB layer and we would not have to combine individual values from different shaders or create 3 attachments. any of the channels can be adapted to the artists need, if the artist doesn't need emission, he can use this channel for something else, the same would go to roughness and metallic. the only CON of this is that the artist would have to make an MRE texture (metallic/roughness/emission) for each composite material.

[adriansnetlis]

If you care how modern rendering engines handle those layers:
base/albedo color - the raw color of surface
metalness - defines wether surfaces behaves as dielectric, metallic or somewhere in between (semiconductors or microsurface mixes)
roughness - amount of microsurface roughness
specular - used to control fresnel's F0 (reflectivity at grazing angles), any values above 0 provide full fresnel reflections at grazing angles
Those are the main settings. There can be additional parameters like subsurface, subsurface color, clearcoat (very useful for car paints), clearcoat roughness, emission, transparency etc.

However, I would love to see current material tab just get destroyed and overhauled as current Blender material system is horrible. I wouldn't even mind a node-only solution. Everybody loves nodes and everybody loves PBR.

[Jesusemora]

@adriansnetlis yes, but we have a few problems:
1- we are using forward and rendering SOME deferred layers, not all. the lighting is still calculated by the blender pipeline, we would have to remake the entire pipeline to use deferred and there are issues like transparency and antialiasing wich have to be implemented diferently or are imposible.
2- we don't have access to light data in 2D filters, specular and to some extend subsurface would be useless for now.
3- there is a problem of performance, layers must be optimized or there will be problems with video memory and buffers. we can't have too many values in realtime.
just being able to make a SSR 2D filter will make blender look as good as unity and some unreal games. we are still far from unreal, we don't have raytracing, or tesselation, but it will be a huge leap forward.

[panzergame]

From the material node side, i implemented a OutputAttachment node which just get as input a vec4 and an attachment index to output:

[lordloki]

Windows build to test: https://mega.nz/#!JpEBSIBb!pkt8QYxPSlZxz45PWvCMghmBHTVi2nFJRTCLIUxfexI

[panzergame]

The last state of ge_render_attachment allow to read all the output attachment via bgl_RenderedTextures[8].
Example:
ge_render_attachment.zip

Also the user here will not control the first attachment (used as default color output) and only 7 attachments are modifiable by the user.

@Jesusemora: Is it on the right way for you ?

[lordloki]

New build with lastest commits: https://mega.nz/#!p4EREa6B!p1tivUq3E68CpgmjrQCKnsoOlwp9Ua-Bwspv0SyR2cs

[Jesusemora]

@panzergame i can't test the file due to the OS i use, so i will talk about what's been written. I think the attachment output is a good idea, is there a difference beetwen color and vector? i think vector can use negative values? would this be a problem for the normal buffer?
bgl_RenderedTextures[8] is good. shouldn't the first attachment be 0?
the rest is perfect as it is, 7 buffers (we can put 3 b/w textures into a single buffer using nodes, so that increases the amount of data we can use in a shader) is the same amount used in unreal, so it could be adapted for deferred in the future. UPBGE games are going to look great!

[panzergame]

is there a difference beetwen color and vector? i think vector can use negative values? would this be a problem for the normal buffer?

From GLSL color is vector, but depending on the attachment image format you can have limitation to normalized and positive values : HDR_NONE is the equivalent of RGBA8 (8 bits per channel, all channels are normalized and positive), HDR_HALF_FLOAT and HDR_FULL_FLOAT are simply floats with different precisions, they can negative and as big or small the precision allows.

bgl_RenderedTextures[8] is good. shouldn't the first attachment be 0?

I wrote 8 to tell that there's 8 attachments:
uniform sampler2D bgl_RenderedTextures[8].

Anyway currently this array contains for the first attachment bgl_RenderedTexture. But the indices used by the user in node materials start from 0 to mean first "extra" attachment.
So the user have to add 1 to get the correct attachment from the index set in UI, this is clunky.

There's two solutions:

• Remove the first attachment in bgl_RenderedTexturesand let only the 7 "extra" attachment and so rename the array by something bgl_DataTextures[7].
• Always define in UI a first attachment for the default color output. The drawback of this solutions is that the user have to know it and the previous HDR setting will be moved into the one of the first attachment.

@Jesusemora, @adriansnetlis : What do you think ?

[panzergame]

Is it fine to choose number 1 ?

[Jesusemora]

@panzergame yes, i was just confused by the 8, lists/arrays always start on 0. either solution is fine, we get the screen from blg_RenderedTexture in 2Dfilter anyways. would removing the first attachment have an impact on performance? other than that i don't see a problem, as long as it's behavior is documented it shouln't be an issue with users.
to create a scene with SSR many of the objects on the scene will have to be node materials. are node materials as fast as normal materials? if not it would be good to have a way of assigning attachments from the texture tab. but it's not urgent, and the material tab will have to be redesigned sooner or later for PBR and to make values like roughness affect both shaders (transform roughness into hardness for specular and rough for oren-nayar, or use diffuse intensity for hardness thus simulating PBR for simpler shaders, etc).

[BluePrintRandom]

we should have the ability to use the principled node from cycles as a ui
for a SSR based shader

On Tue, Jul 31, 2018 at 9:58 AM, Jesusemora [e-mail]
wrote:

@panzergame https://github.com/panzergame yes, i was just confused by
the 8, lists/arrays always start on 0. either solution is fine, we get the
screen from blg_RenderedTexture in 2Dfilter anyways. would removing the
first attachment have an impact on performance? other than that i don't see
a problem, as long as it's behavior is documented it shouln't be an issue
with users.
to create a scene with SSR many of the objects on the scene will have to
be node materials. are node materials as fast as normal materials? if not
it would be good to have a way of assigning attachments from the texture
tab. but it's not urgent, and the material tab will have to be redesigned
sooner or later for PBR and to make values like roughness affect both
shaders (transform roughness into hardness for specular and rough for
oren-nayar, or use diffuse intensity for hardness thus simulating PBR for
simpler shaders, etc).

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/UPBGE/blender/issues/718#issuecomment-409293313, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AG25WaeGHiZe79HDMZ1HZR0zr0lF-4LYks5uMIzDgaJpZM4UqaW9
.

[panzergame]

to create a scene with SSR many of the objects on the scene will have to be node materials. are node materials as fast as normal materials? if not it would be good to have a way of assigning attachments from the texture tab. but it's not urgent, and the material tab will have to be redesigned sooner or later for PBR and to make values like roughness affect both shaders (transform roughness into hardness for specular and rough for oren-nayar, or use diffuse intensity for hardness thus simulating PBR for simpler shaders, etc).

For the moment is implemented only a node to output to extra attachment, but a material tab UI will come. Else material tab is the same as a node material in the source and in the way the GLSL is generated.

@BluePrintRandom : This node is for some reason using GLSL builtins concerning lights… It needs to be remade totally…

[Jesusemora]

For the moment is implemented only a node to output to extra attachment, but a material tab UI will
come. Else material tab is the same as a node material in the source and in the way the GLSL is generated.

great.

This node is for some reason using GLSL builtins concerning lights… It needs to be remade totally…

the cycles nodes are just previews of the materials, the real shaders are the ones wich do the rendering, and work differently from realtime. in some cases the shaders are just diffuse and look exactly the same. the best thing would be to create a new PBR shader for game, a node with ins for all the values and textures and outs to work with attachments.

[youle31]

As this feature has been merged in last upbge master, I close this.

You can add a normal attachment in render tab.

In the shader, you declare:

uniform sampler2D bgl_DataTextures[7];

Then if your attachment is in the slot number 0:

vec4 normalTex = texture(bgl_DataTextures[0], gl_TexCoord[0].st);

About other types of attachments, there are ways to recreate position (for example) from bgl_DepthTexture). You can have a look at Sebastien Mestre SSR, Global Illumination... on BA. For now, this doesn't really work perfectly. If someone wants to work on it, test normal attachment to implement some filters, this is cool :) I close this to clean the tracker for now