Three.js: Unreal Bloom and Renderer Transparency issue

[fire67]

Description of the problem

I am currently using the Unreal Bloom post with a transparent renderer.

renderer = new THREE.WebGLRenderer({ alpha: true });
renderer.setClearColor(0xFF0000, 0);

Unfortunately, the Unreal Bloom pass doesn't seem to take in account the alpha channel of the coming texture when doing the blur and it apparently comes from getSeperableBlurMaterial in _js\postprocessing\UnrealBloomPass.js_.

The fragment's alpha channel is always set to 1.0 which results in a complete removal of the previous alpha values when doing the additive blending at the end.

This is happening in the latest version of Three.js r92 and you can experiment about this issue using the Unreal Bloom example

[WestLangley]

In getSeperableBlurMaterial()you can try a work-around:

gl_FragColor = vec4( diffuseSum / weightSum, 0.1 );\n\

Remember, it is an example, so you are free to modify it.

[fire67]

Thanks for the answer @WestLangley !
Yeah this would help keeping the transparency but would destroy the bloom effect (which is really cool btw) around the object. I think a more cleaner solution would be to apply weight on alpha channel too.

Any solution would help me a lot :)

[WestLangley]

/ping @spidersharma03

[fire67]

I achieved to find a solution for this but I would like to know your opinion @spidersharma03 and @WestLangley.

void main()
{
    vec2 invSize = 1.0 / texSize;
    float fSigma = float(SIGMA);
    float weightSum = gaussianPdf(0.0, fSigma);
    float alphaSum = 0.0;
    vec3 diffuseSum = texture2D(colorTexture, vUv).rgb * weightSum;
    for( int i = 1; i < KERNEL_RADIUS; i ++ )
    {
        float x = float(i);
        float weight = gaussianPdf(x, fSigma);
        vec2 uvOffset = direction * invSize * x;

        vec4 sample1 = texture2D( colorTexture, vUv + uvOffset);
        float weightAlpha = sample1.a * weight;
        diffuseSum += sample1.rgb * weightAlpha;
        alphaSum += weightAlpha;
        weightSum += weight;

        vec4 sample2 = texture2D( colorTexture, vUv - uvOffset);
        weightAlpha = sample2.a * weight;
        diffuseSum += sample2.rgb * weightAlpha;
        alphaSum += weightAlpha;
        weightSum += weight;

    }
    alphaSum /= weightSum;
    diffuseSum /= alphaSum; // Should apply discard here if alphaSum is 0
    gl_FragColor = vec4(diffuseSum.rgb, alphaSum);
}

[WestLangley]

Sorry, I have no opinion on this.

[spidersharma03]

Sorry @fire67 , was away for a few days. Yeah, what you have done seems correct. I didn't handle the transparency.

[fire67]

No problem, thank you for the answer @spidersharma03 !
While testing the effect I noticed a visual difference when pushing some values very high.

Do you have any idea where does this comes from ?

[spidersharma03]

@fire67 , Not sure actually. Do you have the changed code in some branch? I will try to take a look.
Thanks!

[fire67]

@spidersharma03 No, unfortunately the code is not available on a branch. Do you want me to do it ?
Just to let you know, there's nothing more than the main function in the previous post.

[jcyuan]

@fire67 @spidersharma03 hi, I tried to use your main() code to replace the old one in example but all bloom effect gone although the background takes effect it's transparent.

this.renderer = new THREE.WebGLRenderer({
        canvas: view,
        alpha:true
});

//and pass
this.$unrealBloom = new UnrealBloomPass(new THREE.Vector2(this.app.stageWidth, this.app.stageHeight), 3.2, .6, 0.7 );
this.$composer.addPass(this.$unrealBloom);

this.$blurPass = new ShaderPass(HorizontalBlurShader);
this.$blurPass.uniforms['h'].value = 0;
this.$composer.addPass(this.$blurPass);

this.$blurPass.renderToScreen = true;

please help, thanks.

[jcyuan]

ok so the problem is caused by my setting:
this.$renderer.context.enable(this.$renderer.context.SAMPLE_ALPHA_TO_COVERAGE);
the alpha channel of user RTT will not be handled by lowlevel for A2C in WebGL1.0 (it does in WebGl2.0)

the changed shader works well actually, thank you very much @fire67

[shokunin000]

Is it possible to increase bloom based on emissive values? Trying to utilize those values to increase strength on glowing objects but this shader code is bit above my head! Any help would be much appreciated

[robhawkes]

@fire67 Did you manage to get this working with the bloom keeping the colour of the object? As soon as I try to use the changed shader the bloom just goes pure white, rather than taking the object colour as before.

I also tried disabling rendering.alpha but the bloom was still pure white.

Any ideas?

void main() {\n\
  vec2 invSize = 1.0 / texSize;\
  float fSigma = float(SIGMA);\
  float weightSum = gaussianPdf(0.0, fSigma);\
  float alphaSum = 0.0;\
  vec3 diffuseSum = texture2D( colorTexture, vUv).rgb * weightSum;\
  for( int i = 1; i < KERNEL_RADIUS; i ++ ) {\
    float x = float(i);\
    float weight = gaussianPdf(x, fSigma);\
    vec2 uvOffset = direction * invSize * x;\
    vec4 sample1 = texture2D( colorTexture, vUv + uvOffset);\
    float weightAlpha = sample1.a * weight;\
    diffuseSum += sample1.rgb * weightAlpha;\
    alphaSum += weightAlpha;\
    weightSum += weight;\
    vec4 sample2 = texture2D( colorTexture, vUv - uvOffset);\
    weightAlpha = sample2.a * weight;\
    diffuseSum += sample2.rgb * weightAlpha;\
    alphaSum += weightAlpha;\
    weightSum += weight;\
  }\
  alphaSum /= weightSum;\
  diffuseSum /= alphaSum;\
  gl_FragColor = vec4(diffuseSum.rgb, alphaSum);\n\
}

Before:

After:

[robhawkes]

I've been mulling this over for the past few days and decided to attempt a slightly different approach that takes the original (non-transparent) colour calculations, leaves them unmodified and then adds on top the new alpha calculations from @fire67 with a couple of minor tweaks to utilise the same weights as the colour calculations. I don't know if this is the right way to do it, and the calculations could probably be tidied up, but it works.

void main() {\n\
  vec2 invSize = 1.0 / texSize;\
  float fSigma = float(SIGMA);\
  float weightSum = gaussianPdf(0.0, fSigma);\
  float alphaSum = 0.0;\
  vec3 diffuseSum = texture2D( colorTexture, vUv).rgb * weightSum;\
  for( int i = 1; i < KERNEL_RADIUS; i ++ ) {\
    float x = float(i);\
    float w = gaussianPdf(x, fSigma);\
    vec2 uvOffset = direction * invSize * x;\
    vec4 sample1 = texture2D( colorTexture, vUv + uvOffset);\
    vec4 sample2 = texture2D( colorTexture, vUv - uvOffset);\
    diffuseSum += (sample1.rgb + sample2.rgb) * w;\
    weightSum += 2.0 * w;\
    float weightAlpha = sample1.a * w;\
    alphaSum += weightAlpha;\
    weightAlpha = sample2.a * w;\
    alphaSum += weightAlpha;\
  }\
  gl_FragColor = vec4(diffuseSum/weightSum, alphaSum/weightSum);\n\
}

Previous approach (posted earlier in this issue):

New approach combining original logic with updated logic posted earlier in the issue:

[robhawkes]

Tidy things up a bit and it seems to work just as well:

void main() {\n\
  vec2 invSize = 1.0 / texSize;\
  float fSigma = float(SIGMA);\
  float weightSum = gaussianPdf(0.0, fSigma);\
  float alphaSum = 0.0;\
  vec3 diffuseSum = texture2D( colorTexture, vUv).rgb * weightSum;\
  for( int i = 1; i < KERNEL_RADIUS; i ++ ) {\
    float x = float(i);\
    float w = gaussianPdf(x, fSigma);\
    vec2 uvOffset = direction * invSize * x;\
    vec4 sample1 = texture2D( colorTexture, vUv + uvOffset);\
    vec4 sample2 = texture2D( colorTexture, vUv - uvOffset);\
    diffuseSum += (sample1.rgb + sample2.rgb) * w;\
    alphaSum += (sample1.a + sample2.a) * w;\
    weightSum += 2.0 * w;\
  }\
  gl_FragColor = vec4(diffuseSum/weightSum, alphaSum/weightSum);\n\
}

I've noticed some little (single pixel) artefacts that appear once in a while but seem to be random or based on something that only occurs on some frames as they disappear on the next frame. Not sure if it's caused by the changes I made or something unrelated to this issue.

[robhawkes]

I've done some more testing and can confirm that the artefacts that I was seeing are unrelated to Three.js and the changes that I made to the bloom shader, so that's good.

[mrdoob]

/ping @spidersharma03

[spidersharma03]

I will see the results with a colored scene and come back.

[spidersharma03]

@robhawkes , I see that you initialize the alphaSum to 0. Did you simply try making something like this::

vec4 diffuseSum = texture2D( colorTexture, vUv)* weightSum;\ ... ... diffuseSum /= weightSum;

[spidersharma03]

@robhawkes , basically something like this::

                                    float fSigma = float(SIGMA);
                float weightSum = gaussianPdf(0.0, fSigma);
                vec4 diffuseSum = texture2D( colorTexture, vUv) * weightSum;
                for( int i = 1; i < KERNEL_RADIUS; i ++ ) {
                    float x = float(i);
                    float w = gaussianPdf(x, fSigma);
                    vec2 uvOffset = direction * invSize * x;
                    vec4 sample1 = texture2D( colorTexture, vUv + uvOffset);
                    vec4 sample2 = texture2D( colorTexture, vUv - uvOffset);
                    diffuseSum += (sample1 + sample2) * w;
                    weightSum += 2.0 * w;
                }
                gl_FragColor = vec4(diffuseSum/weightSum);

I need to test it though

[simongloor]

I tested the last two adjustments.

The version of @robhawkes turns grayscale in very bright areas.

The version of @spidersharma03 comes pretty close to the opaque version when the background is black, but it fades the color twice and doesn't work on bright backgrounds:

I think the problem is that the actual geometry should replace the background of the canvas while the glow should actually be added additive on top of both.

[revyTH]

Tidy things up a bit and it seems to work just as well:

void main() {\n\
  vec2 invSize = 1.0 / texSize;\
  float fSigma = float(SIGMA);\
  float weightSum = gaussianPdf(0.0, fSigma);\
  float alphaSum = 0.0;\
  vec3 diffuseSum = texture2D( colorTexture, vUv).rgb * weightSum;\
  for( int i = 1; i < KERNEL_RADIUS; i ++ ) {\
    float x = float(i);\
    float w = gaussianPdf(x, fSigma);\
    vec2 uvOffset = direction * invSize * x;\
    vec4 sample1 = texture2D( colorTexture, vUv + uvOffset);\
    vec4 sample2 = texture2D( colorTexture, vUv - uvOffset);\
    diffuseSum += (sample1.rgb + sample2.rgb) * w;\
    alphaSum += (sample1.a + sample2.a) * w;\
    weightSum += 2.0 * w;\
  }\
  gl_FragColor = vec4(diffuseSum/weightSum, alphaSum/weightSum);\n\
}

I've noticed some little (single pixel) artefacts that appear once in a while but seem to be random or based on something that only occurs on some frames as they disappear on the next frame. Not sure if it's caused by the changes I made or something unrelated to this issue.

Hi guys, after changes from 101 to 102 it seems that the change on the fragment shader does not work anymore. Probably related to the change "Removed renderTarget and forceClear parameters from WebGLRenderer.render(). Please use .setRenderTarget() and .clear() instead before you perform the rendering. Be aware that it's now necessary to execute renderer.setRenderTarget( null ) in order to unset an active render target."

Any thought?

[robhawkes]

I'm slowly upgrading all my old experiments to work with r108 and after the comment by @revyTH I was worried my workaround for transparency in the bloom shader wasn't going to work any more. I'm unsure if something was actually broken with this in r102 but when I just added the same modified fragment shader I posted previously everything seems to work OK in r108.

For the avoidance of doubt, the background in the Three.js renderer in the screenshots is transparent, and overlaid over a Mapbox GL JS map. Without the modified fragment shader the bloom shader is opaque and you cannot see the map behind.

[robhawkes]

And for comparison, this is the same scene (slightly rotated) with the alternative approach suggest by @spidersharma03

[native1989]

@robhawkes Can you share this working example with mapbox map?

[fuzhenn]

I found a simpler solution:

It doesn't need to take alpha into account during gaussian blur.
At the final step, when combining bloom texture with the base texture, try to caculate an approximate alpha value for bloom rgb.

void main() {
    vec4 baseColor = getTexture( baseTexture );
    vec3 bloom = getTexture( bloomTexture ).rgb;
    //approximate alpha for bloom pixel when baseColor.a is 0 (transparent)
    //you can adjust it to make it more intense or softer
    float bloomAlpha = sqrt((bloom.r + bloom.g + bloom.b) / 3.0);
    //only use bloomAlpha when baseColor.a is 0
    float alpha = mix(bloomAlpha, baseColor.a, sign(baseColor.a));
    gl_FragColor = vec4(baseColor.rgb + bloom, alpha);

    // old code:
    // gl_FragColor = ( getTexture( baseTexture ) + vec4( 1.0 ) * getTexture( bloomTexture ) );
}

rendering result seems to be promising:

[valentincognito]

@fuzhenn That's the only modification you brought to the shader and you were able to set the renderer to alpha true and use a CSS background ?

[MagicBYang]

I'm slowly upgrading all my old experiments to work with r108 and after the comment by @revyTH I was worried my workaround for transparency in the bloom shader wasn't going to work any more. I'm unsure if something was actually broken with this in r102 but when I just added the same modified fragment shader I posted previously everything seems to work OK in r108.

For the avoidance of doubt, the background in the Three.js renderer in the screenshots is transparent, and overlaid over a Mapbox GL JS map. Without the modified fragment shader the bloom shader is opaque and you cannot see the map behind.

Thanks a lot for the mapbox example.
I try the same thing for a few days but always got a white untransparent scene above the mapbox map when I add your shader code.
So I guess the problem is the version of three.js. I tried r116dev and r108.r108 could not work at all
with a total white scene.r116dev display like below:

I stop render the unrealbloom pass the map show up again:

I cant find whats going on here if you still work in this field please show any clue to me if possible.
I saw a few similar issue in mapboxgl.js project too.

[gisonjourney]

I'm slowly upgrading all my old experiments to work with r108 and after the comment by @revyTH I was worried my workaround for transparency in the bloom shader wasn't going to work any more. I'm unsure if something was actually broken with this in r102 but when I just added the same modified fragment shader I posted previously everything seems to work OK in r108.
For the avoidance of doubt, the background in the Three.js renderer in the screenshots is transparent, and overlaid over a Mapbox GL JS map. Without the modified fragment shader the bloom shader is opaque and you cannot see the map behind.

Thanks a lot for the mapbox example.
I try the same thing for a few days but always got a white untransparent scene above the mapbox map when I add your shader code.
So I guess the problem is the version of three.js. I tried r116dev and r108.r108 could not work at all
with a total white scene.r116dev display like below:

I stop render the unrealbloom pass the map show up again:

I cant find whats going on here if you still work in this field please show any clue to me if possible.
I saw a few similar issue in mapboxgl.js project too.

hello,have you resolved this problem about unreal bloom of mapbox,can you give me some resolutions

[gisonjourney]

@robhawkes Can you share this working example with mapbox map?

[gisonjourney]

@robhawkes Can you share this working example with mapbox map?

hi，do you have any solutions about mapbox to share with me