Obs-studio: Images with alpha channel are incorrectly scaled

[marcan]

Platform

Operating system and version: All (tested on Arch Linux and macOS)
OBS Studio version: 25.0.8

Expected Behavior

Images with alpha channel should be correctly scaled, particularly along the transparency edges, without introducing extraneous colors.

Current Behavior

OBS currently scales alpha channel images incorrectly, introducing dark or colored edges.

Steps to Reproduce

1. Add a white color source
2. Add an Image source with this image.

Note that that image only has transparent and white pixels. Therefore, the resulting frame should be completely white, as all content is white on white.

1. Scale the image to any non-1:1 scale.

Result:

Additional information

This happens because OBS is currently taking the image with non-premultiplied alpha (as stored in the PNG), and scaling/sampling the color planes independently. This is mathematically incorrect: scaling non premultiplied alpha images is not separable.

The correct way to do this is to convert the image to premultiplied alpha, and then scale it (and composite it appropriately). This eliminates the artifacts and is mathematically correct.

Basically, if you ever have a texture with non-premultiplied alpha and you're sampling it at anything other than 1:1 scale with filtering, that's wrong. Any such instances in OBS should be replaced with a conversion to premultiplied alpha prior to sampling. Here's some background on the issue.

[jpark37]

TLDR known issue, painful to fix. I fixed several bad cases last year, but this particular path is nasty to fix cleanly because it would require touching a lot of code.

The test PNG is actually ambiguous. Transparent black and opaque white are 0/0/0/0 and 1/1/1/1 regardless of straight or premultiplied. The color becomes unambiguous when bilinear filtering is applied by the GPU, interpolating to a premultiplied gray.

Rendering a scaled transparent source directly to the canvas is a known buggy code path where the blend state expects straight alpha from the pixel shader but receives premultiplied. You can work around this issue by setting Scale Filtering to anything other than Default. The clean fix would touch a lot of code, and OBS has more severe color issues in the backlog that should be addressed first, e.g. applying alpha to linear colors.

[marcan]

The test PNG is a standard non-premultiplied image (as required by the PNG spec), which means the colors are x/x/x/0 and 1/1/1/1 (in this case x=0, but it could be anything and the output shouldn't change). You are correct that premultiplying it would still result in 0/0/0/0 1/1/1/1, but the point is that then the blending should happen by interpreting it as premultiplied, and in that case the standard texture sampling would do the right thing.

Indeed turning on scale filtering fixes the issue. The problem is not alpha type confusion, though, as you can see if you try using this image which has a 50% transparent section. The bulk of the section is blended properly, which means the blend state is consistent with the alpha type. The only problem is the edges, so the bug must be only that the sampling is done on straight alpha, not that straight alpha is confused for premultiplied anywhere.

[jpark37]

However you want to frame the problem, the issue is known, and it won't be addressed before the more important ones are fixed.

[marcan]

Sure, I'm just filing it because I didn't see a similar bug open.

[sobotka]

The test PNG is actually ambiguous. Transparent black and opaque white are 0/0/0/0 and 1/1/1/1 regardless of straight or premultiplied.

With associated (premultiplied) both occlusion and emission are expressed. Things like reflections, flares, glows, fires, etc. are entirely valid to be completely unoccluding, with alpha of zero, and emitting with nonzero RGB.

Common misconception with how alpha behaves.

[marcan]

Yup, that's one of the reasons why I think straight alpha is a bad idea (outside UIs). It doesn't let you express flare/bloom effects, while premultiplied does.

[jpark37]

I was just pointing out that even if I had performed an alpha multiplication on the original PNG data, it wouldn't fix the problem. (Obviously, since the data would be identical.) Doing the fix neatly with minimal hacks and without unnecessary draws is not trivial, and a workaround exists for the time being.

[alcomposer]

I have also experienced this issue for lower thirds alpha material. prores4444 media (associated alpha) does not composite correctly:

https://github.com/obsproject/obs-studio/pull/1852#issuecomment-637359558

[marcan]

@alcomposer that's a different problem (premultiplied media being interpreted as straight alpha). The problem this issue is about is that the image is interpreted properly as straight alpha, and blended as such, but it is also naively scaled as straight alpha with simple GPU texture sampling, which is incorrect (GPU texture sampling is always incorrect for straight alpha, except for special cases where you can manually prepare the texture to work well anyway).