Imagesharp: Resize: enable opting-out for alpha premultiplication

[ptasev]

Edit: this turned into a feature request for a ResizeOptions.PremultiplyAlpha property defaulting to true.

Adding up-for-grabs, since it's easy, here are steps for a potential community PR:
https://github.com/SixLabors/ImageSharp/issues/1498#issuecomment-758564510

---

Prerequisites

• [x] I have written a descriptive issue title
• [x] I have verified that I am running the latest version of ImageSharp
• [x] I have verified if the problem exist in both DEBUG and RELEASE mode
• [x] I have searched open and closed issues to ensure it has not already been reported

Description

When using Resize(int width, int height) I am getting very odd black bands in the output image. The results seems quite wrong. I am getting a different results when using GIMP's Bicubic Scale Image function.

Steps to Reproduce

I have attached sample code that results in this issue. I have included the input image in the sample, as well as the output I get on my PC for reference.

System Configuration

• ImageSharp version: 1.0.2
• Other ImageSharp packages and versions: N/A
• Environment (Operating system, version and so on): I am using Win 10 Pro 2004, with Ryzen 2600, and AMD Radeon R9 380.
• .NET Framework version: .NET 5.0
• Additional information: N/A

Input:

My ImageSharp output as seen in GIMP:

GIMP's bicubic scale:

ResizeBugProj.zip

[JimBobSquarePants]

Can you post a side by side comparison of the two outputs?

[ptasev]

I have updated the post.

[JimBobSquarePants]

I’m sorry but I still do not understand what the issue you are trying to highlight is? Is that second image showing the ImageSharp output image as black/white?

[ptasev]

The second image is the output I get from image sharp (yes, highlighting the issue) when using the following code:

using (var img = Image<Rgba32>.Load(@"input.tga"))
{
    img.Mutate(op => op.Resize(img.Width / 2, img.Height / 2));
    img.SaveAsPng(@"resizeOutput.png");
}

[JimBobSquarePants]

Is that a yes?

[ptasev]

Yes it is highlighting the issue. No, it is not black and white, but full 8-bit RGB display in GIMP.

[JimBobSquarePants]

Ok.... So that input image contains data that has an alpha component of zero. GIMP however is displaying that to you, ignoring that component so you can see the color data. (_Looking at the top/right panel it looks like you have explicitly told it to do so?_)

Here's the input image opened with Paint.NET. As you can see, the color picker highlights the pixel component values.

When performing a resizing operation with a pixel format that contains an alpha component the RGB component values must first be associated with (_premultiplied by_) the alpha component to ensure the output is correct. In your case, since the alpha component value is zero, that leads to all components becoming zero.

[R {222}, G {182}, B {140}, A{0}] = [R {222 * 0}, G {182 * 0}, B {140 * 0}, A{0}] = [R {0}, G {0}, B {0}, A{0}]

Due to the GIMP setup this means that GIMP is also displaying the resized pixels that originally contained zero RGBA components as black.

Please note that the returned output from ImageSharp is absolutely the correct behavior based upon the input pixel state.

If you want to ignore the alpha component then you should use Rgb24.

[ptasev]

Hmm, I'm not sure which is correct, but GIMP seems to give a different result when resizing. I understand that due to the alpha ImageSharp is 0-ing out certain parts of the texture. However it seems that GIMP does not do this in its built-in resize function. GIMP seems to resize RGB separately from the alpha channel. GIMPs end result does not 0 out the RGB channel due to the alpha (as can be see in the third picture in the OP).

I need to do some reading on what should really be expected.

This is having an effect on the game I'm working with. The alpha channel isn't meant to mask out parts of the texture, but instead is used to give information on where the game should put player color. This is an RTS game and by player color I mean that player one's warriors would have a blue highlight where the alpha mask is white, and player 2 would have a red highlight where the mask is white for example. I still need the RGB data preserved because everything that is not masked by white alpha channel should be displayed normally. I don't think this is uncommon behavior. From what I've seen when working with texture in GIMP and Photoshop and generating mipmaps for DDS files is that the RGB data is preserved and not cancelled out by the alpha.

It seems the only way for me to do this with ImageSharp then is to create two separate images. One Rgb24, and one L8 with the alpha. Resize them separately, and then put them back together in an Rgba32 image.

[ptasev]

The link you gave is helpful to understand the rationale in ImageSharp's behavior. It makes sense, but doesn't seem to be the desired effect for every scenario as I explained above. I'll try to understand the code of a tool like DirectXTex to see what they do for mipmap resizing/generation.

[antonfirsov]

@JimBobSquarePants we may probably consider adding a new property ResizeOptions.PremultiplyAlpha { get; set; } = true;. This would be a super cheap feature, might worth it even if it's extremely rare that users explicitly need it.

[JimBobSquarePants]

@antonfirsov yep, I’ve been considering that myself.

[ptasev]

Having that option would be great. I guess this turned into a feature request instead.

[antonfirsov]

@ptasev are you interested PR-ing this?

This is the list of things to be done:

• Extend ResizeOptions with the property
• Extend the non-generic ResizeProcessor with the property (take it's value from options)
• Use PixelConversionModifiers.None if this.definition.PremultiplyAlpha == false:
  https://github.com/SixLabors/ImageSharp/blob/5ab593f8079201fcafa97b4dfcc57918ad0fea81/src/ImageSharp/Processing/Processors/Transforms/Resize/ResizeProcessor%7BTPixel%7D.cs#L173-L174
• Add test coverage. This is the hardest thing, but I can help with the steps if needed.
  
  
  
  • Add a test similar to this one, using the "Kaboom" image with PremultiplyAlpha = false
    
    https://github.com/SixLabors/ImageSharp/blob/9b7df1373b2eaf2aeb327c896c6d8a893f203d5a/tests/ImageSharp.Tests/Processing/Processors/Transforms/ResizeTests.cs#L204-L217
  
  
  • Run the test, it will fail, but should save the expected output to tests\Images\ActualOutput\ResizeTests in Debug runs. If you (temporarily) copy the resulting image to tests\Images\External\ReferenceOutput\ResizeTests the test should succeed
  
  
  • I will do this: push the reference image to ImageSharp.Tests.Images, and update the submodules so the new reference image is visible for everyone
  
  

[ptasev]

Yeah, will do!