Skiasharp: [QUESTION] SkRuntimeEffect and making the image background transparent.

[Surfrat]

Hi,

I wish to make an image background transparent and have the following solution as suggested on the Skia Google group:

https://jsfiddle.skia.org/canvaskit/12da8034d9fbf655956ba745d76acc9b9dc053ae78e468256624400b8a1015cb

https://groups.google.com/forum/#!topic/skia-discuss/gjr5-ZpSX80

This solution works very well and I am in the process of converting this to use SkiaSharp.

I ran into a problem with SkRuntimeEffect as it does not seem to be available?

My question's are as follows:

1. Is SkRuntimeEffect available and if not is their an alternative?
2. Is there an existing SkiaSharp solution for doing this? Simply put I want to pick a color in an image and replace it with transparency.

Here is the Skia code:

// #slider0:threshold #slider1:exponent
// Format Code shortcut (Shift + Alt + F) 

const surface = CanvasKit.MakeCanvasSurface(canvas.id);
if (!surface) {
    throw 'Could not make surface';
}

slider1.value = 0.2
slider0.value = 0.4

const getExponent = () => slider1.valueAsNumber * 15;
const getThreshold = () => slider0.valueAsNumber * 3;

const skcanvas = surface.getCanvas();
const paint = new CanvasKit.SkPaint();


const prog = `
 in fragmentProcessor color_map;
 uniform float scale;
 uniform half exp;
 uniform float3 in_colors0;

 void main(float2 p, inout half4 color) {
    half4 texColor = sample(color_map, p);
    if(length(abs(in_colors0 - pow(texColor.rgb, half3(exp) ) )) < scale) discard;
    color = texColor;
 }`;

function main(textureImgData) {
    const context = CanvasKit.currentContext();
    const textureShader = CanvasKit.MakeImageFromEncoded(textureImgData).makeShader(
        CanvasKit.TileMode.Clamp, CanvasKit.TileMode.Clamp);

    const fact = CanvasKit.SkRuntimeEffect.Make(prog);

    function drawFrame() {
        CanvasKit.setCurrentContext(context);
        skcanvas.clear(CanvasKit.BLACK);

        const shader = fact.makeShaderWithChildren([getThreshold(), getExponent(), 1.0, 1.0, 1.0], true, [textureShader]);
        paint.setShader(shader);

        skcanvas.drawRect(CanvasKit.LTRBRect(0, 0, 400, 400), paint);
        surface.flush();
        requestAnimationFrame(drawFrame);
        shader.delete();
    }
    requestAnimationFrame(drawFrame);

}


const loadBrick = fetch(

    'https://cdn.shopify.com/s/files/1/0161/0482/products/AyeGear_T5_tshirt_with_pockets_clothing_fashion_style_hiking_fishing_cycling_scottevest_pickpocket_proof_concealed5.jpg?v=1502127434')
    .then((response) => response.arrayBuffer())
    .then(main);

Thanks

Mark.

[mattleibow]

The work to expose SKRuntimeEffect was started in: https://github.com/mono/SkiaSharp/pull/1321

As soon as the package builds, then you can test a few things and see if it is working as expected.

[mattleibow]

I have been fiddling around here and I think I got something that is pretty nice. Let me know your thoughts on the API:

// input values
SKCanvas canvas = ...;
float threshold = 1.05f;
float exponent = 1.5f;

// shader
var src = @"
in fragmentProcessor color_map;

uniform float scale;
uniform half exp;
uniform float3 in_colors0;

void main(float2 p, inout half4 color) {
    half4 texColor = sample(color_map, p);
    if (length(abs(in_colors0 - pow(texColor.rgb, half3(exp)))) < scale)
        discard;
    color = texColor;
}";
using var effect = SKRuntimeEffect.Create(src, out var errorText);

// input values
var inputs = new SKRuntimeEffectInputs(effect);
inputs.Set("scale", threshold);
inputs.Set("exp", exponent);
inputs.Set("in_colors0", new[] { 1f, 1f, 1f });

// shader values
using var blueShirt = SKImage.FromEncodedData(Path.Combine(PathToImages, "blue-shirt.jpg"));
using var textureShader = blueShirt.ToShader();
var children = new SKRuntimeEffectChildren(effect);
children.Set("color_map", textureShader);

// create actual shader
using var shader = effect.ToShader(inputs, children, true);

// draw as normal
canvas.Clear(SKColors.Black);
using var paint = new SKPaint { Shader = shader };
canvas.DrawRect(SKRect.Create(400, 400), paint);

[Ziriax]

This looks amazing!

The only think I would change is easier constructs to bind the uniforms:

var inputs = new SKRuntimeEffectInputs(effect);
inputs["scale"] = threshold;
inputs["exp"] = exponent;
inputs["colors0"] = new[] { 1f, 1f, 1f };

or

var inputs = new SKRuntimeEffectInputs(effect) 
{
      ["scale"] = threshold,
      ["exp"] = exponent,
      ["colors0"] = new[] { 1f, 1f, 1f }
}

And maybe allow passing Color instead of a array, and maybe System.Numerics.VectorX?

[mattleibow]

@Ziriax Thanks for the feedback. I updated the code to use the much better indexer and/or the dictionary initializer.

With regards to the color/vectorx, I need to finish binding the other math types that is used by skia. The are obsoleting/removing a few of the more clunky ones and may be pretty useful to investigate whether the system versions are compatible.

[Ziriax]

You are doing so much work at such a fast pace, amazing.

[Surfrat]

@mattleibow this is awesome. Super excited to start testing later today. Many thanks.

[h82258652]

It seems the SKRumtimeEffect depends on the surface. I use it with the UWP SKXamlCanvas and the app crashed. But I use SKSwapChainPanel, it works well. :confused:

[mattleibow]

Ah, yeah. That is correct for now.

The current build only has the GPU support. I need to fix up some compiler issues and dependencies for the CPU interpreter for the effects.

[h82258652]

Great! I think it's time for me to learn SKSL. Thanks for your job on SkiaSharp.