Pixi.js: [Docs] Using Sampler2D to share JS typed arrays with custom shaders.

[carlsmith]

I'm new to shaders, and am really struggling to figure out how to use a Sampler2D to share raw data (not an image) with a frag shader.

I've put a pen online with a minimal example of passing an array of ints to a shader (as uniform sampler2D numbers), then updating the first value in the array during each animation frame. The code is commented throughout, and explains what the code is doing (and what it should be doing) in more detail.

Note: The pen uses PIXI V5, Vanilla JS, and the HTML and CSS files are empty.

I'd be really grateful If anyone would be kind enough to help me understand how to do this. Thank you.

[carlsmith]

Anyone?

[ivanpopelyshev]

@carlsmith hey! Since no one else responded, I have to take this issue too.

https://codepen.io/ivanpopelyshev/pen/rRjBXr

Uint8Array works just fine, here, I fixed it for you:

const rawdata = new Uint8Array([230, 149, 20, 103, 167, 236, 71, 186]); // Uint8Array
const numbers = new PIXI.Texture.fromBuffer(rawdata, rawdata.length/4, 1); // division by 4
const uniform = {numbers: numbers}; // That's correct uniforms, the ones you used were from older pixi

If you want to use Uint32Array, you have to specify other numbers in typed array, like "0xff00aa00". Also texImage2D will fail because pixi will try to submit Uint8Array as a format. There's a fix that should work only for webgl2 but in 5 minutes i spent on your issue I failed to use that.

numbers.baseTexture.type = WebGL2RenderingContext.UNSIGNED_INT;

Seems like i have to answer to every single issue about shaders :(

Here's where the magic happens: https://github.com/pixijs/pixi.js/blob/dev/packages/core/src/textures/resources/BufferResource.js#L70

[carlsmith]

Thank you, @ivanpopelyshev. You're a lifesaver.

There's a lot of full-stack devs that started programming in the last ten years or so, and struggled professionally just trying to stay on top of everything happening with the Web, the Cloud, and their favorite scripting languages and server frameworks.

Most of us have tried to go beyond script wizardry, but still have basically no professional experience with anything remotely similar to GPU programming. The situation should improve over time, gradually, but having people that understand this stuff and are willing to help the rest of us find our feet is invaluable.

There probably will be a steady flow of people needing help with shaders (and not enough people who can help them) for a while still, but your support really is appreciated. Making sure every user gets some help when they have an issue does make a big difference to how we feel about the project.

Thanks again, mate.

[carlsmith]

I just wanted to chuck up a complete example for anyone finding this in future. It's a bit too big to add to the API docs.

/* PIXI(v5): SHARING TYPED ARRAYS WITH CUSTOM SHADERS

This example illustrates sharing a `Uint8Array` as a uniform `sampler2D` with
a custom GLSL shader. Typed arrays must be converted to textures to be shared
with a shader. As a result, whenever we mutate the array, we need to let PIXI
know, so it can update the texture (else the GPU will not see the mutation).

PIXI defines a `Filter` class that makes it simple to apply custom shaders to
(the textures of) containers. We can create a `Sprite` from a blank `Texture`
that fills the canvas, and then apply a filter to the sprite.

The example shader uses the first four values (8-bit unsigned ints) from the
array (the first pixel in the texture) as an RGBA color, and makes the whole
canvas that color.

Whenever the canvas is clicked, the JavaScript code increases the first value
in the array, gradually changing the color of the canvas (from black to red).
*/

const {Sprite, Filter, Texture, Application} = PIXI;

const [width, height] = [128, 64];

const shaderSource = `
uniform sampler2D data; // the shared data

void main(void) {

    /* This main function just takes the color of the first data pixel (the
    first four bytes of the shared data) and uses it for every pixel in the
    output, making the canvas that color. */

    gl_FragColor = texture2D(data, vec2(0.0, 0.0));
}`;

// initialize a default pixi app, only setting the width and height...

const app = new Application({width, height});

// initialize the data that will be shared with the shader...

const pix = new Uint8Array(8 * 4);              // eight black (rgba) pixels
const data = new Texture.fromBuffer(pix, 8, 1); // the data as an 8x1 texture

// initialize the custom filter and the texture its applied to...

const filter = new Filter(undefined, shaderSource, {data});
const filterBuffer = new Uint8Array(width * height * 4);
const filterTexture = new Texture.fromBuffer(filterBuffer, width, height);
const filterSprite = new Sprite(filterTexture);

// plumb everything together...

filterSprite.filters = [filter];
app.stage.addChild(filterSprite);
document.body.appendChild(app.view);

document.body.addEventListener("click", function(event) {

    /* This callback handles clicks (on the canvas). It increases the value
    of the first byte in the typed array (alternatively, the red channel of
    the first pixel in the texture). */

    pix[0] += 16;  // increase the red level of the first data pixel
    data.update(); // let the gpu know that the texture needs updating
});