P5.js: Performance improvement for tint

[Ajayneethikannan]

Nature of issue?

• [ ] Found a bug
• [x] Existing feature enhancement
• [ ] New feature request

Most appropriate sub-area of p5.js?

• [x] Core/Environment/Rendering

Which platform were you using when you encountered this?

• [x] Mobile/Tablet (touch devices)
• [x] Desktop/Laptop
• [x] Others (specify if possible) Device independent

Details about the bug:

• p5.js version: 0.7.3
• Web browser and version: Browser independent
• Operating System: OS independent
• Steps to reproduce this:
  
  https://editor.p5js.org/TheSkepticSniper/sketches/go2nvAoAF
  This sketch can be used for checking the performance of tint.
  (Uncomment the tint line to see the frame rate drop)

Feature enhancement details:

Currently , when the user sets a tint to draw images , the sketch suffers a huge reduction in performance ,as the function _getTintedCanvas runs every time any image is drawn using the p5.Renderer2d.prototype.image function , manipulating every pixel of the image's canvas and returning a new tinted canvas ( here I think we can make a small speed improvement by using globalCompositeOperation = 'multiply' to create the tinted image, instead of manually multiplying every pixel with the tint ).

When many images are present , or high resolution images are used , this can lead to performance issues.
So curently, the safest place to use tint is only in setup , as the canvas is only drawn once and the tint calculations are done once.

New feature details:

Some ideas which can be implemented are:

1.Caching the previous tinted canvas:

For every p5.Image object, if we are drawing with a specific tint for that image,
we can store the last tinted image canvas, returned by

cnv = this._getTintedImageCanvas(img);

along with the p5.Image instance, as

img._prevTintedImage = cnv;
img._prevTint = tint;

where img refers to a p5.Image instance
In case the img hasn't been modified (which we can check) or the tint hasn't changed, we can reuse this canvas. Or we can calculated the new tinted canvas and store it here.
This method can be used when the image is not modified frequently, or the tint for the particular image is not changed frequently.

(But this method cannot be applied for video elements )

2.Ability to create new tinted image :

if we can add a functionality like

let tImage = img.tint(12, 121, 12);

we can draw the tinted images as new images, without calling _getTintedCanvas every time.

Would love to hear more from the community if this feature should be considered ,
Thank you!

[micuat]

you're right, I guess the current implementation of tint is quite useless in draw loop. One way of solving this without changing p5js is to use createGraphics
https://editor.p5js.org/micuat/sketches/9xhpkGVF8
which I believe is more intuitive (for a Processing user)

[Ajayneethikannan]

Wow , that's a nice work around @micuat !
If you think it is more intuitive , would this be a good feature implementation,

if we can add a functionality like

let tImage = img.tint(12, 121, 12);

we can draw the tinted images as new images, without calling _getTintedCanvas every time.

instead of using the tinted graphics instance, a new tinted image will be created, which can be used normally in the draw function.

Thank you @micuat !

[micuat]

and you can always hack the image function
https://editor.p5js.org/micuat/sketches/S82r42HqN

as you suggested, chaining these image filters is a cool functionality but I don't know if it's something needed for the community. I guess we need input from other people

[SrBrahma]

Same issue here. Tinting is consuming a nice part of the CPU.

[stalgiag]

I feel like @micuat's first suggestion is the intended way to do this. Maybe a documentation change could held nudge people in this direction? I feel like caching the canvas might be a bit too much for this specific problem.

[fcasanellas]

I would like to change the tint color during the draw(), any idea to improve tint performance on this scenario?

[feuerball11]

As another performance improvement, for alpha drawing only:
You can use
drawingContext.globalAlpha = 0.5;
before drawing the image, and restoring it with
drawingContext.globalAlpha = 1;
This allows for performant alpha drawing.

Maybe this could be patched into Processing with a function like
alpha(a)
noAlpha()
?

[micuat]

@fcasanellas I made a simple sketch of a real-time version using a shader. But it seems the texture coordinate is screwed (you see wireframe of the plane)

https://editor.p5js.org/micuat/sketches/OMRlanQW1

@stalgiag do you have any idea why this is happening? I guess gl_FragCoord does not follow the texture coordinate given by plane()? Do I need to explicitly set texCoord in vertex shader?

[b2renger]

@micuat

I usually do it like this, as you mentioned I usually do explicitly set the texCoord in vs :

https://editor.p5js.org/b2renger/sketches/Pu9PalyUH

Still it seems something is off with plane() so I used a rect instead and everything seems ok

[micuat]

thanks that works perfect. I don't know what the direction now for this thread, but I guess, from p5.js's policy, we won't integrate these hacks into the core. Nevertheless it may be useful to make a documentation about shader based tint (or caching tinted image if static) and point it from the reference.

[b2renger]

I agree, it could be nice to have small library of shaders to manage things like tint, blends or masks in a simple and efficient way - with comprehensive examples

Would love to participate if it were to happen :) I also have a few things ready to go :
https://b2renger.github.io/p5js-shaders/shader-blends/
https://b2renger.github.io/p5js-shaders/shader-mask/index.html

[quitmeyer]

Agreed, running into this same issue while trying to use FFT and tinting an image