Pixi.js: Proper usage of multiple canvases

[luxaritas]

My use case: The main component of my application is handled in Pixi. I'd like to implement the UI using the DOM, which should be overlaid on top of the main application, but some UI elements contain dynamically rendered images which I would like to handle with Pixi. I'm considering handling this my using multiple canvases (one for the main portion of the app, and one for each UI image, or at least set of UI images). This brings up a couple questions:

1) Does this make sense in the first place?
2) How specifically should I go about doing this? Do I have one Pixi.Application per canvas? Couldn't that cause a "Too many active WebGL contexts" issue (and is there a way to work around that if so)?
3) Will this have significant performance impacts? Is there a method I should use in order to bring performance impacts to a viable level?

I have looked at #82 and #4327, but neither directly/completely address my question

[luxaritas]

Just realized I could use PIXI.extract.WebGLExtract.base64 to resolve this case, though still curious how multiple canvases are typically handled

[englercj]

Honestly you are getting into territory where it may be easier (and almost certainly more efficient) to just render your UI in pixi.

[luxaritas]

That’s what we currently do, and honestly it’s a bit of a mess. The layout code is awkward, and some things have to be rendered in the DOM anyways (text inputs, some legacy content stored on the server, etc) so it creates a bunch of layering/masking issues on top of that.

Having some sort of XML-based UI layout description or such (a la WPF, JavaFX, etc) would help I guess, but the fact that some elements have to be in HTML still makes it a bit awkward

[englercj]

Yes, rendering UI in pixi is currently non-trivial. And using extract to copy pixels out of a framebuffer, convert them to a string, convert that to pixels, and upload it to the gpu again under a different context is slow.

Unfortunately those are currently the tradeoffs you have.

As to your other questions:

How specifically should I go about doing this? Do I have one Pixi.Application per canvas?

You can have 0 PIXI.Application instances if you want to, it is just a helper. Completely up to you how to manage this. Application is just a container that creates a Renderer, Container, and starts a ticker for you. You need one WebGLRenderer per canvas, but if you use PIXI.Application to make it or make it yourself is up to you.

Couldn't that cause a "Too many active WebGL contexts" issue (and is there a way to work around that if so)?

Yes, creating multiple WebGLRenderers can (and often will) cause this error. Creating 2 for the lifetime of the page is probably OK. If you destroy and create more though that is where you get into trouble. The browser has a cap to how many contexts you can create, and it controls when old contexts are free. We do everything we can to tell the browser to free a context when you destroy a renderer, but at the end of the day we can't make it do it.

Will this have significant performance impacts?

Having two renderers will probably not negatively affect perceived performance on PC, but might have some impact on mobile. Always measure.

If you only have one renderer and use the extract method, then yes that will be very slow. Reading back from the GPU is slow, serializing to a string is slow, deserializing from a string is slow, and uploading to the GPU is slow. Doing them all is very slow.

Is there a method I should use in order to bring performance impacts to a viable level?

If you must have multiple renderers the best way will be for them to each have their own scene that they render. Don't render in one and copy it to the other, but instead just have two WebGLRenderers, each with a separate scene graph. You should be able to share PIXI.Texture and PIXI.BaseTexture objects between them, we do support resources being used in multiple renderers. But WebGL doesn't support a resource being shared between contexts, so internally we have to create a backing WebGL texture for each Renderer and upload the texture data for each one. This can cause increased GPU memory usage, and potentially texture swapping if there are many textures.

[luxaritas]

All very useful information, and makes a lot of sense - thank you!

Per the extract method, there'd be maybe a series of at a maximum 8 extractions happening on a user-initiated event, so it's not running per tick at least - not sure how slow is slow in that context, but that's always something that can be tested. This isn't something that could be done in a worker could it?

Might be interesting to look into using XML for generating a scene graph. Is this something that would be interesting for Pixi proper, or nah? (Edit: Just saw an example of using Pixi with Vue which looks interesting. Might examine that further)

[englercj]

This isn't something that could be done in a worker could it?

No unfortunately, WebGL APIs are not available in WebWorkers currently.

Might be interesting to look into using XML for generating a scene graph. Is this something that would be interesting for Pixi proper, or nah?

Not for the core, no. I think that is out of scope.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[Elyx0]

I know this is stale but as a time traveler what is the state of what happened in this discussion now? Can Pixi solve this nowadays? Are the workers still restricted?

[englercj]

I'd open a new thread for discussion :)

[Ridvanovskyy]

@englercj Thanks for good explanation in 2018.
I experienced the same problem with WebGL. Currently the browser limit is 16 contexts, but this number may differ from version, platform, and browser. I solved this problem by running forceCanvas. Canvas has no such limitations as it doesn't work with GPU, but creating more than 5-10 contexts is too much in my case and i have performance issues.

[hjujah]

We have sorted this out by using 1 canvas and extending PIXI to make special containers which are tracing specific html elements. In general we were passing the bounds of the DOM elements to those containers and then changing their Y position on scroll. So something like this -> container.position.y = domElementbounds.top + scrollTop. Of couse there is a lot of code around it, but it could be a good starting point for everyone who are wondering how to achieve that ;)

You can see it in action here:
https://synchronized.studio/

Also you might check this three.js example which is using pretty much the same approach:
https://github.com/marioecg/codrops-wave-motion