Two.js: Breaks on Web worker because it uses window instead of self

[michaschwab]

Hi Jono,
Your library looks promising and I'm eager to use it for rendering to canvas and mostly to WebGL. However, I render my canvas in a web worker using OffscreenCanvas, and it seems that this library breaks down in that environment because it makes too many assumptions. I've been trying for a while to get this to work.

One of the main issues is that window is assumed to be defined, as you assign either "window" or "global" to root. If you'd use "self" instead of, or as alternative to, "window", it would work fine in a web worker.

The library also tries to access the document in other places. For example, in line 4922 and 7961, it says:
canvas = (root.document ? root.document.createElement('canvas') : { getContext: _.identity });

Why don't you use the domElement here? Creating the canvas element fails in the worker, and then it ends up with the identity context, so that later, the library does calls like .clearRect() on the string '2d', which fails. I am also confused why the 2d rendering context is used to clear the canvas when the rendering is explicitly set to WebGL? It seems some kind of combination between 2d and WebGL context is used. Could you help me understand what's going on there, and what I could do to make it work without creating new elements in the library?

Thanks for your help!
Micha

[jonobr1]

Thanks for the message! You're looking to use OffscreenCanvas API as in this blog post?

https://developers.google.com/web/updates/2018/08/offscreen-canvas

There are a few issues that make that not possible with Two.js at the moment because the library relies on a few DOM calls:

1. Measuring text width
2. Loading bitmap imagery into the context
3. No reference to self as you mention

I've written spoofs compatible with Node, but yet to try with a WebWorker. I think clearing up those should yield working results. A quick way to test this is to build the app without the SVG Renderer, Sprite, ImageSequence, Texture, and Text classes.

To answer your question about why the WebGL Renderer has a canvas internally: this is because all the 2D shapes are drawn in Canvas2D and then uploaded to WebGL as a texture. There are only quads drawn in WebGL.

I'll make a note to address the window, global, self issue. Do you know an example library that handled it well? Maybe I can reference them for a good paradigm to enable this.

[michaschwab]

Thanks for your quick response and your help! I'm excited you're interested in this!

Yes, I'm using OffscreenCanvas as in that blog post. It works fine when I render the nodes on the canvas manually, but I also got it to work with Stardust (https://github.com/stardustjs/) [except I had to mock a "style" attribute on the OffscreenCanvas].

Do you know an example library that handled it well?

Maybe Stardust is a good starting point for the window, global, self issue. Looking at their source code, at https://stardustjs.github.io/stardust/v0.1.1/stardust.bundle.js, we can see that right at the top, line 2 or something, they have this:
if(typeof window!=="undefined"){g=window}else if(typeof global!=="undefined"){g=global}else if(typeof self!=="undefined"){g=self}else{g=this}
I'm not sure where it comes from, as I don't see this in their source code. It appears to be part of their bundling process with yarn or something. But it works well.

I'd be interested in using Two.js without the SVG renderer, sprice, ImageSequence, and Texture classes. Not using text is a bit harder for my field (data visualization), but it would still be a good start.

To answer your question about why the WebGL Renderer has a canvas internally: this is because all the 2D shapes are drawn in Canvas2D and then uploaded to WebGL as a texture. There are only quads drawn in WebGL.

That's very interesting, thanks! I'm still learning parts of WebGL.

Hope the self thing helps! I think I'll try building Two.js without the classes you mentioned.
Thanks,
Micha

[jonobr1]

Sorry one more tedious question.., I can't test this because I'm currently traveling. Does window or this throw an error in the WebWorker? Or is it simply undefined?

[michaschwab]

No problem!

console.log(window);
Uncaught ReferenceError: window is not defined

console.log(this);
{__esModule: true}

console.log(self);
DedicatedWorkerGlobalScope

[michaschwab]

To clarify:

console.log(this ? 'Evaluates to true' : 'Evaluates to false');
Evaluates to true

[jonobr1]

Awesome thanks. Try giving one of the built files in the latest commit on dev a try. Hopefully that fix works for your. You should get a warning about Two.Text measurements, but it shouldn't be blocking. If it still doesn't work then it's going to take some time to figure this out...

[michaschwab]

The startup works well with this fix!

The rendering still breaks unfortunately because of the issue I mentioned above. In line 6707: ctx.clearRect(..), "ctx" is just the string "2d" because root.document does not exist:

console.log(self.document)
undefined

So it can't create the canvas it uses to do 2d context stuff. I also don't think workers can create elements at all.

If Two.js had a way to accept a secondary canvas element that it uses to do 2d context stuff on, instead of creating it itself, then this could be passed to the worker just like the first one, with transferControlToOffscreen(). It could be passed just like domElement. What do you think?

[jonobr1]

Would supplying a reference to your canvas when constructing a new instance of Two.js suffice? An API around the lines of:
javascript var two = new Two({ type: Two.Types.webgl, domElement: canvasToBeRendered, offscreenElement: referenceCanvasForTextureTransfer });
Where canvasToBeRendered and referenceCanvasForTextureTransfer are both OffscreenCanvas elements?

[michaschwab]

Yeah I think that would be perfect!

[jonobr1]

That's up on dev for you. 👍 Hope it works out!

[jonobr1]

Did that work? Or do you have some example code I could try to debug myself?

[michaschwab]

Sorry I went home for the weekend so I haven't tried it yet, but I'm excited to give it a go today! If it doesn't work I'll provide an example. Thanks for building this so fast!!

[michaschwab]

Did you still want me to build it without the text etc classes? If so, I have not yet tried that again.

But using the normal two.js file, there seems to be a similar issue as before. It does receive the secondary OffscreenCanvas, but it doesn't use it for some parts and crashes when calling clearRect in a updateCanvas function. I uploaded an example on this gist: https://gist.github.com/michaschwab/0ff51ee36fbd0af9cd58a215d0672472 - you can download the code from here.
You can see it live here: https://bl.ocks.org/michaschwab/0ff51ee36fbd0af9cd58a215d0672472
Or fullscreen, you can see the error in the console here: https://bl.ocks.org/michaschwab/raw/0ff51ee36fbd0af9cd58a215d0672472/

To show that it's a problem with Two.js, I also included a few lines of sample drawing code that you can enable instead of using Two.js to see that drawing things works correctly in the worker.

[jonobr1]

This is awesome, thanks! I'll debug on my end and report back.

[jonobr1]

Okay, latest dev branch should have the fix and working. By the way, that is pretty crazy you can send an offscreen canvas to a WebWorker and then the image just updates.... o_O

Thanks for helping out! Curious to see what you make. Please don't hesitate to share when you're done.

[michaschwab]

Awesome, it works!! Thank you and great job.

Welcome to the crazy world of OffscreenCanvas :-)

I'll reach out to you via e-mail now to share what I'm working on!

[michaschwab]

I just realized, I only got it to work with canvas so far, not with WebGL. The code example I sent doesn't seem to work yet, with the error: "Failed to execute 'texImage2D' on 'WebGLRenderingContext': No function was found that matched the signature provided.". I updated the gist above to include the new version of Two.js, so you can see the updated error at https://bl.ocks.org/michaschwab/raw/0ff51ee36fbd0af9cd58a215d0672472/.

I'm not sure why texImage2D doesn't exist on the WebGLRenderingContext.

But it's already great that this now works for canvas!

[jonobr1]

Hmmm, my local test worked in WebGL and your blocks actually works for me too ( in latest Chrome Stable ). Where are you trying to execute this?

[michaschwab]

Oh that's odd! I just don't have texImage2D on WebGLRenderingContext for some reason. I'm using Chrome 69 on Ubuntu.

[jonobr1]

Linnnnnux, I think I should be able to SSH into a server and try. This might take me some time, but it sounds like it may be a Chrome issue... Not sure why texImage2D wouldn't exist. It's part of the WebGL spec.

[michaschwab]

Yeah, I think you more than did your job :) But odd that I haven't encountered this before.