Lottie-web: Offscreen Canvas support for better sandboxing and performance

[malaykeshav]

Since there is now support for OffscreenCanvas on the web, it would be great it lottie web players would support this.

OffscreenCanvas uses a WorkerGlobalScope which allows it access to a few APIs like importScripts which could be used to load other scripts into the worker. Since the worker script runs on a different thread, it provides some level of security.

An offscreen canvas paint support means, the player would have to remove all access to the DOM and window. To further improve security, the player script can also remove the use of XMLHttpRequest to load animation data and instead use the JSON animation data passed on from the main thread.

Our team currently has plans to fork the current web player and modify it with the above mentioned changes (have a working prototype with version 5.5.1) . This however means that whenever there are changes upstream (airbnb/lottie-web), it has to be manually merged.

Are there plans to support OffscreenCanvas or CSS PaintWorklets (which uses PaintWorkletGlobalScope for even more sandboxing)?

[bodymovin]

Hi, not for now, but I'm happy to help if you are looking into it.
Out of curiosity, besides possible performance improvements, what is the reason why you are interested in the player running in a sandboxed thread?

[malaykeshav]

Thanks. I already have a working version for v5.5.1. I was thinking about future upstream changes and reducing its frequency due to the manual changes involved. Unless you suggest a better way to do this that I can land it in here?

The 2 primary reasons are security and performance.Running third party libraries in a sandbox is safer and easier to get through security reviews.

[bodymovin]

Nice! Do you have a link to see what you've been working on?

[malaykeshav]

Here is a codepen of the worker script I have now. I am trying to clean it up further by removing lines and even functions that refer to document, window or make http requests. (Font and image loader mainly)

The 'lottie_worker.js' is the lottie canvas player version 5.5.1
At the end of the file is onmessage() that communicates with the main thread and plays the animation.

I believe this can be expanded to send as many canvases along with the lottie data to the same worker to produce a resulting image.

[bodymovin]

This is really cool!
I'll look into it soon.
If you make any more progress or have any questions meanwhile, let me know.

[DebVortex]

Hi there,

the company I'm working for uses a lot of canvases and for performance reasons, tries to move as much as possible of it into an offscreen canvas worker. We already do that with all our three.js code. Moving lottie there too would be amazing!

For the time being: @malaykeshav could you explain a bit more, what parts you changed? Would it be possible to import lottie directly into the worker and then e.g. override the non-worker-compatible code by overriding it with e.g. prototypes?

[malaykeshav]

@DebVortex I shared the script I have now in the previous comments. You can look at the parts I changed which mostly includes the code that refers to the document or makes any http requests to load text, fonts or images. The script can be loaded directly into the worker.

I suppose importing the lottie player as is and overriding with prototypes could also work. Have not tried this approach. Wouldn't the resulting script size be larger in this case?

[malaykeshav]

I created a basic patch that generates a worker.js and worker.min.js. Both of which can play lottie animations if a json data is passed to them from the main thread.

However I am sure there are better ways to override the methods and generate the final worker script. (My js is a bit rustic)

[transitive-bullshit]

This is really interesting work @malaykeshav 😄 I think it'd be great to get a proper PR for this that's disabled by default and accessible via a canvas renderer option like offscreen: true.

[DebVortex]

@transitive-bullshit +1

With e.g. fetch(url).then(res => res.blob()).then(create​Image​Bitmap).then(img => ...) we also could implement proper image (pre)loading.

[yisibl]

https://twitter.com/flackrw/status/1135714462546182144
https://github.com/flackr/lottie-web/commit/c9a3a1f2f4b571a4a09f0ea8d2fc3fa06ead781d

[pedromcunha]

@malaykeshav great work here, I'm currently trying to implement something similar in our web app and will use your fork as a prototype. One thing I noticed that could be improved though: we should return the lottie animation from the webworker back to the document after initialization in the webworker context. That way the webpage has some control over the animation (pausing, playing, destroying, getting animation data etc). What do you think?

[malaykeshav]

@pedromcunha I am planning to send messages to the worklet for such controls using the onmessage().

I will update the code as I add these controls here: https://cs.chromium.org/chromium/src/third_party/lottie

[pedromcunha]

Awesome. Glad to hear. I've been tweaking your lottie-canvas-worker.js and have a working prototype but unfortunately the canvas output is extremely blurry. Not sure if it's a side affect of the lottie files I'm using or my canvas settings. Have you experienced this yourself @malaykeshav or does it always come out sharp?

[malaykeshav]

Yes I have had that blurry issue too. The problem is that the player has no way of knowing the size of the canvas. So it picks the minimum size. To fix this I am taking the size from the Lottie animation data. But you could also pass in the dimensions from the main thread.

This also means that if the HTML container that has the canvas changes size, you would have to update it manually. At least that's my understanding. Let me know if you find a better alternative.

EDIT: player doesn't have a way to know the canvas size unless it is set before hand. So if you have width and height property set on the canvas element, it works.