Tfjs: Add WASM backend

[nebrelbug]

TensorFlow.js version

1.0.4

Browser version

Chrome 73

Describe the problem or feature request

Has there been any further discussion about using WASM to optimize performance? I noticed there was discussion in #36, but it was decided to wait and eventually tfjs-node was created.

Possible Benefits of WASM

• Run in any WASM environment
• Maybe higher speed than WebGL if users have slow graphics?

[dsmilkov]

Hi, thanks for filing this issue. A WASM backend is on our roadmap for this quarter. We can use this issue for updates.

[startupgurukul]

Folks at intel have been making webml/webnn and recently made some amazing strides of hitting 31fps running coco SSD models.

https://github.com/intel/webml-polyfill/issues/650#issuecomment-475127456

They are implementing it natively mimicking the android NN API and for the changes to reach end user, they have to be merged into chromium source code.

Where as WASM backend would be awesome and I hope it can hit as much performance as they are able to

[startupgurukul]

Hi folks update on this, curious to see the performance compared to tflite and tensorflow

[dsmilkov]

Thanks for the reference. We are closely following the amazing work that Intel has done, led by @huningxin. Our wasm backend will most likely not be able to reach that level of performance since we won't be able to access hardware acceleration beyond SIMD 128bit, but it will provide other benefits, such as:

• acceleration for lower-end mobile devices that don't have webgl float support.
• acceleration on higher-end devices through SIMD and threading.
• consistent performance on cpu, agnostic of the JIT compilation.
• open an avenue for adding iterative linear algebra primitives such as SVD, eigenvectors, matrix inversion, that cannot be done efficiently on the GPU.

We will start developing our WASM backend in the open, on GitHub, very soon. Stay tuned!

[huningxin]

Thanks for the update @dsmilkov . As you know, webml-polyfill currently reuses tfjs-core WebGL kernels for GPU implementation and compiles tf-lite kernels to WASM for CPU implementation. It's a great news that tfjs will have a WASM backend soon. It would simplify the webml-polyfill and benefit the webnn spec development. Highly looking forward to that.

[chirva-ivan]

Hey, guys. Are there any updates on this issue?

[dsmilkov]

We just started by adding the framework (skeleton) code around it: https://github.com/tensorflow/tfjs-core/pull/1863 We will be slowly ramping up on this.

[camsjams]

While fully understanding that this a new and still experimental backend, are you planning on publishing the alpha/beta builds to npm?

[dsmilkov]

We are still very early in the process so publishing something to npm is not useful at this time. I'll ping this issue when we have more updates.

[bhack]

For out of browser WASI I just opened https://github.com/tensorflow/mlir/issues/140 in the MLIR repo.

[rthadur]

This has been done and should be available in latest releases.Thank you.

[tiziano88]

Great news! Which release has now Wasm support? And are there any instructions about it? Looking forward to trying it out.

[nsthorat]

This is actually still in progress we’ll update this when we’re ready to release :)

[Annaero]

This is actually still in progress we’ll update this when we’re ready to release :)

Hello, do you have any updates?

[dsmilkov]

Hi all,

We are excited to share that we just released our first alpha release of the WASM backend.
See the README.md in the tfjs-backend-wasm folder for how to use it. At this point we have about ~30 kernels - see all_kernels.ts for the list.

We would love to get your feedback, testing, and help as we are polishing towards a general release!

Thanks!

[dsmilkov]

Closing this issue since it's a general issue tracking our WASM release. Feel free to file separate issues if you run into problems.

[whatevergeek]

Thanks @dsmilkov
This is great news.

[bhack]

@dsmilkov Are you involved as team in https://bytecodealliance.org/?

[camsjams]

There appears to be a directory called wasm-out missing in the published npm version, I get an error:

Module not found: Error: Can't resolve '../wasm-out/tfjs-backend-wasm' in '/var/www/node_modules/@tensorflow/tfjs-backend-wasm/dist'

Here is a screenshot of installed items:

[dsmilkov]

Thanks for finding that out. We just released 1.4.0-alpha2 with the fix. Let us know how it goes.

[camsjams]

It looks to be working so far in the sense that our application hasn't crashed or thrown errors. I will you know how it goes :+1:

[startupgurukul]

Great new congrats on alpha,

Cameron, pls do share the performance improvements you notice during your experiments with wasm backend.

[chirva-ivan]

Did everything following the example from README and got an error. Do I need some special build of tfjs-core? Any Ideas?

[nsthorat]

@chirva-ivan which browser are you using? If you open the initialization error what does it say?

[chirva-ivan]

@nsthorat using Chrome 78

[nsthorat]

@chirva-ivan is it possible for you to stand up a demo for us to look at?

[dsmilkov]

Are you using Parcel? Can you share a small project/build script that we can reproduce? Thanks!

[yashints]

@dsmilkov Here is a codesandbox with above error:

https://codesandbox.io/s/naughty-montalcini-s0gjk

[yashints]

@dsmilkov I am still getting the error in the browser, was the fix just for node?

[dsmilkov]

I just sent https://github.com/tensorflow/tfjs/pull/2559 for review where I document how to use the WASM backend with bundlers and also added starter projects for webpack and parcel. See update readme from the PR.

Note also that you have to depend on the latest alpha 1.4.0-alpha3, and potentially we might need to release alpha4 since we made some updates to the browser field in package.json.