Angular-cli: How to work with web workers (use a web worker IN cli project)

[jbgarr]

Bug Report or Feature Request (mark with an x)

- [ ] bug report -> please search issues before submitting
- [X] feature request

Versions.

@angular/cli: 1.0.0
node: 6.9.1
os: darwin x64 (macOS Sierra 10.12.3)

Is there a way to import and use a web worker in a component using angular-cli? I'm trying to incorporate a web worker that I wrote into my project, but I'm not having any luck. The web worker worked fine in my project before attempting to convert it to use the CLI (previous version was based on System.js). I'm trying to import my web worker code like this:

import * as MyWorker from 'assets/workers/dropdown.worker.js';

and then I'm trying to create my actual worker instance like this:

let blob = new Blob(['(this.onmessage=', MyWorker.toString(), ')'], { type: "text/javascript" });
let dropdownWorker = new Worker((<any>window).URL.createObjectURL(blob));

The above does not work though, and if I log MyWorker right after I try to import it I see it's just an empty object instead of a function (which is what it is in my other app). Any help on this would be much appreciated!

UPDATE

I just want to clarify what I'm trying to do: I want to use an existing web worker IN my Angular app, not run my whole app IN a web worker. I know there is separate issue already that relates to the trying to run an entire app in a web worker. These are similar, but very different issues.

[bmayen]

Check out https://stackoverflow.com/questions/43276044/angular-cli-generated-app-with-web-workers

Would be really great to get this support by default!

[jbgarr]

Thanks a lot for the reply! I had seen that answer on Stackoverflow, but unfortunately they are trying to actually run their whole Angular app IN a web worker. I am trying to use an existing web worker in my app, so it's a little different. I appreciate the help though. Hopefully someone else can chime in here.

[maxime1992]

There's an opened issue here too : https://github.com/angular/angular-cli/issues/2305

[jbgarr]

@maxime1992 I appreciate your reply. That's a similar, but different issue. It's related to the Stackoverflow question that @bmayen linked to as well. Both of those deal with trying to run an entire Angular app IN a web worker, while I just want to use and run an existing web worker in my app.

[bmayen]

@jbgarr, just curious, why do you want to take this approach? The Angular team has done some amazing work supporting running the entire app in a worker. You might meet with less friction if your use case could allow for that approach instead.

[shlangus]

We were using an inline worker-loader, but now it does not work for us (see #4773). So now we are using file-loader.
It's a little uncomfortable since you have to build a worker before building an app, but it works.

[jbgarr]

@bmayen I have a section of my app that is potential performance bottleneck so I built a custom web worker to handle the heavy lifting on another thread, but I don't necessarily need/want to run my whole app in a worker. I built the original web worker for my app before converting over to the Angular CLI to build/bundle my app and I was hoping I could just use it as is without custom tailoring my build process.

@shlangus I had seen issue #4773 and was a bit disappointed. With your new approach are you having to use ng eject and build out some custom handling of your worker? If so, would you mind sharing whatever custom code you had to add in order to get it working? I was really hoping to avoid ng eject, but it may be required if I want to reuse my web worker as is. Thanks for any insight you can share!

[shlangus]

@jbgarr First of all you need to configure build process for your worker. To do that you can create a separate webpack config such as following. It uses ts-loader so you should install it too.

const config = {
  entry: './src/worker/worker.ts',
  output: {
    filename: './src/worker/index.js'
  },
  resolve: {
    extensions: ['.ts', '.tsx', '.js']
  },
  module: {
    loaders: [
      {test: /\.tsx?$/, loader: 'ts-loader'}
    ]
  }
};

module.exports = config;

Now you are able to run webpack --config path_to_worker_config. It makes sense to add this command as "prestart" script.

Once you have done it you need to get ability to include your worker into your angular app.
You should add "node" in types array in your tsconfig.app.json, then you can just require your worker:

const MyWorker = require('file-loader?name=worker.[hash:20].[ext]!../worker/index.js');
...
worker = new Worker(MyWorker);

This solution doesn't require to perform an ejecting but it also doesn't watch for changes in the worker.

[jbgarr]

@shlangus Thanks a lot for the explanation! I'll have to give that a try when I've got some time. The general idea makes sense. Thanks again, I appreciate it.

[filipesilva]

Heya, I added some information on how to currently use webworkers in https://github.com/angular/angular-cli/issues/2305. I know there's been some conversation here but that's the original issue, and it's better to concentrate discussion in a single one.

[jbgarr]

@filipesilva Thanks for the info and I had seen that issue already, but I think there is a core difference between what I'm asking/trying to do VS what that issue touches on. I am trying to use an existing web worker IN my Angular app, while that issue focuses on running an entire Angular app IN a web worker. I believe these are similar, but very different issues. Can you consider reopening this?

[maxime1992]

@filipesilva in the other issue you talked about service-worker and not web-worker :

Unless it's a typo, maybe you might reopen this (+cc @jbgarr comment)

[filipesilva]

@jbgarr @maxime1992 you're absolutely right, reopening.

[naveedahmed1]

@filipesilva can the steps mentioned in this post be integrated in cli?

https://medium.com/@enriqueoriol/angular-with-web-workers-step-by-step-dc11d5872135

Repo: https://github.com/kaikcreator/angular-cli-web-worker

Demo: https://kaikcreator.github.io/angular-cli-web-worker/

[filipesilva]

@naveedahmed1 Hm... I don't think we can do it in a seamless way, no. It would require app level switches to control the build behaviour.

What could perhaps be done in the interim is provide platformWorkerAppDynamic in @ngtools/webpack.

I can also say we're looking at supporting different platforms in the future (e.g. platform-server) and so platform-worker would also make sense.

[naveedahmed1]

@filipesilva glad to know about support for other platforms in future versions.

Meanwhile, what if the changes in app.module.ts and main.ts are left for the user while the cli:

• Could generate workerLoader.ts (it could be commented and user can uncomment it when required)
• Allow a switch to specify webworker as platform which makes appropriate changes to webpack build process

So, that if a user wants to add support for webworker, they just make changes to app.module.ts and main.ts and specify platform as webworker in build command.

Would it be possible?

[filipesilva]

@naveedahmed1 it would be possible, but it's not a very good idea from the CLI standpoint. It's better to properly provide platform support and a proper API for it, than just trying to get it to work ASAP.

For the moment, the eject based guide seems to be the best option.

[bmayen]

There are two issues being discussed in this thread. OP is asking about having individual webworkers from the context of an app running on the UI thread. Others are discussing what is being requested in #2305 , which is what the eject-based option attempts to solve.

@filipesilva, in general, I think the #2305 approach is most beneficial to the community as this is the official Angular way. Any chance we could get a prioritization of this feature or an idea of the timeline? Would be very helpful for planning when we can fully integrate CLI into our workflow... which we're desperate to do ASAP :)

[filipesilva]

@bmayen I don't have a timeline for it, no. It's something to be considered after platform-server lands, but even then it might not be a 1.x feature.

[naveedahmed1]

@bmayen for individual web workers there is a plugin which can be used https://github.com/haochi/angular2-web-worker

[bmayen]

Thanks @naveedahmed1. May still be of interest to the OP. I'm actually more concerned with #2305, personally. Going to hop out of this thread in favor of that one.

[jbgarr]

@naveedahmed1 Thanks for the plugin link. I'll definitely take a look.

[filipesilva]

Let me know if https://github.com/haochi/angular2-web-worker addresses your needs. The importScripts bit looks like it would work pretty well with a file in assets/.

[naveedahmed1]

@filipesilva should the eject based approach work with Lazyloading?

[filipesilva]

@naveedahmed1 yes, the webpack plugin supports it.

[jerryorta-dev]

Here is a proof of concept project for Angular 4 loading web workers:
web-worker-in-ng

@jbgarr, I was able to create an Angular 4.x project that implements two web workers, transpiles them, and hashes them in a build. No ng eject. The only "gotcha" I forgot to document in the project is the Web Worker JS file can not be the same name as it's TS file if they reside in the same directory. The webpack worker-loader gets confused and loads the ts file even when explicitely telling it to load the .js file.

[CalvinDale]

Considering that web workers are designed to offload computation to a background process, it's not clear why running an entire Angular application as a web worker is particularly beneficial. If everything is running in the same process, then what's the point? Another use case for @jbgarr 's original issue here (running a web-worker within an Angular app) is facilitating shared web workers. Among other uses, shared web workers can provide common access to an IndexedDB database for multiple browser tabs and/or iframes.

[maxime1992]

it's not clear why running an entire Angular application as a web worker is particularly beneficial. If everything is running in the same process, then what's the point?

@CalvinDale that's the thing. If you run Angular into a web worker, the "main" thread is kept free.
So if your app is huge and the change detection cycles are taking too much time for 60fps rendering (blocking the main thread), having the change detection cycle running in his own thread is a game changer. Because your UI is not freezing all the time and at worst, you'll get a small delay before having a refresh with latest information but no freeze and everything smooth.

[CalvinDale]

Interesting feedback @maxime1992. I hadn't considered the case of an extremely large site. The sites that I've worked on haven't been large enough to encounter this issue.

[bmayen]

Also, the way Angular implements this is almost entirely transparent; as long as you're not accessing DOM directly. Huge optimization for free. Managing multiple webworkers manually takes considerably more effort.

[naveedahmed1]

@bmayen thats awesome, do you have any idea how would be handle libraries like hammerjs in case of web workers?

[bmayen]

The problem is that in order to make this work transparently, you need to use Angular bindings. In that case Angular is smart enough to marshal calls between the worker thread and the DOM behind the scenes to update the DOM for you automatically. Outside of that use-case, you need to handle it yourself.

Not sure about hammerjs specifically, but we had an app which needed to update canvases. Had to write a layer to batch canvas calls into a serializable object, pass it over to the DOM thread, have a method there fetch the canvas context, and run the calls against it.

Similar issue when writing to localstorage, cookies, etc. Basically, anything on the window that you can't update via a template binding.

[CalvinDale]

I posted a question related to original feature request here on Stack Overflow.

[saurabhpathak9]

Just curious. https://github.com/haochi/angular2-web-worker worked for @jbgarr or anyone else running individual web workers inside angular app?

[jbgarr]

@saurabhpathak9 I actually never had a chance to try the plugin, so I'm not sure how it works. Hopefully some others can chime in and provide feedback.

[saurabhpathak9]

@jbgarr - Thanks for your reply. Somehow, haochi plugin didn't work for me. But I was able to make it work using @jerryorta-dev proof of concept code!

[saurabhpathak9]

All - For benefit for all, I created a sample project using ag-grid with high frequency updates on websocket, leveraging web-worker for reconnect to websocket with progressive interval. Hope it helps! Also, thanks again to @jerryorta-dev! Grid with Websocket and Webworker

[michaeljota]

I'm sorry for my ignorance if I'm missing something here, but I just read the code of angular2-web-worker and it only wraps a regular web worker in a promise to be consumed. So what's the problem about using regular web workers in the first place? You would only need to add the file in cli config apps[0].assets.web-worker.js won't you?

Yes, I know, you would need to write a Javascript file in an Angular app.

Still, maybe I AM missing something else here, but I don't see angular2-web-worker doing any magic. It is just loading a regular js file at the end, so why not just do that?

[rsohi]

@jbgarr - i came with a similar issue where in i have to use webworkers just for some complex functionality and not for the whole angular application in web workers.

I have used web workers as a service in angular 4 application by creating them on the fly using blobs.

You can check the code on https://github.com/rsohi/web_worker_angularCLI

[michaeljota]

@rsohi That's one of the things what angular2-web-worker does. So, again, maybe I'm missing something here, but I think a regular .js file for the web worker is acceptable.

[shlangus]

@michaeljota I was wondering if you could provide an example how to do what you suggest

[michaeljota]

@shlangus I'll try to setup something this weekend and see how it goes. Any idea what to use in a web worker?

[jerryorta-dev]

This may help:

https://github.com/UIUXEngineering/web-worker-in-ng

On Tue, Apr 17, 2018, 2:25 PM Michael De Abreu [e-mail]
wrote:

@shlangus https://github.com/shlangus I'll try to setup something this
weekend and see how it goes. Any idea what to use in a web worker?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/angular/angular-cli/issues/5885#issuecomment-382111827,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAU0o__3nxJqroYx__2Bqot-TwpsZv4Qks5tpkGTgaJpZM4M3gw8
.

>

Jerry Orta
UI / UX Engineer

[shlangus]

@jerryorta-dev Thank you, I've already seen checked this as well as all the solutions in this tread =) It uses an approach with separate webpack config and build step for worker. I described something like that at the beginning. But @michaeljota is talking about something different and I'm just curious to see one more approach

[jerryorta-dev]

My apologies @shlangus, didn't realize I've been on this thread from my phone. I am curious too if this can be done with the cli.

[michaeljota]

@shahata I actually like the work that @jerryorta-dev did. Still, I think that you can do the same with plain old Javascript, and avoid the additional webpack build process.

[creamylooks]

I had a similar issue a while back, here is how i solved it :
I added my worker files in the assets folder and referenced them within my angular service as follows:
const worker = new Worker('../assets/jsworkers/analyzer.js'); worker.postMessage({ x_data: mydata }); worker.addEventListener('message', function (e) {...})
Hope it helps someone.

[shlangus]

I have just reread the initial post and realized that it is not obvious how worker file is built there. So basically, if you worker is a js file you can use it in the way @creamylooks wrote and I suppose that is what @michaeljota was talking about. But, for example, my case is a worker contains dozens classes written in typescript and contains dependencies such as lodash, rxjs and so on. And I want to be able to process it (compile, minify) in the same way as it is done for whole angular app and that is what worker-loader do.

[FloNeu]

i was looking for a easy way to outsource some expensive functions from within my angular controlers etc. stumbled over this little gem... seems great for some use-cases:
http://www.eslinstructor.net/vkthread/

[klemenoslaj]

@jbgarr I think I have exactly the same problem.
I am migrating AngularJS application that had some web workers due to really heavy computations.

I found a workaround in webworkify-webpack package. I hope it helps.

However, I would really love to see the official support for "standalone" web workers from Angular CLI as well.

[sokolovstas]

@shlangus I have found a way to run it

app.ts

import GroupWorker from '!worker-loader!ts-loader!./group.worker';

typings.d.ts

declare module "!worker-loader*" {
  class WebpackWorker extends Worker {
    constructor();
  }

  export default WebpackWorker;
}

group.worker.ts

import { BoardSettings } from '!ts-loader!../model/board.model';
import { Issue } from '!ts-loader!../model/issue.model';

const ctx: Worker = self as any;
ctx.onmessage = (ev: MessageEvent) => {

But have some cache and compilation performance. Compiler always write:
ERROR in ../projects/web/src/app/board/group.worker.ts(1,31): error TS2307: Cannot find module '!awesome-typescript-loader!../model/board.model'.
../projects/web/src/app/board/group.worker.ts(2,23): error TS2307: Cannot find module '!awesome-typescript-loader!../model/issue.model'.

But:
chunk {main} main.js, main.js.map (main) 3.12 MB [initial] [rendered]
ℹ ｢wdm｣: Compiled successfully.

[dirkluijk]

This is my workaround.

Step 1. Add a custom webpack build to Angular CLI

Since Angular 6 you can use the Angular CLI for custom webpack builds.

Add to angular.json:

  "projects": {
    "your-worker-name": {
      // ...
      "architect": {
        // ...
        "build": {
          "builder": "@angular-devkit/build-webpack:webpack",
          "options": {
            "webpackConfig": "webpack.config.js"
          }
        }
      }

Create a simple webpack config, which builds the worker script to your dist directory, e.g.:

const path = require('path');

const config = {
  entry: './src/your-worker.js',
  mode: 'production',
  output: {
    path: path.resolve(__dirname, 'dist'),
    filename: 'your-worker.js'
  }
};

module.exports = config;

Step 2. include the generated worker script in your Angular app

In angular.json, add to your Angular project:

"scripts": [
  { "input": "dist/your-worker.js", "lazy": true }
]

Step 3: make sure to build your worker script first...

Just run:
$ ng run your-worker-name:build

And/or modify your package.json to run before ng serve and ng build:

"scripts": {
    "ng": "ng",
    "start": "ng run your-worker-name:build && ng serve",
    "build": "ng run your-worker-name:build && ng build",
    "test": "ng test",
    "lint": "ng lint",
    "e2e": "ng e2e"
  }

Step 4: profit!

export class AppComponent {
  ngOnInit(): void {
    const worker = new Worker('your-worker.js');
  }
}

Example

[michaeljota]

I will double check this, but I think that by using Webpack loaders, you'll be fine using Typescript files for the source of the worker.

[dirkluijk]

@michaeljota I did't try that yet. But I guess then you also need to include a typescript loader, as you have a custom webpack config. I didn't want to maintain a huge webpack config. ;)

[jerryorta-dev]

@dirkluijk That is a great solution and something I will use. As an experiment, I wonder how other new Angular 6 CLI features may enhance the architecture -- such as generating the worker code as another app or lib? That may over-engineer this solution for your app. It's just a thought for larger apps.

[sladiri]

@dirkluijk Thank you for your example. I do not see why the scripts array inside angular.json should contain the file? I write the compile worker.js into the assets folder, but maybe that is an anti-pattern in Angular? I can load it fine without its addition to the scripts array.

I have Typescript source, and while it does work, the compiler complains about main.ts accessing the document. That is weird and I could not get rid of the error, since the main.worker.ts does not import anything and does not accesss the DOM. Below is my TS config, where I removed DOM from the libs array.

This is from my project https://github.com/sladiri/web-worker-angular-v7

// webpack.base.config.js
const path = require("path");

module.exports = ({ production = false, ie11 = false }) => ({
  mode: production ? "production" : "development",
  entry: "./src/main.worker.ts",
  // entry: ["../src/polyfills.ts", "./worker/main.worker.ts"],
  output: {
    path: path.resolve(__dirname, "assets", `webworkers${ie11 ? "-ie11" : ""}`),
    filename: "main.worker.js",
  },
  resolve: { extensions: [".ts"] },
  module: {
    rules: [
      {
        test: /\.ts$/,
        loader: "ts-loader",
        options: {
          compilerOptions: {
            baseUrl: "./",
            outDir: `./dist/out-tsc/worker${ie11 ? "-ie11" : ""}`,
            sourceMap: true,
            declaration: false,
            module: "esnext",
            moduleResolution: "node",
            emitDecoratorMetadata: true,
            experimentalDecorators: true,
            target: ie11 ? "es5" : "es2015",
            typeRoots: ["node_modules/@types"],
            lib: ["es2018", "esnext.asynciterable", "webworker"],
          },
        },
      },
    ],
  },
});

[chrillewoodz]

There is actually a package for this:

https://www.npmjs.com/package/ngx-web-worker

I've tried it and it works as far as I can tell.

[mboughaba]

@dirkluijk great and quick solution 👏👏👏

I am wondering if you tried with contenthash or do you have any thought about it

[kiranjholla]

I have tried implementing a version of the solution given by @jerryorta-dev into an Angular 6 library. I used the worker-loader! Webpack Loader hoping to get the Worker as part of a reusable functionality I am packing into an Angular Library.

That approach just wouldn't work. Finally I found this: https://github.com/angular/angular-cli/issues/5885#issuecomment-302802124. Angular Libraries are packaged using ngpackgr which does not use Webpack and hence there is no support for loaders.

What's more, they also discuss the possibility of Angular CLI moving away from Webpack completely. That means that there would be no supported means to deploy standalone web-workers in Angular CLI. All discussion tends to point to the fact that the entire Angular Application can be run in a web-worker. However, that approach wouldn't work if we need to use some libraries (example: D3) which need access to the DOM.

[michaeljota]

@kiranjholla What that comments points out is that actually Angular CLI has moved from a building system in the past (From SystemJS to Webpack) and the CLI is all about abstract the build job in a way that the developer can have a consisten behavior.

The solution @dirkluijk post in his comment should work, _as long as Angular implement Webpack_.

However, I really doubt that they are moving to another building system any time soon. Even if they did, as this is now, I'm sure this would still work, as you are just using another builder task.

[dirkluijk]

Exactly. When Angular will replace webpack for another tool, the CLI will still support it by builders like @angular-devkit/build-webpack.

[kiranjholla]

@michaeljota @dirkluijk That's a fair point. Maybe I misinterpreted the comment and jumped the gun.

However, I like the solution that uses worker-loader because of output hashing. The worker.js file is also hashed and I don't need to worry about manual cache-busting. With ng-packagr, I can no longer use worker-loader to load my worker scripts.

Are you aware of any automated way that I could include a output hash in the worker.js file while importing it using the new Worker('hash.worker.js') approach? Any pointers?

Thanks.

[mboughaba]

@kiranjholla

I already asked this question few comments above... https://github.com/angular/angular-cli/issues/5885#issuecomment-428346967

Also tried on StackOverflow and angular/cli but without luck.

That said, considering that from what I am understanding you have 2 builds, there would be no way to reference the hashed file in the previous file.
https://github.com/angular/angular-cli/issues/12582

In case you find a solution don't hesitate to give feedback here

[shlangus]

@kiranjholla @mboughaba Have you tried https://github.com/angular/angular-cli/issues/5885#issuecomment-295153316?

[mboughaba]

@shlangus Thanks very much for the suggestion! I managed to get it working using your method
This is how I require my file

const StorageWorker: any = require('file-loader?name=storage.[hash:20].[ext]!../../../../dist/storage.js');
// [...]
// ctor
   this.worker = new Worker(StorageWorker);

On the config side, I have the following

const config = {
  entry: {
    storage: './src/worker/storage.ts'
  },
  mode: 'production',
  watch: false,
  module: {
    rules: [
      {
        test: /\.ts$/,
        loader: 'awesome-typescript-loader',
        query: {
          configFileName: './src/tsconfig.worker.json'
        }
      }
    ]
  },
  output: {
    path: path.resolve(__dirname, 'dist-worker'),
    filename: '[name].[contenthash].js'
  }
};

There is a lot of magic, but hey it works!!

I spoke too fast 😞 ... This was loading the storage.js which was left over in the dist folder... As soon as I run the prod build which uses CleanWebpackPlugin.

[julmot]

You might be interested in using inline web workers to avoid the overhead of having to create a separate Webpack configuration with Typescript etc.:
https://www.html5rocks.com/en/tutorials/workers/basics/#toc-inlineworkers

[mhosman]

It could be great to create a clean repository with a full working example of an Angular 7 app with Lazy-Loading, running entirely in a web-worker.

[dirkluijk]

@mhosman This issue is about using web workers IN an Angular app. Not running Angular itself in a worker.

For the latter, see https://legacy-to-the-edge.com/2018/05/01/platform-webworker-in-angular/ (I just googled for it)

[mhosman]

Thanks @dirkluijk, my problem is that when I'm loading modules, loaders and the UI itself is stucked for a couple of seconds. I need a way to run the logic in the background in order to get the animations only working in the main thread.

[filipesilva]

There's ongoing work to add https://github.com/googlechromelabs/worker-plugin in https://github.com/angular/angular-cli/pull/12575.

[mhosman]

One of the problems is when you load modules and material2 animations get stuck.

[angular-automatic-lock-bot[bot]]

This issue has been automatically locked due to inactivity.
Please file a new issue if you are encountering a similar or related problem.

Read more about our automatic conversation locking policy.

_{_This action has been performed automatically by a bot._}