Html: Allow workers & shared workers to be created within a service worker

[jakearchibald]

At the moment, a (shared)worker's lifecycle is linked to related documents, but these should be creatable within a service worker to do things such as image processing, expensive diffing etc without locking up the service worker.

I imagine this means changing the worker's documents to be the worker's clients, where clients can be documents or service workers.

Previous discussion https://github.com/slightlyoff/ServiceWorker/issues/678

[jakearchibald]

+@domenic as you've been looking into workers recently

[wanderview]

I guess I'd like to know how realistic it is that other browsers will implement this. As far as I know, gecko is the only browser that has implemented nested workers so far.

[adamvy-google]

Allowing SharedWorker access from ServiceWorkers would make it at lot more feasible to provide robust offline support with background sync.

In any application with a non-trivial data model, it's pretty typical to layer business logic and caching on top of a raw data store like IDB. Without support for SharedWorkers, it's hard to provide a single source of truth to the applications UI components. Business logic will have to be duplicated between the ServiceWorker and SharedWorkers or UI processes, making it easy to introduce inconsistencies and race conditions.

[rektide]

There is a rather sizable amount of discussion in https://github.com/w3c/ServiceWorker/issues/756 where people seem to think the alternative to what is proposed at the top of the thread - spinning up multiple service workers to prevent blocking - is to have service workers call out to workers to perform work. And based on a twitter discussion, I think this not being the case violates a lot of people's expectations.

An example use case could be https://github.com/w3c/ServiceWorker/issues/744, where someone discusses using BPG image-format-decoding in a ServiceWorker. They ought be able to offload this work into a worker.

[wanderview]

I've thought about just moving ahead with this in Firefox since its not that hard to adapt our current nested worker code to SW. While it might see limited use without Chrome also implementing, it is easily feature detectable and could enable some niche applications to be better implemented.

@jakearchibald How would you feel about us experimenting here?

[wanderview]

Of course, the html spec marks Worker and SharedWorker as [Exposed=(Window,Worker)] so maybe this is already spec'd.

[mkruisselbrink]

The part that isn't spec'd is how the lifetime of a worker would be tied to the lifetime of a service worker. Currently their lifetime is fully defined in terms of associated documents.

[domenic]

Yeah, HTML already does its part here; per step 5 of https://html.spec.whatwg.org/multipage/workers.html#run-a-worker this is supported. But if we want to add something like terminating a worker when the service worker terminates, the service worker spec would need to add appropriate steps.

[wanderview]

Why wouldn't the Worker/SharedWorker GC when the ServiceWorker terminates handle that? Seems equivalent to me.

Edit: Nevermind. I see how it explicitly uses Documents here:

https://html.spec.whatwg.org/multipage/workers.html#the-worker's-lifetime

[wanderview]

Seems like all we need is:

1. Define a "worker parent" that can be either a document or a worker.
2. Re-write the worker lifecycle section to reference "worker parent" instead of document.

[annevk]

I'm a little hesitant about exposing shared workers in more places without commitment from other browsers to support them in the first place. Exposing dedicated workers in service workers seems like a good thing though.

[jakearchibald]

@wanderview sounds like a good idea to me. The way you want to implement this sounds like what I proposed in the OP

I imagine this means changing the worker's documents to be the worker's clients, where clients can be documents or service workers.

[annevk]

315 is about potentially removing shared workers. It seems we're still at a bit of a stalemate there, but hopefully Edge and Safari will come around.

Then there are some other issues:

1. In current implementations the parent of a shared worker can only be a document. Per the specification it can also be a worker, but that's not implemented. Is that not a significant amount of work to change? (If that changes we also need to consider what that means for the definition of "agent cluster", though I suspect I will have to solve that since the SharedArrayBuffer folks didn't...)
2. The lifetime as pointed out above by @wanderview will need various changes to make this work. We'll need to change "the worker's Documents" to "the worker's owners" and make changes to all the relevant algorithms. (Given that ServiceWorkerGlobalScope is a WorkerGlobalScope it's lifetime is already broken I suspect as it can't have any documents. I'm somewhat surprised this hasn't been caught yet. @jungkees?)

[domenic]

In current implementations the parent of a shared worker can only be a document. Per the specification it can also be a worker, but that's not implemented.

I thought it was implemented in Firefox?

[annevk]

No, not per @bakulf yesterday at least:

note that SharedWorkers and ServiceWorkers can be just the head of this chain of workers - basically a worker (any type) [cannot] create a Shared/Service worker.

[wanderview]

I have some long term plans to possibly replace SharedWorker owner with a primitive that essentially maps to the Client type (which in turn represents an environment). In our implementation this would then let the SharedWorker be owned by another Worker. (It would also buy us cross-process support, but that's less relevant here.)

There might be some oddness with the about:blank replacement case, though. It could work out, though, if a SharedWorker object in use by the about:blank is just GC'd when the replacement happens.

[annevk]

Okay, so the main todo items here are after #2520 lands:

1. Make sure service worker lifetime actually makes sense on top of the primitives defined in HTML.
2. Write tests, and in particular test the initial about:blank case and whether that causes workers to stop or not (I suspect implementations may differ in whether they use the document or global object as the owner).
3. We could maybe tie worker lifetimes to the global object instead. That would simplify the setup somewhat. Depends on tests and whether the churn is deemed to be worth it if all implementations would have to change.

[wanderview]

Probably beating a dead horse now, but I thought of another reason to switch SharedWorker/DedicatedWorker owners to globals instead of documents. Consider:

1. An iframe is dynamically created and left in initial about:blank
2. Creator runs: frame.contentWindow.myWorker = new frame.contentWindow.SharedWorker(url);
3. iframe.src attribute is set replacing the window while leaving the global intact

Under the current spec language I'm not sure what would happen here. The SharedWorker DOM objects still exists, but the SharedWorker thread is treating the old about:blank document as its owner, not the new document in the iframe.

If the SharedWorker treated the global as the owner then this would work cleanly. Under most use cases the SharedWorker DOM object will be GC'd with the document which will match current spec'd behavior. It also handles, however, this weird shared global case with about:blank replacement.

[wanderview]

One thing to think about when this is spec'd and implemented:

1. Service Worker foo creates a nested worker bar
2. bar's URL is covered by foo's scope and becomes controlled by foo
3. bar periodically issues fetch() requests triggering fetch events in foo keeping foo alive

It seems we would need to do the same propagation of lifetime budget for nested workers that we do for self-postmessage. The fetch events from a nested worker solely owned by the service worker should not grant new lifetime budget to that service worker.

As an added wrinkle, a shared worker could transition from being solely owned by the service worker to being shared with a window and back again.

Should be fun to implement and test.

[annevk]

I looked into the lifetime issue and it does seem like all user agents associated them with documents at the moment, given test.html:

<script src="/resources/testharness.js"></script>
<script src="/resources/testharnessreport.js"></script>
<iframe></iframe>
<script>
async_test(t => {
  const frame = self[0],
        worker = new frame.Worker("pong.js")
  worker.postMessage("hi")
  worker.onmessage = t.step_func(e => {
    if(e.data === "hi") {
      alert("navigating...")
      frame.frameElement.src = "/common/blank.html"
      frame.frameElement.onload = t.step_func(() => {
        worker.postMessage("navigated")
      })
    } else if(e.data === "navigated") {
      alert("still alive")
    }
  })
})
</script>

and pong.js:

onmessage = e => { postMessage(e.data) }

I also tested removing the iframe element and that has the same effect of the worker no longer telling you anything.

[eklem]

Any more news on this?