Serviceworker: DOMParser within ServiceWorkers

[markuskobler]

I'm trying to figure out the best way to parse HTML in response to a fetch request. In my case, I'm attempting to figure out any dependencies the HTML might have so I can also cache those assets as well

Is the only way todo this currently to use a horrendously complicated regex or am I missing something obvious?

Or put it another way what would be the downside of exposing the DOMParser to the ServiceWorker scope.

[annevk]

DOM implementations in browsers are not thread-safe. And don't use a regexp to parse HTML. You need to write an HTML parser if you want to do that.

[markuskobler]

Ok, that's useful to know. Is that the technical reason why document types are supported on XHR and not on fetch Response?

[domenic]

They aren't supported in web workers.

parse5 works in web workers, as does in fact large parts of the jsdom project (although that is a much larger dependency).

[annevk]

@markuskobler the reason fetch() doesn't support nodes is mostly because we didn't think a potential fetch module requiring all of DOM is a reasonable proposition. It's not really related to the DOM not being available in workers.

[markuskobler]

So in my case, I want the service worker to have a better understanding of the HTML it's caching so it can make a call on which of its dependencies should also be cached. I would have assumed this to be a common use case?

I only mentioned the DOMParser, not because I'm attached to that API, but because this feels like a core responsibility of the browser.

[wanderview]

If you want this done on the client I think you should just use read-through-caching. You can then just use <link rel="prefetch"> for resources you want to aggressively load/cache.

Alternatively, you can have your server pre-compute the list of resources and cache them in your service worker install event.

I think trying to parse html on the fly would be the least efficient way to go here.

Would either of those solutions work for you?

[inian]

I am looking to parse HTML received by the SW to replace links to individual JS files with a single link that combines all of them - for performance reasons, and then send it to the browser for parsing. I am thinking the performance gain from lesser RTTs might offset the extra overhead of running a HTML parser within it. Having a DOMParser in this context would be useful..

Is this a reasonable thing to do within a SW?

[jakearchibald]

A better solution here would be to use HTTP/2, where the overhead of multiple requests is low.

[inian]

Hey Jake,
I understand that but I was thinking of other optimisations too..say injecting preload tags for fonts and other optimisations purely done on the client side HTML..
Plus this would be a solution that we can use right now, before CDNs start adopting HTTP/2 more widely..

[annevk]

You cannot use it right now if we need to make DOM thread-safe first…

[jakearchibald]

If you're wanting to trigger preload earlier, I think the options discussed in https://github.com/slightlyoff/ServiceWorker/issues/920 are better.

If you have to download the whole doc to inject a preload, I think you'll lose performance rather than win

[delapuente]

@annevk I think @inian does not want to access the DOM, he is simply suggesting to use some kind of parser to get the body of the response as structured data instead of dealing with strings.

[RReverser]

@delapuente The thing is that DOMParser creates real DOM nodes (hence the name), so it's not that easy to split functionality of one from the other. Much easier would be to use some external HTML5 parser (like parse5 as per @domenic's suggestion).

Moreover, if you want to use HTML parser for the content you're getting from the server, you will likely want a streaming parser (which DOMParser is not) that would play well with Streaming API where / when it's available, as otherwise Service Worker will become a bottleneck and not a source of optimizations.

[inian]

Ah yes, looking like using a (external) Streaming HTML parser along with the Streams API would be better for the things like injecting the preload tag and so on.. That way, I don't need to wait for the entire document to download..

[RReverser]

@inian Yup (and parse5 has streaming mode).

As for preload specifically, you don't even need HTML parser, as instead of tags you can use Link header to indicate same intent, and header is easy to add even without retrieving the response body.

[inian]

Thanks for clarifying that @RReverser. Some of the people we are working with find it easier to add a script tag to their page than messing with their servers - so if we could all these optimisations just using a SW, it would be cool..so we are working from that angle now..

Thinking about it, we could just add the Link header via the SW too..

[mflux]

I just bumped into this issue where, upon using THREE.js ColladaLoader2 (which uses DOMParser) to break open Collada files (3d models), I tried speeding up the load process by putting it through a web worker. To my surprise, this didn't work simply because DOMParser is not available from within web workers.

It's possible to hack ColladaLoader2 to use a DOMLoader alternative but that's just crazy. It's entirely reasonable for a web worker to be parsing things like DOM.

[joeyparrish]

I think for most purposes, any XML parser would suffice. It would not have to be DOMParser specifically, but depending on a pure JS XML parser seems like too much trouble.

[RReverser]

@joeyparrish No, no, please never use XML parser to parse HTML. Despite visual similarity, they have very different semantics (unless you specifically target XHTML and not HTML5).

[joeyparrish]

True, for HTML. I was thinking of a use case of my own where we use DOMParser to parse XML and would like to be able to do so from a service worker. Since the OP wants to parse HTML, please disregard my comment.