Vscode: web - support to load icons/resources contributed by extensions

[jrieken]

The web companion currently cannot load resources, like icons, because they are served from the remote extension host using the vscode-remote-scheme. There are three potentials approaches to tackle this

1. more uri-transforming. we already know that this is challenging because uris often come as css-rules and we don't look at those strings
2. patch document.createElement and have special cases for style and img tags (and others)
3. use the mutation observer to know when a remote-resource is being requested

So, far option 1 seems tricky, option 2 seems pragmatic, and option 3 seems to be the browser way. There was also an attempt use service workers to serve such resources but only the https-scheme is supported...

cc @mjbvz who has a similar problem with webviews (and explored the service worker route)

[jrieken]

Using the mutation observer and blob-url. Should be possible to reduce flicker to a minimum but no guarantees as mutation observers are run _outside_ the dom-loop.

[mjbvz]

Neat. I think service workers would perhaps be better for this, but as you note they only support https URIs so we'd have to switch away from using vscode-remote anyways.

Mutation observers may give us a somewhat reasonable story for migrating webview content back to the web, before we introduce some sort of proper api for webview extensions to use

[mjbvz]

For actual serving up of the resources themselves, I think the service worker based virtual server approach I've taken for serving up webview content could be used here too. Here's how it could work:

1. Instead of using a unique protocol (vscode-remote), we rewrite remote resources to a unique endpoint (/vscode-remote/authority/path/to/resource) that still uses http or https.
2. When running in the web, we register a service worker that handles requests to the special vscode-remote endpoint.
3. The service worker uses postMessage to ask the main VS Code instance to resolve the resource
4. VS Code resolves the requested resource and posts the result back to the service worker.
5. The service worker then resolves the original request

I don't think we can register a service worker for desktop VS Code.

[jrieken]

I like that. It will help with caching and will allow for a solution in which we rewrite url sync, e.g. instead of using the mutation observer approach I would like to have a place that uses remote-urls marked in code, using some special transformer-function like domresource.

[bpasero]

Sounds good to me as well. We should validate if we can use the same trick in Desktop, would be nice.

[jrieken]

We should validate if we can use the same trick in Desktop, would be nice

I'd say it depends on service workers be available for desktop mode...

[bpasero]

@deepak1556 are you aware of any restrictions in Electron to use service workers?

[bpasero]

I just verified that (in Electron 4) I can register a service worker on the file:// scheme:

However I am not sure why issues such as https://github.com/electron/electron/issues/13740 are still open?

[deepak1556]

Service worker support on desktop is same as web for http(s) schemes, as for custom protocols it works if the scheme is registered with privileges using https://github.com/electron/electron/blob/master/docs/api/protocol.md#protocolregisterschemesasprivilegedcustomschemes .

When it comes to file protocol, electron managed to hook into the service worker registration and get it working, but I would highly suggest not to rely on it because its an origin less scheme , there is less flexibility with caching and worker registration per document.

[bpasero]

@deepak1556 ok, so we can introduce a custom protocol for Desktop and register it as being privileged. Is there any disadvantage from doing this registration?

[deepak1556]

Nope no disadvantage, just make sure that in electron < 5 , we need to setup privileges using two different apis for service workers to work properly.

• protocol.registerStandardSchemes - main process
• webFrame.registerURLSchemeAsPrivileged - render process

Things are much better after 5 where they are unified into one single api https://github.com/microsoft/vscode/pull/75802/commits/d8141042dec03ddcb49acbd690bcdb09f5691433

[bpasero]

Thanks for clarifying this.

[mjbvz]

Here's the service-worker we use for loading local resources inside of webviews: https://github.com/microsoft/vscode/blob/02db6311403f8d9967b13d4a73e764d034aa3677/src/vs/workbench/contrib/webview/browser/pre/service-worker.js#L1

We'll need to register a new worker for VS Code itself but may be able to reuse some of this code. The lifecycle and versioning of service workers can be complicated so we'll want to make sure VS Code is always talking to the expected version of the worker. I've added some initial logic in the webview service worker to handle this bu haven't tested it too much yet

[jrieken]

Yeah, that's good start. Tho we should have a service worker that we author in TypeScript and its own compilation artefact (just like web-workers).

Things we should/could use the "main" service worker for

• fetch images/fonts/etc contributed by extensions (this issue)
• store user state after/while shutting down
• cache JS source files to benefit from Chrome/Edge cached data (see https://v8.dev/blog/code-caching-for-devs#use-service-worker-caches)

[bpasero]

@jrieken @mjbvz one thing to keep in mind is that you can only have 1 service worker per domain, so I feel like we need another set of API facade and component for the hoster to use as part of their service worker infrastructure. In other words, we should not bring up the service worker, the hoster needs to imho.

[jrieken]

In other words, we should not bring up the service worker, the hoster needs to imho.

I think that will make development very slow. So much internal knowledge and tech will be built into this that I don't think it will be feasible, esp. not at first.

[jrieken]

one thing to keep in mind is that you can only have 1 service worker per domain

You can have multiple per domain but they need to tackle different scopes. So, we can claim specialised scopes, like web view, for us and see if there is an opportunity for 'shared' service workers.

[bpasero]

You can have multiple per domain but they need to tackle different scopes.

Ok, I did not know that 👍

[jrieken]

Now on master: contributed URIs, e.g icons or theme files, not go through dom#asDomUri before being inserted into the dom. That util synchronously checks for the vscode-remote-uri and rewrites them to <window.location>/vscode-resources/fetch?<actual_uri>. Those uris are served from a service worker which uses a cache or the render's file service to fullfill the request (as described here https://github.com/microsoft/vscode/issues/75061#issuecomment-504594017). Open questions/work:

• Cache invalidation - all resources are now cached and we need to know when to invalidate those caches...
• How to sync with the SW becoming available - the SW has its own lifecycle and asDomUri might have been called already without the SW intercepting those requests. We should consider also service these request from our server or we block the workbench on the SW being available...
• Make this work for the web and desktop - this is debt as today the desktop has a main-thread handler which we should get rid off.

[bpasero]

We should consider also service these request from our server or we block the workbench on the SW being available...

We could add this to the server however I somewhat like that we currently fully rely on the service worker to do its business so that we can validate it actually works. I feel like we should maybe leave it like that for a bit longer to really stress test the solution.

I feel like the service worker - once activated - should talk to the tab and ask it to re-resolve any URI that was going through the asDomUri method, similar to what the mutation observer did before.

[jrieken]

I feel like the service worker - once activated - should talk to the tab and ask it to re-resolve any URI that was going through the asDomUri method, similar to what the mutation observer did before.

Yeah, that and just start the SW sooner, e.g while code loading for the workbench happens. Last, while debugging I have seen that images/resource that the service worker is going to intercept magically recover. It seems that the browser initiates another request once the service worker is ready. Tho that's only an observation and nothing I am certain about...

[jrieken]

I have created https://github.com/electron/electron/issues/19150 which tracks Electron crashing when trying to load a service working from a custom scheme. fyi @deepak1556

[jrieken]

This is now mostly done and remaining pieces will be tracked as follow-up items