Vscode: Builtin support for reading files from their zip archives

[arcanis]

Rel: #17271

High-level

It would be useful for VSCode to be able to read files even when they're stored within zip archives. Opening /foo/bar.zip/package.json would succeed. Read-only is likely fine, at least for a first iteration.

Motivation

• This feature isn't unheard of. Both vim and emacs support reading files from zip archives (they even go a bit farther by allowing to also edit them, but we might not want to have to deal with the UI complexity for now).

• Package managers are experimenting with virtual install folders. Being able to read third-party files from the archives that contain them will be important to provide a good developer experience (particularly for the "ctrl+click on a symbol to go to its definition" workflow). Zip will be used by Yarn because of its good random access properties.

Suggested Implementation

• The Yarn project ships a wasm build of the libzip as an external package. This would make it easy to use a battle-tested zip library at little cost.

• The size cost would be around ~300KB for the Zip implementation (which could be reused for other purposes - there's a bunch of things that could use Zip support down the road).

• You might actually already have a library able to do that?

Why not an extension

• I've started working on an extension to help with that (vscode-zipfs), but it cannot be integrated as well as I'd like. In particular, it doesn't work with the ctrl+click workflow I mentionned, and running code /foo/bar.zip/package.json cannot open the right file. I believe a builtin integration could provide a seamless integration, which would be beneficial to the users.

[arcanis]

@jrieken Is it something you think VSCode would be interested to merge, provided someone make a reasonable PR?

[jrieken]

Not sure, this isn't highly upvoted and that kinda defines our work. I like the feature, I know this from KDE times. I am not yet convinced that this isn't a good extension candidate tho, e.g instead of code /foo/bar.zip/package.json you would call code --file-uri zip:foo/bar.zip/package.json. Not a huge blocker IMO and not a flow that cannot be improved for all file system providers

[arcanis]

From what I remember, the main pain point was to open the file from within a VSCode instance already launched. For instance, running ctrl+p and pasting the virtual path won't open anything, and I think extensions aren't able to change this behavior at the moment. Similarly, ctrl+click on a symbol that comes from an in-zip file won't do anything.

[jrieken]

This is good feedback and I believe we can find a good solution that applies to all file system providers, e.g.

For instance, running ctrl+p and pasting the virtual path won't open anything,

The problem is that we don't have finalised the API for search result providers. Once that's done, extensions can provide search results and cmd/ctrl+p should just be working like that.

Similarly, ctrl+click on a symbol that comes from an in-zip file won't do anything.

This is a little vague but the theory is that the language server computes a location like zip:/some/archive.zip/some/path.foolang and then vscode opens that. The C# extension does something similar with sources generated from dll-files. Still, it needs a cooperative language server

[arcanis]

The problem is that we don't have finalised the API for search result providers. Once that's done, extensions can provide search results and cmd/ctrl+p should just be working like that.

Is there someplace I can track this work? I've found https://github.com/microsoft/vscode/issues/59922 but it's not clear whether it got merged, reverted, or exposed as a public API.

[jrieken]

https://github.com/microsoft/vscode/issues/73524

[arcanis]

Hey! I've just seen that #73524 got closed with a PR at #75882, but it's not clear to me what the scope of the implementation is and what the examples do exactly. Is the use case I described in the OP now possible?

I guess the main question I have is: how do I catch that VSCode is trying to read a .zip and return an in-memory TextBuffer instead? 🤔

[arcanis]

Friendly ping @jrieken? 😊

[arcanis]

Ping? I'm still not sure whether this can already be implemented in userland (and, if it is, through which APIs), or if there are still changes that need to land into the core (and if so, which issues to track).

cc @isidorn?

[isidorn]

Even though as you mention that the extension can not provide a seemless experience I still think that the extension is the way to go here.
Since we do not want this feature in the core at the moment. However that might change in the future, thus leaving this open as a feature request.

[vscodebot[bot]]

This feature request is now a candidate for our backlog. The community has 60 days to upvote the issue. If it receives 20 upvotes we will move it to our backlog. If not, we will close it. To learn more about how we handle feature requests, please see our documentation.

Happy Coding!

[vscodebot[bot]]

:slightly_smiling_face: This feature request received a sufficient number of community upvotes and we moved it to our backlog. To learn more about how we handle feature requests, please see our documentation.

Happy Coding!

[zeph]

very happy it moved into the official backlog... is it still there pending though?

[mayteio]

I would love to be able to use yarn 2 in my primary workspace, VSCode. Any idea when this might make it into a version?

[heavenkiller2018]

is it still pending?

[arcanis]

While not builtin yet, the Yarn team maintains the ZipFS extension which makes this process easier. The integration is still a bit imperfect until #59650 is fixed, but that's out of our hands now.

[CharlesStover]

The integration is still a bit imperfect until #59650 is fixed, but that's out of our hands now.

That issue appears to be closed now.

[arcanis]

Yep, and the problems I was referring to (TS support for virtual fs) are pretty much fixed. Zip support is still not builtin so I won't close my issue just yet, but at least now there's a reasonable option 🙂