Vscode: Expose 'reading' API for file system providers

[jrieken]

With #47475 we will finalise the file system provider API. With that the only (reading) client of that API will be VS Code itself. We should also allow extensions to use the API, e.g. read a file/folder that's served via a contributed file system provider.

[lostintangent]

This would be extremely compelling for Live Share, since it would allow extensions to access files even on the “guest” side.

[eamodio]

This would also be very valuable for my new proof-of-concept extension RemoteHub

[smikitky]

If I understand correctly, even with this, will each extension developer still have to write something like this?

async function writeToAnywhere(uri, blob) {
  if (uri.scheme === 'file')
    await require('fs.promise').writeFile(uri.fsPath(), blob);
  else
    await vscode.workspace.getFileSystemProviderByScheme(uri.scheme).write(uri, blob);
}

I'm writing an extension that deals with binary files, and I'd love to see an abstraction layer that works transparently with any URI regardless of the scheme, as suggested in #51528. There is already a Node's fs-based FileSystemProvider, which may be used to achieve this.

[jrieken]

If I understand correctly, even with this, will each extension developer still have to write something like this?

No. The goal is to enable something like this await vscode.workspace.writeFile(uri, blob)

[bolinfest]

If you are the one creating the FileSystemProvider, here's something you can do in the interim to make it available to other extensions: https://github.com/facebook/nuclide/commit/8cf697a04c76132f922bf19e27252990dbc83b41.

[TomasHubelbauer]

My extensions Email Viewer and ZIP File System would also benefit from this because both deal with mapping files to virtual file system and reading the original file should be possible from other providers not just file.

If one wants to avoid using Node to read the file, it is possible to hack around it by having VS Code open the TextDocument and then use its getText thus obtaining text of a file from any provider not just file, however this would be much cleaner and also work for binary files, which is what my extensions need.

[GabeDeBacker]

@jrieken - Hmm, that doesn't address the fact that VSCode's UI (such as the file explorer) cannot sync with URI schemes from other providers. That should be a goal as well. Our scheme foo: knows that in some cases it really wants to be a "file:" and have all the VSCode UI goodness, and in other cases, it wants to do a write.

I..e.

export class MyFooFileSystemProvider implements vscode.FileSystemProvider {
    public stat(uri: vscode.Uri): vscode.FileStat | Thenable<vscode.FileStat> {
        return new Promise<vscode.FileStat>((resolve, reject) => {
            if (fs.existsSync(uri.fsPath)) {
                fileType |= vscode.FileType.Redirection;
                resolve({
                    ctime: fsStats.ctimeMs,
                    mtime: fsStats.mtimeMs,
                    size: fsStats.size,
                    type: fileType
                });
            } else {
                  // Do custom stuff.
            }
        });

    public readDirection(uri: vscode.Uri): Thenable<[string, vscode.Uri[]> {
      /// Convert the "foo" URI to "file" URI
    }
}

[matthewarkin]

Hey, I was wondering if there was any progress on this issue or what the community could do to make this feature come to life.
Given the number and breadth of extensions which rely on file system access, the lack of this feature makes it difficult for users to take advantage of the FileSystemProvider api as they're essentially faced with a "Do I just do some sort of like fusefs type set up so I can use plugin X or do I use some cool ftp filesystem provider api which gives a much nicer integration but breaks plugin X"?
(This is actually the issue I'm facing where I built an extension to support a remote file system and it works flawlessly until I try to use some extensions that expect files to be local / error non non-local files.)

[smikitky]

Looks like we are nearing the goal. According to this, plain local workspaces will be based on FileSystemProvider soon. As a bonus, FileSystemProvider will support streaming.

[jrieken]

Very simple API proposal (which isn't that simple to implement tho) would be to expose a FileSystemProvider for this. E.g

export namespace workspace {
  export const fs: FileSystemProvider
}

which then can be used like this vscode.workspace.fs.readFile(someURI) or vscode.workspace.fs.rename(oldUri, newUri). Like our internal IFileService this should work across schemes.

[jrieken]

A little more than just the FileSystemProvider interface is needed. For once, a few functions in there are options (like copy) and for "the one" file system we can offer then all. Also, the FileSystemProvider interface allows to return returns sync or async and for the one file system we will always return async. Anyways, the vscode.workspace.fs-type will be very similar to the current file system interface that we have.

[stevenguh]

Can this API read the directory of file located in the remote using VSCode Remote? It doesn't seem to able to if my extension is running on local. Is there any API I can use so that I can query the file system of the remote without having to install the extension in the remote?

[jrieken]

Can this API read the directory of file located in the remote using VSCode Remote?

@stevenguh Yes, that's what this API can be used for. In essence, every resource that VS Code can open, can be opened via this API.

[stevenguh]

It took me some tinkering to get it working. To access the remote file from an extension installed on a local host, you need to have a right scheme and authority in the Uri object. The scheme is vscode-remote and the authority is ssh-remote+<your-ssh-remote-name> for ssh-remote. It's easier to get these information from a opened remote file by accessing vscode.window.activeTextEditor.document.uri

[jrieken]

Yes, the easiest is to "derive" a URI from an active editor, open document, or the workspace

[stevenguh]

I think it would be great if that information is documented somewhere

[jrieken]

Yeah, missing docs/samples are the reason why this issue is still open... I am very open for ideas/suggestions. So far on my list I have

• how to get the 'first' uri constructed
• how to constructs uris using uri.with and path.posix-utils
• how to decode/encode bytes to strings

[stevenguh]

Sounds good, thank you for all the work.

Maybe a sub-bullet for "how to get the 'first' uri constructed" should have some examples of using this API for ssh remote with an extension on the host machine.

And another sub-bullet for path.posix-utils, I found the use of posix path on windows environment is kinda weird. Maybe that's worth pointing it out. (I am new to this, maybe this is a norm in the vs code extension community) Charts like the ones in https://nodejs.org/api/path.html#path_path_posix could be useful in explaining the posix path on windows?

Also, your doc on the proposed API is missing the information the possible exceptions that each method can throw like the doc in FileSystemProvider? As well, the way to catch specific error is not clear.

[jrieken]

started some sample work here: https://github.com/microsoft/vscode-extension-samples/pull/195

[stevenguh]

That looks like a good start. I have a side question, not sure if you would be able to answer.
Uri can take untitled as the scheme so that when I use openTextDocument(uri: Uri) I can create a resources with a predefined path. However, I am not sure if this option is available for remote locations like remote ssh for example as it uses vscde-remote for scheme.

[jrieken]

However, I am not sure if this option is available for remote locations like remote ssh for example as it uses vscde-remote for scheme.

Yes, I tried to clarify that earlier: everything the editor can do, can be done by this API

[jrieken]

sample are here: https://github.com/microsoft/vscode-extension-samples/tree/master/fsconsumer-sample