Vscode: Api for editor insets

[jrieken]

This issue track the API proposal for editor insets. The current proposal is this

https://github.com/microsoft/vscode/blob/ffe3749d5afb046f289d04f915d8d4ade47c028e/src/vs/vscode.proposed.d.ts#L135-L147

[davidanthoff]

Could there be a feature that such an inset can change its height dynamically depending on the content of the webview?

I would like to use this in the Julia extension in such a way that one can run individual blocks of code from the editor, and then the result of the expression that is run would be shown in an inset below that code. For that result I would like to use a HTML tree view that one can use to explore the structure of a composite type that can be returned from an expression, but that would require that an inset can change its height, for example in reaction to a user expanding a node in the tree.

[jrieken]

Yeah - that's something we can consider but we are also afraid that this makes the editing very nervous. Image you are typing and an inset above you keeps changing height as you type; we will likely fail to keep in the view port stable in those cases.

[davidanthoff]

So the way I would use this is that I would only ever change the height dynamically if the user interacts with some UI element inside the webview, i.e. never in reaction to some event outside of the webview. I think that should work, right?

I've been thinking a bit more about this, and even for many use cases where I don't need the ability to dynamically adjust the height I would often not be able to specify a height when I call createWebviewTextEditorInset. For example, say I'm running a piece of code that generates a static plot figure. What I have is some HTML for that that I want to show, but I wouldn't know what height that would have. So I think I would really need a version of createWebviewTextEditorInset that doesn't take a height parameter at all.

[wadethestealth]

@jrieken I am a developer from nteract/hydrogen, an Atom package that provide inline Jupyter support, and some of the developers and users have wanted us to create an extension in vscode for quite some time as they enjoyed our workflow in Atom. Unfortunely without this api, it is not possible to create our workflow, but now it should be feasible.

In Atom's api, we use a point inside the editor's text buffer to inject an html (technically react) element into the page. Atom allows us to style the element however we wish, as if it was a normal element in a browser, which allows for really unique customization.

I am unsure of the underlying code that allows for this api, but only allowing for a set height is extremely limiting in cases where we don't know the height of our content. You can see this particularly in our case because as we send code for jupyter kernels to execute, we use a tiny loading icon, but then the result that is sent back (which we have no control over and dont know the size of) replaces this icon. From my understanding it seems like once we got the result we would have to render it ourselves to be able to calculate the height and choose a proper inset height (which also is not a multiple of a line number).

Point being, I would love to be able to add in inline html elements, but this webview inset seems restricting because of its height requirements.

[davidanthoff]

@waderyan Not relevant to this discussion, but I would _love_ to see nteract in VS Code!!! Maybe another option would be to just use webviews and the new custom editor option? And then just use the standard nteract UI? Also, there seems to be something going on at https://github.com/microsoft/vscode/pull/86632. And sorry for the noise here, we should probably open a separate issue if we want to discuss the nteract story further.

[tamuratak]

@jrieken, How about exposing the API of updating the height of insets and delegating the UX of inset height entirely to extension authors? It would reduce the number of complaints about the UX to the VS Code dev team. The authors might know better what is good UX for each situation.

    export interface WebviewEditorInset {
        readonly editor: TextEditor;
        readonly line: number;
        readonly height: number;
        readonly webview: Webview;
        readonly onDidDispose: Event<void>;
        dispose(): void;
        updateHeight(height: number): void;
    }

From the technical point of view, because the insets are implemented with iframe, I think the correct height must be sent from the inside of the iframe. The height can not be known outside the frame.

This is an experiment of mine.

• https://github.com/tamuratak/vscode/tree/inset_update_height
• https://github.com/tamuratak/vscode-extension-samples/tree/inset_update_height

[jrieken]

Yeah, something along those lines would make sense. It could also be an API function inside the iframe/webview so that the code inside of it can announce its new height

[tamuratak]

FYI: A related issue #88243 is going on.

[shaman-apprentice]

Out of curiosity: Why not providing the monaco-editor instance? Then we could use all native monaco-editor API directly, without having to wait for a vscode wrapper API (I personally would love that). Is that a security concern?

[tamuratak]

@jrieken I am glad to hear that you are not negative to add API. I have another suggestion. How about separating creating EditorWebviewZone and creating Webview in the zone? Exposing createWebview to extension authors allows us to implement the lazy loading of EditorInset with onDidChangeTextEditorVisibleRanges, which behaves like the lazy loading of img on HTML.

    export interface WebviewEditorInset {
        readonly editor: TextEditor;
        readonly line: number;
        readonly height: number;
        readonly webview: Webview | undefined;
        readonly onDidDispose: Event<void>;
        dispose(): void;
        createWebview(): Promise<Webview | undefined>;
        disposeWebview(): void;
    }

An experiment of mine.

• https://github.com/tamuratak/vscode/tree/expose_create_webview_2
• https://github.com/tamuratak/vscode-extension-samples/tree/example_expose_create_webview/helloworld-sample

[shaman-apprentice]

May I kindly ask for a very rough estimation if / when something like this is to come? :) I am looking forward into this a lot and it would increase my already great joy for vscode ;)

[usernamehw]

It was sort of already asked before, but still not answered: https://github.com/microsoft/vscode/pull/66418#issuecomment-474462079 , https://github.com/microsoft/vscode/pull/66418#issuecomment-479846860

Would it be possible to have an alternative rendering mode of webview floating on top of the code? (work as an absolutely/fixed positioned element)

[xixixao]

Yeah - that's something we can consider but we are also afraid that this makes the editing very nervous. Image you are typing and an inset above you keeps changing height as you type; we will likely fail to keep in the view port stable in those cases.

@jrieken It might be useful to have a general approach for maintaining current scroll position of a cursor across changing content. I'm not sure whether this is technically feasible with Monaco or not. Here's a scenario which could benefit from this (regardless of the inset API):

Consider last selected cursor to be the primary cursor. If I place several cursors in a file, and then start typing new lines, the primary cursor (which I was looking at) quickly disappears from the view.

[davidanthoff]

@jrieken would be awesome if you could give us an update on the status of this.

There is a major discussion going on in the Julia language community right now whether most IDE efforts should focus on VS Code. This feature here is by far the most discussed obstacle right now (because the Atom Julia plugin supports it).

I think just a short status update would be great. Is this just on the backlog for a short while? Are there still plans to eventually finish this? Or has this idea be abandoned?

[jrieken]

There are currently no plans to finalise this API (say within the next 3-6 months)

We are currently focusing on native notebooks support (code editors interleaved with rich output, like jupyter notebooks) and we think that a super-set to this proposal. The title of https://github.com/julia-vscode/julia-vscode/issues/178 suggests that it would also benefit from notebooks support. Once notebooks are done, we will revisit this item.

[davidanthoff]

Thanks for clarifying!

Yes, the notebook stuff is fantastic, I've had a notebook branch for the Julia extension for a couple of weeks now and it is probably the feature I'm most excited about in a long time.

[peey]

While I'm liking notebooks, that seems to be suitable for very highly structured UI, where what you write is limited to the cell structure.

This API seems to be much more powerful since you can inline things in a regular text / code file.

I don't want this to die just because notebooks are able to solve a subset of problems that this does. I hope when you re-visit this after notebooks, this API also makes it through.

[jan-dolejsi]

I also believe there are less-structured languages that do not fit the notebook paradigm, yet they would much benefit form the ability to insert visualization (triggered by a code-action) just above selected constructs that benefit from the _picture is worth a 1000 words_.
I built a very similar experience as the insets that the Source Control inserts to show what has changed in a document.
Here is a year old screencast. I cannot get it to work anymore - ever since Content Security Policy was implemented, I think.