Vscode: Allow extensions to determine if a position is within a fold.

[johnfn]

This is currently the most demanded features of VSCodeVim: https://github.com/VSCodeVim/Vim/issues/1004

The main problem with Folds and Vim is that some motions will skip right over folded areas (like moving up/down). We need to know if we are in a folded area so we can iterate these motions until we are out of the fold.

An API like vscode.window.activeTextEditor.getAllFoldedRegions(): vscode.Range[] would be ideal.

An API like vscode.window.activeTextEditor.isPositionInFold(position: vscode.Position): boolean would also be great.

[ztmdsbt]

Hi, How is issue going?

[xconverge]

@alexandrudima since you brought it up in that other ticket for wrappedLine info. Instead of having the entire files worth of info, is it possible to have properties for the immediate line (inAFold, charLineStart, charLineEnd )? Is the folding information simpler or any different than the wrapped line info? Any suggestions for how else we can get this information?

[aminroosta]

@aeschli If we send a PR for vscode.window.activeTextEditor.isPositionInFold(position: vscode.Position): boolean will you merge it?

[aeschli]

@alexandrudima see @aminroosta request above.

[alexdima]

We have another issue somewhere where we discussed this. The recommended path forward is for the vim extension to continue invoking the cursorMove built-in command with various arguments that can mimic the desired vim motion.

We have no plans to expose view model information (which we consider implementation detail of the editor) to API.

[aminroosta]

@alexandrudima Thanks for your quick response :-)

We had a long talk with @sandy081 in #9786
Unfortunately cursorMove is not enough to implement even the most basic vim commands.

We have no plans to expose view model information (which we consider implementation detail of the editor) to API.

Well, That's very sad to hear considering VIM extensions are useless without this API.

@alexandrudima @aeschli @sandy081 All other editors are doing it!

• Atom editor isFoldedAtScreenRow API.
• Sublime view.is_folded API.
• Visual Studio IOutliningManager API.
• Intellij LogicalToVisualPosition API.
• Eclipse api

Why do you guys think querying the IndentRange[] array returned from computeRanges exposes implementation details?

Sigh!

[ztmdsbt]

Oh godless!
@alexandrudima , I can't believe that. Why you guys think to get "is_folded" exposes the detail implementation?

[Chillee]

@alexandrudima For reasons I am not entirely clear on, you guys have decided not to expose fold information to extensions. That's fine!

However, in the absence of that, there are some things we need alternatives for. There's 2 main things vscodevim needs improvements on. I've read much of the discussion surrounding this, and I don't think either of these suggestions should be particularly objectionable.

First, we don't actually need an is_folded api to deal with moving around folds. All we need is that when cursormove is called with by: 'line', we don't step into folds and expand them. This seems like more of an oversight than anything else; it doesn't make sense to me that you would ever want to step into a fold with cursormove.

Second, expand the API calls for folding. I could come up with a more extensive list of what we would need minimally, but here's a very simple example. For za, we want to be able to toggle the fold. Implementing this requires either an is_in_fold() or toggle_fold API call.

Additional notes: One other thing. Desired column, as in https://github.com/Microsoft/vscode/issues/23045, is very tricky for us to handle, even with the additional API calls provided above. I have some ideas for better ways to handle this, but I'll leave those for another day.

I hope you guys appreciate how much my hack here for fixing folds hurts my soul: https://github.com/VSCodeVim/Vim/pull/1552/files#diff-944e6e33fb41f3b52eec89cd786de750R242

[johnfn]

@alexandrudima

Due to the complexity and quantity of Vim motions, cursorMove is not an acceptable fix for folds inside VSCodeVim.

Nor will it ever be, because VSCodeVim allows custom motion creation and extension support.

I just want to be clear here: Vim's motions are too complex and varied for a Vim extension to ever consider cursorMove.

I hope that you reconsider.

[alexdima]

I am open to reconsidering and I'm sorry for this being so annoyingly hard.

Here are my concerns and my trail of thoughts:

• you know very well that extensions run in a separate process. This is something I fought for quite hard internally (arguing with team members that pioneered the highly successful Eclipse same-process plugin system).
• this has great benefits such as the UI being responsive irrespective of what extensions are doing. i.e. our mantra was you can always save a file even with the extension host process dead. The idea was that we'd also transparently restart the extension host process when it crashes, which is something I haven't gotten to yet.
• this also has some interesting problems, such as the parallelism between the UI and extensions. Since the UI is free to keep on going, it is possible that at a certain time the user interacts with the UI and at the same time an extension is executing.
• to overcome the parallelism problem and to provide a somewhat decent programming model, our APIs will follow the guideline that "reading" is mostly synchronous, while "writing" is asynchronous and might fail.
• the guarantee we make to extensions is that the state they see at any time is a consistent state that VS Code was in at a certain time or that VS Code might still be in. e.g. If VS Code is in a certain state s0 and an extension begins to execute synchronously, the extension will see a complete picture of the state s0 that VS Code was in. It is possible that VS Code transitions via a user gesture to a new state s1, but for the synchronous execution of the extension code, it will see the state s0. If the extension uses a setTimeout or yields execution, it is possible that the API will begin to reflect the state s1.
  
  
  
  • in order for us to maintain this guarantee, all possible synchronous vscode reading API must be fully resolved on the extension host.
  
  
  • in other words, if we have a TextEditor.selection API, we must proactively push the selection, every time it changes, from the UI process to the extension host, even if there is an extension interested in reading the selection or not.
  
  
  • we cannot decide to not push a certain piece of data and fetch it lazily, because by the time the extra piece of information is read from the UI process, it might reflect a newer state and not provide a consistent picture.
  
  
  • e.g. we cannot add a getBla(): Thenable<bla> reading API as this returned value might return the value of bla at state s1, and then the extension code would evaluate half its logic in a state s0 and the other half once the reading API returns, in state s1.
  
  

This was all a big introduction to why my default reply to adding more reading API is always to start the conversation with no. That is because any reading API must be synchronous and must be pushed eagerly by the UI process to the extension host.

I'm always aware of the fact that adding more reading API means sending all the data necessary for it to the extension host even if there is nobody interested in actually reading. For example, I am asked many times to provide a getScopesAtPosition API, since we technically have that information on the UI side or could compute it fairly quickly. But again, that would need to be something pushed eagerly for it to work with the above state guarantee. For example, if we will add scope reading API, I think we should implement it by tokenizing as needed on the extension host process, rather than always pushing tokens to the extension host.

Ok, so enough about general problems. Specifically my concerns on the view / model coordinate system:

• it is important to note that this is something very special. e.g. even in the editor core, most of the code is kept in dark about the existance of a view model or about the existance of the view coordinate system. The only code that needs to know about the view model is the folding contribution and the cursor core navigation commands. i.e. no editing commands care about the view model. And they outweigh the navigation commands by a lot.

• so, IMHO, except for some other specialized things, the only extension consuming such new API would be the vim extension. So we'd introduce the mental burden of the view coordinate system to the API for everybody. Consider a first-time extension writer, how likely is it that there will be confusion regarding two members selection and viewSelection, etc.

• then, consider, the amount of data we need to send when having a large file opened with word wrapping on and resizing it. Consider two large files sitting side by side (so they both change their view - model coordinate mappings) and the slider between the editors is dragged freely.

• this is a lot of data that must travel from the UI thread to the extension host in very many cases where the view model changes whereas today that data only travels to the view and is in certain cases lazily computed as needed.

• and then what pains me is that this data will travel for all folks, irrespective if they are using the vim extension or not.

• finally, I am not happy with our current folding implementation. It is indentation based and I'd like to see it improved via a *provider that can be served by a language server. My concern is that we end up exposing some API that we might regret and we are in a position where we cannot deprecate API...

In conclusion:

• I agree that view coordinates are needed to implement a 100% the vim emulation.
• I agree that folding information is needed to implement at 100% the vim emulation.
• heck, I agree that even the scroll position and viewport size is needed.
• but we need to consider the following to bring those to the extension host:
  
  
  
  • how to expose such API that it is available only when really needed. i.e. vim extension not installed => no view data flows
  
  
  • how to expose such API that we don't shoot ourselves in the foot (at the time when we might be able to collapse a region of a line and not just entire lines)
  
  
  • how to encode everything and sync it over to the extension host in an efficient manner even when the data is needed.
  
  
  • does this block in any way an editor core implementation switch. e.g. https://github.com/alexandrudima/edcore/blob/master/edcore.cpp
  
  

[Chillee]

@alexandrudima Could you explain why cursorMove with by:line and to: down would step into a fold? I understand why you want to limit fold information from the extension, but this just seems like an oversight.

To quote the API command's description:
cursorMove - Move cursor to a logical position in the view.

I don't understand how moving the cursor into a fold would be interpreted as a "logical" position in the view.

This just seems like a strange API decision. If extensions wanted to always move down into a fold, they could just do it manually.

EDIT: Actually, I took a look at the source code and I see why. There is only a modelState (which has no notion of folds or wrapped lines) and there is a viewState (which always knows folds/wrapped lines). Thus, there's no easy way of ignoring only wrapped lines.

However, other than the difficulties with the internal representation, offering a by: foldLine option to cursorMove wouldn't seem to run into any of the other issues you talked about above. We wouldn't need to push this info to the extension thread as it's behind an API call. It also doesn't seem like it would interfere with any kind of core editor change/fold changes.

TL;DR: by: wrappedLine should really be named something like by: viewLine, and we need a by: foldLine.

[aminroosta]

@alexandrudima
There are folding improvements in the September 2017 release for folding regions. 👍
Did you added any new extension APIs ?

[jrieken]

This feature request will not be considered in the next 6-12 months roadmap and as such will be closed to keep the number of issues we have to maintain actionable. Thanks for understanding and happy coding!