Vscode: API for hierarchical document symbol data

[jrieken]

In order to support a real outline view (#5605) and a breadcrumb bar (#9418, #31162) we need better document symbol information. Today, the API doesn't allow providers to return tree data and it also doesn't spec what the ranges of symbols should be. Some language extensions make the range be the whole symbol range, some make it the identifier range. It needs a new or refined API to support hierarchies, I see two options

• allow a tree of SymbolInformation-objects, e.g. add an optional children-property.
• have two range-objects on SymbolInformation-objects, one being the whole range and one being the identifier range. While this looks counter intuitive and worst than having tree-style-data it makes merging of different data sets from different providers easy.

[jrieken]

LSP side of this: https://github.com/Microsoft/language-server-protocol/issues/136

[Leedehai]

Indeed needed..

[jrieken]

We want to define, maybe propose, the API this milestone and build a UI for this in May.

[jrieken]

Proposal is to have a new type like DocumentSymbolInformation which has children and a symbol range. A DocumentSymbolProvider is free to return the new type, or old SymbolInformation objects. In the latter case, we cannot render a tree and because of that providers should signal what they return upon registration.

[eamodio]

FYI, this is also related to https://github.com/Microsoft/vscode/issues/11587

[jrieken]

First cut of the API proposal

export class HierarchicalSymbolInformation {
  name: string;
  kind: SymbolKind;
  location: Location; // use location#range to point to the 'identifier' or 'goto' position
  range: Range; // full symbol range as opposed
  children: HierarchicalSymbolInformation[]; // child items

  constructor(name: string, kind: SymbolKind, location: Location, range: Range);
}

export interface DocumentSymbolProvider {
  provideDocumentSymbols(document: TextDocument, token: CancellationToken): ProviderResult<HierarchicalSymbolInformation | SymbolInformation[]>;
}

[patrys]

I'm the author of the Code Outline extension. Here's my take on this:

allow a tree of SymbolInformation-objects, e.g. add an optional children-property.

This could work nicely for languages where methods can be defined outside of the class body or where symbols belong to namespaces independent of the order of definition. However...

have two range-objects on SymbolInformation-objects, one being the whole range and one being the identifier range. While this looks counter intuitive and worst than having tree-style-data it makes merging of different data sets from different providers easy.

This approach would work better if you're ever planning to incorporate other sources of regions such as the folding provider (I'm planning to show folding regions as containers once executeFoldingRegionProvider becomes available as a command).

[jrieken]

An alternative would be a Hierarchy<T>-type which defines a hierarchy but doesn't interfere with the business object. Something like this:

export class Hierarchy<T> {
    parent: T;
    children: Hierarchy<T>[];
    constructor(element: T);
}

export class SymbolInformation {
    detail: string;
    range: Range;
    //more...
}

export interface DocumentSymbolProvider {
    provideDocumentSymbols(
        document: TextDocument, 
        token: CancellationToken
    ): ProviderResult<SymbolInformation[] | Hierarchy<SymbolInformation>[]>;
}

[felixfbecker]

It seems like https://github.com/Microsoft/language-server-protocol/issues/136 is not the equivalent LSP issue, that issue is about class inheritance hierarchy

[jrieken]

The updated proposal is this

export class DocumentSymbol {

    /**
    * The name of this symbol.
    */
    name: string;

    /**
    * The kind of this symbol.
    */
    kind: SymbolKind;

    /**
    * The full range of this symbol not including leading/trailing whitespace but everything else.
    */
    fullRange: Range;

    /**
    * The range that should be revealed when this symbol is being selected, e.g the name of a function.
    * Must be contained by the [`fullRange`](#DocumentSymbol.fullRange).
    */
    gotoRange: Range;

    /**
    * Children of this symbol, e.g. properties of a class.
    */
    children: DocumentSymbol[];

    /**
    * Creates a new document symbol.
    *
    * @param name The name of the symbol.
    * @param kind The kind of the symbol.
    * @param fullRange The full range of the symbol.
    * @param gotoRange The range that should be reveal.
    */
    constructor(name: string, kind: SymbolKind, fullRange: Range, gotoRange: Range);
}

export interface DocumentSymbolProvider {
    provideDocumentSymbols(document: TextDocument, token: CancellationToken): ProviderResult<SymbolInformation[] | DocumentSymbol[]>;
}

The motivation for having a separate DocumentSymbol-type are

• it allows to tell that a provider has thought about outline as a tree
• it makes the API more document symbol specific: today we have the slightly weird location-property. That's because the SymbolInformation-type is also used for workspace symbols which need the uri-property. Now adding, fullRange as range while the gotoRange is contained in the location-object would be asymmetric and harder to understand.

[DanTup]

Sounds good to me. The name gotoRange sounds a little weird but I can't think of anything better.

Any possibility of some sort of presentation hint that could be used to show things in the Outline without them appearing in the flat-list document symbol list (eg. to support/fix https://github.com/Microsoft/vscode/issues/51332)?

[patrys]

@DanTup There's the folding range provider API so the Outline could incorporate those ranges.

[mofux]

Maybe not directly related, but wouldn't it make sense that we specify a way to provide additional symbol information (like arguments of a function type symbol), so they can be rendered in the outline tree later on?

[DanTup]

@patrys That feels kinda clunky. Folding ranges have all sorts of things in them (like annotations, multiline strings, etc.) and no attached label. It would also be weird to provide a Symbol Provider and a Folding Provider and the Outline to do some merging of the two.

(Also, you should probably be able to nicely implement outline without suppling a folding provider).

@mofux I like that idea - currently I'm mashing them on the end of the name but the outline could render them better if it was more structured (even if it was just a second string).

[mofux]

@DanTup That's what I am currently doing as well (via monaco-editor). Ideally, for my use-case, hovering a symbol in the Outline should render a tooltip similar or equal to the tooltip that I get when hovering a symbol in the editor itself, e.g.:

At the moment, to accomplish this, I am "guessing" the position of the symbol in the editor, and then I use the HoverProvider to get that information. If the DocumentSymbol's gotoRange was reliable enough to pass it to the HoverProvider so it would get me that information that would also be sufficient.

[patrys]

@DanTup Clunky or not if folding ranges had proper types attached to them that's the most straightforward way to provide grouping for things like "imports", "private methods" etc.

[DanTup]

@patrys I think you've misunderstood what I said. I'm not talking about grouping things like imports; imports shouldn't even appear in the outline tree. My comment is about providing data for rendering in the outline tree that is not shown in the flat document symbol list (because when not rendered in a tree it's less useful).

[jrieken]

Maybe not directly related, but wouldn't it make sense that we specify a way to provide additional symbol information (like arguments of a function type symbol), so they can be rendered in the outline tree later on?

@mofux Yeah, an earlier version had a detail-property and I am open for adding it back. Not having it ensures a slick/clean UI which is something we always strive for. Tho we could implement the UI so that we only show the detail when an item is selected

[DanTup]

Not having it ensures a slick/clean UI which is something we always strive for.

I think not having it could encourage people to just add it to the name.. I'd already done that in Dart because it seemed useful. If it's a separate field at least it could be rendered more subtle.

[mofux]

@jrieken Thanks for your thoughts. What shape would this detail property have? As said before, if there was a reliable way to get the hover message of the DocumentSymbol (maybe by its gotoRange) that would be even better as we can keep the DocumentSymbol clean and simple.

Another thought:
From a design perspective, it would IMHO make more sense to have the DocumentSymbol carry a defined set of extra information (return type, arguments etc...), and then have the HoverProvider re-use that information to create the hover label out of that.

[eamodio]

Given that the DocumentSymbol has a children property, maybe the parameters of a function/method could be added as children of it with a new SymbolKind.Parameter or something like that?

[jrieken]

I have added a simple detail: string property (like CompletionItem#detail) and we will render it next to the names of symbol, likely only when selected. I have stayed away from spec'ing things more precisely because we usually try to avoid assigning semantics, e.g. this is a param, this is var-args, etc, but we like to design the API according to our UX needs.

Wrt hovers: We are thinking to simply invoke the existing HoverProviders and to reuse that information. We need for special API - all information needed for that request (document & position) is already there.

[mofux]

Wrt hovers: We are thinking to simply invoke the existing HoverProviders and to reuse that information. We need for special API - all information needed for that request (document & position) is already there.

I'm afraid it isn't that simple, consider these cases that show the hover positions required to get the correct function signature:

// have to hover "funcA"
function funcA(a, b) {}

// have to hover "funcB"
const funcB = (a, b) => {}

// have to hover "function" keyword
this.funcC = function(a, b) {}

// no hover position can reveal the function signature
this.funcD = (a, b) => {}

[jrieken]

I guess it would use the gotoRange so giving the extension control over this. Not sure about the last two samples and how often you use this to refer to the global but as soon as you have inside a function or class it works correctly.

[mattacosta]

Maybe not directly related, but wouldn't it make sense that we specify a way to provide additional symbol information (like arguments of a function type symbol), so they can be rendered in the outline tree later on?

Something else that comes to mind was an editor that I used in the past which had green/yellow/red icons (with static variants) next to public/protected/private methods and properties. It was a quick way to check member ordering without having to run something like tslint.

[KamasamaK]

The ability to have modifiers on kind is proposed in #23927.

[jrieken]

After discussing this today we wanna make two ranges fullRange becomes range and gotoRange will become selectionRange.

[pltrant]

Is it going to be possible for extensions to contribute configurationDefaults for the Outline (to show and to set follow cursor and sort type)? It would seem logical since we can do that for things like the minimap. It's beneficial for creating a curated experience with certain specialized extensions (and users can always override said defaults).

[jrieken]

@pltrant This issue is about the underlying API, not configuration