Vscode: Add strict notebook document edit api

[jrieken]

Today, notebook document cells can be removed and added via the NotebookEditorCellEdit-api. The reason for having the edit-builder approach is that notebooks can change from the UX- and API-side and that it provides a consistent way to apply multiple edits for a given state. However, cell metadata, cell output, and document metadata can be changed directly via setters which conflicts with the approach outlined above. Modifying these properties should only be possible via an explicit edit API, direct modification shouldn't be possible.

In addition to this extended edit API we also need a way to modify a notebook without it showing in an editor. This will help with LS scenarios but will be generally useful. The same edit primitives are required outside of an notebook editor, maybe as part of the workspace edit API.

An extended/updated version of the edit-API looks so:

// edit with vscode.workspace.applyEdit(...)
export interface WorkspaceEdit {
  replaceCells(uri: Uri, start: number, end: number, cells: NotebookCellData[], metadata?: WorkspaceEditEntryMetadata): void;
  replaceCellOutput(uri: Uri, index: number, outputs: CellOutput[], metadata?: WorkspaceEditEntryMetadata): void;
  replaceCellMetadata(uri: Uri, index: number, cellMetadata: NotebookCellMetadata, metadata?: WorkspaceEditEntryMetadata): void;
}

// edit with NotebookEditor
export interface NotebookEditorCellEdit {
  replaceCells(start: number, end: number, cells: NotebookCellData[]): void;
  replaceOutput(index: number, outputs: CellOutput[]): void;
  replaceMetadata(index: number, metadata: NotebookCellMetadata): void;
}

with an samples being

// via editor
const success = await vscode.notebook.activeNotebookEditor.edit(edit => {
  edit.replaceOutputs(0, [{ outputKind: vscode.CellOutputKind.Rich, data: { 'text/markdown': '_Hello_' } }])
})
console.log(success);

// via workspace edit
const edit = new vscode.WorkspaceEdit();
edit.replaceCells(
  someNotebookUri,
  0, 0, [{ cellKind: vscode.CellKind.Markdown, language: 'markdown', outputs: [], source: '_Hello_', metadata: undefined }]
)

const success = await vscode.workspace.applyEdit(edit);
console.log(success);

related to https://github.com/microsoft/vscode/issues/102503, https://github.com/microsoft/vscode/issues/103713

[DonJayamanne]

@jrieken Today kernels can update the cell outputs as well as metadata as part of execution.
Should we (extension authors) be using this new API to update the cell output and metadata instead of directly updating the properties?

[jrieken]

Yes and no. We are planning to remove the ability to update output and metadata via those setters but when refining the execution API we might add more API that models cell execution and I can image that part of that would be fresh output. We will know more in a bit.

However, the edit API is here to stay. This will likely be the backbone for LiveShare and notebooks and the rule of thumb is that edits that are saved to disk, like cells structure, output, and metadata, must be declared by extensions and then applied by the renderer. That ensures consistency since users can make conflicting operations in the UI at the same time.

[DonJayamanne]

Doesn't moving these edit operations into these editor builders mean they can also be undone.
My assumption was that any operation perfomed using such API can be undone.

However, when running a cell and cell output is updated, then hitting undo should not revert the output. That not right.
Or am i misunderstanding the API, i.e. it doesn't necessarily mean it can be undone.

[DonJayamanne]

Also, how can we use this new API to update the NotebookDocument.metadata (NotebookDocumentMetadata).
Will we have a similar API or can that be updated directly?

Finally, can the displayOrder and languages properties of the NotebookDocument be updated directly as well. Asking as these properties are similar to metadata.

[jrieken]

My assumption was that any operation perfomed using such API can be undone.

Yeah, no implicit undo guarantee here. What we want to guarantee is consistency of the modified data.

Also, how can we use this new API to update the NotebookDocument.metadata (NotebookDocumentMetadata).
Will we have a similar API or can that be updated directly?

There is replaceCellMetadata

Finally, can the displayOrder and languages properties of the NotebookDocument be updated directly as well. Asking as these properties are similar to metadata.

Not yet decided. The rule of thumb is that everything that potentially gets saved to disk must go through the edit APIs.

[jrieken]

What's still missing

• [ ] clean-up/remove the old setter that allow modification
• [ ] (maybe) a way to move cells, today you need to remove it and add a new cell
• [ ] via https://github.com/microsoft/vscode/commit/7c986e94f4c3384ca166b535da25b360c70abd61 @rebornix made _replace_CellMetadata a _delta_ operation. My intention was replace so I wonder what the motivation was to make it delta?
• [ ] decide what to do other notebook fields, e.g display order (appears to be unused anyways) and languages (maybe from provider?)

[DonJayamanne]

Will we have a similar API or can that be updated directly?
There is replaceCellMetadata

Thanks, however I'm not referring to cell metadata. I'm referring to the document level metadata.

[jrieken]

Sorry - misread that. Yeah, plan is to also have something analog to update document metadata

[jrieken]

btw - there is now a way to replace the notebook metadata using those edit apis.

[jrieken]

⚠️ @colombod, @DonJayamanne I have pushed the following name changes for the NotebookEditorEdit-API proposal.

      export interface NotebookEditorEdit {
-
-               replaceNotebookMetadata(value: NotebookDocumentMetadata): void;
-
+               replaceMetadata(value: NotebookDocumentMetadata): void;
                replaceCells(start: number, end: number, cells: NotebookCellData[]): void;
                replaceCellOutput(index: number, outputs: CellOutput[]): void;
                replaceCellMetadata(index: number, metadata: NotebookCellMetadata): void;
-
-               replaceOutput(index: number, outputs: CellOutput[]): void;
-               replaceMetadata(index: number, metadata: NotebookCellMetadata): void;
-               insert(index: number, content: string | string[], language: string, type: CellKind, outputs: CellOutput[], metadata: NotebookCellMetadata | undefined): void;
-               delete(index: number): void;
        }

[jrieken]

⚠️ @colombod, @DonJayamanne I have pushed the following name changes for the WorkspaceEdit-API proposal.

        export interface WorkspaceEdit {
                replaceNotebookMetadata(uri: Uri, value: NotebookDocumentMetadata): void;
-               replaceCells(uri: Uri, start: number, end: number, cells: NotebookCellData[], metadata?: WorkspaceEditEntryMetadata): void;
-               replaceCellOutput(uri: Uri, index: number, outputs: CellOutput[], metadata?: WorkspaceEditEntryMetadata): void;
-               replaceCellMetadata(uri: Uri, index: number, cellMetadata: NotebookCellMetadata, metadata?: WorkspaceEditEntryMetadata): void;
+               replaceNotebookCells(uri: Uri, start: number, end: number, cells: NotebookCellData[], metadata?: WorkspaceEditEntryMetadata): void;
+               replaceNotebookCellOutput(uri: Uri, index: number, outputs: CellOutput[], metadata?: WorkspaceEditEntryMetadata): void;
+               replaceNotebookCellMetadata(uri: Uri, index: number, cellMetadata: NotebookCellMetadata, metadata?: WorkspaceEditEntryMetadata): void;
        }

[colombod]

Is this live in latest vscode insiders? I will proceed with the changes.

[jrieken]

This is for next insiders

[jrieken]

Closing this as the implementation is done. Minor tweaks will still happen, e.g it's like that we change the signature of replaceCells to accept a NotebookCellRange

[jabbera]

Apologies for the boneheaded question, but the documentation doesn't quite seem to be there for this feature yet. I would expect this code to replace the text in the selected cell. Instead it creates a new cell above the selected cell. How can I replace the text in a cell:

const activeNotebook = await vscode.notebook.activeNotebookEditor;
let overwriteCell: vscode.NotebookCellData[] = [{
    "cellKind": vscode.CellKind.Code,
    "source": "Aasddaasdasdasd",
    "language": "python",
    "outputs": [],
    "metadata": null
}];

let selectedIndex = activeNotebook.selection.index;

let res = await activeNotebook.edit(x => x.replaceCells(selectedIndex, selectedIndex, overwriteCell));

[jrieken]

Yeah, replaceCell is about inserting and deleting cells. The text content of a cell is a normal text document, so the vscode.workspace.applyEdit(someEdits) will help you.

[jabbera]

Many thanks!