Theia: Support Notebooks

[adrienrenaud]

How do I work with notebooks in Theia? I would like multi-language notebooks like Jupyter or Zeppelin... but inside Theia.

[svenefftinge]

You'd need to create an extension for those formats.

[adrienrenaud]

I almost started the extension @svenefftinge :)
I'm using @theia/mini-browser to embed a notebook served by a jupyter server.
It works fine:

I don't know how mini-browser works. Does it embed the requested url in an iframe? In the past, I have successfully embed jupyter-notebooks in applications using Iframe.

Would you support an extension that needs a external jupyter-server?
If yes, we could think of an extension that:

• starts a jupyter server at the root of the workspace
• open any notebook file with extension .ipynb in a iframe

[svenefftinge]

Nice! Instead of just embedding the jupyter views in an Iframe, I'd rather go for a full jupyter integration, where I can see and manage my running kernels. So basically an integration of the jupyterlab notebook support for Theia. Another alternative is to look at the VS Code extension and integrate that:
https://code.visualstudio.com/docs/python/jupyter-support

[adrienrenaud]

Instead of just embedding the jupyter views in an Iframe, I'd rather go for a full jupyter integration,

I agree but iframes seem to give a quick practical solution.

So basically an integration of the jupyterlab notebook support for Theia.

That would make coding great again. No less.

Another alternative is to look at the VS Code extension

An excellent solution.

[rayh]

@svenefftinge any idea of what would be involved to port that plugin over?

[akosyakov]

@rayh python extension should work in Theia already, could you try: https://github.com/theia-ide/theia/wiki/Testing-VS-Code-extensions

[adrienrenaud]

@akosyakov vscode:extension/ms-python.python works fine (debug, lint, refactor) for me on d380b17fee13d4c3042edd2884e58886d80528dd but I can't open the Python interactive window, neither from the command palette or on clicking on Run Cell. It might be related to using conda environment: see attached log.

theia_jupyter.log

[adrienrenaud]

Adding the conda path in .theia/settings.json (in my case "python.condaPath": "/opt/anaconda3/bin/conda") fix some of the issues. I'm able to open the Python Interactive window: Python: Show Python Interactive window and matplotlib graphs are rendering.
.
But hitting Run Cell does not produce any effect and nothing appears in the backend logs, only in the navigator console:

[jgbradley1]

@adrienrenaud it's not related to the conda environment. I tested in a virtual environment and a regular environment (no virtual environment and no conda installed). Both have the same issue. Run Cell doesn't work in either environment setup.

[BroKun]

I think the integration of jupyterlab will bring a better experience. The editor, kernel manager and command palette can all be integrated. I am currently doing this. Do we need it as an extension? @akosyakov @svenefftinge

[akosyakov]

@BroKun do you mean whether we can have it under theia-ide org as a native Theia extension? It would be fine with me, do it.

[akosyakov]

@adrienrenaud @jgbradley1 We landed webviews support last week. Could you check whether you still have issues with VS Code python extension.

Generally it would be helpful if someone have a look what is going on there.

[fGencturk]

vscode-python extension issue in theia https://github.com/eclipse-theia/theia/issues/4956#issuecomment-564931434

[dannaf]

Any recent progress with this? Looks like @adrienrenaud and others have a nice start, thanks!

Also, is this relevant here? https://github.com/nteract/hydrogen/

[akosyakov]

Issues which observed with the latest version (2020.3.71659) of Python extension:

• vscode.workspace.fs.stat is not implemented:

root ERROR [hosted-plugin: 2692] Error Python Extension: 2020-04-07 12:19:27: stat() failed for "python" TypeError: this.vscfs.stat is not a function
    at v.stat (/tmp/vscode-unpacked/ms-python-release.vsix/extension/out/client/extension.js:1:408800)
    at b.pathExists (/tmp/vscode-unpacked/ms-python-release.vsix/extension/out/client/extension.js:1:411206)
    at b.fileExists (/tmp/vscode-unpacked/ms-python-release.vsix/extension/out/client/extension.js:1:411423)
    at w.fileExists (/tmp/vscode-unpacked/ms-python-release.vsix/extension/out/client/extension.js:1:413580)
    at p.getExecutablePath (/tmp/vscode-unpacked/ms-python-release.vsix/extension/out/client/extension.js:1:467961)
    at C.getActiveInterpreter (/tmp/vscode-unpacked/ms-python-release.vsix/extension/out/client/extension.js:75:1115839)
    at process._tickCallback (internal/process/next_tick.js:68:7)

• Python: Create Blank New Jupyter Notebook command fails with an error in the webview:

/tmp/vscode-unpacked/ms-python-release.vsix/extension/out/datascience-ui/notebook/monaco.bundle.js, /tmp/vscode-unpacked/ms-python-release.vsix/extension/out/datascience-ui/notebook/commons.initial.bundle.js, /tmp/vscode-unpacked/ms-python-release.vsix/extension/out/datascience-ui/notebook/nativeEditor.js is not a valid file name

• Python: Open in Notebook Editor fails with:

root ERROR [hosted-plugin: 2692] Error Python Extension: 2020-04-07 12:28:07: DataScience Error Error: This WebviewPanel is disposed!
    at WebviewPanelImpl.checkIsDisposed (/workspace/theia/packages/plugin-ext/lib/plugin/webviews.js:388:19)
    at WebviewPanelImpl.get [as viewColumn] (/workspace/theia/packages/plugin-ext/lib/plugin/webviews.js:309:18)
    at t.WebPanel.show (/tmp/vscode-unpacked/ms-python-release.vsix/extension/out/client/extension.js:75:190325)
    at process._tickCallback (internal/process/next_tick.js:68:7)

• vscode.workspace.fs.readDirectory is not implemented:

Python Extension (lookForInterpretersInVenvs): TypeError: this.vscfs.readDirectory is not a function
    at RawFileSystem.listdir (/workspace/theia/plugins/vscode-python/out/client/common/platform/fileSystem.js:217:40)
    at FileSystemUtils.listdir (/workspace/theia/plugins/vscode-python/out/client/common/platform/fileSystem.js:321:35)
    at FileSystemUtils.getSubDirectories (/workspace/theia/plugins/vscode-python/out/client/common/platform/fileSystem.js:332:34)
    at FileSystem.getSubDirectories (/workspace/theia/plugins/vscode-python/out/client/common/platform/fileSystem.js:473:27)
    at WorkspaceVirtualEnvService.lookForInterpretersInVenvs (/workspace/theia/plugins/vscode-python/out/client/interpreter/locators/services/baseVirtualEnvService.js:41:14)
    at Promise.all.searchPaths.map (/workspace/theia/plugins/vscode-python/out/client/interpreter/locators/services/baseVirtualEnvService.js:37:58)
    at Array.map (<anonymous>)
    at WorkspaceVirtualEnvService.suggestionsFromKnownVenvs (/workspace/theia/plugins/vscode-python/out/client/interpreter/locators/services/baseVirtualEnvService.js:37:40)
    at process._tickCallback (internal/process/next_tick.js:68:7)

[azatsarynnyy]

FYI:
Python extension relies on CustomEditor API for rendering Jupyter Notebooks.
We (at Red Hat) are going to start working on https://github.com/eclipse-theia/theia/issues/6636

[akosyakov]

It is working from https://github.com/eclipse-theia/theia/pull/7908 if you use the last version which does not make use of the custom editor api, namely: https://github.com/microsoft/vscode-python/releases/tag/2020.5.86806

There are still some issue with loading monaco within the webview:

It should be handled in the separate PR from the VS Code fs support.

[azatsarynnyy]

I tried the latest Python extension and it works with #7908 as well.
As I see from https://github.com/microsoft/vscode-python/pull/12188/files#diff-b1e7b24833976c9e40041d298ee03890R201, the extension actually doesn't depend on CustomEditor Plugins API but it has some experimental feature which the user can opt-in.

[akosyakov]

There are still some issue with loading monaco within the webview:

I've verified that VS Codespaces have the same issues. Nothing to fix here.

[akosyakov]

notebooks from python extension should work nicely now.

[tsmaeder]

Notebooks with the version of the extension you mentioned (2020.5.xxx) do not work on Firefox: I get 404's, for example:

GEThttp://ba7c8bf3-71b8-4260-bbc5-6e2fadff213a.webview.localhost:3000/webview/theia-resource/file///c%3A/Users/thomas/AppData/Local/Temp/vscode-unpacked/ms-python-release.vsix/extension/out/datascience-ui/notebook/nativeEditor.js
[HTTP/1.1 404 Not Found 15ms]

when doing "Python: Create New Blank Jupyter Notebook" The exact same thing works when doing it from Chromium.

[azatsarynnyy]

I've just checked it with the latest Python extension. Works well in Chrome:

In Firefox, I got the same errors as @tsmaeder