Vscode-jupyter: Provide new interactive window experience for editing notebooks directly

[rchiodo]

Reusing the interactive window, when opening an ipynb file, just load it into an interactive window, but with these differences:

• Cells are editable
• There's a convert button
• Cells are reorderable (P2)

[rchiodo]

Start of this is here:
https://github.com/microsoft/vscode-python/tree/rchiodo/edit-notebooks

[AeneasZhu]

Will this feature make python interactive window more jupyter-alike?

[rchiodo]

Sort of. It will change the way you edit cells at least. They'll be inline like a jupyter notebook.

[AeneasZhu]

Sort of. It will change the way you edit cells at least. They'll be inline like a jupyter notebook.

How soon will this update be released? It won't be long, right?

[rchiodo]

Maybe next release, or the one after that. It's the next big thing we're working on.

Here's a gif of what I currently have. Comments welcome

[AeneasZhu]

Maybe next release, or the one after that. It's the next big thing we're working on.

Here's a gif of what I currently have. Comments welcome

Damn Brilliant!! Will this feature be compatible with VIM extension? After this release, we could code on ipynb rather than .py file ?

[rchiodo]

Will this feature be compatible with VIM extension?

Likely not a first. Not sure how the VIM extension changes editing, so we'd have to look into it. Extensions that modify the editor would have to be reapplied to this window.

After this release, we could code on ipynb rather than .py file ?

Yep. That's the idea. Or go between them if you like.

[rchiodo]

Sorry are you suggesting people should use pycharm, or that we should have a pycharm like experience?

[AeneasZhu]

Sorry are you suggesting people should use pycharm, or that we should have a pycharm like experience?

Nope, I didn't suggest PyCharm. I don't use PyCharm actually. I mean PyCharm's jupyter style is another solution for VS Code's python interactive window. Sorry for the comment above, I didn't realise it will cause misundstanding.

[rchiodo]

I mean PyCharm's jupyter style is another solution for VS Code's python interactive window.

That's what I was hoping you meant :). This is also in our plans but will likely happen after the idea I showed here.

[AeneasZhu]

That's what I was hoping you meant :). This is also in our plans but will likely happen after the idea I showed here.

Keep the current python interactive window on .py file but make the display of output like the PyCharm Jupyter or Atom Hydrogen where input each cell has its corresponded output cell. Is that right?

[rchiodo]

Yes the idea is to have 3 ways to do jupyter like work

• Current interactive window where it just appends
• ipynb editor that behaves like jupyter (everything inline)
• Interactive window where cells don't append unless you run a new cell

[rchiodo]

Going on vacation. Work is here:
https://github.com/microsoft/vscode-python/tree/rchiodo/edit-notebooks

Here's the minimum list of stuff left to do:

• Functional tests
• Arrow down not working on last line if wrapped
• Keyboard shortcuts missing
  
  
  
  • dd - delete
  
  
  • insert above/below
  
  
  • others?
  
  
• Export to pdf
• Interactive cell buttons only on mouse hover
• Cleanup UI buttons and dropdowns
• Double check different themes
• Weirdness around startup of server
  
  
  
  • Timeout doesn't work anymore, just fails. May have broken the jupyter session retry logic
  
  
• Change markdown editor to CodeMirror as it supports markdown previewing inline
  
  
  
  • Monaco markdown editing isn't as good as CodeMirror
  
  
• Startup server in background
  
  
  
  • Make an option
  
  
• Select multiple cells
  
  
  
  • Make menus work on all the cells at once
  
  
  • Support merge commands
  
  

[rchiodo]

Maybe also investigate https://github.com/microsoft/azuredatastudio to see how they handle the monaco editor

[AeneasZhu]

Will jupyter extension be compatible with this new interactive window? It seems to be a jupyter notebook inside VSCode?

And debug Cell?

[AeneasZhu]

https://github.com/VSCodeVim/Vim/issues/3973#issuecomment-522954761

Is this implemented with a WebView?

[IanMatthewHuff]

@AeneasZhu. This new notebook editing experience is basically another way of interacting with the Interactive Window experience. In the interactive window your .py file is being used as a base for the source code and the interactive window is just to display a history of what you have executed.

This new editing experience uses the same output elements as the interactive windows, but instead of entering code from a .py file it uses a more jupyter like interface where the code elements are all inline with their output.

It shares much of its code with the existing interactive window, so yes it is also implemented as a WebView and will not initially support VS extensions like VSCodeVim (sadly for me as I use that as well).

[IanMatthewHuff]

Creating work list from Rich's last comment

• [x] UI fixes for Rong (naming, dark theme microsoft/vscode-jupyter#3165)
• [ ] Variables being refreshed on every cell submit
• [ ] Fix existing broken unit tests - Jupyter Execution
• [ ] Fix existing broken functional tests
• [ ] Add functional tests
• [ ] Arrow down not working on last line if wrapped
• [ ] Keyboard shortcuts missing
• [ ] dd - delete
• [ ] insert above/below
  others?
• [ ] Export to pdf
• [ ] Interactive cell buttons only on mouse hover
• [ ] Cleanup UI buttons and dropdowns
• [ ] Double check different themes
• [ ] Weirdness around startup of server
• [ ] Timeout doesn't work anymore, just fails. May have broken the jupyter session retry logic
• [ ] Change markdown editor to CodeMirror as it supports markdown previewing inline
• [ ] Monaco markdown editing isn't as good as CodeMirror
• [ ] Startup server in background
• [ ] Make an option
• [ ] Select multiple cells
• [ ] Make menus work on all the cells at once
• [ ] Support merge commands
• [ ] Final UI icons
• [ ] Editor is slow to load when .ipynb is opened
• [ ] Edit history in input box (up and down arrows) not working
• [ ] Check distrubution.toml file on merge

[AeneasZhu]

@IanMatthewHuff Will this feature be added in Sep update?

[gkaissis]

This is an unbelievably strong addition which I feel should be very actively promoted on the blog or in some sort of announcement. The current state of Jupyter support is acceptable, but the flexibility proposed and implemented by @rchiodo just elevates VSCODE as a whole to another level for working data scientists. I do a lot of my development in Jupyter in the browser but I switch to VSCODE for production code or for FOSS contributions. Being able to shift my entire workflow away from the browser while retaining the same sort of user interface (especially regarding fonts, compare microsoft/vscode-python#5256) is fantastic!
Also from an ergonomic sense, working on an ultrawide display and not having to shift the focus of the eyes from left (code/ #%% cells) to the right (Jupyter-like output) will be a huge improvement (compare microsoft/vscode-jupyter#1444).
Congratulations to all involved, I will try to contribute ASAP as well, just currently completely drowning in work. In that spirit, happy vacation @rchiodo !

[joyceerhl]

Echoing comments above, I'm really excited for this feature to come to the Python VS Code extension! My machine learning class (and several other linear algebra/ML classes at my college) actually requires us to use Jupyter Notebooks (converted to PDF) for homework submissions. The homeworks involve submitting notebooks with code, plots, and significant amounts of markdown with LaTeX equations (markdown/LaTeX is about 70% of the notebook), so I've defaulted to the browser workflow in the interest of directly editing and re-executing markdown cells in place. But I'd love to have the direct editing experience in VS Code—I miss the IntelliSense, Git integration for versioning/collaborating on my notebooks, and not being browser-bound. Can't wait for this to ship!!

[AeneasZhu]

Yes the idea is to have 3 ways to do jupyter like work

• Current interactive window where it just appends
• ipynb editor that behaves like jupyter (everything inline)
• Interactive window where cells don't append unless you run a new cell

I try vscode notebook these days, and I still prefer the old style of python interactive windows, for the following points:

• Unable to debug cell
• Intellisense is not intelligent and doesn't make sense
• No support for vim extension
• No support for intellicode

Will the python interactive window feature the following gif in the future ?

[rchiodo]

@AeneasZhu Intellisense is being tracked by a bug:#7316, although if you have more specific feedback, that would great. It should be partially working.

Debugging cells is something we're going to release later. It's on our todo list.

VIM support is longer term. We probably need support from VS code to support loading extensions into the editor we're hosting.

However, intellicode should be working. What results did you get when you tried it?

[AeneasZhu]

Yes the idea is to have 3 ways to do jupyter like work

• Current interactive window where it just appends
• ipynb editor that behaves like jupyter (everything inline)
• Interactive window where cells don't append unless you run a new cell

https://user-images.githubusercontent.com/44701446/65811893-13adc000-e1f2-11e9-8b94-0e626455f03d.gif

Is this style of 'Interactive window where cells don't append unless you run a new cell' still on the todo list?

[rchiodo]

Yes. This issue here is tracking it:
https://github.com/microsoft/vscode-python/issues/6604

Feel free to upvote it.

[gkaissis]

@rchiodo omg, I just tried the new ipynb editor out, it's UNBELIEVABLE! Many congrats to all involved!

[StevenLi-DS]

@rchiodo the customized color theme like italic seems no longer work and also no autocompletion for () and []. Are those known issues?

[rchiodo]

Sorry @StevenLi-DS but not sure what you mean by customized color theme not working. Can you enter new issues for each thing you're seeing?

Theme support and intellisense support should be identical to before the notebook editor was created. If you're seeing a regression please log an issue.

[qdlmcfresh]

Is there a way to enable nbextensions?

[rchiodo]

No sorry nbextensions are not currently supported. Please enter an issue, if you'd like them to be, so other people can up vote the issue.

[zouhx11]

table of contents feature for notebooks would be great

[rchiodo]

@zouhx11, please feel free to enter a suggestion for a table of contents. This way others can up vote it.

[LustigePerson]

When opening an ipynb file it goes right away to the interactive mode. Is there a way to look at the raw json file as well? Sometimes you want to look at the metadata for example.

[gsajko]

what's even more problematic, I can't compare changes in notebooks, I also goes into the interactive mode.

[rchiodo]

The compare changes in notebooks is a bug:
https://github.com/microsoft/vscode-python/issues/7905

To look at the actual JSON, you have to turn this setting off:
"python.dataScience.useNotebookEditor": "true".

With the setting turned off, ipynb files don't automatically open in the new editor. You can still open them though with a right click menu.

[daddy32]

Hello,

i found about this functionality today - and it is great! It is making me consider switching my current IDE.

I have few minor GUI related issues however, which I haven't seen mentioned around here:

• CTLR + Enter does not work to "run" markdown cell (while Shift + Enter does)
• Navigating cells (using arrows) scrolls "viewpoint" extensively and unnecessarily. Every time I switch focus from one cell to another, whole editor scrolls up or down, even if it is not necessary for highlighting the cell (ie it is already on the screen).
• Selecting the ipynb file in Explorer sidebar runs the whole process of opening json, switching to editor and so on - even if the file is the one that is already currently open in the editor. This has some side effects, like not being able to rename the file by selecting it in sidebar with mouse and pressing F2 - the sidebar loses focus, which is different behavior than for other file types.

Still, I am very thankful for this functionality and look forward to subsequent releases.

OS: Windows 10
Version: 1.39.2

[drcege]

Maybe next release, or the one after that. It's the next big thing we're working on.

Here's a gif of what I currently have. Comments welcome

@rchiodo Hi, how can we remove the horizontal frames with "run and markdown button" in the latest version, as described in this question?

It takes up a lot of space but is useless most of the time. I just want an interface like the one in your GIF (native jupyter-like experience).

[rchiodo]

@rchiodo Hi, how can we remove the horizontal frames with "run and markdown button" in the latest version, as described in this question?

It takes up a lot of space but is useless most of the time. I just want an interface like the one in your GIF (native jupyter-like experience).

That GIF is one of our early versions. We didn't ship it that way.

There is no way to turn off the toolbar. However there is another way to open notebooks that we're working on. See this here:
https://devblogs.microsoft.com/python/notebooks-are-getting-revamped/