Vscode-jupyter: DS: Support other languages

[clouds56]

Is it possible to make jupyter support other language like julia and R?
Or at least provide some interface for other extension to integrate.

[rchiodo]

@clouds56, at the moment no.

[rchiodo]

This would be a good spot to have people vote on other languages. Which ones do you want besides python?

[rchiodo]

Julia?

[rchiodo]

R?

[rchiodo]

Haskell?

[rchiodo]

C#/F#?

[rchiodo]

Scala?

[raghavgautam]

Bash ?

[chrishales709]

SAS?

[dr-br]

cling?

[stevengj]

I don't see any point in voting on languages here — supporting two languages should be the same amount of work as supporting all languages. The whole point of Jupyter is that it defines a language-independent protocol … if there is a general vscode extension for notebooks, it should work with any notebook in any language (i.e. it should be decoupled from vscode-python, analogous to nteract/hydrogen).

That is, when you open a Jupyter notebook file (.ipynb), the kernelspec.name metadata should tell you the name foo of the kernel to run. You then look for a kernelspec file foo/kernel.json in the standard locations. This file looks something like

{
  "display_name": "Julia 1.2.0",
  "argv": [
    "/Applications/Julia-1.2.app/Contents/Resources/julia/bin/julia",
    "-i",
    "--startup-file=yes",
    "--color=yes",
    "--project=@.",
    "/Users/me/.julia/packages/IJulia/src/kernel.jl",
    "{connection_file}"
  ],
  "language": "julia",
  "env": {},
  "interrupt_mode": "signal"
}

and tells you exactly how to launch the kernel (via the argv). You create a connection file, pass this for the {connection_file} argument when starting the kernel, and then talk to the kernel over ZMQ in exactly the same way that you talk to the Python kernel.

[rchiodo]

Unfortunately supporting a kernel is not the same as supporting a language. We run a bunch of stuff that is specific to python in order to get things like theming, variables, startup directories, etc correct. None of those work without figuring out a language neutral or per language way to do them.

We will be supporting other kernels long before we support other languages. This item here:
https://github.com/microsoft/vscode-python/issues/3123
has the beginnings of how this will work.

@hochshi has implemented picking a kernel in remote situations here (which will hopefully go in soon)
https://github.com/microsoft/vscode-python/pull/7790

[al6x]

Supporting just the the script itself, as an interactive notebook, without the Jupyther dependency would be even better.

[rchiodo]

Supporting just the the script itself, as an interactive notebook, without the Jupyther dependency would be even better.

@alexeypetrushin can you elaborate more? Jupyter is just a framework for hosting a process that's running your code. I don't think we'd reinvent this ourselves. Is there something you don't like about having jupyter installed?

[stevengj]

You don't need a dependency on the Jupyter software stack if you simply speak the Jupyter messaging protocol to the kernel. e.g. nteract does this.

(You obviously need the software installed for any kernel you want to use, e.g. you need the ipython kernel installed if you want to talk to Python, which pulls in components from Jupyter. But you wouldn't need e.g. the jupyterlab component. And other languages might not need Jupyter software at all. For example, the Jupyter kernel for Julia has no dependency on the Jupyter software stack — it simply speaks the Jupyter protocol over ZMQ to anything that launches it.)

Unfortunately supporting a kernel is not the same as supporting a language. We run a bunch of stuff that is specific to python in order to get things like theming, variables, startup directories, etc correct. None of those work without figuring out a language neutral or per language way to do them.

By "theming" I guess you mean syntax highlighting etcetera? Why can't you simply read the language field of the kernelspec file and activate the syntax mode for the corresponding language?

Not sure what you mean by "startup directories" … the kernelspec file also tells you exactly how to launch a kernel, including any relevant environment variables.

I'm not speaking theoretically — JupyterLab and nteract do this in a language-neutral way already.

[rchiodo]

There's code we run (python code) that does the following:

• Tell jupyter to create SVGs as well as PNGs
• Change the working directory of the kernel
• Query for variables

This is the stuff that is not platform neutral at the moment, with variables being the toughest to implement. This is really our problem, and not a jupyter specific issue.

We could talk directly to the kernel through stdin/stdout, but then we'd be reinventing the server code that jupyter has for remote situations and it wouldn't work for already running jupyter servers. There's no benefit for us, other than perhaps performance.

[rchiodo]

Given all of that, why do you care if we have a dependency on the jupyter server? Is this causing bugs? Is the download too large? (We might make something simpler if we were to implement this ourselves).

[stevengj]

Tell jupyter to create SVGs as well as PNGs

The Jupyter protocol allows the kernel to send data in multiple formats, including both SVGs and PNGs, so I'm not sure what you're referring to here — there is no need to tell it you want SVGs or any specific format in general, because objects supporting rich display will typically send you data in multiple supported formats and you can pick which one to display.

Maybe you are referring to Matplotlib, which by default does not send SVGs for inline display in order to speed up the display of complicated plots? But in that case there is a standard way for the user to request SVG inline plots, with no need for intervention by the front-end.

Query for variables

The Jupyter protocol already has a language-neutral introspection API.

If there is some other form of introspection that you need, I'm sure @minrk and the other Jupyter developers would be open to discussing adding it to the protocol.

Change the working directory of the kernel

Normally in Jupyter the working directory is the directory where the notebook file is located.

Why do you think you need to change directories? The kernelspec file tells you how to launch the kernel, including any path information.

We could talk directly to the kernel through stdin/stdout.

The Jupyter protocol uses zeromq, not stdio. Have you looked at the messaging spec?

why do you care if we have a dependency on the jupyter server?

Because you are hooking into it in a language-dependent way. If you simply spoke the Jupyter protocol the way it was intended to be used, I personally don't care too much what your software dependencies are, but it's not clear to me why a Jupyter-server dependency would remain.

[rchiodo]

why do you care if we have a dependency on the jupyter server?

Because you are hooking into it in a language-dependent way. If you simply spoke the Jupyter protocol the way it was intended to be used, I personally don't care too much what your software dependencies are, but it's not clear to me why a Jupyter-server dependency would remain.

There is no current JMP for querying variable values. The %who line magic might be used, but it certainly won't page in MBs of dataframe data. We use python code to do this now.

We also switch directories in the kernel based on where the user ran code from. This would be simpler to add a message to the protocol and the %cd line magic can do this too. Although I'm not sure if line magics are supported in all kernels.

SVG formatting is similar. We use a line magic to do this now. Not sure it will work in all kernels.

Our usage of jupyter-server is orthogonal to language independence. It's our python code that we use that's the crux of the problem.

In fact it sounds like you don't care that we use jupyter at all. You just want us to eliminate our python code and push that same functionality into jupyter itself.

This might be what we do to solve multiple languages. Or we might solve different problems with other solutions. Variable enumeration and data fetching might be accomplished by talking to an attached debugger in the kernel. At least for python attaching a debugger has very negligible overhead.

[stevengj]

There is no current JMP for querying variable values. The %who line magic might be used, but it certainly won't page in MBs of dataframe data. We use python code to do this now.

Why do you need to query variable values in the front end?

You can always send an execute_request to the kernel to ask it to evaluate a variable, given the name, and send you back a mimebundle that tells you how to display the value. That's about as much as you can hope to do in a language-independent front-end.

We also switch directories in the kernel based on where the user ran code from.

Isn't this just the directory of the notebook file? Normally Jupyter front ends do this by simply running the "launch kernel" kernelspec command from the directory of the notebook, or whatever working directory you want to use.

"Magics" like %cd are specific to the IPython kernel — they aren't part of the Jupyter spec. Nor are they needed to set the working directory of the kernel, as explained above.

SVG formatting is similar. We use a line magic to do this now. Not sure it will work in all kernels.

SVG rich object display is possible for any kernel that sends image/svg+xml data as part of a mimebundle (e.g. in a display_data message).

If the user wants to tell a particular software package, e.g. matplotlib, to use a particular output format, they should do that themselves (e.g. by executing a notebook cell with the relevant line magic in the matplotlib case). I don't understand why you need to get involved.

[rchiodo]

We query variables here in the front end:

That's our variable explorer. It lists out the variables active in the current kernel and allows you to open a data viewer for anything that supports it. Opening the data viewer executes a bunch of python code to page in the data.

Switching directories in the kernel is done when more than one file is run in the same kernel. We might remove this (and instead have one kernel per file), which would mean the startup directory could be used.

SVG formatting is for our plot viewer support. We use it so we can open high res plots in a separate window.

Internally we ask Jupyter to output both SVG and PNG (with a %config line magic). When the user clicks on that little expand button, we open the SVG. We're likely going to get rid of this too as it slows down Jupyter a bunch on really complicated stuff and potentially inject something into matplotlib or the ipython kernel to instead give us the necessary data to draw a high res plot.

Variable support is really the biggest blocker right now. The other stuff can probably just be dropped.

(We also used to enforce a styling on matplotlib, but we got rid of that code as it caused more trouble than it was worth).

[stevengj]

For the variable explorer, in the short term you could simply not support it for non-Python kernels. Other notebook interfaces don't have this, so I think most people could live without.

To do this in a language-independent way in the long term, you would need:

• a message to request a list of variables — the Jupyter folks might be willing to add this to the messaging spec.

• for each variable, you could simply send an execute_request to get a "static" view of the variable (e.g. a mimebundle) as needed

• for more dynamic views of the variable (e.g. live updates, a zoomable plot, or whatever), the language-independent way to do this would be to use Jupyter's widget framework to allow the kernel to communicate with a Javascript widget dynamically.

[al6x]

Is there something you don't like about having jupyter installed?

A little bit off-topic, feel free to ignore, maybe I miss something. I don't care about Jupyter, I don't want to see it or know about it and I don't want to have it as an extra layer to debug in my code. Basically the only thing I need is a REPL-like script with support for Images (graphs) and VSCode syntax and auto-corrections. The easier it will be and less dependencies it has the better. Seems like straightforward way would be - just start Python (or Julia or Node.JS) process, send it code, get back result and/or image, show it in VSCode, no need for Jupyther. Yes, with that approach you need to have an adapter for each language (I guess that's how you currently using jupyther, but it seems like an overkill for that).

[aolney]

It might be possible to leverage SoS for this, particularly SoS Notebook, which uses a superkernel in Python to access other kernels in a single notebook. It's pretty mature and actively maintained.

I've routinely used it for the past year with Python, R, F#, Torch, and Java. AFAIK you can "drop in" any Jupyter kernel. Special extensions are only needed for features like sending data between kernel (which I never use because you can always hit the disk).

Tagging @BoPeng for his thoughts.

[davidanthoff]

I think the medium term plan here should really be to actually move the Jupyter Notebook support out of the Python extension, and revitalize the Jupyter extension, so that it supports notebook support in a language neutral form. I think anything else is really at odds with the Jupyter goals/philosophy, to be honest.

[BoPeng]

I agree with @davidanthoff that Jupyter support should be removed from the Python extension, and be enhanced to support more languages.

[andycraig]

@rchiodo Thank you for all your work on this extension! I want to highlight that most people are voting directly on the languages rather than upvoting the main issue post, so while the issue itself has only 10 votes, Julia now has 73 votes and R has 35. It seems like there is a fair amount of interest in broader language support.

I’m a contributor to one of the VSCode R extensions and I would love to start adding features to take advantage of R Notebooks in VSCode. If there is something that I can do to help this process along please let me know.

Thank you!

[rchiodo]

Hi @andycraig. This is something we're actively pursuing but is waiting right now on VS code doing some work to support notebooks better. We're waiting for VS code to finalize their designs and then we'll be able to describe how other languages will/can work.

[andycraig]

@rchiodo I appreciate the update and great to hear that there’s still a plan for broader language support for Notebooks in VSCode.

[dcuccia]

This is something we're actively pursuing

@rchiodo that's great news. Where is the best place to follow this progress? FWIW, I just posted on the dotnet-interactive group about .Net integration to the VS Code Jupyter integration:

https://github.com/dotnet/interactive/issues/179

Would this be appropriate as a separate feature request, or is it already being effectively tracked here (or elsewhere)?

[rchiodo]

.NET in particular is being tracked here:
https://github.com/microsoft/vscode-python/issues/7145

Otherwise, yes, this issue is the one to track.

[davidanthoff]

@rchiodo do you plan to move the Jupyter Notebook out of the Python extension entirely and into a language agnostic package?

[rchiodo]

Short answer, not at this time no.

Longer anwer:
We require the code in the python extension just to get jupyter to run with all of the different environments that are supported. However there's a bunch of stuff in flux about starting jupyter, starting the kernel, what UI we use etc. So the future may change where all of this lives.

[davidanthoff]

So even when the new notebook stuff in VSCode itself lands, the Python extension would register itself as the "handler" for Jupyter notebooks? To be honest, that seems really, really unfortunate and not in the spirit of the Jupyter project... Other projects (like nteract) handle this properly, one can easily install nteract, Julia and its Jupyter support and everything works, with no piece of Python installed anywhere. I think supporting that scenario should be a high priority for the VS Code Notebook story.

In my mind this all amounts to almost being a "bad citizen" in this ecosystem: essentially the Python extension here is "grabbing" a file format that is not Python specific but explicitly language neutral... (and just to be clear, I see zero bad intention around this from anyone invovled, please don't interpret the language I'm using here in any case as a criticism of any person involved!! It just seems the easiest language to describe the technical situation here).

[rchiodo]

So even when the new notebook stuff in VSCode itself lands, the Python extension would register itself as the "handler" for Jupyter notebooks?

This is unclear at this time. We may only register as the provider of the IPython kernel and nothing else. We may create another jupyter extension that uses the python extension as a dependency in order to start jupyter. (Depends upon how cells are going to be run in the new VS code world).

But I'm not sure why anybody would care which extension they need to install? It's an implementation detail really. Is it merely the branding?

[davidanthoff]

uses the python extension as a dependency in order to start jupyter

I don't understand that. Why would you want to involve the Python extension or Python at all in starting Jupyter kernels? I think that is really the crux here, I think the right way to implement this is to completely leave out anything Python in that step. You just look in these locations for kernel.json files, and start the binaries specified in there directly, with no need for anything Python at all.

But I'm not sure why anybody would care which extension they need to install? It's an implementation detail really. Is it merely the branding?

Folks that use Jupyter notebooks with other languages don't want to install something like the Python extension, which adds UI around Python, notifications around Python etc. If I'm a Julia user that uses Jupyter notebooks, why should I have to add an extension that adds lots of Python related commands, settings etc. into VS Code to edit Jupyter notebook files that have nothing to do with Python?

[rchiodo]

I don't understand that. Why would you want to involve the Python extension or Python at all in starting Jupyter kernels?

It's not to start the kernel, it's to start Jupyter itself. We attempt to launch jupyter using the user's python environment. There's a lot of code to get different terminals, virtual environments, etc to work. The python extension already has this code to run normal python files, we reuse it to start the jupyter server.

We implemented talking to the kernel this way so we'd talk to a local server or a remote server the same way. (Both local jupyter server and remote jupyter server uses the exact same code to execute cells, etc. We use the jupyter_lab APIS)

Folks that use Jupyter notebooks with other languages don't want to install something like the Python extension, which adds UI around Python, notifications around Python etc.

If they don't use Python, then none of these things should get in the way. I guess we'd only consider not using the python extension if we got a ton of negative feedback from a large population of non python users. So far that hasn't happened.

[stevengj]

As explained above, in principle you don’t need a jupyter process at all. You just talk directly to the kernel with ZMQ. This is what e.g. nteract does. Ideally, your goal should be to eliminate the jupyter dependency, which will also separate this extension from Python.

[davidanthoff]

Here is another way to look at this:

From the Julia extension point of view, we will want to have support for Jupyter Notebooks in VS Code that works without a Python installation. If the only Jupyter notebook support that MS ships is going to be inside the Python extension and will require that Python is installed on a user machine (that is how I understand your current plans, right?), I'm 99% sure that we will roll our own support for Jupyter notebooks once the core notebook UI lands in VS Code.

Is that the vision MS has for this scenario? That every langauge extension ships its own support for Jupyter notebooks? I.e. the Python extension, the Julia extension, the R extension etc. each register themselves as an editor for ipynb files and then ship with their own implementation of Jupyter notebooks?

That seems really less than ideal, for at least two reasons:

• it will lead to a bad user experience, because if a user now has both the Julia and Python extension installed, I assume they would have to make a choice which of these extensions should handle the ipynb file whenever one opens one, right?
• it seems a really unfortunate duplication of effort, and one that the Jupyter team tried super hard to avoid by designing a language neutral story... I.e. the whole philosophy of Jupyter to me at least really seems to avoid exactly this kind of design/situation.

[rchiodo]

Is that the vision MS has for this scenario? That every langauge extension ships its own support for Jupyter notebooks? I.e. the Python extension, the Julia extension, the R extension etc. each register themselves as an editor for ipynb files and then ship with their own implementation of Jupyter notebooks?

No that's not the plan at the moment. Each of those would ship support for a kernel more than likely. Which would go along with language server support, debugging, etc. The idea is to have each extension be responsible for executing cells in its own language (at least that's what we've been bandying about internally).

We might use Jupyter in the python extension. Julia might just talk directly to the julia kernel instead.

Or we might make a generic jupyter extension that supports multiple kernels, where each extension just installs the kernel, but there's an inbetween extension that sits between VS code and actually executing.

Or maybe we'll split our remote server support into a separate extension and then the python extension just talks directly to an IPython kernel. The Julia extension talks directly to the Julia kernel, the R extension etc ...

However, right now, multiple language support is not high on the priority list. Our initial push was to support python data scientists, as that was the majority of the DS market.

I will use this conversation to inform what we ask for from VS code though.

[davidanthoff]

Each of those would ship support for a kernel more than likely.

The idea is to have each extension be responsible for executing cells in its own language

Why would a VS Code extension like the Julia extension be responsible for executing Jupyter notebook cells? In the Jupyter design that is the responsibiliy of the kernel, so that part seems entirely solved by Jupyter, with no need to involve language specific VS Code extensions at all?

Which would go along with language server support, debugging

My understanding of the Jupyter plans around debugging are that they are extending the base Jupyter protocol to support debugging (and have working code for that). In that scenario the debugging support would also be provided by the kernel and _not_ a language specific VS Code extension.

Julia might just talk directly to the julia kernel instead.

Why would Julia talk to the julia kernel? The notebook implementation should talk with the Julia kernel via ZMQ messages.

Or we might make a generic jupyter extension that supports multiple kernels, where each extension just installs the kernel, but there's an inbetween extension that sits between VS code and actually executing.

Why would the VS Code extensions install kernels? I think pretty much every Jupyter language already has an existing kernel setup story that is independent of VS Code, it seems not a good idea to get into the way of that by having VS Code extensions start to install kernels.

No that's not the plan at the moment.

However, right now, multiple language support is not high on the priority list. Our initial push was to support python data scientists, as that was the majority of the DS market.

I don't see how these two statements square. If you continue to ship a Python only notebook implementation in the Python extension, then we will exactly end up in the situation that I outlined above, and which you described as not being the plan :)

I will use this conversation to inform what we ask for from VS code though.

I've been following the notebook branch on the VS Code repo closely, and everything I'm seeing there is _exactly_ spot on and how I think things should be done (namely, nothing Jupyter or language specific at all). I'm specifically worried about the direction the Jupyter integration is taking here.

Here is how I think the Jupyter integration should look like: there should be one VS Code extension called "Jupyter". It should register itself as the handler for ipynb files. It should be entirely language neutral. It should, like nteract, communicate with kernels via ZMQ. It should _not_ rely on any Python Jupyter implementation in any way. It should ideally use the debug extensions of the Jupyter protocol to support debugging scenarios. If there are other features that the current existing Python extension has that can't be handled in a language neutral way, it would be fantastic if the MS team could reach out to the Jupyter team and try to find extensions of the Jupyter protocoll that enable such scenarios.

Is the group at MS that is working on the Jupyter stuff in contact with the core Jupyter team? If not, I think that would be very helpful, just to make sure the high level strategies are somewhat aligned. I'm sure you have those contacts, if not, let me know, happy to put you in touch (a large fraction of the Jupyter team is here on campus). I do think that when a big player like MS starts to enter an area like Jupyter, with a vibrant existing community, that it is important that the big player tries to reach out to that community, engages with it and tries to be a "good player" in that space by roughly aligning with the core philosophy of something like Jupyter. Right now, at least from the outside, the strategy that MS seems to have (at least to me) seems not alighed with the core Jupyter philosophy and design decisions.

Now, having said all of that, MS is obviously free to to whatever they want :) I think if the MS plans stay as Python-centric around Jupyter as they appear right now, then maybe the authors of some of the key language extensions that are interested in a notebook extension that follows the Jupyter spirit should band together and just create a language neutral extension ourselves? Maybe MS would be willing to hand the deprecated "Jupyter" extension over to the core Jupyter group and it could be hosted there? Or something like that.

[eightHundreds]

javascript/typescript

[rchiodo]

Just to summarize here:

• You think it's a bad idea to have extensions install kernels because there's already a process for installing kernels.
• You think there should be a single 'jupyter' extension that supports all languages and it would find and talk to the different kernels (but not using jupyter itself)
• You think this extension would also support debugging once the debugging APIs are finalized.
• And if MS doesn't provide it, the community will.

I think this all depends upon how much work each of these steps are (and what our goals around these steps are).

For example, we run jupyter now for a number of reasons (instead of the kernel directly). Switching to a kernel only mode might be a lot of work, and not something we see as necessary.

We might also install the IPython kernel directly because we don't want customers to have to install it. We might install a number of other kernels too. (Especially if we have the one 'jupyter' extension).

I think the one jupyter extension goes against the spirit of VS code though. VS code provides a base experience and extensions support different areas. Is jupyter an area too big for an extension? I think that's a matter of opinion. Maybe not. They are creating uber extensions (like the Remote one) that encompasses larger bits. It would be nice to make it possible for other extensions to enhance the notebook experience though.

Oh and this bit:

If you continue to ship a Python only notebook implementation in the Python extension, then we will exactly end up in the situation that I outlined above, and which you described as not being the plan

Is not likely once the VS code API is complete. The notebook implementation will live inside of VS code itself. We'll either be providing just the code necessary to run python notebooks (in the python extension) or we'll make it more generic to support all notebooks (and possibly move it into a different extension). It depends upon a number of factors like branding and cost to implement.

[rchiodo]

To answer this question:

Why would a VS Code extension like the Julia extension be responsible for executing Jupyter notebook cells? In the Jupyter design that is the responsibiliy of the kernel, so that part seems entirely solved by Jupyter, with no need to involve language specific VS Code extensions at all?

Yes the kernel is still responsible for executing the code. There still needs to be a piece of code that:

• Finds the kernel
• Starts the kernel in whatever environment is appropriate (for python this requires running the kernel with a specific set of environment variables)
• Translates the new VS code API they're creating into the ZMQ protocol.

At first glance it seems like VS code should just use the ZMQ protocol directly (maybe we can get them to change to this), and VS code should just use the registered kernels on the system and start them directly.

I don't think VS code is going to want to search for kernels though. It will leave that up to an extension.

[dcuccia]

@rchiodo thanks for the pointer to #7145.

And, as a casual observer/user, your above post makes sense to me. FWIW, I will second a comment above that being Jupyter support being tied to this Python extension was initially confusing for me when trying to "discover" the extension I needed, as I knew already that Jupyter was multi-platform. Took 5 min, so no biggie, but it could be a barrier to adoption/discoverability.

[davidanthoff]

Just to summarize here:

Yes, your summary of my view is spot on!

I think the one jupyter extension goes against the spirit of VS code though. VS code provides a base experience and extensions support different areas. Is jupyter an area too big for an extension? I think that's a matter of opinion. Maybe not. They are creating uber extensions (like the Remote one) that encompasses larger bits. It would be nice to make it possible for other extensions to enhance the notebook experience though.

I think you raise an excellent point: there is a bit of a tension between the VS Code model (one extension per language) and the Jupyter model (language neutral protocols that make it feasible to have one UI that works well across different languages _without_ language specific extensions in the client). I'm not sure what the right balance there is, to be honest... I could imagine something like this: a Jupyter VS Code extension speaks with many kernels. It uses some Jupyter protocoll spec for debug, but then maybe has also some hook to allow the Python/Julia/R extension to provide say the language server for the code in a Jupyter notebook. But not clear to me whether that kind of extension-to-extension integration is actually feasible in the VS Code extension model right now?

I think this is an area where it would be particularly valuable if the MS team reached out to the Jupyter team and came up with a collaborative solution.

If you continue to ship a Python only notebook implementation in the Python extension, then we will exactly end up in the situation that I outlined above, and which you described as not being the plan

Is not likely once the VS code API is complete. The notebook implementation will live inside of VS code itself. We'll either be providing just the code necessary to run python notebooks (in the python extension) or we'll make it more generic to support all notebooks (and possibly move it into a different extension). It depends upon a number of factors like branding and cost to implement.

I don't understand this. What do you mean by "python notebooks"? The key question is which piece of code registers itself as the editor for the ipynb file type. My understanding of the core VS Code plans is that the core VS Code product will _not_ register itself as the handler for ipynb files. I think that is _exactly_ the right strategy! So it will have to be an extension that registers itself for that file type. If the Python extension registers itself as the handler for that file type, then I'm pretty sure extensions like the Julia extension will also register themsevles for the same file type and bring along their own notebook implementation, because we want a way to interact with ipynb files _without_ a need to install Python. If MS ends up providing the Jupyter extension as the handler for ipynb files on the other hand, well, then all is good, that is exactly what I am recommending :)

There still needs to be a piece of code that:

Finds the kernel

How to do that is clearly speced here, that really doesn't strike me as very complicated? There are essentially just a bunch of file locations that you need to look at...

Starts the kernel in whatever environment is appropriate (for python this requires running the kernel with a specific set of environment variables)

That is _completely_ speced in a language neutral form here as well! There is no need to do anything Python specific. You read the kernel.json, you run the stuff specified in the argv element and set the env variables to the values in the env element. The whole thing seems really not involved at all.

Translates the new VS code API they're creating into the ZMQ protocol.

Yes, that seems like one of the main "tasks" of a Jupyter extension. But that is code that must already existed here in the Python extension, right? Presumably the Python extension currently is communicating via ZMQ with the Python kernel?

At first glance it seems like VS code should just use the ZMQ protocol directly (maybe we can get them to change to this), and VS code should just use the registered kernels on the system and start them directly.

I really hope that is _not_ the direction! I think the notebook support in VS Code itself should be entirely language _and_ notebook tech stack neutral. All this info about kernels, ZMQ etc. is Jupyter specific, but there are many other notebook implementations and stories out there, and I think the support for notebooks in VS Code core should be entirely neutral with respect to which notebook stack it supports. So in my opinion it should have zero dependency on anything that is Jupyter specific (kernels, ZMQ or anything like that). The implementation they are working on right now follows that model exactly, i.e. what they are building right now has nothing Jupyter specific in it, which is great.

I think a Jupyter extension should find these registered kernels, and implement the Jupyter protocoll (via ZMQ), and understand the ipynb file format (i.e. provide the load and save functionality).

[rchiodo]

Finding kernels and launching kernels is easy if
1) A working jupyter is already installed
1) The user doesn't want to run a different python than the one registered in the kernel.
1) The kernels were installed in the default spot. (This isn't always the case)

Right now number 1 is a prerequisite for us, but number 2 is a non-option. The python extension supports running all sorts of python environments on the user's box and we don't want to make them install a kernel for each. This is where the python specific knowledge comes in.

So ideally there'd be a base Jupyter extension that just supports talking and using the kernels registered with the user's jupyter install, and then another way for other extensions to register their own notebook executors (which don't even need to use the jupyter kernels).

The python extension would do its own custom execution (well probably just install ipython like we do ptvsd, and run the user's python environment to setup the write python paths, etc, and then talk over ZMQ to the ipython kernel). And we're not using ZMQ at the moment. We use the jupyterlab rest APIs. This allows us to talk to a local and remote jupyter server the same way.

Language servers should end up being handled the way they are now. VS code will just pass the 'document' for the notebook to the appropriate language server. We do that now in our custom notebook implementation (although notebooks with multiple languages might be tougher).

Perhaps we'll end up with something like so:
1) A jupyter kernel extension. It finds jupyter kernels and allows them to be used to execute any code. It would talk to them all through the ZMQ protocol.
1) A python extension. It allows the user to use any python environment to execute code on a notebook. It doesn't use the kernel.json to run the kernel, but instead ships IPython itself. This allows non jupyter users to run python code in notebooks and the Interactive Window.
1) A remote extension. This would support talking to a remote jupyter server and would use the Jupyter lab REST API.
1) Language servers implemented by other extensions. These should just work as long as VS code gets the TextDocument for the server correct. VS code is currently discussing whether or not Language servers will need to know about notebook cells or not.

All of that should solve our requirements (python still working the way it does now) and not require everybody under the sun to have their own notebook execution.

[stevengj]

Finding kernels and launching kernels is easy if: A working jupyter is already installed

A working jupyter does not need to be installed, and in fact is not involved at all for an arbitrary kernel.

You just look into the documented filesystem locations for a kernel.json file, parse that file with JSON, and it tells you exactly the path of the executable to launch, the arguments, and the required environment variables.

(Of course, for the IPython kernel, certain components of the Jupyter project are required to be installed for that python. That's not true of other language kernels, however, and normally the presence of a kernelspec file implies that the corresponding components for that language are installed.)

The user doesn't want to run a different python than the one registered in the kernel.

Then they should install another kernelspec file. This is how Jupyter works — each "kernel" specifies one path and one environment, and you install multiple kernelspecs if you want multiple options. Why is this so bad?

You could always add a vscode GUI for creating new kernelspecs if you want a nicer interface for specifying these options, and even a vscode-specific addition to the search directories if you don't want vscode-added kernelspecs to be visible from other Jupyter implementations.

The kernels were installed in the default spot.

The kernelspec file gives you the absolute path of the executable, regardless of where it was installed. The Jupyter spec says that the kernelspec files themselves can only be installed in a finite number of possible locations, configurable by the user via the JUPYTER_PATH environment variable

I strongly agree with @davidanthoff that you should start a dialogue with the Jupyter developers, who are very approachable.

[rchiodo]

You just look into the documented filesystem locations for a kernel.json file, parse that file with JSON, and it tells you exactly the path of the executable to launch, the arguments, and the required environment variables.

We've found this documentation to be lacking in some cases. In those cases, Jupyter itself could find the kernels because they were installed next to jupyter (but they weren't in the documented locations)

Then they should install another kernelspec file. This is how Jupyter works — each "kernel" specifies one path and one environment, and you install multiple kernelspecs if you want multiple options. Why is this so bad?

Because users don't normally have to install kernelspecs. They just run Jupyter and it works (likely because they run Jupyter from the appropriate environment). Additionally we want to support users that don't know about or care about kernels. They just have some '.ipynb' file that somebody gave them and they want to use it. Somebody using VS code doesn't necessarily even know what Jupyter is.

The kernelspec file gives you the absolute path of the executable, regardless of where it was installed.

This is also not always true. Sometimes it just says 'python'.

Microsoft does talk with the jupyter development team on a number of things. I believe that's premature in this case though. VS Code is coming up with an extensibility model for executing cells inside of VS code. When this is more finalized, then they'd put out a general call for comments. I believe at that time we'd have a discussion with Jupyter. Not sure what we'd even discuss right now (other than possibly the discrepancies in the docs vs reality).

[chrismmuir]

This would be a good spot to have people vote on other languages. Which ones do you want besides python?

Julia, please!
Also, R & Haskell would be nice as well.

[dcuccia]

C#, F#, R

[stevengj]

Additionally we want to support users that don't know about or care about kernels. They just have some '.ipynb' file that somebody gave them and they want to use it.

To use a Jupyter notebook file, I don't see how you can get around requiring that the corresponding language's kernel software (e.g. ipython) be installed. But you want to handle the case where the software is installed but misconfigured (missing a kernel file in the standard location)?

Requiring vscode to work around mis-configured software seems like asking for trouble…

Look, I get that you've implemented something that works for you now, and here we are telling you that you should throw it in the trash and re-implement a separate plugin using ZMQ, which will maybe have less functionality in the short term for the Python case that you really care about because it will be limited to the Jupyter protocol. That's painful, and I understand why you would be reluctant. But a Python-centric non-ZMQ non-kernelspec implementation of Jupyter notebooks is really abandoning most of the Jupyter ecosystem and will be a continuing headache going forward.

[rchiodo]

To use a Jupyter notebook file, I don't see how you can get around requiring that the corresponding language's kernel software (e.g. ipython) be installed. But you want to handle the case where the software is installed but misconfigured (missing a kernel file in the standard location)?

We will still be using the IPython kernel to run their code. We just plan on installing it for them (either directly into their environment or next to our extension and just pointing the correct sys paths to it). This is a work in progress and is mostly motivated by the 1000s (out of 10,000s of users) of failures we get with jupyter running or getting kernelspecs to change.

But a Python-centric non-ZMQ non-kernelspec implementation of Jupyter notebooks is really abandoning most of the Jupyter ecosystem and will be a continuing headache going forward.

We don't have this now, nor do we plan on going this way. Our current implementation is using jupyter itself to talk the kernels. This is rather messy because jupyter expects to be run from a single environment (well in most cases). Relying on the user's jupyter installation is turning out to be rather fragile. Talking directly to kernels (over ZMQ) seems like it will be much simpler. The question just becomes who does this talking for other languages. Will we (meaning the python extension) talk to all kernels? Will we create another extension that encompasses all of them?

I'm personally worried about having a monolithic Jupyter extension for VS code that is supposed to be all things for all users. VS code's ecosystem doesn't normally work this way.

Here's an example, you as a Julia developer want to add something that tracks dataframe usage in notebooks so you can present some nice new UI in VS code. How would you do that without being the one true Jupyter extension? Would you have to figure out how to modify the Julia kernel then? Does the Jupyter extension have a way to allow listeners for executing code?

I think the answers to this last question will determine where and how Jupyter users run their kernels (meaning what extensions will they need to install).

[axsk]

A very good answer was already proposed by @davidanthoff:

I could imagine something like this: a Jupyter VS Code extension speaks with many kernels. It uses some Jupyter protocoll spec for debug, but then maybe has also some hook to allow the Python/Julia/R extension to provide say the language server for the code in a Jupyter notebook.

Just to paraphrase in my own words (since i deem some misunderstandings here):

The kernels themselves should not be touched by any VS Code extension, they have their task, and that is running code. The Jupyter VS Code extension could communicate I/O of the UI via the VS Code API. It has to be the task of the VS Code language extensions to account for say syntax highlighting or visualization of tables, so they will have to have some hooks to the Jupyter VS Code extension on the VS Code side where they possibly launch some language server in the running kernel for further language specific communications.

[ipstone]

Hello, I am using jupyter lab mostly with R for the moment. vscode-python would display the cached images, code perfectly - which is already very useful.

But I really wish it can take my installed R kernel and work directly in vscode. Currently, vscode python can find all the python installations on my system, but for the R kernel, it displays:

"Couldn't find kernel R that the notebook was created with".

Is there an option/confirmation that I can supply vscode python the kernel binary path?'

Thanks a lot
_

[rchiodo]

@ipstone It's just not supported at the moment. We're filtering out all kernels that aren't python.

This item here is to investigate supporting the C# kernel (which is currently in progress)
https://github.com/microsoft/vscode-python/issues/7145

Once that's finished, I imagine all kernels should be at least minimally supported. Colorization and intellisense probably won't be correct, but at least you'll be able to run code from other kernels.

[MatthewSteen]

ruby via iruby

[Photonico]

Hello,
I'm using Julia and Python kernel frequently So that I desire the Julia support on Jupyter Notebook.
Could you tell me your plan or progress on this request?

[rchiodo]

The long term plan is to support all languages. Or at least all kernels. It will likely be many months before that happens as notebook support is being revamped.

VS code proper will control all of the editor features, and our team will most likely support language server (for python), running kernels (python and any other on your box), and providing handlers for different mime types. It looks like anybody will be able to provide an extension that can do these things and some parts may come from multiple extensions.

[Photonico]

The long term plan is to support all languages. Or at least all kernels. It will likely be many months before that happens as notebook support is being revamped.

VS code proper will control all of the editor features, and our team will most likely support language server (for python), running kernels (python and any other on your box), and providing handlers for different mime types. It looks like anybody will be able to provide an extension that can do these things and some parts may come from multiple extensions.

Thank you for your information. I expect it so much!
I hope that I can use it before 2021.

[jmora]

Thank you for the good work. The ones I use most often besides of Python are Coconut and Clojure.

I use a number of extensions with Clojure, like Parinfer. It would be good to have an editor like VSC that supports these extensions to edit the code in the cells of the notebook and that supports running the cells, displaying the plots, etc., like VSC for Python.

[sanme98]

The long term plan is to support all languages. Or at least all kernels. It will likely be many months before that happens as notebook support is being revamped.
VS code proper will control all of the editor features, and our team will most likely support language server (for python), running kernels (python and any other on your box), and providing handlers for different mime types. It looks like anybody will be able to provide an extension that can do these things and some parts may come from multiple extensions.

Thank you for your information. I expect it so much!
I hope that I can use it before 2021.

Reemphasize, hope before 2021 mean available in 2020 :). Thanks a lot.

[sam-dehority]

I am also very excited for this feature. It should be noted that this feature is available as an extension in atom https://nteract.io/atom although it's not really as high-powered as the jupyter extension for python is for vscode.

Theirs is built with the design decision that the core functionality should work with any kernel and specific languages are provided by new extensions.

[hdavid16]

It's great to see Jupyter notebooks in Julia working on VS Code. How can we get Julia syntax highlighting in the jupyter files that are running with the Julia kernel?

[rchiodo]

@hdavid16 Julia syntax highlighting requires us to find the language-configuration.json file for the julia language in the global extensions folder for VS code. It looks like VS code does not ship a language for Julia so we won't find the language tmlanguage file.

For example, in my install of VS code, I have this directory here:

C:\Users\Rich\AppData\Local\Programs\Microsoft VS Code\resources\app\extensions\csharp

That has a language-configuration.json file for C# that looks like so:

{
    "comments": {
        "lineComment": "//",
        "blockComment": ["/*", "*/"]
    },
    "brackets": [
        ["{", "}"],
        ["[", "]"],
        ["(", ")"]
    ],
    "autoClosingPairs": [
        ["{", "}"],
        ["[", "]"],
        ["(", ")"],
        { "open": "'", "close": "'", "notIn": ["string", "comment"] },
        { "open": "\"", "close": "\"", "notIn": ["string", "comment"] }
    ],
    "surroundingPairs": [
        ["{", "}"],
        ["[", "]"],
        ["(", ")"],
        ["<", ">"],
        ["'", "'"],
        ["\"", "\""]
    ],
    "folding": {
        "markers": {
            "start": "^\\s*#region\\b",
            "end": "^\\s*#endregion\\b"
        }
    }
}

If there was a similar file for Julia, the syntax highlighting should start working. Since VS code doesn't ship one, you'd have to add one manually to that location or change our code to look elsewhere for it.

[hdavid16]

Thank you @rchiodo for pointing me in the right direction. I was able to get it to work by adding a julia folder inside the extensions folder you mentioned and copying the necessary files from https://github.com/julia-vscode/julia-vscode. Syntax highlighting with Jupyter files running on Julia now works.

[jlperla]

@hdavid16 Is it possible to prepare a PR or explain what you did to get it running? It would be great to put in an update to the vscode julia extension.

[jlperla]

@rchiodo I don't think it is kosher to have the Julia extension install that language support. See https://github.com/julia-vscode/julia-vscode/issues/1620

Can you help us navigate the process to get this into vscode in a way that would be compatible with the jupyter/notebook extension features?

[FuZhiyu]

@rchiodo Thanks for the workaround. It works perfectly for Julia on my laptop. Nevertheless it doesn't seem to work if the notebook is running on a linux server. Is it possible to also have a similar trick to deal with the remote case?

[rchiodo]

@rchiodo I don't think it is kosher to have the Julia extension install that language support. See julia-vscode/julia-vscode#1620

Can you help us navigate the process to get this into vscode in a way that would be compatible with the jupyter/notebook extension features?

@jlperla, The long term support for Julia and other languages will just work once we switch over to VS code's native notebook support. (Cells will be treated like a normal text document instead of the hacky stuff we're doing now in the python extension).

So I don't think it's worthwhile to hack something together so that we can find the language configuration file. It's really just a temporary solution. Although your issue you filed will likely make it work.

[rchiodo]

@FuZhiyu I would suspect the hack we have isn't working with remote because you need to put the language configuration file in a specific location on the remote machine. Look for a folder with the resources\app\extensions and put the language configuration file under a similar location as to where you put it on windows.

[jlperla]

@rchiodo Thanks so much. Looks like things are moving forward on doing it right! https://github.com/microsoft/vscode/issues/105229#issuecomment-683688014

Are there any other things that we should consider getting integrated for first-class julia support in the extension?

As an aside: From a discoverability perspective in the medium-term, were you guys thinking about renaming your extension to make things directly connected to python - or splitting into a jupyter/datascience and a python extension (where the first might require the second?)

[rchiodo]

Are there any other things that we should consider getting integrated for first-class julia support in the extension?

The last remaining bit I can think of Julia support is support for the variable explorer. We attempted to make variable gathering language neutral but it doesn't seem to work.

There's a setting in the extension:

                "python.dataScience.variableQueries": {
                    "type": "array",
                    "description": "Language to query mapping for returning the list of active variables in a Jupyter kernel. Used by the Variable Explorer in both the Interactive Window and Notebooks. Example: \n'[\n{\n   \"language\": \"python\",\n   \"query\": \"%who_ls\",\n  \"parseExpr\": \"'(\\\\w+)'\"\n}\n]'",
                    "scope": "machine",
                    "examples": [
                        [
                            {
                                "language": "python",
                                "query": "_rwho_ls = %who_ls\\nprint(_rwho_ls)",
                                "parseExpr": "'(\\w+)'"
                            },
                            {
                                "language": "julia",
                                "query": "whos",
                                "parseExpr": "'(\\w+)'"
                            }
                        ]
                    ]
                },

That is supposed to allow a user to specify how to retrieve the list of variables. The parseExpr is a regex that we use to split up the results. With each result we then pass that onto a inspect request from the kernel to get the value.

The last time I tried this with Julia the inspect request wasn't working (maybe the returned format doesn't look the same as the format returned by ipython so we fall apart trying to extract the string value). So not sure if this is something you could look at or not. It might require changes in our extension or changes in the Julia kernel.

As an aside: From a discoverability perspective in the medium-term, were you guys thinking about renaming your extension to make things directly connected to python - or splitting into a jupyter/datascience and a python extension (where the first might require the second?)

I believe we're looking into making a 'jupyter' extension. This is at least being discussed. Ideally it would be language neutral and provide the glue to make jupyter kernels and languages work in VS code all up. It would likely have a dependency on the python extension for the cases when our ZMQ libraries don't work.

[andycraig]

Looks like the Jupyter extension now exists: https://github.com/microsoft/vscode-jupyter

[DonJayamanne]

Yes, over the next few days, we'll be moving these issues over to that repo.
Please have a look at the native notebook support in the new extension (we do support more languages in that notebook) & also https://github.com/microsoft/vscode-jupyter/issues/273 for support of other languages in interactive window