Conda-forge.github.io: Packaging JupyterLab extensions

[dbast]

What is the canonical way to provide JupyterLab extensions:

• For some extensions like jupyterlab-git, the package is a combination of a server extension written in python + frontend code in nodejs. Providing this as conda package makes the installation easier.
• other extensions like jupyterlab-plotly-extension are frontend only (contain no python part) and could instead also be installed via the JupyterLab Extension manager only...

In both cases JupyterLab requires to do a nodejs extension rebuild, which would be nice to have it already in a pre-build package, but afaik currently by design can not be done during the conda package build.

The npm stats show that many frontend extensions are installed from npm directly.

On the other side having frontend only extensions all with a version specified as part of an environment.yaml, is still appealing, even though the nodejs rebuild step+time is remaining. That is what came up during packaging and proceeding in https://github.com/conda-forge/staged-recipes/pull/8060

Already available packages:

• Server + Frontend https://github.com/conda-forge/jupyterlab-git-feedstock
• Frontend only https://github.com/conda-forge/jupyterlab-plotly-extension-feedstock

Potentially packages are e.g:

• Frontend only https://github.com/jupyterlab/jupyter-renderers (some are already integrated into default JuputerLab)
• Frontend only https://github.com/bokeh/jupyterlab_bokeh
• Server + Frontend https://github.com/dask/dask-labextension

To connect the dots, also linking to upstream https://github.com/jupyterlab/jupyterlab/issues/2065

Tagging @blink1073 @conda-forge/jupyterlab

[scopatz]

CC @conda-forge/core

[jasongrout]

Also ipywidgets is another example (or many of the jupyter widgets packages) - requiring a separate python or conda install of the python package, and a jupyter labextension install of the javascript side from npm.

[croth1]

@jasongrout, I remember being a bit confused not long ago. The notebook extensions of ipywidgets work right out of the box, but the jupyterlab requires manual installing the lab extensions. Do you think we can find a good automatic solution for lab as well?

[hadim]

I got errors trying to package a server+JLab extension too on conda: https://github.com/conda-forge/jupyter-archive-feedstock/issues/6

Any general solution would indeed very welcome.

[moorepants]

Just throwing in my 2 cents as I've been struggling with how I'm supposed to get plotly graphs to show up in Jupyterlab via conda install commands. From a user's perspective, I expect "conda install plotly" to give me a fully working plotly install, including the display of figures in Jupyter notebooks classic and JupyterLab (note that the plotly feedstock is actually a plotly.py installation). But, second best option would be that I have to do "conda install plotly plotly-jupyterlab plotly-notebook". Having to run any labextension commands (without proper JS deps installed) is just confusing. I don't understand why Jupyter seems to have its own package management system, but it is a complication and source of confusion for users.

[scopatz]

FWIW, there are now these instructions for packaging JLab ext: https://conda-forge.org/docs/maintainer/knowledge_base.html#jupyterlab-extensions

[moorepants]

Nice! That's super helpful and good to have a best practice recommendation.

[moorepants]

I followed the instructions in the documentation and seem to be successfully dropping the node package tarball in the right location, but when I open Jupyterlab or run jupyter lab build the extension is not recognized and nothing is built. Is there something I'm missing. The open PR is here: https://github.com/conda-forge/plotly-feedstock/pull/85

[scopatz]

CC @saulshanabrook

[moorepants]

It looks like JupyterLab 3.0 has a new way to distribute extensions as Python distributions (pip or conda):

https://jupyterlab.readthedocs.io/en/latest/getting_started/changelog.html#extensions-can-be-installed-without-building-jupyterlab-with-nodejs

[jasongrout]

Here's an example jlab extension package just approved yesterday: https://github.com/conda-forge/jupyterlab_widgets-feedstock/blob/master/recipe/meta.yaml that uses the new jlab 3.0 extension system.

[jasongrout]

FYI, JLab 3.0 is not released yet, but the plan is to release the week after JupyterCon, which is this week and next (so release in two weeks).

[moorepants]

So what is the recommendation for conda forge? That the extension is a separate package? For example, should a user expect "conda install jupyterlab plotly" to provide working plotly figures in Jupyter Lab or should it be "conda install jupyterlab plotly plotly-jupyterlab-extension"?

[jasongrout]

I can see good arguments both ways. Making a separate package allows you to decouple version bumps of jlab and plotly, but it is a bit more complicated for the end-user.

For right now, for the ipywidgets core package, we have the jlab package as a separate package to decouple version bumps, as well as make it cleaner to install the python kernel side separate from the frontend jlab extension side.

To try out this new ipywidgets jlab extension, you'd do conda install jupyterlab=3 ipywidgets jupyterlab_widgets -c conda-forge/label/jupyterlab_rc -c conda-forge/label/jupyterlab_server_rc -c conda-forge/label/jupyterlab_widgets_rc. Of course, all those -c channels go away when all the packages are finally released.

[jasongrout]

Note that you can provide a plotly-jupyterlab-extension package and just have plotly depend on it, to get both version decoupling and an easy user experience. If you do that, then the question is whether plotly-jupyterlab-extension should depend on jlab or not. If it does, then plotly depends on jlab, which is perhaps kind of heavy? If it doesn't, then just installing the plotly-jupyterlab-extension won't do anything by itself, you also have to install jlab.

[jasongrout]

If it doesn't, then just installing the plotly-jupyterlab-extension won't do anything by itself, you also have to install jlab.

(Also, if it doesn't, you probably want to at least make a run constraint of the appropriate jlab version, but then there's no way to do that well in python pip until they ship their new solver and enable an equivalent feature)

[jasongrout]

It looks like JupyterLab 3.0 has a new way to distribute extensions as Python distributions (pip or conda):

To be clear, IIRC the JLab 2.0 way of installing and distributing extensions still works as well. The new extension distribution mechanism in 3.0 is in addition to the 2.0 way.

[moorepants]

If it does, then plotly depends on jlab, which is perhaps kind of heavy?

Yes, that should be avoided.