Ipywidgets: IPywidget error in Jlab 2.0

[mattiaswangblad]

I have an ipywidgets extension that works on jlab 1.2.6 but getting some trouble on jlab 2.0 with
jupyterlab-manager 2.0

I'm getting a:

Error: No provider for: jupyter.extensions.jupyterWidgetRegistry.

I'm getting some duplicates on my build which from what I understand might cause this.

WARNING in d3-array
  Multiple versions of d3-array found:
    1.2.4 ./~/d3-geo-projection/~/d3-array from ./~/d3-geo-projection/~/d3-array/src/index.js
    2.4.0 ./~/d3-array from ./~/d3-array/src/index.js

WARNING in entities
  Multiple versions of entities found:
    1.1.2 ./~/entities from ./~/htmlparser2/lib/Tokenizer.js
    2.0.0 ./~/dom-serializer/~/entities from ./~/dom-serializer/~/entities/lib/index.js

WARNING in semver
  Multiple versions of semver found:
    6.3.0 ./~/semver from ./~/@jupyter-widgets/jupyterlab-manager/lib/manager.js
    7.1.3 ./~/vega-embed/~/semver from ./~/vega-embed/~/semver/index.js

Is this an issue in jupyterlab manager or am I missing something else on my dependencies?

[jasongrout]

Can you share exactly what you are doing to migrate it? It usually is just a matter of upgrading your dependency to also include @jupyter-widgets/jupyterlab-manager ^3: https://github.com/jupyter-widgets/ipywidgets/issues/2798#issuecomment-593716999

[consideRatio]

I've been working a long time to try to resolve this, but i fail. This error shows for all my widget related extensions at the moment on JupyterLab startup.

My guess is that one of the extensions made me install an old version, and that influenced every extension. I will try to add jupyter lab clean --extensions before I install my set of extension in case some conda package has allowed itself to install stuff by itself or similar.

But, no matter, I'd love debugging help on this. I've made 10 PRs of to update extensions I used and disabling those that hasn't been updated and I'm now fighting this this remaining issue. Any help to spot what could cause a version mismatch it is greatly appreciated as I now manually go through all the extensions to figure it out. I think ipyvolume will need an update next for me.

(base) jovyan:~$ jupyter labextension list
JupyterLab v2.0.1
Known labextensions:
   app dir: /opt/conda/share/jupyter/lab
        @bokeh/jupyter_bokeh v2.0.0  enabled  OK
        @ijmbarr/jupyterlab_spellchecker v0.1.6  enabled  OK
        @jupyter-widgets/jupyterlab-manager v2.0.0  enabled  OK
        @jupyterlab/geojson-extension v2.0.0  enabled  OK
        @jupyterlab/latex v2.0.0  enabled  OK
        @jupyterlab/shortcutui v0.5.0-beta.2  enabled  OK
        @jupyterlab/toc v3.0.0  enabled  OK
        @lckr/jupyterlab_variableinspector v0.4.0  enabled  OK
        bqplot v0.5.5  enabled  OK
        ipyvolume v0.5.2  enabled  OK
        itkwidgets v0.26.1  enabled  OK
        jupyter-leaflet v0.12.3  enabled  OK
        jupyter-matplotlib v0.7.2  enabled  OK
        jupyter-threejs v2.1.1  enabled  OK
        jupyterlab-datawidgets v6.3.0  enabled  OK
        jupyterlab-kernelspy v2.2.0  enabled  OK
        jupyterlab-plotly v1.5.4  enabled  OK
        jupyterlab_iframe v0.2.2  enabled  OK
        nbdime-jupyterlab v2.0.0  enabled  OK
        plotlywidget v1.5.4  enabled  OK

base) jovyan:~$ jupyter --version
jupyter core     : 4.6.3
jupyter-notebook : 6.0.3
qtconsole        : not installed
ipython          : 7.13.0
ipykernel        : 5.1.4
jupyter client   : 6.0.0
jupyter lab      : 2.0.1
nbconvert        : 5.6.1
ipywidgets       : 7.5.1
nbformat         : 5.0.4
traitlets        : 4.3.3

PRs so far:

• [x] https://github.com/ijmbarr/jupyterlab_spellchecker/pull/19
• [x] https://github.com/jupyterhub/jupyter-server-proxy/pull/174 (Needs release)
• [ ] https://github.com/QuantStack/jupyterlab-drawio/pull/52
• [x] https://github.com/plotly/plotly.py/pull/2245
• [x] https://github.com/plotly/plotly.py/pull/2261
• [x] https://github.com/bokeh/jupyter_bokeh/pull/90
• [x] https://github.com/timkpaine/jupyterlab_iframe/pull/54
• [ ] https://github.com/chaoleili/jupyterlab_tensorboard/pull/21
• [ ] https://github.com/quantopian/qgrid/pull/301
• [ ] https://github.com/jupyter-widgets/jupyterlab-sidecar/pull/38 (Needs test error fixed)

[jasongrout]

@consideRatio - you can go into the actual staging directory and figure out from there which extension is pulling in older versions. Do jupyter lab path to find your application directory, then go into that directory's staging subdirectory. You can inspect the yarn.lock file to see what is being pulled in, and you can use yarn why to ask it to trace the dependencies.

[consideRatio]

Thank you @jasongrout !!!

[consideRatio]

My issues went away by either removing ipyvolume or running jupyter lab clean --extensions before I installed all my extensions in one go (Dockerfile, but i have some conda packages that install before so they may have done some installation ahead of time).

[mattiaswangblad]

Can you share exactly what you are doing to migrate it? It usually is just a matter of upgrading your dependency to also include @jupyter-widgets/jupyterlab-manager ^3: #2798 (comment)

I basically just followed those guidelines and upgraded the dependencies to

@jupyter-widgets/jupyterlab-manager": "^2.0.0",
"@jupyter-widgets/base": "^3",
"@jupyterlab/coreutils": "  ^4.0.0",

I only have my extension except for jupyterlabmanager so shouldn't be anything else really that has old dependencies

[jasongrout]

@jupyter-widgets/jupyterlab-manager": "^2.0.0",

Why do you have a dependency on the jupyterlab manager?

[jasongrout]

Also, try jupyter lab clean to clean out any old dependencies. @mattiaswangblad - what do you get doing jupyter labextension list? Is your code somewhere online?

[mattiaswangblad]

Why do you have a dependency on the jupyterlab manager?

Good question. I guess it might be that it didn't work before jpupyte lab manager version 1.0.2 previously after the jlab 1 release.

Also, try jupyter lab clean to clean out any old dependencies. @mattiaswangblad - what do you get doing jupyter labextension list? Is your code somewhere online?

Tried all kinds of cleaning without success.

jupyter labextension list JupyterLab v2.0.1 Known labextensions: app dir: /Users/mattiasw/anaconda3/share/jupyter/lab @jupyter-widgets/jupyterlab-manager v2.0.0 enabled OK pyesasky v1.2.9 enabled OK
My code (pyesasky) can be found here https://github.com/esdc-esac-esa-int/pyesasky (without any updates for jlab 2)

[jasongrout]

My code (pyesasky) can be found here https://github.com/esdc-esac-esa-int/pyesasky (without any updates for jlab 2)

Your code doesn't ever import anything from the jupyterlab manager, I think you should delete it as a dependency: https://github.com/esdc-esac-esa-int/pyesasky/blob/master/package.json#L34

Same with your jupyterlab dependency - I don't see you importing it anywhere.

Does https://github.com/esdc-esac-esa-int/pyesasky/pull/7 work?

[mattiaswangblad]

Yeah you're probably right about the dependencies but unfortunately didn't help

[jasongrout]

I tested this and it seems to install and compile (I didn't check it in an actual notebook):

conda create -n testenv jupyterlab=2 nodejs ipywidgets
conda activate testenv
jupyter labextension install @jupyter-widgets/jupyterlab-manager
git clone [email protected]:esdc-esac-esa-int/pyesasky.git
# Edit to introduce changes in esdc-esac-esa-int/pyesasky#7
jupyter labextension install ./pyesasky

I don't see the errors you are seeing above. Can you try like this with a clean environment?

[consideRatio]

@jasongrout and for anyone else future reference debugging with the helpful https://github.com/jupyter-widgets/ipywidgets/issues/2826#issuecomment-601135957 earlier in this issue, this is what it looked for me debugging from a Docker image

jovyan@b7548eedef72:/opt/conda/share/jupyter/lab/staging$ node /opt/conda/lib/python3.6/site-packages/jupyterlab/staging/yarn.js why @jupyter-widgets/base
yarn why v1.21.1
[1/4] Why do we have the module "@jupyter-widgets/base"...?
[2/4] Initialising dependency graph...
[3/4] Finding dependency...
[4/4] Calculating file sizes...
=> Found "@jupyter-widgets/[email protected]"
info Reasons this module exists
   - "jupyterlab-datawidgets" depends on it
   - Hoisted from "jupyterlab-datawidgets#@jupyter-widgets#base"
   - Hoisted from "jupyter-threejs#@jupyter-widgets#base"
   - Hoisted from "jupyter-matplotlib#@jupyter-widgets#base"
   - Hoisted from "jupyter-threejs#jupyter-dataserializers#@jupyter-widgets#base"
   - Hoisted from "jupyterlab-datawidgets#jupyter-datawidgets#@jupyter-widgets#base"
info Disk size without dependencies: "220KB"
info Disk size with unique dependencies: "16.79MB"
info Disk size with transitive dependencies: "26.89MB"
info Number of shared dependencies: 35
Done in 0.77s.

I wrote some notes for myself in the Dockerfile following jupyter labextension install <list of packages> --no-build.

# NOTE: At this stage, it is a good time to do some jupyter labextension
#       debugging if something is wrong. It can be done by building the docker
#       file locally, and starting up from the recently built stage.
#
#       docker build -t dsp:local images/
#
#           Removing intermediate container 477702c9e081
#            ---> 8223874599ab
#
#       docker run -it 6a638c97beb7 bash
#
#           # identify where and go to our jupyterlab directory
#           jupyter lab path
#           cd /opt/conda/share/jupyter/lab/staging
#
#           # what dependencies are being requested?
#           cat yarn.lock
#
#           # what packages influences the resolution of a package?
#           node /opt/conda/lib/python3.6/site-packages/jupyterlab/staging/yarn.js why @jupyter-widgets/base

[mattiaswangblad]

Ok so I got it to work now. I guess my idea to update the extension doesn't really work.

I used to update with
npm pack jupyter labextension install pyesasky-1.2.10.tgz
without updating the version number between the dependency updates. Not entirely sure when in this process I first changed version number.

instead of jupyter labextension install .

which creates some random name on the package in the extensions folder. So I guess my updates got caught in some cache even though I ran jupyter lab clean --extensions

I'm a bit unsure which dependency removal fixed this. Still having the same build warnings so they were not the issue. Thanks a lot for all your help @jasongrout