Deck.gl: Pydeck (jupyter lab) does not work together with keplergl

[bergkvist]

I can't seem to get both keplergl and pydeck widgets to work in JupyterLab at the same time

keplergl alone works fine

pip install keplergl==0.2.1 && jupyter labextension install [email protected]

```py
import keplergl
display(keplergl.KeplerGl()) # Displays map

### pydeck alone works fine
```sh
pip install pydeck==0.4.1 && jupyter labextension install @deck.gl/[email protected]

import pydeck
display(pydeck.Deck(mapbox_key='...').show()) # Displays map

---

keplergl and pydeck at the same time does not work

pip install keplergl==0.2.1 && jupyter labextension install [email protected]
pip install pydeck==0.4.1 && jupyter labextension install @deck.gl/[email protected]

import pydeck
display(pydeck.Deck(mapbox_key='...').show()) # Error displaying widget: model not found

import keplergl
display(keplergl.KeplerGl()) # Displays map

---

Reversing installation order reverses the problem:

pip install pydeck==0.4.1 && jupyter labextension install @deck.gl/[email protected]
pip install keplergl==0.2.1 && jupyter labextension install [email protected]

import pydeck
display(pydeck.Deck(mapbox_key='...').show()) # Displays map

import keplergl
display(keplergl.KeplerGl()) # Error displaying widget: model not found

[kylebarron]

It's most likely a deck.gl versioning problem. There's no sandbox in the browser between widgets, and pydeck and keplergl aren't released together, so there's no easy way to be sure that the underlying JS part of the widget is using the same deck.gl version. Because there's no sandbox, the deck version from one will overwrite the deck provided with the other.

I think the necessary workaround is to use separate browser tabs or at least reload the page when switching between pydeck and keplergl.

[bergkvist]

Reloading the browser/using different tabs doesn't have any effect. To switch between keplergl and pydeck I actually need to uninstall/reinstall/rebuild the widgets. And this takes a long time (~10-15 minutes on my laptop).

An alternative would be to create 2 separate docker images and run both of them side-by-side (that way I don't need to wait 10-15 min every time I want to switch).

[bergkvist]

Since jupyterlab extensions are built using webpack, perhaps something like this could be used to avoid deck.gl version conflicts?

• https://stackoverflow.com/questions/49323299/can-webpack-bundle-different-versions-of-the-same-dependency
• https://stackoverflow.com/questions/26414587/how-to-install-multiple-versions-of-package-using-npm

// package.json
"dependencies": {
   "keplergl-deck.gl": "npm:[email protected]",
   "pydeck-deck.gl": "npm:[email protected]"
}

This syntax is available since npm v6.9.0 (Mar 6, 2019)

[ibgreen]

While I agree that deck.gl versions could be a source of problems, the error messages you are receiving seem more related to the jupyter widget API (jupyterWidgetRegistry, get_model, renderModel).

It could of course be some deck issue that prevents initialization but normally I would expect jupyter to initialize first.

@ajduberstein

[kylebarron]

FYI I tried this out and I see in the browser console:

Error: deck.gl - multiple versions detected: 8.2.7 vs 8.2.5
    at Object.push.WQvm (vendors~main.bb9e8da21be7567696e5.js:391674)
    at r (vendors~main.bb9e8da21be7567696e5.js:391657)
    at Object.push.WQvm (vendors~main.bb9e8da21be7567696e5.js:391674)
    at r (vendors~main.bb9e8da21be7567696e5.js:391657)
    at Module.push.WQvm (vendors~main.bb9e8da21be7567696e5.js:391657)
    at r (vendors~main.bb9e8da21be7567696e5.js:391657)
    at Object.<anonymous> (vendors~main.bb9e8da21be7567696e5.js:391674)
    at Object.push.WQvm (vendors~main.bb9e8da21be7567696e5.js:391674)
    at r (vendors~main.bb9e8da21be7567696e5.js:391657)
    at Object.push.WQvm (vendors~main.bb9e8da21be7567696e5.js:391657)

At first glance it _looks_ like that's the cause of the widget model not found errors.

[bergkvist]

This might have been caused by the JupyterLab version (v1.2.5). I'm not 100% sure if this is the reason - but I suddenly got both widgets working at the same time in JupyterLab v2.2.5 now.

In case you are interested, the docker image where I got it working is available at bergkvist/kognilab:v1.1.4 (Docker Hub):

Dockerfile (warning: quite large)

FROM jupyter/scipy-notebook:6d42503c684f as builder
ENV NODE_OPTIONS="--max-old-space-size=4096"

RUN pip install --no-cache-dir jupyter-lsp python-language-server[all] dask_labextension ipympl &&\
    jupyter serverextension enable dask_labextension

RUN jupyter labextension install \
        [email protected] \
        [email protected] \
        @jupyter-widgets/jupyterlab-manager && \
    rm -rf /opt/conda/share/jupyter/lab/staging && \
    fix-permissions $CONDA_DIR $HOME

# Split into 2 steps in order to reduce peak memory consumption
RUN jupyter labextension install \
        @krassowski/jupyterlab-lsp \
        dask-labextension \
        @deck.gl/[email protected] \
        [email protected] && \
    rm -rf /opt/conda/share/jupyter/lab/staging && \
    fix-permissions $CONDA_DIR $HOME



# Compress 5 copy instructions into a single layer in the final image
FROM scratch as labextensions
COPY --from=builder /home/jovyan/.config /home/jovyan/.config
COPY --from=builder /home/jovyan/.jupyter /home/jovyan/.jupyter
COPY --from=builder /opt/conda/etc/jupyter /opt/conda/etc/jupyter
COPY --from=builder /opt/conda/share/jupyter /opt/conda/share/jupyter
COPY --from=builder /opt/conda/lib/python3.8/site-packages /opt/conda/lib/python3.8/site-packages



FROM jupyter/scipy-notebook:6d42503c684f
COPY --from=labextensions / /
COPY --from=graphblas/lagraph:13June2020 / /

USER root
RUN fix-permissions $CONDA_DIR $HOME
RUN ldconfig
RUN apt-get update && apt-get install -yq --no-install-recommends \
        libgdal-dev \
        libsnappy-dev && \
    apt-get clean && rm -rf /var/lib/apt/lists/*
USER jovyan

RUN pip install --no-cache-dir pygeos --no-binary pygeos pygdal==3.0.4.6 pyproj python-snappy fastparquet pyarrow && \
    pip install --no-cache-dir keplergl==0.2.1 pydeck==0.4.1 plotly==4.9.0 matplotlib seaborn tqdm && \
    pip install --no-cache-dir pandapower==2.3.1 scipy==1.5.2 numpy==1.19.1 dask memoization numba && \
    pip install --no-cache-dir email-validator aiofiles python-dotenv livereload==2.6.2 fastapi uvicorn && \
    pip install --no-cache-dir pytest==5.4.3 pytest-watch==4.2.0 pytest-testmon==1.0.2 pytest-html==2.1.1 taskipy==1.2.1 && \
    pip install --no-cache-dir jedi==0.17.2 parso==0.7.1 coverage==5.2.1 line_profiler==3.0.2 memory_profiler==0.57.0 && \
    pip install --no-cache-dir psycopg2-binary geoalchemy2 lxml xlrd requests humanize poetry

ENV JUPYTER_ENABLE_LAB=yes

I'm also not getting any errors regarding conflicting deck.gl-versions in this Docker image.

[ajduberstein]

Thanks for filing the issue, @bergkvist–could we get the output of jupyter labextension list for the broken and working environments? This will help us fix this same problem for other users in the future.

[bergkvist]

@ajduberstein

Broken environment (jupyter labextension list):

JupyterLab v1.2.5
Known labextensions:
   app dir: /usr/share/jupyter/lab
        @deck.gl/jupyter-widget v8.2.7  enabled  OK
        @jupyter-widgets/jupyterlab-manager v1.1.0  enabled  OK
        jupyter-matplotlib v0.5.0  enabled  OK
        jupyterlab-plotly v4.8.0  enabled  OK
        keplergl-jupyter v0.2.1  enabled  OK
        plotlywidget v4.8.0  enabled  OK

Working environment (jupyter labextension list):

JupyterLab v2.2.5
Known labextensions:
   app dir: /opt/conda/share/jupyter/lab
        @bokeh/jupyter_bokeh v2.0.3  enabled  OK
        @deck.gl/jupyter-widget v8.2.7  enabled  OK
        @jupyter-widgets/jupyterlab-manager v2.0.0  enabled  OK
        @krassowski/jupyterlab-lsp v1.1.2  enabled  OK
        dask-labextension v3.0.0  enabled  OK
        jupyter-matplotlib v0.7.3  enabled  OK
        jupyterlab-plotly v4.9.0  enabled  OK
        keplergl-jupyter v0.2.1  enabled  OK
        plotlywidget v4.9.0  enabled  OK

Notice @jupyter-widgets/jupyterlab-manager, which has different major versions (and is a dependency of keplergl-jupyter and @deck.gl/jupyter-widget).