altair.data_transformers.enable('json') leads to invalid data URLs in JupyterLab when switching from Altair 3.3.0 to 4.0.0

[sdelarquier]

To reproduce, use the latest version (as of this writing) of altair and jupyterlab:

altair==4.0.0
ipykernel==5.1.3
ipython==7.10.1
jupyter==1.0.0
jupyter-client==5.3.4
jupyter-console==6.0.0
jupyter-core==4.6.1
jupyterlab==1.2.4
jupyterlab-server==1.0.6
vega-datasets==0.7.0

In Jupyter lab open a new notebook and try:

from vega_datasets import data
import altair as alt

alt.data_transformers.enable('json')
alt.renderers.enable('default')

source = data.cars()

alt.Chart(source).mark_circle(size=60).encode(
    x='Horsepower',
    y='Miles_per_Gallon',
    color='Origin',
    tooltip=['Name', 'Origin', 'Horsepower', 'Miles_per_Gallon']
).interactive()

This leads to an empty chart with the following error in the JavaScript console:

vega@5?noext:1 GET http://localhost:8888/altair-data-4d45d69bbd706eda330e96e79ad4bf46.json 404 (Not Found)
vega@5?noext:1 WARN Infinite extent for field "Horsepower": [Infinity, -Infinity]
vega@5?noext:1 WARN Infinite extent for field "Miles_per_Gallon": [Infinity, -Infinity]

And in the Jupyter logs the error reads:

[W 16:56:10.144 LabApp] 404 GET /altair-data-4d45d69bbd706eda330e96e79ad4bf46.json (::1) 1.60ms referer=http://localhost:8888/lab

If I revert to altair==3.3.0 this error disappears and the data gets correctly loaded from disk and rendered. In that case the data URL (read from from the browser dev tools) becomes http://localhost:8888/files/altair-data-4d45d69bbd706eda330e96e79ad4bf46.json?_xsrf=<blah> (where I replaced the long string at the end with <blah).

[jakevdp]

Thanks for the report, and sorry for the issue. The problem is that jupyterlab's frontend (as opposed to jupyter notebook frontend) references files in the local directory via the files subpath. There are two things you can do to fix this:

1) upgrade your JupyterLab vega extension to the yet-to-be-released version that supports vega-lite 4, and use alt.renderers.enable('mimebundle') – with this setup the JSON data transformer should work properly

2) Use the current setup (that has the distinct advantage of being currently available to install) and hack the json data transformer to work with jupyterlab using something like this:

import altair as alt
from altair.utils.data import to_json
from vega_datasets import data
import os

def to_json_for_lab(data):
    output = to_json(data)
    output['url'] = os.path.join("files", output["url"])
    return output

alt.data_transformers.register('json', to_json_for_lab)
alt.data_transformers.enable('json')

source = data.cars()

alt.Chart(source).mark_point().encode(
    x='Miles_per_Gallon:Q',
    y='Horsepower:Q'
)

1. Use another method, such as the Altair Data Server that doesn't rely on loading data from disk.

I'm going to leave this open as a bug as we figure out how to best support users of the JSON transformer across different frontends with different conventions and constraints.

[jakevdp]

A straightforward fix on the Altair side would be to add a urlpath to the json transformer so that you could use

alt.data_transformers.enable('json', urlpath='files')

[sdelarquier]

Thank you @jakevdp , I'll use the custom transformer for now, that's a very convenient solution.

And while I'm here, thank you for Altair, it is has been a fantastic library for exploratory analysis and looks great in presentations too!

[jakevdp]

With #1872, in the next version of Altair (4.1 or newer) you'll be able to run

alt.data_transformers.enable('json', urlpath='files')

to make the transformer work correctly in JupyterLab with the default renderer. Without a way for the backend to detect what frontend you're using (an idea the Jupyter team has long resisted) I don't think there's any way to make this more automatic.

[sdelarquier]

Nice, thank you @jakevdp ! The only caveat is that you would probably need to specify the full path if you're not working in the JupyterLab root directory (something like urlpath='files/subdir'), but that's a minor cost for the convenience of this transformer!

[pabloinsente]

I ran into this exact same problem with Altair 4.0 and jupyterlab==1.2.4

I tried :

alt.data_transformers.enable('csv')
alt.data_transformers.enable('json')

When I try to render a chart I get (in the shell):

[W 22:34:37.908 LabApp] 404 GET /altair-data-cf2bd851a7c068a3092c55520778b4e8.csv (127.0.0.1) 0.78ms referer=None

...and an empty chart. I revert to altair==3.3.0 and the problem went away.

[jakevdp]

The issue is that localhost filepaths are unpredictable between different flavors of Jupyter notebook.

I'd suggest avoiding the JSON/CSV transformer in favor of the data_server transformer, which should work regardless of what Jupyter frontend you're on.

[ellisonbg]

Just saw this one, I think it is a flat out bug in recent versions of JupyterLab, I will open an issue there for us to track it.

[ellisonbg]

The way files are handled in situations like this is the same in lab/notebook as the underlying server is the same in this matter.