Altair: Name and location of json files created by `alt.data_transformers.enable('json')`

[FlorianGD]

Hello,
First, I love this package that I discovered recently and was able to get up to speed really quickly with the tutorials, so thank you!

I use alt.data_transformers.enable('json') to manage fairly large datasets, but when I make several graphs in a notebook, I get a bunch of altair-data-11217bb8-d26d-481e-99d3-8e8701d4e6cf.json-like files, and it is not easy to know which one is which. Is there a way to:

1. Select a location (e.g. a figure file containing those files) ?
2. Choose a name when rendering the file (maybe with the name of the Chart if provided) ?

Thank you

[jakevdp]

Hi – no, there currently is not any way to control the names of these files.

One thing I would like to do is generate filenames based on a hash of the dataset, so that repeated plots with the same dataset would not lead to multiple copies of the same file, but I've not had time to get to that yet.

[jakevdp]

One way around this is to manually save your dataframe to file, and pass that file path to the chart as a URL.

[ellisonbg]

Yeah, it would be nice to improve the way this is handled along the lines
you mention.

On Thu, May 17, 2018 at 9:01 AM, Jake Vanderplas [e-mail]
wrote:

One way around this is to manually save your dataframe to file, and pass
that file path to the chart as a URL.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/altair-viz/altair/issues/857#issuecomment-389919067,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AABr0Nj1eBesmmFsb06mit2QVMrg_9TCks5tzZ7ngaJpZM4UCt0B
.

--
Brian E. Granger
Associate Professor of Physics and Data Science
Cal Poly State University, San Luis Obispo
@ellisonbg on Twitter and GitHub
[email protected] and [email protected]

[FlorianGD]

OK, thanks for the info. And nice if you can improve that

[jakevdp]

Fixed in #862

[armsp]

@jakevdp I read the file that you merged and perhaps the correct way to handle data with more than 5000 rows is to use saved = alt.to_json(data) and then pass the url like alt.Chart(saved['url']) in a chart?

However, I have been just using the following for a long time cause that I what I had seen in the docs long ago -
alt.data_transformers.enable('json')

So I have a lot of json files from various notebooks.

Is there a way to programmatically find which file belongs to which chart's data i.e get the file name, say by executing something in a cell after adding that cell just below the chart?

[jakevdp]

No, there's no public utility for that, but the filenames are generated deterministically based on the contents of the data, using this code: https://github.com/altair-viz/altair/blob/bf5362cb50c88318aa02d497b8ee1b5e19c7d89f/altair/utils/data.py#L114-L116

[armsp]

@jakevdp I tried the above and it really helped.

Do you think it would be a good idea to have some method that helps with this instead of calling private functions? Will a PR help or should we keep it this way?

I was thinking that whenever anyone calls alt.data_transformer.enable('json') or alt.data_transformer.enable('csv') we can run _data_to_json_string(data) or _data_to_csv_string(data) correspondingly and immediately output the filename or output an object where we can extract the filename from. What do you think?

[jakevdp]

I'm not sure that would work, because the dataset is not known at the time that the transformer is enabled.

[jakevdp]

You can always find the filename in the serialized chart:

import altair as alt
import pandas as pd
df = pd.DataFrame({'x': range(5)})

alt.data_transformers.enable('json')
chart = alt.Chart(df).mark_bar().encode(x='x:Q')

print(chart.to_dict()['data']['url'])

Better would probably be to do the transformation manually if you want to manipulate the output file directly:

data = alt.to_json(df)
print(data['url'])
chart = alt.Chart(data).mark_bar().encode(x='x:Q')

[armsp]

Oh yes, I should have thought about it ... we generally use it before. I think the solutions we have right now is good enough. Thanks a lot @jakevdp !