Vega-lite: Vega-Lite file extension

[ellisonbg]

Please:

• [x] Check for duplicate issues
• [x] Describe how to reproduce the bug / the goal of the new feature request
• [x] Provide an example spec and/or screenshots (if applicable)

JupyterLab has the ability to define openers/viewers for different file types. For example, a user can simply click on a CSV file to view it as a table or edit it in a text editor. Mutiple views of the same file are sync'd in real time through a single model. If we standardized on a file extension for vegalite JSON stored on the file system, it would be simple to offer a live renderer for these files that is separate from the notebook. Would be good to have for vega as well. Some options:

• .vega
• .vegalite
• ...

[arvind]

Other options include .vg and .vl (or even .vg.json and .vl.json to explicitly identify them as JSON) for Vega and Vega-Lite respectively.

[ellisonbg]

or something like .geojson such as vgjson and vljson?

On Fri, Jul 15, 2016 at 5:49 PM, Arvind Satyanarayan <
[email protected]> wrote:

Other options include .vg and .vl (or even .vg.json and .vl.json to
explicitly identify them as JSON) for Vega and Vega-Lite respectively.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/vega/vega-lite/issues/1472#issuecomment-233087273,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AABr0MXiF8Y-MIhc2WVQhyz-UmdSWdkQks5qWA6MgaJpZM4JN10m
.

Brian E. Granger
Associate Professor of Physics and Data Science
Cal Poly State University, San Luis Obispo
@ellisonbg on Twitter and GitHub
[email protected] and [email protected]

[jheer]

I think .vgjson and .vljson are good options. While I love the simplicity of .vg. and .vl, I think including "json" makes is easier to understand the nature of the format. Other options (such as .vegajson and particularly .vegalitejson) strike me as too verbose.

[kanitw]

I strongly think .vg.json,.vl.json is the way to go since when you open
these files with text editor you still get the recognition that they are
json-based files for free.
On Sat, Jul 16, 2016 at 3:00 AM Jeffrey Heer [email protected]
wrote:

I think .vgjson and .vljson are good options. While I love the simplicity
of .vg. and .vl, I think including "json" makes is easier to understand
the nature of the format. Other options (such as .vegajson and
particularly .vegalitejson).strike me as too verbose.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/vega/vega-lite/issues/1472#issuecomment-233099557,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAGypTiN7khDIfuIauOdQzrmiZvnqXABks5qWC0WgaJpZM4JN10m
.

[kanitw]

@ellisonbg @jakevdp

I'm thinking about changing all example json in our repo to .vl.json with our pending 1.2 release (since when you open these files with text editor you still get the recognition that they are json-based files for free.)

Before I do that, I wonder if this would work with JupyterLab's extension checker? (If so, I'll do the change and release 1.2!) :)

[jakevdp]

Sounds good to me. @ellisonbg would know whether the compound extension is compatible.

[ellisonbg]

The compound extension works just fine. After thinking more, that is my top choice...

[kanitw]

Fixed in #1524, waiting for reviews

[davidanthoff]

Is there a chance that we could revisit the file extension choice? I'm running into lots of issues with the two-part extension a la .vl.json. In Windows in particular only the part to the right of the last period in a filename is considered the file extension. There seems to be lots and lots of software out there that is based on that assumption. Starting with Explorer and filename associations with programs, to most of the libraries on Windows that deal with filenames. Just some examples: the .Net support for filenames assumes this, Python's pathlib library will give you only .json for .suffix etc.

My particular issues stems from my attempts to integrate things tightly with julia. We have a very powerful uniform file loading and saving package with FileIO.jl. Essentially I'm 99% there to create an API like this:
````julia
spec = load("data.csv") |> vl"""
PUT YOUR SPEC IN HERE AS A STRING
"""

If you execute the above line on the REPL, it will show the graph visually

Save the spec

spec |> save("myfigure.vegalite")

Now load some data from a CSV file, filter it a bit, pipe it into a vega-lite

spec loaded from disc and save the result as a pdf

load("data.csv") |> @filter(_.age>20) |> load("myfigure.vegalite") |> save("figure.pdf")
````
You probably get the gist (and I hope to also integrate this with the existing VegaLite.jl package that provides a high level julia API for creating specs). I think this could be a really, really powerful user experience (especially when I also integrate it with your fantastic voyager stuff).

BUT, I'm stuck with the file extension :) Julia follows the platform assumption that only the stuff to the right of the last . is the file extension, and so that whole integration into the generic load and save functions won't work with a file extension that deviates from that... I kind of assume that this issue will show up again and again in other contexts, wouldn't it be safer to just follow the standard convention here?

I realize that one appealing aspect of the current solution was that editors would automatically use the proper grammar. But I think that could be relatively easily achieved for some other file extension as well. At least for VS Code (which I know well, I maintain the julia extension) it should be really straightforward to just declare that e.g. .vegalite should use the json grammar.

[ellisonbg]

Yeah, we ran into this as well in JupyterLab, but it was easy to add
handling for that in that context. In JupyterLab, we also handle .vl
files as well, so maybe that is your answer?

On Fri, Dec 22, 2017 at 2:09 PM, David Anthoff [e-mail]
wrote:

Is there a chance that we could revisit the file extension choice? I'm
running into lots of issues with the two-part extension a la .vl.json. In
Windows in particular only the part to the right of the last period in a
filename is considered the file extension. There seems to be lots and lots
of software out there that is based on that assumption. Starting with
Explorer and filename associations with programs, to most of the libraries
on Windows that deal with filenames. Just some examples: the .Net support
for filenames assumes this, Python's pathlib library will give you only
.json for .suffix etc.

My particular issues stems from my attempts to integrate things tightly
with julia. We have a very powerful uniform file loading and saving package
with FileIO.jl. Essentially I'm 99% there to create an API like this:

spec = load("data.csv") |> vl"""PUT YOUR SPEC IN HERE AS A STRING"""# If you execute the above line on the REPL, it will show the graph visually

Save the spec

spec |> save("myfigure.vegalite")

Now load some data from a CSV file, filter it a bit, pipe it into a vega-lite# spec loaded from disc and save the result as a pdfload("data.csv") |> @filter(_.age>20) |> load("myfigure.vegalite") |> save("figure.pdf")

You probably get the gist (and I hope to also integrate this with the
existing VegaLite.jl package that provides a high level julia API for
creating specs). I think this could be a really, really powerful user
experience (especially when I also integrate it with your fantastic voyager
stuff).

BUT, I'm stuck with the file extension :) Julia follows the platform
assumption that only the stuff to the right of the last . is the file
extension, and so that whole integration into the generic load and save
functions won't work with a file extension that deviates from that... I
kind of assume that this issue will show up again and again in other
contexts, wouldn't it be safer to just follow the standard convention here?

I realize that one appealing aspect of the current solution was that
editors would automatically use the proper grammar. But I think that could
be relatively easily achieved for some other file extension as well. At
least for VS Code (which I know well, I maintain the julia extension) it
should be really straightforward to just declare that e.g. .vegalite
should use the json grammar.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/vega/vega-lite/issues/1472#issuecomment-353682418,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AABr0AORk7zv321_DtwidaifGFA3kwiTks5tDCiZgaJpZM4JN10m
.

--
Brian E. Granger
Associate Professor of Physics and Data Science
Cal Poly State University, San Luis Obispo
@ellisonbg on Twitter and GitHub
[email protected] and [email protected]

[domoritz]

I like .vl.json because Vega-Lite is just JSON. If you just use .json and set a $schema property, VSCode will give you auto complete.

[davidanthoff]

For VS Code one can get the same result by doing this config for say the .vegalite extension, I believe?

I also looked into how complicated it would be for the vega VS Code extension to provide this kind of stuff, and it seems more tricky than I thought. Essentially one would have to copy all the stuff from the JSON extension over, which seems a lot of work.

So I think at the end of the day we just can't get full integration with the .vl.json extension into systems that expect something else, and the kind of loading/saving integration I want to pull off for julia falls into that category, so I'll just have to use something else for that. Same true if we want things like file extension integration say in explorer in Windows, AFAIK.

@ellisonbg Yes, I could use the .vl extension. Just wondering, might it make sense to use .vegalite instead? .vl seems awfully short, and there might be conflicts with other file formats?

[domoritz]

You're welcome to use your own extensions. We have chosen .vl.json as it shows that the files are just JSON but with some special meaning. I'm sure we could add support for .vegalite to jupyter lab.

[ellisonbg]

I don't think it would hurt to have jupyterlab also accept .vega and
.vegalite. David, want to submit a PR over there:

Here are the relevant lines:

https://github.com/jupyterlab/jupyterlab/blob/master/packages/vega2-extension/src/index.ts#L157
https://github.com/jupyterlab/jupyterlab/blob/master/packages/vega2-extension/src/index.ts#L165

On Wed, Jan 10, 2018 at 5:04 AM, Dominik Moritz [e-mail]
wrote:

You're welcome to use your own extensions. We have chosen .vl.json as it
shows that the files are just JSON but with some special meaning. I'm sure
we could add support for .vegalite to jupyter lab.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/vega/vega-lite/issues/1472#issuecomment-356596585,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AABr0PmbeXztiuFTBM5QsHh6NaukmLBSks5tJLVtgaJpZM4JN10m
.

--
Brian E. Granger
Associate Professor of Physics and Data Science
Cal Poly State University, San Luis Obispo
@ellisonbg on Twitter and GitHub
[email protected] and [email protected]

[davidanthoff]

David, want to submit a PR over there

Done.

I also saw some MIME types there for vega and vegalite. Does that mean if we send some content from IJulia with those MIME types they will use the vega jupyterlab extension? That would be nice. That would only work with jupyterlab, not the currently shipping notebook, right?

Right now the IJulia integration I have is that it sends some HTML as MIME type text/html. In there I use requirejs to load vega/vega-lite/vega-embed. It works, but is a bit cumbersome.

[ellisonbg]

Yes, both JupyterLab and nteract support these MIME types and will "just
work". We would like to get the classic notebook working with it all, but
it has turned out to be really difficult to get it working with npm
packages without breaking everything. Not sure of the status of that, but
in the meantime the newer frontends offer the new MIME based approach.

Cheers,

Brian

On Wed, Jan 10, 2018 at 12:29 PM, David Anthoff [e-mail]
wrote:

David, want to submit a PR over there

Done.

I also saw some MIME types there for vega and vegalite. Does that mean if
we send some content from IJulia with those MIME types they will use the
vega jupyterlab extension? That would be nice. That would only work with
jupyterlab, not the currently shipping notebook, right?

Right now the IJulia integration I have is that it sends some HTML as MIME
type text/html. In there I use requirejs to load
vega/vega-lite/vega-embed. It works, but is a bit cumbersome.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/vega/vega-lite/issues/1472#issuecomment-356726747,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AABr0AnNYsa3TyZ6tMPcf9eONdfe0Kolks5tJR2wgaJpZM4JN10m
.

--
Brian E. Granger
Associate Professor of Physics and Data Science
Cal Poly State University, San Luis Obispo
@ellisonbg on Twitter and GitHub
[email protected] and [email protected]

[domoritz]

I added the extensions to the Vega 3 version as well: https://github.com/jupyterlab/jupyter-renderers/pull/83.

[ellisonbg]

Thanks Dominik! We will get to reviewing that now that the Beta 1 is out.

On Wed, Jan 10, 2018 at 11:33 PM, Dominik Moritz [e-mail]
wrote:

I added the extensions to the Vega 3 version as well:
jupyterlab/jupyter-renderers#83
https://github.com/jupyterlab/jupyter-renderers/pull/83.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/vega/vega-lite/issues/1472#issuecomment-356850014,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AABr0N_pkbcpccesLZsjIEV-N7ntHckDks5tJbk2gaJpZM4JN10m
.

--
Brian E. Granger
Associate Professor of Physics and Data Science
Cal Poly State University, San Luis Obispo
@ellisonbg on Twitter and GitHub
[email protected] and [email protected]

[domoritz]

@ellisonbg Fantastic. Should we move the Vega 3/ Vega-Lite 2 extension over to Jupyter Lab or should it stay in jupyter renderers?

[davidanthoff]

Thanks guys!

I'm getting pretty close to having a fully functional vega-lite story for julia. Voyager already works. I'm really way excited about bringing all the power of the stack you created over to julia, I think it will be a major, major help for the data science stack.

[domoritz]

@davidanthoff That's super exciting. Keep us posted on your progress!