Altair: Incorporate Vega-Tooltip

[jakevdp]

I think it would be nice to add support for Vega-Tooltip in Altair. This would have to be done at the ipyvega level.

[kanitw]

cc: @zeningqu

[jowens]

👍

[doug]

Can tooltips be expressed with the vega-lite 2.0 selections?

[kanitw]

Can tooltips be expressed with the vega-lite 2.0 selections?

Basic tooltip that leverage title attribute will be expressible as a visual encoding channel in VL2. However, if you want custom tooltip design, then you need to write more custom code or use https://github.com/vega/vega-tooltip (still under migration to VL2, not ready for VL2 yet).

[doug]

Awesome, what is the syntax or the spec for basic tooltip in VL2?

[kanitw]

@doug It will likely be just

{
  ...,
  mark: ...,
  encoding: {
     ...,
     tooltip: {...}
  }
}

But we haven't implemented that yet so things might change a bit. You can follow this issue here: https://github.com/vega/vega-lite/issues/1921.

[ytrofimov]

@jakevdp when will Altair v2.0 see the light? can't wait for tooltips, label layering and boxplots 👍
on a separate note: how can one donate to support the project?

[ajasja]

@jakevdp Hi! Now that vega/vega-lite#1921 has been closed, is it possible to use the tooltip channel in altair as well? Does this work like hover tooltips in Bokeh?

[jakevdp]

This is a bit complicated, because vega-tooltip is a separate package from vega-lite or vega, and requires some extra metadata to control both the look of tooltips and whether tooltips are displayed by default. Currently, none of the supported frontends use vega-tooltip in their renderings, and so tooltips are not possible. The Altair itself has no control over what javascript code the frontends use to render the plots.

I think this is a regrettable situation, and that it would be much cleaner for the whole Vega ecosystem if vega-tooltip were rolled-into vega and handled in a more seamless way, but when I've floated that idea with the Vega team they don't seem to agree.

So, for the foreseeable future, tooltips will not be possible with Altair, although you can make tooltip-like behavior using selections; see e.g. https://altair-viz.github.io/gallery/multiline_tooltip.html

[ajasja]

Thanks, I did see that example; I would like to click (or hover) on a point and then show the point's other values (about six other fields). Could that also be done using selectors? I guess it could be linked to a table that would show the other fields?

PS: Out of curiosity, what does this do:
https://altair-viz.github.io/user_guide/API.html?highlight=tooltip#altair.Tooltip

[jowens]

@ajasja I ended up finding a way to use tooltips (https://owensgroup.github.io/gpustats/) by manually patching the JSON:

    with open(os.path.join(outputdir, title + '.html'), 'w') as f:
        spec = chart.to_dict()
        spec['encoding']['tooltip'] = {"field": "Model", "type": "nominal"}
        f.write(template.format(spec=json.dumps(spec), title=title))

[jakevdp]

You don't need to patch the json to set the tooltip encoding in Vega-Lite; you can use it just like any other encoding: chart.encode(tooltip='Model:N').

That doesn't change the main issue that the vega-tooltip package is not loaded by the renderers, though, which is what is needed to get a listing of all the values, like @ajasja wants.

[kanitw]

@jakevdp Hi! Now that vega/vega-lite#1921 has been closed, is it possible to use the tooltip channel in altair as well? Does this work like hover tooltips in Bokeh?

Tooltip channel is a part of core Vega/Vega-Lite so this should work out of the box?

I think this is a regrettable situation, and that it would be much cleaner for the whole Vega ecosystem if vega-tooltip were rolled-into vega and handled in a more seamless way, but when I've floated that idea with the Vega team they don't seem to agree.

I would like clarify that the vega-tooltip library is more powerful than the built-in tooltip channel in Vega/Vega-Lite. That's partly because vega-tooltip uses HTML to render tooltips, which means it can do much more than standard tooltips enabled by SVG/Canvas (via the tooltip encoding channel). However, it also means that we can't just integrate it into Vega without re-writing entire Vega as Vega only support rendering directly to SVG or Canvas (not HTML).

We're open to integrating it into vega-embed though as vega-embed is aimed more for rendering the Vega on SVG / Canvas in a HTML page.

Currently, none of the supported frontends use vega-tooltip in their renderings, and so tooltips are not possible. The Altair itself has no control over what javascript code the frontends use to render the plots.

I think updating the frontends (like ipyvega?) to include vega-tooltip should be easier than re-architecting Vega to support HTML output.

cc: @jheer @domoritz (They might have slightly different opinions as they know more about vega-embed and the renderers that you use here in Python.)

[jakevdp]

@kanitw – yes, adding vega-tooltip to the frontends would be fairly easy. But once that's done, there's no way for Altair to communicate to those frontends how it wants the tooltip to look, or whether they should be on or off, because the tooltip options are entirely separate from the plot specification (and by design, all that Altair does is generate a valid plot specification). So we'd have to come up with some other configuration mechanism that's separate from the plot itself... maybe a magic function or something... but making that work consistently with every frontend would be a challenge, because the frontend code is entirely separate from Altair itself.

[kanitw]

Oh I see! One solution we could resolve this is to use usermeta.vegaTooltip (https://github.com/vega/vega/issues/1061) or add config.vegaTooltip that will normally be ignored by the main Vega/Vega-Lite but will be read by the Vega Tooltip.

(We can discuss if we want to solve this at the vega-tooltip level or vega-embed level, but it should be doable!)

[ajasja]

@kanitw @jakevdp Great, so the tooltip channel should work out of the box?

I'm probably missing something (most probably step 3 in the vega-tooltip tutorial), but the front-end renderers would only have to include two lines of code

<script src="https://cdn.jsdelivr.net/npm/vega-tooltip@[VERSION]"></script>
<link rel="stylesheet" type="text/css" href="https://cdn.jsdelivr.net/npm/vega-tooltip@[VERSION]/build/vega-tooltip.min.css">

and Altair could have a .configure_tooltips block or similar.

Anyway, the point I'm trying to get across is that tooltips are extremely useful (attached is an real-life example form my work, where the tooltips are invaluable)

[jakevdp]

and Altair could have a .configure_tooltips block or similar.

No, it couldn't. By design, the only thing Altair controls is the spec, which is controlled by the schema, and the vega-lite schema does not include the tooltips configuration (it has to be passed as a separate opt object to the vega-tooltip javascript code).

If we want to allow configuration of tooltips from Altair, that would require merging the vega and vega-tooltip javascript libraries, and putting the tooltip configuration into the actual plot specification – then Altair could control the tooltips.

Anyway, the point I'm trying to get across is that tooltips are extremely useful

I 100% agree that full tooltip configuration in Altair would be invaluable, which is why it's quite unfortunate that it's basically impossible to include at the moment, unless the vega & vega-lite libraries change how they do things.

[jakevdp]

We could include that vega-tooltip javascript call in the frontends, and then tooltips would be turned on for every plot. But there would be no way for Altair to turn them off or configure them in any way, because all Altair does is produce a vega-lite specification, and there's no mechanism by which we can send any extra information to the frontend that's not part of that specification.

[jakevdp]

Another solution would be for Altair to define its own super-schema that includes both the vega-lite schema and the tooltip option schema, and return that instead, so it would be able to send the tooltip options to the renderer.

I don't like that idea, because it seems quite hard to maintain, and would be confusing for users who are expecting actual vega/vega-lite schemas to be produced by Altair. My feeling is that any modification of the schema definition should happen within Vega/Vega-Lite itself, not within Altair.

[domoritz]

I think the easiest and cleanest way is to add Vega-Tooltip to Vega-Embed. Here is the issue: https://github.com/vega/vega-embed/issues/14. Then we can enable tooltips in ipyvega (https://github.com/vega/ipyvega/issues/78) and https://github.com/jupyterlab/jupyterlab/tree/master/packages/vega3-extension. The only remaining question is how it would be enabled. We could just add a little checkbox that enables/disabled tooltips similar to the toggle in https://vega.github.io/editor/. But then the Python code has no control so I prefer a more explicit way. To move forward, we need some specification of how Altair or any other Python code will pass metadata to the renderer (ipyvega or vega3 in JupyterLab).

Regardless, you can already use the tooltip channel (https://vega.github.io/vega-lite/docs/tooltip.html#using-tooltip-channel).

cc @sirahd

[jakevdp]

To move forward, we need some specification of how Altair or any other Python code will pass metadata to the renderer

This is the key: the options are

1) define a new super-schema that only Altair uses, and then adjust all the renderers so that they can either handle normal vega-lite/vega schemas or altair-style super-schemas.

2) adjust the vega/vega-lite schema itself to allow inclusion of tooltip configuration.

I think 1 would be quite awkward because of the proliferation of schemas. It would be similar to the embed-spec from the old version of vega-embed. But if 2 is impossible, then I suppose 1 is what we'll need to do.

[domoritz]

Isn't there some metadata that can be passed to a mimetype renderer? I have strong feelings against an extended specification language just to support tooltips.

[jakevdp]

As far as I know, there's no way to pass anything ti the mimetype renderer beyond the schema itself.

In my mind, tooltips are part of the visualization, so it would make sense to include information about them in the visualization schema. It's confused me for a long time why the tooltip config would be separate.

[domoritz]

What about all the other embed options like renderer, baseURL, or editorUrl? We used to have an embed spec and it was a source of confusion and I want to avoid repeating mistakes.

[jakevdp]

We used to have an embed spec and it was a source of confusion and I want to avoid repeating mistakes.

Exactly: I think a super-schema is a horrible idea, which is why I've been advocating from the beginning that tooltip configuration be wrapped into vega/vega-lite. It's part of the plot, so it should be part of the schema.

[domoritz]

Do we agree that we want to support Vega-Tooltip in Vega-Embed and that we need to find a way to make at least the main options available to the Python world?

The two options are

1) Extend the Vega and Vega-Lite schemas to support these options. This entails a new set of JSON schemas with properties that are going to be ignored by the JS versions of Vega and Vega-Lite but the Python modules ipyvega and vega3 would read it and pass it to Vega-Embed.

2) Pass the metadata separate from the spec (as two arguments to a function). @domoritz prefers this but there is no way to pass metadata to mime renderers in Jupyterlab (@ellisonbg). In ipyvega, this wouldn't be an issue at all.

Unless I am missing something, we need to go with 1. Please correct me @kanitw @jheer!

[ellisonbg]

It is possible to pass metadata along with the MIME renderers in
JupyterLab. That is the second dict returned by the MIME renderers in
Altair v2.

On Mon, Mar 19, 2018 at 4:59 PM, Dominik Moritz [e-mail]
wrote:

Do we agree that we want to support Vega-Tooltip in Vega-Embed and that we
need to find a way to make at least the main options available to the
Python world?

The two options are

1.

Extend the Vega and Vega-Lite schemas to support these options. This
entails a new set of JSON schemas with properties that are going to be
ignored by the JS versions of Vega and Vega-Lite but the Python modules
ipyvega and vega3 would read it and pass it to Vega-Embed.
2.

Pass the metadata separate from the spec (as two arguments to a
function). @domoritz https://github.com/domoritz prefers this but
there is no way to pass metadata to mime renderers in Jupyterlab (
@ellisonbg https://github.com/ellisonbg). In ipyvega, this wouldn't
be an issue at all.

Unless I am missing something, we need to go with 1. Please correct me
@kanitw https://github.com/kanitw @jheer https://github.com/jheer!

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/altair-viz/altair/issues/240#issuecomment-374426065,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AABr0Jor5dN6dqU4NC5RnENowHXd6iGvks5tgEZygaJpZM4KcZGV
.

--
Brian E. Granger
Associate Professor of Physics and Data Science
Cal Poly State University, San Luis Obispo
@ellisonbg on Twitter and GitHub
[email protected] and [email protected]

[jakevdp]

This entails a new set of JSON schemas with properties that are going to be ignored by the JS versions of Vega and Vega-Lite but the Python modules ipyvega and vega3 would read it and pass it to Vega-Embed.

I'm fine with embed options (such as editor URL, etc.) being separate. In my mind, those are not part of the visualization itself. Tooltips, on the other hand, are definitely part of the visualization (they are used to convey information about the data to the user), and I think should thus be part of the schema. Of course, this would require rolling vega-tooltip into vega itself... but you seem quite opposed to that for reasons that are unclear to me. Is there some technical hurdle I'm not understanding that prevents combining the two?

[ellisonbg]

I agree with what @jakevdp is saying here. Having some support for tooltips
in the main schema, but some outside, seems like a leak in the abstractions.

On Mon, Mar 19, 2018 at 5:23 PM, Jake Vanderplas [e-mail]
wrote:

This entails a new set of JSON schemas with properties that are going to
be ignored by the JS versions of Vega and Vega-Lite but the Python modules
ipyvega and vega3 would read it and pass it to Vega-Embed.

I'm fine with embed options (such as editor URL, etc.) being separate. In
my mind, those are not part of the visualization. Tooltips, on the other
hand, are part of the visualization, and I think should thus be part of the
schema. Of course, this would require rolling vega-tooltip into vega
itself... but you seem quite opposed to that for reasons that are unclear
to me. Is there some technical hurdle I'm not understanding that prevents
combining the two?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/altair-viz/altair/issues/240#issuecomment-374430365,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AABr0AXsduMaGyZMoRX0jFa7hGqBVgA4ks5tgEvvgaJpZM4KcZGV
.

--
Brian E. Granger
Associate Professor of Physics and Data Science
Cal Poly State University, San Luis Obispo
@ellisonbg on Twitter and GitHub
[email protected] and [email protected]

[domoritz]

Of course, this would require rolling vega-tooltip into vega itself... but you seem quite opposed to that for reasons that are unclear to me. Is there some technical hurdle I'm not understanding that prevents combining the two?

Ahh, I think I understand where you're coming from now. You want tooltips in the spec now for technical reasons but because you think it semantically belongs into the visualization spec. It makes sense but there is one design principle that we've been following that prevents us from moving Vega-Tooltip into Vega. Vega (and Vega-Lite) are designed to be declarative formats that should be agnostic to the specific renderer or platform. Vega-Tooltip only works in browsers but for example, won't work when we generate an SVG file. For this reason, I would think @jheer, @kanitw, and @arvind are opposed to merging Vega-Tooltip into Vega as well.

Having some support for tooltips
in the main schema, but some outside, seems like a leak in the abstractions.

Are you referring to the tooltip channel?

I am advocating for keeping all embed options outside of the main spec and move it into metadata. Tooltip customization would be part of the embed configuration and passed through to Vega-Tooltip.

[ellisonbg]

Yes I was referring to the tooltip channel when talking about the
abstraction leak. Thanks for the additional background though.

On Mon, Mar 19, 2018 at 5:35 PM, Dominik Moritz [e-mail]
wrote:

Of course, this would require rolling vega-tooltip into vega itself... but
you seem quite opposed to that for reasons that are unclear to me. Is there
some technical hurdle I'm not understanding that prevents combining the two?

Ahh, I think I understand where you're coming from now. You want tooltips
in the spec now for technical reasons but because you think it semantically
belongs into the visualization spec. It makes sense but there is one design
principle that we've been following that prevents us from moving
Vega-Tooltip into Vega. Vega (and Vega-Lite) are designed to be declarative
formats that should be agnostic to the specific renderer or platform.
Vega-Tooltip only works in browsers but for example, won't work when we
generate an SVG file. For this reason, I would think @jheer
https://github.com/jheer, @kanitw https://github.com/kanitw, and
@arvind https://github.com/arvind are opposed to merging Vega-Tooltip
into Vega as well.

Having some support for tooltips
in the main schema, but some outside, seems like a leak in the
abstractions.

Are you referring to the tooltip channel?

I am advocating for keeping all embed options outside of the main spec and
move it into metadata. Tooltip customization would be part of the embed
configuration and passed through to Vega-Tooltip.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/altair-viz/altair/issues/240#issuecomment-374432432,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AABr0LnYodvx1qaXJkPsLK4JZIslA2QBks5tgE7EgaJpZM4KcZGV
.

--
Brian E. Granger
Associate Professor of Physics and Data Science
Cal Poly State University, San Luis Obispo
@ellisonbg on Twitter and GitHub
[email protected] and [email protected]

[jakevdp]

Vega-Tooltip only works in browsers but for example, won't work when we generate an SVG file. For this reason, I would think @jheer, @kanitw, and @arvind are opposed to merging Vega-Tooltip into Vega as well.

OK, but why then did you make an exception to this rule for selections, when selections also work only in the browser and don't work in a generated SVG file? I'd say tooltips are quite similar to selections, no?

[jakevdp]

In my mind, encodings, marks, selections, and tooltips are all part of the visual encoding: they are ways by which chart communicates the contents of the dataset to the user. To make a distinction between tooltips on one hand, and the rest of the visual features on the other, seems quite arbitrary.

[jheer]

A few thoughts on this issue:

1. I agree 100% that we should find a way to provide high-quality tooltips in Altair. I also agree that tooltips make sense as part of a visual specification. Vega includes a number of browser-specific features already: signal binding to DOM elements, input event handling (including from a window object or other queryable browser components!), and, yes, tooltips.
2. Vega's basic tooltip support uses browser built-ins. The tooltip encoding channel takes a string (or string-coercible) value and uses it to populate a browser "title" attribute. However, Vega's tooltip handling is entirely extensible. By overriding the appropriate method in Handler.js arbitrary tooltips can be added. For example, any desired format could be used for the tooltip channel so long as the tooltip handler can appropriately process it. Vega signal expressions could be used to marshal objects that contain desired fields and formatted values. With sufficient effort, one could even implement a handler that shows sub-visualizations within a tooltip, not just formatted text.
3. As a result of point 2, at this point I don't think it makes sense to make major changes to core Vega itself, as any number of tooltip extensions could be added. However, it might make a lot of sense to include an additional JavaScript import that provides the default tooltip support chosen for Altair. For example, an import could simply modify the prototype object for the Handler class, ensuring all Vega instances use the desired tooltip machinery.
4. A remaining question concerns possible extended Vega-Lite support for tooltip encoding, which could be considered depending on what the supported tooltip formats are (e.g., for pass-through to a custom handler) and what level of customization should be supported.

[kanitw]

A remaining question concerns possible extended Vega-Lite support for tooltip encoding, which could be considered depending on what the supported tooltip formats are (e.g., for pass-through to a custom handler) and what level of customization should be supported.

It's worth noting that vega-tooltip is useful in the sense that it provides tooltips that show all fields of the hovered data point by default, so it's not just showing the encoded tooltip string. Thus, I'm not sure how would "extended Vega-Lite support for tooltip encoding" work for this case.

Moreover, I don't know if we can really extend tooltip encoding in a way that's sensible for _any arbitrary_ tooltip extension, especially there is no other at the moment. Even if we have another, the extended encoding that we add here may become a LCD-ish solution as something that vega-tooltip support may not be supported by other extension.

I am advocating for keeping all embed options outside of the main spec and move it into metadata. Tooltip customization would be part of the embed configuration and passed through to Vega-Tooltip.

I still prefer using usermeta field (https://github.com/vega/vega/issues/1061) than embed configuration. This means:

1) Vega and Vega-Lite can just ignore this field as it's just metadata to them, but Vega-Tooltip can still pick up the properties.
2) The config options for vega-tooltip can still be a part of saved .vl.json or .vg.json.
3) We don't need to extend Vega / Vega-Lite schema as usermeta is designed to accept arbitrary object.
4) Vega Tooltip can publish its own vega-tooltip-schema that Altair can read and generate API to output things to usermeta.vegaTooltip.
5) If there are other tooltip extension in the future, they can still use usermeta.xxx to customize their configuration without conflicting with what we do here.

[jheer]

It's worth noting that vega-tooltip is useful in the sense that it provides tooltips that show all fields of the hovered data point by default, so it's not just showing the encoded tooltip string. Thus, I'm not sure how would "extended Vega-Lite support for tooltip encoding" work for this case.

A few possible cases might include:

• What if one doesn't want tooltips shown for a given mark?
• What if one wants only a subset of fields shown in the tooltip?
• What if one wants to include some form of custom formatted string in the tooltip?

I could imagine a way of enabling tooltips with the default setting being to show all data points. Beyond that, however, one might want to use a tooltip encoding channel to convey customizations - potentially at the per-mark level, not just global config.

Perhaps the conversation would be smoother if we first framed it around the set of basic tooltip features we think Altair should support, and at what granularity? This seems to me a good starting point, upstream of questions such as which package and/or people should be responsible for these features.

[kanitw]

I like your point about per-mark level customization. Vega-tooltip should definitely take the tooltip channel into account (we have an issue here https://github.com/vega/vega-tooltip/issues/97, but we haven't started discussing there at all).

But there might be still some customization that still needs to be done at the global level.

---

What if one doesn't want tooltips shown for a given mark?

I think setting value to null or empty string should work -- once we do https://github.com/vega/vega-tooltip/issues/97.

What if one wants to include some form of custom formatted string in the tooltip?

For Vega, I think this is already doable using a Value signal. Vega-Lite doesn't have proper support for this yet though. I'm creating an issue here https://github.com/vega/vega-lite/issues/3560.

What if one wants only a subset of fields shown in the tooltip?

This one is the trickiest as we don't have proper support for in VG/VL for this yet.

[domoritz]

I like the point about being agnostic to the specific tooltip implementation.

[ajasja]

Perhaps the conversation would be smoother if we first framed it around the set of basic tooltip features we think Altair should support, and at what granularity?

I would vote for:

• Displaying a subset of fields
• Support for string formatting and/or HTML passthrough (Bokeh tooltips have solved this nicely; I'm just not sure hw this can be tranlated into a declerative form. )

[jakevdp]

Dome in vega-embed

[ajasja]

@jakevdp Wow, great to see this has been done in vega-embed! Do you have any example how this is then used in Altair/Jupyter Lab combination?

[jakevdp]

@ajasja you can use the tooltip encoding channel; see this for an example: https://github.com/altair-viz/altair/blob/master/altair/vegalite/v2/examples/simple_scatter.py

[SpiritR]

so if I understand correctly, is there currently no support for multi-field (e.g. x,y values) tooltip or any customization (adding titles)?

[domoritz]

In vega-Lite, you can specify multiple fields but I'm not sure whether support for this has landed in Altair. You can specify the title by naming the field title and Vega-Tooltip will use it as the tooltip title.

[jakevdp]

@SpiritR

is there currently no support for multi-field (e.g. x,y values)

This is supported in version 2.1, which has not yet been released. It's also possible in the current release with a bit of a workaround. See #890 for details.

[SpiritR]

This is supported in version 2.1, which has not yet been released. It's also possible in the current release with a bit of a workaround. See #890 for details.

Thanks, I manage to get everything that I needed to work after following that thread. Just running into the issue of getting it to work after embedding that chart via the html format option. Looking forward to version 2.1!

[jakevdp]

FYI: Version 2.1 was released last week.

[ajasja]

@jakevdp

Thanks, works great!

import altair as alt
from vega_datasets import data

iris = data.iris()

alt.Chart(iris).mark_point().encode(
    x='petalWidth',
    y='petalLength',
    color='species',
    tooltip=['species', 'petalWidth', 'petalLength']
).interactive()