Vega-lite: Interacting with Legends & Axes

[brylie]

NVD3, and possibly other charting libraries, allow users to filter charts by interacting with the chart legend:

This is useful for exploratory analysis, and makes toggling categories very easy.

Feature

Is it planned/possible for vega-lite charts to have interactive legends for filtering chart data?

[kanitw]

Thanks for asking. This will likely be a part of v2.x release.

[domoritz]

Interactions with legends was a bit tricky with vega (at least v2). Has that changed? cc @arvind

[arvind]

Yes, with Vega 3, axes and legends can now be interacted with. So this should be supported with Vega-Lite 2.0. The open question with interactions, however, is what defaults should be. For example, if a legend is added to the visualization, should it be interactive by default? Should the interaction it supports also extend to interacting with the plotted data points as well? Etc.

[jheer]

We should definitely try to support these kinds of interaction! Not sure if interactive-by-default is the right way to go, but we would need to be able to provide explicit control in either case.

This relates to the selection project transform. As I understand it, our default design right now is no projection for single/multi point selections, and [x, y] projection for interval selections. For axes and continuous (color gradient) legends, I imagine one could define an interval selection projected to the appropriate field/channel/guide. For discrete legends, we might need to default a single-point or multi-point selection to project just the field(s) backing the legend.

There is also the potential issue that users might just want to be able to (more or less) say: "I want to enable selections on this legend (or axis)" without having to think about the appropriate selection type (interval, etc).

[domoritz]

I don't think we want any interactions by default. The problem is that I don't know what channel we would use (e.g. for highlighting) and how we resolve conflicts if the user uses that channel to encode data.

[kanitw]

RE: default interaction, let's create a new issue (#1658) to discuss that
since it's going a little off topic here.

[brylie]

For what it's worth, NVD3 chart legends are interactive by default.

[domoritz]

We have this on the roadmap for 2.1

[kanitw]

Given that we will follow semantic versioning, it is probably more accurate to say "very early versions of 2.x minor release".

[domoritz]

I think this would be awesome to improve some of the default interactions in pdvega. CC @jakevdp

[domoritz]

Here is the Vega example https://vega.github.io/vega/examples/interactive-legend/

[kenklin]

I need to enhance some of my Vega specs to be more interactive, so I experimented migrating them to Vega-Lite, allowing me to easily add zoom (https://vega.github.io/vega-lite/docs/zoom.html). I was previously only (somewhat) familiar with Vega specs, so this is the first time I've tried Vega-Lite.

My port has been mostly successful with the notable exception of my Vega spec's use of https://vega.github.io/vega/examples/interactive-legend/. After searching, I found _this_ open issue 1657.

Do you have any practical recommendations on how I might proceed? I think my options are:

1) Shelve my Vega-Lite code and resume development on my Vega code.

2) Inquire if there is a way to "manually" modify the Vega-Lite spec to create interactive-legend-like behavior. I am guessing this is a dead end as I infer that Vega-Lite does not have signals (https://github.com/vega/vega-lite/milestone/55).

3) Try compiling (which I haven't ever tried) my Vega-Lite spec and see if I write a routine to programmatically inject the interactive-legend spec sequence into the output Vega spec and use the latter. Then, every time I modify my Vega-Lite, I compile it and post-massage it.

I'm leaning slightly towards option 3, but before I jump into this rabbit hole, I thought I'd solicit any helpful advice.

Thanks in advance,
-Ken

[domoritz]

You can try to manually change a spec to manually perform 3 to see how much change is needed and how you might do it automatically. I think it will be the easiest option. Please report back so we can see how much code we need to add to Vega-Lite to add this feature.

[king612]

+1!

This would be an awesome feature to add. In keeping with Vega-Lite philosophy, maybe just a interactive: true property to behave like the Vega example you reference. Namely, legend labels/symbols behave like checkboxes to select/deselect different series of data within the plot. Shift-click allows multiselect on the legend/series. That would be really sweet. I have a complex chart with 15 - 20 different variable plotted on a timeline, much like the example at the start of this ticket. Currently having to build all the filtering by hand. I suspect using the legend to filter would be a very common use case.

Thanks,
John K.

[jakevdp]

During the Altair tutorial yesterday, I realized you could hack a clickable legend in the current release using selections :smile:

{
  "config": {"view": {"width": 400, "height": 300}},
  "hconcat": [
    {
      "mark": "point",
      "encoding": {
        "color": {"type": "nominal", "field": "Origin", "legend": null},
        "x": {"type": "quantitative", "field": "Horsepower"},
        "y": {"type": "quantitative", "field": "Miles_per_Gallon"}
      },
      "transform": [{"filter": {"selection": "legend"}}]
    },
    {
      "mark": "point",
      "encoding": {
        "color": {
          "condition": {
            "type": "nominal",
            "aggregate": "min",
            "field": "Origin",
            "legend": null,
            "selection": "legend"
          },
          "value": "lightgray"
        },
        "y": {"type": "nominal", "field": "Origin", "title": null}
      },
      "selection": {
        "legend": {
          "type": "multi",
          "encodings": ["color"],
          "on": "click",
          "toggle": "event.shiftKey",
          "resolve": "global",
          "empty": "all"
        }
      }
    }
  ],
  "data": {"url": "data/cars.json"},
  "$schema": "https://vega.github.io/schema/vega-lite/v2.4.1.json"
}

[jheer]

Nice hack! It would be great to eventually have VL/Altair produce true interactive legends that Vega already supports: https://vega.github.io/vega/examples/interactive-legend/

[brylie]

Although, the VegaLite code for interactive legend seems a bit obscure with event handling. Adding interactive: true to the legend definition would ideally be sufficient.

[kanitw]

I don't think it will be as simple as adding interactive: true -- as it will be totally diverging from how interactions are defined by specifying selection and applying them in the rest of the language.

Instead, it's more likely that we will add support for this as a part of the selection framework that we have.

[kanitw]

https://tech.shutterstock.com/rickshaw/ is another library that has some support for this.

[kanitw]

@arvind - if interacting with legend is simply applying encoding projection to the channels with legend, I'm wondering how our grammar would extend to interacting with axes.

Simplying having a projection on x and y won't convey that users want the interaction to apply to axis, not the plot (like the axis brush in this Vega example).

[arvind]

I see a couple of different ways of enabling interactive legends in Vega-Lite:

1. Legends correspond to their own separate/special selections that can then be used, like a regular selection, in the remainder of the specification.

   This approach would mimic @jakevdp's example above and has the following advantages:

   • We maintain compatibility with our selections/interaction grammar: legends are not interactive by default, and users must explicitly specify the result of a legend interaction.

   • Interaction semantics are clear – legend interaction is separate from direct manipulation or input widgets which prevents any confusion (as we saw when direct manipulation co-existed alongside input widgets, #5277)

   • Users do not need to worry about understanding selection semantics (i.e., "what type of selection do I need?") and can just directly invoke the selection to drive conditional encoding or interactive filters.

   But, this approach has some open questions and expressivity bounds:

   • Does each legend correspond to its own selection or is there one selection that maps across all legends? The former is the equivalent of the following explicit specification "colorLegend": {"type": "multi", "encodings": ["color"]} while the latter would be "legend": {"type": "multi", "encodings": ["color", "shape", ...]}. Of course, we could do both.

   • If it were a legend selection, is it only populated when at least one entry is selected across _all_ legends? Or, do we partially populate the selection if at least _one_ legend item is selected? In this latter case, the interactive semantics would be subtly different from what they currently are for selections: currently, we populate a complete selection definition (i.e., all fields) on every interaction event. In the partial population case, we would instead build up a selection step-by-step and calculate the cross product of all selected legend items. In practice, I've not found this to be an issue but felt it worth flagging as it is a subtle nuance in semantics.

   • To drive the same interaction the legends support but via direct manipulation, users would need to explicitly add a new/custom selection and then union its invocation with their existing spec. This is more friction than opting back-in for direct manipulation when a selection is bound to input widgets. Thus, this type of accretive selection would be more difficult.

   • Presumably, a legend could be marked as "interactive": false to disable interaction. If only a legend selection was surfaced, this would be fine; if legend-specific selections were exposed, disabling legend interactivity would potentially require additional changes to the specification (or we could say that its corresponding selection always returns true).

2. Legends are made interactive when the user defines a single or multi projected over its domain. Clicking a legend populates any selections it matches with in a partial fashion as described above. A working implementation of this approach can be found in #5305.

   Advantages include:

   • We maintain compatibility with our selections/interaction grammar: legends are not interactive by default, and users must explicitly specify the result of a legend interaction.

   • Interactive legends are a property of defined selections, in an analogous fashion to input widgets, rather than something separate. It would feel like users get interactive legends "for free" if they were already engaging in specifying a selection. A legends: false would be a selection property, to prevent a selection from being populated by legend interaction. If all selections are marked legends: false, then the legends are no longer interactive (we could imagine surfacing an interactive: false property on legends to disable interactivity across all selections).

   • We get direct manipulation + legend interaction populating the same selections. Besides starting a selection via the legend, and continuing it via direct manipulation (i.e., click a legend, shift-click points) we also gain a higher expressive ceiling (e.g., drill down interactions where a legend partially populates only one field of a selection, and then the user uses direct manipulation to complete it).

   The main disadvantage is that direct manipulation + legend interaction could cause the same confusion folks experienced when direct manipulation was left enabled alongside input widgets. Currently, #5305 treats legends as a separate "view" and applies the selection resolution scheme against it. This approach can yield unintuitive behaviors like the one shown in the following GIF:

   

   Here, we have a "type": "multi", "fields": ["Origin"] selection and all clicks you see are shift-clicks. A potentially confusing situation arises when I click the legend to select "USA" and then shift-click a red highlighted point expecting to toggle it out of the selection. But, this actually adds another entry to the selection state as the visualization and legends are separate views. This behavior is consistent with what happens when shift-clicking in multi-view visualizations (e.g., a SPLOM) but it's unclear whether users will think of a legend as a "separate" view.

I've been thinking about interactive legends long enough that it's no longer clear which set of tradeoff are more desirable, so I'd love folks' thoughts!

[jheer]

Thanks for working through this!

My preference is to define interactive legend support in terms of our existing selection abstraction. Once suitably defined, we could later consider syntactic sugar / shortcuts. Alternatively, higher level APIs like Altair could define their own shortcuts, like the existing interactive() method for charts.

I would expect legend selections to be similar to bound input widgets: a specific opt-in condition. So, I would not expect a legend to automatically become interactive if a selection was defined over it's domain; I would instead expect to specify a selection parameter indicating that legend "binding" was desired. Perhaps we could even reuse the existing bind keyword, e.g., "bind": {"input": "legend"}?

I think selections that map across multiple legends are complicated to reason about given the partial population semantics. Might it be simpler to only allow selections for individual legends, and require users to use boolean operators to combine multiple selections if desired? In this case, it would be a schema violation to define a legend-bound selection over multiple fields / encodings that are not contained within a single legend.

As previously discussed for bound widgets, I would also expect interactive legend selections to not include direct manipulation events on the chart itself by default. I would also be fine with an initial implementation that does not support the combination of interactive legends and direct manipulation in the chart (unless achieved through boolean combinations of multiple, separately defined selections).

Obviously there is some overhead for users to learn that, for example, they need to use single or multi selections for symbol legends and interval selections for quantitative legends. (Note that I don't see the latter included in current proposals, but we should at least track it for the future?) I'm not too concerned about selection type overhead, but perhaps there is an allowable shortcut here... Could one omit the selection type if bound to a legend, and infer it from the encoding information? Doesn't seem a high priority to me, though.

[jheer]

On the topic of interactive color gradient legends, I mocked up a Vega spec that includes a legend-bound brush. Rather than "inject" a mark into a legend group, it simply layers a brush mark on top, while getting the necessary geometry information as part of event handling.

There are two problems remaining, though:

1. The brush position will not respond to resizing / relayout events. It will snap into proper place upon subsequent interactions though.
2. There is not a good mechanism for scale inversion. The color scale is defined in terms of colors, but the brush operates in pixel space. In the example above I recover fractional indices along the pixel domain. While these are easy to map to true domain values for linear scales, things get much more complicated for non-linear scales. The interpolated scales in D3 do not expose a method for mapping from interpolation fractions back to domain values. So, unless I'm overlooking something, we will either need to extend those scale implementations or introduce additional utilities (not unlike our panLinear/Log/Pow/Symlog, etc. vega-util methods).

_UPDATE (Sep 14)_: I opened a PR on d3-scale to add invert methods to interpolated (sequential, diverging) scales: https://github.com/d3/d3-scale/pull/191

[zanarmstrong]

Hi all - @jakevdp pointed me to this thread based on a conversation we were having about the value of non-linear interactive color legends.

Just wanted to add my support for:

• interactive color gradient legends, that allow the user to change the mapping from numbers to colors. Or to switch between a color gradient and a "highlight" color.
• intentionally non-linear colormaps (which are already possible, but might be more accessible/attractive via interactive color legends).

For some context/motivation, I spoke about the relationship between color map and the question you want to ask the data (different color maps ask different questions of the data) in this talk at SciPy in 2018. At work, I've created interactive color legends for data vis-based analysis tools.

(on a related topic, I've also been finding it quite useful recently to discretize my continuous colormaps a la Adam Pearce's coloring maps blog post (@1wheel on github).

thanks all!

[jheer]

Thanks for the input @zanarmstrong! As discussed above, we're looking into supporting interaction with color gradient legends too, so these are useful points to keep in mind.

Discretized color maps (e.g., using quantile or quantize scales) are well-supported (here's a simple example). Obviously, interaction with discretized legends would also be great to support, and might function similarly to multi selections in symbol legends (via "clickable" color bands) and/or akin to interval selections over an ordinal domain (via a brush that selects all values within overlapping color bands).