Vega-lite: Support Graticule transform

[domoritz]

https://vega.github.io/vega/docs/transforms/graticule/

[kanitw]

Just like sequence, should this be a special type of data source?

[domoritz]

Maybe. Maybe we should have a shortcut to add a graticule layer to any map.

[jakevdp]

I'd love to see this support added...

I think it may be a bit confusing to provide this in the form of a transform... my mental map is that transform takes an input dataset and creates a new one, while this seems to conjure data from nothing.

In the Vega-Lite context, it may make the most sense to add a graticule property to the Projection definition, which can be either boolean, or a definition that basically maps to Vega's graticule transform configuration.

What do you think?

[domoritz]

I agree that it shouldn't appear as a transform. As @kanitw said, it could either be a special data source or as you and I suggested a shortcut. I haven't had the time to think through what it means to put this in the projections, though.

[jheer]

I would like to up-vote this issue and also suggest a different approach: add a graticule mark type. The mark type would be a variant of geoshape that accepts additional mark properties (extent* and step*) that are then passed as parameters to an underlying Vega graticule transform.

{
  "mark": {
    "type": "graticule",
    "stepMinor": [15, 15]
  }
}

The advantages of doing this over modifying the projection transform include a more modular design and control over layering (e.g., placing graticules below or above other elements). The disadvantage is that one has to create multiple marks and layer them, which is more cumbersome when there is otherwise only one layer. All told, I suspect the pros outweigh the cons, as maps often have multiple layers anyway (GeoJSON land masses, sphere boundary, overlaid lat/lon points, etc).

[jheer]

Now that #3942 are #4836 addressed, I want to continue my "tour" of VL geographic data support by adding graticules. Here are two of the language design proposals that have been made:

1. Add a graticule property to projections. If defined, graticules will be added to the map (presumably placed in the lowest layer behind other projected marks). The graticule property could either be an object containing parameters for the Vega graticule transform or a boolean to enable / disable graticules with default parameters. The default behavior would match "graticule": false. The object-valued version would need to include mark properties (stroke, strokeWidth, strokeDash, etc) for configuring the visual appearance.

2. Add a new graticule mark type, which behaves like a geoshape mark but is backed by an inserted Vega graticule transform. The mark parameters would include both Vega graticule transform parameters and standard mark parameters like stroke, etc. In addition, a limited set of encodings would be available to configure visual values, but I think they would all need to be {value: ...}-type encodings (not field defs).

There are advantages and disadvantages to each proposal...

1. Adding a graticule property directly to projections is arguably less work for end-users (in most cases, one could simply add "graticule": true to a projection definition and call it a day). There is some similarity to the Vega-Lite design philosophy elsewhere: we don't expose axes or legends directly, but as a subordinate feature of an encoding. That said, projections are already a bit different, as they are a bivariate scale mapping that we explicitly expose.

2. The primary of advantage of adding a new mark type is greater expressive control. One can control layering, placing the graticule at a user-chosen position within a layer stack. One could also directly include graticules in conditional encoding logic, for example changing potentially changing the graticule color, strokeWidth, or opacity in response to selections. That said, it's not clear how useful that really is: the graticule mesh is represented as a single visual (geoshape) object, so we're talking about changes to the entire mesh.

One other potential advantage of option 2 is consistency: graticules are, in the end, a variation of geoshape, and there are other variations to consider. For example, to add a bounding sphere to a graphic, one can use GeoJSON [{type: "sphere"}] as the input data, then use a standard geoshape. If one includes both graticules and sphere, it would be odd to have them specified in very different styles.

Of course, _a counter-proposal along the lines of option 1_ would be to further include a sphere property (either a boolean or an object with stroke, etc mark parameters) on the projection specification. One might also argue that option 1 provides more convenience, without eliminating the option of users including their own custom input GeoJSON (including graticules) if and only if they want more expressive control.

Thoughts? @domoritz @kanitw @willium @jakevdp @jwoLondon

[domoritz]

I have to think a bit more about this but what are your thoughts on a graticule data source? It feels a bit more consistent with the rest of Vega-Lite and we could implement sequence in a similar fashion.

[mattijn]

The first time I encountered the graticule transform in Vega I was a little confused since it just was just there, no input needed.
So once you suggested another approach of graticule as mark type I thought it's a good idea.

But if you want to do a mapping tour in Vega-lite to increase clarity then support for the following can also be considered:

• _map scale_ (to show the true scale at some location(s) on the map, d3 examples: [1, 2])
• _directional map rose_ (to show true north, especially in cases where north-south is not aligned, in other cases it can have ornamental value)
• _map frame_ with _map axes labelling_ (if done well, sometimes no need for graticules anymore)

These options are all for clarity and not for the actual data. If these all need to be included through different mark types, then a few transformations with some default settings and limited customisation is not so bad after all.

[jwoLondon]

I'd favour Jeff's option 2 because of its expressive control (layer control wrt to ocean, landmass, labels etc. is useful). Option 1 seems conceptually confusing to me. A projection is just a coordinate transformation, to my mind, independent of what is projected into the planar space. It would also prevent people from showing a graticule using the default, unspecified, projection.

If simplicity is the priority, making the setting of the graticule on or off as a property of a geoshape mark would make more sense to me than making it a property of the projection.

Personally, I think it would also be useful to have a Tissot's indicatrix option (which could simply be another property of the graticule - one ellipse for each meridian/parallel intersection). Calculating true indicatrices is a little complex (small circle of infinitesimal diameter), but in practice, for the projections used by Vega-Lite, a small radius small circle works well enough.

[jheer]

@domoritz Good question. I'm not adverse to having generative transforms in Vega-Lite! They could be quite useful. It seemed to me, though, that a graticule mark type (in this very specific case) might be more intuitive -- but I wanted to hear what everyone else thinks. I also separately pinged @arvind in case these decisions carry unexamined implications for interactive selections.

[jheer]

I thought more about @domoritz and @mattijn's comments and arrived at a different solution that I think I like best: add a new class of data specifications for "generators". For example: "data": {"graticule": {...params}} or "data": {"sequence": {...params}}. For the compiled Vega we could use an "empty" dataset that includes a generator as the first transform in the pipeline; any other normally-specified transforms would simply be appended to the pipeline in standard form.

I think this would avoid confusion around having transforms as generators, meanwhile we would maintain the full expressiveness of Vega-Lite without affecting any other abstractions -- one would simply use geoshape as-is for the mark type and projections wouldn't change at all. This approach results in the solution with the smallest new API surface area, no (new) concerns relative to selections, and a simpler localized implementation (less chance of bugs, easier maintenance, etc).

[domoritz]

As an example, a Vega-Lite spec could be

{
  "$schema": "https://vega.github.io/schema/vega-lite/v3.json",
  "width": 800,
  "height": 500,
  "projection": {
    "type": "albersUsa"
  },
  "layer": [
    {
      "data": {"graticule": {...params}},
      "mark": "geoshape"
    },
    {
      "data": {
        "url": "data/us-10m.json",
        "format": {
          "type": "topojson",
          "feature": "states"
        }
      },
      "mark": {
        "type": "geoshape",
        "fill": "lightgray",
        "stroke": "white"
      }
    }
  ]
}

[kanitw]

I might be a bit late to this conversation, but I would like to point out that axes and legends are guides (visualizations) of scales and they are "peer" properties of scale in the encoding directive.

As axes and legends are not properties of scales, graticule similarly shouldn't be a property of projection.

That said, the key benefit of the proposal 1 is that it is much less work for end-users.

Another potential design to preserve this property is to have graticule as a top-level directive (as peer a property of projection, just like how axis/legend are peer of scales in the encoding).

{
  "$schema": "https://vega.github.io/schema/vega-lite/v3.json",
  "width": 800,
  "height": 500,
  "projection": {"type": "albersUsa", ... },
  "graticule": true | {
    ... // properties from Vega graticule transforms
    ... // properties of underlying geoshape (e.g., color, strokeWidth and zIndex -- to control it's relative layer to the marks just like for axes)
  }, 
  "data": {
    "url": "data/us-10m.json",
    "format": {
    "type": "topojson",
    "feature": "states"
  }
  "mark": {
    "type": "geoshape",
    "fill": "lightgray",
    "stroke": "white"
  }
}

Having graticule as its own directive has two keys benefits:

• More concise syntax, requiring smaller transition from unit spec for geoshape
• More consistent with how other guides related to scales

While graticules as generators are more expressive, it's a bit involving, requiring users to transition from a unit spec to a layer spec. With the generator design, graticule becomes the only kind of main guides that require explicit layering.
Imagine we want to add projection to classic Polestar/Voyager (without layer extension), this becomes pretty hairy.

Plus, I'm not sure if we really need that the more expressive powerr in general. (Would we ever need to generate two graticules for the same projection?)

Thus, I'm leaning towards voting for the design of graticule as its own guide directive, separate from projection.

[kanitw]

In fact, the idea of graticule as a directive might also make sense for Vega.

I can see users who want to theme graticule consistently would want to use config.graticule instead of having to apply a style name to every single graticule and set the style config for that style name.

[jheer]

@kanitw: I agree that two graticules at the same time seems unlikely, but the ability to control graticule layering is desirable, as pointed out by @jwoLondon. I don't think the design you've sketched would permit that. Also note my point above about including the sphere boundary, which I use a lot in Vega and already requires layering (unless we want _another_ peer for that, too, but that seems excessive).

[jheer]

Also, as maps regularly require layers, the projection and projection-peer designs might have undesirable consequences (both in terms of implementation, and in terms of user understanding) when merging multi-view specifications.

For example, if I define a graticule in a middle layer, what happens? Does it get added under the bottom layer? (Confusing, I tried to put it in the middle layer!)

Or, what do we do when projections successfully merge, but there are multiple conflicting graticule specifications across layers? Pick only one? Show multiple? (Ugh.)

[jwoLondon]

I think the point by @jheer that maps regularly require layers is an important one. This is more than simply a requirement of a particular architecture; it is a fundamental element of cartographic design. Personally, I find being able to split a map design into layers simplifies rather than complicates the design process.

[kanitw]

Also note my point above about including the sphere boundary, which I use a lot in Vega and already requires layering (unless we want another peer for that, too, but that seems excessive).

Perhaps, the guide of projection shouldn't be called graticule, but have another name to represent the guide that combines both graticule and sphere - just like how axis combines ticks, domain (line), labels.

Maybe we can name it geographicGrid? Note that I'm not a map expert, maybe this is misleading name? Another (bad?) name is probably "projectionGuide".
(If we can't find a good name, then I agree we shouldn't do this though.)

"geographicGrid/projectionGuide": {
    // properties of underlying graticule geoshape (e.g., color, strokeWidth and zIndex -- to control it's relative layer to the marks just like for axes)
    graticuleColor: ...,
    ...
    // properties of the sphere boundary
    sphereColor: ...,
    ...
    // Add other things if there are other important part of projection's guide
    ...
},

but the ability to control graticule layering is desirable, as pointed out by @jwoLondon. I don't think the design you've sketched would permit that.

Good point. But perhaps the point you make below is a solution for this.

For example, if I define a graticule in a middle layer, what happens? Does it get added under the bottom layer? (Confusing, I tried to put it in the middle layer!)

Assume that graticule of the projectionGuide appears below the marks by default, I can see us doing:

layer: [
  oceanUnit,
  landMassUnitWithGraticuleEnabled,
  labelUnit,
] 

What do we do when projections successfully merge, but there are multiple conflicting graticule specifications across layers? Pick only one? Show multiple? (Ugh.)

To make it work for the layering above, we should _never_ merge graticule across layers.

If users really specify multiple graticule across layers, then they get what they ask for. (They probably shouldn't do it, but we currently don't prevent people from defining triple/quadruple/...-axes, either)

[jwoLondon]

BTW, there is at lest one use case for multiple graticules (albeit uncommon) - when you wish to style 'major' and 'minor' graticule lines differently (e.g. opaque lines at 30 degree intervals and semi-transparent lines at 5 degree intervals).

[mattijn]

I think from a grammar agnostic it starts in the Vega axes docs:

Axes visualize spatial scale mappings using ticks, grid lines and labels. Vega currently supports axes for Cartesian (rectangular) coordinates

If there is support for geographic/curvated axes then there is no need for a special graticule transform (maybe still under the hood). Gridlines also just appear with "grid": true. Moreover all the grid, label, ticks and zindex options are already defined and understood by users.

Maybe include options for sphere-left, sphere-top for axis orientation.
It's especially confusing now since there is actual Cartesian support in the map section (identity) but it's disconnected from the axes which are only Cartesian.

[jwoLondon]

I agree that conceptually a graticule is equivalent to grid guides in a cartesian plot. But in VL, those gridlines are a property of positional encoding, whereas geoshapes do not require an explicit position encoding.

[kanitw]

Thanks @jwoLondon for your point about multiple graticules.

With the geographicGrid/projectionGuide directive design, we could also support multiple graticules too. (We have a few proposal for how to do the same for axes in https://github.com/vega/vega-lite/issues/214. Once we agree on a solution there, we could apply the same strategy to the projection's guide if we choose the solution.)

[kanitw]

We discuss and figure that an alternative argument is that projection is more like a coordinate systems and given that we can't give a good name to "projectionGuide" (as it combines so many things). Plus, the generator design would be simpler -- as we don't have to deal with guide merging, etc. The logic for it would be quite encapsulated. Thus, we probably go with a generator design for now.

Should we need to extend Voyager in the future, we could revisit the idea of "projectionGuide" as a macro for the generator design.

[kanitw]

@jheer

Actually, for macro, I can see graticule / sphere be a property of geoshape marks that gets expanded to graticular / sphere layers.

{
  "$schema": "https://vega.github.io/schema/vega-lite/v3.json",
  "width": 800,
  "height": 500,
  "projection": {"type": "albersUsa", ... },
  "data": {
    "url": "data/us-10m.json",
    "format": {
    "type": "topojson",
    "feature": "states"
  }
  "mark": {
    "type": "geoshape",
    "fill": "lightgray",
    "stroke": "white",
    "graticule": {
      ... // properties of graticule
    },
    "sphere": {
      ... // properties of sphere
    },
    "indicatrix": {
      ...
    }
  }
}

(But to make macros, we need to support graticules as generators first anyway.)

[domoritz]

I think graticule as a data source makes sense for any proposed design so I'd say let's add that for now and iterate on it later.

[willium]

I agree, that makes the most sense