Vega-lite: UTC timeunits do not return UTC times in Vega-Lite 4.0

[jakevdp]

In Vega-Lite 4.0, UTC timeunits return the same value as their non-UTC counterparts. This was not the case in Vega-Lite 3.X

Here is a simple spec that demonstrates this:

{
  "data": {"values": [{"t": "2020-01-01T12:00:00-08:00"}]},
  "mark": "point",
  "encoding": {
    "x": {"type": "temporal", "field": "t", "timeUnit": "hours", "title": "hours"},
    "y": {"type": "temporal", "field": "t", "timeUnit": "utchours", "title": "utchours"}
  }
}

I viewed this on a computer set to the US/Pacific timezone.

With Vega-Lite 3.4.0 & Vega 5.9.1, I see the expected result: the local hour is 12, and the UTC hour is 20.

With Vega-Lite 4.0.2 & Vega 5.9.1, the result has changed: the UTC timeUnit is the same as the local timeUnit:

This seems to be the case not just for utchours but for every UTC timeUnit Vega-Lite supports: each UTC timeUnit returns identical timestamps as its non-UTC counterpart.

[jheer]

Examining the generated Vega output, I see that the timeunit transformations are not using the timezone parameter. For UTC values, the parameter "timezone": "utc" should be applied. In addition, I see two time scales being generated. I would expect to see one time scale and one utc scale.

[haldenl]

Thank you for the report and helpful details! I think I know where the issue is—the switch to vega time format was incomplete. Addressing today.

[haldenl]

I made a PR, #5780 . When utc timeunits are specified, the fix creates compiled vega such that:

1. Transforms that have the 'utc' timezone.
2. Scales that have the 'utc' type.
3. Labels that are formatted with 'utcFormat' instead of 'timeFormat'.

An inconsistency I'm noticing. With the spec Jake provided, (1) and (2) are not necessary as long as (3) is true. What's more, if (1) and (2) are true but not (3), the result is incorrect (still local timezone). Is this okay for normalization's sake?

[SyntaxRules]

I think I found another manifestation of this same issue, and may be fixed by @haldenl 's PR. I'll post the details here, let me know if this is a separate issue and I'll open up a different one.

Marks on the following chart are on the 1st and 15th of each month, but don't align with the axis marks. The marks are suppose to be in "timeFormat": "utcyearmonthdate" but show off by one.

Here is a link to the spec

I wait until the PR is ready, then try out the fix.

[domoritz]

@SyntaxRules After the fix

This seems right since the dates are parsed as local, right?

[SyntaxRules]

@domoritz Its closer 😄 ! What I'm expecting is the rule mark on "2020-01-01" to match exactly with the axis tick mark on "2020-01-01". In this use case I really do not care about what timezone the user is in, I always want the rule mark to line up.

I pulled up a editor with vega-lite 3.2.1 and this is what I got:

This is what I expect; The marks align perfectly with the axis. 3.2.1 is giving me the desired chart.

[domoritz]

Isn't that incorrect in vl 3.2.1? Your dates are midnight in local time (unless you are in UTC). Your scale and axis are in UTC on the other hand.

[haldenl]

@SyntaxRules
It looks like the issue has to do with the inconsistent parsing of the Date object (see here). In the spec, data is given in year-month-day format, which is interpreted in UTC. The datetime objects provided as tick values are interpreted in local time. If we switch to the same formatting:

{
  "$schema": "https://vega.github.io/schema/vega-lite/v4.json",
  "data": {
    "values": [
      {"d": "2020-01-01"},
      {"d": "2020-01-15"},
      {"d": "2020-02-01"},
      {"d": "2020-02-15"},
      {"d": "2020-03-01"}
    ]
  },
  "mark": "rule",
  "width": 600,
  "height": 100,
  "encoding": {
    "x": {
      "scale": {"domain": ["2019-12-15", "2020-04-01"]},
      "axis": {
        "format": "%Y-%m-%d",
        "orient": "bottom",
        "title": "CY",
        "values": ["2019-12-01", "2020-01-01", "2020-02-01", "2020-03-01"]
      },
      "field": "d",
      "type": "temporal",
      "timeUnit": "utcyearmonthdate"
    }
  }
}

The output is as expected (on the fix branch). I wonder if something changed with datetime usage between 3.2.1 and 4.0. Will investigate.