Vega-lite: Wrong UTC / Local time interpretation
{
"layer": [
{
"mark": {"type": "point", "color": "#3366cc"},
"encoding": {
"x": {"type": "ordinal", "field": "date", "timeUnit": "date"},
"y": {"type": "ordinal", "field": "date", "timeUnit": "hours"}
}
},
{
"mark": {"type": "point", "color": "#dc3912", "shape": "square"},
"encoding": {
"x": {"type": "ordinal", "field": "date", "timeUnit": "date"},
"y": {"type": "ordinal", "field": "date", "timeUnit": "utchours"}
}
}
],
"data": {
"values": [
{"date": "2018-02-01"},
{"date": "2018-02-02 00:00"},
{"date": "2018-02-03 00:00 UTC"},
{"date": "2018-02-04 00:00Z"}
]
},
"$schema": "https://vega.github.io/schema/vega-lite/v2.6.0.json"
}
I have UTC+3 so expected:
as
{
"layer": [
{
"mark": {"type": "point", "color": "#3366cc"},
"encoding": {
"x": {"type": "ordinal", "field": "date", "timeUnit": "date"},
"y": {"type": "ordinal", "field": "local"}
}
},
{
"mark": {"type": "point", "color": "#dc3912", "shape": "square"},
"encoding": {
"x": {"type": "ordinal", "field": "date", "timeUnit": "date"},
"y": {"type": "ordinal", "field": "UTC"}
}
}
],
"data": {
"values": [
{"date": "2018-02-01" ,"local":0,"UTC":21},
{"date": "2018-02-02 00:00","local":0,"UTC":21},
{"date": "2018-02-03 00:00 UTC", "local":3,"UTC":0},
{"date": "2018-02-04 00:00Z","local":3,"UTC":0}
]
},
"$schema": "https://vega.github.io/schema/vega-lite/v2.6.0.json"
}
iliatimofeev
All 12 comments
If you're using UTC, you should use the UTC time units: https://vega.github.io/vega-lite/docs/timeunit.html#utc
domoritz
on 27 Jul 2018
Well I read it twice, I can't find any mention why I can't show UTC or local depending on what I put in data.
only warning about double conversion on scale but I don't use it.
Do not use UTC time unit and the UTC scale type at the same time since that will cause Vega-Lite to shift the output time twice.
The only issue I can see that "2018-02-01" is interpreted as UTC, but i believe that it should be local and if it is ISO than it has to be local if it is not ISO it has to be local accoding to documentation
iliatimofeev
on 27 Jul 2018
Except that JavaScript interprets strings in a particular way: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Date/parse
Thanks for the. We will look at these and see what we can do from the Vega-Lite and Vega side.
domoritz
on 27 Jul 2018
UPDATE - never mind - by adding a scale to the tooltip, it works OK
The parser was doing the right thing, parsing to UTC - I needed to tell the tooltip to display in UTC.
Sorry for the interruption.
Bad way, in action: https://bl.ocks.org/ijlyttle/84ce12ce2c1093070f645a2bae3b8fa0
Better way, in action: https://bl.ocks.org/ijlyttle/bccff7673c98e61b887684dbe597cbd3
ijlyttle
on 27 Jul 2018
@iliatimofeev
Copying a comment from slack to get it here. I also initially expected the same as you, but it turns out the current behavior (shown in the left-most plot) is consistent with the documented behavior of javascript date parsing (see https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Date/parse)
The date time string may be in a simplified ISO 8601 format. For example, "2011-10-10" (just date) or "2011-10-10T14:48:00" (date and time) can be passed and parsed. Where the string is ISO 8601 date only, the UTC time zone is used to interpret arguments. If the string is date and time in ISO 8601 format, it will be treated as local.
When you use an ISO fragment, it it treated as UTC, while a full ISO string is treated as local time.
So the question I suppose is whether this is a big enough problem to tackle adding a custom date parser in Vega. Whether or not that happens, I'm planning to make Altair serialize all dates in full ISO so that they will be treated as local time.
jakevdp
on 27 Jul 2018
vega
{
"$schema": "https://vega.github.io/schema/vega/v3.json",
"autosize": "pad",
"padding": 5,
"data": [
{
"name": "table",
"values": [
{"date": "2018-02-01", "local": 0, "UTC": 21},
{"date": "2018-02-02 00:00", "local": 0, "UTC": 21},
{"date": "2018-02-03 00:00 UTC", "local": 3, "UTC": 0},
{"date": "2018-02-04 00:00Z", "local": 3, "UTC": 0}
],
"transform": [
{"type": "formula", "expr": "toDate(datum['date'])", "as": "timestamp"},
{"type": "formula", "expr": "hours(datum['timestamp'])", "as": "hrs"},
{"type": "formula", "expr": "utchours(datum['timestamp'])", "as": "utchrs"},
{"type": "formula", "expr": "date(datum['timestamp'])", "as": "day"}
]
}
],
"marks": [
{
"name": "layer_0_marks",
"type": "symbol",
"style": ["point"],
"from": {"data": "table"},
"encode": {
"update": {
"opacity": {"value": 0.7},
"fill": {"value": "transparent"},
"stroke": {"value": "#3366cc"},
"x": {"scale": "x", "field": "day"},
"y": {"scale": "y", "field": "hrs"}
}
}
},
{
"name": "layer_1_marks",
"type": "symbol",
"style": ["point"],
"from": {"data": "table"},
"encode": {
"update": {
"opacity": {"value": 0.7},
"shape": {"value": "square"},
"fill": {"value": "transparent"},
"stroke": {"value": "#dc3912"},
"x": {"scale": "x", "field": "day"},
"y": {"scale": "y", "field": "utchrs"}
}
}
}
],
"scales": [
{
"name": "x",
"type": "point",
"domain": {
"fields": [
{"data": "table", "field": "day"}
],
"sort": true
},
"range": {"step": 22},
"padding": 0.5
},
{
"name": "y",
"type": "point",
"domain": {
"fields": [
{"data": "table", "field": "hrs"},
{"data": "table", "field": "utchrs"}
],
"sort": true
},
"range": {"step": 22},
"padding": 0.5
}
],
"axes": [
{
"scale": "x",
"orient": "top",
"grid": false,
"title": "date",
"labelOverlap": true,
"encode": {
"labels": {
"update": {
"text": {"signal": "format(datum.value, ',')"},
"angle": {"value": 270},
"align": {"value": "left"},
"baseline": {"value": "middle"}
}
}
},
"zindex": 1
},
{
"scale": "y",
"orient": "left",
"grid": false,
"title": "local, UTC",
"labelOverlap": true,
"zindex": 1
}
]
}
iliatimofeev
on 27 Jul 2018
@ijlyttle
My issue is with how the tooltip is not parsing to UTC as I would expect
The issue is that "2018-01-01" is parsed as UTC, and then adjusted to local time before being displayed. If you want it to be parsed as local time, then you can use "2018-01-01T00:00:00".
jakevdp
on 27 Jul 2018
@jakevdp In this case, I want to keep everything as UTC so that the rendering is insensitive to the browser's time-zone. I found a fix, the tooltip encoding can accept a UTC scale.
ijlyttle
on 27 Jul 2018
My preferred solution for Altair is to try to keep everything local, because by default vega/vega-lite renders in local time. I'd rather have the default just work than require users to manually flag all outputs as UTC every time they use temporal data.
jakevdp
on 27 Jul 2018
In fact it is the same issue https://github.com/vega/vega-lite/issues/4044 I just found it, sorry for doubling issue.
iliatimofeev
on 28 Jul 2018
domoritz
on 28 Jul 2018
Since it's the same as https://github.com/vega/vega-lite/issues/4044 and https://github.com/vega/vega-lite/issues/4044 is a wontfix, I'll close this too.
Please feel free to reopen if you have a suggestion for better solution.
kanitw
on 8 Jun 2019
Related issues
kanitw
路
3Comments
kanitw
路
3Comments
kanitw
路
4Comments
mcnuttandrew
路
3Comments
domoritz
路
4Comments
Most helpful comment
My preferred solution for Altair is to try to keep everything local, because by default vega/vega-lite renders in local time. I'd rather have the default just work than require users to manually flag all outputs as UTC every time they use temporal data.