Vega-lite: plotting a column name with '.' in it eventually leads to a vega error

Created on 13 Dec 2016 · 17Comments · Source: vega/vega-lite

I'm doing an Altair chart with a color given by color=Color('gpuinfo.name:N', .... This produces an eventual Vega error.

(Why do I have a column with a period in its name? Reading nested JSONs into pandas is nicely handled by json_normalize, which flattens nested JSON levels into dot-separated column names.)

undefined:3
return x["gpuinfo"]["name"];
                   ^

TypeError: Cannot read property 'name' of undefined
    at Object.eval [as get] (eval at <anonymous> (/Users/jowens/Documents/working/vega/node_modules/datalib/src/util.js:147:20), <anonymous>:3:20)
    at Facetor.proto._cellkey (/Users/jowens/Documents/working/vega/node_modules/datalib/src/aggregate/aggregator.js:106:23)
    at Facetor.proto._cell (/Users/jowens/Documents/working/vega/node_modules/datalib/src/aggregate/aggregator.js:114:38)
    at Facetor.proto._add (/Users/jowens/Documents/working/vega/node_modules/datalib/src/aggregate/aggregator.js:151:19)
    at add (/Users/jowens/Documents/working/vega/src/transforms/Aggregate.js:174:31)
    at Array.forEach (native)
    at Aggregate.prototype.transform (/Users/jowens/Documents/working/vega/src/transforms/Aggregate.js:190:13)
    at Aggregate.prototype.evaluate (/Users/jowens/Documents/working/vega/src/transforms/Transform.js:48:15)
    at Node.dataRef (/Users/jowens/Documents/working/vega/src/scene/Scale.js:387:15)
    at Node.ordinal (/Users/jowens/Documents/working/vega/src/scene/Scale.js:127:20)

Source

jowens

Most helpful comment

I found some related issues in Vega 3, and just published an updated version of vega-util that resolves them. By default, Vega will try to parse field names like "a.a" into nested lookups (datum['a']['a']). However, you can indicate _non-nested_ field names by wrapping the name in brackets. For example, the field names [a.a] and ['a.a'] will both map to datum['a.a']. It is also legal to use escape characters: a\\.a -> datum['a.a'].

If you are referring to such fields within an expression, simply use standard JavaScript bracket notation: datum['a.a'].

Vega-Lite probably needs to perform a check to see if the . character is in the field name, and if so, wrap the field name in brackets.

jheer on 25 Jan 2017

👍3

All 17 comments

@willium I thought you already fixed this a while ago. Any clue?

kanitw on 11 Jan 2017

I'll check it out. Weird

willium on 11 Jan 2017

@willium this toy spec should (WIP) work:

{
  "description": "A simple bar chart with embedded data.",
  "data": {
    "values": [
      {"a.a": "A","b": 28}, {"a.a": "B","b": 55}
    ]
  },
  "mark": "bar",
  "encoding": {
    "x": {"field": "a.a", "type": "ordinal"},
    "y": {"field": "b", "type": "quantitative"}
  }
}

@jowens still doesn't work. For debugging purpose.

kanitw on 25 Jan 2017

👍1

If you are referring to such fields within an expression, simply use standard JavaScript bracket notation: datum['a.a'].

Vega-Lite probably needs to perform a check to see if the . character is in the field name, and if so, wrap the field name in brackets.

jheer on 25 Jan 2017

👍3

As discussed in #1890, . needs to be escaped. We will document this behavior.

domoritz on 27 Jan 2017

@domoritz @jakevdp can you offer wisdom on how I should make Altair produce wrapping or backslashes appropriately? Altair code would look something like this: color=Color('gpuinfo.name:N').

jowens on 28 Jan 2017

color=Color('gpuinfo\\.name:N') -> 'field': u'gpuinfo\\.name' ->

undefined:3
return x["gpuinfo\\"]["name"];
                     ^

TypeError: Cannot read property 'name' of undefined

color=Color(['gpuinfo.name:N']) -> traitlets.traitlets.TraitError: The 'shorthand' trait of a Color instance must be a unicode string, but a value of ['gpuinfo.name:N'] <type 'list'> was specified.

jowens on 28 Jan 2017

We have fixed this in Vega-Lite 2/ Vega 3 but since Altair is using the stable VL 1 and VG 2, these fixes may not work.

However, instead of color=Color(['gpuinfo.name:N']), I think it should be color=Color('[gpuinfo.name]:N').

domoritz on 28 Jan 2017

PERFECT that totally works thanks @domoritz

jowens on 28 Jan 2017

@domoritz any wisdom on expressing a filter with a column that has a period in it?

   Chart(). ...
    ).transform_data(
        filter=(expr.df['gpuinfo.name'] == gpu)
    )

appears to eventually result in:

return ((datum.gpuinfo.name=='Tesla K80'));

which throws a Vega error.

undefined:3
"use strict"; return ((datum.gpuinfo.name=='Tesla K80'));
                                    ^

TypeError: Cannot read property 'name' of undefined
    at Object.eval (eval at <anonymous> (/Users/jowens/Documents/working/vega/node_modules/vega-expression/src/index.js:18:31), <anonymous>:3:37)
    at value.fn (/Users/jowens/Documents/working/vega/node_modules/vega-expression/src/index.js:22:27)
    at /Users/jowens/Documents/working/vega/src/transforms/Filter.js:29:9
    at Array.forEach (native)
    at Filter.prototype.transform (/Users/jowens/Documents/working/vega/src/transforms/Filter.js:28:13)
    at Filter.prototype.evaluate (/Users/jowens/Documents/working/vega/src/transforms/Transform.js:48:15)
    at Model.prototype.evaluate (/Users/jowens/Documents/working/vega/node_modules/vega-dataflow/src/Graph.js:329:16)
    at Model.prototype.propagate (/Users/jowens/Documents/working/vega/node_modules/vega-dataflow/src/Graph.js:161:22)
    at Model.prototype.fire (/Users/jowens/Documents/working/vega/src/core/Model.js:159:8)
    at View.prototype.update (/Users/jowens/Documents/working/vega/src/core/View.js:369:11)

jowens on 16 Feb 2017

For the record, this doesn't work either:

    ).transform_data(
        filter="datum['gpuinfo.name'] == gpu"
    )

[Vega Err] PARSE DATA FAILED: source Error: Unknown signal "gpu"
[Vega Err] Error: Unknown signal "gpu"
[Vega Err] Error: Unknown signal "gpu"
/Users/jowens/Documents/working/vega/bin/vg2svg:66
    if (err) { throw err; }
               ^

Error: Unknown signal "gpu"
    at Model.prototype.connect (/Users/jowens/Documents/working/vega/node_modules/vega-dataflow/src/Graph.js:266:15)
    at DataSource.prototype.pipeline (/Users/jowens/Documents/working/vega/node_modules/vega-dataflow/src/DataSource.js:111:9)
    at Model.prototype.data (/Users/jowens/Documents/working/vega/node_modules/vega-dataflow/src/Graph.js:66:58)
    at Model.prototype.data (/Users/jowens/Documents/working/vega/src/core/Model.js:79:24)
    at Function.parseData.datasource (/Users/jowens/Documents/working/vega/src/parse/data.js:55:18)
    at /Users/jowens/Documents/working/vega/src/parse/data.js:38:17
    at Array.forEach (native)
    at Object.parseData [as data] (/Users/jowens/Documents/working/vega/src/parse/data.js:32:16)
    at parse (/Users/jowens/Documents/working/vega/src/parse/spec.js:72:29)
    at Object.parseSpec [as spec] (/Users/jowens/Documents/working/vega/src/parse/spec.js:34:5)

jowens on 16 Feb 2017

I can't test it right now but the error says that the signal gpu is
unknown. If it is a string, wrap it in ' or ".

On Wed, Feb 15, 2017, 19:42 John Owens notifications@github.com wrote:

For the record, this doesn't work either:
).transform_data(
    filter="datum['gpuinfo.name'] == gpu"
)
[Vega Err] PARSE DATA FAILED: source Error: Unknown signal "gpu"
[Vega Err] Error: Unknown signal "gpu"
[Vega Err] Error: Unknown signal "gpu"
/Users/jowens/Documents/working/vega/bin/vg2svg:66
if (err) { throw err; }
^

Error: Unknown signal "gpu"
at Model.prototype.connect (/Users/jowens/Documents/working/vega/node_modules/vega-dataflow/src/Graph.js:266:15)
at DataSource.prototype.pipeline (/Users/jowens/Documents/working/vega/node_modules/vega-dataflow/src/DataSource.js:111:9)
at Model.prototype.data (/Users/jowens/Documents/working/vega/node_modules/vega-dataflow/src/Graph.js:66:58)
at Model.prototype.data (/Users/jowens/Documents/working/vega/src/core/Model.js:79:24)
at Function.parseData.datasource (/Users/jowens/Documents/working/vega/src/parse/data.js:55:18)
at /Users/jowens/Documents/working/vega/src/parse/data.js:38:17
at Array.forEach (native)
at Object.parseData [as data] (/Users/jowens/Documents/working/vega/src/parse/data.js:32:16)
at parse (/Users/jowens/Documents/working/vega/src/parse/spec.js:72:29)
at Object.parseSpec [as spec] (/Users/jowens/Documents/working/vega/src/parse/spec.js:34:5)

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/vega/vega-lite/issues/1775#issuecomment-280191498,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAj86tkUOqG99orRUNOTl6VrdAr3Qaooks5rc5t-gaJpZM4LLVt0
.

domoritz on 16 Feb 2017

I see an error unrelated to the column name: gpu should be "gpu".

jheer on 16 Feb 2017

Oh, right, um, gpu is a Python variable. Can I persuade the filter call to paste in the variable value rather than its name?

jowens on 16 Feb 2017

for gpu in gpus:
    chart[gpu] = Chart(df).mark_point().encode(
    ...
    ).transform_data(
        # filter=(expr.df['gpuinfo.name'] == gpu)
        filter="datum['gpuinfo.name'] == gpu"
    )

jowens on 16 Feb 2017

And this appears to work! Woo!

filter="datum['gpuinfo.name'] == \"%s\"" % gpu,

jowens on 16 Feb 2017

Okie. Sounds like we can close this :)

kanitw on 16 Feb 2017

Was this page helpful?

0 / 5 - 0 ratings

Related issues

Refactor: extract special Expression type

kanitw · 4Comments

Better support for log histogram?

kanitw · 3Comments

Styling points on mark type line

mcnuttandrew · 3Comments

legend's symbolOpacity does not work

swanderz · 4Comments

Add schema property like Vega 3

kanitw · 4Comments