Vega-lite: Problem in DataFormatBase "parse" : "auto"

[RobinL]

The following syntax does not work as expected. The text string "auto" seems to get passed into vega lite as an object like {0: "a", 1: "u" 2: "t" 3: "o"}

"format": {
      "type": "csv",
      "parse": "auto"
    }

See here for an example spec/data:
https://vega.github.io/editor/#/gist/vega-lite/robinl/fd3e4ab316743468ea42e44a9cb150a4/d451f25572515a0db606dedf10ac21155c8df43d/possible_bug.vl.json

Here are some screenshots of the error from the Chrome debugger:

If this is indeed a bug, I'd be happy to attempt a fix of this if someone could point me to the right place in the codebase. I can't see anything obviously wrong here but I didn't get any further than that.

[domoritz]

It's

"format": {
      "type": "csv",
      "format": {
        "parse": "auto"
      }
    }

See https://vega.github.io/vega-lite/docs/data.html#format

[RobinL]

Thanks. Apologies for the mistake.

[domoritz]

Wait, sorry, I misread the code. This is indeed a bug. Apologies.

{
  "$schema": "https://vega.github.io/schema/vega-lite/v2.json",
  "description": "A scatterplot showing horsepower and miles per gallons for various cars.",
  "data": {"url": "data/cars.json", "format": {"parse": "auto"}},
  "mark": "point",
  "encoding": {
    "x": {"field": "Horsepower", "type": "quantitative"},
    "y": {"field": "Miles_per_Gallon", "type": "quantitative"}
  }
}

[domoritz]

I've looked into this and it's a bit more complicated than I thought. The issue is that auto parsing can only appear in the first source but this is a problem in named data sources where we generate a transform instead. The reason is that otherwise Vega changes the input data.

To demonstrate the issue, I created http://blockbuilder.org/domoritz/d359821c870c304153b1cf7764012071.

To resolve this, I am proposing to remove format.parse = auto from Vega-Lite 3. As an alternative, we could add a autoParseAll expression/transform to Vega that parses all input fields.

Thoughts @kanitw @jheer?

[domoritz]

The decision is to remove support for auto parsing from Vega-Lite. We will still use the static analysis to find out how to parse fields but not based on the actual data anymore. The reason for this is that the feature isn't used much anyway and it's tricky to get right together with the static analysis.