Vega-lite: Error: Undefined data set name: "scale_concat_1_child_main"

[cpennington]

I am trying to hconcat a faceted graph over two fields with a summary graph (summing across one of the fields). If I have color set in the first chart, then I get the error listed in the title (Undefined data set name: "scale_concat_1_child_main").

Broken code in Vega Editor

Working code in Vega Editor

{
    "$schema": "https://vega.github.io/schema/vega-lite/v3.json",
    "data": {
        "values": [
            {
                "c1": "Argagarg",
                "c2": "Argagarg",
                "mu": "2-8",
                "win_chance": 0.2,
                "p": 0.005293199778439004,
                "type": "global",
                "cum_p": 0.49384000294641295
            },
            {
                "c1": "Argagarg",
                "c2": "BBB",
                "mu": "2.5-7.5",
                "win_chance": 0.25,
                "p": 0.006317783956548739,
                "type": "global",
                "cum_p": 0.4182141361083573
            },
            {
                "c1": "BBB",
                "c2": "Argagarg",
                "mu": "4-6",
                "win_chance": 0.39999999999999997,
                "p": 0.00599613286508726,
                "type": "global",
                "cum_p": 0.5810908430205882
            },
            {
                "c1": "BBB",
                "c2": "BBB",
                "mu": "2.5-7.5",
                "win_chance": 0.25,
                "p": 0.017437015257325033,
                "type": "global",
                "cum_p": 0.5011573865688447
            }
        ]
    },
    "transform": [
        {
            "joinaggregate": [
                {
                    "op": "mean",
                    "field": "cum_p",
                    "as": "c1_avg_cum_p"
                }
            ],
            "groupby": [
                "c1"
            ]
        }
    ],
    "hconcat": [
        {
            "facet": {
                "row": {
                    "field": "c1",
                    "type": "ordinal"
                },
                "column": {
                    "field": "c2",
                    "type": "ordinal"
                }
            },
            "spec": {
                "mark": {
                    "type": "bar"
                },
                "encoding": {
                    "x": {
                        "field": "mu",
                        "type": "ordinal"
                    },
                    "y": {
                        "field": "p",
                        "type": "quantitative"
                    },
                    "color": { // If I remove this object, the graph works
                        "field": "cum_p",
                        "type": "quantitative"
                    }
                }
            }
        },
        {
            "facet": {
                "row": {
                    "field": "c1",
                    "type": "ordinal"
                }
            },
            "spec": {
                "mark": {
                    "type": "bar"
                },
                "encoding": {
                    "x": {
                        "field": "mu",
                        "type": "ordinal"
                    },
                    "y": {
                        "aggregate": "mean",
                        "field": "p",
                        "type": "quantitative"
                    },
                    "color": {
                        "aggregate": "mean",
                        "field": "c1_avg_cum_p",
                        "type": "quantitative"
                    }
                }
            }
        }
    ]
}

[cpennington]

I did some more digging, and notably, this is part of the delta between the compiled vega definitions:

@@ -401,7 +424,7 @@
     {
       "name": "concat_0_x",
       "type": "band",
-      "domain": {"data": "data_0", "field": "mu", "sort": true},
+      "domain": {"data": "data_2", "field": "mu", "sort": true},
       "range": {"step": {"signal": "concat_0_x_step"}},
       "paddingInner": 0.1,
       "paddingOuter": 0.05
@@ -409,15 +432,27 @@
     {
       "name": "concat_0_y",
       "type": "linear",
-      "domain": {"data": "data_0", "field": "p"},
+      "domain": {"data": "data_2", "fields": ["p_start", "p_end"]},
       "range": [{"signal": "concat_0_child_height"}, 0],
       "nice": true,
       "zero": true
     },
+    {
+      "name": "concat_0_color",
+      "type": "linear",
+      "domain": {"data": "data_2", "field": "cum_p"},
+      "range": "ramp",
+      "interpolate": "hcl",
+      "zero": false
+    },
     {
       "name": "concat_1_x",
       "type": "band",
-      "domain": {"data": "data_3", "field": "mu", "sort": true},
+      "domain": {
+        "data": "scale_concat_1_child_main",
+        "field": "mu",
+        "sort": true
+      },
       "range": {"step": {"signal": "concat_1_x_step"}},
       "paddingInner": 0.1,
       "paddingOuter": 0.05
@@ -425,7 +460,7 @@
     {
       "name": "concat_1_y",
       "type": "linear",
-      "domain": {"data": "data_3", "field": "mean_p"},
+      "domain": {"data": "scale_concat_1_child_main", "field": "mean_p"},
       "range": [{"signal": "concat_1_child_height"}, 0],
       "nice": true,
       "zero": true
@@ -433,7 +468,10 @@
     {
       "name": "concat_1_color",
       "type": "linear",
-      "domain": {"data": "data_3", "field": "mean_c1_avg_cum_p"},
+      "domain": {
+        "data": "scale_concat_1_child_main",
+        "field": "mean_c1_avg_cum_p"
+      },
       "range": "ramp",
       "interpolate": "hcl",
       "zero": false

(these changes are all in the scale key in the vega definition). However, I don't know enough about vega or vega-lite to understand why it's generating those data names, or why those data blocks aren't being included in the compiled vega.

[domoritz]

I could see two reasons for this issue. 1) we are not renaming the data field in assemble to be the correct reference or 2) we are accidentally deleting a required data source because the reference counter is not increased. It's probably 1.

[cpennington]

This also broke when I used a layer inside a facet

[parisholley]

still digging, but commenting out this allowed it to work for me:

https://github.com/vega/vega-lite/blob/026abda553ea980e36b97f3d4e751e1bc50f505f/src/compile/data/optimize.ts#L90-L95

[parisholley]

appears to be this mutator:

https://github.com/vega/vega-lite/blob/026abda553ea980e36b97f3d4e751e1bc50f505f/src/compile/data/optimize.ts#L57

without understanding, not sure where the appropriate place would be to trigger that the node isRequired()

[domoritz]

We do keep a reference count for outputs. If the count is 0, we remove it. We increase the reference counter every single time someone calls getSource. See https://github.com/vega/vega-lite/blob/026abda553ea980e36b97f3d4e751e1bc50f505f/src/compile/data/dataflow.ts#L179.

Thank you for looking into this. You already narrowed down where the issue is by a lot.

[0anton]

I got the same error with altair 3.2.0, when tried to _hconcat()_ _mark_bar()s_, which already had facets from _columns='...'_

pseudo code (not working):

a = alt.Chart(data1).mark_bar().encode(x='Year:O', y='Profile:Q', column='Pos:N')
b = alt.Chart(data2).mark_bar().encode(x='Year:O', y='Profile:Q', column='Pos:N')
a | b

Error: _"Javascript Error: Undefined data set name: "scale_concat_1_child_main""_

[domoritz]

Thanks for the additional example. Could you export the json with to_json()?

[Sh4pe]

I'm experiencing the same error. Here is my minimal example (no transform or color like in the issue).

Is anybody working on this issue? And is there a workaround? (The workaround not using facet but rather concat and filter is not feasible for me, since I have a large number of faceted values).

[trubens71]

I have a slightly more minimal example based on the one from @Sh4pe. What I have found is that the problem disappears if the two offending specs being concatenated are not consecutive. In this example I interject a 3rd spec in the middle which does not use the row or column channel.

update: even more trivially and a better workaround is to interject the simplest spec I can imagine...

    {
      "mark": "text"
    },

HTH

[domoritz]

Thank you @trubens71 for the small example. It'll make it easier for me to debug what's going on.

[RobinL]

As a short term fix, I note that if you add a:

"transform": [
                {
                }
            ],

to each part of the vconcat (e.g. filters that have no effect), then it works.

Note in some examples the transforms seem to have to be _different _ in each part of the vconcat

here is an example

[joelostblom]

I ran into this today and have been troubleshooting for a bit. I noticed that it only happens for concatenating charts using the same data source. When passing charts with different data sources, there are no issues. This is also why the filter workaround works, it creates two separate data sources in the compiled Vega spec.

The reason it does not work with the same data source is that VegaLite for some reason passes a non-existing dataset name to Vega instead of reusing the name of the dataset used in the first concatenated chart. Here is an example of the compiled Vega from @trubens71 's example above:

    {
      "name": "concat_0_y",
      "type": "linear",
      "domain": {"data": "data_2", "field": "a"},
      "range": [{"signal": "concat_0_child_height"}, 0],
      "nice": true,
      "zero": true
    },
    {
      "name": "concat_1_x",
      "type": "linear",
      "domain": {"data": "scale_concat_1_child_main", "field": "x"},
      "range": [0, {"signal": "concat_1_child_width"}],
      "nice": true,
      "zero": false,
      "padding": 5
    },

The second entry is referencing a dataset name that doesn't exist and changing this to data_2 as in the first concatenated chart will fix the issue.

I would PR, but I am not sure where in VL this incorrect name is created and couldn't figure it out from looking briefly at the source, so happy for anyone else to do it, I just want to be able to concatenate faceted charts =)

Additional workarounds to those mentioned is to reorder the dataset before passing it to VegaLite (e.g. in Altair) so that it is not recognized as the same dataset, or using a boxplot (which I think doesn't have this issue because it creates separate aggregated datasets).

I believe a PR for this would also close #5475 and solve half of https://github.com/vega/vega-lite/issues/5261.

[mahadi]

Is there anybody working on this?

[domoritz]

Is there anybody working on this?

Not right now, no.