Cartodb: Using another different quantification method than category makes torque fail

[AbelVM]

Context

Trying to style a Torque layer with TurboCarto using [value] fails randomly. The idea in the background was to use the tips & tricks at http://andrewxhill.com/blog/2015/04/17/torque-unknown/

Steps to Reproduce

1. Load a dataset of points and create a 1-layer map with it
2. Set the aggregation of the layer to Torque
3. Make a 1 frame, 0 time, 16px agg Torque layer in Builder, so the CartoCSS looks like the code below and the map is OK

Map {
  -torque-frame-count: 1;
  -torque-animation-duration: 0;
  -torque-time-attribute: "cartodb_id";
  -torque-aggregation-function: "count(1)";
  -torque-resolution: 16;
  -torque-data-aggregation: linear;
}
#layer {
  marker-width: 7;
  marker-fill: #FFB927;
  marker-fill-opacity: 0.9;
  marker-line-width: 0.5;
  marker-line-color: #FFF;
  marker-line-opacity: 1;
  comp-op: lighter;
}

1. Now, let's customize the CartoCSS so the Torque becomes a bubble map based on aggregation values

Map {
  -torque-frame-count: 1;
  -torque-animation-duration: 0;
  -torque-time-attribute: "cartodb_id";
  -torque-aggregation-function: "count(1)";
  -torque-resolution: 16;
  -torque-data-aggregation: linear;
}
#layer {
  marker-width: ramp([value], range(2, 8), jenks(6));
  marker-fill: #FFB927;
  marker-fill-opacity: 0.9;
  marker-line-width: 0.5;
  marker-line-color: #FFF;
  marker-line-opacity: 1;
  comp-op: lighter;
}

1. Change the basemap

Current Result

It works OK till you changes anything else in the dashboard. V.g.: change the basemap. Then, the map is rendered as expected :

But you get an error in the UI:

And no error message in Chrome console.

But if you force refresh the browser, you get the same message in the UI and the error below in the console:

In the case that your Torque layer is linked to an analysis node instead of a source node, the node stays in "running" mode forever. No matter the kind of analysis.

Expected result

Plain dashboard with no errors

Browser and version

No matter the browser

.carto file

No need of .carto file, any points dataset will reproduce this

Additional info

Same behavior in on-prem 2.0

cc @rochoa

[AbelVM]

If we also add UI to choose the aggregation used in Torque... I would be Sauber!

[andy-esch]

I'm seeing the same error when I do the following:

1. Import a dataset with a time column and a numeric column
2. Choose Torque for aggregation method
3. Choose the numeric column to 'style by', then Builder chooses a category quant method instead of a numeric one (quantiles, equal interval, etc.)
4. Change the quant method to 'quantiles' instead and get the following error: all_month_3: Failed to process "marker-fill" property: unable to compute ramp, column "value" does not exist

The CartoCSS looks like this:

Map {
  -torque-frame-count: 256;
  -torque-animation-duration: 30;
  -torque-time-attribute: "time";
  -torque-aggregation-function: "CDB_Math_Mode(value)";
  -torque-resolution: 4;
  -torque-data-aggregation: linear;
}
#layer {
  marker-width: 7;
  marker-fill: ramp([value], (#ffc6c4, #ee919b, #cc607d, #9e3963, #672044), quantiles, "=");
  marker-fill-opacity: 1;
  marker-line-width: 1;
  marker-line-color: #FFF;
  marker-line-opacity: 1;
  comp-op: lighter;
}
#layer[frame-offset=1] {
  marker-width: 9;
  marker-fill-opacity: 0.5;
}
#layer[frame-offset=2] {
  marker-width: 11;
  marker-fill-opacity: 0.25;
}

So it looks like the column name (here I'm using mag) isn't properly templated into the -torque-aggregation-function property, and the aggregation function is on the category-type (CDB_Math_Mode) instead of something like max or sum. It also looks like the "=" should not be there if it's on numeric ("<=").

[AbelVM]

Any news on this issue?

[jorgesancha]

@rafatower for awareness/triage

[rafatower]

As per rochoa, turbo carto does not support torque layers.

[jorgesancha]

Correct. This would require development. Some related info here: https://github.com/CartoDB/cartodb/issues/9389

[AbelVM]

Will it eventually support turbocarto?

Builder data-driven styling is generating turbocarto syntax in cartocss for Torque, so it crashes.

[AbelVM]

Right now, using plain Builder with a plain points dataset. No hacks involved, just Builder UI:

That selection in the UI generated the CartoCSS below, not edited by hand:

And, as you can see... crashed

[AbelVM]

Frontend related issue with the latest info: https://github.com/CartoDB/cartodb/issues/12328

[xavijam]

Since there are more people involved here, let me explain what's going on.

As @rafatower mentioned, turbo-carto is NOT supported in torque-layers, but it was a essential requirement for Builder release. At that point, we decided to "make" a hack for ~making~ faking it ~work~. We kept using a query wrap in order to calculate the categories -> code and we transform the turbo-carto attribute into old CSS conditionals. Example:

• This is what the user applies in the Builder:

Map {
  -torque-frame-count: 256;
  -torque-animation-duration: 30;
  -torque-time-attribute: "cartodb_id";
  -torque-aggregation-function: "CDB_Math_Mode(value)";
  -torque-resolution: 4;
  -torque-data-aggregation: linear;
}
#layer {
  marker-width: 7;
  marker-fill: ramp([value], (#5F4690, #1D6996, #38A6A5, #0F8554, #73AF48, #EDAD08, #E17C05, #CC503E, #94346E, #6F4070, #666666), (1, 2, 3, 4, 5, 6, 7, 8, 9, 10), "=");
  marker-fill-opacity: 1;
  marker-line-width: 1;
  marker-line-color: #FFF;
  marker-line-opacity: 1;
  comp-op: lighter;
}
#layer[frame-offset=1] {
  marker-width: 9;
  marker-fill-opacity: 0.5;
}
#layer[frame-offset=2] {
  marker-width: 11;
  marker-fill-opacity: 0.25;
}

• But this is the layer info we send to the Maps API looks like this:

{
            "id": "49a1892b-d43f-48ea-8d43-bab3bc980b6e",
            "type": "torque",
            "options": {
                "cartocss": "Map {\n  -torque-frame-count: 256;\n  -torque-animation-duration: 30;\n  -torque-time-attribute: \"cartodb_id\";\n  -torque-aggregation-function: \"CDB_Math_Mode(value)\";\n  -torque-resolution: 4;\n  -torque-data-aggregation: linear;\n}\n#layer {\n  marker-width: 7;\n  marker-fill: #666666;\n  [ value = 1 ] {\n    marker-fill: #5F4690;\n  }\n  [ value = 2 ] {\n    marker-fill: #1D6996;\n  }\n  [ value = 3 ] {\n    marker-fill: #38A6A5;\n  }\n  [ value = 4 ] {\n    marker-fill: #0F8554;\n  }\n  [ value = 5 ] {\n    marker-fill: #73AF48;\n  }\n  [ value = 6 ] {\n    marker-fill: #EDAD08;\n  }\n  [ value = 7 ] {\n    marker-fill: #E17C05;\n  }\n  [ value = 8 ] {\n    marker-fill: #CC503E;\n  }\n  [ value = 9 ] {\n    marker-fill: #94346E;\n  }\n  [ value = 10 ] {\n    marker-fill: #6F4070;\n  }\n  marker-fill-opacity: 1;\n  marker-line-width: 1;\n  marker-line-color: #FFF;\n  marker-line-opacity: 1;\n  comp-op: lighter;\n}\n#layer[frame-offset=1] {\n  marker-width: 9;\n  marker-fill-opacity: 0.5;\n}\n#layer[frame-offset=2] {\n  marker-width: 11;\n  marker-fill-opacity: 0.25;\n}",
                "cartocss_version": "2.1.0",
                "source": {
                    "id": "b0"
                },
                "sql_wrap": "select *, (CASE WHEN \"fid\" = 1 THEN 1 WHEN \"fid\" = 2 THEN 2 WHEN \"fid\" = 3 THEN 3 WHEN \"fid\" = 4 THEN 4 WHEN \"fid\" = 5 THEN 5 WHEN \"fid\" = 6 THEN 6 WHEN \"fid\" = 7 THEN 7 WHEN \"fid\" = 8 THEN 8 WHEN \"fid\" = 9 THEN 9 WHEN \"fid\" = 10 THEN 10  ELSE 11  END) as value FROM (<%= sql %>) __wrapped"
            }
        }

But, we didn't take into account the possibility to use color ramps. First, because it was not available before (in editor), and second because it needs to make a previous request to calculate the values for creating manually the ramp (:scream:). Are we sure we need this now? @noguerol @jorgesancha @kevin-reilly

I feel this is good info about how torque categories work for @CartoDB/frontend mates.

[xavijam]

Also, I'd like to point that if you apply a category torque, it works properly, but if you reload the map, it doesn't (points don't appear) because the stored CartoCSS is the turbo-carto one, not that one with conditionals :(.

[javisantana]

It's clear, we have to remove from the UI any option that is not supported by torque. If we want this we have to spend time adding proper turbocarto to aggregated datasets.

[noguerol]

+1 for removing it unless is easy to implement it right now, which I unfortunately understand is not the case.

[xavijam]

Also, I'd like to point that if you apply a category torque, it works properly, but if you reload the map, it doesn't (points don't appear) because the stored CartoCSS is the turbo-carto one, not that one with conditionals :(.

Found the problem why torque categories doesn't work when user reloads the map -> https://github.com/CartoDB/cartodb.js/pull/1698

[xavijam]

Removed the quantification option and fixed torque categories when map is reloaded.

[noguerol]

_Note for our future selves:_ we posponed the dev of the real thing, turbocarto on aggregations, in favor of this much easier solution, but we will have to tackle it sooner or later.