Elasticsearch: 'bucket_script' not supported as a top-level aggregation?

[davmrtl]

Hi,

Why is the 'bucket_script' not supported as a top-level aggregation?

I have to produce a top-level ratio of sums... The following code gives me an error:

GET /sales-2015.11.*/sale/_search
{
  "query": {
    "match_all": {}
  },
  "aggs": {
    "per_day": {
      "date_histogram": {
        "field": "@timestamp",
        "interval": "day"
      },
      "aggs": {
        "sum_of_sales": {
          "sum": {
            "field": "sales"
          }
        },
        "sum_of_vendors_count": {
          "sum": {
            "field": "vendors_count"
          }
        },
        "ratio_of_sales_per_vendors_count": {
          "bucket_script": {
            "buckets_path": {
              "sum_of_sales": "sum_of_sales",
              "sum_of_vendors_count": "sum_of_vendors_count"
            },
            "script": "sum_of_sales / sum_of_vendors_count"
          }
        }
      }
    },
    "sum_of_all_sales": {
      "sum": {
        "field": "sales"
      }
    },
    "sum_of_all_vendors_count": {
      "sum": {
        "field": "vendors_count"
      }
    },
    "ratio_of_all_sales_per_all_vendors_count": {
      "bucket_script": {
        "buckets_path": {
          "sum_of_all_sales": "sum_of_all_sales",
          "sum_of_all_vendors_count": "sum_of_all_vendors_count"
        },
        "script": "sum_of_all_sales / sum_of_all_vendors_count"
      }
    }
  },
  "size": 0
}

It gives me the following error:

{
   "error": {
      "root_cause": [
         {
            "type": "aggregation_execution_exception",
            "reason": "Invalid pipeline aggregation named [ratio_of_all_sales_per_all_vendors_count] of type [bucket_script]. Only sibling pipeline aggregations are allowed at the top level"
         }
      ],
      "type": "search_phase_execution_exception",
      "reason": "all shards failed",
      "phase": "query",
      "grouped": true,
      "failed_shards": [
         {
            "shard": 0,
            "index": "sales-2015.11.01",
            "node": "N3_ZEgs-RAGPOssZrCcL_w",
            "reason": {
               "type": "aggregation_execution_exception",
               "reason": "Invalid pipeline aggregation named [ratio_of_all_sales_per_all_vendors_count] of type [bucket_script]. Only sibling pipeline aggregations are allowed at the top level"
            }
         }
      ]
   },
   "status": 500
}

How can I get it working?

Thanks!

[clintongormley]

@colings86 Could you take a look at this please?

[colings86]

@davmrtl This is by design. The bucket_script aggregation is a Parent Pipeline Aggregation and therefore needs to be inside a multi-bucket aggregation to run since it expects to be given buckets from its parent aggregation and add its results to those buckets. For now, if you want to run arbitrary calculations using top-level aggregation results you will need to do this in your client side application.

[claudio-viola]

HI!
Is there any other way to implement this functionality not on the client_side?
Are we forced to use scripted metrics https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-metrics-scripted-metric-aggregation.html which will basically perform the aggregation and do the ratio?

Will this ever be implemented?
Thank you!

[ebuildy]

@claudio-viola did you try to use https://www.elastic.co/guide/en/elasticsearch/reference/2.3/search-aggregations-bucket-global-aggregation.html?

[anhzhi]

i get a very ugly solution like:

 {
   "size": 0,
   "aggs": {
      "result": {
         "terms": {"script":"'a big trick'"},
         "aggs": {
            "Total计数": {
                "value_count": {
                    "field": "_index"
                }
            },
            "TCP": {
                "filter": {
                    "term": {
                        "_type": "mprobe_tcp"
                    }
                },
                "aggs": {
                    "计数2": {
                        "value_count": {
                            "field": "_index"
                        }
                    }
                }
            },
            "百分比": {
                "bucket_script": {
                    "buckets_path": {
                        "TCP计数2": "TCP>计数2",
                        "Total计数": "Total计数"
                    },
                    "script": "TCP计数2/Total计数"
                }
            }
        }
      }
   }
}

i wish that can help

[colings86]

Closing in favour of https://github.com/elastic/elasticsearch/issues/14600

[ikazoy]

Thanks @anhzhi , you saved my day!

Just one thing to point out, as of Elasticsearch 7.3 we need params to refer a parameter in script. Otherwise, I got an error painless variable [xxx] is not defined.

            "百分比": {
                "bucket_script": {
                    "buckets_path": {
                        "TCP计数2": "TCP>计数2",
                        "Total计数": "Total计数"
                    },
                    "script": "params.TCP计数2 / params.Total计数"
                }
            }