Elasticsearch: Rollup job creation fails when using comma delimitated indices/index patterns

[jen-huang]

I have two indices, kibana_sample_data_logs and kibana_sample_data_flights. Both of these indices have a date field timestamp.

Attempting to create a rollup job for both of these indices using comma delimitated pattern fails with different errors depending on if wildcards are used.

We should document this limitation more clearly by specifying that only a single index or index pattern is allowed. (Or enhance the feature to allow comma separated list 🙂, but I'm not sure if that enhancement is urgent? When @silne30 and I ran into this, it wasn't under real use case scenarios.)

No wildcards

PUT _rollup/job/test2
{
  "id": "test2",
  "index_pattern": "kibana_sample_data_logs,kibana_sample_data_flights",
  "rollup_index": "rollup_samples",
  "cron": "0 * * * * ?",
  "page_size": 1000,
  "groups": {
    "date_histogram": {
      "interval": "2h",
      "delay": "1d",
      "time_zone": "UTC",
      "field": "timestamp"
    }
  }
}

fails with

[index_not_found_exception] no such index [kibana_sample_data_logs,kibana_sample_data_flights], with { resource.type="index_or_alias" & resource.id="kibana_sample_data_logs,kibana_sample_data_flights" & index_uuid="_na_" & index="kibana_sample_data_logs,kibana_sample_data_flights" }

With wildcards

PUT _rollup/job/test2
{
  "id": "test2",
  "index_pattern": "*data_logs,*data_flights",
  "rollup_index": "rollup_samples",
  "cron": "0 * * * * ?",
  "page_size": 1000,
  "groups": {
    "date_histogram": {
      "interval": "2h",
      "delay": "1d",
      "time_zone": "UTC",
      "field": "@timestamp"
    }
  }
}

fails with

[rollup_action_request_validation_exception] Validation Failed: 1: Could not find any fields in the index/index-pattern that were configured in job;

[elasticmachine]

Pinging @elastic/es-analytics-geo

[cuff-links]

The UI tells you that the index patterns with the comma is valid. `

[polyfractal]

Hmm, I think this is a bug. Not looking at the code right now, but the intention was to allow all index patterns (wildcards, comma-delimited, etc). We use the same index resolve that e.g. search uses, so I'm surprised it's failing.

I think it should work/be supported as long as the fields are in both indices... will take a closer look in a bit.

[elasticmachine]

Pinging @elastic/es-ui (:UI)