Elasticsearch: ILM should support placeholders in rollover_alias

[JohnLyman]

Similar (but simpler than) #36627, one should be able to use {index} placeholder while specifying a index.lifecycle.rollover_alias.

[JohnLyman]

It would also be very useful to have a new {write_index} placeholder that would simply drop the -\d{6} suffix from the index name.

[elasticmachine]

Pinging @elastic/es-core-features

[gwbrown]

Hi, thank you for your interest in Elasticsearch, and in ILM in particular!

Could you give an example of how this would be used?

My first interpretation of your comments is that it's something you would use in an index template, like:

PUT _template/datastream_template
{
  "index_patterns": ["managed-*"],                 
  "settings": {
    "number_of_shards": 1,
    "number_of_replicas": 1,
    "index.lifecycle.name": "my_policy",      
    "index.lifecycle.rollover_alias": "{write_index}-alias"    
  }
}

So that if you created managed-logs-000001, then index.lifecycle.rollover-alias on that index would be set to managed-logs-alias. Is that correct? Or am I misinterpreting you?

[JohnLyman]

Yes, that's exactly what I was thinking. It would allow for a more generic index template for ILM-related indices, instead of (for example) having to define new templates each time a new version of beats is released in our environment.

PUT _template/filebeat
{
  "index_patterns": ["filebeat-*", "metricbeat-*"],                 
  "settings": {
    "index.lifecycle.name": "my_policy",      
    "index.lifecycle.rollover_alias": "{write_index}-alias"    
  }
}

vs.

PUT _template/filebeat-6.5.3
{
  "index_patterns": ["filebeat-6.5.3-*"],                 
  "settings": {
    "index.lifecycle.name": "my_policy",      
    "index.lifecycle.rollover_alias": "filebeat-6.5.3-alias"    
  }
}
PUT _template/filebeat-6.6.0
{
  "index_patterns": ["filebeat-6.6.0-*"],                 
  "settings": {
    "index.lifecycle.name": "my_policy",      
    "index.lifecycle.rollover_alias": "filebeat-6.6.0-alias"    
  }
}
PUT _template/metricbeat-6.5.3
{ ... }
PUT _template/metricbeat-6.6.0
{ ... }

[magnuslarsen]

These would be a fantastic feature to have ({index} and {write_index}). Upvotes from here.

[Battleroid]

Can't stress this enough but this is exactly something we could use. Right now we have many many indices that are dynamically generated. It's not feasible to know ahead of time their names to create the policy for each and every single one just to drop them into a different rollover alias via ILM.

Something that would let us wildcard it and dynamically set the alias would be ideal like @JohnLyman has demonstrated.

[jsanchezs]

Any updates on this?

[dakrone]

Closing this as with the introduction of data streams the management of ILM aliases is not as necessary.