Elasticsearch: ILM: Allow delete by space

[jguay]

Describe the feature:

If you have rollover at 50Gb, you can delete based on age from rollover in 7.4.2. However the requirement for deletion may depend on disk space usage
Example : need to delete oldest index when total size >= 20Tb

This would be greater value when https://github.com/elastic/elasticsearch/issues/44001 is added...
The aim would be index with names "team-a" with limit of 2Tb, and index "team-b" with limit of 500Gb. When this is reached the oldest index for matching pattern gets deleted automatically

Equivalent feature in curator : https://www.elastic.co/guide/en/elasticsearch/client/curator/current/filtertype_space.html

[elasticmachine]

Pinging @elastic/es-core-features (:Core/Features/ILM+SLM)

[fbaligand]

This feature would be great!
in delete phase, be able to specify delete index action in X “rollovers”, like we can specify delete index action in X “days”.
For example, if you specify to delete index “in 5 rollovers”, this will allow to remove index “index-00001” when index “index-00006” is created by rollover.

[reighnman]

This is a feature other ES backed offering have (such as Graylog) which makes it much easier to manage storage.

If I have rollover set to Xdocs/Xsize and a delete phase that only keeps the 10 latest indexes I can estimate disk consumption fairly accurately.

[fbaligand]

I agree with @reighnman: it's very useful to control disk usage and avoid full FS!

[allaboutopensource]

Describe the feature:

If you have rollover at 50Gb, you can delete based on age from rollover in 7.4.2. However the requirement for deletion may depend on disk space usage
Example : need to delete oldest index when total size >= 20Tb

This would be greater value when #44001 is added...
The aim would be index with names "team-a" with limit of 2Tb, and index "team-b" with limit of 500Gb. When this is reached the oldest index for matching pattern gets deleted automatically

Equivalent feature in curator : https://www.elastic.co/guide/en/elasticsearch/client/curator/current/filtertype_space.html

this feature is only to delete the indices not the actual data which is stored in ES nodes. AM i correct?

[fbaligand]

Well, when you delete an index, you delete the data inside the index.

[allaboutopensource]

so u mean to say i can delete the indices automatically using the "delete" phase of lifecycle and that will eventually delete the logs and free up the space in my ES nodes.

[fbaligand]

Yes, that is the aim of ILM delete phase.

[phlegx]

@reighnman is right. Is such a feature planned?

I'm still wondering how other people with the current ILM rollover features are able to estimate the disk space.

One quick workaround for now could be to check the disk space and if disk becomes too full to start a script that deletes all indices but last 10 or so.

[passing]

Not having this feature in ILM still forces us to use curator.
Luckily curator 5.7+ is still compatible with ES 7.x, but not sure if it will still be with ES 8