Elasticsearch: Add loadbalancing between data location

Created on 28 Mar 2017  路  2Comments  路  Source: elastic/elasticsearch

Hi,
I would like to suggest a new feature
where E.S will change location automatically when a data point is up
for e.g
if i put in the configuration
path.data = ["/mnt/first", "/mnt/second"...., "/mnt/storagen"]
path.limit = 90

Now E.S will store data in /mnt/first and when it reach 90% of that mounted disk ti will automatically balance the storage to the next one...

I'll be glad to see this functionality on your service.
Thanks

Most helpful comment

In Elasticsearch, we stripe across the path.data at the shard level (an entire shard will sit on a single disk). This is better than using one path at a time as it parallelizes use of the disks. Additionally, by the proposal here, using a disk until it is nearly filled means that we run dangerously close to not being able to add more data to a shard without it being moved to another disk. I do not think that we should add a feature to only use one disk at a time until it is a threshold is crossed.

All 2 comments

In Elasticsearch, we stripe across the path.data at the shard level (an entire shard will sit on a single disk). This is better than using one path at a time as it parallelizes use of the disks. Additionally, by the proposal here, using a disk until it is nearly filled means that we run dangerously close to not being able to add more data to a shard without it being moved to another disk. I do not think that we should add a feature to only use one disk at a time until it is a threshold is crossed.

@jasontedor thanks , for your explication.

Was this page helpful?
0 / 5 - 0 ratings