Helm-charts: [elasticsearch] Why is default number of primary shards 1?
Curious since according to google searches it is 5 by default but seems to be 1 in the helm chart?
Also, I ask because we have noticed a significant increase in performance correlated with the increase of shard count from 1 to 8 (which in our case makes our shards ~50GB each).
naseemkullah
All 4 comments
please read detailed description here https://github.com/elastic/elasticsearch/pull/30539
nvtkaszpir
on 21 Feb 2020
Thank you!
naseemkullah
on 21 Feb 2020
one more thing, more human-readable explanation is at https://www.elastic.co/blog/elasticsearch-7-0-0-released - at section:
```
Default to one shard
One of the biggest sources of troubles we've seen over the years from
our users has been oversharding --- and defaults play a big role in that.
In Elasticsearch 6.x and prior, we defaulted to 5 shards by default per index.
If you had one daily index for 10 different applications and each had the
default of 5 shards, you were creating 50 shards per day and it wasn't long
before you had thousands of shards even if you were only indexing a few
gigabytes of data per day. Index lifecycle management (ILM) was a first
step to help with this: providing native rollover functions to create
indexes by size instead of (just) by day and built-in shrink functionality
to shrink the number of shards per index. Defaulting indices to 1 shard
is the next step in helping to reduce oversharding. Of course, if you have
another preferred primary shard count, you can set it via the index settings.
```
nvtkaszpir
on 21 Feb 2020
Ah thanks for focusing in on that! Love a good tl;dr
naseemkullah
on 21 Feb 2020
Related issues
ronenmagid
路
8Comments
davidkarlsen
路
5Comments
guding
路
5Comments
krichter722
路
6Comments
diversario
路
7Comments
Most helpful comment
one more thing, more human-readable explanation is at https://www.elastic.co/blog/elasticsearch-7-0-0-released - at section:
```
Default to one shard
One of the biggest sources of troubles we've seen over the years from
our users has been oversharding --- and defaults play a big role in that.
In Elasticsearch 6.x and prior, we defaulted to 5 shards by default per index.
If you had one daily index for 10 different applications and each had the
default of 5 shards, you were creating 50 shards per day and it wasn't long
before you had thousands of shards even if you were only indexing a few
gigabytes of data per day. Index lifecycle management (ILM) was a first
step to help with this: providing native rollover functions to create
indexes by size instead of (just) by day and built-in shrink functionality
to shrink the number of shards per index. Defaulting indices to 1 shard
is the next step in helping to reduce oversharding. Of course, if you have
another preferred primary shard count, you can set it via the index settings.
```