Elasticsearch: Should there be a global `max_concurrent_shard_requests` cluster setting?

[colings86]

Search requests have a query parameter called max_concurrent_shard_requests which limits the number of shards that will be searched concurrently. At the moment this can be set per request and if not set it defaults to min(256, nodeCount * default(index.number_of_shards)). default(index.number_of_shards) means the default value of the index.number_of_shards settings which itself defaults to 5.

I am wondering if it is a good idea to allow admins to change this default using a search.default.max_concurrent_shard_requests cluster setting so they can control this on across all requests knowing the behaviour of their cluster. We can make the default of this cluster setting the same as the current default for the query parameter.

I had initially considered this to be better as an index setting but then realised that if it is an index setting and a search is performed across multiple indices, the setting value might be different on each of the indices and then its ambiguous as to which value should be chosen so I think a cluster setting is appropriate here.

[colings86]

We discussed this and decided not to go ahead with it at this time as it doesn't help resource utilisation in the way I had thought.

[pavdmyt]

decided not to go ahead with it at this time as it doesn't help resource utilisation in the way I had thought

@colings86 can you please provide more details? I'm also interested why
max_concurrent_shard_requests can only be set on per-request basis, but not globally for entire cluster via e.g. _cluster/settings.