Elasticsearch: ElasticSearch ignores indexing slowlog log level settings

[tdhopper]

I am trying to tune how my indexing slowlogs are collected. I know that I can set a threshold for how long an indexing op on a shard must take before it is logged. I would also like to be choose what level I'm logging at. It would appear that modifying this line in logging.yml would allow me to do this. However, unless I am misunderstanding the purpose of that setting, it is being ignored by ElasticSearch.

Here is how I came to that conclusion. I have ElasticSearch 1.3.2 running locally on my Mac. I put these config files in ~/es-config. This elasticsearch.yml file is bare-bones, as you can see. The logging.yml file only modifies this line of the default file. My assumption is that this should mean only WARN level indexing ops are logged.

However, if I start an ES instance using

elasticsearch --config="/Users/tdhopper/es-config/elasticsearch.yml"

and then run this Python script

import elasticsearch, time
es = elasticsearch.Elasticsearch(hosts="localhost")

while True:
    print ".",
    es.index(index="index1", doc_type="test_doc", body = {"hot_body": 1})
    time.sleep(.5)

my ~/es-logs/elasticsearch_index_indexing_slowlog.log file is immediately filled up with lines like

[2014-08-22 10:24:52,162][INFO ][index.indexing.slowlog.index] [War Machine] [index1][2] took[1.4ms], took_millis[1], type[test_doc], id[WH83A0yvRHaQtQ34_6wncg], routing[], source[{"hot_body":1}]
[2014-08-22 10:24:52,666][INFO ][index.indexing.slowlog.index] [War Machine] [index1][1] took[1.5ms], took_millis[1], type[test_doc], id[sErCAr3BR_qWVfp0pnayGw], routing[], source[{"hot_body":1}]

That is, INFO level log statements while I specified WARN as the logging level.

I know that my logging.yml file is being read, because I was able to change the name of the slowlog file with index_indexing_slow_log_file. However, nothing seems to change when index.indexing.slowlog is modified.

I brought this up on the ES mailing list, but no one replied.

[clintongormley]

Sorry it has taken a while to get to this. It does look like a bug to me, but I'm not entirely sure.

[clintongormley]

The setting to control the level of search logging is index.search.slowlog.level

This should be added to the slow log docs https://www.elastic.co/guide/en/elasticsearch/reference/5.0/index-modules-slowlog.html#index-slow-log. Also, these settings are per index and are no longer allowed in the config file. The docs should be updated to reflect this.

[gmoskovicz]

@clintongormley the default slowlog level is TRACE, but we can already control this by using -1 in the threshold. Perhaps we could just remove the level settings for the slowlog and only configure this with the threshold?

Nowadays one can control this with the level setting, or threshold with -1 or a specific value.

[jasontedor]

I opened #29040.