Elastalert: ElastAlert & Elasticsearch Compatibility

[abhishekjiitr]

Does Elastalert's latest version (master) supports ES 6.X?

I have ElastAlert 0.1.38 with ES v6.3.1 with ES Python Client fixed to 6.3.1: works perfectly fine.
I wanted to update my ElastAlert to the latest master branch.

I tried the following options but encountered some problem in all cases with current ElastAlert master & Elasticsearch 6.3.1.

ES Client Version 6.3.1 & ES Client < 7 give same errors
Error Log:
```Elastic Version: 6.3.1
Reading Elastic 6 index mappings:
Reading index mapping 'es_mappings/6/silence.json'
Reading index mapping 'es_mappings/6/elastalert_status.json'
Reading index mapping 'es_mappings/6/elastalert.json'
Reading index mapping 'es_mappings/6/past_elastalert.json'
Reading index mapping 'es_mappings/6/elastalert_error.json'
Index elastalert-test already exists. Skipping index creation.
INFO:elastalert:Starting up
INFO:elastalert:Queried rule Dead Man Switch from 2019-07-15 09:51 UTC to 2019-07-15 10:06 UTC: 2049308 hits
WARNING:elasticsearch:GET https://elasticsearch-data:9200/elastalert-test_silence/_search?_source_include=until%2Cexponent&size=1 [status:404 request:0.033s]
ERROR:root:Error while querying for alert silence status: NotFoundError(404, u'index_not_found_exception', u'no such index')
ERROR:root:Traceback (most recent call last):
File "/usr/local/lib/python2.7/site-packages/elastalert-0.1.39-py2.7.egg/elastalert/elastalert.py", line 1180, in run_all_rules
num_matches = self.run_rule(rule, endtime, self.starttime)
File "/usr/local/lib/python2.7/site-packages/elastalert-0.1.39-py2.7.egg/elastalert/elastalert.py", line 897, in run_rule
if self.is_silenced(rule['name'] + "._silence") or self.is_silenced(silence_cache_key):
File "/usr/local/lib/python2.7/site-packages/elastalert-0.1.39-py2.7.egg/elastalert/elastalert.py", line 1807, in is_silenced
self.handle_error("Error while querying for alert silence status: %s" % (e), {'rule': rule_name})
File "/usr/local/lib/python2.7/site-packages/elastalert-0.1.39-py2.7.egg/elastalert/elastalert.py", line 1829, in handle_error
self.writeback('elastalert_error', body)
File "/usr/local/lib/python2.7/site-packages/elastalert-0.1.39-py2.7.egg/elastalert/elastalert.py", line 1509, in writeback
res = self.writeback_es.index(index=index, body=body)
File "/usr/local/lib/python2.7/site-packages/elasticsearch-6.3.1-py2.7.egg/elasticsearch/client/utils.py", line 76, in _wrapped
return func(args, params=params, *kwargs)
TypeError: index() takes at least 4 arguments (4 given)

ES Client >= 7 (v7.0.2) gives 

Elastic Version: 6.3.1
Reading Elastic 6 index mappings:
Reading index mapping 'es_mappings/6/silence.json'
Reading index mapping 'es_mappings/6/elastalert_status.json'
Reading index mapping 'es_mappings/6/elastalert.json'
Reading index mapping 'es_mappings/6/past_elastalert.json'
Reading index mapping 'es_mappings/6/elastalert_error.json'
Index elastalert-test already exists. Skipping index creation.
INFO:elastalert:Starting up
WARNING:elasticsearch:GET https://elasticsearch-data:9200/elastalert-test/_search?size=1000 [status:400 request:0.042s]
ERROR:root:Error finding recent pending alerts: RequestError(400, u'search_phase_execution_exception', u'No mapping found for [alert_time] in order to sort on') {'sort': {'alert_time': {'order': 'asc'}}, 'query': {'bool': {'filter': {'range': {'alert_time': {'to': '2019-07-15T11:56:06.064798Z', 'from': '2019-07-15T11:55:06.064779Z'}}}, 'must': {'query_string': {'query': '!_exists_:aggregate_id AND alert_sent:false'}}}}}
Traceback (most recent call last):
File "/usr/local/lib/python2.7/site-packages/elastalert-0.1.39-py2.7.egg/elastalert/elastalert.py", line 1537, in find_recent_pending_alerts
res = self.writeback_es.search(index=self.writeback_index, body=query, size=1000)
File "/usr/local/lib/python2.7/site-packages/elasticsearch-7.0.2-py2.7.egg/elasticsearch/client/utils.py", line 84, in _wrapped
return func(args, params=params, *kwargs)
File "/usr/local/lib/python2.7/site-packages/elasticsearch-7.0.2-py2.7.egg/elasticsearch/client/__init__.py", line 819, in search
"GET", _make_path(index, "_search"), params=params, body=body
File "/usr/local/lib/python2.7/site-packages/elasticsearch-7.0.2-py2.7.egg/elasticsearch/transport.py", line 353, in perform_request
timeout=timeout,
File "/usr/local/lib/python2.7/site-packages/elasticsearch-7.0.2-py2.7.egg/elasticsearch/connection/http_requests.py", line 155, in perform_request
self._raise_error(response.status_code, raw_data)
File "/usr/local/lib/python2.7/site-packages/elasticsearch-7.0.2-py2.7.egg/elasticsearch/connection/base.py", line 178, in _raise_error
status_code, error_message, additional_info
RequestError: RequestError(400, u'search_phase_execution_exception', u'No mapping found for [alert_time] in order to sort on')
INFO:elastalert:Disabled rules are: []
INFO:elastalert:Sleeping for 899.818719 seconds

`` This error seems harmless though as theelastalert_status` index is empty, and resolves on the next evaluation.

In general, how does the ElastAlert & Elasticsearch & Elasticsearch Client Compatibility work?

[Theoooooo]

For me, the latest Elasticsearch version is detected correctly when i start up elastalert. Maybe it's an issue related to the specific version of ElastAlert you are running ...

[Qmando]

Sorry this is working properly.. Thank you for trying and reporting issues. The master branch contains of bunch of compatibility changes for various version of Elasticsearch (separate code paths for es5, es6, es6.2, es6.6 😨 what a nightmare). I haven't extensively tested these. I'll make to sure to do so before releasing a new version. Might actually have time to do that this week.

[abhishekjiitr]

Closing this! Thanks @Qmando