Elastalert: The simplest of queries returns with no [query] registered for [query]

[ghost]

I don't know what I'm doing wrong. This is my first sample rule copied from frequency example. This is the query:

filter:
- query:
    query_string:
      query: "level:INFO"

I can run this in Kibana against the same index I have and get lots of rows back. however using elastalert-test-rule rules\frequency.yaml I get the following error:

D:\app\elastalert>elastalert-test-rule rules\frequency.yaml
Successfully loaded SampleRule1

WARNING:elasticsearch:GET http://SAADCPHXVM04:9200/logs-*/_search?ignore_unavailable=true&size=1 [status:400 request:0.0
05s]
Error running your filter:
RequestError(400, u'parsing_exception', {u'status': 400, u'error': {u'line': 1, u'root_cause': [{u'reason': u'no [query]
 registered for [query]', u'type': u'parsing_exception', u'line': 1, u'col': 202}], u'type': u'parsing_exception', u'rea
son': u'no [query] registered for [query]', u'col': 202}})
INFO:elastalert:Note: In debug mode, alerts will be logged to console but NOT actually sent. To send them, use --verbose
.
WARNING:elasticsearch:GET http://SAADCPHXVM04:9200/logs-*/_search?_source_include=%40timestamp%2C%2A&ignore_unavailable=
true&scroll=30s&size=10000 [status:400 request:0.006s]
ERROR:root:Error running query: TransportError(400, u'parsing_exception', u'no [query] registered for [query]')

Would have written the following documents to writeback index (default is elastalert_status):

elastalert_error - {'message': "Error running query: TransportError(400, u'parsing_exception', u'no [query] registered f
or [query]')", 'traceback': ['Traceback (most recent call last):', '  File "d:\\app\\elastalert\\lib\\site-packages\\ela
stalert-0.1.11-py2.7.egg\\elastalert\\elastalert.py", line 311, in get_hits', "    res = self.current_es.search(scroll=s
croll_keepalive, index=index, size=rule['max_query_size'], body=query, ignore_unavailable=True, **extra_args)", '  File
"build\\bdist.win-amd64\\egg\\elasticsearch\\client\\utils.py", line 73, in _wrapped', '    return func(*args, params=pa
rams, **kwargs)', '  File "build\\bdist.win-amd64\\egg\\elasticsearch\\client\\__init__.py", line 625, in search', "
doc_type, '_search'), params=params, body=body)", '  File "build\\bdist.win-amd64\\egg\\elasticsearch\\transport.py", li
ne 318, in perform_request', '    status, headers, data = connection.perform_request(method, url, params, body, ignore=i
gnore, timeout=timeout)', '  File "build\\bdist.win-amd64\\egg\\elasticsearch\\connection\\http_requests.py", line 89, i
n perform_request', '    self._raise_error(response.status_code, raw_data)', '  File "build\\bdist.win-amd64\\egg\\elast
icsearch\\connection\\base.py", line 124, in _raise_error', '    raise HTTP_EXCEPTIONS.get(status_code, TransportError)(
status_code, error_message, additional_info)', "RequestError: TransportError(400, u'parsing_exception', u'no [query] reg
istered for [query]')"], 'data': {'query': {'sort': [{'@timestamp': {'order': 'asc'}}], 'query': {'bool': {'filter': {'b
ool': {'must': [{'range': {'@timestamp': {'gt': '2017-04-19T18:23:14.855Z', 'lte': '2017-04-19T18:38:14.855Z'}}}, {'quer
y': {'query_string': {'query': 'level:INFO'}}}]}}}}}, 'rule': 'SampleRule1'}}

One other thing i noticed was the range for timestamp. In the response coming back this is what i see:
{'range': {'@timestamp': {'gt': '2017-04-19T18:23:14.855Z', 'lte': '2017-04-19T18:38:14.855Z'}}
but in my rule i have:

timeframe:
  days: 10

It looks like that is also ignored
thanks
Jeff

[wirecutter313]

I'm seeing something similar after updating. Old rules that are known to work.

[gbarboza]

Also seeing this for previously working rules after upgrading to 0.1.11

Rules are of the form

filter:
- query:
    query_string:
      query: "field:value"

[Qmando]

Arg! I'm so sorry about this. Fixed now in master. I'll release a new version very soon.

Fixed in https://github.com/Yelp/elastalert/commit/95b3f0ad0bb9450987de09e45bb316a60f7a620b

[ghost]

This problem still continues even after upgrading to elastalert-0.1.12.dist
I just want to make sure it's nothing that I'm doing wrong.
BTW after every update to elastalert we have to go through a period of package finding since compatbilities break. This time the upgrade from 1.11 to 1.12 caused a few ripples on "botocore<1.6.0,>=1.5.0", requests==2.2.1, requests-oauthlib==0.5.0, and PyJWT>=1.4.2

The error from elastalert-test-rule:

D:\app\elastalert>elastalert-test-rule --config config.yaml rules\frequency.yaml
Successfully loaded SampleRule1

INFO:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): SAADCPHXVM04
INFO:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): SAADCPHXVM04
WARNING:elasticsearch:GET http://SAADCPHXVM04:9200/logs-*/_search?ignore_unavailable=true&size=1 [status:400 request:0.0
15s]
Error running your filter:
RequestError(400, u'parsing_exception', {u'status': 400, u'error': {u'line': 1, u'root_cause': [{u'reason': u'no [query]
 registered for [query]', u'type': u'parsing_exception', u'line': 1, u'col': 202}], u'type': u'parsing_exception', u'rea
son': u'no [query] registered for [query]', u'col': 202}})
INFO:elastalert:Note: In debug mode, alerts will be logged to console but NOT actually sent. To send them, use --verbose
.
INFO:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): SAADCPHXVM04
INFO:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): SAADCPHXVM04
INFO:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): SAADCPHXVM04
INFO:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): SAADCPHXVM04
WARNING:elasticsearch:GET http://SAADCPHXVM04:9200/logs-*/_search?_source_include=%40timestamp%2C%2A&ignore_unavailable=
true&scroll=30s&size=10000 [status:400 request:0.016s]
ERROR:root:Error running query: TransportError(400, u'parsing_exception', u'no [query] registered for [query]')

Would have written the following documents to writeback index (default is elastalert_status):

elastalert_error - {'message': "Error running query: TransportError(400, u'parsing_exception', u'no [query] registered f
or [query]')", 'traceback': ['Traceback (most recent call last):', '  File "c:\\programdata\\anaconda2\\lib\\site-packag
es\\elastalert-0.1.11-py2.7.egg\\elastalert\\elastalert.py", line 311, in get_hits', "    res = self.current_es.search(s
croll=scroll_keepalive, index=index, size=rule['max_query_size'], body=query, ignore_unavailable=True, **extra_args)", '
  File "d:\\app\\elastalert\\lib\\site-packages\\elasticsearch\\client\\utils.py", line 73, in _wrapped', '    return fu
nc(*args, params=params, **kwargs)', '  File "d:\\app\\elastalert\\lib\\site-packages\\elasticsearch\\client\\__init__.p
y", line 625, in search', "    doc_type, '_search'), params=params, body=body)", '  File "d:\\app\\elastalert\\lib\\site
-packages\\elasticsearch\\transport.py", line 318, in perform_request', '    status, headers, data = connection.perform_
request(method, url, params, body, ignore=ignore, timeout=timeout)', '  File "d:\\app\\elastalert\\lib\\site-packages\\e
lasticsearch\\connection\\http_requests.py", line 89, in perform_request', '    self._raise_error(response.status_code,
raw_data)', '  File "d:\\app\\elastalert\\lib\\site-packages\\elasticsearch\\connection\\base.py", line 124, in _raise_e
rror', '    raise HTTP_EXCEPTIONS.get(status_code, TransportError)(status_code, error_message, additional_info)', "Reque
stError: TransportError(400, u'parsing_exception', u'no [query] registered for [query]')"], 'data': {'query': {'sort': [
{'@timestamp': {'order': 'asc'}}], 'query': {'bool': {'filter': {'bool': {'must': [{'range': {'@timestamp': {'gt': '2017
-04-23T16:49:21.337Z', 'lte': '2017-04-23T17:04:21.337Z'}}}, {'query': {'query_string': {'query': 'level:INFO'}}}]}}}}},
 'rule': 'SampleRule1'}}

[Qmando]

You can also change your query to

filter:
   - query_string:
      query: "level:INFO"

[ghost]

That worked @Qmando . thanks!