Elastalert: Exception during elastalert startup

[robaca]

We currently install elastalert into a Docker container using pip. Our newest builds now fail on startup with the following stack trace:

INFO:elastalert:Queried rule all errors from 2016-10-25 08:34 UTC to 2016-10-25 08:46 UTC: 0 hits
ERROR:root:Traceback (most recent call last):
  File "/opt/elastalert/elastalert/elastalert.py", line 732, in run_all_rules
    num_matches = self.run_rule(rule, endtime, self.starttime)
  File "/opt/elastalert/elastalert/elastalert.py", line 577, in run_rule
    self.writeback('elastalert_status', body)
  File "/opt/elastalert/elastalert/elastalert.py", line 998, in writeback
    doc_type=doc_type, body=body)
  File "build/bdist.linux-x86_64/egg/elasticsearch/client/utils.py", line 71, in _wrapped
    return func(*args, params=params, **kwargs)
TypeError: create() takes at least 5 arguments (5 given)

ERROR:root:Uncaught exception running rule all errors: create() takes at least 5 arguments (5 given)
Traceback (most recent call last):
  File "/usr/lib/python2.7/runpy.py", line 174, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/opt/elastalert/elastalert/elastalert.py", line 1336, in <module>
    sys.exit(main(sys.argv[1:]))
  File "/opt/elastalert/elastalert/elastalert.py", line 1333, in main
    client.start()
  File "/opt/elastalert/elastalert/elastalert.py", line 695, in start
    self.run_all_rules()
  File "/opt/elastalert/elastalert/elastalert.py", line 736, in run_all_rules
    self.handle_uncaught_exception(e, rule)
  File "/opt/elastalert/elastalert/elastalert.py", line 1229, in handle_uncaught_exception
    self.handle_error('Uncaught exception running rule %s: %s' % (rule['name'], exception), {'rule': rule['name']})
  File "/opt/elastalert/elastalert/elastalert.py", line 1224, in handle_error
    self.writeback('elastalert_error', body)
  File "/opt/elastalert/elastalert/elastalert.py", line 998, in writeback
    doc_type=doc_type, body=body)
  File "build/bdist.linux-x86_64/egg/elasticsearch/client/utils.py", line 71, in _wrapped
TypeError: create() takes at least 5 arguments (5 given)

[jverhoelen]

I experienced the same issue, still didn't find out what's the origin.

[rmj]

Same for me...

[PMDubuc]

This looks like the problem I had when trying to upgrade Python to 2.7.12. I reinstalled 2.7.11 and the problem went away.

[Qmando]

A minor python version broke the code? Whoa! :(

It looks like this bug is in the elasticsearch-python library maybe? It seems like the error itself it caused by some function wrapper they use.

The Elasticsearch.Create function doesn't appear any different. http://elasticsearch-py.readthedocs.io/en/master/api.html#elasticsearch.Elasticsearch.create

[pamelagnutzmann]

Same for me.

[varcheesy123]

same issue. I have also had no luck rolling back the version of both python and elastalert.

[mza97]

same for python 2.7 but working with python 2.6

[amosshapira]

I'm on Ubuntu 14.04 with Python 2.7.6 and get the same error.

Testing this inside a virtualenv and Python 2.7.12 from https://launchpad.net/~fkrull/+archive/ubuntu/deadsnakes-python2.7 (just mentioning this but I don't think it makes a difference for the result), I noticed that "pip list" shows elasticsearch version 5.0.0 even though the requirements.txt file pins the version to 1.3.0

"pip uninstall elasticsearch" failed with "No such file or directory" but after "pip install elasticsearch==1.3.0" the script startted working again.

So I suspect that issue might be a mis-alignment between the elastalert code and Elasticsearch Python version 5.0.0.

For now I seem to have found a temporary work-around.

[SimonPNorra]

Thanks @amosshapira

I have a customized version of the elastalert Dockerfile and I did run into the same issue. I changed my base image to
FROM iron/python:2.7.11

and added
RUN pip install elasticsearch==1.3.0
in the line before

# Install Elastalert.
RUN python setup.py install && \
    pip install -e .

Now the image runs again.

Greetings

[Qmando]

Looks like the create function's id parameter went from defaulting to None to being required in the ES 5.0 release:

https://github.com/elastic/elasticsearch-py/commit/a2cd22dbf6f355943fe6198bad849b18cbed380e

I'll add an explicit id=None to fix this.

In the meantime, unless you are running ES 5.X, you can fix this by running
ES2.X: pip install elasticsearch>=2.0.0,<3.0.0
ES1.X: pip install elasticsearch>=1.0.0,<2.0.0

[amosshapira]

Thanks!

[cecchisandrone]

In my case I solved just doing pip install elasticsearch==1.3.0. I'm using https://hub.docker.com/r/ivankrizsan/elastalert/ image

[amosshapira]

I've just verified today that the latest fix makes this unnecessary.