Elastalert: Uncaught exception running rule Alert on App: Parser must be a string or character stream, not int

[adolia]

Hi there!
I got an issue when I'm trying to run rules with buffer_time > 1 minute:
Uncaught exception running rule Alert on App: Parser must be a string or character stream, not int
Traceback:

"Traceback (most recent call last):",
    "  File \"/usr/lib/python2.7/site-packages/elastalert/elastalert.py\", line 1115, in run_all_rules",
    "    num_matches = self.run_rule(rule, endtime, self.starttime)",
    "  File \"/usr/lib/python2.7/site-packages/elastalert/elastalert.py\", line 814, in run_rule",
    "    if not self.run_query(rule, rule[\u0027starttime\u0027], tmp_endtime):",
    "  File \"/usr/lib/python2.7/site-packages/elastalert/elastalert.py\", line 603, in run_query",
    "    data = self.get_hits(rule, start, end, index, scroll)",
    "  File \"/usr/lib/python2.7/site-packages/elastalert/elastalert.py\", line 417, in get_hits",
    "    hits = self.process_hits(rule, hits)",
    "  File \"/usr/lib/python2.7/site-packages/elastalert/elastalert.py\", line 334, in process_hits",
    "    set_es_key(hit[\u0027_source\u0027], rule[\u0027timestamp_field\u0027], rule[\u0027ts_to_dt\u0027](ts))",
    "  File \"/usr/lib/python2.7/site-packages/elastalert/util.py\", line 112, in ts_to_dt",
    "    dt = dateutil.parser.parse(timestamp)",
    "  File \"/usr/lib/python2.7/site-packages/dateutil/parser.py\", line 1182, in parse",
    "    return DEFAULTPARSER.parse(timestr, **kwargs)",
    "  File \"/usr/lib/python2.7/site-packages/dateutil/parser.py\", line 556, in parse",
    "    res, skipped_tokens = self._parse(timestr, **kwargs)",
    "  File \"/usr/lib/python2.7/site-packages/dateutil/parser.py\", line 675, in _parse",
    "    l = _timelex.split(timestr)         # Splits the timestr into tokens",
    "  File \"/usr/lib/python2.7/site-packages/dateutil/parser.py\", line 192, in split",
    "    return list(cls(s))",
    "  File \"/usr/lib/python2.7/site-packages/dateutil/parser.py\", line 61, in __init__",
    "    \u0027{itype}\u0027.format(itype=instream.__class__.__name__))",
    "TypeError: Parser must be a string or character stream, not int"

Using latest elastalert + elasticsearch 6.2.3
Any idea what is the issue?

Thanks in advance.

[Qmando]

Looks to me like the timestamp field is an integer and you're using the default (ISO8601) timestamp format. Though, I'm not sure how that relates to the buffer_time being different. Check the format of timestamp_field (default @timestamp)

Try either unix or unix_ms

timestamp_type: unix

[adolia]

Hi, Qmando!
Using of timestamp_type: unix_ms will not help. (It was my default setting for timestamp). The reason here, is that the latest Elasticsearch 6.x does not return any data on queries with timestamp in unix format. To fix it need to forcibly specify timestamp format in query. For example:
This one will not work:

{
  "query": {
    "bool": {
      "filter": {
        "bool": {
          "must": [
            {
              "range": {
                "timestamp": {
                  "gt": 1525942768000,
                  "lte": 1525942888000
                }
              }
            },
            {
              "term": {
                "type": "sensor"
              }
            }
          ]
        }
      }
    }
  }
}

This one will return docs:

{
  "query": {
    "bool": {
      "filter": {
        "bool": {
          "must": [
            {
              "range": {
                "timestamp": {
                  "gt": 1525942768000,
                  "lte": 1525942888000,
                  "format": "epoch_millis"
                }
              }
            },
            {
              "term": {
                "type": "sensor"
              }
            }
          ]
        }
      }
    }
  }
}

Also, with buffer_time: minutes: 1 it works correctly!

[Qmando]

Interesting. You need the "format": "epoch_millis" even if the field is mapped to epoch_millis?

Also, I'm still trying to figure out why buffer_time affects this. I'll have to set up ES6 using epoch_millis and test it soon.

[adolia]

Sorry for late reply.
Yes, you need to add "format": "epoch_millis" even if the field is mapped to epoch_millis. I think it's an Elasticsearch bug.
Regarding buffer_time, this issue only occurs when I've used "iso" format type. Maybe some calculations are bad.

[maniankara]

@Qmando or @adolia Do you guys have any updates on this? I stuck with this. Help appreciated !! Thanks

Edit: actually, setting timestamp_type: unix_ms gets me proceeding.

[adolia]

Hi, @maniankara
I've patched elastalert lib as a temporary workaround.
Waiting for a fix in elasticsearch.