Elastalert: Rule type that alerts on average or sum of a field?

[anhlqn]

Hi,

I wonder if ElastAlert has a rule type that alerts on average or sum of a field. With IIS Logs in ES for example:
Alert if average time-taken within 5 minutes is higher than 1000 ms.
Alert if sum of sc-bytes within 5 minutes is larger than 1,000,000,000 Bytes.

I'm looking for something similar to average and sum aggregation on Kibana, and it would be great if ElastAlert has such rules.

Thanks,
Anh

[errordaiwa]

I have looked through ElastAlert config sheet and source code. It seems ElastAlert doesn't support aggregation query from ElasticSearch.

[mr2china]

I also have this requirement ,if have general solution for sum,count,average aggr on multi field of each document ....it will be powerful

[errordaiwa]

@mr2china I made a patch to support these features. The implement seems a little tricky since I'm not familiar with python.

[mr2china]

@errordaiwa ,I only implement by using rule extending way,like sum (divisor)/sum (dividend),which in () is multi-fields to do +,-,*,/ in each document.

[mr2china]

adding, we know es support script like groovy ,it's powerful ,so we have to implement by
python to simulate groovy or other script.

[abiruth84]

Hi!

I was actually wondering the same posted here.
I'm trying to set up a couple of alerts based on the average of 2 fields (requests_queued and time_taken) for a certain period of time and I am not sure how to do it.

@errordaiwa you mentioned a patch to support this but I am not sure yet how to implement it.
Thanks,
Ruth

[errordaiwa]

@abiruth84 add configuration in yaml like this

aggs:
  aggs_key: $key_name
  aggs_form:
    $key_name:
      avg:
        field: requests_queued

[abiruth84]

@errordaiwa sorry to ask again.
From the patch you linked, I just need to add these files that you modified to my current EA version?

• elastalert/elastalert.py
• elastalert/ruletypes.py
• elastalert/util.py

and then in my rule.yaml I add the "aggs" part? Is this "aggs" the rule "type"? do you maybe have some example documented on this change?
sorry but my knowledge with python and the insights of elastalert code is a bit limited at this point.
Thanks a lot for the help!

[errordaiwa]

@abiruth84 yes, but since the origin repository changes a lot after this patch, this way may not work well. Or you can just clone my repository and use the patch branch for testing.

The "aggs" part is not "rule type". It should be on the same level with "rule type". Just as blow

type: spike
aggs:
  aggs_key: $key_name
  aggs_form:
    $key_name:
      avg:
        field: requests_queued

PS. I just implement aggregation query on rule type "Spike" and "Frequency" due to the limited time.

[thomas-b-jackson]

Thanks for the contribution Xingyu. Would it make more sense to implement this in a custom rule rather than in a patch?

[Qmando]

How about open a pull request? :)

[IngaFeick]

+1

[ying1]

+1

[legigor]

+1

[shaharmor]

+1

[omlomloml]

can some one post an sampler rule file?

[etoykan]

+1

[xelnagamex]

+1

[Paraomao]

+1

[vmlinuz15]

+1

[fzyzcjy]

Hi several years have passed... Is there any updates?