Elastalert: How to include total hits in alert_text

Created on 15 Jul 2016 · 10Comments · Source: Yelp/elastalert

Can anyone help on how to include the number hits in the alert text?
Thanks in advance

Source

OElesin

Most helpful comment

Unfortunately you can't add it right now. But, I think adding this line should do the trick.

diff --git a/elastalert/elastalert.py b/elastalert/elastalert.py
index e532e76..c684810 100644
--- a/elastalert/elastalert.py
+++ b/elastalert/elastalert.py
@@ -562,6 +562,7 @@ class ElastAlerter():
         num_matches = len(rule['type'].matches)
         while rule['type'].matches:
             match = rule['type'].matches.pop(0)
+            match['num_hits'] = self.num_hits

Then it should be in the default alert text, or in a custom alert text:

alert_text: "Number of hits: {0}"
alert_text_args: ["num_hits"]

To be specific, this number is the number of documents returned in the most recent query to Elasticsearch. A match might actually contain documents from multiple queries, like if you have a frequency type with a big timeframe. It's the same number reported in the log line Queried rule ... from 2016.. to 2016..: X hits, Y matches, Z alerts sent.

Qmando on 21 Jul 2016

👍3

All 10 comments

Unfortunately you can't add it right now. But, I think adding this line should do the trick.

diff --git a/elastalert/elastalert.py b/elastalert/elastalert.py
index e532e76..c684810 100644
--- a/elastalert/elastalert.py
+++ b/elastalert/elastalert.py
@@ -562,6 +562,7 @@ class ElastAlerter():
         num_matches = len(rule['type'].matches)
         while rule['type'].matches:
             match = rule['type'].matches.pop(0)
+            match['num_hits'] = self.num_hits

Then it should be in the default alert text, or in a custom alert text:

alert_text: "Number of hits: {0}"
alert_text_args: ["num_hits"]

Qmando on 21 Jul 2016

👍3

Been pulling my hair out wondering how to do this. Thanks for the patch!

chock on 21 Jul 2016

@Qmando , I have a question about your last statement. For example:
606 query hits, 6 matches, 5 alerts sent
for a rule that's run every 5 minutes looking for 50 events in a 30 second time frame. I get:
"Number of hits: 606" in each of the 5 alerts. Does this mean that 606 is the total number of documents in the 6 matches and only 5 alerts were sent because one of the matches matched less than 50?

Also, I would like to print out all three values in the alert_text. If that's possible, what are the arg names for matches and alerts?

Thanks

PMDubuc on 22 Jul 2016

What might be happening is that as the 606 hits are being processed, the first 50 might trigger one match, the next 50 might trigger another match, etc. It's taking the 30 second timeframe into account for every set of 50 messages, so the number of matches may differ. It could also change based on query_key. Matches may then be dropped because of realert or a couple other reasons.

You could add

match['num_matches'] = num_matches

to add this to the alert as well.

You can't really get the number of alerts sent because it only calculates that number AFTER trying to send them all :)

I'll try to get this into a real feature, or just make that patch the default.

Qmando on 22 Jul 2016

👍1

Could you include this patch as the default? It's working well for me. I need both num_matches and num_hits. Thanks!

PMDubuc on 28 Oct 2016

Done

Qmando on 12 Jan 2017

👍1

Thanks very much!

PMDubuc on 13 Jan 2017

@Qmando Just to make things complicated, how would one remove the num_hits and num_matches as the default? Took a look at elastalert.py but unsure where to make the change.

If there's even just a way to do this on a per-rule basis, that would work as well.