Elastalert: new_style_string_format not working anymore

[hirso]

Hello,

Sorry if I don't respect standards for issues reporting, first one here.

Since 4e8271d9215b9db5cb49e4182985392a099ea7ba (commit which was working), the "new_style_string_format" is not working.

My alert command is the following. It sends messages from auditbeat logs. It was working well before, it is not working anymore.
I came back to this commit (4e8271d9215b9db5cb49e4182985392a099ea7ba) and it's working again.

alert:
- "command"
command: ["/var/lib/elastalert/commands/send_message.sh", "A sudo was executed on {match[beat][name]} : {match[auditd][data][cmd]}"]
new_style_string_format: true

Not working : A sudo was executed on {match[beat][name]} : {match[auditd][data][cmd]}
Working : A sudo was executed on vm-log-es01 : /usr/bin/python -m elastalert.elastalert --verbose --config /var/lib/elastalert/config.yaml --rule /var/lib/elastalert/custom-rules/audit-sudo.yaml

[Qmando]

The format changed slightly. You need to remove match from the string formatters. Now you can just use {beat[name]} and {auditd[data][cmd]}

I'll add backwards compatibility since I shouldn't be making breaking changes in a minor upgrade, my mistake. Sorry about that.

[hirso]

Hello,
Thank you for your fast answer, it's working perfectly.

[jesferman]

Hello

I'm using version version 0.1.32 and even using the new format you answered in this issue, it doesn't work for me. Here is my example:

new_style_string_format: true       
command: ["send_message.sh", "send_msg", "Server {beat[name]} has high RAM utilization: {beat[system][memory][actual][used][pct]}"]

It does not translate to the actual values.

[Qmando]

Try using {beat.name} and beat.system.memory.actual.used.pct

[jesferman]

Does not work either

[Qmando]

Are those fields even available in your match? If you use the debug alerter, with no custom alert text settings, it will show you all the fields available. You haven't posted the rest of your rule so I have no idea if the issue is related to the string formatter or not.

[jesferman]

Here is the complete rule:

name: "RAM utilization"
index: metricbeat-*
type: frequency
query_key: beat.name
num_events: 1
timeframe:
  minutes: 1
filter:
 - range:
     system.memory.actual.used.pct:
       from: 0.9
       to: 1.1
alert:
  - command
realert:
  minutes: 1
new_style_string_format: true       
command: ["send_message.sh", "send_msg", "Server {beat[name]} has high RAM utilization: {beat[system][memory][actual][used][pct]}"]
pipe_match_json: true

And these are the available fields:

Available terms in first hit:
    beat.version
    beat.name
    beat.hostname
    @timestamp
    metricset.rtt
    metricset.name
    metricset.module
    system.memory.total
    system.memory.used.bytes
    system.memory.used.pct
    system.memory.actual.used.bytes
    system.memory.actual.used.pct
    system.memory.actual.free
    system.memory.swap.total
    system.memory.swap.used.bytes
    system.memory.swap.used.pct
    system.memory.swap.free
    system.memory.free

The same rule was working (with the previous syntax of {match[beat][name]}) before updating to latest version of elastalert.

Thanks for your support.

[Qmando]

This is indeed a bug. I'll fix asap.

In the mean time, using %(beat.name)s should work

[Qmando]

1776 will fix this

[jesferman]

Nice, thank you! 😄

[jesferman]

Hi,

I just updated to version 0.1.33 but it is not working yet. I am using the same example posted above and tried both {match[beat][name]} and {beat[name]} but it doesn't expand to the actual values. Am I missing something?

Thanks.

[Qmando]

Did you try {beat.name}?

[jesferman]

Yes, I also tried it

[Qmando]

Ok. You'll have to just keep using %(beat.name)s.

If the data were

{ "beat": { "name": "something"} }

Then you could use {beat[name]}

However, it seems to be

{ "beat.name": "something" }

For which, you need to use the % version.

My apologies for not realizing this earlier. I've got some ideas to make this more foolproof in the future.

[jesferman]

Ok, I will use the % version.

Thanks for your support.

[Poodrdt]

How about pipe_alert_text? How can I use it it command?
"{[alert_text]}"
"{match[alert_text]}"
"{alert_text}"
does not work.

[Qmando]

Unfortunately the alert text can't be formatted directly into the command. You have to do a slight workaround using some scripting to convert the stdin to a command line arg.

In alert.sh, add

#!/bin/bash
alert_text=`cat`
some_command "$alert_text"

Then have the command just run alert.sh with pipe_alert_text: true

[Poodrdt]

Thx man. I wish this was in the docs 👍