Elastalert: Store rules in ElasticSearch

[thinairwireless]

Is it possible to store the rule configurations in ES instead of relying on a file system rules_folder? In addition to having a rules folder, it would add much flexibility if we could add, delete, list(get) or disable(update) rules via ES APIs. ElastAlert could query ES instead of or in addition to rules_folder for changes to rule configurations.

[dforste]

I think this is a wonderful idea. Especially if there was some kind of kibana app for elastalert. I think the best experience would be elastalert runs in the background somewhere either storing its config in elasticsearch or locally. Maybe you give the option of local only or in the index.

If this is how things get done having some kind of kibana app to go with elastalert index configuration would be essential. Ideally users would be able to specify their own alerts in this framework and may not need access to the server or docker container where elastalert runs.

[diudiuzhi]

It's a good iead. We also need it!

[jagheterfredrik]

I've added this in my branch here if you want to play with it. Note that it's not tested whatsoever.
https://github.com/jagheterfredrik/elastalert/tree/rules_in_es

add to your config.yaml:
rules_in_es: true
es_metadata_index: .elastalert

To add a rule, convert it to JSON:
python -c 'import json,yaml,sys;print json.dumps(yaml.load(sys.stdin.read()))' < test_rule.yaml > test_rule.json
And insert it to elasticsearch:
curl -XPOST localhost:9200/.elastalert/rules/ -d@test_rule.json

[patrickdos]

Hello,

It would be awesome to get jagheterfredrik's work merged into the mainstream repo.
Any plans in that regards ?

Thanks

[davidecavestro]

It could be easily leveraged by external projects, like the Elastalert server/Kibana plugin from bitsensor (not tried yet).

[santosh-kore]

In 2020 Rules Loaders is the best approach i believe....