Elastalert: Logging to the file

[vennca]

Hello,
can anyone help me with setting logging to file /var/log/elastalert/elastalert.log?

I have this configuration but it doesn't work:

/opt/elastalert/config.yaml

run_every:
seconds: 30
buffer_time:
minutes: 5
rules_folder: rules
use_ssl: true
es_host: *
es_port: 9200
es_username: elastalert
es_password: ***
es_send_get_body_as: GET
writeback_index: elastalert_audit

logging:
  version: 1
  incremental: false
  disable_existing_loggers: false
  formatters:
    logline:
      format: '%(asctime)s %(levelname)+8s %(name)+20s %(message)s'

    handlers:
      console:
        class: logging.StreamHandler
        formatter: logline
        level: DEBUG
        stream: ext://sys.stderr

      file:
        class : logging.FileHandler
        formatter: logline
        level: DEBUG
        filename: /var/log/elastalert/elastalert.log

    loggers:
      elastalert:
        level: DEBUG
        handlers: []
        propagate: true

      elasticsearch:
        level: DEBUG
        handlers: []
        propagate: true

      elasticsearch.trace:
        level: DEBUG
        handlers: []
        propagate: true

      '':  # root logger
        level: DEBUG
        handlers:
          - console
          - file
        propagate: false

[root@alerter-as1 elastalert]# systemctl status elastalert
● elastalert.service - Elastalert
   Loaded: loaded (/etc/systemd/system/elastalert.service; enabled; vendor preset: disabled)
   Active: active (running) since St 2020-08-26 09:39:47 CEST; 13min ago
 Main PID: 26064 (python3.6)
   CGroup: /system.slice/elastalert.service
           └─26064 /usr/bin/python3.6 -m elastalert.elastalert --config /opt/elastalert/config.yaml

[nsano-rururu]

It's a setting I've never seen, but is it the setting written in the document?

[vennca]

I found out configuration in the file config.yaml.example:
https://github.com/Yelp/elastalert/blob/master/config.yaml.example

[nsano-rururu]

Is this helpful?
Unable to output elastalert.log #2510

[vennca]

Unfortunately no, because indentation is correct.

[vennca]

Does anyone have a working solution for putting logging into operation?

[vennca]

All I need was how to run the logging for elastalert but thank you @nsano-rururu for detailed tutorial how to set praeco.

[nsano-rururu]

We also recommend asking questions on the Gitter channel
https://gitter.im/Yelp/elastalert

[rehannali]

Simple setup your elastalert with supervisor and mention these settings there to log.
Logging is part of deamon process not elastalert it self.

[wajika]

I have the same problem. Why is the logging configuration set but no logging is generated.

[nsano-rururu]

@vennca

I was able to output the file.

# This is the folder that contains the rule yaml files
# Any .yaml file will be loaded as a rule
rules_folder: rules

# How often ElastAlert will query Elasticsearch
# The unit can be anything from weeks to seconds
run_every:
  minutes: 1

# ElastAlert will buffer results from the most recent
# period of time, in case some log sources are not in real time
buffer_time:
  minutes: 15

# The Elasticsearch hostname for metadata writeback
# Note that every rule can have its own Elasticsearch host
es_host: localhost

# The Elasticsearch port
es_port: 9200

# The AWS region to use. Set this when using AWS-managed elasticsearch
#aws_region: us-east-1

# The AWS profile to use. Use this if you are using an aws-cli profile.
# See http://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-started.html
# for details
#profile: test
profile: default

# Optional URL prefix for Elasticsearch
#es_url_prefix: elasticsearch

# Connect with TLS to Elasticsearch
#use_ssl: True

# Verify TLS certificates
#verify_certs: True

# GET request with body is the default option for Elasticsearch.
# If it fails for some reason, you can pass 'GET', 'POST' or 'source'.
# See http://elasticsearch-py.readthedocs.io/en/master/connection.html?highlight=send_get_body_as#transport
# for details
#es_send_get_body_as: GET

# Option basic-auth username and password for Elasticsearch
#es_username: someusername
#es_password: somepassword

# Use SSL authentication with client certificates client_cert must be
# a pem file containing both cert and key for client
#verify_certs: True
#ca_certs: /path/to/cacert.pem
#client_cert: /path/to/client_cert.pem
#client_key: /path/to/client_key.key

# The index on es_host which is used for metadata storage
# This can be a unmapped index, but it is recommended that you run
# elastalert-create-index to set a mapping
writeback_index: elastalert_status
writeback_alias: elastalert_alerts

# If an alert fails for some reason, ElastAlert will retry
# sending the alert until this time period has elapsed
alert_time_limit:
  days: 2

# Custom logging configuration
# If you want to setup your own logging configuration to log into
# files as well or to Logstash and/or modify log levels, use
# the configuration below and adjust to your needs.
# Note: if you run ElastAlert with --verbose/--debug, the log level of
# the "elastalert" logger is changed to INFO, if not already INFO/DEBUG.
logging:
  version: 1
  incremental: false
  disable_existing_loggers: false
  formatters:
    logline:
      format: '%(asctime)s %(levelname)+8s %(name)+20s %(message)s'

  handlers:
    console:
      class: logging.StreamHandler
      formatter: logline
      level: INFO
      stream: ext://sys.stderr

    file:
      class : logging.FileHandler
      formatter: logline
      level: INFO
      filename: elastalert.log

  loggers:
    elastalert:
      level: INFO
      handlers: []
      propagate: true

    elasticsearch:
      level: INFO
      handlers: []
      propagate: true

    elasticsearch.trace:
      level: INFO
      handlers: []
      propagate: true

    '':  # root logger
      level: INFO
      handlers:
        - console
        - file
      propagate: false

[nsano-rururu]

It worked when I put handlers and loggers in the same position as formatters.

Reference URL
https://gist.github.com/eht16/b2183e1ce8373b8916b72dfa985a20c6
https://qiita.com/SatoshiTerasaki/items/42c80dd9a7da0e8155bd
https://qiita.com/petitviolet/items/86a3cc74d35fd9f6c96c

[nsano-rururu]

I confirmed that a pull request has been issued
https://github.com/Yelp/elastalert/pull/2231

[wajika]

@nsano-rururu Thank you. It worked

[vennca]

Thank you. It works!