Elastalert: Can't start elastalert.service

Created on 23 Jun 2020 · 9Comments · Source: Yelp/elastalert

Hi all,

First time for me to request help on github...and actually beginner with Elastic Stack.

I launched this command to start elastalert :
/usr/bin/elastalert --verbose --config /opt/elastalert/etc/config.yaml

But get this error :
Traceback (most recent call last):
File "/usr/bin/elastalert", line 6, in
from pkg_resources import load_entry_point
File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3241, in
@_call_aside
File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3225, in _call_aside
f(args, *kwargs)
File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3254, in _initialize_master_working_set
working_set = WorkingSet._build_master()
File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 583, in _build_master
ws.require(__requires__)
File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 900, in require
needed = self.resolve(parse_requirements(requirements))
File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 786, in resolve
raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'elastalert==0.2.1' distribution was not found and is required by the application

Thank you for your help !!

Source

anhhuy

All 9 comments

Recent changes: As of Elastalert 0.2.0, you must use Python 3.6. Python 2 will not longer be supported.
https://github.com/Yelp/elastalert/blob/master/README.md#running-elastalert

nsano-rururu on 23 Jun 2020

👍1

I am having the same issue, and I am already running Python 3.6.

MakoWish on 25 Jun 2020

look
https://github.com/Yelp/elastalert/issues/2411

nsano-rururu on 25 Jun 2020

👍1

If the issue has already been resolved, please close the issue

nsano-rururu on 22 Jul 2020

@nsano-rururu The link you provided was not related to my issue. I downgraded the OS from Ubuntu 20.04 LTS back down to 18.04 LTS, and it starts now.

MakoWish on 22 Jul 2020

https://www.tecmint.com/install-python-in-ubuntu/

nsano-rururu on 22 Jul 2020

https://www.techiediaries.com/ubuntu/install-python-3-pip-venv-ubuntu-20-04-19/

nsano-rururu on 22 Jul 2020

I think you should consider using Docker.

| ElastAlert Server Docker image | tag| ElastAlert |Elasticsearch 7 Support| Remarks |
|:-------------------|:----:|:----------:|:----------:|:-----------|
|bitsensor/elastalert|2.0.1 |0.1.39|×||
|bitsensor/elastalert|lastet|0.1.39|×||
|bitsensor/elastalert|3.0.0-beta.0|0.2.0b2|〇||
|bitsensor/elastalert|3.0.0-beta.1|0.2.0b2|〇||
|servercentral/elastalert|latest|0.2.1|〇|bitsensor/elastalert fork
Customize|
|daichi703n/elastalert|0.2.1-dev2|0.2.1|〇|servercentral/elastalert fork
Customize|
|johnsusek/elastalert-server|latest|0.2.4|〇|servercentral/elastalert fork
Customize
Library Update
bug fix|

example

johnsusek/elastalert-server:latest
johnsusek/praeco:latest
Elasticsearch 7.8.0
Kibana 7.8.0

/home/sano/dkwork2/es
|--Dockerfiles
|  |--Dockerfile.elastalert
|--docker-compose.yml
|--es
|  |--config
|  |  |--elasticsearch.yml
|  |--data
|--kibana
|  |--config
|  |  |--kibana.yml
|--praeco
|  |--bin
|  |  |--elastalert-start.sh
|  |  |--elastic_search_status.sh
|  |--config
|  |  |--api.config.json
|  |  |--elastalert.yaml
|  |--nginx_config
|  |  |--default.conf
|  |  |--nginx.conf
|  |--public
|  |  |--favicon.ico
|  |  |--index.html
|  |  |--js
|  |  |  |--cron-ui.min.js
|  |  |--praeco.config.json
|  |--rule_templates
|  |--rules

Dockerfiles/Dockerfile.elastalert

FROM johnsusek/elastalert-server:latest

USER root

RUN apk add --update --no-cache net-snmp net-snmp-tools
RUN apk update && \
    apk add bash curl && \
    rm -rf /var/cache/apk/*

ADD praeco/bin/elastalert-start.sh /usr/local/bin/
ADD praeco/bin/elastic_search_status.sh /usr/local/bin/

RUN chmod +x /usr/local/bin/elastalert-start.sh 
RUN chmod +x /usr/local/bin/elastic_search_status.sh

USER node

ENTRYPOINT ["/usr/local/bin/elastalert-start.sh"]

docker-compose.yml

version: "3.7"
services:
  elasticsearch:
    container_name: elasticsearch
    image: docker.elastic.co/elasticsearch/elasticsearch:7.8.0
    ports:
      - 9200:9200
      - 9300:9300
    environment:
      - ES_JAVA_OPTS=-Xms256m -Xmx512m
      - discovery.type=single-node
    restart: always
    volumes:
      - ./es/data:/usr/share/elasticsearch/data
      - ./es/config/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml
    healthcheck:
        test: ["CMD-SHELL", "curl -f http://localhost:9200 || exit 1"]
        interval: 30s
        timeout: 15s
        retries: 3
        start_period: 180s

  kibana:
    container_name: kibana
    image: docker.elastic.co/kibana/kibana:7.8.0
    ports:
      - 5601:5601
    depends_on:
      - elasticsearch
    restart: always
    volumes:
      - ./kibana/config/kibana.yml:/usr/share/kibana/config/kibana.yml
    healthcheck:
        test: ["CMD-SHELL", "curl -f http://localhost:5601/api/status || exit 1"]
        interval: 30s
        timeout: 15s
        retries: 3
        start_period: 200s

  elastalert:
    container_name: elastalert
    build:
      context: .
      dockerfile: Dockerfiles/Dockerfile.elastalert
    image: elastalert-server:0.2.4
    ports:
      - 3030:3030
      - 3333:3333
    depends_on:
      - elasticsearch
    restart: always
    volumes:
      - ./praeco/config/elastalert.yaml:/opt/elastalert/config.yaml
      - ./praeco/config/api.config.json:/opt/elastalert-server/config/config.json
      - ./praeco/rules:/opt/elastalert/rules
      - ./praeco/rule_templates:/opt/elastalert/rule_templates
    healthcheck:
        test: ["CMD-SHELL", "curl -f http://localhost:3030 || exit 1"]
        interval: 30s
        timeout: 15s
        retries: 3
        start_period: 200s

  praeco:
    container_name: praeco
    image: johnsusek/praeco:latest
    ports:
      - 8080:8080
    depends_on:
      - elastalert
    restart: always
    volumes:
      - ./praeco/public/praeco.config.json:/var/www/html/praeco.config.json
      - ./praeco/nginx_config/nginx.conf:/etc/nginx/nginx.conf
      - ./praeco/nginx_config/default.conf:/etc/nginx/conf.d/default.conf
    healthcheck:
        test: ["CMD-SHELL", "curl -f http://localhost:8080 || exit 1"]
        interval: 30s
        timeout: 15s
        retries: 3
        start_period: 200s

es/config/elasticsearch.yml

cluster.name: "docker-cluster"
network.host: 0.0.0.0
discovery.zen.minimum_master_nodes: 1

kibana/config/kibana.yml

server.name: kibana
server.host: "0"
elasticsearch.hosts: http://elasticsearch:9200
xpack.monitoring.ui.container.elasticsearch.enabled: true

praeco/bin/elastalert-start.sh

#!/bin/bash

set -e

echo "Giving Elasticsearch at $ELASTICSEARCH_URL time to start..."

elastic_search_status.sh

echo "Starting ElastAlert!"
npm start

praeco/bin/elastic_search_status.sh

#!/bin/bash

set -e

if [ $# -gt 0 ]; then
  ES_URL="$1"
elif [[ -n $ELASTICSEARCH_URL ]]; then
  ES_URL="$ELASTICSEARCH_URL"
elif [[ -n $ES_HOST ]] && [[ -n $ES_PORT ]]; then
  ES_URL="http://$ES_HOST:$ES_PORT"
else
  ES_URL="http://elasticsearch:9200"
fi

until [[ "$(curl -fsSL "$ES_URL/_cat/health?h=status" | sed -r 's/^[[:space:]]+|[[:space:]]+$//g')" =~ ^(yellow|green)$ ]]; do
  # printf '+' >&2
  sleep 1
done

echo "Elasticsearch is up and healthy at "$ES_URL"" >&2

praeco/config/api.config.json

Default settings

{
  "appName": "elastalert-server",
  "port": 3030,
  "wsport": 3333,
  "elastalertPath": "/opt/elastalert",
  "verbose": false,
  "es_debug": false,
  "debug": false,
  "rulesPath": {
    "relative": true,
    "path": "/rules"
  },
  "templatesPath": {
    "relative": true,
    "path": "/rule_templates"
  },
  "es_host": "elasticsearch",
  "es_port": 9200,
  "es_username": "",
  "es_password": "",
  "es_ssl": false,
  "writeback_index": "praeco_elastalert_status"
}

praeco/config/elastalert.yaml

Default settings

# The elasticsearch hostname for metadata writeback
# Note that every rule can have its own elasticsearch host
es_host: elasticsearch

# The elasticsearch port
es_port: 9200

# This is the folder that contains the rule yaml files
# Any .yaml file will be loaded as a rule
rules_folder: rules

# How often ElastAlert will query elasticsearch
# The unit can be anything from weeks to seconds
run_every:
  seconds: 60

# ElastAlert will buffer results from the most recent
# period of time, in case some log sources are not in real time
buffer_time:
  minutes: 1

# Optional URL prefix for elasticsearch
#es_url_prefix: elasticsearch

# Connect with TLS to elasticsearch
#use_ssl: True

# Verify TLS certificates
#verify_certs: True

# GET request with body is the default option for Elasticsearch.
# If it fails for some reason, you can pass 'GET', 'POST' or 'source'.
# See http://elasticsearch-py.readthedocs.io/en/master/connection.html?highlight=send_get_body_as#transport
# for details
#es_send_get_body_as: GET

# Option basic-auth username and password for elasticsearch
#es_username: someusername
#es_password: somepassword

# The index on es_host which is used for metadata storage
# This can be a unmapped index, but it is recommended that you run
# elastalert-create-index to set a mapping
writeback_index: praeco_elastalert_status

# If an alert fails for some reason, ElastAlert will retry
# sending the alert until this time period has elapsed
alert_time_limit:
  days: 2

skip_invalid: True

profile: default

praeco/public/praeco.config.json

Default settings

{
  "appUrl": "http://127.0.0.1:8080",
  "errorLoggerUrl": "",
  "hidePreconfiguredFields": []
}

execute

chmod 777 es/data
chmod 777 praeco/rule_templates
chmod 777 praeco/rules
docker-compose up -d
docker logs -f elastalert

Giving Elasticsearch at  time to start...
curl: (7) Failed to connect to elasticsearch port 9200: Connection refused
curl: (7) Failed to connect to elasticsearch port 9200: Connection refused
curl: (7) Failed to connect to elasticsearch port 9200: Connection refused
curl: (7) Failed to connect to elasticsearch port 9200: Connection refused
curl: (7) Failed to connect to elasticsearch port 9200: Connection refused
curl: (7) Failed to connect to elasticsearch port 9200: Connection refused
curl: (7) Failed to connect to elasticsearch port 9200: Connection refused
curl: (7) Failed to connect to elasticsearch port 9200: Connection refused
curl: (7) Failed to connect to elasticsearch port 9200: Connection refused
curl: (7) Failed to connect to elasticsearch port 9200: Connection refused
curl: (7) Failed to connect to elasticsearch port 9200: Connection refused
curl: (7) Failed to connect to elasticsearch port 9200: Connection refused
curl: (7) Failed to connect to elasticsearch port 9200: Connection refused
curl: (7) Failed to connect to elasticsearch port 9200: Connection refused
curl: (7) Failed to connect to elasticsearch port 9200: Connection refused
curl: (7) Failed to connect to elasticsearch port 9200: Connection refused
curl: (7) Failed to connect to elasticsearch port 9200: Connection refused
curl: (7) Failed to connect to elasticsearch port 9200: Connection refused
curl: (7) Failed to connect to elasticsearch port 9200: Connection refused
curl: (7) Failed to connect to elasticsearch port 9200: Connection refused
curl: (7) Failed to connect to elasticsearch port 9200: Connection refused
curl: (7) Failed to connect to elasticsearch port 9200: Connection refused
curl: (7) Failed to connect to elasticsearch port 9200: Connection refused
curl: (7) Failed to connect to elasticsearch port 9200: Connection refused
curl: (7) Failed to connect to elasticsearch port 9200: Connection refused
curl: (7) Failed to connect to elasticsearch port 9200: Connection refused
curl: (7) Failed to connect to elasticsearch port 9200: Connection refused
curl: (7) Failed to connect to elasticsearch port 9200: Connection refused
curl: (7) Failed to connect to elasticsearch port 9200: Connection refused
curl: (7) Failed to connect to elasticsearch port 9200: Connection refused
curl: (7) Failed to connect to elasticsearch port 9200: Connection refused
curl: (7) Failed to connect to elasticsearch port 9200: Connection refused
curl: (7) Failed to connect to elasticsearch port 9200: Connection refused
curl: (7) Failed to connect to elasticsearch port 9200: Connection refused
curl: (7) Failed to connect to elasticsearch port 9200: Connection refused
curl: (7) Failed to connect to elasticsearch port 9200: Connection refused
curl: (7) Failed to connect to elasticsearch port 9200: Connection refused
curl: (7) Failed to connect to elasticsearch port 9200: Connection refused
curl: (7) Failed to connect to elasticsearch port 9200: Connection refused
curl: (7) Failed to connect to elasticsearch port 9200: Connection refused
curl: (7) Failed to connect to elasticsearch port 9200: Connection refused
curl: (7) Failed to connect to elasticsearch port 9200: Connection refused
Elasticsearch is up and healthy at http://elasticsearch:9200
Starting ElastAlert!

> @bitsensor/[email protected] start /opt/elastalert-server
> sh ./scripts/start.sh

14:23:35.503Z  INFO elastalert-server: Config:  No config.dev.json file was found in /opt/elastalert-server/config/config.dev.json.
14:23:35.514Z  INFO elastalert-server: Config:  Proceeding to look for normal config file.
14:23:35.515Z  INFO elastalert-server: Config:  A config file was found in /opt/elastalert-server/config/config.json. Using that config.
14:23:35.527Z  INFO elastalert-server: Router:  Listening for GET request on /.
14:23:35.529Z  INFO elastalert-server: Router:  Listening for GET request on /status.
14:23:35.529Z  INFO elastalert-server: Router:  Listening for GET request on /status/errors.
14:23:35.530Z  INFO elastalert-server: Router:  Listening for GET request on /rules.
14:23:35.533Z  INFO elastalert-server: Router:  Listening for GET request on /rules/:id*.
14:23:35.534Z  INFO elastalert-server: Router:  Listening for POST request on /rules/:id*.
14:23:35.535Z  INFO elastalert-server: Router:  Listening for DELETE request on /rules/:id*.
14:23:35.536Z  INFO elastalert-server: Router:  Listening for GET request on /templates.
14:23:35.537Z  INFO elastalert-server: Router:  Listening for GET request on /templates/:id*.
14:23:35.537Z  INFO elastalert-server: Router:  Listening for POST request on /templates/:id*.
14:23:35.538Z  INFO elastalert-server: Router:  Listening for DELETE request on /templates/:id*.
14:23:35.539Z  INFO elastalert-server: Router:  Listening for PUT request on /folders/:type/:path*.
14:23:35.540Z  INFO elastalert-server: Router:  Listening for DELETE request on /folders/:type/:path*.
14:23:35.540Z  INFO elastalert-server: Router:  Listening for POST request on /test.
14:23:35.541Z  INFO elastalert-server: Router:  Listening for POST request on /silence/:path*.
14:23:35.542Z  INFO elastalert-server: Router:  Listening for GET request on /config.
14:23:35.542Z  INFO elastalert-server: Router:  Listening for POST request on /config.
14:23:35.543Z  INFO elastalert-server: Router:  Listening for POST request on /download.
14:23:35.544Z  INFO elastalert-server: Router:  Listening for GET request on /metadata/elastalert.
14:23:35.544Z  INFO elastalert-server: Router:  Listening for GET request on /metadata/elastalert_status.
14:23:35.545Z  INFO elastalert-server: Router:  Listening for GET request on /metadata/silence.
14:23:35.546Z  INFO elastalert-server: Router:  Listening for GET request on /metadata/elastalert_error.
14:23:35.546Z  INFO elastalert-server: Router:  Listening for GET request on /metadata/past_elastalert.
14:23:35.549Z  INFO elastalert-server: Router:  Listening for GET request on /indices.
14:23:35.550Z  INFO elastalert-server: Router:  Listening for GET request on /mapping/:index.
14:23:35.550Z  INFO elastalert-server: Router:  Listening for POST request on /search/:index.
14:23:35.550Z  INFO elastalert-server: Router:  Listening for GET request on /config.
14:23:35.558Z  INFO elastalert-server: ProcessController:  Starting ElastAlert
14:23:35.559Z  INFO elastalert-server: ProcessController:  Creating index
14:23:40.599Z  INFO elastalert-server:
    ProcessController:  Elastic Version: 7.2.0
    Reading Elastic 6 index mappings:
    Reading index mapping 'es_mappings/6/silence.json'
    Reading index mapping 'es_mappings/6/elastalert_status.json'
    Reading index mapping 'es_mappings/6/elastalert.json'
    Reading index mapping 'es_mappings/6/past_elastalert.json'
    Reading index mapping 'es_mappings/6/elastalert_error.json'
    New index praeco_elastalert_status created
    Done!

14:23:40.599Z  INFO elastalert-server: ProcessController:  Index create exited with code 0
14:23:40.600Z  INFO elastalert-server: ProcessController:  Starting elastalert with arguments [none]
14:23:40.618Z  INFO elastalert-server: ProcessController:  Started Elastalert (PID: 260)
14:23:40.625Z  INFO elastalert-server: Server:  Server listening on port 3030
14:23:40.626Z  INFO elastalert-server: Server:  Websocket listening on port 3333
14:23:40.627Z  INFO elastalert-server: Server:  Server started
14:23:46.410Z  INFO elastalert-server: Routes:  Successfully handled GET request for '/'.
14:24:16.817Z  INFO elastalert-server: Routes:  Successfully handled GET request for '/'.
14:24:47.492Z  INFO elastalert-server: Routes:  Successfully handled GET request for '/'.
14:25:17.941Z  INFO elastalert-server: Routes:  Successfully handled GET request for '/'.
14:25:48.463Z  INFO elastalert-server: Routes:  Successfully handled GET request for '/'.
14:26:19.031Z  INFO elastalert-server: Routes:  Successfully handled GET request for '/'.
14:26:49.636Z  INFO elastalert-server: Routes:  Successfully handled GET request for '/'.
14:27:20.204Z  INFO elastalert-server: Routes:  Successfully handled GET request for '/'.
14:27:50.840Z  INFO elastalert-server: Routes:  Successfully handled GET request for '/'.
14:28:21.417Z  INFO elastalert-server: Routes:  Successfully handled GET request for '/'.
14:28:51.943Z  INFO elastalert-server: Routes:  Successfully handled GET request for '/'.
14:29:22.539Z  INFO elastalert-server: Routes:  Successfully handled GET request for '/'.

nsano-rururu on 22 Jul 2020