Kapacitor: Kapacitor not recording data stream and alerts not visible in kapacitor log

Created on 12 Dec 2016  路  24Comments  路  Source: influxdata/kapacitor

Kapacitor not recording data stream and alerts not visible in kapacitor log

I have a Fedora24(CentOS) vm in windows10 host.I followed the article here.

Following are the details
I tried with the exact same version of Influxdb,kapacitor and telegraf as was mentioned in the article but it seemed like many changes were made in kapacitor 1.0 beta later like
the old task.db database file since it is still needed to do the migration.
[storage]
boltdb = "/var/lib/kapacitor/kapacitor.db"

as I am receiving error exactly at this point.

1.First I tried with
Software Versions:

InfluxDB - 0.13.0
Telegraf - 0.13.1
Kapacitor - 0.13.1

Later on I uninstalled Kapacitor -0.13.1 and installed 1.1.1

Here are the configuration files

1.influxdb.conf

```
logging-enabled = true
pprof-enabled = false

[admin]
enabled = true
bind-address = ":8083"
https-enabled = false
https-certificate = "/etc/ssl/influxdb.pem"

[http]
enabled = true
bind-address = ":8086"
auth-enabled = false
log-enabled = true
write-tracing = false
logging-enabled = true
pprof-enabled = false

[admin]
enabled = true
bind-address = ":8083"
https-enabled = false
https-certificate = "/etc/ssl/influxdb.pem"

[http]
enabled = true
bind-address = ":8086"
auth-enabled = false
log-enabled = true
write-tracing = false
pprof-enabled = false
https-enabled = false
https-certificate = "/etc/ssl/influxdb.pem"
max-row-limit = 10000

[subscriber]
enabled = true
http-timeout = "30s"

[[udp]]
enabled = false

[continuous_queries]
log-enabled = true
enabled = true

Other details are omitted as of now

2.telegraf.conf

[agent]
interval = "1s"

[outputs]

[outputs.influxdb]
urls = ["http://localhost:8086"]
database = "kapacitor_example"
user_agent = "telegraf"

[cpu]
percpu = false
totalcpu = true
drop = ["cpu_time"]

3.Kapacitor.conf

hostname = "localhost"

data_dir = "/var/lib/kapacitor"

default-retention-policy = ""

[http]
bind-address = ":9092"
auth-enabled = false
log-enabled = true
write-tracing = false
pprof-enabled = false
https-enabled = false

hostname = "localhost"

data_dir = "/var/lib/kapacitor"

default-retention-policy = ""

[http]
bind-address = ":9092"
auth-enabled = false
log-enabled = true
write-tracing = false
pprof-enabled = false
https-enabled = false
https-certificate = "/etc/ssl/kapacitor.pem"

[config-override]
enabled = true

[logging]
file = "/var/log/kapacitor/kapacitor.log"
level = "INFO"

[replay]
dir = "/var/lib/kapacitor/replay"

[[influxdb]]
default = true
name = "localhost"
urls = ["http://localhost:8086"]
username = ""
password = ""
ssl-ca = ""
ssl-cert = ""
ssl-key = ""
insecure-skip-verify = false
timeout = "0"
disable-subscriptions = false
udp-buffer = 1000
udp-read-buffer = 0
startup-timeout = "0"

startup-timeout = "5m"
disable-subscriptions = false
subscription-protocol = "http"

4.cpu_alert.tick

stream
|from()
.measurement('cpu')

|alert()
            .crit(lambda: "usage_idle" < 90 )
    .log('/tmp/cpu_idle_alerts.log')
I followed  [https://docs.influxdata.com/kapacitor/v0.12/introduction/getting_started/](url)  for installation.

### Steps to reproduce

kapacitor define cpu_alert -type stream -tick ~/cpu_alert.tick -dbrp kapacitor_example.default

$ kapacitor show cpu_alert
ID: cpu_alert
Error:
Template:
Type: stream
Status: enabled
Executing: true
Created: 10 Dec 16 13:44 PST
Modified: 11 Dec 16 09:39 PST
LastEnabled: 11 Dec 16 09:39 PST
Databases Retention Policies: ["kapacitor_example"."default"]
TICKscript:
stream
|from()
.measurement('cpu')
|alert()
.crit(lambda: "usage_idle" < 90)
.log('/tmp/cpu_idle_alerts.log')

DOT:
digraph cpu_alert {
graph [throughput="0.00 points/s"];

stream0 [avg_exec_time_ns="0s" ];
stream0 -> from1 [processed="0"];

from1 [avg_exec_time_ns="0s" ];
from1 -> alert2 [processed="0"];

alert2 [alerts_triggered="0" avg_exec_time_ns="0s" crits_triggered="0" infos_triggered="0" oks_triggered="0" warns_triggered="0" ];

[ticku@localhost ~]$ kapacitor record stream -task cpu_alert -duration 20s -no-wait
922d81ca-9375-413c-9ffb-7c8fdd68a360
[ticku@localhost ~]$ kapacitor list recordings 922d81ca-9375-413c-9ffb-7c8fdd68a360
ID Type Status Size Date
922d81ca-9375-413c-9ffb-7c8fdd68a360 stream running 0 B 11 Dec 16 09:46 PST

[ticku@localhost ~]$ sudo cat /var/log/kapacitor/kapacitor.log
[run] 2016/12/10 18:16:39 I! Kapacitor starting, version 1.1.1, branch master, commit d1e0173df2b9f44c64b7bbf1a120a1bdb3ef683e
[run] 2016/12/10 18:16:39 I! Go version go1.7.4
[srv] 2016/12/10 18:16:39 I! Kapacitor hostname: localhost
[srv] 2016/12/10 18:16:39 I! ClusterID: e364dd76-166e-4e1c-bbe5-06c1c24aac1e ServerID: e859317a-0ea9-4272-a0bd-3b8ce8eeec03
[task_master:main] 2016/12/10 18:16:39 I! opened
[task_store] 2016/12/10 18:16:39 W! could not open old boltd for task_store. Not performing migration. Remove the task_store.dir configuration to disable migration.
[task_master:main] 2016/12/10 18:16:39 I! Started task: cpu_alert
[smtp] 2016/12/10 18:16:39 I! Starting SMTP service
[stats] 2016/12/10 18:16:39 I! opened service
[httpd] 2016/12/10 18:16:39 I! Starting HTTP service
[httpd] 2016/12/10 18:16:39 I! Authentication enabled: false
[run] 2016/12/10 18:16:39 I! Listening for signals
[httpd] ::1 - - [10/Dec/2016:18:17:21 -0800] "GET /kapacitor/v1/debug/vars HTTP/1.1" 200 1851 "-" "KapacitorClient" f264449f-bf47-11e6-8001-000000000000 4124
[httpd] ::1 - - [10/Dec/2016:18:17:55 -0800] "GET /kapacitor/v1/tasks/cpu_alert?dot-view=attributes&replay-id=&script-format=formatted HTTP/1.1" 200 612 "-" "KapacitorClient" 06b41a2c-bf48-11e6-8002-000000000000 3782
[httpd] ::1 - - [10/Dec/2016:18:17:55 -0800] "PATCH /kapacitor/v1/tasks/cpu_alert HTTP/1.1" 200 614 "-" "KapacitorClient" 06b4d137-bf48-11e6-8003-000000000000 17560
[task_master:main] 2016/12/10 18:17:55 I! Stopped task: cpu_alert
[httpd] ::1 - - [10/Dec/2016:18:17:55 -0800] "PATCH /kapacitor/v1/tasks/cpu_alert HTTP/1.1" 200 440 "-" "KapacitorClient" 06b79c77-bf48-11e6-8004-000000000000 12599
[task_master:main] 2016/12/10 18:17:55 I! Started task: cpu_alert
[httpd] ::1 - - [10/Dec/2016:18:17:55 -0800] "PATCH /kapacitor/v1/tasks/cpu_alert HTTP/1.1" 200 600 "-" "KapacitorClient" 06ba122a-bf48-11e6-8005-000000000000 23309
[httpd] ::1 - - [10/Dec/2016:18:18:03 -0800] "GET /kapacitor/v1/tasks?dot-view=attributes&fields=type&fields=status&fields=executing&fields=dbrps&limit=100&offset=0&pattern=cpu%2A&replay-id=&script-format=formatted HTTP/1.1" 200 201 "-" "KapacitorClient" 0bc55423-bf48-11e6-8006-000000000000 2564
[httpd] ::1 - - [10/Dec/2016:18:18:04 -0800] "GET /kapacitor/v1/tasks/cpu_alert?dot-view=attributes&replay-id=&script-format=formatted HTTP/1.1" 200 600 "-" "KapacitorClient" 0c497bab-bf48-11e6-8007-000000000000 25805
[httpd] ::1 - - [10/Dec/2016:18:18:21 -0800] "GET /kapacitor/v1/tasks?dot-view=attributes&fields=link&limit=100&offset=0&pattern=cpu_alert&replay-id=&script-format=formatted HTTP/1.1" 200 124 "-" "KapacitorClient" 166f8ef7-bf48-11e6-8008-000000000000 1401
[httpd] ::1 - - [10/Dec/2016:18:18:21 -0800] "PATCH /kapacitor/v1/tasks/cpu_alert HTTP/1.1" 200 609 "-" "KapacitorClient" 16700e1c-bf48-11e6-8009-000000000000 12661
[httpd] ::1 - - [10/Dec/2016:18:20:07 -0800] "GET /kapacitor/v1/tasks/cpu_alert?dot-view=attributes&replay-id=&script-format=formatted HTTP/1.1" 200 609 "-" "KapacitorClient" 554f65cc-bf48-11e6-800a-000000000000 14409
[run] 2016/12/10 18:26:14 I! Signal received, initializing clean shutdown...
[run] 2016/12/10 18:26:14 I! Waiting for clean shutdown...
[task_master:main] 2016/12/10 18:26:14 I! Stopped task: cpu_alert
[smtp] 2016/12/10 18:26:14 I! Closing SMTP service
[stats] 2016/12/10 18:26:14 I! closed service
[httpd] 2016/12/10 18:26:14 I! Closed HTTP service
[task_master:main] 2016/12/10 18:26:14 I! closed
[run] 2016/12/10 18:26:14 I! server shutdown completed
[run] 2016/12/10 18:26:17 I! Kapacitor starting, version 1.1.1, branch master, commit d1e0173df2b9f44c64b7bbf1a120a1bdb3ef683e
[run] 2016/12/10 18:26:17 I! Go version go1.7.4
[srv] 2016/12/10 18:26:17 I! Kapacitor hostname: localhost
[srv] 2016/12/10 18:26:17 I! ClusterID: e364dd76-166e-4e1c-bbe5-06c1c24aac1e ServerID: e859317a-0ea9-4272-a0bd-3b8ce8eeec03
[task_master:main] 2016/12/10 18:26:17 I! opened
[task_store] 2016/12/10 18:26:17 W! could not open old boltd for task_store. Not performing migration. Remove the task_store.dir configuration to disable migration.
[task_master:main] 2016/12/10 18:26:18 I! Started task: cpu_alert
[smtp] 2016/12/10 18:26:18 I! Starting SMTP service
[stats] 2016/12/10 18:26:18 I! opened service
[httpd] 2016/12/10 18:26:18 I! Starting HTTP service
[httpd] 2016/12/10 18:26:18 I! Authentication enabled: false
[run] 2016/12/10 18:26:18 I! Listening for signals
[httpd] ::1 - - [10/Dec/2016:18:26:35 -0800] "GET /kapacitor/v1/tasks/cpu_alert?dot-view=attributes&replay-id=&script-format=formatted HTTP/1.1" 200 609 "-" "KapacitorClient" 3d139b0e-bf49-11e6-8001-000000000000 19523
[httpd] ::1 - - [10/Dec/2016:18:26:35 -0800] "PATCH /kapacitor/v1/tasks/cpu_alert HTTP/1.1" 200 612 "-" "KapacitorClient" 3d1714ff-bf49-11e6-8002-000000000000 25929
[task_master:main] 2016/12/10 18:26:36 I! Stopped task: cpu_alert
[httpd] ::1 - - [10/Dec/2016:18:26:36 -0800] "PATCH /kapacitor/v1/tasks/cpu_alert HTTP/1.1" 200 438 "-" "KapacitorClient" 3d1b490d-bf49-11e6-8003-000000000000 20700
[task_master:main] 2016/12/10 18:26:36 I! Started task: cpu_alert
[httpd] ::1 - - [10/Dec/2016:18:26:36 -0800] "PATCH /kapacitor/v1/tasks/cpu_alert HTTP/1.1" 200 599 "-" "KapacitorClient" 3d1e8e28-bf49-11e6-8004-000000000000 25508
[httpd] ::1 - - [10/Dec/2016:18:26:49 -0800] "GET /kapacitor/v1/tasks?dot-view=attributes&fields=link&limit=100&offset=0&pattern=cpu_alert&replay-id=&script-format=formatted HTTP/1.1" 200 124 "-" "KapacitorClient" 45614b9a-bf49-11e6-8005-000000000000 7634
[httpd] ::1 - - [10/Dec/2016:18:26:49 -0800] "PATCH /kapacitor/v1/tasks/cpu_alert HTTP/1.1" 200 610 "-" "KapacitorClient" 45629fad-bf49-11e6-8006-000000000000 20701
[httpd] ::1 - - [10/Dec/2016:18:27:04 -0800] "GET /kapacitor/v1/tasks/cpu_alert?dot-view=attributes&replay-id=&script-format=formatted HTTP/1.1" 200 610 "-" "KapacitorClient" 4e24e582-bf49-11e6-8007-000000000000 16534
[run] 2016/12/10 18:31:11 I! Signal received, initializing clean shutdown...
[run] 2016/12/10 18:31:11 I! Waiting for clean shutdown...
[task_master:main] 2016/12/10 18:31:11 I! Stopped task: cpu_alert
[smtp] 2016/12/10 18:31:11 I! Closing SMTP service
[stats] 2016/12/10 18:31:11 I! closed service
[httpd] 2016/12/10 18:31:11 I! Closed HTTP service
[task_master:main] 2016/12/10 18:31:11 I! closed
[run] 2016/12/10 18:31:11 I! server shutdown completed
[run] 2016/12/10 18:32:01 I! Kapacitor starting, version 1.1.1, branch master, commit d1e0173df2b9f44c64b7bbf1a120a1bdb3ef683e
[run] 2016/12/10 18:32:01 I! Go version go1.7.4
[srv] 2016/12/10 18:32:01 I! Kapacitor hostname: localhost
[srv] 2016/12/10 18:32:01 I! ClusterID: e364dd76-166e-4e1c-bbe5-06c1c24aac1e ServerID: e859317a-0ea9-4272-a0bd-3b8ce8eeec03
[task_master:main] 2016/12/10 18:32:01 I! opened
[task_store] 2016/12/10 18:32:01 W! could not open old boltd for task_store. Not performing migration. Remove the task_store.dir configuration to disable migration.
[task_master:main] 2016/12/10 18:32:01 I! Started task: cpu_alert
[smtp] 2016/12/10 18:32:01 I! Starting SMTP service
[stats] 2016/12/10 18:32:01 I! opened service
[httpd] 2016/12/10 18:32:01 I! Starting HTTP service
[httpd] 2016/12/10 18:32:01 I! Authentication enabled: false
[run] 2016/12/10 18:32:01 I! Listening for signals
[httpd] ::1 - - [10/Dec/2016:18:32:14 -0800] "GET /kapacitor/v1/tasks/cpu_alert?dot-view=attributes&replay-id=&script-format=formatted HTTP/1.1" 200 610 "-" "KapacitorClient" 06a0858c-bf4a-11e6-8001-000000000000 3791
[httpd] ::1 - - [10/Dec/2016:18:32:20 -0800] "GET /kapacitor/v1/tasks/cpu_alert?dot-view=attributes&replay-id=&script-format=formatted HTTP/1.1" 200 610 "-" "KapacitorClient" 0a602e3d-bf4a-11e6-8002-000000000000 8258
[httpd] ::1 - - [10/Dec/2016:18:32:20 -0800] "PATCH /kapacitor/v1/tasks/cpu_alert HTTP/1.1" 200 611 "-" "KapacitorClient" 0a6197b0-bf4a-11e6-8003-000000000000 14542
[task_master:main] 2016/12/10 18:32:20 I! Stopped task: cpu_alert
[httpd] ::1 - - [10/Dec/2016:18:32:20 -0800] "PATCH /kapacitor/v1/tasks/cpu_alert HTTP/1.1" 200 438 "-" "KapacitorClient" 0a641f42-bf4a-11e6-8004-000000000000 9198
[task_master:main] 2016/12/10 18:32:20 I! Started task: cpu_alert
[httpd] ::1 - - [10/Dec/2016:18:32:20 -0800] "PATCH /kapacitor/v1/tasks/cpu_alert HTTP/1.1" 200 600 "-" "KapacitorClient" 0a65ca90-bf4a-11e6-8005-000000000000 38318
[httpd] ::1 - - [10/Dec/2016:18:32:24 -0800] "GET /kapacitor/v1/tasks/cpu_alert?dot-view=attributes&replay-id=&script-format=formatted HTTP/1.1" 200 600 "-" "KapacitorClient" 0cb42c9f-bf4a-11e6-8006-000000000000 9891
[httpd] ::1 - - [10/Dec/2016:18:32:28 -0800] "GET /kapacitor/v1/tasks?dot-view=attributes&fields=link&limit=100&offset=0&pattern=cpu_alert&replay-id=&script-format=formatted HTTP/1.1" 200 124 "-" "KapacitorClient" 0f6127b7-bf4a-11e6-8007-000000000000 2811
[httpd] ::1 - - [10/Dec/2016:18:32:28 -0800] "PATCH /kapacitor/v1/tasks/cpu_alert HTTP/1.1" 200 610 "-" "KapacitorClient" 0f620002-bf4a-11e6-8008-000000000000 16844
[run] 2016/12/10 18:38:28 I! Signal received, initializing clean shutdown...
[run] 2016/12/10 18:38:28 I! Waiting for clean shutdown...
[task_master:main] 2016/12/10 18:38:28 I! Stopped task: cpu_alert
[smtp] 2016/12/10 18:38:28 I! Closing SMTP service
[stats] 2016/12/10 18:38:28 I! closed service
[httpd] 2016/12/10 18:38:28 I! Closed HTTP service
[task_master:main] 2016/12/10 18:38:28 I! closed
[run] 2016/12/10 18:38:28 I! server shutdown completed
[run] 2016/12/10 18:38:52 I! Kapacitor starting, version 1.1.1, branch master, commit d1e0173df2b9f44c64b7bbf1a120a1bdb3ef683e
[run] 2016/12/10 18:38:52 I! Go version go1.7.4
[srv] 2016/12/10 18:38:52 I! Kapacitor hostname: localhost
[srv] 2016/12/10 18:38:52 I! ClusterID: e364dd76-166e-4e1c-bbe5-06c1c24aac1e ServerID: e859317a-0ea9-4272-a0bd-3b8ce8eeec03
[task_master:main] 2016/12/10 18:38:52 I! opened
[task_store] 2016/12/10 18:38:52 W! could not open old boltd for task_store. Not performing migration. Remove the task_store.dir configuration to disable migration.
[task_master:main] 2016/12/10 18:38:52 I! Started task: cpu_alert
[smtp] 2016/12/10 18:38:52 I! Starting SMTP service
[stats] 2016/12/10 18:38:52 I! opened service
[httpd] 2016/12/10 18:38:52 I! Starting HTTP service
[httpd] 2016/12/10 18:38:52 I! Authentication enabled: false
[run] 2016/12/10 18:38:52 I! Listening for signals
[httpd] ::1 - - [10/Dec/2016:18:39:13 -0800] "GET /kapacitor/v1/tasks/cpu_alert?dot-view=attributes&replay-id=&script-format=formatted HTTP/1.1" 200 610 "-" "KapacitorClient" 0090d939-bf4b-11e6-8001-000000000000 4844
[httpd] ::1 - - [10/Dec/2016:18:39:13 -0800] "PATCH /kapacitor/v1/tasks/cpu_alert HTTP/1.1" 200 610 "-" "KapacitorClient" 0091c034-bf4b-11e6-8002-000000000000 24413
[task_master:main] 2016/12/10 18:39:13 I! Stopped task: cpu_alert
[httpd] ::1 - - [10/Dec/2016:18:39:13 -0800] "PATCH /kapacitor/v1/tasks/cpu_alert HTTP/1.1" 200 437 "-" "KapacitorClient" 0095fb86-bf4b-11e6-8003-000000000000 16514
[task_master:main] 2016/12/10 18:39:13 I! Started task: cpu_alert
[httpd] ::1 - - [10/Dec/2016:18:39:13 -0800] "PATCH /kapacitor/v1/tasks/cpu_alert HTTP/1.1" 200 600 "-" "KapacitorClient" 0098b80b-bf4b-11e6-8004-000000000000 26238
[httpd] ::1 - - [10/Dec/2016:18:39:20 -0800] "GET /kapacitor/v1/tasks?dot-view=attributes&fields=link&limit=100&offset=0&pattern=cpu_alert&replay-id=&script-format=formatted HTTP/1.1" 200 124 "-" "KapacitorClient" 048759ae-bf4b-11e6-8005-000000000000 1720
[httpd] ::1 - - [10/Dec/2016:18:39:20 -0800] "PATCH /kapacitor/v1/tasks/cpu_alert HTTP/1.1" 200 611 "-" "KapacitorClient" 0487d50f-bf4b-11e6-8006-000000000000 8508
[httpd] ::1 - - [10/Dec/2016:18:39:24 -0800] "GET /kapacitor/v1/tasks/cpu_alert?dot-view=attributes&replay-id=&script-format=formatted HTTP/1.1" 200 611 "-" "KapacitorClient" 0707ad3e-bf4b-11e6-8007-000000000000 14243
[httpd] ::1 - - [11/Dec/2016:09:39:04 -0800] "GET /kapacitor/v1/tasks/cpu_alert?dot-view=attributes&replay-id=&script-format=formatted HTTP/1.1" 200 611 "-" "KapacitorClient" b57659a9-bfc8-11e6-8008-000000000000 37256
[httpd] ::1 - - [11/Dec/2016:09:39:04 -0800] "PATCH /kapacitor/v1/tasks/cpu_alert HTTP/1.1" 200 615 "-" "KapacitorClient" b57dca07-bfc8-11e6-8009-000000000000 35760
[task_master:main] 2016/12/11 09:39:04 I! Stopped task: cpu_alert
[httpd] ::1 - - [11/Dec/2016:09:39:04 -0800] "PATCH /kapacitor/v1/tasks/cpu_alert HTTP/1.1" 200 441 "-" "KapacitorClient" b58356b2-bfc8-11e6-800a-000000000000 19168
[task_master:main] 2016/12/11 09:39:04 I! Started task: cpu_alert
[httpd] ::1 - - [11/Dec/2016:09:39:04 -0800] "PATCH /kapacitor/v1/tasks/cpu_alert HTTP/1.1" 200 602 "-" "KapacitorClient" b5865a7c-bfc8-11e6-800b-000000000000 21753
[httpd] ::1 - - [11/Dec/2016:09:42:35 -0800] "GET /kapacitor/v1/tasks?dot-view=attributes&fields=type&fields=status&fields=executing&fields=dbrps&limit=100&offset=0&pattern=cp%2A&replay-id=&script-format=formatted HTTP/1.1" 200 201 "-" "KapacitorClient" 3342378a-bfc9-11e6-800c-000000000000 15366
[httpd] ::1 - - [11/Dec/2016:09:42:35 -0800] "GET /kapacitor/v1/tasks/cpu_alert?dot-view=attributes&replay-id=&script-format=formatted HTTP/1.1" 200 602 "-" "KapacitorClient" 33ced0aa-bfc9-11e6-800d-000000000000 5718
[httpd] ::1 - - [11/Dec/2016:09:44:37 -0800] "POST /kapacitor/v1/recordings/stream HTTP/1.1" 201 214 "-" "KapacitorClient" 7bfc88ff-bfc9-11e6-800e-000000000000 37514
[httpd] ::1 - - [11/Dec/2016:09:44:59 -0800] "GET /kapacitor/v1/tasks?dot-view=attributes&fields=type&fields=status&fields=executing&fields=dbrps&limit=100&offset=0&pattern=fccd05f3-be59-4e93-83db-67027ecd4a12&replay-id=&script-format=formatted HTTP/1.1" 200 43 "-" "KapacitorClient" 8913636c-bfc9-11e6-800f-000000000000 1115
[httpd] ::1 - - [11/Dec/2016:09:46:39 -0800] "POST /kapacitor/v1/recordings/stream HTTP/1.1" 201 213 "-" "KapacitorClient" c4f25780-bfc9-11e6-8010-000000000000 11718
[httpd] ::1 - - [11/Dec/2016:09:46:59 -0800] "GET /kapacitor/v1/recordings?fields=type&fields=size&fields=date&fields=status&limit=100&offset=0&pattern=922d81ca-9375-413c-9ffb-7c8fdd68a360 HTTP/1.1" 200 223 "-" "KapacitorClient" d0d0fa26-bfc9-11e6-8011-000000000000 645

[ticku@localhost ~]$ influx
Visit https://enterprise.influxdata.com to register for updates, InfluxDB server management, and monitoring.
Connected to http://localhost:8086 version 0.13.0
InfluxDB shell version: 0.13.0

show subscriptions

name: telegraf

retention_policy name mode destinations
default kapacitor-e364dd76-166e-4e1c-bbe5-06c1c24aac1e ANY [http://localhost:9092]

name: _internal

retention_policy name mode destinations
monitor kapacitor-e364dd76-166e-4e1c-bbe5-06c1c24aac1e ANY [http://localhost:9092]

name: kapacitor_example

retention_policy name mode destinations
default kapacitor-e364dd76-166e-4e1c-bbe5-06c1c24aac1e ANY [http://localhost:9092]

[httpd] 2016/12/10 17:42:30 ::1 - - [10/Dec/2016:17:42:30 -0800] POST /write?consistency=any&db=telegraf&precision=s&rp=default HTTP/1.1 204 0 - InfluxDBClient 1416bc85-bf43-11e6-8032-000000000000 6.05208ms
[httpd] 2016/12/10 17:42:40 ::1 - - [10/Dec/2016:17:42:40 -0800] POST /write?consistency=any&db=telegraf&precision=s&rp=default HTTP/1.1 204 0 - InfluxDBClient 1a0b9efe-bf43-11e6-8033-000000000000 7.417738ms
[httpd] 2016/12/10 17:42:40 ::1 - - [10/Dec/2016:17:42:40 -0800] POST /write?consistency=any&db=telegraf&precision=s&rp=default HTTP/1.1 204 0 - InfluxDBClient 1a0b9fbc-bf43-11e6-8034-000000000000 10.765394ms
[httpd] 2016/12/10 17:42:50 ::1 - - [10/Dec/2016:17:42:50 -0800] POST /write?consistency=any&db=telegraf&precision=s&rp=default HTTP/1.1 204 0 - InfluxDBClient 200197ed-bf43-11e6-8036-000000000000 8.655367ms
[httpd] 2016/12/10 17:42:50 ::1 - - [10/Dec/2016:17:42:50 -0800] POST /write?consistency=any&db=telegraf&precision=s&rp=default HTTP/1.1 204 0 - InfluxDBClient 200197ed-bf43-11e6-8035-000000000000 11.39447ms
[httpd] 2016/12/10 17:43:00 ::1 - - [10/Dec/2016:17:43:00 -0800] POST /write?consistency=any&db=telegraf&precision=s&rp=default HTTP/1.1 204 0 - InfluxDBClient 25f6f89b-bf43-11e6-8037-000000000000 4.489744ms
[httpd] 2016/12/10 17:43:00 ::1 - - [10/Dec/2016:17:43:00 -0800] POST /write?consistency=any&db=telegraf&precision=s&rp=default HTTP/1.1 204 0 - InfluxDBClient 25f74eb0-bf43-11e6-8038-000000000000 4.027544ms
[httpd] 2016/12/10 17:43:10 ::1 - - [10/Dec/2016:17:43:10 -0800] POST /write?consistency=any&db=telegraf&precision=s&rp=default HTTP/1.1 204 0 - InfluxDBClient 2becdaa1-bf43-11e6-8039-000000000000 4.398767ms
[httpd] 2016/12/10 17:43:10 ::1 - - [10/Dec/2016:17:43:10 -0800] POST /write?consistency=any&db=telegraf&precision=s&rp=default HTTP/1.1 204 0 - InfluxDBClient 2bed6aa4-bf43-11e6-803a-000000000000 2.589761ms
[httpd] 2016/12/10 17:43:20 ::1 - - [10/Dec/2016:17:43:20 -0800] POST /write?consistency=any&db=telegraf&precision=s&rp=default HTTP/1.1 204 0 - InfluxDBClient 31e2dd1e-bf43-11e6-803b-000000000000 4.905191ms
[httpd] 2016/12/10 17:43:20 ::1 - - [10/Dec/2016:17:43:20 -0800] POST /write?consistency=any&db=telegraf&precision=s&rp=default HTTP/1.1 204 0 - InfluxDBClient 31e3037a-bf43-11e6-803c-000000000000 14.243782ms
[run] 2016/12/10 17:43:25 Signal received, initializing clean shutdown...
[run] 2016/12/10 17:43:25 Waiting for clean shutdown...
[copier] 2016/12/10 17:43:25 copier listener closed
[cluster] 2016/12/10 17:43:25 cluster service accept error: network connection closed
[shard-precreation] 2016/12/10 17:43:25 Precreation service terminating
[snapshot] 2016/12/10 17:43:25 snapshot listener closed
[continuous_querier] 2016/12/10 17:43:25 continuous query service terminating
[retention] 2016/12/10 17:43:25 retention policy enforcement terminating
[monitor] 2016/12/10 17:43:25 shutting down monitor system
[monitor] 2016/12/10 17:43:25 terminating storage of statistics
[subscriber] 2016/12/10 17:43:27 closed service
[run] 2016/12/10 17:43:27 server shutdown completed
[ticku@localhost influxdb]$

As it can be seen the error lies in
`[task_store] 2016/12/10 18:16:39 W! **could not open old boltd for task_store. Not performing migration. Remove the `task_store.dir` configuration to disable migration.**`

How to resolve this?
I tried all possible ways to do that:-
1.Added [subscriber] in kapacitor-0.13.1.
2.Referred to all the available forums and articles available online but none of them provided a solution.
3.I also tried in docker but on enabling the cpu_alert neither the kapacitor records any stream nor is the log file created.'show subscriptions' lists subscriptions correctly and there are no errors in influxdb logs



docker run -d --name influxdb -p 8083:8083 -p 8086:8086 influxdb

docker run -d --name telegraf --net=container:influxdb telegraf

After creating tick folder in root of the host I created tickscript - cpu_alert.tick to  that ~/tick folder.

Add tickscript - cpu_alert.tick to  that ~/tick folder.

Add ip of influxdb into kapacitor config file and run below command.

docker run -p 9092:9092 -d --link influxdb:influxdb --name kapacitor -v ~/Downloads/kapacitor.conf:/etc/kapacitor/kapacitor.conf:ro -v ~/tick/:/root -w=/root kapacitor

kapacitor define cpu_alert -type stream -tick ~/cpu_alert.tick -dbrp telegraf.autogen

kapacitor enable cpu_alert
```
Tried restarting the containers but it doesnot seem to work.

Most helpful comment

Looks like impossible to download matched versions of telegraf, influxdb and kapacitor, the current available versions are 1.6, 1.5 and 1.4 respectively. Don't know how is the release policy and why they must be consistent but can not be downloaded with same version. I met issue with data points are captured by kapacitor but alert are not sent like this.

All 24 comments

+1

@ipsha1545 The major version of InfluxDB and Kapacitor must match. Meaning if you are using 0.13.X InfluxDB you need to be using Kapacitor 0.13.X. Since you have upgraded Kapacitor to 1.1.X of Kapacitor you should also upgrade InflxuDB to 1.1.X.

Specifically there are lots of bug fixes for the subscriptions in version 1.0 of Kapacitor/InfluxDB so upgrading my solve your problem.

As it can be seen the error lies in
[task_store] 2016/12/10 18:16:39 W! could not open old boltd for task_store. Not performing migration. Remove thetask_store.dirconfiguration to disable migration.

This is not important, it is just complaining that it could not find an older (0.11) format of the kapacitor database file. That warning is going to be removed in the next release.

You may also find that the output of kapacitor stats ingress is useful in debugging ingest issues with Kapacitor.

I upgraded Influxdb to 1.1.x but still the issue persists.Is there any way to fix it?

Is there any specific version of kapacitor,influxdb and telegraph for which the entire setup will work fine without issues,the stream of data will be visible in kapacitor and the alerts can be visible in the log file either on docker or on localhost?

I'm having the same issue. I upgraded everything to the latest listed on the website today. I can't get anything to work, yet all the communications seem to be working.

@yatahaze @ipsha1545 Any version >= 1.0 should be working without issues. Can you share the output of kapacitor stats ingress?

Sure,
# kapacitor stats ingress Database Retention Policy Measurement Points Received _internal monitor cq 299 _internal monitor database 1196 _internal monitor httpd 299 _internal monitor queryExecutor 299 _internal monitor runtime 299 _internal monitor shard 23322 _internal monitor subscriber 2093 _internal monitor tsm1_cache 23322 _internal monitor tsm1_engine 23322 _internal monitor tsm1_filestore 23322 _internal monitor tsm1_wal 23322 _internal monitor write 299 _kapacitor autogen edges 888 _kapacitor autogen ingress 15628 _kapacitor autogen kapacitor 298 _kapacitor autogen runtime 298 telegraf default ceph 139308 telegraf default cpu 10764 telegraf default disk 31993 telegraf default diskio 28106 telegraf default disksmart 1144 telegraf default kernel 1196 telegraf default mem 1196 telegraf default mysql 299 telegraf default mysql_users 613 telegraf default net 16744 telegraf default processes 1196 telegraf default procstat 598 telegraf default sensors_custom 199 telegraf default swap 1196 telegraf default system 1196 telegraf default ups_apc 598 tm default customer_overdue 68 tm default error_checker 353 tm default inventory_count 50 tm default invoices 204 tm default invoices_90d 216 tm default invoices_ytd 220 tm default item_count 168 tm default order_status 77 tm default order_total 12 tm default order_total_status 24 tm default oversold 134 tm default sales_ptd 220 tm default sales_type 84 tm default sales_type2 588 tm default stock 1370 tm default stock_percentage 220 tm default transactions 208 tm default transactions_launch 964 tm default transactions_theme 1280 tm default transactions_themes 152 tm default transactions_type 668 tm default whos_online 238 tm default whos_online_provinces 105 tm default whos_online_total 50

Other random info I can provide:

  • Telegraf is running on multiple hosts
  • Influx and Kapacitor are on the same LXC VM
  • I've tried tick scripts from the examples (the disk stream example) and that also produced no results, although I'm not sure if there was anything for it to complain about.
  • I have been using Influx & Telegraf for about 6 months now with no issues.
  • The getting started guide references a command to check the CPU usage, that command takes forever because I have so much data in my DB from all the hosts. It's possible my basic test tick script is asking for too much, but applying limits and trying other examples is yielding the same result.

@yatahaze Looks like Kapacitor is receiving the data, can you share a TICKscript that isn't working? The entire output of kapacitor show TASK_NAME would be best.

Here is the TICKscript and the output.

    |from()
        .measurement('cpu')
    |alert()
        .crit(lambda: "usage_idle" < 90)
        .log('/tmp/cpu_idle_alerts.log')

```[ticku@localhost ~]$ kapacitor show cpu_alert
ID: cpu_alert
Error:
Template:
Type: stream
Status: enabled
Executing: true
Created: 19 Dec 16 23:29 PST
Modified: 19 Dec 16 23:29 PST
LastEnabled: 19 Dec 16 23:29 PST
Databases Retention Policies: ["telegraf"."autogen"]
TICKscript:
stream
|from()
.measurement('cpu-total')
|alert()
.crit(lambda: "usage_idle" < 90)
.log('/tmp/alerts.log')

DOT:
digraph cpu_alert {
graph [throughput="0.00 points/s"];

stream0 [avg_exec_time_ns="0s" ];
stream0 -> from1 [processed="0"];

from1 [avg_exec_time_ns="0s" ];
from1 -> alert2 [processed="0"];

alert2 [alerts_triggered="0" avg_exec_time_ns="0s" crits_triggered="0" infos_triggered="0" oks_triggered="0" warns_triggered="0" ];
```

Here's mine:

# kapacitor show cpu_alert
ID: cpu_alert
Error:
Template:
Type: stream
Status: disabled
Executing: false
Created: 19 Dec 16 10:21 PST
Modified: 20 Dec 16 09:03 PST
LastEnabled: 01 Jan 01 00:00 UTC
Databases Retention Policies: ["telegraf"."autogen"]
TICKscript:
stream
    // Select just the cpu measurement from our example database.
    |from()
        .measurement('cpu')
    |alert()
        .crit(lambda: "usage_idle" < 100)
        // Whenever we get an alert write it to a file.
        .log('/tmp/alerts.log')

DOT:
digraph cpu_alert {
stream0 -> from1;
from1 -> alert2;
}

And the other I've been testing:

# kapacitor show disk_alert_stream
ID: disk_alert_stream
Error:
Template:
Type: stream
Status: disabled
Executing: false
Created: 19 Dec 16 11:14 PST
Modified: 19 Dec 16 11:51 PST
LastEnabled: 01 Jan 01 00:00 UTC
Databases Retention Policies: ["telegraf"."autogen"]
TICKscript:
// disk_alert_stream

// metric: used_percent
// available_fields: free","inodes_free","inodes_total","inodes_used","total","used"

// TELEGRAF CONFIGURATION
// [[inputs.disk]]
//   ignore_fs = ["tmpfs", "devtmpfs"]

// DEFINE: kapacitor define disk_alert_stream -type stream -tick disk/disk_alert_stream.tick -dbrp telegraf.autogen
// ENABLE: kapacitor enable disk_alert_stream

// Parameters
var info = 75

var warn = 85

var crit = 92

var period = 10s

var every = 10s

// Dataframe
var data = stream
    |from()
        .database('telegraf')
        .retentionPolicy('autogen')
        .measurement('disk')
        .groupBy('host')
    |window()
        .period(period)
        .every(every)
    |mean('used_percent')
        .as('stat')

// Thresholds
var alert = data
    |alert()
        .id('{{ index .Tags "host"}}/disk_used')
        .message('{{ .ID }}:{{ index .Fields "stat" }}')
        .info(lambda: "stat" > info)
        .warn(lambda: "stat" > warn)
        .crit(lambda: "stat" > crit)

// Alert
alert
        .log('/tmp/disk_alert_log.txt')

DOT:
digraph disk_alert_stream {
stream0 -> from1;
from1 -> window2;
window2 -> mean3;
mean3 -> alert4;
}

And my command:

# kapacitor record stream -task cpu_alert -duration 5s

Results in:

# kapacitor list recordings
ID                                   Type    Status    Size      Date
e39ced51-2b32-4a62-88a3-f9c3ccc92fbf stream  running   0 B       20 Dec 16 09:04 PST
b1e7c9d4-4cec-46ff-a8ff-e22090ac4969 stream  running   0 B       19 Dec 16 12:16 PST

(2nd one is from yesterday)

[ticku@localhost ~]$ kapacitor stats ingress
Database Retention Policy Measurement Points Received
_internal monitor cq 760
_internal monitor database 1520
_internal monitor httpd 760
_internal monitor queryExecutor 760
_internal monitor runtime 760
_internal monitor shard 5320
_internal monitor subscriber 3040
_internal monitor tsm1_cache 5320
_internal monitor tsm1_engine 5320
_internal monitor tsm1_filestore 5320
_internal monitor tsm1_wal 5320
_internal monitor write 760
_kapacitor autogen edges 3795
_kapacitor autogen ingress 18970
_kapacitor autogen kapacitor 759
_kapacitor autogen nodes 2277
_kapacitor autogen runtime 759
telegraf autogen cpu 3796
telegraf autogen disk 3800
telegraf autogen diskio 8916
telegraf autogen kernel 760
telegraf autogen mem 760
telegraf autogen processes 760
telegraf autogen swap 1520
telegraf autogen system 1520
``

+1

Having same issues with influxdb 1.1.1, kapacitor 1.1.1

confirmed that influxdb+kapacitor 1.1.0 works--was able to get through the getting started guide now. Hope this helps narrow things down to a range of changelists. ;)

same issue here

+1
i have the same issue

[dev_dean@VM_61_2_centos ~]$ kapacitor -url http://172.1.56.5:9092 stats ingress
Database   Retention Policy Measurement    Points Received
_internal  monitor          cq                         220
_internal  monitor          database                   440
_internal  monitor          httpd                      220
_internal  monitor          queryExecutor              220
_internal  monitor          runtime                    220
_internal  monitor          shard                     1760
_internal  monitor          subscriber                3514
_internal  monitor          tsm1_cache                1760
_internal  monitor          tsm1_engine               1760
_internal  monitor          tsm1_filestore            1760
_internal  monitor          tsm1_wal                  1760
_internal  monitor          write                      220
_kapacitor autogen          edges                      573
_kapacitor autogen          ingress                   3224
_kapacitor autogen          kapacitor                  108
_kapacitor autogen          nodes                      291
_kapacitor autogen          runtime                    108
telegraf   autogen          cpu                       4806
telegraf   autogen          disk                      1314
telegraf   autogen          diskio                    1314
telegraf   autogen          kernel                     438
telegraf   autogen          mem                        438
telegraf   autogen          mongodb                    218
telegraf   autogen          net                        876
telegraf   autogen          netstat                    438
telegraf   autogen          processes                  438
telegraf   autogen          redis                      220
telegraf   autogen          redis_keyspace             660
telegraf   autogen          swap                       876
telegraf   autogen          system                     876
[dev_dean@VM_61_2_centos ~]$ kapacitor -url http://172.1.56.5:9092 show cpu_alert
ID: cpu_alert
Error: 
Template: 
Type: stream
Status: enabled
Executing: true
Created: 24 Jan 17 06:07 UTC
Modified: 24 Jan 17 06:08 UTC
LastEnabled: 24 Jan 17 06:08 UTC
Databases Retention Policies: ["telegraf"."default"]
TICKscript:
stream
    // Select just the cpu measurement from our example database.
    |from()
        .database('telegraf')
        .measurement('cpu')
    |alert()
        .crit(lambda: "usage_idle" < 100)
        // Whenever we get an alert write it to a file.
        .log('/tmp/alerts.log')

DOT:
digraph cpu_alert {
graph [throughput="0.00 points/s"];

stream0 [avg_exec_time_ns="0s" ];
stream0 -> from1 [processed="0"];

from1 [avg_exec_time_ns="0s" ];
from1 -> alert2 [processed="0"];

alert2 [alerts_triggered="0" avg_exec_time_ns="0s" crits_triggered="0" infos_triggered="0" oks_triggered="0" warns_triggered="0" ];
}

kapacitor logs:

[srv] 2017/01/24 06:06:12 D! opening service: *task_store.Service
[task_store] 2017/01/24 06:06:12 W! could not open old boltd for task_store. Not performing migration. Remove the `task_store.dir` configuration to disable migration.
[srv] 2017/01/24 06:06:12 D! opened service: *task_store.Service

i running influx and kapacitor in docker in kubernetes...

influxdb 1.1.1
kapacitor 1.1.1

I'm experience the same issues with

  • InfluxDB 1.1.0
  • Kapacitor 1.1.0

Here is some extra information and troubleshooting I've been doing, any extra input would be appreciated:

Hello, I'm having an issue with Kapacitor and data ingestion/alerting. The data appears
to be incoming from subscription channels from Influx, however Kapacitor streams
do not appear active.

Kapacitor and InfluxDB are running inside of separate docker containers on same
host.

Any help in diagnosing this problem is greatly appreciated.

Relevant Configuration

The relevant configuration from my Kapacitor config is:

hostname = "kapacitor"
data_dir = "/root/.kapacitor"
default-retention-policy = "one_year"

[http]
  bind-address = "0.0.0.0:9092"
  auth-enabled = false
  log-enabled = true
  write-tracing = false
  pprof-enabled = false
  https-enabled = false
  https-certificate = "/etc/ssl/kapacitor.pem"

[replay]
  dir = "/root/.kapacitor/replay"

[[influxdb]]
  enabled = true
  default = true
  name = "influxdb"
  urls = ["http://influxdb:8086"]
  username = ""
  password = ""
  timeout = 0
  subscription-protocol="http"
  udp-buffer = 1000
  udp-read-buffer = 0
  disable-subscriptions = false

[logging]
  file = "STDERR"
  level = "DEBUG"

The Testing TICK Script

I am trying to run 1 task, the cpu task. Kapacitor knows about it and thinks
it is running.

/ # kapacitor show cpu
ID: cpu
Error:
Template:
Type: stream
Status: enabled
Executing: true
Created: 27 Jan 17 19:57 UTC
Modified: 27 Jan 17 19:57 UTC
LastEnabled: 27 Jan 17 19:57 UTC
Databases Retention Policies: ["metrics"."default"]
TICKscript: 
var data = stream
        |from()
    .database('metrics')
    .measurement('docker_container_cpu')

data
    |deadman(100.0, 10s)|log().prefix('DEADMAN')

data
    |log().prefix('STANDARD')

DOT:
digraph cpu {
graph [throughput="0.00 points/s"];

stream0 [avg_exec_time_ns="0s" ];
stream0 -> from1 [processed="0"];

from1 [avg_exec_time_ns="0s" ];
from1 -> log2 [processed="0"];

log2 [avg_exec_time_ns="0s" ];
log2 -> alert3 [processed="0"];

alert3 [alerts_triggered="0" avg_exec_time_ns="0s" crits_triggered="0" infos_triggered="0" oks_triggered="0" warns_triggered="0" ];

stats4 [avg_exec_time_ns="0s" ];
stats4 -> derivative5 [processed="2"];

derivative5 [avg_exec_time_ns="0s" ];
derivative5 -> alert6 [processed="1"];

alert6 [alerts_triggered="1" avg_exec_time_ns="0s" crits_triggered="1" infos_triggered="0" oks_triggered="0" warns_triggered="0" ];
}

Here I'm trying to get Kapacitor log anything to STDERR, and if nothing comes in
the stream, then the deadman prints picks up.

I can confirm in InfluxDB there is a metrics table. I do not have a default
retention policy, or one by the name. However, I'm thinking the Databases Retention Policies specification just means it will use the default retention
policy of the metrics table.

Troubleshooting

Kapacitor Logs

Many logs similar to this:

[httpd] 10.0.0.138 - - [27/Jan/2017:17:09:50 +0000] "POST /write?consistency=&db=metrics&precision=ns&rp=one_year HTTP/1.1" 204 0 "-" "InfluxDBClient" 699b238e-e4b3-11e6-ae28-000000000000 22832

Show up in the logs. I am taking these logs to mean that the InfluxDB
subscription is indeed working.

I have confirmed the InfluxDB subscriptions to the database exist by looking in
the InfluxDB and finding the kapacitor subscriptions, targeted at the correct host.

However, the deadman node kicks off every 10seconds:

[cpu:alert5] 2017/01/27 22:43:10 D! CRITICAL alert triggered id:node 'from1' in task 'cpu' msg:node 'from1' in task 'cpu' is dead: %!f(MISSING) points/10s. data:&{stats map[] [time emitted] [[2017-01-27 22:42:50 +0000 UTC 0]]}
[cpu:alert10] 2017/01/27 22:43:10 D! CRITICAL alert triggered id:node 'stream0' in task 'cpu' msg:node 'stream0' in task 'cpu' is dead: %!f(MISSING) points/10s. data:&{stats map[] [time emitted] [[2017-01-27 22:42:50 +0000 UTC 0]]}
[cpu:log6] 2017/01/27 22:43:10 I! DEADMAN {"Name":"stats","Database":"","RetentionPolicy":"","Group":"","Dimensions":{"ByName":false,"TagNames":null},"Tags":null,"Fields":{"emitted":0},"Time":"2017-01-27T22:42:50Z"}

This data stream is updated in InfluxDB every 10 seconds, and we never see any
logging from the |log().prefix('STANDARD') portion of the debugging TICK file.

write-tracing

If I turn on write tracing, I can see values for CPU to come through:

cpu,cpu=cpu-total,host=argon-odinperf usage_guest=0,usage_guest_nice=0,usage_idle=88.1011465381056,usage_iowait=0.028270771164317015,usage_irq=0,usage_nice=0,usage_softirq=0.21674257892678755,usage_steal=0,usage_system=2.2333909219421186,usage_user=9.420449191315278 1485557630000000000

So I know that I'm getting valid data from Influx.

i updated the kapacitor and influxdb to 1.2. Now it all working

Hi is the 1.2-alpine good image too? Cause I can see that I received something while i do Kapacitor Stats ingress. However, I neither get any alerts nor my kapacitor record stream, records anything. It just stays like its hung without exciting. Any help?

Looks like impossible to download matched versions of telegraf, influxdb and kapacitor, the current available versions are 1.6, 1.5 and 1.4 respectively. Don't know how is the release policy and why they must be consistent but can not be downloaded with same version. I met issue with data points are captured by kapacitor but alert are not sent like this.

+1 Also it's 2019

+1

Was this page helpful?
0 / 5 - 0 ratings

Related issues

davidhiendl picture davidhiendl  路  6Comments

sslupsky picture sslupsky  路  3Comments

nathanielc picture nathanielc  路  8Comments

phemmer picture phemmer  路  5Comments

R4scal picture R4scal  路  4Comments