Icinga2: Servicegroup table for livestatus is not refreshed when service is removed

Created on 15 Jul 2019 · 5Comments · Source: Icinga/icinga2

Describe the bug

When a service that belongs to a servicegroup is deleted from a host, the servicegroup is not updated in livestatus.

To Reproduce

Add a host called called e.g. example.localdomain
Add services to host, e.g. service1, service2, service3
Add a service group with all hosts matching a regex "^example.*":

object ServiceGroup "example" {
  display_name = "example"
  assign where regex("^example.*", host.name)
}

Delete service3 from host through api:

curl -k -s -u root:icinga -H 'Accept: application/json' \
 -X DELETE 'https://localhost:5665/v1/objects/services/example.localdomain!service3?cascade=1&pretty=1'
 ```
5. Notice that the service is still in the servicegroup example:

echo -e "GET servicegroups\n" | nc -U /var/run/icinga2/cmd/livestatus |grep example


## Expected behavior
The service should no longer be part of the servicegroup in livestatus

## Your Environment

* Version used (`icinga2 --version`):

icinga2 --version

icinga2 - The Icinga 2 network monitoring daemon (version: r2.10.5-1)

Copyright (c) 2012-2019 Icinga GmbH (https://icinga.com/)
License GPLv2+: GNU GPL version 2 or later http://gnu.org/licenses/gpl2.html
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

System information:
Platform: Debian GNU/Linux
Platform version: 9 (stretch)
Kernel: Linux
Kernel version: 4.9.0-9-amd64
Architecture: x86_64

Build information:
Compiler: GNU 6.3.0
Build host: cb654124b660

Application information:

General paths:
Config directory: /etc/icinga2
Data directory: /var/lib/icinga2
Log directory: /var/log/icinga2
Cache directory: /var/cache/icinga2
Spool directory: /var/spool/icinga2
Run directory: /run/icinga2

Old paths (deprecated):
Installation root: /usr
Sysconf directory: /etc
Run directory (base): /run
Local state directory: /var

Internal paths:
Package data directory: /usr/share/icinga2
State path: /var/lib/icinga2/icinga2.state
Modified attributes path: /var/lib/icinga2/modified-attributes.conf
Objects path: /var/cache/icinga2/icinga2.debug
Vars path: /var/cache/icinga2/icinga2.vars
PID path: /run/icinga2/icinga2.pid

* Operating System and version:

lsb_release -a

No LSB modules are available.
Distributor ID: Debian
Description: Debian GNU/Linux 9.9 (stretch)
Release: 9.9
Codename: stretch

* Enabled features (`icinga2 feature list`):

icinga2 feature list

Disabled features: debuglog elasticsearch gelf influxdb mainlog opentsdb perfdata statusdata
Enabled features: api checker command compatlog graphite ido-mysql livestatus notification syslog

* Icinga Web 2 version and modules (System - About):
N/A
* Config validation (`icinga2 daemon -C`):

icinga2 feature list

Disabled features: debuglog elasticsearch gelf influxdb mainlog opentsdb perfdata statusdata
Enabled features: api checker command compatlog graphite ido-mysql livestatus notification syslog
root@icinga-test-srv01:~# icinga2 daemon -C
[2019-07-15 12:58:16 +0200] information/cli: Icinga application loader (version: r2.10.5-1)
[2019-07-15 12:58:16 +0200] information/cli: Loading configuration file(s).
[2019-07-15 12:58:18 +0200] information/ConfigItem: Committing config item(s).
[2019-07-15 12:58:18 +0200] warning/ApiListener: Attribute 'key_path' for object 'api' of type 'ApiListener' is deprecated and should not be used.
[2019-07-15 12:58:18 +0200] warning/ApiListener: Attribute 'ca_path' for object 'api' of type 'ApiListener' is deprecated and should not be used.
[2019-07-15 12:58:18 +0200] warning/ApiListener: Attribute 'cert_path' for object 'api' of type 'ApiListener' is deprecated and should not be used.
[2019-07-15 12:58:18 +0200] warning/ApiListener: Please read the upgrading documentation for v2.8: https://icinga.com/docs/icinga2/latest/doc/16-upgrading-icinga-2/
[2019-07-15 12:58:18 +0200] information/ApiListener: My API identity: icinga-test-srv01.localdomain
[2019-07-15 12:58:28 +0200] information/WorkQueue: #4 (DaemonUtility::LoadConfigFiles) items: 0, rate: 74.6667/s (4480/min 4480/5min 4480/15min);
[2019-07-15 12:58:28 +0200] information/WorkQueue: #6 (ApiListener, SyncQueue) items: 0, rate: 0/s (0/min 0/5min 0/15min);
[2019-07-15 12:58:28 +0200] information/WorkQueue: #7 (GraphiteWriter, graphite) items: 0, rate: 0/s (0/min 0/5min 0/15min);
[2019-07-15 12:58:28 +0200] information/WorkQueue: #5 (ApiListener, RelayQueue) items: 0, rate: 0/s (0/min 0/5min 0/15min);
[2019-07-15 12:58:28 +0200] information/WorkQueue: #8 (IdoMysqlConnection, ido-mysql) items: 0, rate: 0/s (0/min 0/5min 0/15min);
[2019-07-15 12:58:29 +0200] warning/ApplyRule: Apply rule 'mail-host-utveckling' (in /local/icinga2/checked-out-config/conf/notification.conf: 68:1-68:49) for type 'Notification' does not match anywhere!
[2019-07-15 12:58:29 +0200] warning/ApplyRule: Apply rule 'mail-service-utveckling' (in /local/icinga2/checked-out-config/conf/notification.conf: 79:1-79:55) for type 'Notification' does not match anywhere!
[2019-07-15 12:58:29 +0200] information/ConfigItem: Instantiated 11658 Services.
[2019-07-15 12:58:29 +0200] information/ConfigItem: Instantiated 1 LivestatusListener.
[2019-07-15 12:58:29 +0200] information/ConfigItem: Instantiated 1 SyslogLogger.
[2019-07-15 12:58:29 +0200] information/ConfigItem: Instantiated 1 IcingaApplication.
[2019-07-15 12:58:29 +0200] information/ConfigItem: Instantiated 1301 Hosts.
[2019-07-15 12:58:29 +0200] information/ConfigItem: Instantiated 5 NotificationCommands.
[2019-07-15 12:58:29 +0200] information/ConfigItem: Instantiated 13988 Notifications.
[2019-07-15 12:58:29 +0200] information/ConfigItem: Instantiated 1 NotificationComponent.
[2019-07-15 12:58:29 +0200] information/ConfigItem: Instantiated 47 HostGroups.
[2019-07-15 12:58:29 +0200] information/ConfigItem: Instantiated 1 ApiListener.
[2019-07-15 12:58:29 +0200] information/ConfigItem: Instantiated 69 Downtimes.
[2019-07-15 12:58:29 +0200] information/ConfigItem: Instantiated 1 GraphiteWriter.
[2019-07-15 12:58:29 +0200] information/ConfigItem: Instantiated 6 Comments.
[2019-07-15 12:58:29 +0200] information/ConfigItem: Instantiated 1 CheckerComponent.
[2019-07-15 12:58:29 +0200] information/ConfigItem: Instantiated 1 Zone.
[2019-07-15 12:58:29 +0200] information/ConfigItem: Instantiated 1 ExternalCommandListener.
[2019-07-15 12:58:29 +0200] information/ConfigItem: Instantiated 2 Endpoints.
[2019-07-15 12:58:29 +0200] information/ConfigItem: Instantiated 3 ApiUsers.
[2019-07-15 12:58:29 +0200] information/ConfigItem: Instantiated 1 CompatLogger.
[2019-07-15 12:58:29 +0200] information/ConfigItem: Instantiated 379 Users.
[2019-07-15 12:58:29 +0200] information/ConfigItem: Instantiated 1 IdoMysqlConnection.
[2019-07-15 12:58:29 +0200] information/ConfigItem: Instantiated 299 CheckCommands.
[2019-07-15 12:58:29 +0200] information/ConfigItem: Instantiated 8 UserGroups.
[2019-07-15 12:58:29 +0200] information/ConfigItem: Instantiated 43 ServiceGroups.
[2019-07-15 12:58:29 +0200] information/ConfigItem: Instantiated 10 TimePeriods.
[2019-07-15 12:58:29 +0200] information/ScriptGlobal: Dumping variables to file '/var/cache/icinga2/icinga2.vars'
[2019-07-15 12:58:29 +0200] information/cli: Finished validating the configuration file(s).
```

arelivestatus help wanted

Source

mickenordin

All 5 comments

That's basically the same thing as with #4084 - such dependencies are not re-evaluated atm. Therefore closing as duplicate.

dnsmichi on 15 Jul 2019

Moving the discussion here:

@dnsmichi I created #7317 and I am not 100% sure that this is the same problem. The icinga2 api notices that the service is removed and since group membership is stored in the service, not in the servicegroup there is no problem for the api here.

What is needed to fix #7317 would instead be to refresh the servicegroup table when a service or host is deleted, or the hostgroup table only if a host is deleted.

Further more, it works when adding hosts or services through the api, livestatus catches that and adds the service to the servicegroup table, so it seems that there is allready code for updating the table, it just isn't run on delete.

I see. Livestatus does some extra magic here which works differently than the rest of the core. Feel free to look into a patch, we won't push any resources into the Livestatus feature anymore.

dnsmichi on 15 Jul 2019

😕2

@lippserd Do we even still maintain livestatus?