Icinga2: NotificationCommand misbehaving in 2.12

Created on 4 Sep 2020  Â·  16Comments  Â·  Source: Icinga/icinga2

Describe the bug

We have noticed that the CommandNotification object had a breaking behavior change between 2.11.5 and 2.12.0.

We are using a icinga2opsgenie integration using a configuration close to this one. while upgrading icinga2 to 2.12.0 we suddenly didn't receive any alert on OpsGenie anymore for the service problems.
After investigations, we noticed that the notification command generated by icinga2 was wrong as shown below:

'/usr/bin/icinga2opsgenie' '-sdt' '0' '-lss' 'UNKNOWN' '-lssc' '1598969681' '-lssi' '3' '-mha' '3' '-msa' '5' '-s' 'service_passive_host_check_placeholder' '-sa' '1' '-sc' 'dummy' '-sdn' 'service_passive_host_check_placeholder' '-sds' '164280.386972' '-lsch' '1599133838' '-set' '0' '-sl' '0.000554' '-so' 'No check results received. Last result time: 2020-09-03 11:45:38 +0000' '-spd' '' '-ss' 'UNKNOWN' '-ssi' '3' '-sst' 'HARD' '-t' 'PROBLEM' '-tags' 'dev' '-teams' '[REDACTED]' '-ho' 'PING OK - Packet loss = 0%, RTA = 2.54 ms' '-ha' '1' '-haddr' 'â…œREDACTED]' '-hal' '[REDACTED]' '-hdn' '[REDACTED]' '-hds' '3692919.058979' '-hdt' '0' '-het' '4.028298' '-hgns' 'dev' '-hl' '0.000411' '-hn' '[REDACTED]' '-entityType' 'service' '-hpd' 'rta=2.537000ms;3000.000000;5000.000000;0.000000 pl=0%;80;100;0' '-hs' 'UP' '-hsi' '0' '-hst' 'HARD' '-ldt' '2020-09-03 11:52:41 +0000' '-lhc' '1599133940' '-lhs' 'UP' '-lhsc' '1595441042' '-lhsi' '0'

If you give it a closer look, you'll notice that the -spd option is being added to the command line although its value evaluates to empty.
The consequence is that the icinga2opsgenie command kind of shifts all the remaining options by one (okay, this is maybe something wrong with the Golang's flag parser).
Then we decided to update the configuration to use a more explicit syntax that proved to correctly skip options whose values evaluate to empty as shown below, unfortunately this gave us the same results:

  arguments = {
    "-entityType" = "service"
    "-t" = {
      value = "$notification.type$"
    }
    "-spd" = {
      value = "$service.perfdata$"
    }
  }

Expected behavior

With icinga2 2.11.x arguments whose values evaluate to empty are correctly skipped from the generated NotificationCommand.

Your Environment

  • Version used (icinga2 --version):
icinga2 - The Icinga 2 network monitoring daemon (version: r2.12.0-1)

Copyright (c) 2012-2020 Icinga GmbH (https://icinga.com/)
License GPLv2+: GNU GPL version 2 or later <http://gnu.org/licenses/gpl2.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

System information:
  Platform: Debian GNU/Linux
  Platform version: 10 (buster)
  Kernel: Linux
  Kernel version: 4.14.[REDACTED]
  Architecture: x86_64

Build information:
  Compiler: GNU 8.3.0
  Build host: runner-hh8q3bz2-project-298-concurrent-0

Application information:

General paths:
  Config directory: /etc/icinga2
  Data directory: /var/lib/icinga2
  Log directory: /var/log/icinga2
  Cache directory: /var/cache/icinga2
  Spool directory: /var/spool/icinga2
  Run directory: /run/icinga2

Old paths (deprecated):
  Installation root: /usr
  Sysconf directory: /etc
  Run directory (base): /run
  Local state directory: /var

Internal paths:
  Package data directory: /usr/share/icinga2
  State path: /var/lib/icinga2/icinga2.state
  Modified attributes path: /var/lib/icinga2/modified-attributes.conf
  Objects path: /var/cache/icinga2/icinga2.debug
  Vars path: /var/cache/icinga2/icinga2.vars
  PID path: /run/icinga2/icinga2.pid
  • Operating System and version:
Distributor ID: Debian
Description:    Debian GNU/Linux 10 (buster)
Release:    10
Codename:   buster
  • Enabled features (icinga2 feature list):
Enabled features: api checker command graphite ido-mysql mainlog notification
  • Config validation (icinga2 daemon -C):
[2020-09-04 14:08:22 +0000] information/cli: Icinga application loader (version: r2.12.0-1)
[2020-09-04 14:08:22 +0000] information/cli: Loading configuration file(s).
[2020-09-04 14:08:22 +0000] information/ConfigItem: Committing config item(s).
[2020-09-04 14:08:22 +0000] information/ApiListener: My API identity: [REDACTED]
[2020-09-04 14:08:22 +0000] information/ConfigItem: Instantiated 39 HostGroups.
[2020-09-04 14:08:22 +0000] information/ConfigItem: Instantiated 1 FileLogger.
[2020-09-04 14:08:22 +0000] information/ConfigItem: Instantiated 2 NotificationCommands.
[2020-09-04 14:08:22 +0000] information/ConfigItem: Instantiated 1 NotificationComponent.
[2020-09-04 14:08:22 +0000] information/ConfigItem: Instantiated 145 Notifications.
[2020-09-04 14:08:22 +0000] information/ConfigItem: Instantiated 1 IcingaApplication.
[2020-09-04 14:08:22 +0000] information/ConfigItem: Instantiated 50 Hosts.
[2020-09-04 14:08:22 +0000] information/ConfigItem: Instantiated 1 ApiListener.
[2020-09-04 14:08:22 +0000] information/ConfigItem: Instantiated 6 Comments.
[2020-09-04 14:08:22 +0000] information/ConfigItem: Instantiated 1 GraphiteWriter.
[2020-09-04 14:08:22 +0000] information/ConfigItem: Instantiated 1 CheckerComponent.
[2020-09-04 14:08:22 +0000] information/ConfigItem: Instantiated 4 Zones.
[2020-09-04 14:08:22 +0000] information/ConfigItem: Instantiated 1 ExternalCommandListener.
[2020-09-04 14:08:22 +0000] information/ConfigItem: Instantiated 3 Endpoints.
[2020-09-04 14:08:22 +0000] information/ConfigItem: Instantiated 2 ApiUsers.
[2020-09-04 14:08:22 +0000] information/ConfigItem: Instantiated 1 IdoMysqlConnection.
[2020-09-04 14:08:22 +0000] information/ConfigItem: Instantiated 242 CheckCommands.
[2020-09-04 14:08:22 +0000] information/ConfigItem: Instantiated 1 User.
[2020-09-04 14:08:22 +0000] information/ConfigItem: Instantiated 95 Services.
[2020-09-04 14:08:22 +0000] information/ScriptGlobal: Dumping variables to file '/var/cache/icinga2/icinga2.vars'
[2020-09-04 14:08:22 +0000] information/cli: Finished validating the configuration file(s).
areconfiguration bug

Most helpful comment

Ok, so I ran more tests, and I confirm that version v2.12.0-1-gade891bbf fixes the issue.

All 16 comments

Hello @ziirish and thank you for reporting!

Please could you test v2.12.0 + #8174?

https://nextcloud.icinga.com/index.php/s/cHRZCJbeBg8MCyt

Best,
AK

Hello,

Sure, I'll give it a try and let you know if things are better.

Thanks

Seems not to work as expected or I have discovered a different issue

OS: Ubuntu 20.04

icinga2 - The Icinga 2 network monitoring daemon (version: v2.12.0-1-gade891bbf)

Copyright (c) 2012-2020 Icinga GmbH (https://icinga.com/)
License GPLv2+: GNU GPL version 2 or later <http://gnu.org/licenses/gpl2.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

System information:
  Platform: Ubuntu
  Platform version: 20.04.1 LTS (Focal Fossa)
  Kernel: Linux
  Kernel version: 5.8.8-arch1-1
  Architecture: x86_64

Build information:
  Compiler: GNU 8.3.0
  Build host: 8e67d8135f5d
  OpenSSL version: OpenSSL 1.1.1f  31 Mar 2020

Application information:

General paths:
  Config directory: /etc/icinga2
  Data directory: /var/lib/icinga2
  Log directory: /var/log/icinga2
  Cache directory: /var/cache/icinga2
  Spool directory: /var/spool/icinga2
  Run directory: /run/icinga2

Old paths (deprecated):
  Installation root: /usr
  Sysconf directory: /etc
  Run directory (base): /run
  Local state directory: /var

Internal paths:
  Package data directory: /usr/share/icinga2
  State path: /var/lib/icinga2/icinga2.state
  Modified attributes path: /var/lib/icinga2/modified-attributes.conf
  Objects path: /var/cache/icinga2/icinga2.debug
  Vars path: /var/cache/icinga2/icinga2.vars
  PID path: /run/icinga2/icinga2.pid

CheckCommand

object CheckCommand "check_test" {
  import "plugin-check-command"
  import "ipv4-or-ipv6"

  command = [ PluginDir + "/check_test.sh" ]

  arguments = {
    "-H" = {
      value = "$check_test_address$"
      description = "Hostname"
      required = true
    }
    "-t" = {
      value = "$check_test_foo$"
    }
  }

  vars.check_test_address = "$check_address$"
  vars.check_test_foo = 123
}

Check Skript

Very very basic. ;-)

# cat /usr/lib/nagios/plugins/check_test.sh
#!/bin/sh

echo $@ >> /tmp/params

Try 1

apply Service "check_test" {
  import "generic-service"

  check_command = "check_test"

  assign where 1 == 1
}

Result

./check_test.sh -H 127.0.0.1 -t 123

This looks like expected

Try 2

apply Service "check_test" {
  import "generic-service"

  check_command = "check_test"

  vars.check_test_foo = null

  assign where 1 == 1
}

Result

./check_test.sh -H 127.0.0.1 -t

Empty value for -t

Try 3

apply Service "check_test" {
  import "generic-service"

  check_command = "check_test"

  vars.check_test_foo = ""

  assign where 1 == 1
}

Result

./check_test.sh -H 127.0.0.1 -t

Empty value for -t

@phibos What exactly "Seems not to work"?

8214.conf

object CheckerComponent "checker" { }

object CheckCommand "8214" {
    command = [ "echo" ]
    arguments = {
        "-h" = {
            value = "$host.name$"
        }
        "-t" = {
            value = "$8214_t$"
        }
    }
}

object Host "null" {
    check_command = "8214"
    vars["8214_t"] = null
}

object Host "empty" {
    check_command = "8214"
    vars["8214_t"] = ""
}

object Host "h" {
    check_command = "8214"
    vars["8214_t"] = "h"
}

v2.11.x / #8174

➜  icinga2 git:(e9b8fc234) prefix/sbin/icinga2 daemon -x notice -c 8214.conf
[2020-09-21 11:11:57 +0200] information/cli: Icinga application loader (version: v2.11.0; debug)
[2020-09-21 11:11:57 +0200] notice/cli: Spawning seemless worker process doing the actual work
[2020-09-21 11:11:57 +0200] notice/cli: Spawned worker process (PID 86226), waiting for it to load its config
[2020-09-21 11:11:57 +0200] information/cli: Loading configuration file(s).
[2020-09-21 11:11:57 +0200] notice/ConfigCompiler: Compiling config file: 8214.conf
[2020-09-21 11:11:57 +0200] information/ConfigItem: Committing config item(s).
[2020-09-21 11:11:57 +0200] notice/WorkQueue: Spawning WorkQueue threads for 'DaemonUtility::LoadConfigFiles'
[2020-09-21 11:11:57 +0200] information/ConfigItem: Instantiated 1 CheckCommand.
[2020-09-21 11:11:57 +0200] information/ConfigItem: Instantiated 3 Hosts.
[2020-09-21 11:11:57 +0200] information/ConfigItem: Instantiated 1 IcingaApplication.
[2020-09-21 11:11:57 +0200] information/ConfigItem: Instantiated 1 CheckerComponent.
[2020-09-21 11:11:57 +0200] information/ScriptGlobal: Dumping variables to file '/Users/aklimov/NET/WS/icinga2/prefix/var/cache/icinga2/icinga2.vars'
[2020-09-21 11:11:57 +0200] notice/WorkQueue: Stopped WorkQueue threads for 'DaemonUtility::LoadConfigFiles'
[2020-09-21 11:11:57 +0200] notice/cli: Notifying umbrella process (PID 86204) about the config loading success
[2020-09-21 11:11:57 +0200] notice/cli: Worker process successfully loaded its config
[2020-09-21 11:11:57 +0200] notice/cli: Waiting for the umbrella process to let us doing the actual work
[2020-09-21 11:11:57 +0200] notice/cli: The umbrella process let us continuing
[2020-09-21 11:11:57 +0200] information/ConfigItem: Triggering Start signal for config items
[2020-09-21 11:11:57 +0200] information/CheckerComponent: 'checker' started.
[2020-09-21 11:11:57 +0200] information/ConfigItem: Activated all objects.
[2020-09-21 11:11:57 +0200] notice/WorkQueue: Stopped WorkQueue threads for 'DaemonCommand::Run'
[2020-09-21 11:11:57 +0200] notice/ConfigObjectUtility: Package _api doesn't exist yet, creating it.
[2020-09-21 11:11:57 +0200] notice/ApiListener: Updating object authority for local objects.
[2020-09-21 11:11:57 +0200] notice/Process: Running command 'echo' '-h' 'empty' '-t' '': PID 86227
[2020-09-21 11:11:57 +0200] notice/Process: PID 86227 ('echo' '-h' 'empty' '-t' '') terminated with exit code 0
[2020-09-21 11:11:57 +0200] notice/Checkable: State Change: Checkable 'empty' hard state change from DOWN to UP detected.
[2020-09-21 11:12:02 +0200] notice/CheckerComponent: Pending checkables: 0; Idle checkables: 3; Checks/s: 0.0166667
[2020-09-21 11:12:05 +0200] notice/Process: Running command 'echo' '-h' 'h' '-t' 'h': PID 86232
[2020-09-21 11:12:05 +0200] notice/Process: PID 86232 ('echo' '-h' 'h' '-t' 'h') terminated with exit code 0
[2020-09-21 11:12:05 +0200] notice/Checkable: State Change: Checkable 'h' hard state change from DOWN to UP detected.
[2020-09-21 11:12:07 +0200] notice/CheckerComponent: Pending checkables: 0; Idle checkables: 3; Checks/s: 0.0333333
[2020-09-21 11:12:12 +0200] notice/CheckerComponent: Pending checkables: 0; Idle checkables: 3; Checks/s: 0.0333333
[2020-09-21 11:12:17 +0200] notice/CheckerComponent: Pending checkables: 0; Idle checkables: 3; Checks/s: 0.0333333
[2020-09-21 11:12:22 +0200] notice/CheckerComponent: Pending checkables: 0; Idle checkables: 3; Checks/s: 0.0333333
[2020-09-21 11:12:27 +0200] notice/CheckerComponent: Pending checkables: 0; Idle checkables: 3; Checks/s: 0.0333333
[2020-09-21 11:12:32 +0200] notice/CheckerComponent: Pending checkables: 0; Idle checkables: 3; Checks/s: 0.0333333
[2020-09-21 11:12:37 +0200] notice/CheckerComponent: Pending checkables: 0; Idle checkables: 3; Checks/s: 0.0333333
[2020-09-21 11:12:42 +0200] notice/CheckerComponent: Pending checkables: 0; Idle checkables: 3; Checks/s: 0.0333333
[2020-09-21 11:12:42 +0200] notice/Process: Running command 'echo' '-h' 'null' '-t' '': PID 86234
[2020-09-21 11:12:42 +0200] notice/Process: PID 86234 ('echo' '-h' 'null' '-t' '') terminated with exit code 0
[2020-09-21 11:12:42 +0200] notice/Checkable: State Change: Checkable 'null' hard state change from DOWN to UP detected.
[2020-09-21 11:12:47 +0200] notice/CheckerComponent: Pending checkables: 0; Idle checkables: 3; Checks/s: 0.05
^C[2020-09-21 11:12:48 +0200] notice/cli: Got signal 2, forwarding to seemless worker (PID 86226)
[2020-09-21 11:12:48 +0200] information/Application: Received request to shut down.
[2020-09-21 11:12:48 +0200] information/Application: Shutting down...
[2020-09-21 11:12:48 +0200] information/CheckerComponent: 'checker' stopped.
[2020-09-21 11:12:48 +0200] information/ConfigObject: Dumping program state to file '/Users/aklimov/NET/WS/icinga2/prefix/var/lib/icinga2/icinga2.state'
[2020-09-21 11:12:48 +0200] information/IcingaApplication: Icinga has shut down.
[2020-09-21 11:12:48 +0200] notice/cli: Seemless worker (PID 86226) stopped, stopping as well
➜  icinga2 git:(e9b8fc234)

v2.12.0

➜  icinga2 git:(338d0aaa8) prefix/sbin/icinga2 daemon -x notice -c 8214.conf
[2020-09-21 11:08:29 +0200] information/cli: Icinga application loader (version: v2.12.0; debug)
[2020-09-21 11:08:29 +0200] notice/cli: Spawning seemless worker process doing the actual work
[2020-09-21 11:08:29 +0200] notice/cli: Spawned worker process (PID 84904), waiting for it to load its config
[2020-09-21 11:08:29 +0200] information/cli: Loading configuration file(s).
[2020-09-21 11:08:29 +0200] notice/ConfigCompiler: Compiling config file: 8214.conf
[2020-09-21 11:08:29 +0200] notice/ConfigCompiler: Compiling config file: /Users/aklimov/NET/WS/icinga2/prefix/var/lib/icinga2/api/packages/_api/include.conf
[2020-09-21 11:08:29 +0200] notice/ConfigCompiler: Compiling config file: /Users/aklimov/NET/WS/icinga2/prefix/var/lib/icinga2/api/packages/_api/eac27b59-e3fb-4098-9212-e95ed626abcc/include.conf
[2020-09-21 11:08:29 +0200] notice/ConfigCompiler: Compiling config file: /Users/aklimov/NET/WS/icinga2/prefix/var/lib/icinga2/api/packages/_api/eac27b59-e3fb-4098-9212-e95ed626abcc/../active.conf
[2020-09-21 11:08:29 +0200] information/ConfigItem: Committing config item(s).
[2020-09-21 11:08:29 +0200] notice/WorkQueue: Spawning WorkQueue threads for 'DaemonUtility::LoadConfigFiles'
[2020-09-21 11:08:29 +0200] information/ConfigItem: Instantiated 1 CheckCommand.
[2020-09-21 11:08:29 +0200] information/ConfigItem: Instantiated 3 Hosts.
[2020-09-21 11:08:29 +0200] information/ConfigItem: Instantiated 1 IcingaApplication.
[2020-09-21 11:08:29 +0200] information/ConfigItem: Instantiated 1 CheckerComponent.
[2020-09-21 11:08:29 +0200] information/ScriptGlobal: Dumping variables to file '/Users/aklimov/NET/WS/icinga2/prefix/var/cache/icinga2/icinga2.vars'
[2020-09-21 11:08:29 +0200] notice/WorkQueue: Stopped WorkQueue threads for 'DaemonUtility::LoadConfigFiles'
[2020-09-21 11:08:29 +0200] notice/cli: Notifying umbrella process (PID 84890) about the config loading success
[2020-09-21 11:08:29 +0200] notice/cli: Worker process successfully loaded its config
[2020-09-21 11:08:29 +0200] notice/cli: Waiting for the umbrella process to let us doing the actual work
[2020-09-21 11:08:29 +0200] notice/cli: The umbrella process let us continuing
[2020-09-21 11:08:29 +0200] notice/ConfigCompiler: Compiling config file: /Users/aklimov/NET/WS/icinga2/prefix/var/lib/icinga2/modified-attributes.conf
[2020-09-21 11:08:29 +0200] information/ConfigItem: Triggering Start signal for config items
[2020-09-21 11:08:29 +0200] information/CheckerComponent: 'checker' started.
[2020-09-21 11:08:29 +0200] information/ConfigItem: Activated all objects.
[2020-09-21 11:08:29 +0200] notice/WorkQueue: Stopped WorkQueue threads for 'DaemonCommand::Run'
[2020-09-21 11:08:29 +0200] notice/ApiListener: Updating object authority for local objects.
[2020-09-21 11:08:29 +0200] notice/Process: Running command 'echo' '-h' 'empty' '-t' '': PID 84906
[2020-09-21 11:08:29 +0200] notice/Process: PID 84906 ('echo' '-h' 'empty' '-t' '') terminated with exit code 0
[2020-09-21 11:08:29 +0200] notice/Checkable: State Change: Checkable 'empty' hard state change from DOWN to UP detected.
[2020-09-21 11:08:34 +0200] notice/CheckerComponent: Pending checkables: 0; Idle checkables: 3; Checks/s: 0.0166667
[2020-09-21 11:08:37 +0200] notice/Process: Running command 'echo' '-h' 'h' '-t' 'h': PID 84909
[2020-09-21 11:08:37 +0200] notice/Process: PID 84909 ('echo' '-h' 'h' '-t' 'h') terminated with exit code 0
[2020-09-21 11:08:37 +0200] notice/Checkable: State Change: Checkable 'h' hard state change from DOWN to UP detected.
[2020-09-21 11:08:39 +0200] notice/CheckerComponent: Pending checkables: 0; Idle checkables: 3; Checks/s: 0.0333333
[2020-09-21 11:08:44 +0200] notice/CheckerComponent: Pending checkables: 0; Idle checkables: 3; Checks/s: 0.0333333
[2020-09-21 11:08:49 +0200] notice/CheckerComponent: Pending checkables: 0; Idle checkables: 3; Checks/s: 0.0333333
[2020-09-21 11:08:54 +0200] notice/CheckerComponent: Pending checkables: 0; Idle checkables: 3; Checks/s: 0.0333333
[2020-09-21 11:08:59 +0200] notice/CheckerComponent: Pending checkables: 0; Idle checkables: 3; Checks/s: 0.0333333
[2020-09-21 11:09:04 +0200] notice/CheckerComponent: Pending checkables: 0; Idle checkables: 3; Checks/s: 0.0333333
[2020-09-21 11:09:09 +0200] notice/CheckerComponent: Pending checkables: 0; Idle checkables: 3; Checks/s: 0.0333333
[2020-09-21 11:09:14 +0200] notice/CheckerComponent: Pending checkables: 0; Idle checkables: 3; Checks/s: 0.0333333
[2020-09-21 11:09:14 +0200] notice/Process: Running command 'echo' '-h' 'null' '-t': PID 84910
[2020-09-21 11:09:14 +0200] notice/Process: PID 84910 ('echo' '-h' 'null' '-t') terminated with exit code 0
[2020-09-21 11:09:14 +0200] notice/Checkable: State Change: Checkable 'null' hard state change from DOWN to UP detected.
^C[2020-09-21 11:09:18 +0200] notice/cli: Got signal 2, forwarding to seemless worker (PID 84904)
[2020-09-21 11:09:18 +0200] information/Application: Received request to shut down.
[2020-09-21 11:09:18 +0200] information/Application: Shutting down...
[2020-09-21 11:09:18 +0200] information/CheckerComponent: 'checker' stopped.
[2020-09-21 11:09:18 +0200] information/ConfigObject: Dumping program state to file '/Users/aklimov/NET/WS/icinga2/prefix/var/lib/icinga2/icinga2.state'
[2020-09-21 11:09:18 +0200] information/IcingaApplication: Icinga has shut down.
[2020-09-21 11:09:18 +0200] notice/cli: Seemless worker (PID 84904) stopped, stopping as well
➜  icinga2 git:(338d0aaa8)

@ziirish What means "a configuration close to this one"?

"-spd" = {
  value = "$service.perfdata$"
}
➜  icinga2 git:(e9b8fc234) prefix/sbin/icinga2 console
Icinga 2 (version: v2.11.0)
Type $help to view available commands.
<1> => Service().perfdata
       ^^^^^^^^^^^^^^^^^^
Invalid field access (for value of type 'Service'): 'perfdata'
<2> =>

I still have the same issue as well with this version:

```
icinga2 - The Icinga 2 network monitoring daemon (version: v2.12.0-1-gade891bbf)

Copyright (c) 2012-2020 Icinga GmbH (https://icinga.com/)
License GPLv2+: GNU GPL version 2 or later http://gnu.org/licenses/gpl2.html
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

System information:
Platform: Debian GNU/Linux
Platform version: 10 (buster)
Kernel: Linux
Kernel version: 4.19.0-5-cloud-amd64
Architecture: x86_64

Build information:
Compiler: GNU 8.3.0
Build host: 8e67d8135f5d
OpenSSL version: OpenSSL 1.1.1d 10 Sep 2019

Application information:

General paths:
Config directory: /etc/icinga2
Data directory: /var/lib/icinga2
Log directory: /var/log/icinga2
Cache directory: /var/cache/icinga2
Spool directory: /var/spool/icinga2
Run directory: /run/icinga2

Old paths (deprecated):
Installation root: /usr
Sysconf directory: /etc
Run directory (base): /run
Local state directory: /var

Internal paths:
Package data directory: /usr/share/icinga2
State path: /var/lib/icinga2/icinga2.state
Modified attributes path: /var/lib/icinga2/modified-attributes.conf
Objects path: /var/cache/icinga2/icinga2.debug
Vars path: /var/cache/icinga2/icinga2.vars
PID path: /run/icinga2/icinga2.pid
```

The arguments whose value evaluate to empty are added to the command line where they weren't in v2.11.x.

Please share a service, its CheckCommand and its host.

I'm using icinga-director's API to publish my hosts with apply-rules to create my services.

Here is the generated example in /var/lib/icinga2/api/zones//director/hosts.conf:

object Host "brokennotif" {
    import "generic-host"

    display_name = "Broken Notif"
    address = "127.0.0.1"
    groups = [ "somegroup" ]
    vars.vision = {
        service = {
            "wrong_website" = {
                display_name = "Check Wrong Website"
                tags = [  ]
                type = "http"
                url = "thisiswrong"
            }
        }
    }
}

My apply rules are:

apply Service "service_http_" for (service => config in host.vars.vision.service) {
  display_name = config.display_name
  check_command = "http"
  import "generic-service"

  if (config.tags) {
    vars.tags = config.tags
  }

  vars.http_vhost = config.url
  vars.http_uri = "/"

  assign where config.type == "http"
}

The check_command is the one provided with icinga-director:

object CheckCommand "http" {
    import "plugin-check-command"
    command = [ PluginDir + "/check_http" ]
    timeout = 1m
    arguments += {
        "--invert-regex" = {
            description = "Return CRITICAL if found, OK if not"
            set_if = "$http_invertregex$"
        }
        "--no-body" = {
            description = "Don't wait for document body: stop reading after headers"
            set_if = "$http_ignore_body$"
        }
        "--sni" = {
            description = "Enable SSL/TLS hostname extension support (SNI)"
            set_if = "$http_sni$"
        }
        "-4" = {
            description = "Use IPv4 connection"
            set_if = "$http_ipv4$"
        }
        "-6" = {
            description = "Use IPv6 connection"
            set_if = "$http_ipv6$"
        }
        "-A" = {
            description = "String to be sent in http header as User Agent"
            value = "$http_useragent$"
        }
        "-C" = {
            description = "Minimum number of days a certificate has to be valid. This parameter explicitely sets the port to 443 and ignores the URL if passed."
            value = "$http_certificate$"
        }
        "-E" = {
            description = "Print additional perfdata"
            set_if = "$http_extendedperfdata$"
        }
        "-H" = {
            description = "Host name argument for servers using host headers (virtual host)"
            value = "$http_vhost$"
        }
        "-I" = {
            description = "IP address or name (use numeric address if possible to bypass DNS lookup)"
            value = "$http_address$"
        }
        "-J" = {
            description = "Name of file contains the client certificate (PEM format)"
            value = "$http_clientcert$"
        }
        "-K" = {
            description = "Name of file contains the private key (PEM format)"
            value = "$http_privatekey$"
        }
        "-L" = {
            description = "Wrap output in HTML link"
            set_if = "$http_link$"
        }
        "-M" = {
            description = "Warn if document is more than seconds old"
            value = "$http_maxage$"
        }
        "-P" = {
            description = "URL encoded http POST data"
            value = "$http_post$"
        }
        "-R" = {
            description = "Search page for case-insensitive regex"
            order = 2
            value = "$http_expect_body_eregi$"
        }
        "-S" = {
            description = "Connect via SSL"
            set_if = "$http_ssl$"
        }
        "-S1" = {
            description = "Connect via SSL version TLSv1"
            set_if = "$http_ssl_force_tlsv1$"
        }
        "-S1+" = {
            description = "Connect via SSL version TLSv1 and newer"
            set_if = "$http_ssl_force_tlsv1_or_higher$"
        }
        "-S1.1" = {
            description = "Connect via SSL version TLSv1.1"
            set_if = "$http_ssl_force_tlsv1_1$"
        }
        "-S1.1+" = {
            description = "Connect via SSL version TLSv1.1 and newer"
            set_if = "$http_ssl_force_tlsv1_1_or_higher$"
        }
        "-S1.2" = {
            description = "Connect via SSL version TLSv1.2"
            set_if = "$http_ssl_force_tlsv1_2$"
        }
        "-S1.2+" = {
            description = "Connect via SSL version TLSv1.2 and newer"
            set_if = "$http_ssl_force_tlsv1_2_or_higher$"
        }
        "-S2" = {
            description = "Connect via SSL version SSLv2"
            set_if = "$http_ssl_force_sslv2$"
        }
        "-S2+" = {
            description = "Connect via SSL version SSLv2 and newer"
            set_if = "$http_ssl_force_sslv2_or_higher$"
        }
        "-S3" = {
            description = "Connect via SSL version SSLv3"
            set_if = "$http_ssl_force_sslv3$"
        }
        "-S3+" = {
            description = "Connect via SSL version SSLv3 and newer"
            set_if = "$http_ssl_force_sslv3_or_higher$"
        }
        "-T" = {
            description = "Specify Content-Type header when POSTing"
            value = "$http_contenttype$"
        }
        "-a" = {
            description = "Username:password on sites with basic authentication"
            value = "$http_auth_pair$"
        }
        "-b" = {
            description = "Username:password on proxy-servers with basic authentication"
            value = "$http_proxy_auth_pair$"
        }
        "-c" = {
            description = "Response time to result in critical status (seconds)"
            value = "$http_critical_time$"
        }
        "-d" = {
            description = "String to expect in the response headers"
            value = "$http_headerstring$"
        }
        "-e" = {
            description = "Comma-delimited list of strings, at least one of them is expected in the first (status) line of the server response (default: HTTP/1.)"
            value = "$http_expect$"
        }
        "-f" = {
            description = "How to handle redirect pages"
            value = "$http_onredirect$"
        }
        "-j" = {
            description = "Set http method (for example: HEAD, OPTIONS, TRACE, PUT, DELETE)"
            value = "$http_method$"
        }
        "-k" = {
            description = "Any other tags to be sent in http header"
            value = "$http_header$"
        }
        "-l" = {
            description = "Allow regex to span newline"
            order = 1
            set_if = "$http_linespan$"
        }
        "-m" = {
            description = "Minim page size required:Maximum page size required"
            value = "$http_pagesize$"
        }
        "-p" = {
            description = "Port number (default: 80)"
            value = "$http_port$"
        }
        "-r" = {
            description = "Search page for regex"
            order = 2
            value = "$http_expect_body_regex$"
        }
        "-s" = {
            description = "String to expect in the content"
            value = "$http_string$"
        }
        "-t" = {
            description = "Seconds before connection times out"
            value = "$http_timeout$"
        }
        "-u" = {
            description = "URL to GET or POST (default: /)"
            value = "$http_uri$"
        }
        "-v" = {
            description = "Show details for command-line debugging"
            set_if = "$http_verbose$"
        }
        "-w" = {
            description = "Response time to result in warning status (seconds)"
            value = "$http_warn_time$"
        }
    }
    vars.check_address = {
        arguments = [  ]
        deprecated = false
        name = "<anonymous>"
        side_effect_free = false
        type = "Function"
    }
    vars.check_ipv4 = "$http_ipv4$"
    vars.check_ipv6 = "$http_ipv6$"
    vars.http_address = "$check_address$"
    vars.http_invertregex = false
    vars.http_linespan = false
    vars.http_link = false
    vars.http_sni = false
    vars.http_ssl = false
    vars.http_verbose = false
}

Surprisingly, the "optional" arguments of the ckeck command seem correctly handled. We only have the problem with the Notification Command.

Notification command is the same as shown here with a few adjustments where I explicitly mark the arguments as optional like this:

object NotificationCommand "opsgenie-service-notification" {
  import "plugin-notification-command"

  vars.sgns = {{ service.groups.join(",") }}
  command = [ "/tmp/dump_notif.sh" ]
  arguments = {
    "-entityType" = "service"
    "-t" = {
      value = "$notification.type$"
    }
    "-ldt" = {
      value = "$icinga.long_date_time$"
    }
    "-hn" = {
      value = "$host.name$"
    }
    "-hdn" = {
      value = "$host.display_name$"
    }
    "-hal" = {
      value = "$host.display_name$"
    }
[...]

From your log output, here is what seem wrong to me:

[2020-09-21 11:11:57 +0200] notice/Process: Running command 'echo' '-h' 'empty' '-t' '': PID 86227
[2020-09-21 11:11:57 +0200] notice/Process: PID 86227 ('echo' '-h' 'empty' '-t' '') terminated with exit code 0

I expect the command to be 'echo' '-h' 'empty'.
This is how it works in v2.11.x.

Notification command is the same as shown here with a few adjustments

Please share the full adjusted Notification command config.

object NotificationCommand "opsgenie-service-notification" {
  import "plugin-notification-command"

  vars.sgns = {{ service.groups.join(",") }}
  command = [ "/tmp/dump_notif.sh" ]
  arguments = {
    "-entityType" = "service"
    "-t" = {
      value = "$notification.type$"
    }
    "-ldt" = {
      value = "$icinga.long_date_time$"
    }
    "-hn" = {
      value = "$host.name$"
    }
    "-hdn" = {
      value = "$host.display_name$"
    }
    "-hal" = {
      value = "$host.display_name$"
    }
    "-haddr" = {
      value = "$host.address$"
    }
    "-hs" = {
      value = "$host.state$"
    }
    "-hsi" = {
      value = "$host.state_id$"
    }
    "-lhs" = {
      value = "$host.last_state$"
    }
    "-lhsi" = {
      value = "$host.last_state_id$"
    }
    "-hst" = {
      value = "$host.state_type$"
    }
    "-ha" = {
      value = "$host.check_attempt$"
    }
    "-mha" = {
      value = "$host.max_check_attempts$"
    }
    "-hl" = {
      value = "$host.latency$"
    }
    "-het" = {
      value = "$host.execution_time$"
    }
    "-hds" = {
      value = "$host.duration_sec$"
    }
    "-hdt" = {
      value = "$host.downtime_depth$"
    }
    "-hgn" = {
      value = "$host.group$"
    }
    "-lhc" = {
      value = "$host.last_check$"
    }
    "-lhsc" = {
      value = "$host.last_state_change$"
    }
    "-ho" = {
      value = "$host.output$"
    }
    "-hpd" = {
      value = "$host.perfdata$"
    }
    "-s" = {
      value = "$service.name$"
    }
    "-sdn" = {
      value = "$service.display_name$"
    }
    "-ss" = {
      value = "$service.state$"
    }
    "-ssi" = {
      value = "$service.state_id$"
    }
    "-lss" = {
      value = "$service.last_state$"
    }
    "-lssi" = {
      value = "$service.last_state_id$"
    }
    "-sst" = {
      value = "$service.state_type$"
    }
    "-sa" = {
      value = "$service.check_attempt$"
    }
    "-sc" = {
      value = "$service.check_command$"
    }
    "-msa" = {
      value = "$service.max_check_attempts$"
    }
    "-sl" = {
      value = "$service.latency$"
    }
    "-set" = {
      value = "$service.execution_time$"
    }
    "-sds" = {
      value = "$service.duration_sec$"
    }
    "-sdt" = {
      value = "$service.downtime_depth$"
    }
    "-lsch" = {
      value = "$service.last_check$"
    }
    "-lssc" = {
      value = "$service.last_state_change$"
    }
    "-so" = {
      value = "$service.output$"
    }
    "-sgns" = {
      value = "$command.vars.sgns$"
    }
    "-spd" = {
      value = "$service.perfdata$"
    }
    "-teams" = {
      value = "$host.vars.team$"
    }
    "-hgns" = {
      value = {{
        var hostgroups_array = []
        var hostgroups = macro("$host.groups$")
        for (var hostgroup_name in hostgroups) {
          var hostgroup = get_host_group(hostgroup_name)
          if (hostgroup.display_name) {
            hostgroups_array.add(hostgroup.display_name)
          } else {
            hostgroups_array.push(hostgroup_name)
          }
        }
        return hostgroups_array.join(",")
      }}
    }
    "-tags" = {
      value = {{
        var tags_array = []
        var host_tags = macro("$host.vars.tags$")
        var service_tags = macro("$service.vars.tags$")
        if (host_tags) {
          for (var item in host_tags) {
            tags_array.add(item)
          }
        }
        if (service_tags) {
          for (var item in service_tags) {
            tags_array.add(item)
          }
        }
        return tags_array.join(",")
      }}
    }
  }
}

object NotificationCommand "opsgenie-host-notification" {
  import "plugin-notification-command"

  command = [ "/usr/bin/icinga2opsgenie" ]
  arguments = {
    "-entityType" = "host"
    "-t" = {
      value = "$notification.type$"
    }
    "-ldt" = {
      value = "$icinga.long_date_time$"
    }
    "-hn" = {
      value = "$host.display_name$"
    }
    "-hdn" = {
      value = "$host.display_name$"
    }
    "-hal" = {
      value = "$host.name$"
    }
    "-haddr" = {
      value = "$host.address$"
    }
    "-hs" = {
      value = "$host.state$"
    }
    "-hsi" = {
      value = "$host.state_id$"
    }
    "-lhs" = {
      value = "$host.last_state$"
    }
    "-lhsi" = {
      value = "$host.last_state_id$"
    }
    "-hst" = {
      value = "$host.state_type$"
    }
    "-ha" = {
      value = "$host.check_attempt$"
    }
    "-mha" = {
      value = "$host.max_check_attempts$"
    }
    "-hl" = {
      value = "$host.latency$"
    }
    "-het" = {
       value = "$host.execution_time$"
    }
    "-hds" = {
      value = "$host.duration_sec$"
    }
    "-hdt" = {
      value = "$host.downtime_depth$"
    }
    "-hgn" = {
      value = "$host.group$"
    }
    "-lhc" = {
      value = "$host.last_check$"
    }
    "-lhsc" = {
      value = "$host.last_state_change$"
    }
    "-ho" = {
      value = "$host.output$"
    }
    "-hpd" = {
      value = "$host.perfdata$"
    }
    "-teams" = {
      value = "$host.vars.team$"
    }
    "-hgns" = {
      value = {{
        var hostgroups_array = []
        var hostgroups = macro("$host.groups$")
        for (var hostgroup_name in hostgroups) {
          var hostgroup = get_host_group(hostgroup_name)
          if (hostgroup.display_name) {
            hostgroups_array.add(hostgroup.display_name)
          } else {
            hostgroups_array.push(hostgroup_name)
          }
        }
        return hostgroups_array.join(",")
      }}
    }
    "-tags" = {
      value = {{
        var host_tags = macro("$host.vars.tags$")
        if (host_tags) {
          return host_tags.join(",")
        }
        return ""
      }}
    }
  }
}

object User "opsgenie" {
  import "generic-user"
  display_name = "OpsGenie Contact"
}

apply Notification "notify-opsgenie-service" to Service {
  command = "opsgenie-service-notification"
  users = ["opsgenie"]
  assign where service
}

apply Notification "notify-opsgenie-host" to Host {
  command = "opsgenie-host-notification"
  users = ["opsgenie"]
  assign where host
}

Hm... this config...

object CheckerComponent "checker" { }

object NotificationComponent "notification" { }

object CheckCommand "perfdata" {
    command = [ "bash", "-c", "echo ' |lolcat=1'; exit 2" ]
}

object CheckCommand "no-perfdata" {
    command = [ "bash", "-c", "exit 2" ]
}

object Host "perfdata" {
    check_command = "perfdata"
    max_check_attempts = 1
}

object Host "no-perfdata" {
    check_command = "no-perfdata"
    max_check_attempts = 1
}

object NotificationCommand "perfdata" {
    command = [ "echo" ]
    arguments = {
        "-p" = {
            value = "$host.perfdata$"
        }
    }
}

object User "admin" { }

apply Notification "perfdata" to Host {
    command = "perfdata"
    users = [ "admin" ]
    assign where true
}

... shows the same behavior on v2.12 and v2.11...

➜  tmp.Lbse0QH0 git:(e9b8fc234) prefix/sbin/icinga2 daemon -x notice -c 8214.conf
[2020-09-21 13:08:36 +0200] information/cli: Icinga application loader (version: v2.11.0; debug)
[2020-09-21 13:08:36 +0200] notice/cli: Spawning seemless worker process doing the actual work
[2020-09-21 13:08:36 +0200] notice/cli: Spawned worker process (PID 94046), waiting for it to load its config
[2020-09-21 13:08:36 +0200] information/cli: Loading configuration file(s).
[2020-09-21 13:08:36 +0200] notice/ConfigCompiler: Compiling config file: 8214.conf
[2020-09-21 13:08:36 +0200] information/ConfigItem: Committing config item(s).
[2020-09-21 13:08:36 +0200] notice/WorkQueue: Spawning WorkQueue threads for 'DaemonUtility::LoadConfigFiles'
[2020-09-21 13:08:36 +0200] information/ConfigItem: Instantiated 2 CheckCommands.
[2020-09-21 13:08:36 +0200] information/ConfigItem: Instantiated 2 Hosts.
[2020-09-21 13:08:36 +0200] information/ConfigItem: Instantiated 1 IcingaApplication.
[2020-09-21 13:08:36 +0200] information/ConfigItem: Instantiated 2 Notifications.
[2020-09-21 13:08:36 +0200] information/ConfigItem: Instantiated 1 NotificationCommand.
[2020-09-21 13:08:36 +0200] information/ConfigItem: Instantiated 1 User.
[2020-09-21 13:08:36 +0200] information/ConfigItem: Instantiated 1 CheckerComponent.
[2020-09-21 13:08:36 +0200] information/ConfigItem: Instantiated 1 NotificationComponent.
[2020-09-21 13:08:36 +0200] information/ScriptGlobal: Dumping variables to file '/var/folders/c2/k37yy5v51qzd5_rfrwd_sd7r0000gn/T/tmp.Lbse0QH0/prefix/var/cache/icinga2/icinga2.vars'
[2020-09-21 13:08:36 +0200] notice/WorkQueue: Stopped WorkQueue threads for 'DaemonUtility::LoadConfigFiles'
[2020-09-21 13:08:36 +0200] notice/cli: Notifying umbrella process (PID 94031) about the config loading success
[2020-09-21 13:08:36 +0200] notice/cli: Worker process successfully loaded its config
[2020-09-21 13:08:36 +0200] notice/cli: Waiting for the umbrella process to let us doing the actual work
[2020-09-21 13:08:36 +0200] notice/cli: The umbrella process let us continuing
[2020-09-21 13:08:36 +0200] information/ConfigItem: Triggering Start signal for config items
[2020-09-21 13:08:36 +0200] information/NotificationComponent: 'notification' started.
[2020-09-21 13:08:36 +0200] information/CheckerComponent: 'checker' started.
[2020-09-21 13:08:36 +0200] information/ConfigItem: Activated all objects.
[2020-09-21 13:08:36 +0200] notice/WorkQueue: Stopped WorkQueue threads for 'DaemonCommand::Run'
[2020-09-21 13:08:36 +0200] notice/ConfigObjectUtility: Package _api doesn't exist yet, creating it.
[2020-09-21 13:08:36 +0200] notice/ApiListener: Updating object authority for local objects.
[2020-09-21 13:08:36 +0200] notice/Process: Running command 'bash' '-c' 'echo ' |lolcat=1'; exit 2': PID 94047
[2020-09-21 13:08:36 +0200] notice/Process: PID 94047 ('bash' '-c' 'echo ' |lolcat=1'; exit 2') terminated with exit code 2
[2020-09-21 13:08:36 +0200] notice/Checkable: State Change: Checkable 'perfdata' hard state change from DOWN to DOWN detected.
[2020-09-21 13:08:36 +0200] information/Checkable: Checkable 'perfdata' has 1 notification(s). Checking filters for type 'Problem', sends will be logged.
[2020-09-21 13:08:36 +0200] notice/Notification: Attempting to send notifications of type 'Problem' for notification object 'perfdata!perfdata'.
[2020-09-21 13:08:36 +0200] information/Notification: Sending 'Problem' notification 'perfdata!perfdata' for user 'admin'
[2020-09-21 13:08:36 +0200] notice/Process: Running command 'echo' '-p' 'lolcat=1': PID 94048
[2020-09-21 13:08:36 +0200] information/Notification: Completed sending 'Problem' notification 'perfdata!perfdata' for checkable 'perfdata' and user 'admin' using command 'perfdata'.
[2020-09-21 13:08:36 +0200] notice/Process: PID 94048 ('echo' '-p' 'lolcat=1') terminated with exit code 0
[2020-09-21 13:08:41 +0200] notice/CheckerComponent: Pending checkables: 0; Idle checkables: 2; Checks/s: 0.0166667
[2020-09-21 13:08:44 +0200] notice/Process: Running command 'bash' '-c' 'exit 2': PID 94155
[2020-09-21 13:08:44 +0200] notice/Process: PID 94155 ('bash' '-c' 'exit 2') terminated with exit code 2
[2020-09-21 13:08:44 +0200] notice/Checkable: State Change: Checkable 'no-perfdata' hard state change from DOWN to DOWN detected.
[2020-09-21 13:08:44 +0200] information/Checkable: Checkable 'no-perfdata' has 1 notification(s). Checking filters for type 'Problem', sends will be logged.
[2020-09-21 13:08:44 +0200] notice/Notification: Attempting to send notifications of type 'Problem' for notification object 'no-perfdata!perfdata'.
[2020-09-21 13:08:44 +0200] information/Notification: Sending 'Problem' notification 'no-perfdata!perfdata' for user 'admin'
[2020-09-21 13:08:44 +0200] notice/Process: Running command 'echo' '-p' '': PID 94156
[2020-09-21 13:08:44 +0200] information/Notification: Completed sending 'Problem' notification 'no-perfdata!perfdata' for checkable 'no-perfdata' and user 'admin' using command 'perfdata'.
[2020-09-21 13:08:44 +0200] notice/Process: PID 94156 ('echo' '-p' '') terminated with exit code 0
[2020-09-21 13:08:46 +0200] notice/CheckerComponent: Pending checkables: 0; Idle checkables: 2; Checks/s: 0.0333333
[2020-09-21 13:08:51 +0200] notice/CheckerComponent: Pending checkables: 0; Idle checkables: 2; Checks/s: 0.0333333
[2020-09-21 13:08:56 +0200] notice/CheckerComponent: Pending checkables: 0; Idle checkables: 2; Checks/s: 0.0333333
[2020-09-21 13:09:01 +0200] notice/CheckerComponent: Pending checkables: 0; Idle checkables: 2; Checks/s: 0.0333333
[2020-09-21 13:09:06 +0200] notice/CheckerComponent: Pending checkables: 0; Idle checkables: 2; Checks/s: 0.0333333
[2020-09-21 13:09:11 +0200] notice/CheckerComponent: Pending checkables: 0; Idle checkables: 2; Checks/s: 0.0333333
^C[2020-09-21 13:09:15 +0200] notice/cli: Got signal 2, forwarding to seemless worker (PID 94046)
[2020-09-21 13:09:15 +0200] information/Application: Received request to shut down.
[2020-09-21 13:09:15 +0200] information/Application: Shutting down...
[2020-09-21 13:09:15 +0200] information/CheckerComponent: 'checker' stopped.
[2020-09-21 13:09:15 +0200] information/NotificationComponent: 'notification' stopped.
[2020-09-21 13:09:15 +0200] information/ConfigObject: Dumping program state to file '/var/folders/c2/k37yy5v51qzd5_rfrwd_sd7r0000gn/T/tmp.Lbse0QH0/prefix/var/lib/icinga2/icinga2.state'
[2020-09-21 13:09:15 +0200] information/IcingaApplication: Icinga has shut down.
[2020-09-21 13:09:15 +0200] notice/cli: Seemless worker (PID 94046) stopped, stopping as well
➜  tmp.Lbse0QH0 git:(e9b8fc234)

@Al2Klimov Thanks for your examples. If we are mixing two issues we could split the issues and the discussion.

2.11.x

[2020-09-21 11:11:57 +0200] notice/Process: Running command 'echo' '-h' 'empty' '-t' '': PID 86227
[2020-09-21 11:11:57 +0200] notice/Process: PID 86227 ('echo' '-h' 'empty' '-t' '') terminated with exit code 0
...
[2020-09-21 11:12:42 +0200] notice/Process: Running command 'echo' '-h' 'null' '-t' '': PID 86234
[2020-09-21 11:12:42 +0200] notice/Process: PID 86234 ('echo' '-h' 'null' '-t' '') terminated with exit code 0

2.12

[2020-09-21 11:08:29 +0200] notice/Process: Running command 'echo' '-h' 'empty' '-t' '': PID 84906
[2020-09-21 11:08:29 +0200] notice/Process: PID 84906 ('echo' '-h' 'empty' '-t' '') terminated with exit code 0
...
[2020-09-21 11:09:14 +0200] notice/Process: Running command 'echo' '-h' 'null' '-t': PID 84910
[2020-09-21 11:09:14 +0200] notice/Process: PID 84910 ('echo' '-h' 'null' '-t') terminated with exit code 0

Expected

[2020-09-21 11:08:29 +0200] notice/Process: Running command 'echo' '-h' 'empty' '-t' '': PID 84906
[2020-09-21 11:08:29 +0200] notice/Process: PID 84906 ('echo' '-h' 'empty' '-t' '') terminated with exit code 0
...
[2020-09-21 11:09:14 +0200] notice/Process: Running command 'echo' '-h' 'null': PID 84910
[2020-09-21 11:09:14 +0200] notice/Process: PID 84910 ('echo' '-h' 'null') terminated with exit code 0

Setting a variable to "" (empty string) and icinga passes the argument with the empty string as value to the check command, than this is what I would expect. But setting a variable to null (value, not string) should not pass the argument or the value to the check command. At least there is a unset command to unset a variable.

Example

The example below has been tested on Ubuntu 20.04 and Debian 10.

$ apt-get install monitoring-plugins-basic

Values for warning and critical set to null. No arguments are passed. The command uses the internal default values.

$ /usr/lib/nagios/plugins/check_procs                   
PROCS OK: 5 processes | procs=5;;;0;

Setting the values for warning and critical to "" (empty string)(or in Icinga 2.11.x to null). Works in this case.

$ /usr/lib/nagios/plugins/check_procs -w '' -c ''
PROCS OK: 5 processes | procs=5;;;0;

Setting the values to null (value, not string) in Icinga 2.12 results in an unpredicted behaviour.

$ /usr/lib/nagios/plugins/check_procs -w -c   
PROCS WARNING: 5 processes | procs=5;-c;;0;
$ /usr/lib/nagios/plugins/check_procs -c -w
PROCS CRITICAL: 5 processes | procs=5;;-w;0;

But setting a variable to null (value, not string) should not pass the argument or the value to the check command.

This was also my opinion due to #7558 and I've made #7567. But then https://github.com/Icinga/icinga2/pull/7567#issuecomment-673158037 and https://community.icinga.com/t/command-argument-will-be-handed-over-to-a-service-even-if-the-value-isn-t-set-since-icinga-v2-12/5243 complained. IMAO the feature #7558 is not worth breaking users' setups, so I've made #8174.

Ok, so I ran more tests, and I confirm that version v2.12.0-1-gade891bbf fixes the issue.

Fixed by #8174

Was this page helpful?
0 / 5 - 0 ratings