Icinga2: Add support for in-memory negation of CheckCommand results

[dnsmichi]

Expected Behavior

• In-memory negate options.
• Simple user interface, no extra functions or overloaded content
• Re-use the values of our DSL constants Ok, Warning, etc.
• Documentation in the advanced topics area or ITL.

Current Behavior

One needs to incorporate the negate plugin into the current CheckCommand which is cumbersome.

Possible Solution

Add support for built-in (in-memory) status negation. Handling should be available in all CheckCommand objects and automatically inject the check result creation in libmethods. This involves not only pluginchecktask but also our internal in-memory checks.

  vars.negate_ok = Critical
  vars.negate_critical = OK

  vars.negate_replace = "substr"

Context

@dgoetz asked for this on NET Discourse.

[Al2Klimov]

Hey guys, I just thought:

Why not to abstract this to a general check result transformation configurable via our DSL (and implemented completely via our DSL?) and this negate stuff being a concrete use case.

E.g. I've a bunch of DNSBL checks in my private setup. They regularily time out, but I don't care and would like to just turn UNKNOWN to OK.

[dnsmichi]

I've removed this from 2.11, since the argument is valid. Needs re-iteration on the implementation details.

[dnsmichi]

@dgoetz rescheduling this for later this year. This needs a better design round after 2.11 has been released.

[dnsmichi]

Something like this:

object CheckCommand "..." {

  negate_state = true
  negate_output = {{ 
    //make the original output available in this scope
   return output.replace("a", "b")
  }}
}

The scope is a problem, since you may not have access to the original objects on a command endpoint. So this can only be used as command output replacement strategy.

Just a thought from yesterday evening, I'm not sure about it yet.

[Al2Klimov]

I've got the following idea on this:

Either Checkable and/or CheckCommand have a new config attribute "transform_check_result". This shall be either not set (null) or a function which takes one parameter – the check result – and may modify it:

transform_check_result = function(cr) {
  cr.exit_status = 3 - cr.exit_status
}

[dnsmichi]

Such an "event interceptor" could make other things possible, as we know the almighty "check_snmp" whose output is far too generic.

Still, I'd like to take another round here with a design story and follow-up decisions with @lippserd and @bobapple after 2.11 is done.

[htriem]

Let's look into the negate CheckCommand implementation and then decide on further iteration on check result transformation. Discussed this with @dnsmichi offline.