Windows_exporter: Windows Services - Report against Degraded State

[pete-leese]

Hi,

How can I query windows services in a "degraded" state? Nagios is able to do this which I am trying to replace with Prometheus :) Not found any values that would tell me a windows service is running in a degraded status.

Thanks :)

Pete

[carlpett]

Well, I guess the question is what Nagios means by "degraded"? This is, as far as I am aware, not a state which Windows itself has. So Nagios must be combining multiple points of information?

[pete-leese]

https://msdn.microsoft.com/en-us/library/aa394418(v=vs.85).aspx

[martinlindhe]

Okay, this should be easy to implement.

We already have collector/service.go exporting some values from Win32_Service.

We just need to extend it to also include the value of Status:

    Data type: string
    Access type: Read-only
    Qualifiers: MaxLen (10), DisplayName ("Status")

    Current status of the object. Various operational and nonoperational statuses can be defined.
Operational statuses include: "OK", "Degraded", and "Pred Fail" (an element, such as a SMART-
enabled hard disk drive, may be functioning properly but predicting a failure in the near future).
Nonoperational statuses include: "Error", "Starting", "Stopping", and "Service". The latter, "Service",
could apply during mirror-resilvering of a disk, reload of a user permissions list, or other administrative
work. Not all such work is online, yet the managed element is neither "OK" nor in one of the other
states.

[pete-leese]

That would be great.

We have a few in house services that randomly fall over but still appear to be running.

Historically we see this from the status of the service with what nagios is reporting so we can take the appropriate action.

Cheers

Pete

[martinlindhe]

I feel helping to reduce number of nagios installs is a good cause. I have had my share of troubles with nagios in my days, too.

[carlpett]

Ah, I misunderstood. I was thinking about State, not Status. Obviously different things (naming things is hard, right?).
The cardinality of the service metrics will get pretty high if we add all the listed ~states~ statuses, though, and I'm not sure how many of them overlap with the state already. Should we start with just a gauge wmi_service_degraded{name="<service-name>"} 0/1, or is there value in having the others too?

[pete-leese]

I would see the behaviour inline with how Service state operates.

example:

wmi_service_status{name=",status="OK"} 0
wmi_service_status{name=",status="Degraded"} 1
wmi_service_status{name=",status="Error"} 0

Possible values:

OK ("OK")
Error ("Error")
Degraded ("Degraded")
Unknown ("Unknown")
Pred Fail ("Pred Fail")
Starting ("Starting")
Stopping ("Stopping")
Service ("Service")
Stressed ("Stressed")
NonRecover ("NonRecover")
No Contact ("No Contact")
Lost Comm ("Lost Comm")

[carlpett]

Yes, that is the other option. It does increase the number of metrics rather a lot, though, so I just wanted to have the discussion on if the data is valuable, or if it largely mirrors the service_state metric anyway.

[pete-leese]

Is there anything further that needs discussion before a decision can be made on how this is implemented ? Cheers Pete

[pete-leese]

Are there any plans for this being in the 0.3.0 release? Just trying to plan ahead updating of the exporter.

[carlpett]

It was not included in 0.3.0 (which was released on Friday last week). When I or someone else can find time to implement it, it will get included in the next version.

[pete-leese]

Looking forward to this being unplanted.

My nagios monitoring is all but decommissioned and only monitoring sub service states.

Would love to be able turn it off :)