Windows_exporter: Add configuration file for collectors flags

[sbakharev]

Collectors' flags can only be configured via command line arguments. And some of the collectors would need them customized for optimisation purposes (minimizing metrics series count) in almost any usage scenario - process and service collectors are the first candidates. In a perfect world you would configure the flags only once during installation but in real life things tend to change sometimes. Meanwhile the resultant complete command line string with flags may be quite long - up to almost 700 bytes in my case.

When you need to change the flags you basically have two options:

• Reinstall the exporter with new flags values
• Change the command line arguments of exporter service manually with regedit or with a script of some sort (PowerShell for example) and restart the service

Both ways are painful. Reinstalling may require involving administator in case you don't have sufficient rights at the server, changing the command line arguments of a service is simply inconvenient, and in both cases you have to deal with escaping characters (in a potentially very long string) since many of the flags are regular expressions or just may contain braces and other special chars.

Things would be way simpler if we could store flags in a good old key-value configuration file like this:

collector.iis.app-whitelist=AppPool_1_Name|AppPool_2_Name|AppPool_3_Name
collector.iis.site-whitelist=Site_1_Name
collector.service.services-where=Name='Service_1_Name' OR Name='Service_2_Name'
collector.logical_disk.volume-whitelist=C\:

P.S. Thanks to all the community for the tool. Despite some minor performance issues we are quite satisfied with it :)

[16srana]

:top: much-needed feature, especially when you spend half a day struggling just to identify how many quotes to use #589

[muxto]

+1

[Yorikup]

+1

[xxMRIxx]

+1

[NePS-007]

I don't like when you have to work painfully, so +1

[carlpett]

We've previously rejected this on grounds keeping things simple. But perhaps it needs to be revisited if real use cases require 700 characters of commandline, which can hardly be called simple. The reconfiguration aspect is also a frequent complaint which would be resolved this way.

So, we'll need to figure out how this should work technically and make this play nice with the existing codebase, as well as keeping backwards compatibility. I'd still expect commandline configuration to be the primary method to use, and it cannot break during an upgrade.
I'll try to find some time for this, but it might take a while. If anyone is willing to start working on it, let me know and I can provide some guidance.

[breed808]

@carlpett I'd be keen to start working on it. Did you have any requirements in mind?

[carlpett]

@breed808 The primary thing would be making sure it integrates with the current CLI handling seamlessly. I wrote code in Kingpin, the library we use, for this a couple of years ago, but it remains on the v3/unstable branch. I know the Alertmanager folks did a local adaptation of that for the amtool cli, but haven't looked that closely on exactly how it works. This seems like a good place to start.

Once that works we can talk about the file format etc, but getting this integration in place is the crucial bit to making sure we don't have endless bugs from maintaining flags and config files in parallel.

[sdwire]

I'm thrilled to see this discussion happening. Installation of the exporter is pretty much a one-time event, and we'd like to see it applied uniformly across every VM we deploy. Configuring the exporter, however, with choices on which processes and services should have their detailed metrics exported, is something that changes over time as we learn and as we change the software on the VM.

Personally, I would hope that this would work in such a way that a) a config file can include other config files and b) changes in the config files will be honored during the next scrape - without needing to restart the exporter.

Thank you for being interested in solving this.

[breed808]

I've created an initial implementation in the linked pull request. This will allow users to specify any existing kingpin flags in a configuration file (default location %programfiles%\windows_exporter\config.yml).
Once we're comfortable with the integration, we can look into implementing child configs and automatic reloading of configs.

[fischerman]

@breed808 Awesome work so far!

I was wondering how I would manage the config files. We're are using SCCM for the deployment of the .msi file. Every change is fairly expensive and rollout is very slow. I was thinking of using GPOs (still widely used as I've recently rediscovered) to distribute the config files. They can simply drop a file in some directory and the windows_exporter picks up the changes. To that end, it would be awesome to have drop-in configurations. We currently have the use-case that we need to monitor a particular process on a set of machines for performance analyses, meaning enabling the process collector and setting the whitelist. Being able to do that on top of the generic configuration would be great.

I'm keen to hear how everyone plans to manage the config files.

[sbakharev]

Just came here to say that we've upgraded to the new version with the config file support and it's just totally awesome. Thanks a lot to everyone who made this happen :)