Node_exporter: Proposal: Use shell commands to extend node_exporter

[idweball]

Extending the node_exporter collector requires writing golang code, which is a very difficult task for an operation and maintenance worker who is not familiar with golang.

Is it possible to add a shell-type collector to collect metrics by executing shell commands? Such as: https://github.com/ozhiwei/shell_exporter

My English is too bad. This proposal uses Google Translate, hoping to understand what I mean.

[paulfantom]

Since there is already ozhiwei/shell_exporter why not use it instead of adding complexity to node_exporter? If not, you can always run shell scripts alongside and use textfile collector to ingest data from those.

[SuperQ]

We explicitly have a policy against using external processes in the node_exporter for a variety of safety reasons.

What I have thought about is adding a helper tool to make managing and running textfile helper scripts easier.

A config file like this:

global:
  interval: 1m
  timeout: 10s
scripts:
- cmd: /path/to/textfile_helper.sh
  interval: 5m
  timeout: 1m
  file: textfile.prom

Each script would be managed by a goroutine, and the stdout would be sent atomically to the textfile.

[roidelapluie]

That would be great, especially if that tool can fork as different users to execute processes, but still write files with the running user.

[idweball]

From a security perspective, adding a help tool to the textfile to replace the shell-type collector may be a good choice.

In this way, system administrators can also easily expand the ability of node_export to collect metrics. No need to use crontab and textfile as before. After all, crontab can only be accurate to the minute level.

Of course, it is better to have a shell type collector.

[discordianfish]

I'm not sure we should promote using the textfile collector. It's breaking the pull based model. There are few cases where I would prefer the textfile collector over direct instrumentation, an exporter or the pushgateway for batch stuff. Most uses of it are indeed probably better solved by using a shell exporter/collector..

The question is just whether to build that into the node-exporter. In general an exporter is intended to export 'one thing' which is 'node/host level metrics' in case of the node-exporter. A shell script could gather any kind of metrics and maybe people will abuse that to expose app level metrics. But then again, there are a lot of site specific node level metrics for which having a shell collector in node-exporter seems like the right fit. I'm also not that much worried about "safety", we can just do a similar locking like we do in the statfs collector to prevent leaking processes..