Telegraf: Prometheus output replaces "/" to "_"

[Bregor]

Bug report

Starting from release 1.3.2 there is function sanitize in package prometheus_client which replaces anything but ^a-zA-Z0-9_ to _.
/ is not included in invalidNameCharRE, so all filesystem mounts looks like _var_lib_postgresql which is confusing.

Relevant telegraf.conf:

[inputs.diskio]
[[inputs.disk]]
ignore_fs = ["tmpfs", "devtmpfs"]

[outputs.prometheus_client]
listen = ":9126"

System info:

$ telegraf --version
Telegraf v1.3.2 (git: release-1.3 0f419e9a0d294ab4d68bb7d1c6b28e60d362239b)
$ uname -r
4.8.0-54-generic

Steps to reproduce:

1. Upgrade telegraf to 1.3.2
2. Voila!

Expected behavior:

To return / as /

Actual behavior:

All slashes replaced with underscores

[freeseacher]

got the same behavior.
my metrics has been doubled

[Bregor]

By digging deeper I realised, sanitization rules for _labels_ are used to sanitize _metric_ names.
Following expressions are identical:

• https://github.com/prometheus/prometheus/blob/master/util/strutil/strconv.go#L23
• https://github.com/influxdata/telegraf/blob/master/plugins/outputs/prometheus_client/prometheus_client.go#L20

And also sanitization methods:

• https://github.com/prometheus/prometheus/blob/master/util/strutil/strconv.go#L42-L44
• https://github.com/influxdata/telegraf/blob/master/plugins/outputs/prometheus_client/prometheus_client.go#L170-L172

In Prometheus these rules are only for _labels_, but Telegraf uses them for _metric names_ sanitization, which is terrible wrong.

[danielnelson]

Looking over the changes and the previous version, it looks that the difference is that we are now applying the sanitize function label values. Metric names were sanitized in the previous Telegraf release.

Here are the allowed values for metrics names and labels. Technically we should allow : in the metric name as well, but this would be a breaking change so I won't do it just yet.

I'll link to a pull request soon removing the sanitization of label values.