Telegraf: OpenTSDB does not sanitize all required characters

Created on 28 Jul 2017 · 3Comments · Source: influxdata/telegraf

Bug report

If metric name or tag name has the character "[", the tsdb won't parse i.e we will lose that mertic.

Relevant telegraf.conf:

[agent]
  interval = "5s"
  round_interval = true
  metric_buffer_limit = 1000
  flush_interval = "5s"
  quiet = false
  hostname = ""

[[outputs.opentsdb]]
  host = "tcp://<HOST>"
  port = 4343

[[inputs.win_perf_counters]]
  [[inputs.win_perf_counters.object]]
    Instances = ["*"]
    Counters = [
      "% Idle Time",
      "% Interrupt Time",
      "% Privileged Time",
      "% User Time",
      "% Processor Time",
    ]
    Measurement = "win_cpu"
    IncludeTotal=true

  [[inputs.win_perf_counters.object]]
    ObjectName = "LogicalDisk"
    Instances = ["*"]
    Counters = [
      "Free Megabytes",
      "Disk Reads/sec",
      "Disk Writes/sec",
      "Disk Bytes/sec",
      "% Idle Time",
      "Current Disk Queue Length",
    ]
    Measurement = "win_disk"

  [[inputs.win_perf_counters.object]]
    ObjectName = "Network Interface"
    Counters = [
      "Bytes Sent/sec",
      "Bytes Received/sec",
    ]
    Instances = ["*"]
    Measurement = "win_network"

  [[inputs.win_perf_counters.object]]
    ObjectName = "Memory"
    Counters = [
      "Available Bytes",
      "Committed Bytes",
      "Cache Bytes",
      "Pages/sec",
      "Pool Nonpaged Bytes",
      "Pool Paged Bytes",
    ]
    Instances = ["------"]
    Measurement = "win_mem"

System info:

Windows 10, Telegraf v1.3.5 (Downloaded exe from https://portal.influxdata.com/downloads)

Steps to reproduce:

Set the output to OpenTSDB (Telnet Mode)

[[outputs.opentsdb]]
  host = "tcp://<HOST>"
  port = 4343

The windows performance counters names must have the character "[",. For example, my laptops network interface counter name is Broadcom_NetLink_[TM]_Gigabit_Ethernet which has "[". (You can get the counter names with Get-Counter command in windows power shell)
Start the Telegraf Windows Service (Or run in Windows power shell) and capture traffic with wireshark to see what Telegraf is sending and what it is getting response from tsdb server.

What I saw in my capture is

put: illegal argument: Invalid tag value ("Broadcom_NetLink_[TM]_Gigabit_Ethernet"): illegal character: [

and this is the metric, it is trying to send

win_network_Bytes_Received_persec <ts> <value> host=utron instance=Broadcom_NetLink_[TM]_Gigabit_Ethernet objectname=Network_Interface

Expected behavior:

It should send metrics even if it has any special character. In code, it should replace the "[" with some other

Proposal:

In code, it should replace the "[" with some other

areopentsdb bug

Source

veerendra2

All 3 comments

FWIW, I have the same problem with the parentheses characters "(" and ")".

The OpenTSDB validateString() logic allows the following:

(('a' <= c && c <= 'z') 
   || ('A' <= c && c <= 'Z')
   || ('0' <= c && c <= '9') 
   || c == '-' 
   || c == '_' 
   || c == '.'
   || c == '/' 
   || Character.isLetter(c) 
   || isAllowSpecialChars(c)))

(where isAllowSpecialChars() allows characters from tsd.core.tag.allow_specialchars, which is empty by default)