Telegraf: how to use data_foramt on two types of values MQTT addon

Created on 25 Sep 2019  路  3Comments  路  Source: influxdata/telegraf

say i have two topics, one is string one is float
how do i configure the [[inputs.mqtt_consumer]] to give me both?

i am getting the below error
2019-09-25T12:11:10Z E! [outputs.influxdb] when writing to [http://localhost:8086]: received error partial write: field type conflict: input field "value" on measurement "mqtt_consumer" is type string, already exists as type integer dropped=1; discarding points

when i try it my way
```

#

INPUTS

#

Windows Performance Counters plugin.

These are the recommended method of monitoring system metrics on windows,

as the regular system plugins (inputs.cpu, inputs.mem, etc.) rely on WMI,

which utilize more system resources.

#

See more configuration examples at:

https://github.com/influxdata/telegraf/tree/master/plugins/inputs/win_perf_counters

[[inputs.mqtt_consumer]]
## MQTT broker URLs to be used. The format should be scheme://host:port,
## schema can be tcp, ssl, or ws.
servers = ["tcp://127.0.0.1:1883"]

## Topics that will be subscribed to.
topics = [
"/XXX",
]
data_format = "value"
data_type = "integer" # required

[[inputs.mqtt_consumer]]
## MQTT broker URLs to be used. The format should be scheme://host:port,
## schema can be tcp, ssl, or ws.
servers = ["tcp://127.0.0.1:1883"]

## Topics that will be subscribed to.
topics = [
"/SSS",
]
data_format = "value"
data_type = "string" # required
```

i am trying to get all values back from MQTT but i cannot do only
data_format = "value" data_type = "string" # required

Most helpful comment

You have a few options, example configs below are untested and might need tweaked.

You can have multiple plugins, one for each type and use name_override to vary the measurement name:

[[inputs.mqtt_consumer]]
  name_override = "my_integer"
  topics = ["/XXX"]
  data_format = "value"
  data_type = "integer"

[[inputs.mqtt_consumer]]
  name_override = "my_string"
  topics = ["/SSS"]
  data_format = "value"
  data_type = "string"

You can have a single plugin reading all values as strings, and rename the fields and convert the integers using processors:

[[inputs.mqtt_consumer]]
  topics = ["/#"]
  data_format = "value"
  data_type = "string"

[[processors.rename]]
  order = 1
  [processors.rename.tagpass]
    topic = ["/SSS"]

  [[processors.rename.replace]]
    field = "value"
    dest = "my_string"

[[processors.rename]]
  order = 2
  [processors.rename.tagpass]
    topic = ["/XXX"]

  [[processors.rename.replace]]
    field = "value"
    dest = "my_integer"

[[processors.converter]]
  order = 3
  [processors.converter.tagpass]
    topic = ["/XXX"]

  [[processors.converter.fields]]
    integer = ["my_integer"]

Last "option" is to write a format that can be converted automatically on read, such as InfluxDB Line Protocol which will give you the most control over how the data looks. I recommend this if it is possible to modify the incoming data:
```toml
[[inputs.mqtt_consumer]]
topics = ["/#"]
data_format = "influx"

All 3 comments

You have a few options, example configs below are untested and might need tweaked.

You can have multiple plugins, one for each type and use name_override to vary the measurement name:

[[inputs.mqtt_consumer]]
  name_override = "my_integer"
  topics = ["/XXX"]
  data_format = "value"
  data_type = "integer"

[[inputs.mqtt_consumer]]
  name_override = "my_string"
  topics = ["/SSS"]
  data_format = "value"
  data_type = "string"

You can have a single plugin reading all values as strings, and rename the fields and convert the integers using processors:

[[inputs.mqtt_consumer]]
  topics = ["/#"]
  data_format = "value"
  data_type = "string"

[[processors.rename]]
  order = 1
  [processors.rename.tagpass]
    topic = ["/SSS"]

  [[processors.rename.replace]]
    field = "value"
    dest = "my_string"

[[processors.rename]]
  order = 2
  [processors.rename.tagpass]
    topic = ["/XXX"]

  [[processors.rename.replace]]
    field = "value"
    dest = "my_integer"

[[processors.converter]]
  order = 3
  [processors.converter.tagpass]
    topic = ["/XXX"]

  [[processors.converter.fields]]
    integer = ["my_integer"]

Last "option" is to write a format that can be converted automatically on read, such as InfluxDB Line Protocol which will give you the most control over how the data looks. I recommend this if it is possible to modify the incoming data:
```toml
[[inputs.mqtt_consumer]]
topics = ["/#"]
data_format = "influx"

last one sounds great but i am not able to make it work
i am getting below error after i am connected to the broker

2019-10-07T05:22:35Z I! [agent] Config: Interval:10s, Quiet:false, Hostname:"QA-GAD", Flush Interval:10s 2019-10-07T05:22:35Z I! [inputs.mqtt_consumer] Connected [tcp://127.0.0.1:1883] 2019-10-07T05:23:24Z E! [inputs.mqtt_consumer] Error in plugin: metric parse error: expected tag at 1:2: "9" 2019-10-07T05:23:25Z E! [inputs.mqtt_consumer] Error in plugin: metric parse error: expected tag at 1:4: "Gad" 2019-10-07T05:23:26Z E! [inputs.mqtt_consumer] Error in plugin: metric parse error: expected tag at 1:2: "9"

`  [[inputs.mqtt_consumer]]
 ## MQTT broker URLs to be used. The format should be scheme://host:port,
 ## schema can be tcp, ssl, or ws.
 servers = ["tcp://127.0.0.1:1883"]
  topics = ["/#"]
  data_format = "influx"
  `

what did you mean when you said if i can change my data?

In order to use the last example you would need to write data in InfluxDB Line Protocol format. The linked page shows all the details, but the most basic integer and string examples are:

my_measurement my_integer=42i
my_measurement my_string="my string"
Was this page helpful?
0 / 5 - 0 ratings