Telegraf: inputs.snmp - Collection took longer than expected; not complete after interval of 10s

[chench00]

Greetings!

Not sure if this is a bug or expected behavior but I've found a couple topics similar to this subject but can't get a clear answer if this is safe to ignore. Everything works fine otherwise.

Relevant telegraf.conf:

[[processors.regex]]
  [[processors.regex.fields]]
    key = "log-dates"
    pattern = "^(?P<YYYY>\\d{4})(?P<MM>\\d{2})(?P<DD>\\d{2})(?P<HH>\\d{2})(?P<mm>\\d{2})(?P<ss>\\d{2})\\.(?P<SSSSSS>\\d{6})(?P<ZZ>[-+]\\d{3,4})$"
    replacement = "${YYYY}-${MM}-${DD} ${HH}:${mm}:${ss}"

[[inputs.snmp]]
  agents = [ "192.168.139.6:161" , "192.168.139.8:161" , "192.168.139.9:161" ]
  version = 1
  community = "public"
  name = "idrac-hosts"
  timeout = "10s"

  [[inputs.snmp.field]]
     name = "system-name"
     oid  = ".1.3.6.1.2.1.1.5.0"
     is_tag = true

  [[inputs.snmp.field]]
     name = "system-osname"
     oid  = ".1.3.6.1.4.1.674.10892.5.1.3.6.0"

  [[inputs.snmp.field]]
     name = "system-osversion"
     oid  = ".1.3.6.1.4.1.674.10892.5.1.3.14.0"

  [[inputs.snmp.field]]
     name = "system-model"
     oid  = ".1.3.6.1.4.1.674.10892.5.1.3.12.0"

  [[inputs.snmp.field]]
     name = "idrac-url"
     oid  = ".1.3.6.1.4.1.674.10892.5.1.1.6.0"

  [[inputs.snmp.field]]
     name = "power-state"
     oid  = ".1.3.6.1.4.1.674.10892.5.2.4.0"

  [[inputs.snmp.field]]
     name = "system-uptime"
     oid  = ".1.3.6.1.4.1.674.10892.5.2.5.0"

  [[inputs.snmp.field]]
     name = "system-servicetag"
     oid  = ".1.3.6.1.4.1.674.10892.5.1.3.2.0"

  [[inputs.snmp.field]]
     name = "system-globalstatus"
     oid  = ".1.3.6.1.4.1.674.10892.5.2.1.0"

 [[inputs.snmp.field]]
    name = "fan4-speed"
    oid  = ".1.3.6.1.4.1.674.10892.5.4.700.12.1.6.1.4"

 [[inputs.snmp.field]]
    name = "fan5-speed"
    oid  = ".1.3.6.1.4.1.674.10892.5.4.700.12.1.6.1.5"

 [[inputs.snmp.field]]
    name = "fan6-speed"
    oid  = ".1.3.6.1.4.1.674.10892.5.4.700.12.1.6.1.6"

 [[inputs.snmp.field]]
    name = "inlet-temp"
    oid  = ".1.3.6.1.4.1.674.10892.5.4.700.20.1.6.1.1"

 [[inputs.snmp.field]]
    name = "exhaust-temp"
    oid  = ".1.3.6.1.4.1.674.10892.5.4.700.20.1.6.1.2"

 [[inputs.snmp.field]]
    name = "cpu1-temp"
    oid  = ".1.3.6.1.4.1.674.10892.5.4.700.20.1.6.1.3"

 [[inputs.snmp.field]]
    name = "cpu2-temp"
    oid  = ".1.3.6.1.4.1.674.10892.5.4.700.20.1.6.1.4"

 [[inputs.snmp.field]]
    name = "cmos-batterystate"
    oid  = ".1.3.6.1.4.1.674.10892.5.4.600.50.1.6.1.1"

 [[inputs.snmp.field]]
    name = "system-watts"
    oid  = ".1.3.6.1.4.1.674.10892.5.4.600.30.1.6.1.3"

System info:

Telegraf 1.15.1
Ubuntu 18.04

Docker

Steps to reproduce:

1. ...Run the above configuration
2. ...Watch the logs

Expected behavior:

No warning messages

Actual behavior:

Aug 09 12:49:00 TIG telegraf[981]: 2020-08-09T16:49:00Z W! [inputs.snmp] Collection took longer than expected; not complete after interval of 10s
Aug 09 12:55:10 TIG telegraf[981]: 2020-08-09T16:55:10Z W! [inputs.snmp] Collection took longer than expected; not complete after interval of 10s
Aug 09 12:56:50 TIG telegraf[981]: 2020-08-09T16:56:50Z W! [inputs.snmp] Collection took longer than expected; not complete after interval of 10s
Aug 09 12:58:10 TIG telegraf[981]: 2020-08-09T16:58:10Z W! [inputs.snmp] Collection took longer than expected; not complete after interval of 10s
Aug 09 13:04:40 TIG telegraf[981]: 2020-08-09T17:04:40Z W! [inputs.snmp] Collection took longer than expected; not complete after interval of 10s
Aug 09 13:08:10 TIG telegraf[981]: 2020-08-09T17:08:10Z W! [inputs.snmp] Collection took longer than expected; not complete after interval of 10s
Aug 09 13:12:30 TIG telegraf[981]: 2020-08-09T17:12:30Z W! [inputs.snmp] Collection took longer than expected; not complete after interval of 10s
Aug 09 13:18:10 TIG telegraf[981]: 2020-08-09T17:18:10Z W! [inputs.snmp] Collection took longer than expected; not complete after interval of 10s
Aug 09 13:20:20 TIG telegraf[981]: 2020-08-09T17:20:20Z W! [inputs.snmp] Collection took longer than expected; not complete after interval of 10s
Aug 09 13:22:10 TIG telegraf[981]: 2020-08-09T17:22:10Z W! [inputs.snmp] Collection took longer than expected; not complete after interval of 10s

Additional info:

[ssoroka]

It sounds like one of the agents is maybe taking longer than 10s to respond? You could try setting the interval higher for the snmp input to collect less frequently. It's not really a problem (Telegraf will skip intervals if it needs to, and will not overlap collections), but the log messages might be annoying. As far as I know this is just a direct result of how long it takes to collect the data from the SNMP device.

[Hipska]

Indeed, do you really need it to be polled every 10 seconds? I usually take 60s as minimum for snmp devices.

[chench00]

@ssoroka and @Hipska You're absolutely correct and I have changed to a 60s interval as I agree that there's no need for a anything less than that. I have yet to see the message. Newbie mistake. Thank you for your help.