Telegraf: `socket_listener` unable to decode compressed TCP payloads

[quikkian]

Relevant telegraf.conf:

[[inputs.socket_listener]]
  service_address = "tcp://:8096"
  data_format = "influx"
  content_encoding = "gzip"

System info:

Telegraf: Telegraf 1.14.0 (git: HEAD fefd7ff1)
OS: Windows Server 2016 Standard - Version 1607 build 14393.3630

Telegraf installed as a Windows Service by telegraf.exe

Docker

Steps to reproduce:

1. Start a Telegraf instance listening with socket_listener with the provided configuration to listen for TCP connections with gzip compressed payloads
2. Write compressed line protocol messages to the TCP socket that Telegraf is listening on
   
   
   
   1. This can be done with the build artifact from abandoned PR 7355
   
   

Expected behavior:

Telegraf can decode and consume line protocol messages compressed with gzip on a TCP socket

Actual behavior:

Telegraf unable to decode and consume line protocol messages compressed with gzip on a TCP socket

Additional info:

Log file messages:

2020-04-20T20:13:46Z I! [inputs.socket_listener] Listening on tcp://[::]:8096
2020-04-20T20:14:50Z E! [inputs.socket_listener] Unable to decode incoming line: unexpected EOF
2020-04-20T20:14:50Z E! [inputs.socket_listener] Unable to decode incoming line: gzip: invalid header
...

Telegraf_CompressedTCP.pcapng.zip

[danielnelson]

Quick look at the code, I think the UDP side is working, but not TCP. It uses line based message framing, but that would require textual encoding of the input, gzip data would only work if it happens not to contain any newline bytes.

To fix this, we would have to provide control over message framing with the TCP sockets, for example a message could have a header with the byte length or we could use byte stuffing. This would need to be mirrored on the output side in order to round trip.

related #6025

[m4ce]

Having the same issue.. just spent hours debugging it..

[m4ce]

@danielnelson, any chance this could be fixed? I think having a header with the data length is doable?

[danielnelson]

The configurable message framing would work for sure, but one of my colleagues pointed out that it might be possible to support gzip+tcp without it. The content decoder we are using now is batch based, instead we would need to create a new version that supports gzip/gunzip in a streaming manner.

I guess you are probably wondering if I will actually fix this soon, let me try the streaming method today and if that works then I can get it. If we need to do the message framing method I will have to leave it for the community to address (only update the docs for now).

[m4ce]

Thanks @danielnelson !

[danielnelson]

I'm putting the fix in for 1.15.0 since I feel it ended up being a bit too much for a patch release.