Logstash: The kafka consumer clientid is suffixed by 0,1,2...

Created on 23 Aug 2018  路  6Comments  路  Source: elastic/logstash

using kafka input plugin, I set client_id=d9f37fcb and consumer_threads => 3

[org.apache.kafka.clients.FetchSessionHandler] [Consumer clientId=d9f37fcb-0, groupId=default_logstash1535012319052] Error sending fetch request (sessionId=INVALID, epoch=INITIAL) to node 99135: org.apache.kafka.common.errors.DisconnectException.

[org.apache.kafka.clients.FetchSessionHandler] [Consumer clientId=d9f37fcb-1, groupId=default_logstash1535012319052] Error sending fetch request (sessionId=INVALID, epoch=INITIAL) to node 99135: org.apache.kafka.common.errors.DisconnectException.

[org.apache.kafka.clients.FetchSessionHandler] [Consumer clientId=d9f37fcb-2, groupId=default_logstash1535012319052] Error sending fetch request (sessionId=INVALID, epoch=INITIAL) to node 99135: org.apache.kafka.common.errors.DisconnectException.

Most helpful comment

same issue, but not automatically recover as @SimplyAmuthan said. Kafka clients will stop consuming and need to restart

All 6 comments

could I set it without changement?

@yubobo I am experiencing the same problem with, how did you resolve this? could you please help me fix this problem

Thank you
Regards
Amuthan

Error sending fetch request (sessionId=INVALID, epoch=INITIAL) to node 3: org.apache.kafka.common.errors.DisconnectException.
I met this exception with logstash 6.5.3 too, are there any solution for it?

Actually, it is an internal error and not at all a blocker, if you notice closely it is INFO, Kafka recover it from automatically.

I have seen it too.
Error sending fetch request (sessionId=INVALID, epoch=INITIAL) to node 2: org.apache.kafka.common.errors.DisconnectException
When i used consumer.commitAsync(); or consumer.commitSync();
How can i fix it?THX
The kafka version is 2.2.4
The SpringBoot version is 2.1.3Release

same issue, but not automatically recover as @SimplyAmuthan said. Kafka clients will stop consuming and need to restart

Was this page helpful?
0 / 5 - 0 ratings