Kafka-node: Messages delivered repeatedly

[kryptx]

When we deployed a service depending on version 2.6.0, we noticed we were receiving messages many times. In the worst cases, thousands of times across our consumer group. These deliveries often happen (on one host) many times even within the same millisecond. They have the same topic, same partition, same offset, and same payload.

Number of brokers, number of partitions seems irrelevant. Reverting to kafka-node 2.3.0 alleviated the issue.

I don't have debug output or repro steps, but I can probably get them if you can't think of any way this might happen.

Environment

• Node version: 6.13.1
• Kafka-node version: 2.6.0
• Kafka version: 0.9.0

[ufmemo]

Same here... spent two days looking at this issue... fixed by reverting to 2.5.0.

[kunle-e]

I've struggled with this for many hours. Went back to an older version and the problem went away.

[anajavi]

Same thing.

Environment:

• Node version: 8.11.1
• Kafka-node version: 2.6.0
• Kafka version: 1.0.1

[hyperlink]

Looks like this is caused by V2 fetch changes. The issue reveals itself when connecting to multiple brokers and unfortunately our tests only connects to one broker.

I'm reverting that commit and publish it as patch release shortly.

[philipp-eisen]

+1

[enjine]

confirmed. locking to version 2.5.0 restores expected behavior. When can we expect the patch to be available?

[hyperlink]

It's published as 2.6.1.

[anajavi]

2.6.1 works for me

[hyperlink]

@enjine @ph1l337 @kunle-e @ufmemo @kryptx have 2.6.1 resolve the issue?

[kryptx]

Yes, 2.6.1 works for me too.