Kafka-node: ConsumerGroup : TimeoutError: Request timed out after 30000ms

[CoolSmiley]

Questions?

I have a ConsumerGroup making use of the new kafkaHost option to connect to brokers directly. At runtime, I'm observing this error emitted out every 30 seconds

TimeoutError: Request timed out after 30000ms
at new TimeoutError (...../node_modules/kafka-node/lib/errors/TimeoutError.js:6:9)
at Timeout.setTimeout ...../node_modules/kafka-node/lib/kafkaClient.js:385:14)
at ontimeout (timers.js:386:11)
at tryOnTimeout (timers.js:250:5)
at Timer.listOnTimeout (timers.js:214:5) message: 'Request timed out after 30000ms' }

Irrespective of this error, I can see the consumer fetch cycle complete and messages getting consumed.

What exactly is this requestTimeout option when using the new KafkaClient ?

Bug Report

Environment

• Node version: 6.12.x
• Kafka-node version: 2.3.1
• Kafka version: 1.0.0

For specific cases also provide

• Number of Brokers: 2
• Number partitions for topic:

Include Sample Code to reproduce behavior

let options = {
            kafkaHost: "localhost:9092,localhost:9093",
            groupId: ...,
            fetchMaxWaitMs: ....,
            fetchMinBytes: ...,
            fetchMaxBytes: ...,
        };
kafkaConsumer = new ConsumerGroup(options, kafkaTopic);

Include output with Debug turned on

Thanks for your contribution!

[hyperlink]

This error is emitted if there's has been no response from the request made to the broker within that time. Unfortunately there's no context so it's unknown what API request timed out. It could be a great future enhancement.

Also you can also set requestTimeout to false to disable this feature entirely.

[emmaccacca]

If set requestTimeout to false, you will get
{ TimeoutError: Request timed out after falsems
at new TimeoutError (/usr/src/app/node_modules/kafka-node/lib/errors/TimeoutError.js:6:9)
at Timeout.setTimeout [as _onTimeout] (/usr/src/app/node_modules/kafka-node/lib/kafkaClient.js:737:14)
at ontimeout (timers.js:475:11)
at tryOnTimeout (timers.js:310:5)
at Timer.listOnTimeout (timers.js:270:5) message: 'Request timed out after falsems' }

[hyperlink]

Double check a boolean false is being passed in the options. We have code that checks for it.

[CoolSmiley]

If you do use the requestTimeout feature, make sure to set the value of requestTimeout greater than fetchMaxWaitMs.

[skumar568]

@CoolSmiley Where we need to set fetchMaxWaitMs?

[Talgatio]

Have same issue. When I set requestTimeout: false
return TimeoutError: Request timed out after falsems
@hyperlink Can you help me?

[Talgatio]

@hyperlink This exception called from waitUntilReady method 990 line in kafkaClient.js file.
There is no check on false

[Talgatio]

@hyperlink This code snippet solves my problem
https://pastebin.com/CXBcdLQQ
But I can't create pr for this

[hyperlink]

@hyperlink This code snippet solves my problem
https://pastebin.com/CXBcdLQQ
But I can't create pr for this

It won't work. You can't bypass the ready event. The broker socket needs to get the API version before accepting with any other requests.

[Talgatio]

how I can fix it from my side?

[Talgatio]

https://www.npmjs.com/package/kafka-schema-registry
this is my npm module for work with kafka-node

[Talgatio]

in 1001 line you get const timeout = this.options.requestTimeout;
It's ok?

[hyperlink]

in 1001 line you get const timeout = this.options.requestTimeout;

This line? I don't see an issue with this line.

[specialgirlgotoheaven]

do the question solved ?I meet the same problem.Can you help me ? @CoolSmiley @emmaccacca @Talgatio @skumar568 @hyperlink