Kafka-node: apiSupport uncaught exception when a node disconnects and broker.apiSupport comes empty

[jemapi]

Hi,

On high load we're getting an uncaught error:

apiSupport is empty

That is related to: https://github.com/SOHU-Co/kafka-node/issues/750, it's suppoused to be fix but we're still experiencing some problems, we had a look at the code and we think that in kafkaClient.js file for the sendRequest method you shoud try / catch this when you call the sendToBroker method, something like this

... try { sendToBroker(payload, leader, callback); } catch (ex) { callback(ex); } ...

Bug Report

Environment

• Node version: 8.11.1
• Kafka-node version: 2.6.1
• Kafka version: 0.11

For specific cases also provide

• Number of Brokers: 3
• Number partitions for topic: 3

Include Sample Code to reproduce behavior

Just gave load to a KafkaClient producing messages and cut off some broker. You should see the uncaught error.

[hyperlink]

Can you attach the stack trace for this error?

[jemapi]

Sure, here it is:

04062018@04:26:37 Error: AssertionError [ERR_ASSERTION]: apiSupport is empty
    at getSupportedForRequestType (/opt/app/node_modules/kafka-node/lib/kafkaClient.js:717:3)
    at async.ensureAsync (/opt/app/node_modules/kafka-node/lib/kafkaClient.js:765:19)
    at /opt/app/node_modules/kafka-node/node_modules/async/dist/async.js:3255:12
    at /opt/app/node_modules/kafka-node/node_modules/async/dist/async.js:74:12
    at /opt/app/node_modules/kafka-node/lib/kafkaClient.js:801:13
    at /opt/app/node_modules/kafka-node/node_modules/async/dist/async.js:3866:24
    at replenish (/opt/app/node_modules/kafka-node/node_modules/async/dist/async.js:998:17)
    at iterateeCallback (/opt/app/node_modules/kafka-node/node_modules/async/dist/async.js:983:17)
    at /opt/app/node_modules/kafka-node/node_modules/async/dist/async.js:958:16
    at /opt/app/node_modules/kafka-node/node_modules/async/dist/async.js:3871:13
    at /opt/app/node_modules/kafka-node/node_modules/async/dist/async.js:3249:30
    at Immediate.<anonymous> (/opt/app/node_modules/kafka-node/node_modules/async/dist/async.js:119:16)
    at runCallback (timers.js:781:20)
    at tryOnImmediate (timers.js:743:5)
    at processImmediate [as _immediateCallback] (timers.js:714:5)

[HaraldKorneliussen]

Is there anything new on this issue? I get the same error message if kafka hasn't fully started when I try to make a KafkaClient:

AssertionError [ERR_ASSERTION]: apiSupport is empty
    at getSupportedForRequestType ([Project prefix]\node_modules\kafka-node\lib\kafkaClient.js:942:3)
    at async.series.cb ([Project prefix]\node_modules\kafka-node\lib\kafkaClient.js:878:29)
    at [Project prefix]\node_modules\async\dist\async.js:3880:24
    at replenish ([Project prefix]\node_modules\async\dist\async.js:1011:17)
    at iterateeCallback ([Project prefix]\node_modules\async\dist\async.js:995:17)
    at [Project prefix]\node_modules\async\dist\async.js:969:16
    at [Project prefix]\node_modules\async\dist\async.js:3885:13
    at KafkaClient.onReady ([Project prefix]\node_modules\kafka-node\lib\kafkaClient.js:959:5)
    at Object.onceWrapper (events.js:273:13)
    at KafkaClient.emit (events.js:182:13)

I tried manually adding @jemapi 's try-catch, but that didn't make a difference. What's the right way to make sure the connection is ready for it before trying to make a KafkaClient? (Also related, what's the recommended way of avoiding crashing if the connection goes down?)

[SergeyCherman]

Same issue here, able to reproduce by producing a message after taking kakfa/zookeeper down.

[chrisandrews7]

Similar issue here, able to reproduce by trying to connect to kafka/zookeeper when its down.

Looks to be caused when connecting calls loadMetadataForTopics() which fails to handle an error thrown by getSupportedForRequestType().

Have tested changing the loadMetadataForTopics() to prove the idea, and it works:

try {
  var correlationId = this.nextId();
  var supportedCoders = getSupportedForRequestType(broker, 'metadata');
  var request = supportedCoders.encoder(this.clientId, correlationId, topics);

  this.queueCallback(broker.socket, correlationId, [supportedCoders.decoder, cb]);
  broker.write(request);
} catch (err) {
  callback(err);
}   

[hyperlink]

I was able to reproduce this reliably when KafkaClients connects to a broker that's not 100% initialized. When this happens the initial socket connection made but is then closed abruptly.

[Manjukb]

this issue still exists i keep getting apiSupport is empty if broker is not available

kafka-node : 4.0.2

[Manjukb]

still issue is not solved now in 4.1.3 ?? plz respond any workaround ???

[hyperlink]

@Manjukb what version of Kafka are you using?

[markoscalderon]

Still happening in 4.1.3

[rorysavage77]

Yes, I can confirm this is still happening in 4.1.3 and is a serious problem.

Kafka: kafka-2.11
NodeJS: v8.9.3

[rorysavage77]

8 Jul 21:51:57 - [red] Uncaught Exception:
8 Jul 21:51:57 - { AssertionError [ERR_ASSERTION]: apiSupport is empty
at getSupportedForRequestType (/usr/src/node-red/node_modules/kafka-node/lib/kafkaClient.js:974:3)
at async.series.cb (/usr/src/node-red/node_modules/kafka-node/lib/kafkaClient.js:911:33)
at /usr/src/node-red/node_modules/async/dist/async.js:3880:24
at replenish (/usr/src/node-red/node_modules/async/dist/async.js:1011:17)
at iterateeCallback (/usr/src/node-red/node_modules/async/dist/async.js:995:17)
at /usr/src/node-red/node_modules/async/dist/async.js:969:16
at /usr/src/node-red/node_modules/async/dist/async.js:3885:13
at KafkaClient.onReady (/usr/src/node-red/node_modules/kafka-node/lib/kafkaClient.js:998:5)
at Object.onceWrapper (events.js:313:30)
at emitNone (events.js:106:13)
generatedMessage: false,
name: 'AssertionError [ERR_ASSERTION]',
code: 'ERR_ASSERTION',
actual: false,
expected: true,

[rorysavage77]

So this issue is still occurring. Basically, when we have new machines come online, and once their kafka client emits that they are "Ready", we sometimes see this error. It only happens during initial Kafka startup, so this is in line with the observations mentioned above.

Kafka Node Version: 5.0.0
NodeJS: v8.9.3
Kafka Version: 2.11

[rorysavage77]

I opened a fresh new bug.

[sashadu]

Happens for me as well,
Kafka Node Version: 5.0.0
NodeJS: v13.0.1
Kafka Version: 2.5.0