Librdkafka: Facing issue while creating SSL connection between librdkafka client and KAFKA server

Created on 12 Feb 2018  路  3Comments  路  Source: edenhill/librdkafka

We followed the steps from the following link to create SSL connection.

https://github.com/edenhill/librdkafka/wiki/Using-SSL-with-librdkafka

Description

when librdkafka client is trying to connect to KAFKA server, we are getting following error in consumer trace
% ERROR CALLBACK: kcea#consumer-1: error: -181 ssl://abc.abc.aa.bb:9092/bootstrap: SSL handshake failed: SSL syscall error

where following error is excerpt from Kafka server.

[2018-02-12 20:12:55,333] WARN Unexpected error from /abc.abc.abc.aa; closing connection (org.apache.kafka.common.network.Selector)
org.apache.kafka.common.network.InvalidReceiveException: Invalid receive (size = 369295617 larger than 104857600)
at org.apache.kafka.common.network.NetworkReceive.readFromReadableChannel(NetworkReceive.java:95)
at org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java:75)
at org.apache.kafka.common.network.KafkaChannel.receive(KafkaChannel.java:203)
at org.apache.kafka.common.network.KafkaChannel.read(KafkaChannel.java:167)
at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:379)
at org.apache.kafka.common.network.Selector.poll(Selector.java:326)
at kafka.network.Processor.poll(SocketServer.scala:499)

could you please let us know if we are missing some steps here.

librdkafka version - librdkafka-0.11.0
Kafka server version - kafka_2.11-0.11.0.0

question

Most helpful comment

Looks like you tried to connect to a non-SSL listener.
Verify that 9092 is the SSL listener port on the broker.

All 3 comments

Looks like you tried to connect to a non-SSL listener.
Verify that 9092 is the SSL listener port on the broker.

Thanks @edenhill , after providing values for following parameters, i was able to connect to kafka server.
listeners=SSL://:9092
advertised.listeners=SSL://:9092
inter.broker.listener.name=SSL

now i have one more query - in server.properties of KAFKA server, we have specified the all TLS versions as
ssl.enabled.protocols=TLSv1.2,TLSv1.1,TLSv1

while librdkafka client with security.protocol=ssl connects to kafka server, how does librdkakfa client decides which TLS version needs to use ?.

My guess is the broker and client negotiates the best common protocol suite to use.
For reference, librdkafka is using the SSLv23_method client initializer:
https://www.openssl.org/docs/man1.0.2/ssl/SSL_CTX_new.html

Was this page helpful?
0 / 5 - 0 ratings