We followed the steps from the following link to create SSL connection.
https://github.com/edenhill/librdkafka/wiki/Using-SSL-with-librdkafka
when librdkafka client is trying to connect to KAFKA server, we are getting following error in consumer trace
% ERROR CALLBACK: kcea#consumer-1: error: -181 ssl://abc.abc.aa.bb:9092/bootstrap: SSL handshake failed: SSL syscall error
where following error is excerpt from Kafka server.
[2018-02-12 20:12:55,333] WARN Unexpected error from /abc.abc.abc.aa; closing connection (org.apache.kafka.common.network.Selector)
org.apache.kafka.common.network.InvalidReceiveException: Invalid receive (size = 369295617 larger than 104857600)
at org.apache.kafka.common.network.NetworkReceive.readFromReadableChannel(NetworkReceive.java:95)
at org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java:75)
at org.apache.kafka.common.network.KafkaChannel.receive(KafkaChannel.java:203)
at org.apache.kafka.common.network.KafkaChannel.read(KafkaChannel.java:167)
at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:379)
at org.apache.kafka.common.network.Selector.poll(Selector.java:326)
at kafka.network.Processor.poll(SocketServer.scala:499)
could you please let us know if we are missing some steps here.
librdkafka version - librdkafka-0.11.0
Kafka server version - kafka_2.11-0.11.0.0
Looks like you tried to connect to a non-SSL listener.
Verify that 9092 is the SSL listener port on the broker.
Thanks @edenhill , after providing values for following parameters, i was able to connect to kafka server.
listeners=SSL://:9092
advertised.listeners=SSL://:9092
inter.broker.listener.name=SSL
now i have one more query - in server.properties of KAFKA server, we have specified the all TLS versions as
ssl.enabled.protocols=TLSv1.2,TLSv1.1,TLSv1
while librdkafka client with security.protocol=ssl connects to kafka server, how does librdkakfa client decides which TLS version needs to use ?.
My guess is the broker and client negotiates the best common protocol suite to use.
For reference, librdkafka is using the SSLv23_method client initializer:
https://www.openssl.org/docs/man1.0.2/ssl/SSL_CTX_new.html
Most helpful comment
Looks like you tried to connect to a non-SSL listener.
Verify that 9092 is the SSL listener port on the broker.