Confluent-kafka-dotnet: Can I produce and subscribe on Secure and Unsecured ports?

[vinodres]

I have a mix of applications that are developed in .NET and JAVA. I am using v0.9.5 client for .NET based applications. Recently security was enabled on Kafka brokers (v0.10.2). So we have brokers with 9092 port secured. But we also have port 9091 left unsecured.

As .NET client cannot use security yet, is it possible for a .NET client to consume messages on unsecure 9091 port while JAVA application is producing messages on secure 9092 port for the same topic?

Once the security features start working in .NET Kafka client, I will switch using secure port. I am trying to figure out if above mentioned solution is possible as a temporary measure or both producers and consumers have to use same ports (either secure or unsecured). I am looking for suggestions.

[treziac]

I don't see why it wouldn't work. I don't use ssl myself, but given the doc https://kafka.apache.org/documentation/#security_rolling_upgrade
I see nothing which prevent this

[vinodres]

Good to know that it will work. For some reason I am not seeing the .NET consumer receiving messages. No errors being output on client side. The consumer is silently waiting. I am doing further investigation on my side to make sure everything is configured properly. I will get back after I complete my investigation.

[vinodres]

Found the root cause. I was using NullDeserializer on the .NET consumer where as Java based producer was publishing using StringSerializer for Key. Once I switched the .NET consumer to use StringDeserializer for key, I started receiving messages.

[mhowlett]

note: i would have expect the NullDeserializer to have been throwing exceptions in the scenario described above - not silently waiting (this will be changed in 0.9.6 to raise an OnConsumeError event).