Schema-registry: `kafka-avro-console-consumer` should handle non-Avro types

[xiongtx]

Expected Behavior

kafka-avro-console-consumer should support different deserializers for key and value, instead of assuming that key _and_ value are Avro.

E.g. we have many topics w/ string key and Avro value. kafka-avro-console-consumer can't print the keys.

Actual Behavior

kafka-avro-console-consumer (or more specifically AvroMessageFormatter) doesn’t accept key.deserializer property like kafka-console-consumer (or more specifically DefaultMessageFormatter).

This means that we can’t deserialize a topic where messages are non-Avro (e.g. String) and values are Avro with --property print.key=true.

Steps to Reproduce

1. Have a topic with non-Avro key (e.g. string) and Avro value
2. Use kafka-avro-console-consumer to consume messages from the topic
3. See that there's no way for the console consumer to deserialize the keys

[mageshn]

@xiongtx would you like to take it forward? you can may be optionally accept a key deserializer and if not present assume avro.

[akloeber]

kafka-avro-console-consumer even has an option --key-deserializer (at least output of kafka-avro-console-consumer -h says so) but that doesn't work neither. It always uses org.apache.kafka.common.serialization.ByteArrayDeserializer.

Options I tried:

--property 'key.deserializer=org.apache.kafka.common.serialization.StringDeserializer'
--key-deserializer org.apache.kafka.common.serialization.StringDeserializer 

[vinipuh5]

It works fine in my environment. Try to put the equal sign between '--key-deserializer' and the value.
Example:
kafka-avro-console-consumer --topic MyTopic --zookeeper localhost:2181 --property print.key=true --key-deserializer=org.apache.kafka.common.serialization.StringDeserializer

[rayokota]

This issue has been resolved.

[NickEm]

@rayokota Can you, please, reopen this one or create another one (because it can be not related) or explain what is going wrong
I have docker setup of _zookeeper,kafka,schemaregisry_ of official version 5.4.1
I go to _schemaregisry_ container and tried to execute different queries, f.e:
/usr/bin/kafka-avro-console-consumer --bootstrap-server kafka:9092 --topic Identity.TestCreateLifecycleEvents --value-deserializer=io.confluent.kafka.serializers.KafkaAvroDeserializer --property schema.registry.url=http://schemaregistry:8081 --property value.deserializer=io.confluent.kafka.serializers.KafkaAvroDeserializer
or
docker exec -it les-schema bash -c "/usr/bin/kafka-avro-console-consumer --bootstrap-server kafka:9092 --topic Identity.TestCreateLifecycleEvents --value-deserializerio.confluent.kafka.serializers.KafkaAvroDeserializer --property schema.registry.url='http://schemaregistry:8081' --property value.deserializer=io.confluent.kafka.serializers.KafkaAvroDeserializer"
And everytime my consumer is initialized with value.deserializer = class org.apache.kafka.common.serialization.ByteArrayDeserializer, question is why?