Confluent-kafka-dotnet: When Consume Message , throw System.ArgumentException

[Alvin1128]

use kafka-console-consumer.bat is ok,

use below code throw exception，what is the cause of what may be

thanks

`var config = new Dictionary
{
{ "group.id", "test-consumer-group" },
{ "bootstrap.servers", brokerList}
};

        using (var consumer = new Consumer<Null, string>(config, null, new StringDeserializer(Encoding.UTF8)))
        {

            consumer.Assign(new List<TopicPartitionOffset> { new TopicPartitionOffset(topics, 0, 0) });
            while (true)
            {
                Message<Null, string> msg;
                try
                {
                    if (consumer.Consume(out msg))
                    {
                        Console.WriteLine("Topic: {0}, Partition: {1} Offset: {2} {3}",
                            msg.Topic, msg.Partition, msg.Offset, msg.Value);
                    }
                }
                catch (Exception ex)
                {
                    Console.WriteLine(DateTime.Now.ToShortTimeString() + " " + ex.Message);
                }
            }
        }`

[fadijabi]

Oh! I thought I am having an issue in my code! I am having the same issue..

the exception is: NullDeserializer may only be used to deserialize data that is null.

[Alvin1128]

@edenhill

I found that when the message is string type does not go wrong, if the message is json type will report this error，Will it be wrong with deserializer?

this is detail exception message

System.ArgumentException: NullDeserializer may only be used to deserialize data that is null.
在 Confluent.Kafka.Serialization.NullDeserializer.Deserialize(Byte[] data)
在 Confluent.Kafka.Serialization.MessageExtensions.DeserializeTKey,TValue
在 Confluent.Kafka.Consumer2.Consume(Message2& message, Int32 millisecondsTimeout)
在 Confluent.Kafka.Consumer2.Consume(Message2& message)
在 Kafka_Consumer.Program.Main(String[] args) 位置 c:\Users\LinkedCare\Documents\Visual Studio 2013\Projects\BeautifulDotNet\Kafka\Program.cs:行号 36

[mhowlett]

NullDeserializer considers it an error if it receives a value that is not null and throws an exception. PR #179 changes the behaviour of the consumer so that if a deserialiser throws and exception this is exposed via the OnConsumeError event instead (which will generally be more convenient). This change will be in 0.9.6. For now, you should handle the exception.

[ghost]

I'm using 0.11.3 version of Kafka and i'm seeing the NullDeserializer exception during the OnConsumeError event but i'm not certain as to what could be causing this. My message is JSON, could it be that the message is invalid or somehow not able to be deserialized or is it that my deserializer needs to be NOT passed in with an initialization but, instead, as a NULL value? I even checked my key values and they are not null, not an empty string, but an integer array with ints stored in them, successfully. What is going wrong here?

[mhowlett]

If you're using NullDeserializer, it expects the message key or value it's deserializing to be null and will throw an exception otherwise (you say your message is a string, so it will throw). If you want to ignore values, try the IgnoreDeserializer instead.

[ghost]

Well, i'm not even getting to the point of deserialization. I fail when i attempt to get out a message from an assigned partition: Consumer.Consume(out message, TimeSpan.FromSeconds(1)). I setup my message object but there are no exceptions and i am only using this line as a conditional to attempt to deserialize, but the Broker never puts the record into my "message" object. Why is that?

[ghost]

The answer to this question: the Consumer and Producer both have to match their key serializers or else Kafka gets confused and won't function, properly. In this case, this will cause Kafka to NOT be able to consume messages since the Reader was expecting a key deserializer that was null but the Producer gave one that was NOT null.