Schema-registry: Regression: field default value of type short not handled correctly

[gunnarmorling]

We're running into a regression with 5.1.0. Fields with a default value of type short fail to be converted to Avro fields now:

Unknown datum class: class java.lang.Short
    at org.apache.avro.util.internal.JacksonUtils.toJson(JacksonUtils.java:87)
    at org.apache.avro.util.internal.JacksonUtils.toJsonNode(JacksonUtils.java:48)
    at org.apache.avro.Schema$Field.<init>(Schema.java:423)
    at org.apache.avro.Schema$Field.<init>(Schema.java:415)
    at io.confluent.connect.avro.AvroData.addAvroRecordField(AvroData.java:964)
    at io.confluent.connect.avro.AvroData.fromConnectSchema(AvroData.java:847)
    at io.confluent.connect.avro.AvroData.fromConnectSchemaWithCycle(AvroData.java:942)
    at io.confluent.connect.avro.AvroData.fromConnectSchema(AvroData.java:834)
    at io.confluent.connect.avro.AvroData.addAvroRecordField(AvroData.java:963)
    at io.confluent.connect.avro.AvroData.fromConnectSchema(AvroData.java:847)
    at io.confluent.connect.avro.AvroData.fromConnectSchema(AvroData.java:689)
    at io.confluent.connect.avro.AvroData.fromConnectSchema(AvroData.java:683)
    at io.confluent.connect.avro.AvroData.fromConnectData(AvroData.java:328)
    at io.confluent.connect.avro.AvroConverter.fromConnectData(AvroConverter.java:76)

This appears to be caused by https://github.com/confluentinc/schema-registry/pull/853/: previously, default values were converted into the Avro type here in the converter (by invoking the Schema$Field constructor that takes a JsonNode for the default value), whereas the new logic relies on this being done by Avro (by invoking the Schema$Field constructor that takes any object as default value). Hence the new code triggers Avro 1.8.1's org.apache.avro.util.internal.JacksonUtils.toJson() method which lacks support for short values.

The issue could be fixed by either up-casting byte and short values (for INT8/INT16) to int before passing the value to Avro. Or Avro could be adjusted to accept these values; not sure what's their stance given only 32/64 bits are in the list of supported schema types, although it'd seem reasonable to accept byte/short instances as default for the wider types, too.

[renatomefi]

@mageshn would you be willing to take a look at this one? Cheers!

[mageshn]

@renatomefi the issue is already being looked by @rhauch and he is working to get a fix out for this.

[javierholguera]

We are observing the same error for other types that used to be supported, like date and bigdecimal.

[jrmma]

Is there a workaround for this? I'm facing this issue in a debezium setup with v0.8.0.Final, v0.8.2 & v0.8.3.Final as well.

[rhauch]

Marking this as resolved by #985, which will be in the next CP 5.2.0 release and the upcoming CP 5.1.1 patch release.

This is a client-side change in the AvroConverter classes used in Connect installations. So upgrade your Connect installations to CP 5.2.0 or 5.1.1 (when available) or upgrade the kafka-connect-avro-converter dependency.

[gunnarmorling]

Excellent, great news!

Is there any ETA for 5.1.1, @rhauch?

[jrmma]

Thanks!

[ankit45621]

Hello everyone,

I am getting same issue and issue yet not solved with version 5.1.1. Anyone help me for this issue.