Schema-registry: Upgrade to Avro 1.9.1

[rocketraman]

Avro 1.9 has been released. The Confluent suite of tools should update to it.

Here are some highlights of the new release:

https://blog.godatadriven.com/apache-avro-1-9-release

The main incompatibility I have seen so far with Confluent Schema Registry, is that Avro 1.8 pulls in jackson 1.x as a transitive dependency, and schema registry contains a small amount of code that uses and exposes this dependency in order to manipulate JSON:

https://github.com/confluentinc/schema-registry/search?q=org.codehaus&unscoped_q=org.codehaus

It seems this code is all internal to SR, but with Avro 1.9 if Jackson 1.x isn't placed onto the user's classpath explicitly, the serdes will fail to start, with a ClassNotFoundException thrown. That being said, if client code puts Jackson 1.x onto their dependency list, Avro 1.9 seems to work quite well with the current version of the SR and the Kafka Avro serde, at least based on my testing so far.

In Avro 1.9, Jackson is now an implementation dependency rather than an API dependency, so SR will need to either depend on Jackson 1.x (yuck), depend on and upgrade to Jackson 2.x, or manipulate JSON a different way that doesn't require a third-party dependency.

[rayokota]

Avro is working through some regressions that will be fixed in Avro 1.9.1, so we'll probably at least wait for 1.9.1. For example, https://issues.apache.org/jira/browse/AVRO-2400

[rocketraman]

Avro 1.9.1 is now released, and includes a fix for the issue AVRO-2400 mentioned above.

[ivan-klass]

We also really need this as we have object-to-avro mapping via sksamuel/avro4s which now uses 1.9.1 Avro. This is especially a problem now for connectors where we need avro-to-connect converter