Schema-registry: KafkaAvroDeserializer should minimize dependency on kafka server class

[phaas]

I've attempted to declare a KafkaAvroDeserializer as a spring @Bean, which exposed it to BeanDefinitionPostProcessors.

While the class itself works without having the kafka server jar on the class path, it would be nice if the new consumer/producer-related files were truly independent of the kafka server.

This is really an issue with the PersistenceAnnotationBeanPostProcessor and not this project, but it would be appreciated if the client plays nice with other code.

__Stacktrace__

Caused by: java.lang.IllegalStateException: Failed to introspect bean class [io.confluent.kafka.serializers.KafkaAvroDeserializer] for persistence metadata: could not find class that it depends on
    at org.springframework.orm.jpa.support.PersistenceAnnotationBeanPostProcessor.findPersistenceMetadata(PersistenceAnnotationBeanPostProcessor.java:396) ~[spring-orm-4.1.9.RELEASE.jar:4.1.9.RELEASE]
    at org.springframework.orm.jpa.support.PersistenceAnnotationBeanPostProcessor.postProcessMergedBeanDefinition(PersistenceAnnotationBeanPostProcessor.java:333) ~[spring-orm-4.1.9.RELEASE.jar:4.1.9.RELEASE]
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyMergedBeanDefinitionPostProcessors(AbstractAutowireCapableBeanFactory.java:923) ~[spring-beans-4.1.5.RELEASE.jar:4.1.5.RELEASE]
    ... 83 common frames omitted
Caused by: java.lang.NoClassDefFoundError: kafka/utils/VerifiableProperties
    at java.lang.Class.getDeclaredMethods0(Native Method) ~[na:1.8.0_91]
    at java.lang.Class.privateGetDeclaredMethods(Class.java:2701) ~[na:1.8.0_91]
    at java.lang.Class.getDeclaredMethods(Class.java:1975) ~[na:1.8.0_91]
    at org.springframework.orm.jpa.support.PersistenceAnnotationBeanPostProcessor.buildPersistenceMetadata(PersistenceAnnotationBeanPostProcessor.java:420) ~[spring-orm-4.1.9.RELEASE.jar:4.1.9.RELEASE]
    at org.springframework.orm.jpa.support.PersistenceAnnotationBeanPostProcessor.findPersistenceMetadata(PersistenceAnnotationBeanPostProcessor.java:392) ~[spring-orm-4.1.9.RELEASE.jar:4.1.9.RELEASE]
    ... 85 common frames omitted
Caused by: java.lang.ClassNotFoundException: kafka.utils.VerifiableProperties
    at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1854) ~[catalina.jar:7.0.65]
    at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1703) ~[catalina.jar:7.0.65]
    ... 90 common frames omitted

__Example Configuration__

    /**
     * FIXME KafkaAvroDeserializer (optionally) uses a class from the kafka server jar. Since this
     * is not present on the class path, that part of the class can't be introspected.
     * <p>
     * The persistence bean postprocessor checks every bean for @PersistenceContext and related
     * annotations. When it accesses the method using the "kafka/utils/VerifiableProperties" class,
     * it fails with a NoClassDefFoundError.
     * <p>
     * Workaround: Don't declare the serializer as a spring bean, since the benefit of having a singleton
     * is minimal.
     *
     * @return
     */
//  @Bean
    public KafkaAvroDeserializer kafkaAvroDeserializer() {
        // Must get back specific records
        Map<String, Object> props = new HashMap<>();
        props.put(KafkaAvroDeserializerConfig.SPECIFIC_AVRO_READER_CONFIG, true);
        props.put(KafkaAvroDeserializerConfig.SCHEMA_REGISTRY_URL_CONFIG, schemaRegistryUrl);
        return new KafkaAvroDeserializer(schemaRegistryClient(), props);
    }

[larsduelfer]

Hi all, I have the same issue at the moment. Any news on this yet?

[Wiwun]

Also have this issue

[abhishekagarwal87]

For a workaround, one can add a empty class with same package name in their code. Not a perfect solution but works nevertheless.

[pranay-1995]

Include kafka dependency in pom.xml of your application e.g.,

<dependency>
            <groupId>org.apache.kafka</groupId>
            <artifactId>kafka_2.11</artifactId>
</dependency>

[rayokota]

Currently kafka-avro-serializer relies on classes such as MessageReader, MessageFormatter, VerifiableProperties from Kafka