Cp-docker-images: Confluent Metrics Reporter : failed

[yosshi825]

After running "docker-compose up -d" , I get the following error:

• docker-compose.yml

  kafka7:
    image: confluentinc/cp-kafka:5.1.2
    container_name: confluent_kafka7
    ports:
      - 9092:9092
      - 9999:9999
    environment:
      - KAFKA_BROKER_ID=7
      - KAFKA_ZOOKEEPER_CONNECT=xxxxxx
      - KAFKA_ADVERTISED_LISTENERS=xxxxxx
      - KAFKA_DELETE_TOPIC_ENABLE=true
      - KAFKA_UNCLEAN_LEADER_ELECTION_ENABLE=FALSE
      - KAFKA_LOG_RETENTION_HOURS=72
      - KAFKA_MIN_INSYNC_REPLICAS=2
      - KAFKA_NUM_NETWORK_THREADS=12
      - KAFKA_NUM_IO_THREADS=24
      - KAFKA_NUM_REPLICA_FETCHERS=25
      - KAFKA_REPLICA_FETCH_MAX_BYTES=104857600
      - KAFKA_REPLICA_FETCH_RESPONSE_MAX_BYTES=104857600
      - KAFKA_MESSAGE_MAX_BYTES=104857600
      - KAFKA_NUM_PARTITIONS=14
      - KAFKA_DEFAULT_REPLICATION_FACTOR=3
      - JMX_PORT=9999
      - KAFKA_HEAP_OPTS=-Xms32G -Xmx32G
      - KAFKA_inter_broker_protocol_version=2.1
      - KAFKA_log_message_format_version=2.1
      - KAFKA_METRIC_REPORTERS=io.confluent.metrics.reporter.ConfluentMetricsReporter
      - CONFLUENT_METRICS_REPORTER_BOOTSTRAP_SERVERS=xxxxxx
      - CONFLUENT_METRICS_REPORTER_ZOOKEEPER_CONNECT=xxxxxx
      - CONFLUENT_METRICS_REPORTER_TOPIC_PARTITIONS=14
      - CONFLUENT_METRICS_REPORTER_TOPIC_REPLICAS=3
      - CONFLUENT_METRICS_ENABLE=true
      - CONFLUENT_SUPPORT_CUSTOMER_ID=anonymous

• error message

[2019-03-15 10:45:57,039] INFO [TransactionCoordinator id=7] Starting up. (kafka.coordinator.transaction.TransactionCoordinator)
[2019-03-15 10:45:57,040] INFO [TransactionCoordinator id=7] Startup complete. (kafka.coordinator.transaction.TransactionCoordinator)
[2019-03-15 10:45:57,040] INFO [Transaction Marker Channel Manager 7]: Starting (kafka.coordinator.transaction.TransactionMarkerChannelManager)
[2019-03-15 10:45:57,065] ERROR [KafkaServer id=7] Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
org.apache.kafka.common.KafkaException: io.confluent.metrics.reporter.ConfluentMetricsReporter ClassNotFoundException exception occurred
    at org.apache.kafka.common.config.AbstractConfig.getConfiguredInstances(AbstractConfig.java:357)
    at kafka.server.DynamicMetricsReporters.createReporters(DynamicBrokerConfig.scala:725)
    at kafka.server.DynamicMetricsReporters.<init>(DynamicBrokerConfig.scala:673)
    at kafka.server.DynamicBrokerConfig.addReconfigurables(DynamicBrokerConfig.scala:212)
    at kafka.server.KafkaServer.startup(KafkaServer.scala:304)
    at io.confluent.support.metrics.SupportedServerStartable.startup(SupportedServerStartable.java:114)
    at io.confluent.support.metrics.SupportedKafka.main(SupportedKafka.java:66)
Caused by: java.lang.ClassNotFoundException: io.confluent.metrics.reporter.ConfluentMetricsReporter
    at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:348)
    at org.apache.kafka.common.utils.Utils.loadClass(Utils.java:335)
    at org.apache.kafka.common.utils.Utils.newInstance(Utils.java:324)
    at org.apache.kafka.common.config.AbstractConfig.getConfiguredInstances(AbstractConfig.java:355)
    ... 6 more
[2019-03-15 10:45:57,068] INFO [KafkaServer id=7] shutting down (kafka.server.KafkaServer)
[2019-03-15 10:45:57,069] INFO [SocketServer brokerId=7] Stopping socket server request processors (kafka.network.SocketServer)
[2019-03-15 10:45:57,072] INFO [SocketServer brokerId=7] Stopped socket server request processors (kafka.network.SocketServer)
[2019-03-15 10:45:57,072] INFO [Kafka Request Handler on Broker 7], shutting down (kafka.server.KafkaRequestHandlerPool)
[2019-03-15 10:45:57,074] INFO [Kafka Request Handler on Broker 7], shut down completely (kafka.server.KafkaRequestHandlerPool)
[2019-03-15 10:45:57,075] INFO [KafkaApi-7] Shutdown complete. (kafka.server.KafkaApis)
[2019-03-15 10:45:57,076] INFO [ExpirationReaper-7-topic]: Shutting down (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2019-03-15 10:45:57,204] INFO [ExpirationReaper-7-topic]: Stopped (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2019-03-15 10:45:57,204] INFO [ExpirationReaper-7-topic]: Shutdown completed (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)

[AkshayAnurag]

Please use

image: confluentinc/cp-enterprise-kafka:latest
instead of
image: confluentinc/cp-kafka:5.1.2

to create the broker. Metrics is only available in Confluent Enterprise.

[mingliangguo]

@AkshayAnurag what needs to be taken out to make cp-kafka work? I ran into the similar issue, even after I remove all of the environment variables related to metrics, it still throws the same exception.

[AkshayAnurag]

@mingliangguo Can you please share the log file having the errors?

[gary-mguo]

@AkshayAnurag apologize for the confusion, It seems the pod didn't restart itself after I removed the metric related environment variables. It works well after I did a manually restart. Thanks for your quick response, do appreciate it!

[AkshayAnurag]

@gary-mguo Happy to help!

[pmihalcin]

I'm hitting this issue when running in Gitlab CI environment where support-metrics.confluent.io is not reachable.

Kafka container definition doesn't contain any metrics related configuration: https://github.com/testcontainers/testcontainers-spring-boot/blob/develop/embedded-kafka/src/main/java/com/playtika/test/kafka/configuration/KafkaContainerConfiguration.java#L95-L124

Docker image confluentinc/cp-kafka:5.4.1 tries to publish metrics even though it's community image: https://hub.docker.com/r/confluentinc/cp-kafka

Is this expected behavior?

[gAmUssA]

hey @pmihalcin

Confluent Proactive Support (you can read more about it here
) enabled by default in confluent images. To disable it in docker you can use

export KAFKA_CONFLUENT_SUPPORT_METRICS_ENABLE=false

Hope this helps!

[pmihalcin]

Many thanks! This helps.

[miaozhongyin]

hey @pmihalcin

Confluent Proactive Support (you can read more about it here
) enabled by default in confluent images. To disable it in docker you can use

export KAFKA_CONFLUENT_SUPPORT_METRICS_ENABLE=false

Hope this helps!

@gAmUssA Thank you for your solutino,It's fine.