Cp-docker-images: Error while attempting to create/find topic(s) 'quickstart-avro-offsets'

[nabi-rd]

Hi,
Trying to run Kafka Connect using this tutorial. Kafka Connect Docker container stops straight away after running.

[2017-10-05 04:18:52,360] ERROR Uncaught exception in herder work thread, exiting: (org.apache.kafka.connect.runtime.distributed.DistributedHerder) org.apache.kafka.connect.errors.ConnectException: Error while attempting to create/find topic(s) 'quickstart-avro-offsets' at org.apache.kafka.connect.util.TopicAdmin.createTopics(TopicAdmin.java:245) at org.apache.kafka.connect.storage.KafkaOffsetBackingStore$1.run(KafkaOffsetBackingStore.java:99) at org.apache.kafka.connect.util.KafkaBasedLog.start(KafkaBasedLog.java:126) at org.apache.kafka.connect.storage.KafkaOffsetBackingStore.start(KafkaOffsetBackingStore.java:109) at org.apache.kafka.connect.runtime.Worker.start(Worker.java:146) at org.apache.kafka.connect.runtime.AbstractHerder.startServices(AbstractHerder.java:99) at org.apache.kafka.connect.runtime.distributed.DistributedHerder.run(DistributedHerder.java:194) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.InvalidReplicationFactorException: replication factor: 3 larger than available brokers: 1 at org.apache.kafka.common.internals.KafkaFutureImpl.wrapAndThrow(KafkaFutureImpl.java:45) at org.apache.kafka.common.internals.KafkaFutureImpl.access$000(KafkaFutureImpl.java:32) at org.apache.kafka.common.internals.KafkaFutureImpl$SingleWaiter.await(KafkaFutureImpl.java:89) at org.apache.kafka.common.internals.KafkaFutureImpl.get(KafkaFutureImpl.java:213) at org.apache.kafka.connect.util.TopicAdmin.createTopics(TopicAdmin.java:226) ... 11 more Caused by: org.apache.kafka.common.errors.InvalidReplicationFactorException: replication factor: 3 larger than available brokers: 1 [2017-10-05 04:18:52,366] DEBUG stopping org.eclipse.jetty.server.Server@7a4446e2 (org.eclipse.jetty.util.component.AbstractLifeCycle) [2017-10-05 04:18:52,368] DEBUG Graceful shutdown org.eclipse.jetty.server.Server@7a4446e2 by (org.eclipse.jetty.server.Server) [2017-10-05 04:18:52,369] DEBUG stopping ServerConnector@68309532{HTTP/1.1}{0.0.0.0:28083} (org.eclipse.jetty.util.component.AbstractLifeCycle) [2017-10-05 04:18:52,369] INFO Kafka Connect stopping (org.apache.kafka.connect.runtime.Connect)

I do have the Kafka topic though. Any ideas?

[victorchicu]

You or whatever it is was try to create topic with replicas parameter which is equal to 3 but you do not have enough brokers services setted up for this.
Go in server.properties configuration file and try to uncomment the following line:
#confluent.metrics.reporter.topic.replicas=1

[nabi-rd]

After running kafka container, I uncommented that line in server.properties but that did not solve my issue either!

[kccheung]

Hi, I faced exactly the same problem when using kafka connect to existing kafka using the confluentinc/cp-enterprise-kafka docker image.

[victorchicu]

oops! i think I gave wrong config.
check this one "offsets.topic.replication.factor=1" if it's equals 3 try to put 1.

[nabi-rd]

It is already set to 1. Still doesn't work!

[kccheung]

same as @nabi-rd I have already set "offsets.topic.replication.factor=1" to 1. Still doesn't work!

I need to go for standalone mode of Connect in the meantime.

[hjespers]

where the docker command line says

-e CONNECT_OFFSETS_TOPIC_REPLICATION_FACTOR=1 \

add in also

-e CONNECT_CONFIG_STORAGE_REPLICATION_FACTOR= 1 \
-e CONNECT_STATUS_STORAGE_REPLICATION_FACTOR= 1 \

Also make sure your "latest" image in not an old image by explicitly stating the image to be

confluentinc/cp-kafka-connect:4.0.0

[filipamb]

Problem was caused by wrong properties included into tutorial. Proper command (which worked for me) to launch kafka connect docker is:

docker run -d \
--name=kafka-connect-avro \
--net=host \
-e CONNECT_BOOTSTRAP_SERVERS=localhost:29092 \
-e CONNECT_REST_PORT=28083 \
-e CONNECT_GROUP_ID="quickstart-avro" \
-e CONNECT_CONFIG_STORAGE_TOPIC="quickstart-avro-config" \
-e CONNECT_OFFSET_STORAGE_TOPIC="quickstart-avro-offsets" \
-e CONNECT_STATUS_STORAGE_TOPIC="quickstart-avro-status" \
-e CONNECT_CONFIG_STORAGE_REPLICATION_FACTOR=1 \
-e CONNECT_STATUS_STORAGE_REPLICATION_FACTOR=1 \
-e CONNECT_OFFSET_STORAGE_REPLICATION_FACTOR=1 \
-e CONNECT_KEY_CONVERTER="io.confluent.connect.avro.AvroConverter" \
-e CONNECT_VALUE_CONVERTER="io.confluent.connect.avro.AvroConverter" \
-e CONNECT_KEY_CONVERTER_SCHEMA_REGISTRY_URL="http://localhost:8081" \
-e CONNECT_VALUE_CONVERTER_SCHEMA_REGISTRY_URL="http://localhost:8081" \
-e CONNECT_INTERNAL_KEY_CONVERTER="org.apache.kafka.connect.json.JsonConverter" \
-e CONNECT_INTERNAL_VALUE_CONVERTER="org.apache.kafka.connect.json.JsonConverter" \
-e CONNECT_REST_ADVERTISED_HOST_NAME="localhost" \
-e CONNECT_LOG4J_ROOT_LOGLEVEL=DEBUG \
-v /tmp/quickstart/file:/tmp/quickstart \
-v /tmp/quickstart/jars:/etc/kafka-connect/jars \
confluentinc/cp-kafka-connect:4.0.0

I changed or added bolded parts of this command

[RichardRTS]

So, Which is the solution?

[rahulkohli23]

I am facing the same issue while deploying kafka-connect with kubernetes.

Below are the yaml configs I am using

apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaConnect
metadata:
  name: my-connect-cluster
  annotations:
    strimzi.io/use-connector-resources: "true"
#  # use-connector-resources configures this KafkaConnect
#  # to use KafkaConnector resources to avoid
#  # needing to call the Connect REST API directly
spec:
  version: 2.7.0
  replicas: 1
  bootstrapServers: my-cluster-kafka-bootstrap.default.svc:9092
  config:
    group.id: connect-cluster
    offset.storage.topic: 1
    config.storage.topic: 1
    status.storage.topic: 1