Schema-registry: Cannot connect to kafka from schema-registry inside a docker-compose network

[raghavan20]

Hi,

I am trying to bring up schema-registry as part of docker-compose environment. I have ensured that Zookeeper and Kafka are functional. But when I start schema-registry service later using docker-compose up schema-registry, it fails to communicate to kafka or rather not seem to communicate to kafka socket passed as environment variable, SCHEMA_REGISTRY_KAFKASTORE_BOOTSTRAP_SERVERS. Could you please help identify the issue?

Docker compose file

version: '3'
services:
  zookeeper:
    image: confluentinc/cp-zookeeper:5.0.0
    volumes:
      - ~/dev/data/zk-data:/var/lib/zookeeper/data
      - ~/dev/logs/zk-txn-logs:/var/lib/zookeeper/log
    environment:
      - ZOOKEEPER_CLIENT_PORT=2181
    ports:
      - 2181:2181
  kafka:
    image: confluentinc/cp-kafka:5.0.0
    volumes:
      - ~/dev/data/kafka-data:/var/lib/kafka/data
    environment:
      - KAFKA_BROKER_ID=1
      - KAFKA_ZOOKEEPER_CONNECT=zookeeper:2181
      - KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://127.0.0.1:9092
      - KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR=1
    links:
      - zookeeper
    ports:
      - 9092:9092
  schema-registry:
    image: confluentinc/cp-schema-registry:5.0.0
    environment:
      - SCHEMA_REGISTRY_KAFKASTORE_CONNECTION_URL=zookeeper:2181
      - SCHEMA_REGISTRY_HOST_NAME=schema-registry
      - SCHEMA_REGISTRY_LISTENERS=http://0.0.0.0:8081
      - SCHEMA_REGISTRY_KAFKASTORE_BOOTSTRAP_SERVERS=PLAINTEXT://kafka:9092
    links:
      - zookeeper
      - kafka
    ports:
      - 8081:8081

Error

schema-registry_1  | [main] INFO org.apache.kafka.clients.admin.AdminClientConfig - AdminClientConfig values: 
schema-registry_1  |    bootstrap.servers = [127.0.0.1:9092]
...
schema-registry_1  | ===> ENV Variables ...      
schema-registry_1  | ALLOW_UNSIGNED=false                                   
schema-registry_1  | COMPONENT=schema-registry                           
schema-registry_1  | CONFLUENT_DEB_VERSION=1                                       
schema-registry_1  | CONFLUENT_MAJOR_VERSION=5                            
schema-registry_1  | CONFLUENT_MINOR_VERSION=0                                                           
schema-registry_1  | CONFLUENT_MVN_LABEL=                                           
schema-registry_1  | CONFLUENT_PATCH_VERSION=0                                                                                                                                
schema-registry_1  | CONFLUENT_PLATFORM_LABEL=                                        
schema-registry_1  | CONFLUENT_VERSION=5.0.0                                           
schema-registry_1  | CUB_CLASSPATH=/etc/confluent/docker/docker-utils.jar    
schema-registry_1  | HOME=/root                                                 
schema-registry_1  | HOSTNAME=388072d27a78                                                                                            
schema-registry_1  | KAFKA_VERSION=2.0.0                                                                                                     
schema-registry_1  | LANG=C.UTF-8                                                                                       
schema-registry_1  | PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin                                   
schema-registry_1  | PWD=/                                                                                              
schema-registry_1  | PYTHON_PIP_VERSION=8.1.2                                                                            
schema-registry_1  | PYTHON_VERSION=2.7.9-1          
schema-registry_1  | SCALA_VERSION=2.11                       
schema-registry_1  | SCHEMA_REGISTRY_HOST_NAME=schema-registry                
schema-registry_1  | SCHEMA_REGISTRY_KAFKASTORE_BOOTSTRAP_SERVERS=PLAINTEXT://kafka:9092
schema-registry_1  | SCHEMA_REGISTRY_KAFKASTORE_CONNECTION_URL=zookeeper:2181
schema-registry_1  | SCHEMA_REGISTRY_LISTENERS=http://0.0.0.0:8081                                                                                                                   
schema-registry_1  | SHLVL=1                                                                               
schema-registry_1  | ZULU_OPENJDK_VERSION=8=8.30.0.1                                                       
schema-registry_1  | _=/usr/bin/env                                                                                     
schema-registry_1  | affinity:container==ea23849bf423c4cbec570036a9d6461bc1f48a31762c6c76c102ac020e362c00    
...
...
schema-registry_1  | [main] INFO org.apache.kafka.common.utils.AppInfoParser - Kafka version : 2.0.0-cpNone
schema-registry_1  | [main] INFO org.apache.kafka.common.utils.AppInfoParser - Kafka commitId : ca8d91be74ec83ed
schema-registry_1  | [kafka-admin-client-thread | adminclient-1] WARN org.apache.kafka.clients.NetworkClient - [AdminClient clientId=adminclient-1] Connection to node -1 could not be established. Broker may not be available.
schema-registry_1  | [kafka-admin-client-thread | adminclient-1] WARN org.apache.kafka.clients.NetworkClient - [AdminClient clientId=adminclient-1] Connection to node -1 could not be established. Broker may not be available.
schema-registry_1  | [kafka-admin-client-thread | adminclient-1] WARN org.apache.kafka.clients.NetworkClient - [AdminClient clientId=adminclient-1] Connection to node -1 could not be established. Broker may not be available.
schema-registry_1  | [kafka-admin-client-thread | adminclient-1] WARN org.apache.kafka.clients.NetworkClient - [AdminClient clientId=adminclient-1] Connection to node -1 could not be established. Broker may not be available.
schema-registry_1  | [kafka-admin-client-thread | adminclient-1] WARN org.apache.kafka.clients.NetworkClient - [AdminClient clientId=adminclient-1] Connection to node -1 could not be established. Broker may not be available.
schema-registry_1  | [kafka-admin-client-thread | adminclient-1] WARN org.apache.kafka.clients.NetworkClient - [AdminClient clientId=adminclient-1] Connection to node -1 could not be established. Broker may not be available.

[OneCricketeer]

- KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://127.0.0.1:9092

Kafka is only listening within the Container, not on the external network.

Please refer to the cp-docker-images docker-compose file. While that is using Zookeeper connection, the property that I mention is not using 127.0.0.1 or localhost for the internal network, it uses broker:9092

You also should only list one of Kafka or Zookeeper for the schema registry, not both

[rayokota]

Closing this issue, please free to reopen. Thanks!