Ksql: CREATE/SHOW CONNECTOR Failure

[Sarlken]

ksql> SHOW CONNECTORS;io.confluent.ksql.util.KsqlServerException: org.apache.http.conn.HttpHostConnectException: Connect to localhost:8083 [localhost/127.0.0.1] failed: Connection refused (Connection refused) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to localhost:8083 [localhost/127.0.0.1] failed: Connection refused (Connection refused) Caused by: Could not connect to the server. Caused by: Could not connect to the server.
After installing KSQLDB to my virtual machine, when I try to create JDBC Connector for my mysql db I receive the error below:

io.confluent.ksql.util.KsqlServerException: org.apache.http.conn.HttpHostConnectException: Connect to localhost:8083 [localhost/127.0.0.1] failed: Connection refused (Connection refused) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to localhost:8083 [localhost/127.0.0.1] failed: Connection refused (Connection refused) Caused by: Could not connect to the server. Caused by: Could not connect to the server.

I receive the same error when I try to create the connector. I think it's related with Kafka Connect component inside of the KSQLDB, since all the queries related with the KSQL part is functioning really well. However, all the queries related with Kafka Connector fails and gives the same error message as above.

The versions are CLI v0.6.0, Server v0.6.0
The SQL statements I run SHOW CONNECTORS;

I would be glad for any help about how to resolve this error.

[agavra]

The reason this doesn't work is that you need to configure ksqlDB with a connect property file.

If you are intending to use a local embedded connect worker, you need to supply a connect.properties file to ksqlDB. You can take a look at https://github.com/confluentinc/ksql/blob/master/config/connect.properties for a sample configuration (make sure to update the bootstrap.server and the schema registry URLs). You will need to mount this file via docker volumes (we're working on a way to make this easier) and then set KSQL_KSQL_CONNECT_WORKER_CONFIG: /path/to/volume/connect.properties

[Sarlken]

Thanks for your response. I modified connect.properties and created a path in my virtual machine as you said, then copied the file to the path and added the line below to the ksqldb's docker-compose.yml file as below:

**version: "3"
services:
    schema-registry:
    image: "[my_image_path]"
    networks:
      ntwrk-zkpr-stck:
      ntwrk-kfk-stck:
    depends_on:
      - srvc-apch-zkpr0
      - srvc-apch-zkpr1
      - srvc-apch-zkpr2
      - srvc-kfk-brkr0
      - srvc-kfk-brkr1
      - srvc-kfk-brkr2
    environment:
      SCHEMA_REGISTRY_HOST_NAME: schema-registry
      SCHEMA_REGISTRY_KAFKASTORE_CONNECTION_URL: srvc-apch-zkpr0:2181, srvc-apch-zkpr0:2181, srvc-apch-zkpr0:2181
      SCHEMA_REGISTRY_LISTENERS: "http://0.0.0.0:8081"
    ports:
      - 8081:8081
  ksqldb-server:
    image: [my_image_path]
    hostname: ksqldb-server
    container_name: ksqldb-server
    ports:
      - "8089:8089"
    environment:
      KSQL_LISTENERS: http://0.0.0.0:8089
      KSQL_BOOTSTRAP_SERVERS: [ip_of_my_vm]:9092,[ip_of_my_vm]:9093,[ip_of_my_vm]:9094
      KSQL_KSQL_LOGGING_PROCESSING_STREAM_AUTO_CREATE: "true"
      KSQL_KSQL_LOGGING_PROCESSING_TOPIC_AUTO_CREATE: "true"
      KSQL_KSQL_CONNECT_WORKER_CONFIG: /etc/ksqldb/config/connect.properties
  ksqldb-cli:
    image: [my_image_path]
    container_name: ksqldb-cli
    depends_on:
      - ksqldb-server
    entrypoint: /bin/sh
    tty: true
networks:
  ntwrk-zkpr-stck:
  ntwrk-kfk-stck:**

PS: Image values are the docker repositories that I pushed so no problem about them.

After doing these configurations, ksqldb-server is always failing. I couldn't understand why.

In this point, I would like to ask whether kSQLDB has an embedded schema registry or do I need to install schema registry in the docker-compose file above?

Thanks for the help.

[McHardex]

Hi @agavra , I tried the solution you suggested above and after doing so, I keep getting the same error just like @Sarlken mentioned.

Here is my docker-compose.yml file

version: '2'

services:
  zookeeper:
    image: confluentinc/cp-zookeeper:5.3.1
    hostname: zookeeper
    container_name: zookeeper
    ports:
      - "2181:2181"
    environment:
      ZOOKEEPER_CLIENT_PORT: 2181
      ZOOKEEPER_TICK_TIME: 2000

  broker:
    image: confluentinc/cp-enterprise-kafka:5.3.1
    hostname: broker
    container_name: broker
    depends_on:
      - zookeeper
    ports:
      - "9092:9092"
      - "29092:29092"
    environment:
      KAFKA_BROKER_ID: 1
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://broker:29092,PLAINTEXT_HOST://localhost:9092
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
      KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS: 0

  schema-registry:
    image: confluentinc/cp-schema-registry:5.3.1
    hostname: schema-registry
    container_name: schema-registry
    depends_on:
      - zookeeper
      - broker
    ports:
      - "8081:8081"
    environment:
      SCHEMA_REGISTRY_HOST_NAME: schema-registry
      SCHEMA_REGISTRY_KAFKASTORE_CONNECTION_URL: zookeeper:2181

  connect:
    image: confluentinc/cp-kafka-connect:latest
    hostname: connect
    container_name: connect
    depends_on:
      - zookeeper
      - broker
      - schema-registry
    ports:
      - "8083:8083"
    environment:
      CONNECT_BOOTSTRAP_SERVERS: broker:29092
      CONNECT_REST_ADVERTISED_HOST_NAME: localhost
      CONNECT_REST_PORT: 8083
      CONNECT_GROUP_ID: ksql-connect-cluster
      CONNECT_OFFSET_STORAGE_TOPIC: ksql-connect-configs
      CONNECT_CONFIG_STORAGE_TOPIC: ksql-connect-topics
      CONNECT_STATUS_STORAGE_TOPIC: ksql-connect-statuses
      CONNECT_CONFIG_STORAGE_REPLICATION_FACTOR: 1
      CONNECT_OFFSET_FLUSH_INTERVAL_MS: 10000
      CONNECT_OFFSET_STORAGE_REPLICATION_FACTOR: 1
      CONNECT_STATUS_STORAGE_REPLICATION_FACTOR: 1
      CONNECT_KEY_CONVERTER: org.apache.kafka.connect.storage.StringConverter
      CONNECT_VALUE_CONVERTER: io.confluent.connect.avro.AvroConverter
      CONNECT_VALUE_CONVERTER_SCHEMA_REGISTRY_URL: http://schema-registry:8081
      CONNECT_INTERNAL_KEY_CONVERTER: "org.apache.kafka.connect.json.JsonConverter"
      CONNECT_INTERNAL_VALUE_CONVERTER: "org.apache.kafka.connect.json.JsonConverter"
      CONNECT_ZOOKEEPER_CONNECT: 'zookeeper:2181'
      # CLASSPATH required due to CC-2422
      CLASSPATH: C:/Users/User01/Documents/confluent/share/confluent-hub-components
      # CONNECT_PRODUCER_INTERCEPTOR_CLASSES: "io.confluent.monitoring.clients.interceptor.MonitoringProducerInterceptor"
      # CONNECT_CONSUMER_INTERCEPTOR_CLASSES: "io.confluent.monitoring.clients.interceptor.MonitoringConsumerInterceptor"
      CONNECT_PLUGIN_PATH: C:/Users/User01/Documents/confluent/share/confluent-hub-components
      # CONNECT_LOG4J_LOGGERS: org.apache.zookeeper=ERROR,org.I0Itec.zkclient=ERROR,org.reflections=ERROR
      KSQL_KSQL_CONNECT_WORKER_CONFIG: C:/Users/User01/Documents/confluent/share/java/connect.properties
    volumes: 
      - C:/Users/User01/Documents/confluent/share/java/connect.properties

  ksqldb-server:
    image: confluentinc/ksqldb-server:0.6.0
    hostname: ksqldb-server
    container_name: ksqldb-server
    depends_on:
      - broker
    ports:
      - "8088:8088"
    environment:
      KSQL_LISTENERS: http://0.0.0.0:8088
      KSQL_BOOTSTRAP_SERVERS: broker:29092
      KSQL_KSQL_LOGGING_PROCESSING_STREAM_AUTO_CREATE: "true"
      KSQL_KSQL_LOGGING_PROCESSING_TOPIC_AUTO_CREATE: "true"
      KSQL_KSQL_CONNECT_WORKER_CONFIG: C:/Users/User01/Documents/confluent/share/java/connect.properties
    volumes: 
      - C:/Users/User01/Documents/confluent/share/java/connect.properties

  ksqldb-cli:
    image: confluentinc/ksqldb-cli:0.6.0
    container_name: ksqldb-cli
    depends_on:
      - broker
      - ksqldb-server
    entrypoint: /bin/sh
    tty: true

  postgres:
    # *-----------------------------*
    # To connect to the DB: 
    #   docker-compose exec postgres bash -c 'psql -U $POSTGRES_USER $POSTGRES_DB'
    # *-----------------------------*
    image: postgres
    hostname: postgres
    container_name: postgres
    environment:
     - POSTGRES_USER=postgres
     - POSTGRES_PASSWORD=postgres
    # volumes:
    #  - ./data/postgres:/docker-entrypoint-initdb.d/

I checked the log of the ksqldb-server container and this is the response:
$ docker logs ksqldb-server ===> Configuring ksqlDB... ===> Launching ksqlDB Server... [2019-12-23 10:59:56,473] INFO KsqlConfig values: ksql.access.validator.enable = auto ksql.avro.maps.named = true ksql.connect.url = http://localhost:8083 ksql.connect.worker.config = C:/Users/User01/Documents/confluent/share/java/connect.properties ksql.extension.dir = ext ksql.functions.substring.legacy.args = false ksql.insert.into.values.enabled = true ksql.internal.topic.replicas = 1 ksql.metric.reporters = [] ksql.metrics.extension = null ksql.metrics.tags.custom = ksql.named.internal.topics = on ksql.output.topic.name.prefix = ksql.persistence.wrap.single.values = true ksql.persistent.prefix = query_ ksql.query.inject.legacy.map.values.node = false ksql.query.persistent.active.limit = 2147483647 ksql.query.pull.enable = true ksql.query.pull.routing.timeout.ms = 30000 ksql.query.pull.skip.access.validator = false ksql.query.pull.streamsstore.rebalancing.timeout.ms = 10000 ksql.query.stream.groupby.rowkey.repartition = false ksql.schema.registry.url = http://localhost:8081 ksql.security.extension.class = null ksql.service.id = default_ ksql.sink.partitions = null ksql.sink.replicas = null ksql.sink.window.change.log.additional.retention = 1000000 ksql.transient.prefix = transient_ ksql.udf.collect.metrics = false ksql.udf.enable.security.manager = true ksql.udfs.enabled = true ksql.windowed.session.key.legacy = false ssl.cipher.suites = null ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1] ssl.endpoint.identification.algorithm = https ssl.key.password = null ssl.keymanager.algorithm = SunX509 ssl.keystore.location = null ssl.keystore.password = null ssl.keystore.type = JKS ssl.protocol = TLS ssl.provider = null ssl.secure.random.implementation = null ssl.trustmanager.algorithm = PKIX ssl.truststore.location = null ssl.truststore.password = null ssl.truststore.type = JKS (io.confluent.ksql.util.KsqlConfig:347) [2019-12-23 10:59:56,563] INFO KsqlRestConfig values: access.control.allow.headers = access.control.allow.methods = access.control.allow.origin = authentication.method = NONE authentication.realm = authentication.roles = [*] authentication.skip.paths = [] compression.enable = true debug = false idle.timeout.ms = 30000 ksql.healthcheck.interval.ms = 5000 ksql.server.command.response.timeout.ms = 5000 ksql.server.exception.uncaught.handler.enable = false ksql.server.install.dir = /usr ksql.server.preconditions = [] ksql.server.websockets.num.threads = 5 listeners = [http://0.0.0.0:8088] metric.reporters = [] metrics.jmx.prefix = rest-utils metrics.num.samples = 2 metrics.sample.window.ms = 30000 metrics.tag.map = [] port = 8080 query.stream.disconnect.check = 1000 request.logger.name = io.confluent.rest-utils.requests resource.extension.classes = [] response.mediatype.default = application/json response.mediatype.preferred = [application/json] rest.servlet.initializor.classes = [] shutdown.graceful.ms = 1000 ssl.cipher.suites = [] ssl.client.auth = false ssl.client.authentication = NONE ssl.enabled.protocols = [] ssl.endpoint.identification.algorithm = null ssl.key.password = [hidden] ssl.keymanager.algorithm = ssl.keystore.location = ssl.keystore.password = [hidden] ssl.keystore.type = JKS ssl.protocol = TLS ssl.provider = ssl.trustmanager.algorithm = ssl.truststore.location = ssl.truststore.password = [hidden] ssl.truststore.type = JKS websocket.path.prefix = /ws websocket.servlet.initializor.classes = [] (io.confluent.ksql.rest.server.KsqlRestConfig:347) [2019-12-23 10:59:56,630] INFO Logging initialized @2909ms to org.eclipse.jetty.util.log.Slf4jLog (org.eclipse.jetty.util.log:169) [2019-12-23 10:59:56,950] INFO Adding listener: http://0.0.0.0:8088 (io.confluent.rest.ApplicationServer:314) [2019-12-23 10:59:57,055] INFO KsqlConfig values: ksql.access.validator.enable = auto ksql.avro.maps.named = true ksql.connect.url = http://localhost:8083 ksql.connect.worker.config = C:/Users/User01/Documents/confluent/share/java/connect.properties ksql.extension.dir = ext ksql.functions.substring.legacy.args = false ksql.insert.into.values.enabled = true ksql.internal.topic.replicas = 1 ksql.metric.reporters = [] ksql.metrics.extension = null ksql.metrics.tags.custom = ksql.named.internal.topics = on ksql.output.topic.name.prefix = ksql.persistence.wrap.single.values = true ksql.persistent.prefix = query_ ksql.query.inject.legacy.map.values.node = false ksql.query.persistent.active.limit = 2147483647 ksql.query.pull.enable = true ksql.query.pull.routing.timeout.ms = 30000 ksql.query.pull.skip.access.validator = false ksql.query.pull.streamsstore.rebalancing.timeout.ms = 10000 ksql.query.stream.groupby.rowkey.repartition = false ksql.schema.registry.url = http://localhost:8081 ksql.security.extension.class = null ksql.service.id = default_ ksql.sink.partitions = null ksql.sink.replicas = null ksql.sink.window.change.log.additional.retention = 1000000 ksql.transient.prefix = transient_ ksql.udf.collect.metrics = false ksql.udf.enable.security.manager = true ksql.udfs.enabled = true ksql.windowed.session.key.legacy = false ssl.cipher.suites = null ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1] ssl.endpoint.identification.algorithm = https ssl.key.password = null ssl.keymanager.algorithm = SunX509 ssl.keystore.location = null ssl.keystore.password = null ssl.keystore.type = JKS ssl.protocol = TLS ssl.provider = null ssl.secure.random.implementation = null ssl.trustmanager.algorithm = PKIX ssl.truststore.location = null ssl.truststore.password = null ssl.truststore.type = JKS (io.confluent.ksql.util.KsqlConfig:347) [2019-12-23 10:59:57,187] INFO KsqlConfig values: ksql.access.validator.enable = auto ksql.avro.maps.named = true ksql.connect.url = http://localhost:8083 ksql.connect.worker.config = C:/Users/User01/Documents/confluent/share/java/connect.properties ksql.extension.dir = ext ksql.functions.substring.legacy.args = false ksql.insert.into.values.enabled = true ksql.internal.topic.replicas = 1 ksql.metric.reporters = [] ksql.metrics.extension = null ksql.metrics.tags.custom = ksql.named.internal.topics = on ksql.output.topic.name.prefix = ksql.persistence.wrap.single.values = true ksql.persistent.prefix = query_ ksql.query.inject.legacy.map.values.node = false ksql.query.persistent.active.limit = 2147483647 ksql.query.pull.enable = true ksql.query.pull.routing.timeout.ms = 30000 ksql.query.pull.skip.access.validator = false ksql.query.pull.streamsstore.rebalancing.timeout.ms = 10000 ksql.query.stream.groupby.rowkey.repartition = false ksql.schema.registry.url = http://localhost:8081 ksql.security.extension.class = null ksql.service.id = default_ ksql.sink.partitions = null ksql.sink.replicas = null ksql.sink.window.change.log.additional.retention = 1000000 ksql.transient.prefix = transient_ ksql.udf.collect.metrics = false ksql.udf.enable.security.manager = true ksql.udfs.enabled = true ksql.windowed.session.key.legacy = false ssl.cipher.suites = null ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1] ssl.endpoint.identification.algorithm = https ssl.key.password = null ssl.keymanager.algorithm = SunX509 ssl.keystore.location = null ssl.keystore.password = null ssl.keystore.type = JKS ssl.protocol = TLS ssl.provider = null ssl.secure.random.implementation = null ssl.trustmanager.algorithm = PKIX ssl.truststore.location = null ssl.truststore.password = null ssl.truststore.type = JKS (io.confluent.ksql.util.KsqlConfig:347) [2019-12-23 10:59:57,262] INFO ProcessingLogConfig values: ksql.logging.processing.rows.include = false ksql.logging.processing.stream.auto.create = true ksql.logging.processing.stream.name = KSQL_PROCESSING_LOG ksql.logging.processing.topic.auto.create = true ksql.logging.processing.topic.name = ksql.logging.processing.topic.partitions = 1 ksql.logging.processing.topic.replication.factor = 1 (io.confluent.ksql.logging.processing.ProcessingLogConfig:347) [2019-12-23 10:59:58,535] INFO Blacklist file: /usr/ext/resource-blacklist.txt not found. No classes will be blacklisted (io.confluent.ksql.function.Blacklist:55) [2019-12-23 10:59:59,225] INFO Adding function ENTRIES for method public java.util.List io.confluent.ksql.function.udf.array.Entries.entriesInt(java.util.Map,boolean) (io.confluent.ksql.function.UdfLoader:143) [2019-12-23 10:59:59,709] INFO Adding function ENTRIES for method public java.util.List io.confluent.ksql.function.udf.array.Entries.entriesBigInt(java.util.Map,boolean) (io.confluent.ksql.function.UdfLoader:143) [2019-12-23 10:59:59,718] INFO Adding function ENTRIES for method public java.util.List io.confluent.ksql.function.udf.array.Entries.entriesDouble(java.util.Map,boolean) (io.confluent.ksql.function.UdfLoader:143) [2019-12-23 10:59:59,738] INFO Adding function ENTRIES for method public java.util.List io.confluent.ksql.function.udf.array.Entries.entriesBoolean(java.util.Map,boolean) (io.confluent.ksql.function.UdfLoader:143) [2019-12-23 10:59:59,749] INFO Adding function ENTRIES for method public java.util.List io.confluent.ksql.function.udf.array.Entries.entriesString(java.util.Map,boolean) (io.confluent.ksql.function.UdfLoader:143) [2019-12-23 10:59:59,767] INFO Adding function GENERATE_SERIES for method public java.util.List io.confluent.ksql.function.udf.array.GenerateSeries.generateSeriesInt(int,int,int) (io.confluent.ksql.function.UdfLoader:143) [2019-12-23 10:59:59,771] INFO Adding function GENERATE_SERIES for method public java.util.List io.confluent.ksql.function.udf.array.GenerateSeries.generateSeriesInt(int,int) (io.confluent.ksql.function.UdfLoader:143) [2019-12-23 10:59:59,784] INFO Adding function GENERATE_SERIES for method public java.util.List io.confluent.ksql.function.udf.array.GenerateSeries.generateSeriesLong(long,long,int) (io.confluent.ksql.function.UdfLoader:143) [2019-12-23 10:59:59,789] INFO Adding function GENERATE_SERIES for method public java.util.List io.confluent.ksql.function.udf.array.GenerateSeries.generateSeriesLong(long,long) (io.confluent.ksql.function.UdfLoader:143) [2019-12-23 10:59:59,868] INFO Adding function datetostring for method public java.lang.String io.confluent.ksql.function.udf.datetime.DateToString.dateToString(int,java.lang.String) (io.confluent.ksql.function.UdfLoader:143) [2019-12-23 10:59:59,871] INFO Adding function stringtodate for method public int io.confluent.ksql.function.udf.datetime.StringToDate.stringToDate(java.lang.String,java.lang.String) (io.confluent.ksql.function.UdfLoader:143) [2019-12-23 10:59:59,898] INFO Adding function stringtotimestamp for method public long io.confluent.ksql.function.udf.datetime.StringToTimestamp.stringToTimestamp(java.lang.String,java.lang.String) (io.confluent.ksql.function.UdfLoader:143) [2019-12-23 10:59:59,899] INFO Adding function stringtotimestamp for method public long io.confluent.ksql.function.udf.datetime.StringToTimestamp.stringToTimestamp(java.lang.String,java.lang.String,java.lang.String) (io.confluent.ksql.function.UdfLoader:143) [2019-12-23 10:59:59,908] INFO Adding function timestamptostring for method public java.lang.String io.confluent.ksql.function.udf.datetime.TimestampToString.timestampToString(long,java.lang.String) (io.confluent.ksql.function.UdfLoader:143) [2019-12-23 10:59:59,908] INFO Adding function timestamptostring for method public java.lang.String io.confluent.ksql.function.udf.datetime.TimestampToString.timestampToString(long,java.lang.String,java.lang.String) (io.confluent.ksql.function.UdfLoader:143) [2019-12-23 10:59:59,910] INFO Adding function unix_date for method public int io.confluent.ksql.function.udf.datetime.UnixDate.unixDate() (io.confluent.ksql.function.UdfLoader:143) [2019-12-23 10:59:59,911] INFO Adding function unix_timestamp for method public long io.confluent.ksql.function.udf.datetime.UnixTimestamp.unixTimestamp() (io.confluent.ksql.function.UdfLoader:143) [2019-12-23 10:59:59,915] INFO Adding function geo_distance for method public java.lang.Double io.confluent.ksql.function.udf.geo.GeoDistance.geoDistance(double,double,double,double) (io.confluent.ksql.function.UdfLoader:143) [2019-12-23 10:59:59,916] INFO Adding function geo_distance for method public java.lang.Double io.confluent.ksql.function.udf.geo.GeoDistance.geoDistance(double,double,double,double,java.lang.String) (io.confluent.ksql.function.UdfLoader:143) [2019-12-23 10:59:59,948] INFO Adding function AS_ARRAY for method public final java.util.List io.confluent.ksql.function.udf.list.AsArray.asArray(java.lang.Object[]) (io.confluent.ksql.function.UdfLoader:143) [2019-12-23 10:59:59,950] INFO Adding function slice for method public java.util.List io.confluent.ksql.function.udf.list.Slice.slice(java.util.List,java.lang.Integer,java.lang.Integer) (io.confluent.ksql.function.UdfLoader:143) [2019-12-23 10:59:59,951] INFO Adding function AS_MAP for method public final java.util.Map io.confluent.ksql.function.udf.map.AsMap.asMap(java.util.List,java.util.List) (io.confluent.ksql.function.UdfLoader:143) [2019-12-23 10:59:59,954] INFO Adding function Abs for method public java.lang.Double io.confluent.ksql.function.udf.math.Abs.abs(java.lang.Double) (io.confluent.ksql.function.UdfLoader:143) [2019-12-23 10:59:59,955] INFO Adding function Abs for method public java.math.BigDecimal io.confluent.ksql.function.udf.math.Abs.abs(java.math.BigDecimal) (io.confluent.ksql.function.UdfLoader:143) [2019-12-23 10:59:59,956] INFO Adding function Abs for method public java.lang.Double io.confluent.ksql.function.udf.math.Abs.abs(java.lang.Integer) (io.confluent.ksql.function.UdfLoader:143) [2019-12-23 10:59:59,957] INFO Adding function Abs for method public java.lang.Double io.confluent.ksql.function.udf.math.Abs.abs(java.lang.Long) (io.confluent.ksql.function.UdfLoader:143) [2019-12-23 10:59:59,997] INFO Adding function exp for method public java.lang.Double io.confluent.ksql.function.udf.math.Exp.exp(java.lang.Double) (io.confluent.ksql.function.UdfLoader:143) [2019-12-23 10:59:59,999] INFO Adding function exp for method public java.lang.Double io.confluent.ksql.function.udf.math.Exp.exp(java.lang.Long) (io.confluent.ksql.function.UdfLoader:143) [2019-12-23 11:00:00,000] INFO Adding function exp for method public java.lang.Double io.confluent.ksql.function.udf.math.Exp.exp(java.lang.Integer) (io.confluent.ksql.function.UdfLoader:143) [2019-12-23 11:00:00,002] INFO Adding function Floor for method public java.lang.Double io.confluent.ksql.function.udf.math.Floor.floor(java.math.BigDecimal) (io.confluent.ksql.function.UdfLoader:143) [2019-12-23 11:00:00,004] INFO Adding function Floor for method public java.lang.Double io.confluent.ksql.function.udf.math.Floor.floor(java.lang.Double) (io.confluent.ksql.function.UdfLoader:143) [2019-12-23 11:00:00,006] INFO Adding function Floor for method public java.lang.Double io.confluent.ksql.function.udf.math.Floor.floor(java.lang.Long) (io.confluent.ksql.function.UdfLoader:143) [2019-12-23 11:00:00,027] INFO Adding function Floor for method public java.lang.Double io.confluent.ksql.function.udf.math.Floor.floor(java.lang.Integer) (io.confluent.ksql.function.UdfLoader:143) [2019-12-23 11:00:00,030] INFO Adding function ln for method public java.lang.Double io.confluent.ksql.function.udf.math.Ln.ln(java.lang.Double) (io.confluent.ksql.function.UdfLoader:143) [2019-12-23 11:00:00,030] INFO Adding function ln for method public java.lang.Double io.confluent.ksql.function.udf.math.Ln.ln(java.lang.Integer) (io.confluent.ksql.function.UdfLoader:143) [2019-12-23 11:00:00,031] INFO Adding function ln for method public java.lang.Double io.confluent.ksql.function.udf.math.Ln.ln(java.lang.Long) (io.confluent.ksql.function.UdfLoader:143) [2019-12-23 11:00:00,039] INFO Adding function Round for method public java.math.BigDecimal io.confluent.ksql.function.udf.math.Round.round(java.math.BigDecimal) (io.confluent.ksql.function.UdfLoader:143) [2019-12-23 11:00:00,039] INFO Adding function Round for method public java.lang.Long io.confluent.ksql.function.udf.math.Round.round(long) (io.confluent.ksql.function.UdfLoader:143) [2019-12-23 11:00:00,040] INFO Adding function Round for method public java.lang.Long io.confluent.ksql.function.udf.math.Round.round(int) (io.confluent.ksql.function.UdfLoader:143) [2019-12-23 11:00:00,048] INFO Adding function Round for method public java.math.BigDecimal io.confluent.ksql.function.udf.math.Round.round(java.math.BigDecimal,java.lang.Integer) (io.confluent.ksql.function.UdfLoader:143) [2019-12-23 11:00:00,068] INFO Adding function Round for method public java.lang.Long io.confluent.ksql.function.udf.math.Round.round(java.lang.Double) (io.confluent.ksql.function.UdfLoader:143) [2019-12-23 11:00:00,069] INFO Adding function Round for method public java.lang.Double io.confluent.ksql.function.udf.math.Round.round(java.lang.Double,java.lang.Integer) (io.confluent.ksql.function.UdfLoader:143) [2019-12-23 11:00:00,072] INFO Adding function sign for method public java.lang.Integer io.confluent.ksql.function.udf.math.Sign.sign(java.lang.Double) (io.confluent.ksql.function.UdfLoader:143) [2019-12-23 11:00:00,078] INFO Adding function sign for method public java.lang.Integer io.confluent.ksql.function.udf.math.Sign.sign(java.lang.Integer) (io.confluent.ksql.function.UdfLoader:143) [2019-12-23 11:00:00,079] INFO Adding function sign for method public java.lang.Integer io.confluent.ksql.function.udf.math.Sign.sign(java.lang.Long) (io.confluent.ksql.function.UdfLoader:143) [2019-12-23 11:00:00,081] INFO Adding function sqrt for method public java.lang.Double io.confluent.ksql.function.udf.math.Sqrt.sqrt(java.lang.Double) (io.confluent.ksql.function.UdfLoader:143) [2019-12-23 11:00:00,088] INFO Adding function sqrt for method public java.lang.Double io.confluent.ksql.function.udf.math.Sqrt.sqrt(java.lang.Long) (io.confluent.ksql.function.UdfLoader:143) [2019-12-23 11:00:00,088] INFO Adding function sqrt for method public java.lang.Double io.confluent.ksql.function.udf.math.Sqrt.sqrt(java.lang.Integer) (io.confluent.ksql.function.UdfLoader:143) [2019-12-23 11:00:00,108] INFO Adding function elt for method public java.lang.String io.confluent.ksql.function.udf.string.Elt.elt(int,java.lang.String[]) (io.confluent.ksql.function.UdfLoader:143) [2019-12-23 11:00:00,112] INFO Adding function field for method public int io.confluent.ksql.function.udf.string.Field.field(java.lang.String,java.lang.String[]) (io.confluent.ksql.function.UdfLoader:143) [2019-12-23 11:00:00,114] INFO Adding function initcap for method public java.lang.String io.confluent.ksql.function.udf.string.InitCap.initcap(java.lang.String) (io.confluent.ksql.function.UdfLoader:143) [2019-12-23 11:00:00,127] INFO Adding function mask_keep_left for method public java.lang.String io.confluent.ksql.function.udf.string.MaskKeepLeftKudf.mask(java.lang.String,int) (io.confluent.ksql.function.UdfLoader:143) [2019-12-23 11:00:00,129] INFO Adding function mask_keep_left for method public java.lang.String io.confluent.ksql.function.udf.string.MaskKeepLeftKudf.mask(java.lang.String,int,java.lang.String,java.lang.String,java.lang.String,java.lang.String) (io.confluent.ksql.function.UdfLoader:143) [2019-12-23 11:00:00,131] INFO Adding function mask_keep_right for method public java.lang.String io.confluent.ksql.function.udf.string.MaskKeepRightKudf.mask(java.lang.String,int) (io.confluent.ksql.function.UdfLoader:143) [2019-12-23 11:00:00,148] INFO Adding function mask_keep_right for method public java.lang.String io.confluent.ksql.function.udf.string.MaskKeepRightKudf.mask(java.lang.String,int,java.lang.String,java.lang.String,java.lang.String,java.lang.String) (io.confluent.ksql.function.UdfLoader:143) [2019-12-23 11:00:00,151] INFO Adding function mask for method public java.lang.String io.confluent.ksql.function.udf.string.MaskKudf.mask(java.lang.String) (io.confluent.ksql.function.UdfLoader:143) [2019-12-23 11:00:00,152] INFO Adding function mask for method public java.lang.String io.confluent.ksql.function.udf.string.MaskKudf.mask(java.lang.String,java.lang.String,java.lang.String,java.lang.String,java.lang.String) (io.confluent.ksql.function.UdfLoader:143) [2019-12-23 11:00:00,165] INFO Adding function mask_left for method public java.lang.String io.confluent.ksql.function.udf.string.MaskLeftKudf.mask(java.lang.String,int) (io.confluent.ksql.function.UdfLoader:143) [2019-12-23 11:00:00,166] INFO Adding function mask_left for method public java.lang.String io.confluent.ksql.function.udf.string.MaskLeftKudf.mask(java.lang.String,int,java.lang.String,java.lang.String,java.lang.String,java.lang.String) (io.confluent.ksql.function.UdfLoader:143) [2019-12-23 11:00:00,170] INFO Adding function mask_right for method public java.lang.String io.confluent.ksql.function.udf.string.MaskRightKudf.mask(java.lang.String,int) (io.confluent.ksql.function.UdfLoader:143) [2019-12-23 11:00:00,198] INFO Adding function mask_right for method public java.lang.String io.confluent.ksql.function.udf.string.MaskRightKudf.mask(java.lang.String,int,java.lang.String,java.lang.String,java.lang.String,java.lang.String) (io.confluent.ksql.function.UdfLoader:143) [2019-12-23 11:00:00,199] INFO Adding function replace for method public java.lang.String io.confluent.ksql.function.udf.string.Replace.replace(java.lang.String,java.lang.String,java.lang.String) (io.confluent.ksql.function.UdfLoader:143) [2019-12-23 11:00:00,201] INFO Adding function split for method public java.util.List io.confluent.ksql.function.udf.string.SplitKudf.split(java.lang.String,java.lang.String) (io.confluent.ksql.function.UdfLoader:143) [2019-12-23 11:00:00,203] INFO Adding function substring for method public java.lang.String io.confluent.ksql.function.udf.string.Substring.substring(java.lang.String,java.lang.Integer,java.lang.Integer) (io.confluent.ksql.function.UdfLoader:143) [2019-12-23 11:00:00,218] INFO Adding function substring for method public java.lang.String io.confluent.ksql.function.udf.string.Substring.substring(java.lang.String,java.lang.Integer) (io.confluent.ksql.function.UdfLoader:143) [2019-12-23 11:00:00,219] INFO Adding function url_decode_param for method public java.lang.String io.confluent.ksql.function.udf.url.UrlDecodeParamKudf.decodeParam(java.lang.String) (io.confluent.ksql.function.UdfLoader:143) [2019-12-23 11:00:00,220] INFO Adding function url_encode_param for method public java.lang.String io.confluent.ksql.function.udf.url.UrlEncodeParamKudf.encodeParam(java.lang.String) (io.confluent.ksql.function.UdfLoader:143) [2019-12-23 11:00:00,222] INFO Adding function url_extract_fragment for method public java.lang.String io.confluent.ksql.function.udf.url.UrlExtractFragmentKudf.extractFragment(java.lang.String) (io.confluent.ksql.function.UdfLoader:143) [2019-12-23 11:00:00,227] INFO Adding function url_extract_host for method public java.lang.String io.confluent.ksql.function.udf.url.UrlExtractHostKudf.extractHost(java.lang.String) (io.confluent.ksql.function.UdfLoader:143) [2019-12-23 11:00:00,252] INFO Adding function url_extract_parameter for method public java.lang.String io.confluent.ksql.function.udf.url.UrlExtractParameterKudf.extractParam(java.lang.String,java.lang.String) (io.confluent.ksql.function.UdfLoader:143) [2019-12-23 11:00:00,259] INFO Adding function url_extract_path for method public java.lang.String io.confluent.ksql.function.udf.url.UrlExtractPathKudf.extractPath(java.lang.String) (io.confluent.ksql.function.UdfLoader:143) [2019-12-23 11:00:00,277] INFO Adding function url_extract_port for method public java.lang.Integer io.confluent.ksql.function.udf.url.UrlExtractPortKudf.extractPort(java.lang.String) (io.confluent.ksql.function.UdfLoader:143) [2019-12-23 11:00:00,279] INFO Adding function url_extract_protocol for method public java.lang.String io.confluent.ksql.function.udf.url.UrlExtractProtocolKudf.extractProtocol(java.lang.String) (io.confluent.ksql.function.UdfLoader:143) [2019-12-23 11:00:00,289] INFO Adding function url_extract_query for method public java.lang.String io.confluent.ksql.function.udf.url.UrlExtractQueryKudf.extractQuery(java.lang.String) (io.confluent.ksql.function.UdfLoader:143) [2019-12-23 11:00:00,302] INFO Adding UDAF name=collect_list from path=internal class=class io.confluent.ksql.function.udaf.array.CollectListUdaf (io.confluent.ksql.function.UdafLoader:71) [2019-12-23 11:00:00,338] INFO Adding UDAF name=collect_list from path=internal class=class io.confluent.ksql.function.udaf.array.CollectListUdaf (io.confluent.ksql.function.UdafLoader:71) [2019-12-23 11:00:00,339] INFO Adding UDAF name=collect_list from path=internal class=class io.confluent.ksql.function.udaf.array.CollectListUdaf (io.confluent.ksql.function.UdafLoader:71) [2019-12-23 11:00:00,341] INFO Adding UDAF name=collect_list from path=internal class=class io.confluent.ksql.function.udaf.array.CollectListUdaf (io.confluent.ksql.function.UdafLoader:71) [2019-12-23 11:00:00,343] INFO Adding UDAF name=collect_list from path=internal class=class io.confluent.ksql.function.udaf.array.CollectListUdaf (io.confluent.ksql.function.UdafLoader:71) [2019-12-23 11:00:00,368] INFO Adding UDAF name=collect_set from path=internal class=class io.confluent.ksql.function.udaf.array.CollectSetUdaf (io.confluent.ksql.function.UdafLoader:71) [2019-12-23 11:00:00,369] INFO Adding UDAF name=collect_set from path=internal class=class io.confluent.ksql.function.udaf.array.CollectSetUdaf (io.confluent.ksql.function.UdafLoader:71) [2019-12-23 11:00:00,370] INFO Adding UDAF name=collect_set from path=internal class=class io.confluent.ksql.function.udaf.array.CollectSetUdaf (io.confluent.ksql.function.UdafLoader:71) [2019-12-23 11:00:00,371] INFO Adding UDAF name=collect_set from path=internal class=class io.confluent.ksql.function.udaf.array.CollectSetUdaf (io.confluent.ksql.function.UdafLoader:71) [2019-12-23 11:00:00,372] INFO Adding UDAF name=collect_set from path=internal class=class io.confluent.ksql.function.udaf.array.CollectSetUdaf (io.confluent.ksql.function.UdafLoader:71) [2019-12-23 11:00:00,374] INFO Adding UDAF name=avg from path=internal class=class io.confluent.ksql.function.udaf.average.AverageUdaf (io.confluent.ksql.function.UdafLoader:71) [2019-12-23 11:00:00,389] INFO Adding UDAF name=avg from path=internal class=class io.confluent.ksql.function.udaf.average.AverageUdaf (io.confluent.ksql.function.UdafLoader:71) [2019-12-23 11:00:00,396] INFO Adding UDAF name=avg from path=internal class=class io.confluent.ksql.function.udaf.average.AverageUdaf (io.confluent.ksql.function.UdafLoader:71) [2019-12-23 11:00:00,398] INFO Adding UDAF name=histogram from path=internal class=class io.confluent.ksql.function.udaf.map.HistogramUdaf (io.confluent.ksql.function.UdafLoader:71) [2019-12-23 11:00:00,399] INFO Adding UDAF name=sum_list from path=internal class=class io.confluent.ksql.function.udaf.sum.ListSumUdaf (io.confluent.ksql.function.UdafLoader:71) [2019-12-23 11:00:00,400] INFO Adding UDAF name=sum_list from path=internal class=class io.confluent.ksql.function.udaf.sum.ListSumUdaf (io.confluent.ksql.function.UdafLoader:71) [2019-12-23 11:00:00,407] INFO Adding UDAF name=sum_list from path=internal class=class io.confluent.ksql.function.udaf.sum.ListSumUdaf (io.confluent.ksql.function.UdafLoader:71) [2019-12-23 11:00:00,409] INFO Adding UDAF name=WindowEnd from path=internal class=class io.confluent.ksql.function.udaf.window.WindowEndKudaf (io.confluent.ksql.function.UdafLoader:71) [2019-12-23 11:00:00,410] INFO Adding UDAF name=WindowStart from path=internal class=class io.confluent.ksql.function.udaf.window.WindowStartKudaf (io.confluent.ksql.function.UdafLoader:71) [2019-12-23 11:00:00,429] INFO UDFs can't be loaded as as dir /usr/ext doesn't exist or is not a directory (io.confluent.ksql.function.UserFunctionLoader:86) [2019-12-23 11:00:00,450] INFO KafkaJsonDeserializerConfig values: json.fail.unknown.properties = true json.key.type = class io.confluent.ksql.rest.entity.CommandId json.value.type = class java.lang.Object (io.confluent.kafka.serializers.KafkaJsonDeserializerConfig:347) [2019-12-23 11:00:00,451] INFO KafkaJsonDeserializerConfig values: json.fail.unknown.properties = true json.key.type = class java.lang.Object json.value.type = class io.confluent.ksql.rest.server.computation.Command (io.confluent.kafka.serializers.KafkaJsonDeserializerConfig:347) [2019-12-23 11:00:00,834] INFO KafkaJsonSerializerConfig values: json.indent.output = false (io.confluent.kafka.serializers.KafkaJsonSerializerConfig:347) [2019-12-23 11:00:00,890] INFO KafkaJsonSerializerConfig values: json.indent.output = false (io.confluent.kafka.serializers.KafkaJsonSerializerConfig:347) [2019-12-23 11:00:08,406] INFO KsqlRestConfig values: access.control.allow.headers = access.control.allow.methods = access.control.allow.origin = authentication.method = NONE authentication.realm = authentication.roles = [*] authentication.skip.paths = [/v1/metadata, /v1/metadata/id, /healthcheck] compression.enable = true debug = false idle.timeout.ms = 30000 ksql.healthcheck.interval.ms = 5000 ksql.server.command.response.timeout.ms = 5000 ksql.server.exception.uncaught.handler.enable = false ksql.server.install.dir = /usr ksql.server.preconditions = [] ksql.server.websockets.num.threads = 5 listeners = [http://0.0.0.0:8088] metric.reporters = [] metrics.jmx.prefix = rest-utils metrics.num.samples = 2 metrics.sample.window.ms = 30000 metrics.tag.map = [] port = 8080 query.stream.disconnect.check = 1000 request.logger.name = io.confluent.rest-utils.requests resource.extension.classes = [] response.mediatype.default = application/json response.mediatype.preferred = [application/json] rest.servlet.initializor.classes = [] shutdown.graceful.ms = 1000 ssl.cipher.suites = [] ssl.client.auth = false ssl.client.authentication = NONE ssl.enabled.protocols = [] ssl.endpoint.identification.algorithm = null ssl.key.password = [hidden] ssl.keymanager.algorithm = ssl.keystore.location = ssl.keystore.password = [hidden] ssl.keystore.type = JKS ssl.protocol = TLS ssl.provider = ssl.trustmanager.algorithm = ssl.truststore.location = ssl.truststore.password = [hidden] ssl.truststore.type = JKS websocket.path.prefix = /ws websocket.servlet.initializor.classes = [] (io.confluent.ksql.rest.server.KsqlRestConfig:347) [2019-12-23 11:00:08,421] ERROR Failed to start KSQL (io.confluent.ksql.rest.server.KsqlServerMain:63) java.io.IOException: Is a directory at sun.nio.ch.FileDispatcherImpl.read0(Native Method) at sun.nio.ch.FileDispatcherImpl.read(FileDispatcherImpl.java:46) at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223) at sun.nio.ch.IOUtil.read(IOUtil.java:197) at sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:159) at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:65) at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:109) at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:103) at java.io.InputStream.read(InputStream.java:101) at java.util.Properties$LineReader.readLine(Properties.java:435) at java.util.Properties.load0(Properties.java:353) at java.util.Properties.load(Properties.java:341) at org.apache.kafka.common.utils.Utils.loadProps(Utils.java:564) at io.confluent.ksql.rest.server.ConnectExecutable.of(ConnectExecutable.java:41) at io.confluent.ksql.rest.server.KsqlServerMain.createExecutable(KsqlServerMain.java:109) at io.confluent.ksql.rest.server.KsqlServerMain.main(KsqlServerMain.java:59)

[McHardex]

I got this to work by adding KSQL_KSQL_CONNECT_URL to my ksqldb-server environment

KSQL_KSQL_CONNECT_URL: http://connect:8083

[basharat58]

@McHardex I am following https://ksqldb.io/quickstart.html (Windows 10) and have modified my docker file to be similar to yours above. It is:

---
version: '2'

services:
  zookeeper:
    image: confluentinc/cp-zookeeper:5.3.1
    hostname: zookeeper
    container_name: zookeeper
    ports:
      - "2181:2181"
    environment:
      ZOOKEEPER_CLIENT_PORT: 2181
      ZOOKEEPER_TICK_TIME: 2000

  broker:
    image: confluentinc/cp-enterprise-kafka:5.3.1
    hostname: broker
    container_name: broker
    depends_on:
      - zookeeper
    ports:
      - "9092:9092"
      - "29092:29092"
    environment:
      KAFKA_BROKER_ID: 1
      KAFKA_ZOOKEEPER_CONNECT: 'zookeeper:2181'
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://broker:9092,PLAINTEXT_HOST://localhost:29092
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
      KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS: 0

  schema-registry:
    image: confluentinc/cp-schema-registry:5.3.1
    hostname: schema-registry
    container_name: schema-registry
    depends_on:
      - zookeeper
      - broker
    ports:
      - "8081:8081"
    environment:
      SCHEMA_REGISTRY_HOST_NAME: schema-registry
      SCHEMA_REGISTRY_KAFKASTORE_CONNECTION_URL: zookeeper:2181

  connect:
    image: confluentinc/cp-kafka-connect:latest
    hostname: connect
    container_name: connect
    depends_on:
      - zookeeper
      - broker
      - schema-registry
    ports:
      - "8083:8083"
    environment:
      CONNECT_BOOTSTRAP_SERVERS: broker:29092
      CONNECT_REST_ADVERTISED_HOST_NAME: localhost
      CONNECT_REST_PORT: 8083
      CONNECT_GROUP_ID: ksql-connect-cluster
      CONNECT_OFFSET_STORAGE_TOPIC: ksql-connect-configs
      CONNECT_CONFIG_STORAGE_TOPIC: ksql-connect-topics
      CONNECT_STATUS_STORAGE_TOPIC: ksql-connect-statuses
      CONNECT_CONFIG_STORAGE_REPLICATION_FACTOR: 1
      CONNECT_OFFSET_FLUSH_INTERVAL_MS: 10000
      CONNECT_OFFSET_STORAGE_REPLICATION_FACTOR: 1
      CONNECT_STATUS_STORAGE_REPLICATION_FACTOR: 1
      CONNECT_KEY_CONVERTER: org.apache.kafka.connect.storage.StringConverter
      CONNECT_VALUE_CONVERTER: io.confluent.connect.avro.AvroConverter
      CONNECT_VALUE_CONVERTER_SCHEMA_REGISTRY_URL: http://schema-registry:8081
      CONNECT_INTERNAL_KEY_CONVERTER: "org.apache.kafka.connect.json.JsonConverter"
      CONNECT_INTERNAL_VALUE_CONVERTER: "org.apache.kafka.connect.json.JsonConverter"
      CONNECT_ZOOKEEPER_CONNECT: 'zookeeper:2181'
      #CLASSPATH: C:/Users/User01/Documents/confluent/share/confluent-hub-components      
      #CONNECT_PLUGIN_PATH: C:/Users/User01/Documents/confluent/share/confluent-hub-components      
      KSQL_KSQL_CONNECT_WORKER_CONFIG: C:/Users/basharat.hussain/GoogleTime/Kafka/connect.properties
    volumes: 
      - C:/Users/basharat.hussain/GoogleTime/Kafka/connect.properties

  ksqldb-server:
    image: confluentinc/ksqldb-server:0.6.0
    hostname: ksqldb-server
    container_name: ksqldb-server
    depends_on:
      - broker
    ports:
      - "8088:8088"
    environment:
      KSQL_LISTENERS: http://0.0.0.0:8088
      KSQL_BOOTSTRAP_SERVERS: broker:29092
      KSQL_KSQL_LOGGING_PROCESSING_STREAM_AUTO_CREATE: "true"
      KSQL_KSQL_LOGGING_PROCESSING_TOPIC_AUTO_CREATE: "true"
      KSQL_KSQL_CONNECT_WORKER_CONFIG: C:/Users/basharat.hussain/GoogleTime/Kafka/connect.properties
      KSQL_KSQL_CONNECT_URL: http://connect:8083
    volumes: 
        - C:/Users/basharat.hussain/GoogleTime/Kafka/connect.properties

  ksqldb-cli:
    image: confluentinc/ksqldb-cli:0.6.0
    container_name: ksqldb-cli
    depends_on:
      - broker
      - ksqldb-server
    entrypoint: /bin/sh
    tty: true

When I try to run docker-compose up it gives me an error:

I see you are referencing (althoug I copied your file, I don't have confluent-hub-components on my machine:

CLASSPATH: C:/Users/User01/Documents/confluent/share/confluent-hub-components #CONNECT_PLUGIN_PATH: C:/Users/User01/Documents/confluent/share/confluent-hub-components

where can I download this from and how?
Also do you know why I am getting this issue?

[McHardex]

@basharat58 My configuration has since changed. But however the error you're getting can be fixed.
Your classpath should reference should look something like this:

CONNECT_PLUGIN_PATH: '/usr/share/java,/usr/share/confluent-hub-components/,/data/connect-jars'

You do not have to modify that as it references the connect folder structure. Whatever jar you mount to connect will be installed in one of those directories so connect will check the three directories for any jar and use it.

To check your container folder structure:
docker-compose exec -it container-name bash

in your case, that would be:
docker-compose exec -it connect bash

you should use volumes to mount your jar. You can download them from confluent-hub. Alternatively, you can also download while starting up if you do not want to have them on your local machine. But that means you will have to download them every time you start up the containers.
I hope my new connect config can guide you:

connect:
    image: confluentinc/cp-kafka-connect:5.3.1
    hostname: connect
    container_name: connect
    depends_on:
      - broker
      - schema-registry
    ports:
      - 8083:8083
    environment:
      CONNECT_LOG4J_APPENDER_STDOUT_LAYOUT_CONVERSIONPATTERN: "[%d] %p %X{connector.context}%m (%c:%L)%n"
      CONNECT_CUB_KAFKA_TIMEOUT: 300
      CONNECT_BOOTSTRAP_SERVERS: "broker:29092"
      CONNECT_REST_ADVERTISED_HOST_NAME: 'connect'
      CONNECT_REST_PORT: 8083
      CONNECT_GROUP_ID: connect-group
      CONNECT_CONFIG_STORAGE_TOPIC: connect-group-configs
      CONNECT_OFFSET_STORAGE_TOPIC: connect-group-offsets
      CONNECT_STATUS_STORAGE_TOPIC: connect-group-status
      CONNECT_KEY_CONVERTER: io.confluent.connect.avro.AvroConverter
      CONNECT_KEY_CONVERTER_SCHEMA_REGISTRY_URL: "http://schema-registry:8081"
      CONNECT_VALUE_CONVERTER: io.confluent.connect.avro.AvroConverter
      CONNECT_VALUE_CONVERTER_SCHEMA_REGISTRY_URL: "http://schema-registry:8081"
      CONNECT_INTERNAL_KEY_CONVERTER: 'org.apache.kafka.connect.json.JsonConverter'
      CONNECT_INTERNAL_VALUE_CONVERTER: 'org.apache.kafka.connect.json.JsonConverter'
      CONNECT_LOG4J_ROOT_LOGLEVEL: 'INFO'
      CONNECT_LOG4J_LOGGERS: 'org.apache.kafka.connect.runtime.rest=WARN,org.reflections=ERROR'
      CONNECT_LOG4J_LOGGERS: 'org.apache.kafka.connect.runtime.rest=WARN,org.reflections=ERROR,org.eclipse.jetty.server=DEBUG'
      CONNECT_CONFIG_STORAGE_REPLICATION_FACTOR: '1'
      CONNECT_OFFSET_STORAGE_REPLICATION_FACTOR: '1'
      CONNECT_STATUS_STORAGE_REPLICATION_FACTOR: '1'
      CONNECT_PLUGIN_PATH: '/usr/share/java,/usr/share/confluent-hub-components/,/data/connect-jars'
      # External secrets config
      CONNECT_CONFIG_PROVIDERS: 'file'
      CONNECT_CONFIG_PROVIDERS_FILE_CLASS: 'org.apache.kafka.common.config.provider.FileConfigProvider'
    # volumes:
      # - C:\Users\User01\Downloads\connectors:/data/connect-jars
    #   - ./csv:/data/csv
    command: 
      # In the command section, $ are replaced with $$ to avoid the error 'Invalid interpolation format for "command" option'
      - bash 
      - -c 
      - |
        echo "Installing connector plugins"
        confluent-hub install --no-prompt debezium/debezium-connector-mysql:latest
        confluent-hub install --no-prompt confluentinc/kafka-connect-elasticsearch:7.5.1
        # confluent-hub install --no-prompt jcustenborder/kafka-connect-twitter:0.3.33
        confluent-hub install --no-prompt confluentinc/kafka-connect-jdbc:5.3.2
        # confluent-hub install --no-prompt confluentinc/kafka-connect-syslog:latest
        # confluent-hub install --no-prompt neo4j/kafka-connect-neo4j:1.0.2
        echo "Launching Kafka Connect worker"
        /etc/confluent/docker/run & 
        #
        sleep infinity

remember to comment out either the volume or the online jar download scripts depending on which one you want to go for.

[basharat58]

You mentioned downloading the jar from confluent-hub, how do I go about downloading the Elasticsearch jar from confluent-hub in Windows 10?

[McHardex]

Just visit confluence hub. They have all the jars you need there

On Friday, February 21, 2020, basharat58 [e-mail] wrote:

You mentioned downloading the jar from confluent-hub, how do I go about
downloading the Elasticsearch jar from confluent-hub in Windows 10?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/confluentinc/ksql/issues/4127?email_source=notifications&email_token=AH24M27MW3HHAIL5RM3PHMTRD73BNA5CNFSM4J2BC2F2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEMTFVVI#issuecomment-589716181,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AH24M22ES6GVWMDVORFWWYDRD73BNANCNFSM4J2BC2FQ
.

--
Adebisi Oluwabukunmi J.
Software Developer
Mobile: [phone]
Email: [email protected]