Ksql: Make KSQL_CONNECT-* params work within ksqldb docker images

[apurvam]

Is your feature request related to a problem? Please describe.
It is currently not possible to get KSQL_CONNECT params working (instead versus passing in a connect properties file), and ksqlDB exits with just "ksqldb-server exited with code 255" in the Docker logs

The KSQL_KSQL_CONNECT_WORKER_CONFIG config works.

Describe the solution you'd like
It would be more ergonomic configure embedded workes with the KSQL_CONNECT-* config variables in the docker image.

[apurvam]

cc @ybyzek

[agavra]

I'm pretty sure we fixed this?

      KSQL_BOOTSTRAP_SERVERS: "broker:9092"
      KSQL_HOST_NAME: ksqldb-server
      KSQL_LISTENERS: "http://0.0.0.0:8088"
      KSQL_CACHE_MAX_BYTES_BUFFERING: 0
      KSQL_KSQL_SCHEMA_REGISTRY_URL: "http://schema-registry:8081"
      KSQL_KSQL_LOGGING_PROCESSING_STREAM_AUTO_CREATE: "true"
      KSQL_KSQL_LOGGING_PROCESSING_TOPIC_AUTO_CREATE: "true"
      KSQL_LOG4J_OPTS: "-Dlog4j.configuration=file:/etc/ksql/log4j-rolling2.properties -Dlog4j.debug"
      KSQL_CONNECT_GROUP_ID: "ksql-connect-cluster"
      KSQL_CONNECT_BOOTSTRAP_SERVERS: "broker:9092"
      KSQL_CONNECT_KEY_CONVERTER: "io.confluent.connect.avro.AvroConverter"
      KSQL_CONNECT_KEY_CONVERTER_SCHEMA_REGISTRY_URL: "http://schema-registry:8081"
      KSQL_CONNECT_VALUE_CONVERTER: "io.confluent.connect.avro.AvroConverter"
      KSQL_CONNECT_VALUE_CONVERTER_SCHEMA_REGISTRY_URL: "http://schema-registry:8081"
      KSQL_CONNECT_CONFIG_STORAGE_TOPIC: "ksql-connect-configs"
      KSQL_CONNECT_OFFSET_STORAGE_TOPIC: "ksql-connect-offsets"
      KSQL_CONNECT_STATUS_STORAGE_TOPIC: "ksql-connect-statuses"
      KSQL_CONNECT_CONFIG_STORAGE_REPLICATION_FACTOR: 1
      KSQL_CONNECT_OFFSET_STORAGE_REPLICATION_FACTOR: 1
      KSQL_CONNECT_STATUS_STORAGE_REPLICATION_FACTOR: 1
      KSQL_CONNECT_PLUGIN_PATH: "/etc/ksql/connectors"

That works for me. If you're referring to the cp-ksqldb image I think you're right that work needs to be done there as well.

[agavra]

see https://github.com/confluentinc/ksql/pull/4260

[apurvam]

thanks @agavra what works needs to be done on the cp-ksqldb images?

[agavra]

port the PR above to the docker scripts in ksql-images. I can take care of that sometime this week

[agavra]

Closing this issue and tracking it internally.