Ksql: Unable to start ksqlDB,

Created on 20 Apr 2020  路  9Comments  路  Source: confluentinc/ksql

Hello,

Could you please help, I am trying to install and start ksqlDB using docker-compose with already installed Apache Kafka version 2.11

Apache Kafka uses SASL PLAIN mechanism.

This is my docker-compose.yml file content:


version: '2'

services:
  ksqldb-server:
    image: confluentinc/ksqldb-server:0.8.1
    hostname: ksqldb-server
    container_name: ksqldb-server
    ports:
      - "8088:8088"
    environment:
      KSQL_LISTENERS: http://0.0.0.0:8088
      KSQL_BOOTSTRAP_SERVERS: "10.10.10.10:9092"
      KSQL_KSQL_LOGGING_PROCESSING_STREAM_AUTO_CREATE: "true"
      KSQL_KSQL_LOGGING_PROCESSING_TOPIC_AUTO_CREATE: "true"
      KSQL_SECURITY_PROTOCOL: "SASL_PLAINTEXT"
      KSQL_SASL_MECHANISM: "PLAIN"
      KSQL_SASL_JAAS_CONFIG: "org.apache.kafka.common.security.plain.PlainLoginModule required username=\"admin\" password=\"password\";"
  ksqldb-cli:
    image: confluentinc/ksqldb-cli:0.8.1
    container_name: ksqldb-cli
    depends_on:
      - ksqldb-server
    entrypoint: /bin/sh
    tty: true

After starting with command - docker-compose up, I am faced with message

ksqldb-server | [2020-04-20 13:51:06,507] WARN The configuration 'sasl.jaas.config' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig:355) ksqldb-server | [2020-04-20 13:51:06,507] WARN The configuration 'sasl.jaas.config' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig:355) ksqldb-server | [2020-04-20 13:51:07,450] ERROR Failed to start KSQL (io.confluent.ksql.rest.server.KsqlServerMain:63) ksqldb-server | io.confluent.ksql.util.KsqlServerException: Could not get Kafka authorized operations! ksqldb-server | at io.confluent.ksql.services.KafkaClusterUtil.isAuthorizedOperationsSupported(KafkaClusterUtil.java:54) ksqldb-server | at io.confluent.ksql.security.KsqlAuthorizationValidatorFactory.create(KsqlAuthorizationValidatorFactory.java:52) ksqldb-server | at io.confluent.ksql.rest.server.KsqlRestApplication.buildApplication(KsqlRestApplication.java:711) ksqldb-server | at io.confluent.ksql.rest.server.KsqlRestApplication.buildApplication(KsqlRestApplication.java:629) ksqldb-server | at io.confluent.ksql.rest.server.KsqlServerMain.createExecutable(KsqlServerMain.java:101) ksqldb-server | at io.confluent.ksql.rest.server.KsqlServerMain.main(KsqlServerMain.java:59) ksqldb-server | Caused by: java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.UnsupportedVersionException: Attempted to write a non-default includeClusterAuthorizedOperations at version 7 ksqldb-server | at org.apache.kafka.common.internals.KafkaFutureImpl.wrapAndThrow(KafkaFutureImpl.java:45) ksqldb-server | at org.apache.kafka.common.internals.KafkaFutureImpl.access$000(KafkaFutureImpl.java:32) ksqldb-server | at org.apache.kafka.common.internals.KafkaFutureImpl$SingleWaiter.await(KafkaFutureImpl.java:89) ksqldb-server | at org.apache.kafka.common.internals.KafkaFutureImpl.get(KafkaFutureImpl.java:260) ksqldb-server | at io.confluent.ksql.services.KafkaClusterUtil.isAuthorizedOperationsSupported(KafkaClusterUtil.java:52) ksqldb-server | ... 5 more ksqldb-server | Caused by: org.apache.kafka.common.errors.UnsupportedVersionException: Attempted to write a non-default includeClusterAuthorizedOperations at version 7

question
Source
picture shekp

Most helpful comment

@pimpelsang @tarekabouzeid @shekp can you all try running the ksqlDB server with the following config set to true/false ksql.access.validator.enable=false
That should avoid making that call to includeAuthorizedOperations

picture stevenpyzhang on 21 Jul 2020
👍2

All 9 comments

I just ran into that issue. I think it's because KSQL does DescribeClusterOptions().includeAuthorizedOperations() which seems to have been introduced in Kafka 2.3.

https://github.com/confluentinc/ksql/blob/c49b86c7b39de182a653350fdd466f76490e627f/ksqldb-engine/src/main/java/io/confluent/ksql/services/KafkaClusterUtil.java#L49
https://issues.apache.org/jira/browse/KAFKA-7922

ateijelo picture ateijelo on 22 Apr 2020
👍2

You mention you're using Kafka 2.11, I think that's the version of Scala that your brokers are using. The most recent version of Kafka as of this comment is 2.5.

ateijelo picture ateijelo on 22 Apr 2020
👍1

You mention you're using Kafka 2.11, I think that's the version of Scala that your brokers are using. The most recent version of Kafka as of this comment is 2.5.

Hello,
Thank you for help.
You are right, we use 2.1.1 Kafka version, and it seems, that we have faced with this issue https://issues.apache.org/jira/browse/KAFKA-7922

shekp picture shekp on 23 Apr 2020

Hi,
i am using Kafka version 1.1.1 and facing same error as well, i enabled raised the log level to TRACE and below is the detailed error:

[2020-05-04 18:42:40,829] TRACE [AdminClient clientId=adminclient-1] Metadata is ready to use. (org.apache.kafka.clients.admin.internals.AdminMetadataManager:159)
[2020-05-04 18:42:40,829] TRACE [AdminClient clientId=adminclient-1] Metadata is ready to use. (org.apache.kafka.clients.admin.internals.AdminMetadataManager:159)
[2020-05-04 18:42:40,829] TRACE [AdminClient clientId=adminclient-1] Found least loaded node xxxx:9092 (id: 1001 rack: null) connected with no in-flight requests (org.apache.kafka.clients.NetworkClient:677)
[2020-05-04 18:42:40,829] TRACE [AdminClient clientId=adminclient-1] Found least loaded node xxxx:9092 (id: 1001 rack: null) connected with no in-flight requests (org.apache.kafka.clients.NetworkClient:677)
[2020-05-04 18:42:40,829] TRACE [AdminClient clientId=adminclient-1] Assigned Call(callName=listNodes, deadlineMs=1588610620826) to node xxxx:9092 (id: 1001 rack: null) (org.apache.kafka.clients.admin.KafkaAdminClient:986)
[2020-05-04 18:42:40,829] TRACE [AdminClient clientId=adminclient-1] Assigned Call(callName=listNodes, deadlineMs=1588610620826) to node xxxx:9092 (id: 1001 rack: null) (org.apache.kafka.clients.admin.KafkaAdminClient:986)
[2020-05-04 18:42:40,829] DEBUG [AdminClient clientId=adminclient-1] Sending MetadataRequestData(topics=[], allowAutoTopicCreation=true, includeClusterAuthorizedOperations=true, includeTopicAuthorizedOperations=false) to xxxx:9092 (id: 1001 rack: null). correlationId=7 (org.apache.kafka.clients.admin.KafkaAdminClient:1037)
[2020-05-04 18:42:40,829] DEBUG [AdminClient clientId=adminclient-1] Sending MetadataRequestData(topics=[], allowAutoTopicCreation=true, includeClusterAuthorizedOperations=true, includeTopicAuthorizedOperations=false) to xxxx:9092 (id: 1001 rack: null). correlationId=7 (org.apache.kafka.clients.admin.KafkaAdminClient:1037)
SLF4J: Failed toString() invocation on an object of type [org.apache.kafka.common.requests.MetadataRequest]
Reported exception:
org.apache.kafka.common.errors.UnsupportedVersionException: Attempted to write a non-default includeClusterAuthorizedOperations at version 5
[2020-05-04 18:42:40,830] DEBUG [AdminClient clientId=adminclient-1] Using older server API v5 to send METADATA [FAILED toString()] with correlation id 7 to node 1001 (org.apache.kafka.clients.NetworkClient:510)
[2020-05-04 18:42:40,830] DEBUG [AdminClient clientId=adminclient-1] Using older server API v5 to send METADATA [FAILED toString()] with correlation id 7 to node 1001 (org.apache.kafka.clients.NetworkClient:510)
[2020-05-04 18:42:40,831] DEBUG [AdminClient clientId=adminclient-1] Version mismatch when attempting to send MetadataRequestData(topics=[], allowAutoTopicCreation=true, includeClusterAuthorizedOperations=true, includeTopicAuthorizedOperations=false) with correlation id 7 to 1001 (org.apache.kafka.clients.NetworkClient:488)
org.apache.kafka.common.errors.UnsupportedVersionException: Attempted to write a non-default includeClusterAuthorizedOperations at version 5
[2020-05-04 18:42:40,831] DEBUG [AdminClient clientId=adminclient-1] Version mismatch when attempting to send MetadataRequestData(topics=[], allowAutoTopicCreation=true, includeClusterAuthorizedOperations=true, includeTopicAuthorizedOperations=false) with correlation id 7 to 1001 (org.apache.kafka.clients.NetworkClient:488)
org.apache.kafka.common.errors.UnsupportedVersionException: Attempted to write a non-default includeClusterAuthorizedOperations at version 5
[2020-05-04 18:42:40,834] TRACE [AdminClient clientId=adminclient-1] Entering KafkaClient#poll(timeout=60003) (org.apache.kafka.clients.admin.KafkaAdminClient:1271)
[2020-05-04 18:42:40,834] TRACE [AdminClient clientId=adminclient-1] Entering KafkaClient#poll(timeout=60003) (org.apache.kafka.clients.admin.KafkaAdminClient:1271)
[2020-05-04 18:42:40,834] TRACE [AdminClient clientId=adminclient-1] KafkaClient#poll retrieved 1 response(s) (org.apache.kafka.clients.admin.KafkaAdminClient:1273)
[2020-05-04 18:42:40,834] TRACE [AdminClient clientId=adminclient-1] KafkaClient#poll retrieved 1 response(s) (org.apache.kafka.clients.admin.KafkaAdminClient:1273)
[2020-05-04 18:42:40,834] DEBUG [AdminClient clientId=adminclient-1] Call(callName=listNodes, deadlineMs=1588610620826) failed with non-retriable exception after 1 attempt(s) (org.apache.kafka.clients.admin.KafkaAdminClient:734)
java.lang.Exception: UnsupportedVersionException: Attempted to write a non-default includeClusterAuthorizedOperations at version 5
at org.apache.kafka.clients.admin.KafkaAdminClient$Call.fail(KafkaAdminClient.java:735)
at org.apache.kafka.clients.admin.KafkaAdminClient$AdminClientRunnable.handleResponses(KafkaAdminClient.java:1115)
at org.apache.kafka.clients.admin.KafkaAdminClient$AdminClientRunnable.processRequests(KafkaAdminClient.java:1280)
at org.apache.kafka.clients.admin.KafkaAdminClient$AdminClientRunnable.run(KafkaAdminClient.java:1203)
at java.lang.Thread.run(Thread.java:745)
[2020-05-04 18:42:40,834] DEBUG [AdminClient clientId=adminclient-1] Call(callName=listNodes, deadlineMs=1588610620826) failed with non-retriable exception after 1 attempt(s) (org.apache.kafka.clients.admin.KafkaAdminClient:734)
java.lang.Exception: UnsupportedVersionException: Attempted to write a non-default includeClusterAuthorizedOperations at version 5
at org.apache.kafka.clients.admin.KafkaAdminClient$Call.fail(KafkaAdminClient.java:735)
at org.apache.kafka.clients.admin.KafkaAdminClient$AdminClientRunnable.handleResponses(KafkaAdminClient.java:1115)
at org.apache.kafka.clients.admin.KafkaAdminClient$AdminClientRunnable.processRequests(KafkaAdminClient.java:1280)
at org.apache.kafka.clients.admin.KafkaAdminClient$AdminClientRunnable.run(KafkaAdminClient.java:1203)
at java.lang.Thread.run(Thread.java:745)
[2020-05-04 18:42:40,834] TRACE [AdminClient clientId=adminclient-1] Trying to choose nodes for [] at 1588610560834 (org.apache.kafka.clients.admin.KafkaAdminClient:961)
[2020-05-04 18:42:40,834] TRACE [AdminClient clientId=adminclient-1] Trying to choose nodes for [] at 1588610560834 (org.apache.kafka.clients.admin.KafkaAdminClient:961)
[2020-05-04 18:42:40,834] TRACE [AdminClient clientId=adminclient-1] Entering KafkaClient#poll(timeout=299942) (org.apache.kafka.clients.admin.KafkaAdminClient:1271)
[2020-05-04 18:42:40,834] TRACE [AdminClient clientId=adminclient-1] Entering KafkaClient#poll(timeout=299942) (org.apache.kafka.clients.admin.KafkaAdminClient:1271)
[2020-05-04 18:42:40,834] TRACE [AdminClient clientId=adminclient-1] KafkaClient#poll retrieved 0 response(s) (org.apache.kafka.clients.admin.KafkaAdminClient:1273)
[2020-05-04 18:42:40,834] TRACE [AdminClient clientId=adminclient-1] KafkaClient#poll retrieved 0 response(s) (org.apache.kafka.clients.admin.KafkaAdminClient:1273)
[2020-05-04 18:42:40,835] TRACE [AdminClient clientId=adminclient-1] Trying to choose nodes for [] at 1588610560835 (org.apache.kafka.clients.admin.KafkaAdminClient:961)
[2020-05-04 18:42:40,835] TRACE [AdminClient clientId=adminclient-1] Trying to choose nodes for [] at 1588610560835 (org.apache.kafka.clients.admin.KafkaAdminClient:961)
[2020-05-04 18:42:40,835] TRACE [AdminClient clientId=adminclient-1] Entering KafkaClient#poll(timeout=299941) (org.apache.kafka.clients.admin.KafkaAdminClient:1271)
[2020-05-04 18:42:40,835] ERROR Failed to start KSQL (io.confluent.ksql.rest.server.KsqlServerMain:63)
io.confluent.ksql.util.KsqlServerException: Could not get Kafka authorized operations!
at io.confluent.ksql.services.KafkaClusterUtil.isAuthorizedOperationsSupported(KafkaClusterUtil.java:51)
at io.confluent.ksql.security.KsqlAuthorizationValidatorFactory.create(KsqlAuthorizationValidatorFactory.java:52)
at io.confluent.ksql.rest.server.KsqlRestApplication.buildApplication(KsqlRestApplication.java:639)
at io.confluent.ksql.rest.server.KsqlRestApplication.buildApplication(KsqlRestApplication.java:567)
at io.confluent.ksql.rest.server.KsqlServerMain.createExecutable(KsqlServerMain.java:100)
at io.confluent.ksql.rest.server.KsqlServerMain.main(KsqlServerMain.java:59)
Caused by: java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.UnsupportedVersionException: Attempted to write a non-default includeClusterAuthorizedOperations at version 5
at org.apache.kafka.common.internals.KafkaFutureImpl.wrapAndThrow(KafkaFutureImpl.java:45)
at org.apache.kafka.common.internals.KafkaFutureImpl.access$000(KafkaFutureImpl.java:32)
at org.apache.kafka.common.internals.KafkaFutureImpl$SingleWaiter.await(KafkaFutureImpl.java:89)
at org.apache.kafka.common.internals.KafkaFutureImpl.get(KafkaFutureImpl.java:260)
at io.confluent.ksql.services.KafkaClusterUtil.isAuthorizedOperationsSupported(KafkaClusterUtil.java:49)
... 5 more
Caused by: org.apache.kafka.common.errors.UnsupportedVersionException: Attempted to write a non-default includeClusterAuthorizedOperations at version 5
[2020-05-04 18:42:40,835] TRACE [AdminClient clientId=adminclient-1] Entering KafkaClient#poll(timeout=299941) (org.apache.kafka.clients.admin.KafkaAdminClient:1271)

so is there any workaround this or the only option to run KSQLDB is having Kafka 2.3+ ?

Best Regards,

tarekabouzeid picture tarekabouzeid on 4 May 2020
👍2

Also run into issue, running Kafka brokers 1.1... Can't use ksqlDB?

pimpelsang picture pimpelsang on 18 May 2020
👍1

@pimpelsang @tarekabouzeid @shekp can you all try running the ksqlDB server with the following config set to true/false ksql.access.validator.enable=false
That should avoid making that call to includeAuthorizedOperations

stevenpyzhang picture stevenpyzhang on 21 Jul 2020
👍2

Hi @stevenpyzhang ,

This configuration successfully started ksqlDB server for me, will continue testing ksqlDB functionalities as well, thanks so much for such important input, i really suggest that this to be added to confluent official documentation in order to make ksqlDB runs against kafka versions older than 2.3.

Best Regards,

tarekabouzeid picture tarekabouzeid on 21 Jul 2020

The server isn't supposed to not start up in these cases, the behavior was fixed in https://github.com/confluentinc/ksql/pull/5617 and the fix should be in starting from ksqlDB 0.10.0

stevenpyzhang picture stevenpyzhang on 24 Aug 2020

@pimpelsang @tarekabouzeid @shekp can you all try running the ksqlDB server with the following config set to true/false ksql.access.validator.enable=false
That should avoid making that call to includeAuthorizedOperations

Just a feedback:

The confluentinc/ksqldb-server:0.11.0 shows: 

org.apache.kafka.common.config.ConfigException: Invalid value false for configuration ksql.access.validator.enable: String must be one of: on, off, auto

So, change to ksql.access.validator.enable=off works.

RafaelGSS picture RafaelGSS on 19 Oct 2020
Was this page helpful?
0 / 5 - 0 ratings

Related issues

apurvam picture apurvam  路  3Comments
apurvam picture apurvam  路  4Comments
rmoff picture rmoff  路  4Comments
Harshith2396 picture Harshith2396  路  4Comments
bhargav516 picture bhargav516  路  3Comments