Charts: kafka-3.0.13 SSL handshake errors

[MrPink]

Which chart:

kafka-3.0.13

Description
Authentication fails with SSL errors when auth.enable=true is set

Steps to reproduce the issue:

helm install -n kafka --set auth.enabled=true --set auth.certificatesSecret=kafka-certificates --set auth.certificatesPassword=<cert password> --set sslEndpointIdentificationAlgorithm= bitnami/kafka

Describe the results you received:

[2019-07-09 08:43:28,709] INFO [SocketServer brokerId=1001] Failed authentication with kafka-0.kafka-headless.default.svc.cluster.local/10.1.0.124 (SSL handshake failed) (org.apache.kafka.common.network.Selector)

Describe the results you expected:

Successful authentication

Additional information you deem important (e.g. issue happens only occasionally):

I created the certificates with kafka-generate-ssl.sh from https://github.com/confluentinc/confluent-platform-security-tools/blob/master/kafka-generate-ssl.sh

I've tried setting the sslEndpointIdentificationAlgorithm to '' and HTTPS with the same result

kubectl describe pod kafka-0

Name:               kafka-0
Namespace:          default
Priority:           0
PriorityClassName:  <none>
Node:               docker-desktop/192.168.65.3
Start Time:         Tue, 09 Jul 2019 10:34:38 +0200
Labels:             app.kubernetes.io/component=kafka
                    app.kubernetes.io/instance=kafka
                    app.kubernetes.io/managed-by=Tiller
                    app.kubernetes.io/name=kafka
                    controller-revision-hash=kafka-67fd95575
                    helm.sh/chart=kafka-3.0.13
                    statefulset.kubernetes.io/pod-name=kafka-0
Annotations:        <none>
Status:             Running
IP:                 10.1.0.124
Controlled By:      StatefulSet/kafka
Containers:
  kafka:
    Container ID:   docker://36d2fe88032feeac4e65216a6df7d13db6453571d76df6160bede83d244b790b
    Image:          docker.io/bitnami/kafka:2.3.0-debian-9-r4
    Image ID:       docker-pullable://bitnami/kafka@sha256:7c377418272ae8d6c33b4cc1d892cdc22bb364dd0a0d29c1337155fea935c859
    Port:           9092/TCP
    Host Port:      0/TCP
    State:          Running
      Started:      Tue, 09 Jul 2019 10:34:40 +0200
    Ready:          True
    Restart Count:  0
    Liveness:       tcp-socket :kafka delay=10s timeout=5s period=10s #success=1 #failure=2
    Readiness:      tcp-socket :kafka delay=5s timeout=5s period=10s #success=1 #failure=6
    Environment:
      MY_POD_IP:                                            (v1:status.podIP)
      MY_POD_NAME:                                         kafka-0 (v1:metadata.name)
      KAFKA_CFG_ZOOKEEPER_CONNECT:                         kafka-zookeeper
      KAFKA_PORT_NUMBER:                                   9092
      KAFKA_CFG_LISTENERS:                                 SASL_SSL://:$(KAFKA_PORT_NUMBER)
      KAFKA_CFG_ADVERTISED_LISTENERS:                      SASL_SSL://$(MY_POD_NAME).kafka-headless.default.svc.cluster.local:$(KAFKA_PORT_NUMBER)
      KAFKA_OPTS:                                          -Djava.security.auth.login.config=/opt/bitnami/kafka/conf/kafka_jaas.conf
      KAFKA_BROKER_USER:                                   user
      KAFKA_BROKER_PASSWORD:                               <set to the key 'kafka-broker-password' in secret 'kafka'>  Optional: false
      KAFKA_INTER_BROKER_USER:                             admin
      KAFKA_INTER_BROKER_PASSWORD:                         <set to the key 'kafka-inter-broker-password' in secret 'kafka'>  Optional: false
      KAFKA_CERTIFICATE_PASSWORD:                         <password>
      ALLOW_PLAINTEXT_LISTENER:                            no
      KAFKA_CFG_BROKER_ID:                                 -1
      KAFKA_CFG_DELETE_TOPIC_ENABLE:                       false
      KAFKA_HEAP_OPTS:                                     -Xmx1024m -Xms1024m
      KAFKA_CFG_LOG_FLUSH_INTERVAL_MESSAGES:               10000
      KAFKA_CFG_LOG_FLUSH_INTERVAL_MS:                     1000
      KAFKA_CFG_LOG_RETENTION_BYTES:                       1073741824
      KAFKA_CFG_LOG_RETENTION_CHECK_INTERVALS_MS:          300000
      KAFKA_CFG_LOG_RETENTION_HOURS:                       168
      KAFKA_CFG_LOG_MESSAGE_FORMAT_VERSION:                
      KAFKA_CFG_MAX_MESSAGE_BYTES:                         1000012
      KAFKA_CFG_SEGMENT_BYTES:                             1073741824
      KAFKA_CFG_LOGS_DIRS:                                 /opt/bitnami/kafka/data
      KAFKA_CFG_DEFAULT_REPLICATION_FACTOR:                1
      KAFKA_CFG_OFFSETS_TOPIC_REPLICATION_FACTOR:          1
      KAFKA_CFG_TRANSACTION_STATE_LOG_REPLICATION_FACTOR:  1
      KAFKA_CFG_SSL_ENDPOINT_IDENTIFICATION_ALGORITHM:     
      KAFKA_CFG_TRANSACTION_STATE_LOG_MIN_ISR:             1
      KAFKA_CFG_NUM_IO_THREADS:                            8
      KAFKA_CFG_NUM_NETWORK_THREADS:                       3
      KAFKA_CFG_NUM_PARTITIONS:                            1
      KAFKA_CFG_NUM_RECOVERY_THREADS_PER_DATA_DIR:         1
      KAFKA_CFG_SOCKET_RECEIVE_BUFFER_BYTES:               102400
      KAFKA_CFG_SOCKET_REQUEST_MAX_BYTES:                  104857600
      KAFKA_CFG_SOCKET_SEND_BUFFER_BYTES:                  102400
      KAFKA_CFG_ZOOKEEPER_CONNECT_TIMEOUT_MS:              6000
    Mounts:
      /bitnami/kafka from data (rw)
      /opt/bitnami/kafka/conf/certs/ from kafka-certificates (ro)
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-bhvhx (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             True 
  ContainersReady   True 
  PodScheduled      True 
Volumes:
  data:
    Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  data-kafka-0
    ReadOnly:   false
  kafka-certificates:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  kafka-certificates
    Optional:    false
  default-token-bhvhx:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-bhvhx
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type    Reason     Age   From                     Message
  ----    ------     ----  ----                     -------
  Normal  Scheduled  18m   default-scheduler        Successfully assigned default/kafka-0 to docker-desktop
  Normal  Pulled     18m   kubelet, docker-desktop  Container image "docker.io/bitnami/kafka:2.3.0-debian-9-r4" already present on machine
  Normal  Created    18m   kubelet, docker-desktop  Created container kafka
  Normal  Started    18m   kubelet, docker-desktop  Started container kafka

Version of Helm and Kubernetes:

• Output of helm version:

Client: &version.Version{SemVer:"v2.12.3", GitCommit:"eecf22f77df5f65c823aacd2dbd30ae6c65f186e", GitTreeState:"clean"}
Server: &version.Version{SemVer:"v2.12.3", GitCommit:"eecf22f77df5f65c823aacd2dbd30ae6c65f186e", GitTreeState:"clean"}

• Output of kubectl version:

Client Version: version.Info{Major:"1", Minor:"14", GitVersion:"v1.14.1", GitCommit:"b7394102d6ef778017f2ca4046abbaa23b88c290", GitTreeState:"clean", BuildDate:"2019-04-08T17:11:31Z", GoVersion:"go1.12.1", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"14", GitVersion:"v1.14.3", GitCommit:"5e53fd6bc17c0dec8434817e69b04a25d8ae0ff0", GitTreeState:"clean", BuildDate:"2019-06-06T01:36:19Z", GoVersion:"go1.12.5", Compiler:"gc", Platform:"linux/amd64"}

I managed to get this working locally with docker-compose but no such luck with Kubernetes

any assistance or guidance appreciated

Thanks,
Will

[marcosbc]

Hi @MrPink, we recommend you to check the Security section in the container repository. Especially for the following notes:

• When prompted to enter a password, use the same one for all.
• Set the Common Name or FQDN values to your Kafka container hostname, e.g. kafka.example.com. After entering this value, when prompted "What is your first and last name?", enter this value as well.

In addition to that, there is an example docker-compose.yml which we were able to successfully configure. The main change in comparison with other examples is the addition of the hostname property:

kafka:
  (...)
  hostname: kafka.example.com

Once you have generated the certificates keeping those three things in mind, you should be able to get it working.

[MrPink]

Hi @marcosbc

Thanks for your prompt reply

I did read the Security section before, the problem I actually have is when I set --set replicaCount=3 I then have multiple hostnames.

I've adjusted the keytool command in kafka-generate-ssl.sh to do

keytool -keystore $KEYSTORE_WORKING_DIRECTORY/$KEYSTORE_FILENAME \
    -alias  default.svc.cluster.local -validity $VALIDITY_IN_DAYS -genkey -keyalg RSA -ext SAN=dns:kafka-0.kafka-headless.default.svc.cluster.local,dns:kafka-1.kafka-headless.default.svc.cluster.local,dns:kafka-2.kafka-headless.default.svc.cluster.local,dns:kafka-zookeeper-0.kafka-headless.default.svc.cluster.local,dns:kafka-zookeeper-1.kafka-headless.default.svc.cluster.local,dns:kafka-zookeeper-2.kafka-headless.default.svc.cluster.local

Unfortunately this does not seem to work

Thanks,
Will

[anoopl]

I have the same issue. I have created the JKS files with SAN and added -ext SAN=DNS:kafka-0.kafka-headless.kafka.svc.cluster.local,DNS:kafka-1.kafka-headless.kafka.svc.cluster.local,DNS:kafka-2.kafka-headless.kafka.svc.cluster.local

I get this error: [2019-07-11 08:21:16,368] INFO [SocketServer brokerId=1008] Failed authentication with kafka-1.kafka-headless.kafka.svc.cluster.local/10.4.2.76 (SSL handshake failed) (org.apache.kafka.common.network.Selector).

I can see this SAN set on the keystore like below:

SubjectAlternativeName [
  DNSName: kafka-0.kafka-headless.kafka.svc.cluster.local
  DNSName: kafka-1.kafka-headless.kafka.svc.cluster.local
  DNSName: kafka-2.kafka-headless.kafka.svc.cluster.local
]

[marcosbc]

We will work on reproducing this, but in the meantime keep in mind there are various calls to keytool, as the hostname needs to be specified a couple of times during the script. We will get back to you once we have any update.

[marcosbc]

I have been looking into this and unfortunately it seems not to work.

The main problem is that the certificate file and keystore would not show the Subject Alternative Name. We were able to fix this by following these steps:

• Remove any -ext parameter to Keytool and apply this change to the script:

  --- a/kafka-generate-ssl.sh 2019-06-05 12:22:41.778240058 +0200
  +++ b/kafka-generate-ssl.sh 2019-07-12 16:38:41.564707336 +0200
  @@ -169,7 +169,8 @@
  echo "You will be prompted for the trust store's private key password."
  openssl x509 -req -CA $CA_CERT_FILE -CAkey $trust_store_private_key_file \
  -in $KEYSTORE_SIGN_REQUEST -out $KEYSTORE_SIGNED_CERT \

  • -days $VALIDITY_IN_DAYS -CAcreateserial
  • -days $VALIDITY_IN_DAYS -CAcreateserial \

  • -extfile openssl-req.conf -extensions v3_req

    creates $KEYSTORE_SIGN_REQUEST_SRL which is never used or needed.

    echo

• Create openssl-req.conf file containing the following content:

  [req]
  distinguished_name = req_distinguished_name
  req_extensions = v3_req
  prompt = no

  [v3_req]
  keyUsage = keyEncipherment, dataEncipherment
  extendedKeyUsage = serverAuth
  subjectAltName = @alt_names
  [alt_names]
  DNS.1 = kafka-0.kafka-headless.default.svc.cluster.local
  DNS.2 = kafka-1.kafka-headless.default.svc.cluster.local
  DNS.3 = kafka-2.kafka-headless.default.svc.cluster.local

• Ensure the SAN was added to kafka.keystore.jks:

  $ keytool -list -v -keystore keystore/kafka.keystore.jks
  (...)
  #3: ObjectId: 2.5.29.17 Criticality=false
  SubjectAlternativeName [
  DNSName: kafka-0.kafka-headless.default.svc.cluster.local
  DNSName: kafka-1.kafka-headless.default.svc.cluster.local
  DNSName: kafka-2.kafka-headless.default.svc.cluster.local
  ]

However, even with the proper SAN set in the keystore, Kafka would fail to start:

[2019-07-12 14:37:08,031] ERROR [KafkaServer id=1001] Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
org.apache.kafka.common.KafkaException: org.apache.kafka.common.config.ConfigException: Invalid value javax.net.ssl.SSLHandshakeException: General SSLEngine problem for configuration A client SSLEngine created with the provided settings can't connect to a server SSLEngine created with those settings.
    at org.apache.kafka.common.network.SaslChannelBuilder.configure(SaslChannelBuilder.java:160)
    at org.apache.kafka.common.network.ChannelBuilders.create(ChannelBuilders.java:146)
(...)

I checked on the official documentation for Confluent (which is a distribution on Kafka) and they mention that each broker should have their own keystore. See the Brokers section. If that is the case the chart would require changes in order to support that feature.

So I have the feeling Kafka does not support SAN for certificates. Unfortunately I couldn't investigate more on this, but if you do and find anything, please share.

[anoopl]

I was able to get this working with the following script attached and ssl.conf file.
I have added -ext SAN to the keystore generation:
```keytool -keystore $KEYSTORE_WORKING_DIRECTORY/$KEYSTORE_FILENAME \
-alias localhost -validity $VALIDITY_IN_DAYS -genkey -keyalg RSA \
-ext SAN=DNS:kafka-0.kafka-headless.kafka.svc.cluster.local,DNS:kafka-1.kafka-headless.kafka.svc.cluster.local,DNS:kafka-2.kafka-headless.kafka.svc.cluster.local

Then:

openssl x509 -req -CA $CA_CERT_FILE -CAkey $trust_store_private_key_file \
-in $KEYSTORE_SIGN_REQUEST -out $KEYSTORE_SIGNED_CERT \
-days $VALIDITY_IN_DAYS -CAcreateserial -extfile ssl.cnf -extensions req_ext

[Archive.zip](https://github.com/bitnami/charts/files/3389871/Archive.zip)

I can send a PR if required. But I guess that script is not for Kubernetes.

I still have issue with kafka-exporter pod:

Error Init Kafka Client
panic: kafka: client has run out of available brokers to talk to (Is your cluster reachable?)

goroutine 1 [running]:
main.NewExporter(0xc420045b50, 0x1, 0x1, 0x100, 0x8e9778, 0x0, 0x8e9778, 0x0, 0x0, 0x8e9778, ...)
/home/travis/gopath/src/github.com/danielqsj/kafka_exporter/kafka_exporter.go:205 +0x92d
main.main()
/home/travis/gopath/src/github.com/danielqsj/kafka_exporter/kafka_exporter.go:439 +0x1d6c
```
I am not sure how to setup the SSL cert for this client
@marcosbc @MRPink

[marcosbc]

Thanks @anoopl! We were also able to get it to work with your changes:

--- a/kafka-generate-ssl.sh 2019-06-05 12:22:41.778240058 +0200
+++ b/kafka-generate-ssl.sh 2019-07-15 17:59:59.987428608 +0200
@@ -142,7 +142,8 @@
 # https://docs.oracle.com/javase/7/docs/api/javax/net/ssl/X509ExtendedTrustManager.html

 keytool -keystore $KEYSTORE_WORKING_DIRECTORY/$KEYSTORE_FILENAME \
-  -alias localhost -validity $VALIDITY_IN_DAYS -genkey -keyalg RSA
+  -alias localhost -validity $VALIDITY_IN_DAYS -genkey -keyalg RSA \
+  -ext SAN=DNS:kafka-0.kafka-headless.kafka.svc.cluster.local,DNS:kafka-1.kafka-headless.kafka.svc.cluster.local,DNS:kafka-2.kafka-headless.kafka.svc.cluster.local

 echo
 echo "'$KEYSTORE_WORKING_DIRECTORY/$KEYSTORE_FILENAME' now contains a key pair and a"
@@ -169,7 +170,8 @@
 echo "You will be prompted for the trust store's private key password."
 openssl x509 -req -CA $CA_CERT_FILE -CAkey $trust_store_private_key_file \
   -in $KEYSTORE_SIGN_REQUEST -out $KEYSTORE_SIGNED_CERT \
-  -days $VALIDITY_IN_DAYS -CAcreateserial
+  -days $VALIDITY_IN_DAYS -CAcreateserial \
+  -extfile ssl.cnf -extensions req_ext
 # creates $KEYSTORE_SIGN_REQUEST_SRL which is never used or needed.

 echo

And the following OpenSSL configuration file ssl.cnf:

[ req ]
#default_bits       = 2048
#default_md     = sha256
#default_keyfile    = privkey.pem
distinguished_name  = req_distinguished_name
attributes      = req_attributes
req_extensions      = req_ext
[ req_distinguished_name ]
countryName         = Country Name (2 letter code)
countryName_min         = 2
countryName_max         = 2
stateOrProvinceName     = State or Province Name (full name)
localityName            = Locality Name (eg, city)
0.organizationName      = Organization Name (eg, company)
organizationalUnitName      = Organizational Unit Name (eg, section)
commonName          = Common Name (eg, fully qualified host name)
commonName_max          = 64
emailAddress            = Email Address
emailAddress_max        = 64

[ req_attributes ]
challengePassword       = A challenge password
challengePassword_min       = 4
challengePassword_max       = 20
[ req_ext ]
subjectAltName = @alt_names
[ alt_names ]
DNS.1 = kafka-0.kafka-headless.default.svc.cluster.local
DNS.2 = kafka-1.kafka-headless.default.svc.cluster.local
DNS.3 = kafka-2.kafka-headless.default.svc.cluster.local

I've created an internal task for finding the best way to support this feature without needing to patch the thirdparty kafka-generate-ssl.sh script (i.e. by contributing the changes), our main concern is the -extfile change may not be accepted so it might need to be workaround.

As for your issue with kafka-exporter, currently it's a thirdparty container (danielqsj/kafka-exporter) so it is separate from this issue. However, it seems like it has already been addressed here, please check it: https://github.com/danielqsj/kafka_exporter/issues/51.

[anoopl]

Thanks @marcosbc for the reply.
I will try the kafka-exporter work-around

[MrPink]

Many thanks @anoopl that's working for me 👍

[anoopl]

@marcosbc @MrPink
How can we create certificates for the client using this script?

[juan131]

Hi @anoopl

I'm afraid we didn't have time to work on the internal task @marcosbc created to find the best way to support this feature without needing to patch the third-party kafka-generate-ssl.sh script.

We'll update this issue once we have more time to investigate it so we can provide a generic solution to support SSL. Thanks for you consideration and patience.

[anoopl]

@juan131 Thanks for the reply, I have fixed the client certs creation issue as well.
I can add a gist with all these commands if needed.
Another thing I noticed is ServicePort is hardcoded as 9092 on the templates svc.yaml. But with SSL it should be 9093. Do you accept PR from outside?

[juan131]

Yes of course! @anoopl
Feel free to create a PR and I'd be glad to review it!

Regarding the client certs creation, could you share the commands please?

[anoopl]

@juan131 I have created a Gist for the command I used for Server & Client cert creation.

https://gist.github.com/anoopl/85d869f7a85a70c6c60542922fc314a8

Regarding the Hardcoded port. I was mentioning this line:
https://github.com/bitnami/charts/blob/master/bitnami/kafka/templates/svc.yaml#L24

After proper tests and deploy it again. I can send the PR

[juan131]

Thanks so much for sharing the Gist!! Regarding the hardcoded port, I just fixed in this PR: https://github.com/bitnami/charts/pull/1337

[anoopl]

@juan131 Great that you fixed it, please change here as well:
https://github.com/bitnami/charts/blob/master/bitnami/kafka/templates/svc-headless.yaml#L16
I have created a PR: https://github.com/bitnami/charts/pull/1339

[juan131]

Your PR have been approved and merged @anoopl ! Thanks for it!

[anoopl]

Thanks @juan131

[polinchw]

I'm having the same problem.

[2019-09-17 19:16:15,221] INFO [Controller id=1025, targetBrokerId=1027] Failed authentication with kafka-ssl-1.kafka-ssl-headless.kafka-ssl.svc.cluster.local/10.233.65.61 (SSL handshake failed) (org.apache.kafka.common.network.Selector) [2019-09-17 19:16:15,221] ERROR [Controller id=1025, targetBrokerId=1027] Connection to node 1027 (kafka-ssl-1.kafka-ssl-headless.kafka-ssl.svc.cluster.local/10.233.65.61:9092) failed authentication due to: SSL handshake failed (org.apache.kafka.clients.NetworkClient)

[juan131]

Hi @polinchw

Did you follow the instructions @anoopl shared? You can find some useful scripts in the Gist below:

https://gist.github.com/anoopl/85d869f7a85a70c6c60542922fc314a8

[polinchw]

Yes I tried that but I got an PKIX error on my client side.

[polinchw]

I set my kafka advertised listener to my public dns and I'm getting through ssl and auth now, but I'm getting a different error...

2019-09-18 13:43:39.746  INFO 13359 --- [           main] o.a.k.c.s.authenticator.AbstractLogin    : Successfully logged in.
2019-09-18 13:43:39.928  WARN 13359 --- [           main] o.a.k.clients.consumer.ConsumerConfig    : The configuration 'spring.json.trusted.packages' was supplied but isn't a known config.
2019-09-18 13:43:39.931  INFO 13359 --- [           main] o.a.kafka.common.utils.AppInfoParser     : Kafka version : 2.0.1
2019-09-18 13:43:39.931  INFO 13359 --- [           main] o.a.kafka.common.utils.AppInfoParser     : Kafka commitId : fa14705e51bd2ce5
2019-09-18 13:43:40.403  INFO 13359 --- [           main] org.apache.kafka.clients.Metadata        : Cluster ID: RbjU1cZIQdGkAyT4tX48_g
2019-09-18 13:43:40.462  INFO 13359 --- [           main] o.a.k.clients.consumer.ConsumerConfig    : ConsumerConfig values: 



2019-09-18 13:31:25.337 ERROR 12191 --- [ad | producer-1] o.s.k.support.LoggingProducerListener    : Exception thrown when sending a message with key='null' and payload='A simple test message' to topic mytopic:

org.apache.kafka.common.errors.TimeoutException: Expiring 3 record(s) for mytopic-0: 30095 ms has passed since batch creation plus linger time

[juan131]

I set my kafka advertised listener to my public dns and I'm getting through ssl and auth now

That's great! Glad you were able to solve the SSL

Regarding the second error, I'm not an expert in Kafka but it looks like it's a connectivity issues. You might get more information to debug this issue from actual Kafka users. For instance, they were discussing something similar in the Stackoverflow question below:

https://stackoverflow.com/questions/48227857/spring-boot-kafka-producer-thrown-exception-with-key-null

[polinchw]

@juan131 Thanks I'll take a look at that stackoverflow URL.

[anoopl]

@juan131 I had to configure Container Port, Service Port and Listeners on Port 9093 to SSL working. I created a PR: https://github.com/bitnami/charts/pull/1540

I have tested this by deploying and connecting to Kafka using client called Kafkatool and created a Topic successfully!

[juan131]

Hi @anoopl

Thanks so much for the PR! I just approved and merged it.

[sbavi2]

Hi @anoopl,

Were you able to get pass the kafka-exporters issue? I was following this thread from beginning and was able to use your script summerizing server and client jks files. This resolved the Kafka interbroker communication however, im still not able to resolve the kafka-exporter issue

[anoopl]

@sbavi2 I could not solve the exporter issue. As we need to maintain a new docker image with option to pass the TLS certs as k8s secret or something. This work is pending

[cedricve]

Still having the exporter issue. Works when auth is disabled, which is not suitable for my usecase.

[juan131]

Hi @cedricve @anoopl @sbavi2

Sorry we lost the track of your comments.

The exporter is no longer a third-party image, we're using this image: https://github.com/bitnami/bitnami-docker-kafka-exporter. Therefore, we should come up with a solution to ensure you can use it even if SSL or SASAL authentication is enabled for external connections in the Kafka brokers.

I just created an internal task to work on this. We'll keep you updated about our progress.

[juan131]

Hi everyone,

I'd like to share with you that we introduced today some major changes in the latest revision of the Kafka Docker image, and the latest version of the Kafka Helm chart.

These changes, introduced in the chart version 11.0.0, refactor how listeners and authentication are configured on Kafka, allowing you to have different listeners for different purposes, and configure different authentication protocols on them.

The supported listeners will be:

• INTERNAL: used for inter-broker communications.
• CLIENT: used for communications with clients that are within the same network as Kafka brokers.
• EXTERNAL: used for communications with clients that are on a different network.

To use a more complex configuration, you can mount your own server.properties and kafka_jaas.conf configuration files.

Due to these changes, you'll have to adapt your values.yaml to continue using the chart.

Find more information in the link below:

• https://github.com/bitnami/charts/tree/master/bitnami/kafka#enable-security-for-kafka-and-zookeeper
• https://github.com/bitnami/charts/tree/master/bitnami/kafka#to-1100
• https://github.com/bitnami/bitnami-docker-kafka#250-debian-10-r51