On the following line, Kafka fails to start if there is a Kubernetes service called "kafka". This is because Kubernetes creates env variables for each service, one of them being KAFKA_PORT.
The startup script should not exit when KAFKA_PORT exists, if advertised listeners are otherwise valid.
@analytik do you have a workaround or suggested solution what would be backward compatible?
Would turning this fatal error into a warning be sufficient? This crashing behavior is somehow intentionally backwards incompatible as it is.
Sorry for the delay; yes, downgrading it from a fatal error to a warning would fix this particular case.
Again, please note that this only happens if there is a kubernetes service called "kafka", which I'm not sure is a best practice. We have services for individual nodes, e.g. kafka00, kafka01, kafka02, and "kafka" round-robins those three for convenience. But I assume people running single-node clusters might run into this problem, though, since "kafka" seems as reasonable name as any.
As a workaround you can unset KAFKA_PORT and execute /etc/confluent/docker/run. This works OK for me (it's a part of StatefulSet manifest):
command:
- sh
- -exc
- |
unset KAFKA_PORT && \
export KAFKA_BROKER_ID=${HOSTNAME##*-} && \
export KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://${POD_IP}:9092 && \
exec /etc/confluent/docker/run
Most helpful comment
As a workaround you can unset
KAFKA_PORTand execute/etc/confluent/docker/run. This works OK for me (it's a part ofStatefulSetmanifest):