Quarkus: Kafka Streams Bootstrap Servers Problem

[waldimiro]

Describe the bug
Admin client can't establish the connection to bootstrap server. I'm sure that the host name is correct, when I use it in Kafka Messaging like mp.messaging.connector.smallrye-kafka.bootstrap.servers=my.host.name:9093 there are no problems.

Expected behavior
Admin client establishes the connection.

Actual behavior
Admin client can't establish the connection.

Kafka Consumer Config logs:
...
bootstrap.servers = [my.host.name/< unresolved >:9093] (Why < unresolved > ??? When I use the same server to mp.messaging, I don't have the < unresolved >)
...
In Console can I see:
...
Connection to node -1 (localhost/127.0.0.1:9093) could not be established. Broker may not be available.
...
ERROR [io.qua.kaf.str.run.KafkaStreamsProducer] (pool-6-thread-1) Failed to get topic names from broker: java.util.concurrent.TimeoutException

Configuration

quarkus.kafka-streams.bootstrap-servers=my.host.name:9093
quarkus.kafka-streams.application-id=...
quarkus.kafka-streams.topics=...

kafka-streams.security.inter.broker.protocol=SSL
kafka-streams.security.protocol=SSL
kafka-streams.ssl.truststore.location=...
kafka-streams.ssl.truststore.password=...
kafka-streams.ssl.keystore.location=...
kafka-streams.ssl.keystore.password=...
kafka-streams.ssl.key.password=...

kafka-streams.cache.max.bytes.buffering=10240
kafka-streams.commit.interval.ms=1000
kafka-streams.metadata.max.age.ms=500
kafka-streams.auto.offset.reset=earliest
kafka-streams.metrics.recording.level=DEBUG

Environment (please complete the following information):

• Output of java -version: openjdk version "14" 2020-03-17
  OpenJDK Runtime Environment (build 14+36-1461)
  OpenJDK 64-Bit Server VM (build 14+36-1461, mixed mode, sharing)

• Quarkus version or git rev: 1.7.0.Final

[gsmet]

/cc @rquinio

[quarkusbot]

/cc @gunnarmorling

[waldimiro]

I manipulated in debug modus the class io.quarkus.kafka.streams.runtime.KafkaStreamProducer by removing the "< unresolved >" and it works fine.

Please check the method:

private static String asString(List<InetSocketAddress> addresses) {
    return (String)addresses.stream().map(InetSocketAddress::toString).collect(Collectors.joining(","));
}

It is correct to use the InetSocketAddress.toString() method in this case (please think about IP ranges too)? It exists a workaround for this problem? I need KafkaStreams as soon as possible :)

[vietk]

Hello,

Are you sure you have a correct DNS setup ?
It seems that the JVM cannot resolve your DNS : my.host.name.

Now the possible reason why you are seeing a < unresolved > is that you may have upgraded to JDK 14 recently, the way InetSocketAddress.toString() works has evolved there :

Additionally, the string format for unresolved addresses has been changed. The method now represents the literal IP address with the token , for example: foo/:80 instead of foo:80. This is based on InetAddress::toString, which returns a string of the form "hostname / literal IP address". To retrieve a string representation of the hostname, or the string form of the address if it doesn't have a hostname, use InetSocketAddress::getHostString, rather than parsing the string representation.

You can check it at https://www.oracle.com/java/technologies/javase/14-relnote-issues.html#Diffs.

At least, quarkus could check if the address has been resolved and fail if it's not the case before passing it too the admin client.

It surprising that the AdminClient works when you modify the value in debug, at the end the AdminClient will resolve the address too ...

Regards

[waldimiro]

Using Java 11 solves the problem. Thanks.

[gsmet]

@waldimiro so it's something we should fix in KafkaStreamProducer right? We shouldn't use toString(), right?

[gsmet]

@waldimiro could you check if this PR fixes your issue with JDK 14: https://github.com/quarkusio/quarkus/pull/11639 ?

This doc might help https://github.com/quarkusio/quarkus/blob/master/CONTRIBUTING.md#checking-an-issue-is-fixed-in-master (just use my PR branch instead of master)