Cmak: support for 0.10.0.0?

Created on 31 May 2016 · 31Comments · Source: yahoo/CMAK

Source

hengheng0haha

👍64

Most helpful comment

I would guess it shouldn't be too difficult. Just need to create shaded jars for kafka 0.9.0.1 and 0.10.0.0 so both are supported. Will work on it this month.

patelh on 11 Jul 2016

👍39 🎉12 ❤10

All 31 comments

Getting continuously WARNINGS like this when connecting to Kafka 0.10.0

[warn] k.c.ConsumerFetcherManager$LeaderFinderThread - [KafkaManagerOffsetCache_04160b48ba5b-1464964908716-d110efc9-leader-finder-thread], Failed to find leader for Set([__consumer_offsets,16], [__consumer_offsets,2], [__consumer_offsets,39], [__consumer_offsets,15], [__consumer_offsets,33], [__consumer_offsets,38], [__consumer_offsets,9], [__consumer_offsets,7], [__consumer_offsets,31], [__consumer_offsets,48], [__consumer_offsets,24], [__consumer_offsets,19], [__consumer_offsets,11], [__consumer_offsets,22], [__consumer_offsets,26], [__consumer_offsets,17], [__consumer_offsets,10], [__consumer_offsets,35], [__consumer_offsets,36], [__consumer_offsets,34], [__consumer_offsets,49], [__consumer_offsets,44], [__consumer_offsets,23], [__consumer_offsets,40], [__consumer_offsets,47], [__consumer_offsets,45], [__consumer_offsets,43], [__consumer_offsets,46], [__consumer_offsets,37], [__consumer_offsets,28], [__consumer_offsets,27], [__consumer_offsets,25], [__consumer_offsets,6], [__consumer_offsets,5], [__consumer_offsets,3], [__consumer_offsets,18], [__consumer_offsets,14], [__consumer_offsets,41], [__consumer_offsets,42], [__consumer_offsets,4], [__consumer_offsets,12], [__consumer_offsets,21], [__consumer_offsets,30], [__consumer_offsets,1], [__consumer_offsets,32], [__consumer_offsets,13], [__consumer_offsets,8], [__consumer_offsets,20], [__consumer_offsets,0], [__consumer_offsets,29])
kafka.common.KafkaException: fetching topic metadata for topics [Set(__consumer_offsets)] from broker [ArrayBuffer(BrokerEndPoint(0,localhost,9092))] failed
        at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:73) ~[org.apache.kafka.kafka_2.11-0.9.0.1.jar:na]
        at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:94) ~[org.apache.kafka.kafka_2.11-0.9.0.1.jar:na]
        at kafka.consumer.ConsumerFetcherManager$LeaderFinderThread.doWork(ConsumerFetcherManager.scala:66) ~[org.apache.kafka.kafka_2.11-0.9.0.1.jar:na]
        at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:63) [org.apache.kafka.kafka_2.11-0.9.0.1.jar:na]
Caused by: java.nio.channels.ClosedChannelException: null
        at kafka.network.BlockingChannel.send(BlockingChannel.scala:110) ~[org.apache.kafka.kafka_2.11-0.9.0.1.jar:na]
        at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:75) ~[org.apache.kafka.kafka_2.11-0.9.0.1.jar:na]
        at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:74) ~[org.apache.kafka.kafka_2.11-0.9.0.1.jar:na]
        at kafka.producer.SyncProducer.send(SyncProducer.scala:119) ~[org.apache.kafka.kafka_2.11-0.9.0.1.jar:na]
        at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:59) ~[org.apache.kafka.kafka_2.11-0.9.0.1.jar:na]
        ... 3 common frames omitted
[warn] k.c.ClientUtils$ - Fetching topic metadata with correlation id 247 for topics [Set(__consumer_offsets)] from broker [BrokerEndPoint(0,localhost,9092)] failed
java.nio.channels.ClosedChannelException: null
        at kafka.network.BlockingChannel.send(BlockingChannel.scala:110) ~[org.apache.kafka.kafka_2.11-0.9.0.1.jar:na]
        at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:75) ~[org.apache.kafka.kafka_2.11-0.9.0.1.jar:na]
        at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:74) ~[org.apache.kafka.kafka_2.11-0.9.0.1.jar:na]
        at kafka.producer.SyncProducer.send(SyncProducer.scala:119) ~[org.apache.kafka.kafka_2.11-0.9.0.1.jar:na]
        at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:59) [org.apache.kafka.kafka_2.11-0.9.0.1.jar:na]
        at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:94) [org.apache.kafka.kafka_2.11-0.9.0.1.jar:na]
        at kafka.consumer.ConsumerFetcherManager$LeaderFinderThread.doWork(ConsumerFetcherManager.scala:66) [org.apache.kafka.kafka_2.11-0.9.0.1.jar:na]
        at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:63) [org.apache.kafka.kafka_2.11-0.9.0.1.jar:na]

weand on 3 Jun 2016

any update with such issue ? as such new kafka version has been released. by they way, is there any plan to have more metric option to monitor each topic data size (accumulated).

bwzhang2011 on 6 Jun 2016

Consumers consuming from this topic
is null.

for 0.10.0.0 with jmx

twtiyb on 8 Jun 2016

I recently upgraded the kafka cluster to 0.10. But in Kafka-Manager it gives me a option to select 0.8.2.X as the kafka version. How do you access a kafka 0.10 cluster?

sambit19 on 10 Jun 2016

kafka manager docs say that it supports 0.8.x and 0.9 of kafka. So 0.10 won't work.

sambit19 on 10 Jun 2016

When do you plan to add 0.10 support?

hakanilter on 17 Jun 2016

👍25

Forecast for 0.10 support ? Thanks

RodrigoLimasss on 20 Jun 2016

👍17

zhjt80 on 2 Jul 2016

👎2

KulikovaLidia on 7 Jul 2016

👎1

jantoniucci on 11 Jul 2016

I would guess it shouldn't be too difficult. Just need to create shaded jars for kafka 0.9.0.1 and 0.10.0.0 so both are supported. Will work on it this month.

patelh on 11 Jul 2016

👍39 🎉12 ❤10

Do you have any updates for this issue?

hakanilter on 15 Aug 2016

seua on 20 Aug 2016

Newest version is 0.10.0.1 . It would be great to support that as well :)

nwolff on 23 Aug 2016

👍2

@nwolff - PR #282 includes support for 0.10.0.1 (though I've only tested on 0.10.0.0).

qfissler on 25 Aug 2016

👍3

@qfissler Just tested your fork on kafka 0.10.0.1 and it works well. Thanks a lot.

nwolff on 25 Aug 2016

Does anyway know if I can see the client producers instance(s) within a cluster and for a given topic. jmx metrics exists like kafka.server-->Produce --> byte-rate