Cmak: Metrics are alwawys Zero

Created on 18 Nov 2016  ·  4Comments  ·  Source: yahoo/CMAK

I failed to see the data size from brokers' metrics. I have checked several similar issues, but still can't find a solution in my case. My JMX_PORT is set 9999. Thanks for you help!

I got these two errors in the kafka-manager log.

[error] k.m.j.KafkaJMX$ - Failed to connect to service:jmx:rmi:///jndi/rmi://localhost:9999/jmxrmi
java.io.IOException: Failed to retrieve RMIServer stub: javax.naming.CommunicationException [Root exception is java.rmi.ConnectIOException: error during JRMP connection establishment; nested exception is:
    javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake]
    at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:369) ~[na:1.8.0_31]
    at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:270) ~[na:1.8.0_31]
    at kafka.manager.jmx.KafkaJMX$.doWithConnection(KafkaJMX.scala:57) ~[kafka-manager.kafka-manager-1.3.2.1-sans-externalized.jar:na]
    at kafka.manager.actor.cluster.BrokerViewCacheActor$$anonfun$kafka$manager$actor$cluster$BrokerViewCacheActor$$updateBrokerMetrics$1$$anonfun$apply$27$$anonfun$apply$3.apply$mcV$sp(BrokerViewCacheActor.scala:358) [kafka-manager.kafka-manager-1.3.2.1-sans-externalized.jar:na]
    at kafka.manager.actor.cluster.BrokerViewCacheActor$$anonfun$kafka$manager$actor$cluster$BrokerViewCacheActor$$updateBrokerMetrics$1$$anonfun$apply$27$$anonfun$apply$3.apply(BrokerViewCacheActor.scala:355) [kafka-manager.kafka-manager-1.3.2.1-sans-externalized.jar:na]
    at kafka.manager.actor.cluster.BrokerViewCacheActor$$anonfun$kafka$manager$actor$cluster$BrokerViewCacheActor$$updateBrokerMetrics$1$$anonfun$apply$27$$anonfun$apply$3.apply(BrokerViewCacheActor.scala:355) [kafka-manager.kafka-manager-1.3.2.1-sans-externalized.jar:na]
    at scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24) [org.scala-lang.scala-library-2.11.8.jar:na]
    at scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24) [org.scala-lang.scala-library-2.11.8.jar:na]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_31]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_31]
Caused by: javax.naming.CommunicationException: null
    at com.sun.jndi.rmi.registry.RegistryContext.lookup(RegistryContext.java:122) ~[na:1.8.0_31]
    at com.sun.jndi.toolkit.url.GenericURLContext.lookup(GenericURLContext.java:205) ~[na:1.8.0_31]
    at javax.naming.InitialContext.lookup(InitialContext.java:417) ~[na:1.8.0_31]
    at javax.management.remote.rmi.RMIConnector.findRMIServerJNDI(RMIConnector.java:1929) ~[na:1.8.0_31]
    at javax.management.remote.rmi.RMIConnector.findRMIServer(RMIConnector.java:1896) ~[na:1.8.0_31]
    at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:287) ~[na:1.8.0_31]
    at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:270) ~[na:1.8.0_31]
    at kafka.manager.jmx.KafkaJMX$.doWithConnection(KafkaJMX.scala:57) ~[kafka-manager.kafka-manager-1.3.2.1-sans-externalized.jar:na]
    at kafka.manager.actor.cluster.BrokerViewCacheActor$$anonfun$kafka$manager$actor$cluster$BrokerViewCacheActor$$updateBrokerMetrics$1$$anonfun$apply$27$$anonfun$apply$3.apply$mcV$sp(BrokerViewCacheActor.scala:358) [kafka-manager.kafka-manager-1.3.2.1-sans-externalized.jar:na]
    at kafka.manager.actor.cluster.BrokerViewCacheActor$$anonfun$kafka$manager$actor$cluster$BrokerViewCacheActor$$updateBrokerMetrics$1$$anonfun$apply$27$$anonfun$apply$3.apply(BrokerViewCacheActor.scala:355) [kafka-manager.kafka-manager-1.3.2.1-sans-externalized.jar:na]
Caused by: java.rmi.ConnectIOException: error during JRMP connection establishment; nested exception is:
    javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake
    at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:304) ~[na:1.8.0_31]
    at sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:202) ~[na:1.8.0_31]
    at sun.rmi.server.UnicastRef.newCall(UnicastRef.java:342) ~[na:1.8.0_31]
    at sun.rmi.registry.RegistryImpl_Stub.lookup(Unknown Source) ~[na:1.8.0_31]
    at com.sun.jndi.rmi.registry.RegistryContext.lookup(RegistryContext.java:118) ~[na:1.8.0_31]
    at com.sun.jndi.toolkit.url.GenericURLContext.lookup(GenericURLContext.java:205) ~[na:1.8.0_31]
    at javax.naming.InitialContext.lookup(InitialContext.java:417) ~[na:1.8.0_31]
    at javax.management.remote.rmi.RMIConnector.findRMIServerJNDI(RMIConnector.java:1929) ~[na:1.8.0_31]
    at javax.management.remote.rmi.RMIConnector.findRMIServer(RMIConnector.java:1896) ~[na:1.8.0_31]
    at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:287) ~[na:1.8.0_31]
Caused by: javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake
    at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:980) ~[na:1.8.0_31]
    at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1363) ~[na:1.8.0_31]
    at sun.security.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:735) ~[na:1.8.0_31]
    at sun.security.ssl.AppOutputStream.write(AppOutputStream.java:123) ~[na:1.8.0_31]
    at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82) ~[na:1.8.0_31]
    at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140) ~[na:1.8.0_31]
    at java.io.DataOutputStream.flush(DataOutputStream.java:123) ~[na:1.8.0_31]
    at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:229) ~[na:1.8.0_31]
    at sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:202) ~[na:1.8.0_31]
    at sun.rmi.server.UnicastRef.newCall(UnicastRef.java:342) ~[na:1.8.0_31]
Caused by: java.io.EOFException: SSL peer shut down incorrectly
    at sun.security.ssl.InputRecord.read(InputRecord.java:505) ~[na:1.8.0_31]
    at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:961) ~[na:1.8.0_31]
    at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1363) ~[na:1.8.0_31]
    at sun.security.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:735) ~[na:1.8.0_31]
    at sun.security.ssl.AppOutputStream.write(AppOutputStream.java:123) ~[na:1.8.0_31]
    at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82) ~[na:1.8.0_31]
    at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140) ~[na:1.8.0_31]
    at java.io.DataOutputStream.flush(DataOutputStream.java:123) ~[na:1.8.0_31]
    at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:229) ~[na:1.8.0_31]
    at sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:202) ~[na:1.8.0_31]

and

[error] k.m.a.c.BrokerViewCacheActor - Failed to get topic metrics for broker BrokerIdentity(0,localhost,9092,9999,false)
java.io.IOException: Failed to retrieve RMIServer stub: javax.naming.CommunicationException [Root exception is java.rmi.ConnectIOException: error during JRMP connection establishment; nested exception is:
    javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake]
    at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:369) ~[na:1.8.0_31]
    at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:270) ~[na:1.8.0_31]
    at kafka.manager.jmx.KafkaJMX$.doWithConnection(KafkaJMX.scala:57) ~[kafka-manager.kafka-manager-1.3.2.1-sans-externalized.jar:na]
    at kafka.manager.actor.cluster.BrokerViewCacheActor$$anonfun$kafka$manager$actor$cluster$BrokerViewCacheActor$$updateTopicMetrics$1$$anonfun$apply$24$$anonfun$apply$25$$anonfun$apply$2.apply$mcV$sp(BrokerViewCacheActor.scala:326) ~[kafka-manager.kafka-manager-1.3.2.1-sans-externalized.jar:na]
    at kafka.manager.actor.cluster.BrokerViewCacheActor$$anonfun$kafka$manager$actor$cluster$BrokerViewCacheActor$$updateTopicMetrics$1$$anonfun$apply$24$$anonfun$apply$25$$anonfun$apply$2.apply(BrokerViewCacheActor.scala:323) ~[kafka-manager.kafka-manager-1.3.2.1-sans-externalized.jar:na]
    at kafka.manager.actor.cluster.BrokerViewCacheActor$$anonfun$kafka$manager$actor$cluster$BrokerViewCacheActor$$updateTopicMetrics$1$$anonfun$apply$24$$anonfun$apply$25$$anonfun$apply$2.apply(BrokerViewCacheActor.scala:323) ~[kafka-manager.kafka-manager-1.3.2.1-sans-externalized.jar:na]
    at scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24) ~[org.scala-lang.scala-library-2.11.8.jar:na]
    at scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24) ~[org.scala-lang.scala-library-2.11.8.jar:na]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[na:1.8.0_31]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ~[na:1.8.0_31]
Caused by: javax.naming.CommunicationException: null
    at com.sun.jndi.rmi.registry.RegistryContext.lookup(RegistryContext.java:122) ~[na:1.8.0_31]
    at com.sun.jndi.toolkit.url.GenericURLContext.lookup(GenericURLContext.java:205) ~[na:1.8.0_31]
    at javax.naming.InitialContext.lookup(InitialContext.java:417) ~[na:1.8.0_31]
    at javax.management.remote.rmi.RMIConnector.findRMIServerJNDI(RMIConnector.java:1929) ~[na:1.8.0_31]
    at javax.management.remote.rmi.RMIConnector.findRMIServer(RMIConnector.java:1896) ~[na:1.8.0_31]
    at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:287) ~[na:1.8.0_31]
    at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:270) ~[na:1.8.0_31]
    at kafka.manager.jmx.KafkaJMX$.doWithConnection(KafkaJMX.scala:57) ~[kafka-manager.kafka-manager-1.3.2.1-sans-externalized.jar:na]
    at kafka.manager.actor.cluster.BrokerViewCacheActor$$anonfun$kafka$manager$actor$cluster$BrokerViewCacheActor$$updateTopicMetrics$1$$anonfun$apply$24$$anonfun$apply$25$$anonfun$apply$2.apply$mcV$sp(BrokerViewCacheActor.scala:326) ~[kafka-manager.kafka-manager-1.3.2.1-sans-externalized.jar:na]
    at kafka.manager.actor.cluster.BrokerViewCacheActor$$anonfun$kafka$manager$actor$cluster$BrokerViewCacheActor$$updateTopicMetrics$1$$anonfun$apply$24$$anonfun$apply$25$$anonfun$apply$2.apply(BrokerViewCacheActor.scala:323) ~[kafka-manager.kafka-manager-1.3.2.1-sans-externalized.jar:na]
Caused by: java.rmi.ConnectIOException: error during JRMP connection establishment; nested exception is:
    javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake
    at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:304) ~[na:1.8.0_31]
    at sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:202) ~[na:1.8.0_31]
    at sun.rmi.server.UnicastRef.newCall(UnicastRef.java:342) ~[na:1.8.0_31]
    at sun.rmi.registry.RegistryImpl_Stub.lookup(Unknown Source) ~[na:1.8.0_31]
    at com.sun.jndi.rmi.registry.RegistryContext.lookup(RegistryContext.java:118) ~[na:1.8.0_31]
    at com.sun.jndi.toolkit.url.GenericURLContext.lookup(GenericURLContext.java:205) ~[na:1.8.0_31]
    at javax.naming.InitialContext.lookup(InitialContext.java:417) ~[na:1.8.0_31]
    at javax.management.remote.rmi.RMIConnector.findRMIServerJNDI(RMIConnector.java:1929) ~[na:1.8.0_31]
    at javax.management.remote.rmi.RMIConnector.findRMIServer(RMIConnector.java:1896) ~[na:1.8.0_31]
    at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:287) ~[na:1.8.0_31]
Caused by: javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake
    at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:980) ~[na:1.8.0_31]
    at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1363) ~[na:1.8.0_31]
    at sun.security.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:735) ~[na:1.8.0_31]
    at sun.security.ssl.AppOutputStream.write(AppOutputStream.java:123) ~[na:1.8.0_31]
    at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82) ~[na:1.8.0_31]
    at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140) ~[na:1.8.0_31]
    at java.io.DataOutputStream.flush(DataOutputStream.java:123) ~[na:1.8.0_31]
    at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:229) ~[na:1.8.0_31]
    at sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:202) ~[na:1.8.0_31]
    at sun.rmi.server.UnicastRef.newCall(UnicastRef.java:342) ~[na:1.8.0_31]
Caused by: java.io.EOFException: SSL peer shut down incorrectly
    at sun.security.ssl.InputRecord.read(InputRecord.java:505) ~[na:1.8.0_31]
    at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:961) ~[na:1.8.0_31]
    at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1363) ~[na:1.8.0_31]
    at sun.security.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:735) ~[na:1.8.0_31]
    at sun.security.ssl.AppOutputStream.write(AppOutputStream.java:123) ~[na:1.8.0_31]
    at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82) ~[na:1.8.0_31]
    at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140) ~[na:1.8.0_31]
    at java.io.DataOutputStream.flush(DataOutputStream.java:123) ~[na:1.8.0_31]
    at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:229) ~[na:1.8.0_31]
    at sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:202) ~[na:1.8.0_31]
picture Angryrou

Most helpful comment

image
don't select jmx with ssl

picture neicunlideyu on 12 Jan 2017
👍5

All 4 comments

Here is what the UI shows
kafka-manager-1

Angryrou picture Angryrou on 18 Nov 2016

image
don't select jmx with ssl

neicunlideyu picture neicunlideyu on 12 Jan 2017
👍5

@neicunlideyu Thank you! It works now! Do you know why?

Angryrou picture Angryrou on 14 Jan 2017

打开JMX端口，重启broker，如果没起来，可能需要将主机名添加到hosts。再@neicunlideyu 按这个配置来改

cai574159008 picture cai574159008 on 6 Feb 2018
Was this page helpful?
0 / 5 - 0 ratings

Related issues

BDeus picture BDeus  ·  5Comments
captainbupt picture captainbupt  ·  8Comments
isharakaru picture isharakaru  ·  3Comments
jmax01 picture jmax01  ·  4Comments
rudrasharma picture rudrasharma  ·  6Comments