Cmak: Login module not specified in JAAS config - kafka-manager-1.3.3.21

[cmcbugg]

Hi

I am running Kafka with SASL_PLAINTEXT. I have configed Kafka-manager to use this and provided a jaas file with the correct options (well options that work for all other consumers) - but when starting kafka-manager I get the error "Login module not specified in JAAS config"

The consumers & offsets are not visible in the UI.

version kafka-manager-1.3.3.21

jaas file:

KafkaClient {
   com.sun.security.auth.module.Krb5LoginModule required
   useKeyTab=true
   keyTab="/etc/security/keytabs/kafka.service.keytab"
   storeKey=true
   useTicketCache=false
   serviceName="kafka"
   principal="XXXXXXX";
};

2018-10-29 22:20:20,650 - [ERROR] - from kafka.manager.actor.cluster.KafkaStateActor in kafka-manager-system-akka.actor.default-dispatcher-2 
KafkaTopicOffsetGetter exception 
org.apache.kafka.common.KafkaException: Failed to construct kafka consumer
    at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:793) ~[org.apache.kafka.kafka-clients-1.1.0.jar:na]
    at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:644) ~[org.apache.kafka.kafka-clients-1.1.0.jar:na]
    at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:624) ~[org.apache.kafka.kafka-clients-1.1.0.jar:na]
    at kafka.manager.actor.cluster.KafkaStateActor$KafkaTopicOffsetGetter$$anonfun$run$7$$anonfun$apply$10$$anonfun$apply$mcV$sp$14.apply(KafkaStateActor.scala:1490) ~[kafka-manager.kafka-manager-1.3.3.21-sans-externalized.jar:na]
    at kafka.manager.actor.cluster.KafkaStateActor$KafkaTopicOffsetGetter$$anonfun$run$7$$anonfun$apply$10$$anonfun$apply$mcV$sp$14.apply(KafkaStateActor.scala:1468) ~[kafka-manager.kafka-manager-1.3.3.21-sans-externalized.jar:na]
    at scala.collection.Iterator$class.foreach(Iterator.scala:891) ~[org.scala-lang.scala-library-2.11.12.jar:na]
    at scala.collection.AbstractIterator.foreach(Iterator.scala:1334) ~[org.scala-lang.scala-library-2.11.12.jar:na]
    at scala.collection.MapLike$DefaultKeySet.foreach(MapLike.scala:174) ~[org.scala-lang.scala-library-2.11.12.jar:na]
    at kafka.manager.actor.cluster.KafkaStateActor$KafkaTopicOffsetGetter$$anonfun$run$7$$anonfun$apply$10.apply$mcV$sp(KafkaStateActor.scala:1468) ~[kafka-manager.kafka-manager-1.3.3.21-sans-externalized.jar:na]
    at scala.util.control.Breaks.breakable(Breaks.scala:38) ~[org.scala-lang.scala-library-2.11.12.jar:na]
    at kafka.manager.actor.cluster.KafkaStateActor$KafkaTopicOffsetGetter$$anonfun$run$7.apply(KafkaStateActor.scala:1467) ~[kafka-manager.kafka-manager-1.3.3.21-sans-externalized.jar:na]
    at kafka.manager.actor.cluster.KafkaStateActor$KafkaTopicOffsetGetter$$anonfun$run$7.apply(KafkaStateActor.scala:1436) ~[kafka-manager.kafka-manager-1.3.3.21-sans-externalized.jar:na]
    at scala.Option.fold(Option.scala:158) ~[org.scala-lang.scala-library-2.11.12.jar:na]
    at kafka.manager.actor.cluster.KafkaStateActor$KafkaTopicOffsetGetter.run(KafkaStateActor.scala:1436) ~[kafka-manager.kafka-manager-1.3.3.21-sans-externalized.jar:na]
    at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_191]

Caused by: java.lang.IllegalArgumentException: Login module not specified in JAAS config
    at org.apache.kafka.common.security.JaasConfig.<init>(JaasConfig.java:66) ~[org.apache.kafka.kafka-clients-1.1.0.jar:na]
    at org.apache.kafka.common.security.JaasContext.load(JaasContext.java:90) ~[org.apache.kafka.kafka-clients-1.1.0.jar:na]
    at org.apache.kafka.common.security.JaasContext.loadClientContext(JaasContext.java:84) ~[org.apache.kafka.kafka-clients-1.1.0.jar:na]
    at org.apache.kafka.common.network.ChannelBuilders.create(ChannelBuilders.java:119) ~[org.apache.kafka.kafka-clients-1.1.0.jar:na]
    at org.apache.kafka.common.network.ChannelBuilders.clientChannelBuilder(ChannelBuilders.java:65) ~[org.apache.kafka.kafka-clients-1.1.0.jar:na]
    at org.apache.kafka.clients.ClientUtils.createChannelBuilder(ClientUtils.java:88) ~[org.apache.kafka.kafka-clients-1.1.0.jar:na]
    at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:710) ~[org.apache.kafka.kafka-clients-1.1.0.jar:na]
    ... 14 common frames omitted

[ghost]

Same problem here, any solution?

[jmreymond]

Same problem with last release 1.3.3.22

[patelh]

are u specifying the jaas config in the kafka manger cluster config?

[jmreymond]

I think so and it is the same issue as #591

[patelh]

Are you not sure where you specify the JAAS config? There 2 places. One is through conf file as command line argument when starting kafka manager. The other is through the cluster config inside of kafka manger where you create/modify the cluster config.

[lanxia]

@patelh same problem with last release 1.3.3.22. And when I create the cluster config using kafka manager through web UI, I'm confused with the SASL JAAS Config, should I fill with the path to the config file or the content of the file?

And another question, I have multiple Kafka Cluster (each with a zookeeper cluster using different SASL), how could I specify the SASL information about the zookeeper cluster so I can use manager to create/delete topic ? I think the first way you mentioned above just for the zookeeper cluster where manager to store the meta data not for the broker's zookeeper cluster.

[jmreymond]

Using kafka manager UI, it is better but not perfect. Consumers are not displayed
I have these message

2019-01-15 14:08:28,364 - [INFO] - from org.apache.zookeeper.ClientCnxn in kafka-manager-system-akka.actor.default-dispatcher-2-SendThread(114.221.241.80:2181)
Opening socket connection to server 114.221.241.80/114.221.241.80:2181. Will not attempt to authenticate using SASL (unknown error)

and this messsage:

2019-01-15 14:08:30,161 - [ERROR] - from kafka.manager.actor.cluster.package$TryLogErrorHelper in KafkaManagedOffsetCache
Failed to create consumer for offset topic for cluster KAFKA_BPCE-IT_INT_ant
org.apache.kafka.common.KafkaException: Failed to construct kafka consumer
        at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:793) ~[org.apache.kafka.kafka-clients-1.1.0.jar:na]
        at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:644) ~[org.apache.kafka.kafka-clients-1.1.0.jar:na]
        at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:624) ~[org.apache.kafka.kafka-clients-1.1.0.jar:na]
        at kafka.manager.actor.cluster.KafkaManagedOffsetCache.kafka$manager$actor$cluster$KafkaManagedOffsetCache$$createKafkaConsumer(KafkaStateActor.scala:283) ~[kafka-manager.kafka-manager-1.3.3.22-sans-externalized.jar:na]
        at kafka.manager.actor.cluster.KafkaManagedOffsetCache$$anonfun$run$2.apply(KafkaStateActor.scala:327) ~[kafka-manager.kafka-manager-1.3.3.22-sans-externalized.jar:na]

followed by

2019-01-15 14:08:30,178 - [ERROR] - from kafka.manager.actor.cluster.KafkaStateActor in kafka-manager-system-akka.actor.default-dispatcher-4
consumerProperties:{key.deserializer=org.apache.kafka.common.serialization.ByteArrayDeserializer, sasl.mechanism=SCRAM-SHA-512, bootstrap.servers=a-bilibgk002.dom101.intres:9093, group.id=kafka.manager.actor.cluster.KafkaStateActor.KafkaTopicOffsetGetter, sasl.jaas.config=/appli/kafka/kafka-manager-1.3.3.22/conf/jaas.conf, value.deserializer=org.apache.kafka.common.serialization.ByteArrayDeserializer, security.protocol=SASL_PLAINTEXT} WARNING arguments left: 1

2019-01-15 14:08:30,178 - [ERROR] - from kafka.manager.actor.cluster.KafkaStateActor in kafka-manager-system-akka.actor.default-dispatcher-4
KafkaTopicOffsetGetter exception
org.apache.kafka.common.KafkaException: Failed to construct kafka consumer
        at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:793) ~[org.apache.kafka.kafka-clients-1.1.0.jar:na]
        at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:644) ~[org.apache.kafka.kafka-clients-1.1.0.jar:na]
        at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:624) ~[org.apache.kafka.kafka-clients-1.1.0.jar:na]
        at kafka.manager.actor.cluster.KafkaStateActor$KafkaTopicOffsetGetter$$anonfun$run$7$$anonfun$apply$10$$anonfun$apply$mcV$sp$14.apply(KafkaStateActor.scala:1490) ~[kafka-manager.kafka-manager-1.3.3.22-sans-externalized.jar:na]
        at kafka.manager.actor.cluster.KafkaStateActor$KafkaTopicOffsetGetter$$anonfun$run$7$$anonfun$apply$10$$anonfun$apply$mcV$sp$14.apply(KafkaStateActor.scala:1468) ~[kafka-manager.kafka-manager-1.3.3.22-sans-externalized.jar:na]
        at scala.collection.Iterator$class.foreach(Iterator.scala:891) ~[org.scala-lang.scala-library-2.11.12.jar:na]
        at scala.collection.AbstractIterator.foreach(Iterator.scala:1334) ~[org.scala-lang.scala-library-2.11.12.jar:na]
        at scala.collection.MapLike$DefaultKeySet.foreach(MapLike.scala:174) ~[org.scala-lang.scala-library-2.11.12.jar:na]
        at kafka.manager.actor.cluster.KafkaStateActor$KafkaTopicOffsetGetter$$anonfun$run$7$$anonfun$apply$10.apply$mcV$sp(KafkaStateActor.scala:1468) ~[kafka-manager.kafka-manager-1.3.3.22-sans-externalized.jar:na]
        at scala.util.control.Breaks.breakable(Breaks.scala:38) ~[org.scala-lang.scala-library-2.11.12.jar:na]
        at kafka.manager.actor.cluster.KafkaStateActor$KafkaTopicOffsetGetter$$anonfun$run$7.apply(KafkaStateActor.scala:1467) ~[kafka-manager.kafka-manager-1.3.3.22-sans-externalized.jar:na]

followd by

2019-01-15 14:08:40,259 - [ERROR] - from kafka.manager.actor.cluster.KafkaStateActor in kafka-manager-system-akka.actor.default-dispatcher-2
KafkaTopicOffsetGetter exception
org.apache.kafka.common.KafkaException: Failed to construct kafka consumer
        at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:793) ~[org.apache.kafka.kafka-clients-1.1.0.jar:na]
        at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:644) ~[org.apache.kafka.kafka-clients-1.1.0.jar:na]
        at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:624) ~[org.apache.kafka.kafka-clients-1.1.0.jar:na]
        at kafka.manager.actor.cluster.KafkaStateActor$KafkaTopicOffsetGetter$$anonfun$run$7$$anonfun$apply$10$$anonfun$apply$mcV$sp$14.apply(KafkaStateActor.scala:1490) ~[kafka-manager.kafka-manager-1.3.3.22-sans-externalized.jar:na]
        at kafka.manager.actor.cluster.KafkaStateActor$KafkaTopicOffsetGetter$$anonfun$run$7$$anonfun$apply$10$$anonfun$apply$mcV$sp$14.apply(KafkaStateActor.scala:1468) ~[kafka-manager.kafka-manager-1.3.3.22-sans-externalized.jar:na]
        at scala.collection.Iterator$class.foreach(Iterator.scala:891) ~[org.scala-lang.scala-library-2.11.12.jar:na]
        at scala.collection.AbstractIterator.foreach(Iterator.scala:1334) ~[org.scala-lang.scala-library-2.11.12.jar:na]
        at scala.collection.MapLike$DefaultKeySet.foreach(MapLike.scala:174) ~[org.scala-lang.scala-library-2.11.12.jar:na]
        at kafka.manager.actor.cluster.KafkaStateActor$KafkaTopicOffsetGetter$$anonfun$run$7$$anonfun$apply$10.apply$mcV$sp(KafkaStateActor.scala:1468) ~[kafka-manager.kafka-manager-1.3.3.22-sans-externalized.jar:na]
        at scala.util.control.Breaks.breakable(Breaks.scala:38) ~[org.scala-lang.scala-library-2.11.12.jar:na]
        at kafka.manager.actor.cluster.KafkaStateActor$KafkaTopicOffsetGetter$$anonfun$run$7.apply(KafkaStateActor.scala:1467) ~[kafka-manager.kafka-manager-1.3.3.22-sans-externalized.jar:na]
        at kafka.manager.actor.cluster.KafkaStateActor$KafkaTopicOffsetGetter$$anonfun$run$7.apply(KafkaStateActor.scala:1436) ~[kafka-manager.kafka-manager-1.3.3.22-sans-externalized.jar:na]
        at scala.Option.fold(Option.scala:158) ~[org.scala-lang.scala-library-2.11.12.jar:na]
        at kafka.manager.actor.cluster.KafkaStateActor$KafkaTopicOffsetGetter.run(KafkaStateActor.scala:1436) ~[kafka-manager.kafka-manager-1.3.3.22-sans-externalized.jar:na]
        at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_181]
Caused by: java.lang.IllegalArgumentException: Login module not specified in JAAS config
        at org.apache.kafka.common.security.JaasConfig.<init>(JaasConfig.java:66) ~[org.apache.kafka.kafka-clients-1.1.0.jar:na]
        at org.apache.kafka.common.security.JaasContext.load(JaasContext.java:90) ~[org.apache.kafka.kafka-clients-1.1.0.jar:na]
        at org.apache.kafka.common.security.JaasContext.loadClientContext(JaasContext.java:84) ~[org.apache.kafka.kafka-clients-1.1.0.jar:na]
        at org.apache.kafka.common.network.ChannelBuilders.create(ChannelBuilders.java:119) ~[org.apache.kafka.kafka-clients-1.1.0.jar:na]
        at org.apache.kafka.common.network.ChannelBuilders.clientChannelBuilder(ChannelBuilders.java:65) ~[org.apache.kafka.kafka-clients-1.1.0.jar:na]
        at org.apache.kafka.clients.ClientUtils.createChannelBuilder(ClientUtils.java:88) ~[org.apache.kafka.kafka-clients-1.1.0.jar:na]
        at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:710) ~[org.apache.kafka.kafka-clients-1.1.0.jar:na]
        ... 14 common frames omitted

2019-01-15 14:08:45,283 - [ERROR] - from kafka.manager.actor.cluster.KafkaStateActor in kafka-manager-system-akka.actor.default-dispatcher-2
consumerProperties:{key.deserializer=org.apache.kafka.common.serialization.ByteArrayDeserializer, sasl.mechanism=SCRAM-SHA-512, bootstrap.servers=a-bilibgk002.dom101.intres:9093, group.id=kafka.manager.actor.cluster.KafkaStateActor.KafkaTopicOffsetGetter, sasl.jaas.config=/appli/kafka/kafka-manager-1.3.3.22/conf/jaas.conf, value.deserializer=org.apache.kafka.common.serialization.ByteArrayDeserializer, security.protocol=SASL_PLAINTEXT} WARNING arguments left: 1

2019-01-15 14:08:45,284 - [ERROR] - from kafka.manager.actor.cluster.KafkaStateActor in kafka-manager-system-akka.actor.default-dispatcher-2
KafkaTopicOffsetGetter exception

[doddys]

Hi,

I was having the same problem, but I managed to resolve the error, at least in my config.

Basically the JAAS Config from the UI expect a JAAS file content (KafkaClient Section), not a path to the jaas file.

[jmreymond]

yes, it seems better :-)
now, it is a problem with invalid credentials

[warn] o.a.k.c.NetworkClient - [Consumer clientId=consumer-1, groupId=KMOffsetCache-bilibgk007.dom101.intres] Connection to node -1 terminated during authentication. This may indicate that authentication failed due to invalid credentials.

[lanxia]

@doddys
thanks a lot, and I have a question that the create/delete topic feature is
ok ?

doddys [e-mail] 于2019年2月4日周一 下午3:34写道：

Hi,

I was having the same problem, but I managed to resolve the error, at
least in my config.

Basically the JAAS Config from the UI expect a JAAS file content
(KafkaClient Section), not a path to the jaas file.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/yahoo/kafka-manager/issues/577#issuecomment-460154170,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AEI0tzb5FxCO5ahCcb3-kH2veUGEr_2vks5vJ-JtgaJpZM4YAspY
.

[bigdatasunil]

Hi,

I was having the same problem, but I managed to resolve the error, at least in my config.

Basically the JAAS Config from the UI expect a JAAS file content (KafkaClient Section), not a path to the jaas file.

Yeah, i also struggled on this, finally it is working, even i was passing the jaas file path, after changing to content it started working. Thanks

[doddys]

@doddys thanks a lot, and I have a question that the create/delete topic feature is ok ? doddys [e-mail] 于2019年2月4日周一 下午3:34写道：
…

Yes I am able to create and delete topic. I am using kafka user keytab.

[maldo91]

@doddys I'm getting the same error. Can you please share your jaas.conf file?

[yantzu]

@maldo91 I use content likes below, seems work, no error in application log
org.apache.kafka.common.security.plain.PlainLoginModule required username="xxx" password="xxxx";

[jmreymond]

@yantzu Are you OK with the list of consumers ?

[jmreymond]

good news (at least for me), it is working :-)
Great, thanks for the job

[cwienands1]

I ran in to the same issue as OP and finally got this working by entering the sasl.jaas.config value directly in the UI when creating a cluster. Has anybody else noticed that Kafka Manager then keeps printing out the SASL password in plaintext to the console repeatedly?

2019-09-23 15:44:21,573 - [INFO] k.m.a.c.KafkaStateActor - SASL Mechanism =SASL_MECHANISM_SCRAM512
2019-09-23 15:44:21,573 - [INFO] k.m.a.c.KafkaStateActor - SASL JAAS config=org.apache.kafka.common.security.scram.ScramLoginModule required username="superuser" password="***";

Any way to avoid this?

[wjw870970]

@doddys thanks a lot, and I have a question that the create/delete topic feature is ok ? doddys [email protected] 于2019年2月4日周一 下午3:34写道：
…

Yes I am able to create and delete topic. I am using kafka user keytab.
hi doddys ,how to confiig JAAS file content for user keytab ？

[XLCYun]

Hi, guys, I solved this problem by login to zookeeper by zkCli.sh and delete our kafka-manager config stored in zookeeper. I misconfigured the Java JAAS Config when I add a new cluster in GUI(by setting it to a path, yet it requires just a string java JAAS config, not a file containing it), so I cannot restart the CMAK, so I delete the config for the cluster in zookeeper, and restart CMAK, and add the cluster again. Now it's all fine.
Hope this help.

[HistoryGift]

Hi,
I was having the same problem, but I managed to resolve the error, at least in my config.
Basically the JAAS Config from the UI expect a JAAS file content (KafkaClient Section), not a path to the jaas file.

Yeah, i also struggled on this, finally it is working, even i was passing the jaas file path, after changing to content it started working. Thanks
can you share you "SASL JAAS Config (only applies to SASL based security)" on UI ?

[tanuj83]

it is not working for me, can any one please provide complete steps?