Confluent-kafka-dotnet: No such configuration property: "sasl.kerberos.keytab"

[coloya-liou]

Hi, I'm new here.
I've been trying to apply the MIT Kerberos authentication with the use of a keytab.

Below is my code (in .NET).

public static void Main()
{
string topicName = "test-topic";
var producerConfig = new ProducerConfig
{
BootstrapServers = "bootstrapServer ",
SecurityProtocol = SecurityProtocolType.Sasl_Ssl,
SaslMechanism = SaslMechanismType.Gssapi,
SaslKerberosServiceName = "ksfka",
SaslKerberosPrincipal = "[email protected]",
SaslKerberosKeytab = @"D:kafka.keytab",
SaslKerberosKinitCmd = @"kinit -k -t D:kafka.keytab [email protected]"
};

using (var producer = new Producer(producerConfig))
{
Console.WriteLine($"{producer.Name} producing on {topicName}. q to exit.");
string text="123";
while ((text = Console.ReadLine()) != "q")
{
var deliveryReport = producer.ProduceAsync(topicName, new Message { Value = text });
deliveryReport.ContinueWith(task =>
{
Console.WriteLine($"Partition: {task.Result.Partition}, Offset: {task.Result.Offset}, Message: {text}");
});
}
}
}

The following situation has occurred.

System.InvalidOperationException
HResult=0x80131509
Message=No such configuration property: "sasl.kerberos.keytab"
Source=Confluent.Kafka
StackTrace:
Confluent.Kafka.Impl.SafeConfigHandle.Set(String name, String value)
Confluent.Kafka.Producer.<>c__DisplayClass33_0.<.ctor>b__6(KeyValuePair2 kvp) System.Collections.Generic.List1.ForEach(Action1 action) Confluent.Kafka.Producer..ctor(IEnumerable1 config)
Confluent.Kafka.Producer2..ctor(IEnumerable1 config, SerializerGenerator1 keySerializerGenerator, SerializerGenerator1 valueSerializerGenerator)
Confluent.Kafka.Producer2..ctor(IEnumerable1 config, Serializer1 keySerializer, Serializer1 valueSerializer)
DotNetExample.Main() in D:DoNetKafkaProgram.cs:line 21

How can I solve it?

Checklist

Please provide the following information:

• [ ] Confluent.Kafka nuget version:1.0.0-beta
• [ ] Apache Kafka version: cloudera kafka 3.1.0
• [ ] Client configuration:
• [ ] Operating system:windows 10 build 1803
• [ ] Provide logs (with "debug" : "..." as necessary in configuration)
• [ ] Provide broker log excerpts
• [ ] Critical issue

[mhowlett]

This config property is apparently only supported on the linux/osx builds of librdkafka. On windows, I believe we only support credentials of the logged in user.

[mhowlett]

labelled this as 'enhancement'. We should update our documentation which is not at all clear on this.

[mathsigit]

Hi @mhowlett ,
I have the same issue ,and could you please describe more detail about how to use credentials of the logged in user on Windows?
Thanks a lot.

[mnajib89]

Hi @mhowlett
I'm having the same issue while trying to consume in .Net client.
Is there any alternative for consuming a Kerberos enabled producer in .Net?

[WendelEstrada]

Hi @mhowlett
I'm having the same issue in .NET Core on Windows.
Do you have any news?

[laxman-munigala]

Hi @mhowlett Any updates on this please. This is preventing .NET on Windows to connect and use Kafka. Any help and insights would be greatly helpful.

[MANYAM-HARI]

Hi @mhowlett Any updates on this, please. This is preventing .NET on Windows to connect and use Kafka. Any help and insights would be greatly helpful.

[COMENGUGUR]

Hello, is there any solution for fixing this problem?

[meena1977]

Hi @mhowlett , Any solutions to this? Any help would be appreciated...

[akkaladevi1992]

Hi @mhowlett, Is there any solution for fixing this problem?

[rajeshg81]

This config property is apparently only supported on the linux/osx builds of librdkafka. On windows, I believe we only support credentials of the logged in user.

@mhowlett, Do you have any update on this please....I'm using C# Confluent Kafka and getting same error... I'm using SASL Kerberos authentication.

[SmugStacy]

This is a windows-nix-ish workaround, but I used version 1.4.3 of confluent.kafka and .Net Core 3.1 running under Windows subsystem for linux (CentOS 7) on a Windows 10 host.
https://github.com/yuk7/CentWSL/releases

Other than installing the .Net core runtime (or SDK) in the subsystem, I needed to make sure the following two packages were also installed:
sasl dependencies:
sudo yum install cyrus-sasl-gssapi cyrus-sasl-devel
kerberos client dependencies
sudo yum install krb5-workstation krb5-libs krb5-auth-dialog

After that, I created a new ProducerConfig instance, passed it into the ProducerBuilder class, published the solution using the correct RuntimeIdentifiers for centos, copied it over to the subsystem, and it transmitted messages to the cluster without any issues.
Note: My config used SaslMechanism.Gssapi, SecurityProtocol.SaslPlaintext, and a headless keytab file+principal.

[Darudes]

Hi, I'm new here.
I've been trying to apply the MIT Kerberos authentication with the use of a keytab.

Below is my code (in .NET).

public static void Main()
{
string topicName = "test-topic";
var producerConfig = new ProducerConfig
{
BootstrapServers = "bootstrapServer ",
SecurityProtocol = SecurityProtocolType.Sasl_Ssl,
SaslMechanism = SaslMechanismType.Gssapi,
SaslKerberosServiceName = "ksfka",
SaslKerberosPrincipal = "[email protected]",
SaslKerberosKeytab = @"D:kafka.keytab",
SaslKerberosKinitCmd = @"kinit -k -t D:kafka.keytab [email protected]"
};

using (var producer = new Producer(producerConfig))
{
Console.WriteLine($"{producer.Name} producing on {topicName}. q to exit.");
string text="123";
while ((text = Console.ReadLine()) != "q")
{
var deliveryReport = producer.ProduceAsync(topicName, new Message { Value = text });
deliveryReport.ContinueWith(task =>
{
Console.WriteLine($"Partition: {task.Result.Partition}, Offset: {task.Result.Offset}, Message: {text}");
});
}
}
}

The following situation has occurred.

System.InvalidOperationException
HResult=0x80131509
Message=No such configuration property: "sasl.kerberos.keytab"
Source=Confluent.Kafka
StackTrace:
Confluent.Kafka.Impl.SafeConfigHandle.Set(String name, String value)
Confluent.Kafka.Producer.<>c__DisplayClass33_0.<.ctor>b__6(KeyValuePair2 kvp) System.Collections.Generic.List1.ForEach(Action1 action) Confluent.Kafka.Producer..ctor(IEnumerable1 config)
Confluent.Kafka.Producer2..ctor(IEnumerable1 config, SerializerGenerator1 keySerializerGenerator, SerializerGenerator1 valueSerializerGenerator)
Confluent.Kafka.Producer2..ctor(IEnumerable1 config, Serializer1 keySerializer, Serializer1 valueSerializer)
DotNetExample.Main() in D:DoNetKafkaProgram.cs:line 21

How can I solve it?

Checklist

Please provide the following information:

• [ ] Confluent.Kafka nuget version:1.0.0-beta
• [ ] Apache Kafka version: cloudera kafka 3.1.0
• [ ] Client configuration:
• [ ] Operating system:windows 10 build 1803
• [ ] Provide logs (with "debug" : "..." as necessary in configuration)
• [ ] Provide broker log excerpts
• [ ] Critical issue

[jcasement]

Same problem here: On Windows, Kafka uses the service account which may be fine for a Windows console app or a Windows service but if you're trying to use Kafka producer from a WebAPI this does not work at least for me since the service account for IIS is really under the app pool control and not an AD account. Why these properties are exposed here gives you a sense that you could use them but in the end it looks like in this Kafka (1.3.0) version you can't. Believe me, I gone down the proverbial rabbit hole to create the keytab file and circle back around to put this in with zero to show for it.
Has anyone solved this with strictly a Windows approach? (sorry but the CentOS solution is a none starter for most)