Confluent-kafka-dotnet: "Invalid librdkafka version 901ff, expected at least 903ff" on Linux

[roederja2]

Description

When running version 0.11.0-RC1 on Red Hat linux I get an exception: "Invalid librdkafka version 901ff, expected at least 903ff". Looks like there was a mistake with the native library for that platform?

How to reproduce

Build and run an application that uses the library on RHEL7 in .net core.

Checklist

Please provide the following information:

• [ ] Confluent.Kafka nuget version: 0.11.0-RC1
• [ ] Apache Kafka version: 0.10.1
• [ ] Client configuration:
• [ ] Operating system: REHL 7
• [ ] Provide logs (with "debug" : "..." as necessary in configuration)
• [ ] Provide broker log excerpts
• [ ] Critical issue

[mhowlett]

you'll need to install librdkafka manually for RHEL currently (the nuget package is broken and we won't fix this in the current release).

thanks for letting us know you need this.

[roederja2]

Hm, It seems very easy to add the binaries to the package? I can build them for you if that is the problem.

The error message above was caused by me still having a dependency on the old rdkafka library which ships with an old version of the library for RHEL... That's a nasty incompatibility.

[edenhill]

Bundling pre-built librdkafka binaries is not a problem in itself, but librdkafka's dependencies are.
It depends on openssl, cyrus-sasl, zlib, liblz4, etc, and there is no good way to handle those dependencies automatically from high-level bindings, afaik.
And the errors those dependencies may cause (mismatch, missing, etc) are much harder to identify and troubleshoot for both the end-user and us as maintainers.

So at this time we've decided not to bundle librdkafka but depend on the system's standard packaging methods to provide librdkafka (through Confluent repositories to guarantee the latest version).

We are working on improving this situation.

[treziac]

@roederja2 yep, you should uninstall all rdkafka-dotnet dependancies, as its shipped librdkafka lib would override confluent-kafka-dotnet or even system ones

[ah-]

I think we could provide useful binaries for common distributions that make it easy to use with not too much effort. Statically link zlib/liblz4, .net core already depends on openssl so that will be available in some form, which leaves only cyrus-sasl. Maybe we could even make that optional?

That would make it massively easier to run confluent-kafka-dotnet on .net core, having to bundle a matching librdkafka out of band isn't trivial.

[edenhill]

The librdkafka.redist package (which is based on your original package, @ah- ! :) ) already provides this, and NuGet should sort out the dotnet's dependency on it, so I think we're basically fine here, except for the missing RedHat-built .so files which we'll look into automatic soon.

[treziac]

For info, it should be possible to use existing rdkafka and confluent-kafka side by side (to compare or for upgrading), just prevent copy from the runtimes of rd.kafka.internals (see here for references : https://docs.microsoft.com/en-us/nuget/consume-packages/package-references-in-project-files)

This mean only the dll (or so or other native lib) from librdkafka.redist will be copied to runtime. As librdkafka is API compatible (\o/), this won't be an issue (but it will change some params - read release notes of librdkafka since 0.9.1)

[roederja2]

@edenhill Which confluent RPM repository are you referring to?

[treziac]

http://docs.confluent.io/current/installation.html#installation-yum

[joproulx]

Sorry to piggyback on this issue (let me know if I should open a new one), but I am trying to use the nuget Confluent.Kafka (0.11.6) in a .Net Core 2.2 project on a windows machine and I got the exact same error. Is there some kind of workaround that I can use to fix this issue?

[mhowlett]

This error indicates the .net runtime is finding and loading an earlier version of librdkafka than expected. Are you referencing any librdkafka.redist package explicitly? (if so, remove that). Do you have librdkafka.dll installed anywhere else on your system? (if so, try removing this). Finally, try version 1.0-RC1 which fixes many problems with 0.11.6.

[joproulx]

@mhowlett I was using the NServiceBus.Transport.Kafka package which seemed to cause the issue. After removing it, it was complaining that it couldn't find the librdkafka library. I then installed the latest librdkafka.redist (0.11.6) package and it worked.

Thanks for the quick reply!