Librdkafka: dynamic linking doesn't work

[ghost]

Hello,

If I try to use dynamic link I always get undefined references.

I installed the libraries in /usr/local/lib ane try to compile rdkafka_example.c using

gcc -I/usr/local/include/librdkafka -L/usr/local/lib -lrdkafka rdkafka_example.c

But I get link errors

/tmp/cczYMsjA.o: In function rd_kafka_message_errstr': rdkafka_example.c:(.text+0x31): undefined reference tord_kafka_err2str'
/tmp/cczYMsjA.o: In function logger': rdkafka_example.c:(.text+0x1f0): undefined reference tord_kafka_name'
/tmp/cczYMsjA.o: In function msg_delivered': rdkafka_example.c:(.text+0x263): undefined reference tord_kafka_err2str'
/tmp/cczYMsjA.o: In function msg_consume': rdkafka_example.c:(.text+0x2e7): undefined reference tord_kafka_topic_name'
rdkafka_example.c:(.text+0x357): undefined reference to rd_kafka_topic_name' /tmp/cczYMsjA.o: In functionmetadata_print':
rdkafka_example.c:(.text+0x5ba): undefined reference to rd_kafka_err2str' rdkafka_example.c:(.text+0x6fc): undefined reference tord_kafka_err2str'
/tmp/cczYMsjA.o: In function sig_usr1': rdkafka_example.c:(.text+0x765): undefined reference tord_kafka_dump'
/tmp/cczYMsjA.o: In function main': rdkafka_example.c:(.text+0x7ea): undefined reference tord_kafka_conf_new'
rdkafka_example.c:(.text+0x7f3): undefined reference to rd_kafka_topic_conf_new' rdkafka_example.c:(.text+0x887): undefined reference tord_kafka_conf_set'
.....

[edenhill]

Hey,

that's weird.

• is this latest master?
• What version of gcc and what OS is this on?
• Try putting -lrdkafka at the end of the gcc argument list.
• output from ls -la /usr/local/lib | grep rdkafka

Thanks

[ghost]

putting -lrdkafka at the end of the gcc arguments list works.

[edenhill]

Great! Some (older) gcc versions are pickier than others on linking order.

[gvd]

I'm having some linking issues with version 5d8d4f01864be70b7a04056905e4646bceae2ab9

I can only seem to get it to link to the static libraries. Using the dynamic libraries I get a bunch of undefined reference errors. I tried compiling the cpp example using the dynamic libs but this results in the same errors. I compile the example using static libs using:

/usr/bin/c++ -g -O2 -fPIC -Wall -Werror -Wfloat-equal -Wpointer-arith -Wno-non-virtual-dtor -I/home/user/dev/install/kafka/include/librdkafka rdkafka_example.cpp -o rdkafka_example_cpp -g ../src-cpp/librdkafka++.a ../src/librdkafka.a -lpthread -lz -lstdc++

That works fine. When trying to compile the cpp example using the dynamic libs:

/usr/bin/c++ -g -O2 -fPIC -Wall -Werror -Wfloat-equal -Wpointer-arith -Wno-non-virtual-dtor -I/home/user/dev/install/kafka/include/librdkafka rdkafka_example.cpp -L/home/user/dev/install/kafka/lib -o rdkafka_example_cpp -lpthread -lz -lstdc++ -lrdkafka++ -lrdkafka

/tmp/ccrQv7rs.o: In function ExampleEventCb::event_cb(RdKafka::Event&)': /home/user/dev/git/librdkafka/examples/rdkafka_example.cpp:92: undefined reference toRdKafka::err2str(RdKafka::ErrorCode)'
/home/user/dev/git/librdkafka/examples/rdkafka_example.cpp:75: undefined reference to RdKafka::err2str(RdKafka::ErrorCode)' /tmp/ccrQv7rs.o: In functionmain':
/home/user/dev/git/librdkafka/examples/rdkafka_example.cpp:139: undefined reference to RdKafka::Conf::create(RdKafka::Conf::ConfType)' /home/user/dev/git/librdkafka/examples/rdkafka_example.cpp:140: undefined reference toRdKafka::Conf::create(RdKafka::Conf::ConfType)'
/tmp/ccrQv7rs.o: In function std::string::_M_data() const': /usr/include/c++/4.8/bits/basic_string.h:293: undefined reference toRdKafka::Conf::DEBUG_CONTEXTS'
/tmp/ccrQv7rs.o: In function main': /home/user/dev/git/librdkafka/examples/rdkafka_example.cpp:276: undefined reference toRdKafka::version()'
/home/user/dev/git/librdkafka/examples/rdkafka_example.cpp:276: undefined reference to RdKafka::version_str()' /home/user/dev/git/librdkafka/examples/rdkafka_example.cpp:338: undefined reference toRdKafka::Producer::create(RdKafka::Conf_, std::string&)'
/home/user/dev/git/librdkafka/examples/rdkafka_example.cpp:350: undefined reference to RdKafka::Topic::create(RdKafka::Handle_, std::string const&, RdKafka::Conf_, std::string&)' /home/user/dev/git/librdkafka/examples/rdkafka_example.cpp:484: undefined reference toRdKafka::wait_destroyed(int)'
/home/user/dev/git/librdkafka/examples/rdkafka_example.cpp:375: undefined reference toRdKafka::err2str(RdKafka::ErrorCode)' /home/user/dev/git/librdkafka/examples/rdkafka_example.cpp:401: undefined reference toRdKafka::Consumer::create(RdKafka::Conf_, std::string&)'
/home/user/dev/git/librdkafka/examples/rdkafka_example.cpp:413: undefined reference to RdKafka::Topic::create(RdKafka::Handle*, std::string const&, RdKafka::Conf*, std::string&)' /home/user/dev/git/librdkafka/examples/rdkafka_example.cpp:425: undefined reference toRdKafka::err2str(RdKafka::ErrorCode)'
collect2: error: ld returned 1 exit status

Note that /home/user/dev/install/kafka/lib contains the installed libs.

[edenhill]

The linker script for the C library was run on the C++ library, hiding all of the C++ symbols.

Fixed in latest master.

Thanks for reporting this!

[bhanushrestha]

I am still getting same compilation error as above while trying to build with gcc compiler the librdkafka cpp example code.

./Librdkafkaexample.o: In function metadata_print': /home/lenovo/cpp-workspace/LibrdKafkaTest/Debug/../Librdkafkaexample.cpp:91: undefined reference toRdKafka::err2str(RdKafka::ErrorCode)'
/home/lenovo/cpp-workspace/LibrdKafkaTest/Debug/../Librdkafkaexample.cpp:121: undefined reference to RdKafka::err2str(RdKafka::ErrorCode)' ./Librdkafkaexample.o: In functionmain':
/home/lenovo/cpp-workspace/LibrdKafkaTest/Debug/../Librdkafkaexample.cpp:258: undefined reference to RdKafka::Conf::create(RdKafka::Conf::ConfType)' /home/lenovo/cpp-workspace/LibrdKafkaTest/Debug/../Librdkafkaexample.cpp:259: undefined reference toRdKafka::Conf::create(RdKafka::Conf::ConfType)'
/home/lenovo/cpp-workspace/LibrdKafkaTest/Debug/../Librdkafkaexample.cpp:411: undefined reference to RdKafka::get_debug_contexts()' /home/lenovo/cpp-workspace/LibrdKafkaTest/Debug/../Librdkafkaexample.cpp:411: undefined reference toRdKafka::version()'
/home/lenovo/cpp-workspace/LibrdKafkaTest/Debug/../Librdkafkaexample.cpp:411: undefined reference to RdKafka::version_str()' /home/lenovo/cpp-workspace/LibrdKafkaTest/Debug/../Librdkafkaexample.cpp:476: undefined reference toRdKafka::Producer::create(RdKafka::Conf_, std::string&)'
/home/lenovo/cpp-workspace/LibrdKafkaTest/Debug/../Librdkafkaexample.cpp:488: undefined reference to RdKafka::Topic::create(RdKafka::Handle_, std::string const&, RdKafka::Conf_, std::string&)' /home/lenovo/cpp-workspace/LibrdKafkaTest/Debug/../Librdkafkaexample.cpp:513: undefined reference toRdKafka::err2str(RdKafka::ErrorCode)'
/home/lenovo/cpp-workspace/LibrdKafkaTest/Debug/../Librdkafkaexample.cpp:542: undefined reference toRdKafka::Consumer::create(RdKafka::Conf_, std::string&)' /home/lenovo/cpp-workspace/LibrdKafkaTest/Debug/../Librdkafkaexample.cpp:554: undefined reference toRdKafka::Topic::create(RdKafka::Handle, std::string const&, RdKafka::Conf, std::string&)'
/home/lenovo/cpp-workspace/LibrdKafkaTest/Debug/../Librdkafkaexample.cpp:566: undefined reference toRdKafka::err2str(RdKafka::ErrorCode)' /home/lenovo/cpp-workspace/LibrdKafkaTest/Debug/../Librdkafkaexample.cpp:602: undefined reference toRdKafka::Producer::create(RdKafka::Conf*, std::string&)'
/home/lenovo/cpp-workspace/LibrdKafkaTest/Debug/../Librdkafkaexample.cpp:615: undefined reference toRdKafka::Topic::create(RdKafka::Handle_, std::string const&, RdKafka::Conf_, std::string&)' /home/lenovo/cpp-workspace/LibrdKafkaTest/Debug/../Librdkafkaexample.cpp:630: undefined reference toRdKafka::err2str(RdKafka::ErrorCode)'
/home/lenovo/cpp-workspace/LibrdKafkaTest/Debug/../Librdkafkaexample.cpp:651: undefined reference toRdKafka::wait_destroyed(int)' ./Librdkafkaexample.o: In functionExampleEventCb::event_cb(RdKafka::Event&)':
/home/lenovo/cpp-workspace/LibrdKafkaTest/Debug/../Librdkafkaexample.cpp:153: undefined reference toRdKafka::err2str(RdKafka::ErrorCode)' /home/lenovo/cpp-workspace/LibrdKafkaTest/Debug/../Librdkafkaexample.cpp:170: undefined reference toRdKafka::err2str(RdKafka::ErrorCode)'

Even though all the libraries are included and the linkers as given in documentation i get this error. Can anyone help?

[edenhill]

Please provide your linking command line.

[bhanushrestha]

@edenhill I used Eclipse IDE that generated this command on console:
_g++ -o "LibrdKafkaTest" ./Librdkafkaexample.o -lrdkafka -lpthread -lrt_

I set the linker to have -lz as well as per your documentation in the github project but that gave error as :

_/usr/bin/ld: cannot find -lz
collect2: error: ld returned 1 exit status_

[edenhill]

You dont seem to be linking with the C++ library, try: -lrdkafka++

[bhanushrestha]

Thanks a ton @edenhill . It worked and now the project is building. Thanks again for the quick response.

[harunkucuksabanoglu]

Hello,
I run the commands below but gcc gives error: rdkafka.h: No such file or directory
Please help.

./configure --prefix=/home/gateway/kafkalib
make
make install

I created kafka.conf under ld.so.conf.d folder. (content: /home/gateway/kafkalib)
echo $LD_LIBRARY_PATH
/usr/local/lib:/home/gateway/kafkalib/lib

gcc -O3 Wall -lrdkafka -lz -lpthread -lrt kafkaexample.c

[edenhill]

You need to put the libraries at the end of the command line:
gcc -O3 -Wall kafkaexample.c -lrdkafka -lz -lpthread -lrt

[harunkucuksabanoglu]

The same error occurs again.

[edenhill]

Would you mind pasting the actual error message?

[harunkucuksabanoglu]

Line 11: #include "rdkafka.h"

Error :
kafkaexample.c:11:21: error: rdkafka.h: No such file or directory

[edenhill]

If you've installed librdkafka to a non-standard location you'll also need to provide the include path to the compiler, e.g.:

gcc -O3 -Wall -I/home/gateway/kafkalib/include/librdkafka kafkaexample.c -lrdkafka -lz -lpthread -lrt

[harunkucuksabanoglu]

Error :
/usr/bin/ld: cannot find -lrdkafka
collect2: ld returned 1 exit status

[edenhill]

These are pretty basic C compilation/building concepts and are outside the scope of librdkafka open source support.

gcc -O3 -Wall -I/home/gateway/kafkalib/include/librdkafka kafkaexample.c -L/home/gateway/kafkalib/lib -lrdkafka -lz -lpthread -lrt

[harunkucuksabanoglu]

Final code is like below and it worked successfully.
Thanks for your help and quick response.

gcc -O3 -Wall -L/home/gateway/kafkalib/lib -lrdkafka -lz -lpthread -lrt -I/home/gateway/kafkalib/include/librdkafka kafkaexample.c