Librdkafka: Problem about install librdkafka

[siyuanmami]

Hi edenhill,
During installing librdkafka on my ubuntu 14.04 system, I came across this error when executing ./configure command:

root@ip-10-0-0-18:~/kafka_client/librdkafka-0.8.6# ./configure
using cache file config.cache
checking for OS or distribution... ok (Ubuntu)
checking for C compiler from CC env... failed
checking for gcc (by command)... ok (cached)
checking for C++ compiler from CXX env... failed
checking for C++ compiler (g++)... ok (cached)
checking executable ld... ok (cached)
checking executable nm... ok (cached)
checking executable objdump... ok (cached)
checking executable strip... ok (cached)
checking for pkgconfig (by command)... ok (cached)
checking for install (by command)... ok (cached)
checking for PIC (by compile)... ok (cached)
checking for GNU-compatible linker options... ok (cached)
checking for GNU linker-script ld flag... ok (cached)
checking for __atomic_32 (by compile)... ok (cached)
checking for __atomic_64 (by compile)... ok (cached)
checking for socket (by compile)... ok (cached)
parsing version '0x00080600'... ok (0.8.6)
checking for libpthread (by pkg-config)... failed
checking for libpthread (by compile)... ok
checking for zlib (by pkg-config)... ok
checking for librt (by pkg-config)... failed
checking for librt (by compile)... ok
checking for nm (by env NM)... ok (cached)
checking for perl (by command)... ok (cached)
Generated rdkafka.pc
Generated Makefile.config
Generated config.h

Configuration summary:
prefix /usr/local
ARCH x86_64
CPU generic
MKL_DISTRO Ubuntu
CC gcc
CXX g++
LD ld
NM nm
OBJDUMP objdump
STRIP strip
CPPFLAGS -g -O2 -fPIC -Wall -Werror -Wfloat-equal -Wpointer-arith
PKG_CONFIG pkg-config
INSTALL install
LIB_LDFLAGS -shared -Wl,-soname,$(LIBFILENAME)
LDFLAG_LINKERSCRIPT -Wl,--version-script=
RDKAFKA_VERSION_STR 0.8.6
LIBS -lpthread -lz -lrt
CFLAGS
CXXFLAGS -Wno-non-virtual-dtor
SYMDUMPER $(NM) -D
exec_prefix /usr/local
bindir /usr/local/bin
sbindir /usr/local/sbin
libexecdir /usr/local/libexec
datadir /usr/local/share
sysconfdir /usr/local/etc
sharedstatedir /usr/local/com
localstatedir /usr/local/var
libdir /usr/local/lib
includedir /usr/local/include
infodir /usr/local/info
mandir /usr/local/man
Generated config.cache

Now type 'make' to build

for those failed items, do I need to fix them before executing make command?
I am not familiar with shell script, could you help to tell me if I miss anything? c complier, c++ complier, or something like that ?
I tried executing make command yesterday, but looked the installation failed ( I can not use librdkafka ) although there was no error prompt during the installation.

Thanks!

[DEvil0000]

cached may not be a good thing - try without cached configure..
you will need to provide some more input to get a helpful answer. (make output etc..)

[edenhill]

The "failed" configure tests are not typically critical, they only try to find different alternatives, or to enable/disable certain futures.
If a required test fails it will be very evident by a large error printout at the end of ./configure execution.

As @DEvil0000 mentions you will need to provide the output of make and sudo make install.
Also make sure that your own application has /usr/local/lib in its library path (e.g., /etc/ld.so.conf or LD_LIBRARY_PATH).

I see that you are running on Ubuntu so you could try sudo apt-get install librdkafka1

[siyuanmami]

@DEvil0000 Thank you for you advice. I tried to install it again, and it worked now.

[siyuanmami]

@edenhill
OK. Thank you for your explanation. I have installed it successfully now even with those "failed" items.
So actually, I can install it by sudo apt-get install librdkafka1 instead, right ?

[edenhill]

Installing the debian/ubuntu package is easier but might give you an older version (which is usually okay), For the latest stuff you'll need to build your own and use steps given above.

[siyuanmami]

@edenhill
Hi edenhill,
I am installing librdkafka on centos 6.5, and after executing ./configure, I found the check for these items failed:
checking for __atomic_32 (by compile)... failed
checking for __atomic_32_lib (by compile)... failed
checking for __sync_32 (by compile)... ok
checking for __atomic_64 (by compile)... failed
checking for __atomic_64_lib (by compile)... failed

Could you help to tell me if there is anything necessary I miss to install ?
Thanks!

[edenhill]

librdkafka requires 64-bit atomics but it seems like your arch or toolchain does not provide it.
You can provide the full output of ./configure, and uname -a and gcc --version

Thanks

[siyuanmami]

@edenhill
Thanks for you help.

1. The full output of ./configure is as follows:
   checking for OS or distribution... ok (CentOS)
   checking for C compiler from CC env... failed
   checking for gcc (by command)... ok
   checking for C++ compiler from CXX env... failed
   checking for C++ compiler (g++)... ok
   checking executable ld... ok
   checking executable nm... ok
   checking executable objdump... ok
   checking executable strip... ok
   checking for pkgconfig (by command)... ok
   checking for install (by command)... ok
   checking for PIC (by compile)... ok
   checking for GNU-compatible linker options... ok
   checking for GNU linker-script ld flag... ok
   checking for __atomic_32 (by compile)... failed
   checking for __atomic_32_lib (by compile)... failed
   checking for __sync_32 (by compile)... ok
   checking for __atomic_64 (by compile)... failed
   checking for __atomic_64_lib (by compile)... failed
   checking for __sync_64 (by compile)... ok
   checking for socket (by compile)... ok
   parsing version '0x00080600'... ok (0.8.6)
   checking for libpthread (by pkg-config)... failed
   checking for libpthread (by compile)... ok
   checking for zlib (by pkg-config)... ok
   checking for librt (by pkg-config)... failed
   checking for librt (by compile)... ok
   checking for nm (by env NM)... ok (cached)
   checking for perl (by command)... ok
   Generated rdkafka.pc
   Generated Makefile.config
   Generated config.h

Configuration summary:
prefix /usr/local
ARCH x86_64
CPU generic
MKL_DISTRO CentOS
CC gcc
CXX g++
LD ld
NM nm
OBJDUMP objdump
STRIP strip
CPPFLAGS -g -O2 -fPIC -Wall -Werror -Wfloat-equal -Wpointer-arith
PKG_CONFIG pkg-config
INSTALL install
LIB_LDFLAGS -shared -Wl,-soname,$(LIBFILENAME)
LDFLAG_LINKERSCRIPT -Wl,--version-script=
RDKAFKA_VERSION_STR 0.8.6
LIBS -lpthread -lz -lrt
CFLAGS
CXXFLAGS -Wno-non-virtual-dtor
SYMDUMPER $(NM) -D
exec_prefix /usr/local
bindir /usr/local/bin
sbindir /usr/local/sbin
libexecdir /usr/local/libexec
datadir /usr/local/share
sysconfdir /usr/local/etc
sharedstatedir /usr/local/com
localstatedir /usr/local/var
libdir /usr/local/lib
includedir /usr/local/include
infodir /usr/local/info
mandir /usr/local/man
Generated config.cache

Now type 'make' to build

2.The output of uname -a is:
Linux iZ94d7vqlolZ 2.6.32-431.23.3.el6.x86_64 #1 SMP Thu Jul 31 17:20:51 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux

3.The output of gcc --version is:
gcc (GCC) 4.4.7 20120313 (Red Hat 4.4.7-16)
Copyright (C) 2010 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Thanks!

[edenhill]

There is no error, you have all you need to build and install the library.

[siyuanmami]

@edenhill

Thanks, edenhill.
But not sure why after I install this php extension which is based on librdkafka, it tells me:
PHP Startup: Unable to load dynamic library '/usr/lib64/php/modules/kafka.so' - librdkafka.so.1: cannot open shared object file: No such file or directory in Unknown on line 0
https://github.com/EVODelavega/phpkafka

[edenhill]

You probably need to add /usr/local/lib to your library search path, since that's where librdkafka is installed by default.

So you could try doing export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib before running your PHP script, or update /etc/ld.so.conf to include /usr/local/lib

[edenhill]

https://github.com/EVODelavega/phpkafka#requirements

[siyuanmami]

@edenhill
Thanks for you great help.
I've tried export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib, it still does not work.
The same error.

[siyuanmami]

@edenhill
By the way, I uninstalled librdkafka by executing 'make clean', and installed it again, is this alright ?

Thanks!

[edenhill]

Try this:
./configure --prefix=/usr
sudo make install

[siyuanmami]

@edenhill
Do you mean ./configure --prefix=/usr for installing librdkafka ?

[edenhill]

Yes
Den 16 okt 2015 11:15 skrev "siyuanmami" [email protected]:

@edenhill https://github.com/edenhill
Do you mean ./configure --prefix=/usr for installing librdkafka ?

—
Reply to this email directly or view it on GitHub
https://github.com/edenhill/librdkafka/issues/370#issuecomment-148662999
.

[siyuanmami]

@edenhill
Then librdkafka will be installed under /usr, right ?

[edenhill]

Yes
Den 16 okt 2015 11:16 skrev "siyuanmami" [email protected]:

@edenhill https://github.com/edenhill
Then librdkafka will be installed under /usr, right ?

—
Reply to this email directly or view it on GitHub
https://github.com/edenhill/librdkafka/issues/370#issuecomment-148663174
.

[siyuanmami]

@edenhill
Do I need to "make clean" the previous one before "./configure --prefix=/usr" ?
Then I need to export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr, right ?

Thanks!

[edenhill]

Nope, just reconfigure and install
Den 16 okt 2015 11:34 skrev "siyuanmami" [email protected]:

@edenhill https://github.com/edenhill
Do I need to "make clean" the previous one before "./configure
--prefix=/usr" ?
Then I need to export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr, right ?

Thanks!

—
Reply to this email directly or view it on GitHub
https://github.com/edenhill/librdkafka/issues/370#issuecomment-148666430
.

[siyuanmami]

@edenhill
OK.
May I know the reason of doing so ?
Thanks!

[edenhill]

/usr is already in the default library search path.
Den 16 okt 2015 11:37 skrev "siyuanmami" [email protected]:

@edenhill https://github.com/edenhill
OK.
May I know the reason of doing so ?
Thanks!

—
Reply to this email directly or view it on GitHub
https://github.com/edenhill/librdkafka/issues/370#issuecomment-148667613
.

[edenhill]

Did this make it work?

[siyuanmami]

@edenhill
Thanks you so much for you patience!
I am so glad that I installed that php extension successfully after I updated /etc/ld.so.conf to include /usr/local/lib !
I really appreciate your help and have learned a lot from you!
^_^

[edenhill]

Perfect!

[edenhill]

@sp-suresh The actual build error is probably above the pasted output

[edenhill]

Those errors look like v8 version errors:

../src/Int64.cc:85:43: error: no matching function for call to ‘v8::Value::ToNumber()’
         obj = new Int64(info[0]->ToNumber());

and are not related to librdkafka in any way.
File the issue on node-rdkafka instead.

[amikar]

sudo apt-get install build-essential

Worked for me