Confluent-kafka-go: Could not determine kind of name for C.RD_KAFKA_EVENT_STATS

[rb16]

Pulled latest changes. when i am trying run go code getting this error
could not determine kind of name for C.RD_KAFKA_EVENT_STATS

Here my machine configuration

GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/rb/workspace"
GORACE=""
GOROOT="/usr/local/Cellar/go/1.8.1/libexec"
GOTOOLDIR="/usr/local/Cellar/go/1.8.1/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/jm/rf97g_5d6p91cpp6s_jgzn140000gn/T/go-build788309128=/tmp/go-build -gno-record-gcc-switches -fno-common"
CXX="clang++"
CGO_ENABLED="1"
PKG_CONFIG="pkg-config"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"

[edenhill]

Hi,.. You need to build and install librdkafka master, or use go client
release 0.9.4

Den 11 maj 2017 1:27 em skrev "Ram Bharose Rana" [e-mail]:

Pulled latest changes. when i am trying run go code getting this error
could not determine kind of name for C.RD_KAFKA_EVENT_STATS

Here my machine configuration

GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/rb/workspace"
GORACE=""
GOROOT="/usr/local/Cellar/go/1.8.1/libexec"
GOTOOLDIR="/usr/local/Cellar/go/1.8.1/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/jm/rf97g_5d6p91cpp6s_jgzn140000gn/T/go-build788309128=/tmp/go-build -gno-record-gcc-switches -fno-common"
CXX="clang++"
CGO_ENABLED="1"
PKG_CONFIG="pkg-config"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/confluentinc/confluent-kafka-go/issues/61, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAgCvrtTYMPgcY4dhBOZOG03UdO2zSnCks5r4vCQgaJpZM4NX3NV
.

[rb16]

Thanks, i got working 👍

[mewwts]

I have librdkafka installed, but still see this error. Could you be a bit more specific on what causes this?

[edenhill]

@mewwts The master branch of the Go client requires the master branch of librdkafka (the development is kept in synch).

So you have two options:

• use the latest release of the Go client (v0.9.4) with a recent release of librdkafka (>=v0.9.4)
• remove any installed versions of librdkafka, clone librdkafka master and build and install it.

[mewwts]

Thanks a bunch @edenhill - that was clarifying!

[gaurav36]

@edenhill Hi edenhill,
1) I have installed librdkafka from source code from git clone https://github.com/edenhill/librdkafka.git and doing make install.
After then cloning "go get -u github.com/confluentinc/confluent-kafka-go/kafka" its giving following error:

pkg-config --cflags rdkafka
Package rdkafka was not found in the pkg-config search path.
Perhaps you should add the directory containing `rdkafka.pc'
to the PKG_CONFIG_PATH environment variable
No package 'rdkafka' found
pkg-config: exit status 1

2) If i insalled librdkafka from binary (latest version is 0.9.5) and if i clone confluent-kafka-go its giving following error:
$ go get gopkg.in/confluentinc/confluent-kafka-go.v0/examples/go-kafkacat
github.com/confluentinc/confluent-kafka-go/kafka
could not determine kind of name for C.RD_KAFKA_EVENT_STATS

Do i need to install librdkafka 0.9.4 for making it sync with confluent-kafka-go v0.9.4-RC2 ?

Any clue on this issue ?

[edenhill]

pkg-config will need to find the rdkafka.pc file that is installed when you do 'make install' in librdkafka, the location depends on ./configure --prefix and defaults to /usr/local.
It might be the case that /usr/local is not in your PKG_CONFIG_PATH, the easiest fix is to install librdkafka into /usr, like this:

$ ./configure --prefix /usr
$ make
$ sudo make install

[gaurav36]

@edenhill thank you very much for your reply. I did the same things previously that you have described in your comment. But still the same error after cleaning up every thing and re installing librdkafka from source then doing
$ go get -u github.com/confluentinc/confluent-kafka-go/kafka

Its showing following error again:
pkg-config --cflags rdkafka Package rdkafka was not found in the pkg-config search path. Perhaps you should add the directory containing "rdkafka.pc" to the PKG_CONFIG_PATH environment variable No package "rdkafka" found pkg-config: exit status 1

If i install librdkafka from rpm then importing kafka using
$ go get -u github.com/confluentinc/confluent-kafka-go/kafka
Its showing following error:
02:10 PM Tue Jun 06$ go get -u github.com/confluentinc/confluent-kafka-go/kafka github.com/confluentinc/confluent-kafka-go/kafka /tmp/go-build990109538/github.com/confluentinc/confluent-kafka-go/kafka/_obj/event.cgo2.o: In function _cgo_2117b3ae5bb7_Cfunc_rd_kafka_event_stats: confluentinc/confluent-kafka-go/kafka/event.go:127: undefined reference to rd_kafka_event_stats collect2: error: ld returned 1 exit status

I cleaned up everything and started again both way its showing the same error.

[edenhill]

Sorry for the delay.

The first problem seems to be an installation path problem, librdkafka is installed into a location where pkg-config will not look. You can probably set up PKG_CONFIG_PATH to point to the directory containing the installed rdkafka.pc file.

The latter problem is that the RPM package is too old, you should use the Confluent YUM repositories to make sure you have the latest version of librdkafka installed:
http://docs.confluent.io/current/installation.html#rpm-packages-via-yum

I would go for this second approach.

[liaoyizhe]

@edenhill
I'm sorry to bother you
The program reported an error

/tmp/go-build129217119/command-line-arguments/_obj/exe/producer_example: error while loading shared libraries: librdkafka.so.1: cannot open shared object file: No such file or directory

how can i fix it?
thank u.

[edenhill]

@liaoyizhe Make sure to install librdkafka v0.11.0 as instructed here:
https://github.com/confluentinc/confluent-kafka-go/#installing-librdkafka

[lghinet]

hello,
kind of having the same problem, but on windows with visual studio code

# pkg-config --cflags rdkafka
pkg-config: exec: "pkg-config": executable file not found in %PATH%
Process exiting with code: 2

i dont have pkg-config on windows
@edenhill you mention two options

So you have two options:
use the latest release of the Go client (v0.9.4) with a recent release of librdkafka (>=v0.9.4)
remove any installed versions of librdkafka, clone librdkafka master and build and install it.

how can i use the first one ? if i want the latest librdkafka release from
https://github.com/edenhill/librdkafka/releases/tag/v0.11.0 they only provide sources which i have to compile right ?
can i use microsoft windows nuget ?! with go ? ..
thanks

[lisadeng]

Hi @edenhill ,
Sorry to bother you.
Actually, couple of years ago, you helped me figure out an issue of kafka php client. :)
Recently I had a try on kafka golang client, after installing librdkafka by apt-get install librdkafka, I came across the error:

2017/12/11 07:51:34 ERROR ▶ 0005 Failed to build the application: # github.com/confluentinc/confluent-kafka-go/kafka
../github.com/confluentinc/confluent-kafka-go/kafka/00version.go:45:2: error: #error "confluent-kafka-go requires librdkafka v0.11.0 or later. Install the latest version of librdkafka from the Confluent repositories, see http://docs.confluent.io/current/installation.html"

Then I build it from source, another error occurred:

2017/12/11 08:07:06 ERROR ▶ 0005 Failed to build the application: # github.com/confluentinc/confluent-kafka-go/kafka
/tmp/go-build337827339/github.com/confluentinc/confluent-kafka-go/kafka/_obj/consumer.cgo2.o: In function _cgo_dbdabba0f90b_Cfunc_rd_kafka_offsets_store': /tmp/go-build/github.com/confluentinc/confluent-kafka-go/kafka/_obj/cgo-gcc-prolog:306: undefined reference tord_kafka_offsets_store'
/tmp/go-build337827339/github.com/confluentinc/confluent-kafka-go/kafka/_obj/event.cgo2.o: In function _cgo_dbdabba0f90b_Cfunc_rd_kafka_event_stats': /tmp/go-build/github.com/confluentinc/confluent-kafka-go/kafka/_obj/cgo-gcc-prolog:98: undefined reference tord_kafka_event_stats'
/tmp/go-build337827339/github.com/confluentinc/confluent-kafka-go/kafka/_obj/offset.cgo2.o: In function _cgo_dbdabba0f90b_Cfunc_rd_kafka_offsets_for_times': /tmp/go-build/github.com/confluentinc/confluent-kafka-go/kafka/_obj/cgo-gcc-prolog:61: undefined reference tord_kafka_offsets_for_times'
/tmp/go-build337827339/github.com/confluentinc/confluent-kafka-go/kafka/_obj/producer.cgo2.o: In function do_produce': ../github.com/confluentinc/confluent-kafka-go/kafka/producer.go:41: undefined reference tord_kafka_producev'
collect2: error: ld returned 1 exit status

File rdkafka++.pc and rdkafka.pc exist under folder /usr/lib/pkgconfig, and export PKG_CONFIG_PATH=/usr/lib/pkgconfig was executed.

Could you help to give me some advice? Not sure what went wrong.
Thanks a lot!

[edenhill]

That looks like it is finding librdkafka >0.11.0 when compiling (rdkafka.h header file) but not when linking (library file).

I suggest removing all installed versions of librdkafka from your system, using both your package manager and by searching for rdkafka.h and librdkafka.so*.

Then build and install librdkafka 0.11.3 (latest version) and then rebuild the Go client (go clean -i ./... first).

[lisadeng]

@edenhill
Thank you so much for your help.
It works!

[edenhill]

@lisadeng Glad to hear!