Librdkafka: bro-dist unkown

[dlewis2017]

Please point me in the correct direction if this is not the right place to post this issue/question, Thank you.

Description

After following the installation instructions on https://www.bro.org/sphinx/components/bro-plugins/kafka/README.html , I get to this command for compiling the bro plugin:
./configure --bro-dist=$BRO_SRC
it says "Unknown option bro-dist="

How to reproduce

Follow installation instruction https://www.bro.org/sphinx/components/bro-plugins/kafka/README.html up until the command shown above

Checklist

Please provide the following information:

• [X] librdkafka version (release number or git tag): 0.9.4
• [X] Apache Kafka version: Confluent Kafka 2.11
• [X] librdkafka client configuration: Bro 2.5.1
• [X] Operating system: RedHat Linux
• [ ] Using the legacy Consumer
• [ ] Using the high-level KafkaConsumer
• [ ] Provide logs (with debug=.. as necessary) from librdkafka
• [ ] Provide broker log excerpts
• [ ] Critical issue

COMMAND RESULTS

_Result of: ./configure_

using cache file config.cache
checking for OS or distribution... ok (Linux)
checking for C compiler from CC env... failed
checking for gcc (by command)... ok (cached)
checking for C++ compiler from CXX env... failed
checking for C++ compiler (g++)... ok (cached)
checking executable ld... ok (cached)
checking executable nm... ok (cached)
checking executable objdump... ok (cached)
checking executable strip... ok (cached)
checking for pkgconfig (by command)... ok (cached)
checking for install (by command)... ok (cached)
checking for PIC (by compile)... ok (cached)
checking for GNU-compatible linker options... ok (cached)
checking for GNU linker-script ld flag... ok (cached)
checking for __atomic_32 (by compile)... ok (cached)
checking for __atomic_64 (by compile)... ok (cached)
checking for socket (by compile)... ok (cached)
parsing version '0x000904ff'... ok (0.9.4)
checking for libpthread (by pkg-config)... failed
checking for libpthread (by compile)... ok (cached)
checking for zlib (by pkg-config)... ok
checking for zlib (by compile)... ok (cached)
checking for libcrypto (by pkg-config)... ok
checking for libcrypto (by compile)... ok (cached)
checking for liblz4 (by pkg-config)... failed
checking for liblz4 (by compile)... failed (disable)
checking for libssl (by pkg-config)... ok
checking for libssl (by compile)... ok (cached)
checking for libsasl2 (by pkg-config)... failed
checking for libsasl2 (by compile)... failed (disable)
checking for libsasl (by pkg-config)... failed
checking for libsasl (by compile)... failed (disable)
checking for regex (by compile)... ok (cached)
checking for librt (by pkg-config)... failed
checking for librt (by compile)... ok (cached)
checking for strndup (by compile)... ok (cached)
checking for nm (by env NM)... ok (cached)
checking for python (by command)... ok (cached)
Generated Makefile.config
Generated config.h

Configuration summary:
prefix /usr/local
ARCH x86_64
CPU generic
GEN_PKG_CONFIG y
ENABLE_DEVEL n
ENABLE_VALGRIND n
ENABLE_REFCNT_DEBUG n
ENABLE_SHAREDPTR_DEBUG n
ENABLE_LZ4 y
ENABLE_SSL y
ENABLE_SASL y
MKL_APP_NAME librdkafka
MKL_APP_DESC_ONELINE The Apache Kafka C/C++ library
MKL_DISTRO Linux
CC gcc
CXX g++
LD ld
NM nm
OBJDUMP objdump
STRIP strip
CPPFLAGS -g -O2 -fPIC -Wall -Wsign-compare -Wfloat-equal -Wpointer-arith
PKG_CONFIG pkg-config
INSTALL install
LIB_LDFLAGS -shared -Wl,-soname,$(LIBFILENAME)
LDFLAG_LINKERSCRIPT -Wl,--version-script=
RDKAFKA_VERSION_STR 0.9.4
MKL_APP_VERSION 0.9.4
LIBS -lpthread -lz -lz -lcrypto -lcrypto -lssl -lcrypto -lssl -lrt
CFLAGS
CXXFLAGS -Wno-non-virtual-dtor
SYMDUMPER $(NM) -D
exec_prefix /usr/local
bindir /usr/local/bin
sbindir /usr/local/sbin
libexecdir /usr/local/libexec
datadir /usr/local/share
sysconfdir /usr/local/etc
sharedstatedir /usr/local/com
localstatedir /usr/local/var
libdir /usr/local/lib
includedir /usr/local/include
infodir /usr/local/info
mandir /usr/local/man
Generated config.cache

_Result of: make_

make[1]: Entering directory /home/ec2-user/bro/plugins/librdkafka-0.9.4/src' gcc -MD -MP -g -O2 -fPIC -Wall -Wsign-compare -Wfloat-equal -Wpointer-arith -c rdkafka.c -o rdkafka.o gcc -MD -MP -g -O2 -fPIC -Wall -Wsign-compare -Wfloat-equal -Wpointer-arith -c rdkafka_broker.c -o rdkafka_broker.o gcc -MD -MP -g -O2 -fPIC -Wall -Wsign-compare -Wfloat-equal -Wpointer-arith -c rdkafka_msg.c -o rdkafka_msg.o gcc -MD -MP -g -O2 -fPIC -Wall -Wsign-compare -Wfloat-equal -Wpointer-arith -c rdkafka_topic.c -o rdkafka_topic.o gcc -MD -MP -g -O2 -fPIC -Wall -Wsign-compare -Wfloat-equal -Wpointer-arith -c rdkafka_conf.c -o rdkafka_conf.o gcc -MD -MP -g -O2 -fPIC -Wall -Wsign-compare -Wfloat-equal -Wpointer-arith -c rdkafka_timer.c -o rdkafka_timer.o gcc -MD -MP -g -O2 -fPIC -Wall -Wsign-compare -Wfloat-equal -Wpointer-arith -c rdkafka_offset.c -o rdkafka_offset.o gcc -MD -MP -g -O2 -fPIC -Wall -Wsign-compare -Wfloat-equal -Wpointer-arith -c rdkafka_transport.c -o rdkafka_transport.o gcc -MD -MP -g -O2 -fPIC -Wall -Wsign-compare -Wfloat-equal -Wpointer-arith -c rdkafka_buf.c -o rdkafka_buf.o gcc -MD -MP -g -O2 -fPIC -Wall -Wsign-compare -Wfloat-equal -Wpointer-arith -c rdkafka_queue.c -o rdkafka_queue.o gcc -MD -MP -g -O2 -fPIC -Wall -Wsign-compare -Wfloat-equal -Wpointer-arith -c rdkafka_op.c -o rdkafka_op.o gcc -MD -MP -g -O2 -fPIC -Wall -Wsign-compare -Wfloat-equal -Wpointer-arith -c rdkafka_request.c -o rdkafka_request.o gcc -MD -MP -g -O2 -fPIC -Wall -Wsign-compare -Wfloat-equal -Wpointer-arith -c rdkafka_cgrp.c -o rdkafka_cgrp.o gcc -MD -MP -g -O2 -fPIC -Wall -Wsign-compare -Wfloat-equal -Wpointer-arith -c rdkafka_pattern.c -o rdkafka_pattern.o gcc -MD -MP -g -O2 -fPIC -Wall -Wsign-compare -Wfloat-equal -Wpointer-arith -c rdkafka_partition.c -o rdkafka_partition.o gcc -MD -MP -g -O2 -fPIC -Wall -Wsign-compare -Wfloat-equal -Wpointer-arith -c rdkafka_subscription.c -o rdkafka_subscription.o gcc -MD -MP -g -O2 -fPIC -Wall -Wsign-compare -Wfloat-equal -Wpointer-arith -c rdkafka_assignor.c -o rdkafka_assignor.o gcc -MD -MP -g -O2 -fPIC -Wall -Wsign-compare -Wfloat-equal -Wpointer-arith -c rdkafka_range_assignor.c -o rdkafka_range_assignor.o gcc -MD -MP -g -O2 -fPIC -Wall -Wsign-compare -Wfloat-equal -Wpointer-arith -c rdkafka_roundrobin_assignor.c -o rdkafka_roundrobin_assignor.o gcc -MD -MP -g -O2 -fPIC -Wall -Wsign-compare -Wfloat-equal -Wpointer-arith -c rdkafka_feature.c -o rdkafka_feature.o gcc -MD -MP -g -O2 -fPIC -Wall -Wsign-compare -Wfloat-equal -Wpointer-arith -c rdcrc32.c -o rdcrc32.o gcc -MD -MP -g -O2 -fPIC -Wall -Wsign-compare -Wfloat-equal -Wpointer-arith -c rdaddr.c -o rdaddr.o gcc -MD -MP -g -O2 -fPIC -Wall -Wsign-compare -Wfloat-equal -Wpointer-arith -c rdrand.c -o rdrand.o gcc -MD -MP -g -O2 -fPIC -Wall -Wsign-compare -Wfloat-equal -Wpointer-arith -c rdlist.c -o rdlist.o gcc -MD -MP -g -O2 -fPIC -Wall -Wsign-compare -Wfloat-equal -Wpointer-arith -c tinycthread.c -o tinycthread.o gcc -MD -MP -g -O2 -fPIC -Wall -Wsign-compare -Wfloat-equal -Wpointer-arith -c rdlog.c -o rdlog.o gcc -MD -MP -g -O2 -fPIC -Wall -Wsign-compare -Wfloat-equal -Wpointer-arith -c rdstring.c -o rdstring.o gcc -MD -MP -g -O2 -fPIC -Wall -Wsign-compare -Wfloat-equal -Wpointer-arith -c rdkafka_event.c -o rdkafka_event.o sgcc -MD -MP -g -O2 -fPIC -Wall -Wsign-compare -Wfloat-equal -Wpointer-arith -c rdkafka_metadata.c -o rdkafka_metadata.o udo gcc -MD -MP -g -O2 -fPIC -Wall -Wsign-compare -Wfloat-equal -Wpointer-arith -c rdregex.c -o rdregex.o gcc -MD -MP -g -O2 -fPIC -Wall -Wsign-compare -Wfloat-equal -Wpointer-arith -c rdports.c -o rdports.o gcc -MD -MP -g -O2 -fPIC -Wall -Wsign-compare -Wfloat-equal -Wpointer-arith -c rdkafka_metadata_cache.c -o rdkafka_metadata_cache.o gcc -MD -MP -g -O2 -fPIC -Wall -Wsign-compare -Wfloat-equal -Wpointer-arith -c rdavl.c -o rdavl.o magcc -MD -MP -g -O2 -fPIC -Wall -Wsign-compare -Wfloat-equal -Wpointer-arith -c snappy.c -o snappy.o ke gcc -MD -MP -g -O2 -fPIC -Wall -Wsign-compare -Wfloat-equal -Wpointer-arith -c rdgz.c -o rdgz.o Creating shared library librdkafka.so.1 gcc -shared -Wl,-soname,librdkafka.so.1 -Wl,--version-script=librdkafka.lds rdkafka.o rdkafka_broker.o rdkafka_msg.o rdkafka_topic.o rdkafka_conf.o rdkafka_timer.o rdkafka_offset.o rdkafka_transport.o rdkafka_buf.o rdkafka_queue.o rdkafka_op.o rdkafka_request.o rdkafka_cgrp.o rdkafka_pattern.o rdkafka_partition.o rdkafka_subscription.o rdkafka_assignor.o rdkafka_range_assignor.o rdkafka_roundrobin_assignor.o rdkafka_feature.o rdcrc32.o rdaddr.o rdrand.o rdlist.o tinycthread.o rdlog.o rdstring.o rdkafka_event.o rdkafka_metadata.o rdregex.o rdports.o rdkafka_metadata_cache.o rdavl.o snappy.o rdgz.o -o librdkafka.so.1 -lpthread -lz -lz -lcrypto -lcrypto -lssl -lcrypto -lssl -lrt Creating static library librdkafka.a ar rcs librdkafka.a rdkafka.o rdkafka_broker.o rdkafka_msg.o rdkafka_topic.o rdkafka_conf.o rdkafka_timer.o rdkafka_offset.o rdkafka_transport.o rdkafka_buf.o rdkafka_queue.o rdkafka_op.o rdkafka_request.o rdkafka_cgrp.o rdkafka_pattern.o rdkafka_partition.o rdkafka_subscription.o rdkafka_assignor.o rdkafka_range_assignor.o rdkafka_roundrobin_assignor.o rdkafka_feature.o rdcrc32.o rdaddr.o rdrand.o rdlist.o tinycthread.o rdlog.o rdstring.o rdkafka_event.o rdkafka_metadata.o rdregex.o rdports.o rdkafka_metadata_cache.o rdavl.o snappy.o rdgz.o Generating pkg-config file rdkafka.pc Checking librdkafka integrity librdkafka.so.1 OK librdkafka.a OK Symbol visibility OK make[1]: Leaving directory/home/ec2-user/bro/plugins/librdkafka-0.9.4/src'
make[1]: Entering directory /home/ec2-user/bro/plugins/librdkafka-0.9.4/src-cpp' Generating pkg-config file rdkafka++.pc Checking librdkafka++ integrity librdkafka++.so.1 OK librdkafka++.a OK make[1]: Leaving directory/home/ec2-user/bro/plugins/librdkafka-0.9.4/src-cpp'
make -C examples
make[1]: Entering directory `/home/ec2-user/bro/plugins/librdkafka-0.9.4/examples'
gcc -g -O2 -fPIC -Wall -Wsign-compare -Wfloat-equal -Wpointer-arith -I../src rdkafka_example.c -o rdkafka_example \
../src/librdkafka.a -lpthread -lz -lz -lcrypto -lcrypto -lssl -lcrypto -lssl -lrt
ins rdkafka_example is ready
Run producer (write messages on stdin)
./rdkafka_example -P -t -p

or consumer
./rdkafka_example -C -t -p

More usage options:
./rdkafka_example -h
gcc -g -O2 -fPIC -Wall -Wsign-compare -Wfloat-equal -Wpointer-arith -I../src rdkafka_performance.c -o rdkafka_performance \
../src/librdkafka.a -lpthread -lz -lz -lcrypto -lcrypto -lssl -lcrypto -lssl -lrt
tal rdkafka_performance is ready
Run producer
./rdkafka_performance -P -t -p -s

or consumer
./rdkafka_performance -C -t -p

More usage options:
./rdkafka_performance -h
g++ -g -O2 -fPIC -Wall -Wsign-compare -Wfloat-equal -Wpointer-arith -Wno-non-virtual-dtor -I../src-cpp rdkafka_example.cpp -o rdkafka_example_cpp \
../src-cpp/librdkafka++.a ../src/librdkafka.a -lpthread -lz -lz -lcrypto -lcrypto -lssl -lcrypto -lssl -lrt -lstdc++
lgcc -g -O2 -fPIC -Wall -Wsign-compare -Wfloat-equal -Wpointer-arith -I../src rdkafka_consumer_example.c -o rdkafka_consumer_example \
../src/librdkafka.a -lpthread -lz -lz -lcrypto -lcrypto -lssl -lcrypto -lssl -lrt
rdkafka_consumer_example is ready

./rdkafka_consumer_example

More usage options:
./rdkafka_consumer_example -h
g++ -g -O2 -fPIC -Wall -Wsign-compare -Wfloat-equal -Wpointer-arith -Wno-non-virtual-dtor -I../src-cpp rdkafka_consumer_example.cpp -o rdkafka_consumer_example_cpp \
../src-cpp/librdkafka++.a ../src/librdkafka.a -lpthread -lz -lz -lcrypto -lcrypto -lssl -lcrypto -lssl -lrt -lstdc++
g++ -g -O2 -fPIC -Wall -Wsign-compare -Wfloat-equal -Wpointer-arith -Wno-non-virtual-dtor -I../src-cpp kafkatest_verifiable_client.cpp -o kafkatest_verifiable_client \
../src-cpp/librdkafka++.a ../src/librdkafka.a -lpthread -lz -lz -lcrypto -lcrypto -lssl -lcrypto -lssl -lrt -lstdc++
make[1]: Leaving directory /home/ec2-user/bro/plugins/librdkafka-0.9.4/examples' Updating Checking integrity CONFIGURATION.md OK examples/rdkafka_example OK examples/rdkafka_performance OK examples/rdkafka_example_cpp OK make[1]: Entering directory/home/ec2-user/bro/plugins/librdkafka-0.9.4/src'
Checking librdkafka integrity
librdkafka.so.1 OK
librdkafka.a OK
Symbol visibility OK
make[1]: Leaving directory /home/ec2-user/bro/plugins/librdkafka-0.9.4/src' make[1]: Entering directory/home/ec2-user/bro/plugins/librdkafka-0.9.4/src-cpp'
Checking librdkafka++ integrity
librdkafka++.so.1 OK
librdkafka++.a OK
make[1]: Leaving directory `/home/ec2-user/bro/plugins/librdkafka-0.9.4/src-cpp'

_Result of: sudo make install_

make[1]: Entering directory /home/ec2-user/bro/plugins/librdkafka-0.9.4/src' Install librdkafka to /usr/local install -d $DESTDIR/usr/local/include/librdkafka ; \ install -d $DESTDIR/usr/local/lib ; \ install rdkafka.h $DESTDIR/usr/local/include/librdkafka ; \ install librdkafka.a $DESTDIR/usr/local/lib ; \ install librdkafka.so.1 $DESTDIR/usr/local/lib ; \ [ -f "rdkafka.pc" ] && ( \ install -d $DESTDIR/usr/local/lib/pkgconfig ; \ install -m 0644 rdkafka.pc $DESTDIR/usr/local/lib/pkgconfig \ ) ; \ (cd $DESTDIR/usr/local/lib && ln -sf librdkafka.so.1 librdkafka.so) make[1]: Leaving directory/home/ec2-user/bro/plugins/librdkafka-0.9.4/src'
make[1]: Entering directory /home/ec2-user/bro/plugins/librdkafka-0.9.4/src-cpp' Install librdkafka++ to /usr/local install -d $DESTDIR/usr/local/include/librdkafka ; \ install -d $DESTDIR/usr/local/lib ; \ install rdkafkacpp.h $DESTDIR/usr/local/include/librdkafka ; \ install librdkafka++.a $DESTDIR/usr/local/lib ; \ install librdkafka++.so.1 $DESTDIR/usr/local/lib ; \ [ -f "rdkafka++.pc" ] && ( \ install -d $DESTDIR/usr/local/lib/pkgconfig ; \ install -m 0644 rdkafka++.pc $DESTDIR/usr/local/lib/pkgconfig \ ) ; \ (cd $DESTDIR/usr/local/lib && ln -sf librdkafka++.so.1 librdkafka++.so) make[1]: Leaving directory/home/ec2-user/bro/plugins/librdkafka-0.9.4/src-cpp'

[rolandyoung]

Looking at the page you refer to, it seems you have already successfully installed librdafka and are now trying to follow the instructions to "compile this Bro plugin using the following commands".

The instructions you are following have omitted to tell you to change back to the directory into which you have downloaded the Bro plugin. (In fact, those instructions don't even seem to tell you where to download the plugin from, but that is another issue...) You are seeing the error you report because --bro-dist option is not an option understood by the librdkafka .configure script. It is almost certainly an option to the configuration script for the plugin whose instructions you are following..

If this information does not help you solve your problem, you need to refer to the support for the Bro project (https://www.bro.org/support/index.html); this is not an issue with the librdkafka software.