Confluent-kafka-go: panic: Invalid value "SASL_SSL" for configuration property "security.protocol"

[vanipolnedi]

Description

Hi,
When trying to connect to kafka consumer using golang, getting below error :
[ 2019_12_12_12:45:05 ] ----> Kafka Consumer Connection Error (kafka.Error=Invalid value "sasl_ssl" for configuration property "security.protocol")

Code:

   func main() {
c, err := kafka.NewConsumer(&kafka.ConfigMap{
    "bootstrap.servers": "localhost",
    "group.id":          "myGroup",
    "auto.offset.reset": "earliest",
    "sasl.mechanisms":   "PLAIN",
    "security.protocol": "SASL_SSL",
})
if err != nil {
    panic(err)
}
c.SubscribeTopics([]string{"myTopic", "^aRegex.*[Tt]opic"}, nil)
for {
    msg, err := c.ReadMessage(-1)
    if err == nil {
        fmt.Printf("Message on %s: %s\n", msg.TopicPartition, string(msg.Value))
    } else {
        // The client will automatically try to recover from all errors.
        fmt.Printf("Consumer error: %v (%v)\n", err, msg)
    }
}

c.Close()}

Checklist

Please provide the following information:

• [ ] confluent-kafka-go: confluent-kafka-go.v1 and librdkafka version (v1.3.0)
• [x] Apache Kafka version: kafka_2.11-2.0.0
• [ ] Client configuration: &kafka.ConfigMap{ "bootstrap.servers": "localhost", "group.id": "myGroup", "auto.offset.reset": "earliest", "sasl.mechanisms": "PLAIN", "security.protocol": "SASL_SSL", })
• [ ] Operating system: centos

[edenhill]

It seems like your librdkafka library was built without SSL support.

[vanipolnedi]

@edenhill followed below steps for installation:
git clone https://github.com/edenhill/librdkafka.git
cd librdkafka
./configure --prefix /usr
make
sudo make install

can u please suggest me how to build with SSL support??

[edenhill]

You need to check the output from configure to see if it detects OpenSSL or not.
On Centos you will need the openssl-devel package installed prior to configure

[vanipolnedi]

Thank you @edenhill .... previously i have installed librdkafka first and then openssl ... now uninstalled and installed librdkafka again ... its working fine

[vanipolnedi]

@edenhill i followed same steps in docker (first installed openssl and next librdkafka ) too but it is throwing same error .. can u please help me

[edenhill]

Please post the output "./configure .."

[vanipolnedi]

@edenhill
Step 11/24 : RUN openssl version
---> Running in 3d072e559a8f
OpenSSL 1.1.1d 10 Sep 2019
Removing intermediate container 3d072e559a8f
Step 12/24 : RUN git clone https://github.com/edenhill/librdkafka.git
---> Running in 7bcaef39b8d2
Cloning into 'librdkafka'...
Removing intermediate container 7bcaef39b8d2
---> 901352f3af66
Step 13/24 : WORKDIR /go/src/librdkafka
---> Running in a0689f772ae6
Removing intermediate container a0689f772ae6
---> 940499c89655
Step 14/24 : RUN ./configure --prefix /usr
---> Running in 36ceab70bdc8
checking for OS or distribution... ok (debian)
checking for C compiler from CC env... failed
checking for gcc (by command)... ok
checking for C++ compiler from CXX env... failed
checking for C++ compiler (g++)... ok
checking executable ld... ok
checking executable nm... ok
checking executable objdump... ok
checking executable strip... ok
checking for pkgconfig (by command)... ok
checking for install (by command)... ok
checking for PIC (by compile)... ok
checking for GNU-compatible linker options... ok
checking for GNU linker-script ld flag... ok
checking for __atomic_32 (by compile)... ok
checking for __atomic_64 (by compile)... ok
checking for socket (by compile)... ok
parsing version '0x010300ff'... ok (1.3.0)
checking for librt (by pkg-config)... failed
checking for librt (by compile)... ok
checking for libpthread (by pkg-config)... failed
checking for libpthread (by compile)... ok
checking for c11threads (by pkg-config)... failed
checking for c11threads (by compile)... ok
checking for libdl (by pkg-config)... failed
checking for libdl (by compile)... ok
checking for zlib (by pkg-config)... failed
checking for zlib (by compile)... failed (disable)
checking for libcrypto (by pkg-config)... failed
checking for libcrypto (by compile)... failed (disable)
checking for libsasl2 (by pkg-config)... failed
checking for libsasl2 (by compile)... failed (disable)
checking for libsasl (by pkg-config)... failed
checking for libsasl (by compile)... failed (disable)
checking for libzstd (by pkg-config)... failed
checking for libzstd (by compile)... failed (disable)
checking for libm (by pkg-config)... failed
checking for libm (by compile)... ok
checking for liblz4 (by pkg-config)... failed
checking for liblz4 (by compile)... failed (disable)
checking for rapidjson (by compile)... failed (disable)
checking for crc32chw (by compile)... ok
checking for regex (by compile)... ok
checking for strndup (by compile)... ok
checking for strlcpy (by compile)... failed (disable)
checking for strerror_r (by compile)... ok
checking for pthread_setname_gnu (by compile)... ok
checking for nm (by env NM)... ok (cached)
checking for python (by command)... ok
checking for getrusage (by compile)... ok
Generated Makefile.config
Generated config.h

Configuration summary:
prefix /usr
MKL_DISTRO debian
SOLIB_EXT .so
ARCH x86_64
CPU generic
GEN_PKG_CONFIG y
ENABLE_ZSTD y
ENABLE_SSL y
ENABLE_GSSAPI y
ENABLE_DEVEL n
ENABLE_VALGRIND n
ENABLE_REFCNT_DEBUG n
ENABLE_SHAREDPTR_DEBUG n
ENABLE_LZ4_EXT y
ENABLE_C11THREADS y
MKL_APP_NAME librdkafka
MKL_APP_DESC_ONELINE The Apache Kafka C/C++ library
CC gcc
CXX g++
LD ld
NM nm
OBJDUMP objdump
STRIP strip
CPPFLAGS -g -O2 -fPIC -Wall -Wsign-compare -Wfloat-equal -Wpointer-arith -Wcast-align
PKG_CONFIG pkg-config
INSTALL install
LIB_LDFLAGS -shared -Wl,-soname,$(LIBFILENAME)
LDFLAG_LINKERSCRIPT -Wl,--version-script=
RDKAFKA_VERSION_STR 1.3.0
MKL_APP_VERSION 1.3.0
LIBS -lm -ldl -lpthread -lrt -lpthread -lrt
CXXFLAGS -Wno-non-virtual-dtor
SYMDUMPER $(NM) -D
exec_prefix /usr
bindir /usr/bin
sbindir /usr/sbin
libexecdir /usr/libexec
datadir /usr/share
sysconfdir /usr/etc
sharedstatedir /usr/com
localstatedir /usr/var
runstatedir /usr/var/run
libdir /usr/lib
includedir /usr/include
infodir /usr/info
mandir /usr/man
BUILT_WITH GCC GXX PKGCONFIG INSTALL GNULD LDS C11THREADS LIBDL PLUGINS HDRHISTOGRAM SNAPPY SOCKEM CRC32C_HW
Generated config.cache

Now type 'make' to build
Removing intermediate container 36ceab70bdc8
---> 9f721f58e1f7

[edenhill]

checking for libcrypto (by pkg-config)... failed
checking for libcrypto (by compile)... failed (disable)

It fails to find openssl/libssl/libcrypto.

Make sure you install the openss-devel (libssl-dev on debian) package prior to running configure.

[vanipolnedi]

@edenhill thank you .. its working fine after installing libssl-dev

[deepnirmal]

This should work

sudo apt-get remove --auto-remove librdkafka-dev
sudo apt-get install libssl-dev
sudo curl -Lk -o /root/librdkafka-1.3.0.tar.gz https://github.com/edenhill/librdkafka/archive/v1.3.0.tar.gz &&       tar -xzf /root/librdkafka-1.3.0.tar.gz -C /root &&       cd /root/librdkafka-1.3.0 &&       ./configure --prefix /usr && make && make install && make clean && ./configure --clean