Librdkafka: Issues compiling the library on MacOS High Sierra

[vvraskin]

Read the FAQ first: https://github.com/edenhill/librdkafka/wiki/FAQ

Description

It seems that there is a conflict between the openssl version that I have and the headers that are required for the compilation.
When I make the latest master version I get the following error (sorry for the long output):

Creating shared library librdkafka.1.dylib
gcc  -shared -dynamiclib -Wl,-install_name,/usr/local/lib/librdkafka.1.dylib rdkafka.o rdkafka_broker.o rdkafka_msg.o rdkafka_topic.o rdkafka_conf.o rdkafka_timer.o rdkafka_offset.o rdkafka_transport.o rdkafka_buf.o rdkafka_queue.o rdkafka_op.o rdkafka_request.o rdkafka_cgrp.o rdkafka_pattern.o rdkafka_partition.o rdkafka_subscription.o rdkafka_assignor.o rdkafka_range_assignor.o rdkafka_roundrobin_assignor.o rdkafka_feature.o rdcrc32.o crc32c.o rdmurmur2.o rdaddr.o rdrand.o rdlist.o tinycthread.o rdlog.o rdstring.o rdkafka_event.o rdkafka_metadata.o rdregex.o rdports.o rdkafka_metadata_cache.o rdavl.o rdkafka_sasl.o rdkafka_sasl_plain.o rdkafka_interceptor.o rdkafka_msgset_writer.o rdkafka_msgset_reader.o rdkafka_header.o rdvarint.o rdbuf.o rdunittest.o rdkafka_sasl_cyrus.o rdkafka_sasl_scram.o snappy.o rdgz.o rdkafka_lz4.o xxhash.o rddl.o rdkafka_plugin.o -o librdkafka.1.dylib -lpthread -lz -L/usr/local/Cellar/lz4/1.8.1.2/lib -llz4 -L/usr/local/Cellar/openssl/1.0.2n/lib -lssl -lsasl2 -ldl
Undefined symbols for architecture x86_64:
  "_BIO_ctrl", referenced from:
      _rd_base64_encode in rdkafka_sasl_scram.o
  "_BIO_f_base64", referenced from:
      _rd_kafka_sasl_scram_fsm in rdkafka_sasl_scram.o
      _rd_base64_encode in rdkafka_sasl_scram.o
  "_BIO_free_all", referenced from:
      _rd_kafka_sasl_scram_fsm in rdkafka_sasl_scram.o
      _rd_base64_encode in rdkafka_sasl_scram.o
  "_BIO_new", referenced from:
      _rd_kafka_sasl_scram_fsm in rdkafka_sasl_scram.o
      _rd_base64_encode in rdkafka_sasl_scram.o
  "_BIO_new_mem_buf", referenced from:
      _rd_kafka_sasl_scram_fsm in rdkafka_sasl_scram.o
  "_BIO_push", referenced from:
      _rd_kafka_sasl_scram_fsm in rdkafka_sasl_scram.o
      _rd_base64_encode in rdkafka_sasl_scram.o
  "_BIO_read", referenced from:
      _rd_kafka_sasl_scram_fsm in rdkafka_sasl_scram.o
  "_BIO_s_mem", referenced from:
      _rd_base64_encode in rdkafka_sasl_scram.o
  "_BIO_set_flags", referenced from:
      _rd_kafka_sasl_scram_fsm in rdkafka_sasl_scram.o
      _rd_base64_encode in rdkafka_sasl_scram.o
  "_BIO_write", referenced from:
      _rd_base64_encode in rdkafka_sasl_scram.o
  "_CRYPTO_cleanup_all_ex_data", referenced from:
      _rd_kafka_transport_ssl_term in rdkafka_transport.o
  "_CRYPTO_num_locks", referenced from:
      _rd_kafka_transport_ssl_init in rdkafka_transport.o
  "_CRYPTO_set_id_callback", referenced from:
      _rd_kafka_transport_ssl_term in rdkafka_transport.o
      _rd_kafka_transport_ssl_init in rdkafka_transport.o
  "_CRYPTO_set_locking_callback", referenced from:
      _rd_kafka_transport_ssl_term in rdkafka_transport.o
      _rd_kafka_transport_ssl_init in rdkafka_transport.o
  "_ERR_error_string_n", referenced from:
      _rd_kafka_ssl_error in rdkafka_transport.o
  "_ERR_get_error_line_data", referenced from:
      _rd_kafka_ssl_error in rdkafka_transport.o
  "_ERR_remove_thread_state", referenced from:
      _rd_kafka_broker_thread_main in rdkafka_broker.o
  "_EVP_PKEY_free", referenced from:
      _rd_kafka_transport_ssl_ctx_init in rdkafka_transport.o
  "_EVP_PKEY_new", referenced from:
      _rd_kafka_transport_ssl_ctx_init in rdkafka_transport.o
  "_EVP_sha1", referenced from:
      _rd_kafka_sasl_scram_conf_validate in rdkafka_sasl_scram.o
  "_EVP_sha256", referenced from:
      _rd_kafka_sasl_scram_conf_validate in rdkafka_sasl_scram.o
  "_EVP_sha512", referenced from:
      _rd_kafka_sasl_scram_conf_validate in rdkafka_sasl_scram.o
  "_HMAC", referenced from:
      _rd_kafka_sasl_scram_Hi in rdkafka_sasl_scram.o
      _rd_kafka_sasl_scram_HMAC in rdkafka_sasl_scram.o
  "_OPENSSL_add_all_algorithms_noconf", referenced from:
      _rd_kafka_transport_ssl_init in rdkafka_transport.o
  "_PKCS12_free", referenced from:
      _rd_kafka_transport_ssl_ctx_init in rdkafka_transport.o
  "_PKCS12_parse", referenced from:
      _rd_kafka_transport_ssl_ctx_init in rdkafka_transport.o
  "_SHA1", referenced from:
      _rd_kafka_sasl_scram_conf_validate in rdkafka_sasl_scram.o
  "_SHA256", referenced from:
      _rd_kafka_sasl_scram_conf_validate in rdkafka_sasl_scram.o
  "_SHA512", referenced from:
      _rd_kafka_sasl_scram_conf_validate in rdkafka_sasl_scram.o
  "_X509_STORE_set_flags", referenced from:
      _rd_kafka_transport_ssl_ctx_init in rdkafka_transport.o
  "_X509_free", referenced from:
      _rd_kafka_transport_ssl_ctx_init in rdkafka_transport.o
      _rd_kafka_transport_io_serve in rdkafka_transport.o
  "_X509_new", referenced from:
      _rd_kafka_transport_ssl_ctx_init in rdkafka_transport.o
  "_X509_verify_cert_error_string", referenced from:
      _rd_kafka_transport_io_serve in rdkafka_transport.o
  "_d2i_PKCS12_fp", referenced from:
      _rd_kafka_transport_ssl_ctx_init in rdkafka_transport.o
  "_sk_pop_free", referenced from:
      _rd_kafka_transport_ssl_ctx_init in rdkafka_transport.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[1]: *** [librdkafka.1.dylib] Error 1
make: *** [libs] Error 2

Has someone seen this error before?

Thank you.

Checklist

• [x] librdkafka version (release number or git tag): v0.11.3
• [x] Operating system: MacOS High Sierra 10.13.3

[edenhill]

Install openssl from homebrew and rerun:

$ ./configure --clean
$ ./configure
$ make

[vvraskin]

Unfortunately reinstalling openssl did not help.
The problem seems to be related to the conflict of openssl versions (macos native and the one installed by brew). Specifying the binaries and headers explicitly by exporting the env vars unravels the mystery:

export LDFLAGS=-L/usr/local/opt/openssl/lib
export CPPFLAGS=-I/usr/local/opt/openssl/include