Node-rdkafka: linker command failed with exit code 1 (use -v to see invocation) with v2.3.3

[raghu64]

Failed to install node-rdkafka v2.3.3 with the following error

Environment Information

• OS [e.g. Mac, Arch, Windows 10]: Mac
• Node Version [e.g. 8.2.1]: 8.9.4
• NPM Version [e.g. 5.4.2]: 5.6.0
• C++ Toolchain [e.g. Visual Studio, llvm, g++]: g++
• node-rdkafka version [e.g. 2.3.3]: 2.3.3

Steps to Reproduce
npm install -S node-rdkakfa

Additional context

Creating shared library librdkafka.1.dylib
gcc -L/Users/raghu/martel-marketo2/node_modules/node-rdkafka/build/deps -shared -dynamiclib -Wl,-install_name,/Users/raghu/martel-marketo2/node_modules/node-rdkafka/build/deps/librdkafka.1.dylib rdkafka.o rdkafka_broker.o rdkafka_msg.o rdkafka_topic.o rdkafka_conf.o rdkafka_timer.o rdkafka_offset.o rdkafka_transport.o rdkafka_buf.o rdkafka_queue.o rdkafka_op.o rdkafka_request.o rdkafka_cgrp.o rdkafka_pattern.o rdkafka_partition.o rdkafka_subscription.o rdkafka_assignor.o rdkafka_range_assignor.o rdkafka_roundrobin_assignor.o rdkafka_feature.o rdcrc32.o crc32c.o rdmurmur2.o rdaddr.o rdrand.o rdlist.o tinycthread.o rdlog.o rdstring.o rdkafka_event.o rdkafka_metadata.o rdregex.o rdports.o rdkafka_metadata_cache.o rdavl.o rdkafka_sasl.o rdkafka_sasl_plain.o rdkafka_interceptor.o rdkafka_msgset_writer.o rdkafka_msgset_reader.o rdkafka_header.o rdvarint.o rdbuf.o rdunittest.o rdkafka_sasl_cyrus.o rdkafka_sasl_scram.o snappy.o rdgz.o rdkafka_lz4.o xxhash.o lz4.o lz4frame.o lz4hc.o rddl.o rdkafka_plugin.o -o librdkafka.1.dylib -lsasl2 -L/usr/local/Cellar/openssl/1.0.2o_1/lib -lssl  -lz -ldl -lpthread
Undefined symbols for architecture x86_64:
  "_BIO_ctrl", referenced from:
      _rd_base64_encode in rdkafka_sasl_scram.o
  "_BIO_f_base64", referenced from:
      _rd_kafka_sasl_scram_fsm in rdkafka_sasl_scram.o
      _rd_base64_encode in rdkafka_sasl_scram.o
  "_BIO_free_all", referenced from:
      _rd_kafka_sasl_scram_fsm in rdkafka_sasl_scram.o
      _rd_base64_encode in rdkafka_sasl_scram.o
  "_BIO_new", referenced from:
      _rd_kafka_sasl_scram_fsm in rdkafka_sasl_scram.o
      _rd_base64_encode in rdkafka_sasl_scram.o
  "_BIO_new_mem_buf", referenced from:
      _rd_kafka_sasl_scram_fsm in rdkafka_sasl_scram.o
  "_BIO_push", referenced from:
      _rd_kafka_sasl_scram_fsm in rdkafka_sasl_scram.o
      _rd_base64_encode in rdkafka_sasl_scram.o
  "_BIO_read", referenced from:
      _rd_kafka_sasl_scram_fsm in rdkafka_sasl_scram.o
  "_BIO_s_mem", referenced from:
      _rd_base64_encode in rdkafka_sasl_scram.o
  "_BIO_set_flags", referenced from:
      _rd_kafka_sasl_scram_fsm in rdkafka_sasl_scram.o
      _rd_base64_encode in rdkafka_sasl_scram.o
  "_BIO_write", referenced from:
      _rd_base64_encode in rdkafka_sasl_scram.o
  "_CRYPTO_cleanup_all_ex_data", referenced from:
      _rd_kafka_transport_ssl_term in rdkafka_transport.o
  "_CRYPTO_num_locks", referenced from:
      _rd_kafka_transport_ssl_init in rdkafka_transport.o
  "_CRYPTO_set_id_callback", referenced from:
      _rd_kafka_transport_ssl_term in rdkafka_transport.o
      _rd_kafka_transport_ssl_init in rdkafka_transport.o
  "_CRYPTO_set_locking_callback", referenced from:
      _rd_kafka_transport_ssl_term in rdkafka_transport.o
      _rd_kafka_transport_ssl_init in rdkafka_transport.o
  "_ERR_error_string_n", referenced from:
      _rd_kafka_ssl_error in rdkafka_transport.o
  "_ERR_get_error_line_data", referenced from:
      _rd_kafka_ssl_error in rdkafka_transport.o
  "_ERR_remove_thread_state", referenced from:
      _rd_kafka_broker_thread_main in rdkafka_broker.o
  "_EVP_PKEY_free", referenced from:
      _rd_kafka_transport_ssl_ctx_init in rdkafka_transport.o
  "_EVP_PKEY_new", referenced from:
      _rd_kafka_transport_ssl_ctx_init in rdkafka_transport.o
  "_EVP_sha1", referenced from:
      _rd_kafka_sasl_scram_conf_validate in rdkafka_sasl_scram.o
  "_EVP_sha256", referenced from:
      _rd_kafka_sasl_scram_conf_validate in rdkafka_sasl_scram.o
  "_EVP_sha512", referenced from:
      _rd_kafka_sasl_scram_conf_validate in rdkafka_sasl_scram.o
  "_HMAC", referenced from:
      _rd_kafka_sasl_scram_Hi in rdkafka_sasl_scram.o
      _rd_kafka_sasl_scram_HMAC in rdkafka_sasl_scram.o
  "_OPENSSL_add_all_algorithms_noconf", referenced from:
      _rd_kafka_transport_ssl_init in rdkafka_transport.o
  "_PKCS12_free", referenced from:
      _rd_kafka_transport_ssl_ctx_init in rdkafka_transport.o
  "_PKCS12_parse", referenced from:
      _rd_kafka_transport_ssl_ctx_init in rdkafka_transport.o
  "_SHA1", referenced from:
      _rd_kafka_sasl_scram_conf_validate in rdkafka_sasl_scram.o
  "_SHA256", referenced from:
      _rd_kafka_sasl_scram_conf_validate in rdkafka_sasl_scram.o
  "_SHA512", referenced from:
      _rd_kafka_sasl_scram_conf_validate in rdkafka_sasl_scram.o
  "_X509_STORE_set_flags", referenced from:
      _rd_kafka_transport_ssl_ctx_init in rdkafka_transport.o
  "_X509_free", referenced from:
      _rd_kafka_transport_ssl_ctx_init in rdkafka_transport.o
      _rd_kafka_transport_io_serve in rdkafka_transport.o
  "_X509_new", referenced from:
      _rd_kafka_transport_ssl_ctx_init in rdkafka_transport.o
  "_X509_verify_cert_error_string", referenced from:
      _rd_kafka_transport_io_serve in rdkafka_transport.o
  "_d2i_PKCS12_fp", referenced from:
      _rd_kafka_transport_ssl_ctx_init in rdkafka_transport.o
  "_sk_pop_free", referenced from:
      _rd_kafka_transport_ssl_ctx_init in rdkafka_transport.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [librdkafka.1.dylib] Error 1
make[1]: *** [libs] Error 2
make: *** [_Users_raghu_martel_marketo2_node_modules_node_rdkafka_deps_librdkafka_gyp_librdkafka_target_build_dependencies.intermediate] Error 2
rm _Users_raghu_martel_marketo2_node_modules_node_rdkafka_deps_librdkafka_gyp_librdkafka_target_build_dependencies.intermediate
gyp ERR! build error 
gyp ERR! stack Error: `make` failed with exit code: 2
gyp ERR! stack     at ChildProcess.onExit (/Users/raghu/.nvm/versions/node/v8.9.4/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:258:23)
gyp ERR! stack     at emitTwo (events.js:126:13)
gyp ERR! stack     at ChildProcess.emit (events.js:214:7)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:198:12)
gyp ERR! System Darwin 17.5.0
gyp ERR! command "/Users/raghu/.nvm/versions/node/v8.9.4/bin/node" "/Users/raghu/.nvm/versions/node/v8.9.4/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
gyp ERR! cwd /Users/raghu/martel-marketo2/node_modules/node-rdkafka
gyp ERR! node -v v8.9.4
gyp ERR! node-gyp -v v3.6.2
gyp ERR! not ok 
npm WARN [email protected] No license field.

[caged]

I ran into a similar error on node 9.5.0 using npm 6.0.1. I have openssl installed through homebrew and was able to successfully install using:

CFLAGS=-I/usr/local/opt/openssl/include LDFLAGS=-L/usr/local/opt/openssl/lib npm install --save node-rdkafka

[ankon]

Looks like a duplicate of #373

[battlecow]

I will confirm that @Caged 's solution works for Mac with node v10.1.0 and npm 6.1.0, without the extra build flags it won't compile.

[sugarandmagic]

Worked for me with node v8.9.4 and npm 5.6.0 - thank you @Caged for saving me a lot of legwork

[jpwilliams]

@Caged's suggestion worked for me too.

Why does this cause it to work, @Caged? What's missing from the current build script?

[FuzzOli87]

Same bug on node 10.3.0 and NPM 6.1.0

Using the flags that @Caged mentioned solved it.

[webmakersteve]

Updated the README. Re-open the issue if the problem persists after the workaround.

[zhengdai]

CFLAGS=-I/usr/local/opt/openssl/include LDFLAGS=-L/usr/local/opt/openssl/lib npm install --save node-rdkafka

works for me on mac big sur