pulsar-client-cpp not able to build with static link

[vli02]

Expected behavior

Build pulsar-client-cpp lib with static linking all libs.

Set LINK_STATIC to ON in CMakeList.txt and run ./docker-build.sh, expect to build the cpp client lib with static linking.

Actual behavior

Build failed.

Steps to reproduce

In pulsar/pulsar-client-cpp/

$ git diff CMakeLists.txt
diff --git a/pulsar-client-cpp/CMakeLists.txt b/pulsar-client-cpp/CMakeLists.txt
index 38a47742..7fb08401 100644
--- a/pulsar-client-cpp/CMakeLists.txt
+++ b/pulsar-client-cpp/CMakeLists.txt
@@ -24,7 +24,7 @@ set(CMAKE_MODULE_PATH ${CMAKE_MODULE_PATH} "${CMAKE_SOURCE_DIR}/cmake_modules")
option(BUILD_TESTS "Build tests" ON)
MESSAGE(STATUS "BUILD_TESTS: " ${BUILD_TESTS})

-option(LINK_STATIC "Link against static libraries" OFF)
+option(LINK_STATIC "Link against static libraries" ON)
MESSAGE(STATUS "LINK_STATIC: " ${LINK_STATIC})

option(USE_LOG4CXX "Build with Log4cxx support" OFF)
$ ./docker-build.sh
...
[ 38%] Linking CXX shared library libpulsar.so
...
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/libboost_filesystem.a(operations.o): relocation R_X86_64_32 against `.rodata.str1.1' can not be used when making a shared object; recompile with -fPIC
/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/libboost_filesystem.a: error adding symbols: Bad value
collect2: error: ld returned 1 exit status
lib/CMakeFiles/pulsarShared.dir/build.make:2097: recipe for target 'lib/libpulsar.so.2.1.0-incubating-SNAPSHOT' failed
make[2]: * [lib/libpulsar.so.2.1.0-incubating-SNAPSHOT] Error 1
make[2]: Leaving directory '/pulsar/pulsar-client-cpp'
CMakeFiles/Makefile2:158: recipe for target 'lib/CMakeFiles/pulsarShared.dir/all' failed
make[1]: [lib/CMakeFiles/pulsarShared.dir/all] Error 2
make[1]: Leaving directory '/pulsar/pulsar-client-cpp'
Makefile:86: recipe for target 'all' failed
make: ** [all] Error 2

System configuration

Pulsar version: 2.0

[merlimat]

Hi @vli02. The problem is that to build a "fat" libpulsar.so that contains all dependencies, these dependencies need to be compiled with -fPIC .

Unfortunately, that is not the case with most library that ship in Ubuntu/RHEL.

For example, for Python binding we have a Docker image with all the dependencies compiled correctly.

Same for building statically linked RPM package (there is a PR in progress at https://github.com/apache/incubator-pulsar/pull/1963). The Docker image, based on Centos 7, is already available as apachepulsar/pulsar-build:centos-7. You can use that image, that comes with required dependencies, to build a statically linked libpulsar.so that doesn't depend on any other library.

[vli02]

Hi @merlimat I just had a quick try with apachepulsar/pulsar-build:centos-7, build failed early in cmake:

$ BUILD_IMAGE_VERSION=centos-7 ./docker-build.sh
---- Build Pulsar C++ client using image apachepulsar/pulsar-build:centos-7
centos-7: Pulling from apachepulsar/pulsar-build
...
Digest: sha256:233a5b70ae3546d14ff4605a09b74c91b6f6b3b1980fc14ed9e96c4760a62dad
Status: Downloaded newer image for apachepulsar/pulsar-build:centos-7
-- The C compiler identification is GNU 4.8.5
-- The CXX compiler identification is GNU 4.8.5
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- BUILD_TESTS:  ON
-- LINK_STATIC:  OFF
-- USE_LOG4CXX:  OFF
-- CMAKE_BUILD_TYPE:  RelWithDebInfo
-- Found ZLIB: /usr/local/lib/libz.so (found version "1.2.11")
-- Found PythonLibs: /usr/lib64/libpython2.7.so (found version "2.7.5")
-- PYTHON: 2.7.5
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - not found
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE
-- Boost version: 1.64.0
-- Found the following Boost libraries:
--   program_options
--   filesystem
--   regex
--   thread
--   system
--   python
-- Found OpenSSL: /usr/lib64/libcrypto.so (found version "1.0.2k")
clang-tidy not found
clang-format not found
CMake Error: The following variables are used in this project, but they are set to NOTFOUND.
Please set them or make sure they are set and tested correctly in the CMake files:
GMOCK_INCLUDE_PATH
   used as include directory in directory /pulsar/pulsar-client-cpp
...
GMOCK_LIBRARY_PATH
    linked by target "main" in directory /pulsar/pulsar-client-cpp/tests
GTEST_INCLUDE_PATH
   used as include directory in directory /pulsar/pulsar-client-cpp
...
LOG4CXX_LIBRARY_PATH
    linked by target "pulsarShared" in directory /pulsar/pulsar-client-cpp/lib
...
-- Configuring incomplete, errors occurred!
See also "/pulsar/pulsar-client-cpp/CMakeFiles/CMakeOutput.log".
See also "/pulsar/pulsar-client-cpp/CMakeFiles/CMakeError.log".

[merlimat]

@vli02 The Gmock and Gtest are not included in the Docker image. For the RPM I was building with the cmake . -DBUILD_TESTS=OFF so that they're not required.

For log4cxx, #1963 has a fix to remove the dependency when USE_LOG4CXX=OFF (which is the default). Will merge soon

[merlimat]

@vli02 the fix to compile in the centos docker image was merged earlier today

[vli02]

Thank you! @merlimat

[sijie]

@vli02 can you verify if the latest master fixes the problem?

[sijie]

This has been fixed.

[sekarpdkt]

I tried it in Ubuntu. Still fails.

[ 89%] Linking CXX shared library libpulsar.so
/usr/bin/ld: /usr/lib/x86_64-linux-gnu/libboost_regex.a(regex.o): relocation R_X86_64_PC32 against symbol `_ZTVN5boost9exceptionE' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: final link failed: Bad value
collect2: error: ld returned 1 exit status
lib/CMakeFiles/pulsarShared.dir/build.make:2370: recipe for target 'lib/libpulsar.so.2.4.0-SNAPSHOT' failed
make[2]: *** [lib/libpulsar.so.2.4.0-SNAPSHOT] Error 1
CMakeFiles/Makefile2:153: recipe for target 'lib/CMakeFiles/pulsarShared.dir/all' failed
make[1]: *** [lib/CMakeFiles/pulsarShared.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
[ 90%] Linking CXX static library libpulsar.a
[ 90%] Built target pulsarStatic
Makefile:129: recipe for target 'all' failed
make: *** [all] Error 2

[f7753]

CentOS 7 still has this issue

 $ cmake -DPYTHON_INCLUDE_DIR=/usr/include/python2.7 -DPYTHON_LIBRARY=/usr/lib/python2.7/config/libpython2.7.so -fPIC . -DBUILD_TESTS=OFF
-- BUILD_TESTS:  OFF
-- BUILD_PYTHON_WRAPPER:  ON
-- LINK_STATIC:  OFF
-- USE_LOG4CXX:  OFF
-- CMAKE_BUILD_TYPE:  RelWithDebInfo
-- Boost version: 1.53.0
-- Found the following Boost libraries:
--   program_options
--   regex
--   system
-- PYTHON:
-- Boost version: 1.53.0
-- Found the following Boost libraries:
--   python
-- HAS_ZSTD: 0
CMake Error at lib/CMakeLists.txt:41 (set_target_properties):
  set_target_properties called with incorrect number of arguments.


CMake Error at lib/CMakeLists.txt:45 (set_target_properties):
  set_target_properties called with incorrect number of arguments.


-- Using Boost Python libs: /usr/lib64/libboost_python-mt.so
clang-tidy not found
clang-format not found
CMake Error: The following variables are used in this project, but they are set to NOTFOUND.
Please set them or make sure they are set and tested correctly in the CMake files:
GMOCK_INCLUDE_PATH

[wolfstudy]

@f7753 You can update clang-format version, because in CMakeLists.txt:

# `make format` option
if (NOT APPLE AND NOT WIN32)
    set(CLANG_FORMAT_VERSION "5.0")
endif()

Or, we can ignore make check-format. In centos7 OS, the default version of clang is 3.4.2

---

Then in apachepulsar/pulsar-build:centos-7 image, when i exec make -j12, error as follows:

[root@1fc7007643de pulsar-client-cpp]# make -j12
[ 86%] Built target pulsarShared
[ 86%] Built target pulsarStatic
[ 87%] Built target SampleConsumerListenerCApi
[ 91%] Built target SampleConsumerCApi
[ 91%] Built target SampleProducer
[ 89%] Built target SampleProducerCApi
[ 91%] Built target SampleConsumerListener
[ 93%] Built target SampleConsumer
[ 89%] Built target SampleAsyncProducer
[ 94%] Built target perfProducer
[ 93%] Built target SampleReaderCApi
[ 95%] Built target perfConsumer
[ 95%] Linking CXX shared library _pulsar.so
/usr/bin/ld: cannot find -lstdc++
collect2: error: ld returned 1 exit status
make[2]: *** [python/_pulsar.so] Error 1
make[1]: *** [python/CMakeFiles/_pulsar.dir/all] Error 2
make: *** [all] Error 2

I tried to exec yum install -y libstdc++-static.x86_64, this problem was fixed.

[f7753]

After pulled the latest docker images and the source code of pulsar-client-cpp, as @wolfstudy refered above, it works well for me now. Thanks

[f7753]

BTW, when I compile these code in docker environment, I use the default CPU resource settings， and then I compile with the param -j12, it would throw the error below:

c++: internal compiler error: Killed (program cc1plus)
Please submit a full bug report,
with preprocessed source if appropriate.
See <http://bugzilla.redhat.com/bugzilla> for instructions.
make[2]: *** [lib/CMakeFiles/pulsarShared.dir/BatchMessageContainer.cc.o] Error 4
make[2]: *** Waiting for unfinished jobs....

So, if you see the log above, please make sure that your compile params match the docker resource settings.