Cudf: [FEA] Compiling the libs on CentOS (with or without Conda's help)

[MasoodK]

Is your feature request related to a problem? Please describe.

Building libcudf is not possible with standard setups on CentOS; current errors:

• gcc (GCC) 7.3.1 20180303 (Red Hat 7.3.1-5)
  Building libcudf: make -j
  Error: libNVStrings.so: undefined reference to `std::runtime_error::runtime_error(char const*)@GLIBCXX_3.4.21'

• gcc (GCC) 8.2.1 20180905 (Red Hat 8.2.1-3)
  Building libcudf: cmake .. -DCMAKE_INSTALL_PREFIX=$CONDA_PREFIX -DCMAKE_CXX11_ABI=OFF
  Error: #error -- unsupported GNU version! gcc versions later than 7 are not supported!

gcc has been upgraded in the above cases using:
yum install centos-release-scl
yum install devtoolset-7-gcc*
scl enable devtoolset-7 bash

Describe the solution you'd like
Detailed instructions regarding how to compile on CentOS (which may or may not involve Conda libs, cf. #688)

Describe alternatives you've considered
NA

Additional context
My env is described in #796; this issue is created following the discussions there.

[jrhemstad]

Thanks for creating the new issue @MasoodK.

[awthomp]

I was able to overcome this issue by pulling the updated libstdc++ library from Anaconda and using it to replace the one that comes natively with devtoolset-X. I used devtoolset-6 rather than 7, but the process should be the same.

cp $CONDA_PREFIX/lib/libstdc++.so.6.0.25 /usr/lib64
rm /usr/lib64/libstdc++.so.6
ln -s /usr/lib64/libstdc++.so.6.0.25 /usr/lib64/libstdc++.so.6

[MasoodK]

Thanks @awthomp! It did not work for me, however. Here is the process and logs:

Installing gcc v6:
yum install centos-release-scl
yum install devtoolset-6-gcc*
scl enable devtoolset-6 bash

[root@ip-10-0-0-11 cpp]# which gcc
/opt/rh/devtoolset-6/root/usr/bin/gcc

[root@ip-10-0-0-11 cpp]# gcc --version
gcc (GCC) 6.3.1 20170216 (Red Hat 6.3.1-3)

Find and replace the "libstdc++.so" lib used:
Find the lib:
cd /opt/rh/devtoolset-6
find . -name "libstdc++.so*"
Found:
/opt/rh/devtoolset-6/root/usr/lib/gcc/x86_64-redhat-linux/6.3.1/libstdc++.so
(There is also the 32-bit version here which was left alone: gcc/x86_64-redhat-linux/6.3.1/32/libstdc++.so)

Replace the libstdc++.so lib used

• mv /opt/rh/devtoolset-6/root/usr/lib/gcc/x86_64-redhat-linux/6.3.1/libstdc++.so /opt/rh/devtoolset-6/root/usr/lib/gcc/x86_64-redhat-linux/6.3.1/libstdc++.so.bak
• cp $CONDA_PREFIX/lib/libstdc++.so.6.0.25 /opt/rh/devtoolset-6/root/usr/lib/gcc/x86_64-redhat-linux/6.3.1/libstdc++.so
  (libstdc++.so.6.0.25 is present there)

cmake .. -DCMAKE_INSTALL_PREFIX=$CONDA_PREFIX -DCMAKE_CXX11_ABI=OFF
Error:

gmake[3]: *** [Makefile:144: all] Error 2
gmake[2]: *** [CMakeFiles/Arrow.dir/build.make:111: Arrow-prefix/src/Arrow-stamp/Arrow-build] Error 2
gmake[1]: *** [CMakeFiles/Makefile2:73: CMakeFiles/Arrow.dir/all] Error 2
gmake: *** [Makefile:84: all] Error 2
CMake Error at cmake/Modules/ConfigureArrow.cmake:67 (message):
  Building Arrow failed: 2
Call Stack (most recent call first):
CMakeLists.txt:91 (include)

Output/error: https://gist.github.com/MasoodK/885953ee60e358d7ad12ee8cbb61454f
CMakeOutput.log: https://gist.github.com/MasoodK/0be345a8cec91e7500fa798a44b9bce9

Let me know if you have some thoughts on this. Thanks.
PS: I haven't dug into what it's complaining about.

[awthomp]

Here's what I did. As a word of warning, I was using a CentOS 7 Docker image as a basis for this work, so your paths and what not will probably vary.

#Centos Updates -- Assuming base CentOS 7 image
yum update -y
yum install -y centos-release-scl-rh
yum-config-manager --enable rhel-server-rhscl-7-rpms
yum install -y devtoolset-6 \
    bzip2 \
    git \
    zlib-devel \
    wget
scl enable devtoolset-6 bash

# Download Anaconda and install
wget https://repo.continuum.io/archive/Anaconda3-2018.12-Linux-x86_64.sh
chmod +x Anaconda3-2018.12-Linux-x86_64.sh
./Anaconda3-2018.12-Linux-x86_64.sh

# Set environment variables
export PATH=/root/anaconda3/bin:$PATH
export CONDA_PREFIX=/root/anaconda3

# cuDF
export CUDF_HOME=/root/home/cudf
git clone --recurse-submodules https://github.com/rapidsai/cudf.git $CUDF_HOME
cd $CUDF_HOME
conda env create --name cudf_dev --file conda/environments/cudf_dev.yml
source activate cudf_dev

# Fix libstdc++ shared object by pulling from Anaconda
cp /root/anaconda3/lib/libstdc++.so.6.0.25 /usr/lib64
rm /usr/lib64/libstdc++.so.6
ln -s /usr/lib64/libstdc++.so.6.0.25 /usr/lib64/libstdc++.so.6

# Install cuDF according to documentation
cd $CUDF_HOME/cpp
mkdir build
cd build
cmake .. -DCMAKE_INSTALL_PREFIX=$CONDA_PREFIX/envs/cudf_dev
export LD_LIBRARY_PATH=$CONDA_PREFIX/envs/cudf_dev/lib:$LD_LIBRARY_PATH
export LIBRARY_PATH=$CONDA_PREFIX/envs/cudf_dev/lib:$LIBRARY_PATH
make -j
make install

make python_cffi
make install_python
cd $CUDF_HOME/python
python setup.py build_ext --inplace

export CUDA_HOME=/usr/local/cuda
export NUMBAPRO_NVVM=$CUDA_HOME/nvvm/lib64/libnvvm.so
export NUMBAPRO_LIBDEVICE=$CUDA_HOME/nvvm/libdevice

python setup.py install

[MasoodK]

Thank you @awthomp for taking the time to document your steps! I'll try to give this another go sometime later.

[mike-wendt]

@MasoodK we have CentOS containers released and @rlratzel was able to get this to work in his build scripts that are posted here github.com/rapidsai/build

Are you still having issues or can we close this issue?

[MasoodK]

Thanks for the update @mike-wendt ! You may close this issue if someone has confirmed that this works now on CentOS. I personally had to move on; I got lucky as I could install/run the v.0.5 version binary from PyPI on CentOS and it's working fine for now. At some point, I may to need to compile/build again; I'll check out the pointers then. Thanks!

[mike-wendt]

Ok @MasoodK if you have future issues feel free to file a new issue and ask for @rlratzel who has done builds on CentOS for our team.