Rocksdb: Rocksdb not link against libdl

[huydx]

Note: Please use Issues only for bug reports. For questions, discussions, feature requests, etc. post to dev group: https://www.facebook.com/groups/rocksdb.dev

Expected behavior

Build shared lib with make shared_lib
Generated librocksdb.so file should link with libdl.
Not link against libdl will make build with app fail with .so file when it try to use dlopen method.

Actual behavior

It's not linked with libdl

Steps to reproduce the behavior

I'm using centos:latest docker file

Install latest dependencies using

yum install -y cmake make gcc git wget snappy snappy-devel zlib zlib-devel bzip2 bzip2-devel lz4-devel make which gcc-g++

Install shared lib using

make -j8 shared_lib

Generated so file not linked with libdl

[root@38af411d27af rocksdb]# ldd librocksdb.so
    linux-vdso.so.1 =>  (0x00007ffe15be1000)
    libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f082bb3c000)
    librt.so.1 => /lib64/librt.so.1 (0x00007f082b934000)
    libsnappy.so.1 => /lib64/libsnappy.so.1 (0x00007f082b72d000)
    libz.so.1 => /lib64/libz.so.1 (0x00007f082b517000)
    libbz2.so.1 => /lib64/libbz2.so.1 (0x00007f082b307000)
    liblz4.so.1 => /lib64/liblz4.so.1 (0x00007f082b0f1000)
    libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007f082adea000)
    libm.so.6 => /lib64/libm.so.6 (0x00007f082aae8000)
    libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f082a8d1000)
    libc.so.6 => /lib64/libc.so.6 (0x00007f082a504000)
    /lib64/ld-linux-x86-64.so.2 (0x00005614cfe1d000)

May be I'm missing something or it's expected behaviour?

[sagar0]

Wondering if there's a need to install any special package for getting libdl?
@mrambacher since you introduced support for loading dyanamic libraries into RocksDB, any chance you know about this?

[mrambacher]

What operating system and compiler are you using? We did not have to
install any special packages here, but I can look into it for you if I know
what I am trying to find.

On Tue, Jul 2, 2019 at 3:23 PM Sagar Vemuri [e-mail]
wrote:

Wondering if there's a need to install any special package for getting
libdl?
@mrambacher https://github.com/mrambacher since you introduced support
for loading dyanamic libraries into RocksDB, any chance you know about this?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/facebook/rocksdb/issues/5530?email_source=notifications&email_token=AABCTBFOIAHCGBCNTFIW5VTP5OTMRA5CNFSM4H4Z2XIKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZCJPQY#issuecomment-507811779,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AABCTBCYLSKXYDLTOOXOZQDP5OTMRANCNFSM4H4Z2XIA
.

[huydx]

@mrambacher I'm using gcc-c++ default package of centos 7. (which version at 4.8 I think) (c++ (GCC) 4.8.5 20150623 (Red Hat 4.8.5-36)). It could be easy reproduced inside container with Dockerfile looks like

FROM centos:latest
RUN yum -y update && yum install -y cmake make gcc git wget snappy snappy-devel zlib zlib-devel bzip2 bzip2-devel lz4-devel make which gcc-g++

As far as I could tell after take a simple look at Makefile, seems that ldl only be linked when we include tbb package?

[huydx]

gentle ping @mrambacher , any update for the issue?

[mrambacher]

The yum command you sent me does not appear to work. Yum could not install gcc-g++ for me and I had no g++ compiler and could not build anything Rocks-related.

When I changed it to "yum install gcc-c++", everything builds and runs fine. I also have a libdl.so on my machine and the env_test passes:

ldd env_test
linux-vdso.so.1 => (0x00007fff479ad000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007fa384492000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fa384276000)
librt.so.1 => /lib64/librt.so.1 (0x00007fa38406e000)
libsnappy.so.1 => /lib64/libsnappy.so.1 (0x00007fa383e68000)
libz.so.1 => /lib64/libz.so.1 (0x00007fa383c52000)
libbz2.so.1 => /lib64/libbz2.so.1 (0x00007fa383a42000)
liblz4.so.1 => /lib64/liblz4.so.1 (0x00007fa38382d000)
libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007fa383526000)
libm.so.6 => /lib64/libm.so.6 (0x00007fa383224000)
libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007fa38300e000)
libc.so.6 => /lib64/libc.so.6 (0x00007fa382c41000)
/lib64/ld-linux-x86-64.so.2 (0x00007fa384696000)

Does that solution work for you?

[riversand963]

This is what I got on my machine.

$ldd librocksdb.so
        linux-vdso.so.1 =>  (0x00007ffc0b3c3000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fc73eaba000)
        librt.so.1 => /lib64/librt.so.1 (0x00007fc73e8b2000)
        libz.so.1 => /lib64/libz.so.1 (0x00007fc73e69c000)
        libzstd.so.1 => /lib64/libzstd.so.1 (0x00007fc73e3ff000)
        libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007fc73e0f8000)
        libm.so.6 => /lib64/libm.so.6 (0x00007fc73ddf6000)
        libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007fc73dbe0000)
        libc.so.6 => /lib64/libc.so.6 (0x00007fc73d813000)
        /lib64/ld-linux-x86-64.so.2 (0x00007fc73f43b000)
$ldd env_test
        linux-vdso.so.1 =>  (0x00007ffec95b8000)
        libdl.so.2 => /lib64/libdl.so.2 (0x00007f6826e90000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f6826c74000)
        librt.so.1 => /lib64/librt.so.1 (0x00007f6826a6c000)
        libz.so.1 => /lib64/libz.so.1 (0x00007f6826856000)
        libzstd.so.1 => /lib64/libzstd.so.1 (0x00007f68265b9000)
        libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007f68262b2000)
        libm.so.6 => /lib64/libm.so.6 (0x00007f6825fb0000)
        libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f6825d9a000)
        libc.so.6 => /lib64/libc.so.6 (0x00007f68259cd000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f6827094000)

[dkrutsko]

We solved it by adding -ldl to our LDFLAGS. The real question is why it's all of a sudden needed? is shared_lib no longer working? or are they using dlopen for other features, like loading plugins.

[mrambacher]

I am working on adding loadable plugins to RocksDB and introduced the ability to do dlopen to add those features. I plan to write up a Wiki on the feature in the next week or so.

[huydx]

thanks for supporting @mrambacher. quick update on our side: currently we did some work around (include override make file) so the problem is not very critical for us now

[ajkr]

Sorry I didn't see a conclusion. Should librocksdb.so be built with -ldl now? I just tried compiling something against a freshly built librocksdb.so and got the following errors:

$ g++ -std=c++11 -I./include/ -L./ ./tmp.cpp -lrocksdb
.//librocksdb.so: undefined reference to `dlopen'
.//librocksdb.so: undefined reference to `dlclose'
.//librocksdb.so: undefined reference to `dlerror'
.//librocksdb.so: undefined reference to `dlsym'

[mrambacher]

Fixed by #6660