conan install should fix the runpath of installed dependencies

Created on 22 Mar 2018 · 9Comments · Source: conan-io/conan

Lately, I have tried to build and use dynamic shared object for my project and its dependencies.

I have reproduce my layout project in this simple more simpler example https://github.com/AlexandreBossard/conan-deptest

The build.sh build and launch the app.bin.
The dependency is as follow:
app.bin -> libbd -> libconnector

After running the according conan create/install and such, the built app.bin fails to launch out of the box. Which is sad, because to me, it was the whole point of using conan!

On linux and mac, it fails for relatively the same reason, (haven't tried windows):

on linux, libdb.so cannot find the libconnector.so
on osx, app.bin cannot find the the libdb.dynlib

The whole problem seems to be how, conan (or cmake) handle the runpath flag. The runpath of shared objects in the conan cache is not set.

$ readelf -d .conan/data/connector/0.0.1/alex/dev/package/160b4bac5177b0f9a5f4857d00317fe0862a8a02/lib/libconnector.so 

Dynamic section at offset 0xde8 contains 26 entries:
  Tag        Type                         Name/Value
 0x0000000000000001 (NEEDED)             Shared library: [libstdc++.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]
 0x000000000000000e (SONAME)             Library soname: [libconnector.so]
#[...]

$ readelf -d .conan/data/db/0.0.3/alex/dev/package/991309e0b1e72d26aa0af568021a9c64f8f207d6/lib/libdb.so 

Dynamic section at offset 0xdc8 contains 27 entries:
  Tag        Type                         Name/Value
 0x0000000000000001 (NEEDED)             Shared library: [libconnector.so]
 0x0000000000000001 (NEEDED)             Shared library: [libstdc++.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]
 0x000000000000000e (SONAME)             Library soname: [libdb.so]
#[...]

$ readelf -d app.bin 

Dynamic section at offset 0xda8 contains 29 entries:
  Tag        Type                         Name/Value
 0x0000000000000001 (NEEDED)             Shared library: [libdb.so]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]
 0x000000000000001d (RUNPATH)            Library runpath: [/home/alexandre/.conan/data/db/0.0.3/alex/dev/package/991309e0b1e72d26aa0af568021a9c64f8f207d6/lib:/home/alexandre/.conan/data/connector/0.0.1/alex/dev/package/160b4bac5177b0f9a5f4857d00317fe0862a8a02/lib]
#[...]

$ ldd bin/app.bin 
        linux-vdso.so.1 =>  (0x00007ffef89bd000)
        libdb.so => /home/alexandre/.conan/data/db/0.0.3/alex/dev/package/991309e0b1e72d26aa0af568021a9c64f8f207d6/lib/libdb.so (0x00007f71f5bd5000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f71f57f5000)
        libconnector.so => not found
        libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f71f546f000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f71f5fd9000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f71f5119000)
        libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f71f4f02000)

As you can see, conan or cmake (both?) did set the runpath of app.bin, but not in libdb.so, libconnector.so. The app.bin runpath seems to point to all its dependencies, direct and indirect.

But, the runtime linker ld.so does not use the runpath of app.bin for the children dependencies. It only use the runpath definition of the current object it is resolving, as stated by the man of ld.so.

Using the directories specified in the DT_RUNPATH dynamic section attribute of the binary if present. Such directories are searched only to find those objects required by DT_NEEDED (direct dependencies) entries and do not apply to those objects' children, which must themselves have their own DT_RUNPATH entries. This is unlike DT_RPATH, which is applied to searches for all children in the dependency tree.

So how is conan is supposed to handle that, or is it ? by the way https://github.com/conan-io/conan/issues/1303 has a lot of informations regarding this issue.

Today, you can workaround this issue by either:

Ask cmake to set the shared object's runpath to point to its depencies, which are hopefully in the conan cache, with SET(CMAKE_INSTALL_RPATH_USE_LINK_PATH TRUE). That would only work if I compile everything locally with a conan install --build. It won't work if i just want to pull a binary package in the conan cache and use it. Also, handle this with auto-tools and other build system could be harder to do.
use import to copy all the .so. This is tedious and not very elegant IHMO. We have a lot of heavy shared object, i would like not to copy everywhere. Might be the only solution for Windows.
use conan virtualenv generator, which will set the LD_LIBRARY_PATH (among other things). It kind of break/not work with my (fish) shell, means you have to use a posix shell (which i don't, I like it hard, i know), you have to remember to enable/disable the virtualenv or see programs launched by that shell explodes.

So, Is it possible for conan to set/fix the runpath of the installed packages? Linux has the patchelf, mac os install_name_toolutilities. Windows is borken, so conan might have to fallback to an implicit import like mechanism. My point is, conan install has all the infornation to do it correctly. It will allow developers/consumers of conan dependencies to build AND run their project out of the box.

Source

AlexandreBossard

👍7

Most helpful comment

Oh! The perfect linker storm. Just recently been troubleshooting a similar issue.
Here's my 2 cents:

@AlexandreBossard:

binaries inside the package folders in the local cache should not set absolute RPATH/RUNPATH, as these would make them non-relocatable (remember you can upload those same binaries to a remote, etc). One cannot make the assumption that other users will install them in the same location. Even more, one cannot make the assumption that the package_id (which is the name of the folder the binaries are placed), will remain the same for the same settings/options/dependencies.

The problem you are experiencing in conan-deptest (excellent isolated case, by the way!) is the behaviour that you are documenting, between DT_RPATH and DT_RUNPATH.
It is CMake (and not conan), that passes the -rpath flag with the correct paths to the dependent libraries, when building the executable.
The problem is that newer versions of the linker in some platforms, emit DT_RUNPATH where they used to emit DT_RPATH. Indeed this is the case as readelf -d is listing the RUNPATH tag.
This is why @niosHD has not been able to reproduce the problem: his linker is probably emitting RPATH instead.

To see exactly which locations the linker is searching at runtime, you can try to run your app like this:
env LD_DEBUG=files,libs ./app.bin

You will see an output like this:
``` 1038: file=libdb.so [0]; needed by ./app.bin [0]
1038: find library=libdb.so [0]; searching
1038: search path= .... (RUNPATH from file ./app.bin)

notice how it says `RUNPATH`.

Further down the line, you will see:

  1038: find library=libconnector.so [0]; searching
  1038:  search cache=/etc/ld.so.cache
  1038:  search path= ... (system search path)

Notice how for this library, the one that is missing, it is NOT looking in the `RUNPATH` at all.

Now let's see what happens on an Older version of ubuntu where `RPATH` is emitted instead:
```32269:   find library=libconnector.so [0]; searching
     32269:  search path=...        (RPATH from file ./app.bin)

which finds it in the correct location and lets the application run.

The way to replicate this behaviour on versions of the linker that emit RUNPATH instead, is to pass the --disable-new-dtags flag to the linker.
Which you can do like this with CMake:
target_link_libraries(app.bin "-Wl,--disable-new-dtags")

Then your application will run properly.

Obviously, as you mention, this is OK for the cmake build tree, but for relocatable packages including your application, using absolute paths in the rpath is not the best idea in terms of relocation...

luiscarox on 20 Jul 2018

👍2

All 9 comments

Yes, we are aware of this. The Conan default or if want to call it recommended way is to clear the rpaths (just filename) and import to same bin directory all the shared. Then you still need as you said the LD_LIBRARY_PATH etc in a shell, so not ideal.

The idea always is let the user choose the best approach, so a helper to patch the rpaths could be good, the issue is, if you modify the package once installed, the manifest verification will fail and probably we should block it if you try to upload, and I'm sure other problems could raise.

Of course it's doable, but it requires some more features than the rpaths "patchers". It could be a complex one.

lasote on 22 Mar 2018

Shipping is a different problem altogether. Signing should not be too much of an issue. You only check the signature after you downloaded it. If its on your filesystem, the signature guaranty does not mean much. Anyway, you could keep the original signed binary for later use (shipping), and patch the runpath of a copy.

My idea is not to modify the binary for the upload, only during the install. If there is a runpath in the download shared object, there is a high chance it won't be relevant once installed locally. So the tool, let's assume conan, only have to patch the downloaded shared objects after signature verification and unpacking.

IIRC cmake do/knows how to do this during the cmake install, I suspect other generators support this.
If that's the case, triggering the install rule for installing in the conan cache could be a simple approach.

AlexandreBossard on 22 Mar 2018

Yes, I understand what you mean. I only was trying to explain that the issue goes further than patch the rpaths. The Conan model, currently is not supposing the packages in the local cache as something freeze like files installed in the /usr/bin.

The packages in the cache can be moved and uploaded, so some additional check or improvement is needed, I bet it will have more implications, for example if you import the artifacts to a local directory the rpaths should be patched again.

But anyway! It is a feature I would like to have, but it would require work. We could start by providing a tool to patch rpaths and then improve the integration with automatic flows.

lasote on 22 Mar 2018

Thank you for the minimal "working" example, really great to have one! I just gave it a try and, to my surprise, the runtime linker on my system seems to propagate the runpath from the app to the libraries because executing app.bin directly works.

In general, I currently use the virtualenv generator as a workaround and setup (DY)LD_LIBRARY_PATH and PATH in my packages accordingly. I assume that is best practice but am not sure to be honest. As last resort, if I consume external packages which do not setup the variables accordingly, I fall back to the virtualrunenv generator. Anyway, I definitely agree that that some better solution would be great.

My stab at this is the proposed conan exec (see PR #2516) command which is the result of the discussions in #2461. conan exec is basically a wrapper around conan install, the virtualenv and virtualrunenv generators, os.exec, and permits to execute a command in the environment of packages.

Therefore, launching of the app through conan exec hopefully should make it work. If you give the PR a try, the following command should do the trick:
conan exec --path app/ -o '*:shared=True' ./app/build/bin/app.bin

Furthermore, launching other shells like fish should also work and kind of replaces the virtualenv generator. Please give it a try and join the discussion on conan exec.

niosHD on 23 Mar 2018