Cargo: Build a binary with dependencies dynamically linked and loaded

[mssun]

Current cargo can only build a "fat" binary, meaning that all dependencies will be static linked into the final binary. However, rustc support to dynamic load a rust library. For example, I can use the --crate-type dylib flag to generate an so library (for Linux target) and use the -C prefer-dynamic flag to dynamically link the library. By this way, multiple binaries can share a same library (in the loader's search path: LD_LIBRARY_PATH) instead of keeping multiple copies.

I found that current cargo cannot generate a "thin" binary. After examining the sources of cargo, I found these following reasons.

1. Firstly, cargo does not allow to pass --crate-type dylib flags by using RUSTFLAGS environment. Since cargo depends on a stdout of a dummy compilation pass for further configuration.
   
   
   
   • https://github.com/rust-lang/cargo/blob/master/src/cargo/ops/cargo_rustc/context.rs#L249
   
   
2. The default --crate-type value for a dependency library is rlib, and this value is not customizable.
   
   
   
   • https://github.com/rust-lang/cargo/blob/master/src/cargo/ops/cargo_rustc/context.rs#L201
   
   
   • https://github.com/rust-lang/cargo/blob/master/src/cargo/util/toml/targets.rs#L131
   
   

The easiest way to fix this issue is to provide a config flag to set the default --crate-type. Then, people like me can use dylib flags to build dependencies and finally build a "thin" binary.

However, it was not as easy as I though. I tried to modify cargo to build with dylib by default. Some libraries can be built, but some library are not. (And this may be another issue) For example, libraries with no_std attribute:

$ unicode-segmentation git:(master) cargo build -vv
    Updating registry `https://github.com/rust-lang/crates.io-index`
   Compiling unicode-segmentation v1.2.0 (file:///Users/mssun/Code/unicode-segmentation)
     Running `rustc --crate-name unicode_segmentation src/lib.rs --crate-type lib --emit=dep-info,link -C debuginfo=2 -C metadata=699be049c18ca985 -C extra-filename=-699be049c18ca985 --out-dir /Users/mssun/Code/unicode-segmentation/target/debug/deps -L dependency=/Users/mssun/Code/unicode-segmentation/target/debug/deps`
    Finished dev [unoptimized + debuginfo] target(s) in 0.69 secs
$ unicode-segmentation git:(master) rustc --crate-name unicode_segmentation src/lib.rs --crate-type lib --emit=dep-info,link -C debuginfo=2 -C metadata=699be049c18ca985 -C extra-filename=-699be049c18ca985 --out-dir /Users/mssun/Code/unicode-segmentation/target/debug/deps -L dependency=/Users/mssun/Code/unicode-segmentation/target/debug/deps
$ unicode-segmentation git:(master) rustc --crate-name unicode_segmentation src/lib.rs --crate-type dylib --emit=dep-info,link -C debuginfo=2 -C metadata=699be049c18ca985 -C extra-filename=-699be049c18ca985 --out-dir /Users/mssun/Code/unicode-segmentation/target/debug/deps -L dependency=/Users/mssun/Code/unicode-segmentation/target/debug/deps
error: language item required, but not found: `panic_fmt`

error: language item required, but not found: `eh_personality`

error: aborting due to 2 previous errors

Anyways, if there is a plan to support such kind of need, I would like to help.

[alexcrichton]

Thanks for the report! What would be the eventual goal of this support? I don't think this would speed up compilations, unfortunately :(

[mssun]

The main goal is to use Rust as a system language like C/C++. That is to provide dynamic linked binaries and shared libraries. The shared libraries can be linked by all binaries which depend on them.

This scenario is a bit like libc in Rust. All Rust binaries (except crates with no-std) rely on libc and dynamically load them at runtime.

Obviously, there are some advantages of dynamic linking. I guess one is to minimize disk usages for some embedded devices. In addition, updating libraries no longer need to rebuild binaries. Of course, this will definitely cause some compatibility issues (e.g., how to manage different versions). But, overall, I think it's still worth to do.

[alexcrichton]

Ah ok in that case I think this is less of a Cargo issue and moreso a Rust/rustc issue. Rust right now has no stable ABI by default so you have to opt-in with extern functions and #[no_mangle]. It's true that Cargo doesn't help much with the way it builds crates today but it's also not clear how it would actually change to support this use case.

[mssun]

What do you mean by "no stable ABI by default so you have to opt-in with extern functions and #[no_mangle]"?

Right now, I can export rust functions as dylib for other binary like this:

$ cat lib.rs
pub fn foo() {
    println!("Hello from hello-world-lib");
}

$ cat main.rs
extern crate hello_world_lib;
use hello_world_lib::*;
fn main() {
    foo();
    println!("Hello, world!");
}

$ rustc --crate-name hello_world_lib lib.rs --crate-type dylib --emit=dep-info,link -C prefer-dynamic

$ rustc main.rs --extern hello_world_lib=libhello_world_lib.so

$ LD_LIBRARY_PATH=~/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib:. ./main
Hello from hello-world-lib
Hello, world!

$ ldd main
        linux-vdso.so.1 =>  (0x00007ffd561fd000)
        libhello_world_lib.so => not found
        libstd-5e796e3137ff8efb.so => not found
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f245b566000)
        /lib64/ld-linux-x86-64.so.2 (0x000055a55db1c000)

Correct me if I'm wrong, because of the unstable ABI, binaries compiled with later rustc versions will not successfully call to functions of dylib compiled with old rustc.

[mssun]

Here is the discussion of ABI stability: https://github.com/rust-lang/rfcs/issues/600

Seems that this is a long-term (more than two years) discussion.

[alexcrichton]

@mssun the problem with your example is that the foo function does not have a stable symbol name across compilers nor even a stable ABI across compilers. You're only guaranteed that the one compiler you just used will work with linking against that dynamic library.

It's true that the upstream discussion has gone on for a long time, this is a very thorny and not-super-high-priority issue, meaning that progress tends to be slow.

[carols10cents]

I'm going to close this since it depends on the stable API RFC discussion. Thanks!

[FrankHB]

I don't think it appropriate to close this issue by forwarding the problem to rustc. Technically, ABI stability is not a must to resolve the problem. As a tool concerning with dependency management, cargo could do something more than simply relying on some fixed set of binary dependencies, e.g. #2552.

[piegamesde]

The linked Rust ABI issue this was blocked on has been closed; there won't be a stable Rust ABI any time soon.

But I second the thought that some form of dynamic linking would still be useful in a few use cases, even if it requires compiling all dependencies with the same rustc version or similar restrictions.