Rust: Tracking issue for musl host toolchain

[mati865]

TODO:

• [x] Installable host toolchain (https://github.com/rust-lang/rust/pull/58575)
  
  
  
  • [x] Rustup support (https://github.com/rust-lang/rustup.rs/pull/1882)
  
  
• [x] Tests passing as the host (https://github.com/rust-lang/rust/pull/60474)
• [ ] Convenient way to link dynamically (e.g. $arch-unknown-linux-dynmusl triple)
• [x] static-pie support https://github.com/rust-lang/rust/issues/53968 (done in https://github.com/rust-lang/rust/pull/70740)

For people looking how to use as it dynamic target:

RUSTFLAGS="-C target-feature=-crt-static" cargo build

[wangbj]

It seems x86_64-unknown-linux-musl target doesn't honor --enable-default-pie, a different behavior compare to x86_64-unknown-linux-gnu. Is this a known issue?

[mati865]

@wangbj I don't think Rust supports --enable-default-pie for any target. Could you provide more details?

[wangbj]

hmm, the problem seems caused by musl-gcc wrapper instead of rustc:

details: https://gist.github.com/wangbj/ead5fa8c96418de7c9050bc354fbf353

--enable-default-pie is a gcc configure flag. sorry for the noise. (shortened the text to a gist).

[Cogitri]

FWIW, Void has been hacking around this for some time: https://github.com/void-linux/void-packages/blob/742a7d53eb0b373bf8dad3b51db3380c5cdf8dc1/srcpkgs/rust/patches/musl-dont-use-crt-static.patch and https://github.com/void-linux/void-packages/blob/742a7d53eb0b373bf8dad3b51db3380c5cdf8dc1/srcpkgs/rust/patches/link-musl-dynamically.patch

I'm currently working on this for Alpine too (BTW, maybe I can ping you on the PR for this @mati865 so you can take a look at it?).

[mati865]

I knew about Alpine and Void. IMO Alpine approach is better because <arch>-unknown-linux-musl remain static by default (it behaves like upstream) and new target <arch>-alpine-linux-musl links dynamically by default.

I had looked at that PR and libc changes will be incompatible with upstream rust behaviour. Official Rust packages when statically linking to musl create executables that all fully static (run basically everywhere), that patch will create static executables that depend on musl runtime (use have to install musl to be able to run them).

[Cogitri]

Yup, but at least on Void we've come to terms that the Rust in the repos is only really meant for packages in the repos, while users interested in rust development should use upstream Rust via rustip.

[smaeul]

I had looked at that PR and libc changes will be incompatible with upstream rust behaviour. Official Rust packages when statically linking to musl create executables that all fully static (run basically everywhere), that patch will create static executables that depend on musl runtime (use have to install musl to be able to run them).

That doesn't make sense. Static executables by definition don't depend on having musl installed. Are you referring to rust-lang/libc#1327? That change would require having a musl-targeting toolchain installed for rustc to link (static) musl-targeting binaries. But any static binaries compiled by rustc would still be totally independent and require no musl installation.

[mati865]

@smaeul I'm referring to https://github.com/alpinelinux/aports/blob/21c1983ce828ead1afc9e80aa77ede330c662182/community/rust/link-musl-dynamically.patch

[smaeul]

@mati865 Okay, so that patch is misnamed. It doesn't have anything to do with static versus dynamic linking. It makes the same change as rust-lang/libc#1327 -- using the C toolchain's copy of libc.a instead of bundling it inside liblibc.rlib. And my comment above applies to it as well.

Alpine also ships custom targets that use the C toolchain's crt*.o instead of rustc shipping a second copy. Since on Alpine targets rustc's copy of crt*.o are not used, this patch stops rustc from making the copy. (This change is unrelated to the liblibc change.)

[mati865]

Thanks for the explanation.

[thedrow]

53968 also seems to be related but it's not included in the OP's checklist.

Am I correct?

[mati865]

Updated

[thedrow]

61328 also seems important.

[mati865]

I don't see how #61328 relates to this, there is no issue when using target-feature=-crt-static.
It applies to musl toolchain in general but it has nothing to do with using musl toolchain natively.

[gnzlbg]

In https://github.com/rust-lang/libc/pull/1327, a #[cfg(target_env = "dyn_musl")] or similar was proposed to support both statically and dynamically linked musl environments. I don't know if that would be a good way to make progress here though.

How do other run-times deal with this ? Is it possible to statically link glibc and others into Rust binaries ? Maybe we need a #[cfg(target_env_linkage = "static/dynamic/...")] system instead of duplicating target_envs per target.

[mati865]

AFAIK musl is the only target which you can link statically and dynamically at same time.

Duplicating target_env is easy but for me it sounds like a bad workaround because user would have to install 2 toolchains that have exactly the same libs and the only difference comes to target-feature=-crt-static.
With static nobundle it would be third 99% identical toolchain.

Ideally they should use single shared sysroot because all that is required can be handled at runtime.
I think Rust is not capable of it yet so this would have to go through RFC first.

[smaeul]

How do other run-times deal with this ? Is it possible to statically link glibc and others into Rust binaries ?

The two relevant target attributes are crt_static_respected and crt_static_allows_dylibs.

For glibc (and all other non-musl unix-like platforms), crt_static_respected is false, so the -C target-feature=+crt-static option is ignored by the backend and still dynamically links libc, and therefore dynamic native libraries can still be linked. So the only effect of the option is that it matches
#[cfg(target_feature = "crt-static")].

For MSVC, crt_static_respected and crt_static_allows_dylibs are both true, so -C target-feature=+crt-static only affects the C runtime, and other (rust or native) libraries can still be loaded as DLLs. This is because on Windows, there's not really such a thing as a "static binary".

musl is in the unique situation of crt_static_respected being true and crt_static_allows_dylibs being false. Since musl uses ELF (and this would apply to glibc too if it supported static linking), linking libc statically requires all other libraries to also be linked statically. For native libraries, this means #[link(kind = "static")] or #[link(kind = "static-nobundle)]".

rustc currently handles this limitation correctly for rust libraries and direct native dependencies, but not for native libraries that are transitive dependencies of a dependent crate.

I had a patch (#55566) that "fixed" this by changing the semantics of #[link] with no kind in the +crt-static and not crt_static_allows_dylibs case. But the default kind is documented to be dylib, so that change was unacceptable. Instead, we need a patch to produce an error, similar to the non-transitive case here.

Maybe we need a #[cfg(target_env_linkage = "static/dynamic/...")] system instead of duplicating target_envs per target.

We already have that: #[cfg(target_feature = "crt-static")].

---

The problem in rust-lang/libc#1327 is that we try to support cross-compilation to musl without having the proper sysroot installed. The whole musl_root system is a giant hack to support that, and it breaks in the native case, where we want to use the libc.a that is actually present on the system (and updated by the package manager, etc.). Distributions that ship a native musl rust have to add a patch similar to the one in that PR (specifically e.g. this one for Adélie and Gentoo).

[Cogitri]

I had a patch (#55566) that "fixed" this

Fwiw Alpine Linux ships a similar patch (although we don't use crt-static for pur distro packages). Good to know what's the status on that, thanks.

https://github.com/alpinelinux/aports/blob/master/community/rust/musl-fix-static-linking.patch

Distributions that ship a native musl rust have to add a patch similar to the one in that PR (specifically e.g. this one for Adélie and Gentoo).

Void Linux and Alpine Linux carry a similiar patch: https://github.com/alpinelinux/aports/blob/master/community/rust/link-musl-dynamically.patch

[gnzlbg]

The problem in rust-lang/libc#1327 is that we try to support cross-compilation to musl without having the proper sysroot installed.

Many production users rely on this feature, so we are kind of constrained on not breaking this. If the only way to achieve that is have a second set of targets, then that's the way it is.

[smaeul]

Many production users rely on this feature, so we are kind of constrained on not breaking this. If the only way to achieve that is have a second set of targets, then that's the way it is.

What about shipping libc.a alongside rust, like we already do for crt*.o? Then we could link it static-nobundle from liblibc.rlib instead of bundling it inside. And then if we only add a -L for that directory to the linker args when cross-compiling, it would properly use the system libc.a for the native case as well. (And distro packagers could simply rm rust's copy of libc.a instead of patching).

[gnzlbg]

I think I would prefer something that also works great for distro packages. While they could tune things when packaging, a distro that uses musl as its stdlib, will have users that might install rust via rustup, and they will want their "default" rust target to dynamically link the system musl, instead of doing something else, right?

[smaeul]

Yes. That's why most (all except Void) musl distributions ship distro-specific targets (usually matching their $CHOST) that default to dynamic linking. On the other hand, the "static by default" semantics of the current official musl targets are well documented, and x86_64-unknown-linux-musl is already available in rustup.

The two feasible solutions I can think of are:

1. Introduce new targets that are dynamic by default – by changing the vendor (e.g. x86_64-dynamic-linux-musl), not by changing the environment (e.g. x86_64-unknown-linux-musldyn), which would break all existing uses of #[cfg(target_env = "musl")].
2. (At the risk of making the mess worse) override the crt_static_default = true in musl targets when the host environment is also musl, or alternatively change it to crt_static_default = false and override it when cross compiling.

[smaeul]

Linking these here for future reference, more cases where bundling libc.a into liblibc.rlib breaks things:
https://github.com/rust-lang/rust/issues/71564
https://github.com/rust-lang/libc/pull/1704