Rust: fs::metadata() crashes on FreeBSD 12 due to layout change in stat.h
The layout of struct stat used to be this in FreeBSD 11, in FreeBSD 12 it's now this. This causes the FreeBSD-specific implementation of std::fs::metadata() to crash for 1.18 and nightly as of 17/06/15, as it's stack frame get's boiled up.
Can be reproduced via
use std::fs;
fn main() {
println!("{:?}", fs::metadata("Cargo.toml"));
}
Downstream this appeared in jwilm/alacritty#618
lukaslueg
All 38 comments
I would like to point a pre-RFC discussion about introducing a way in Rust to permit libc to represent differents layouts per OS version. I think there is need to move forward.
semarie
on 16 Jun 2017
We discussed this at libs triage today and the conclusion was that this is indeed bad! We would be fine in the very near future adding dynamic detection to do the right thing at runtime. In the long term we decided to see how @semarie's rfc plays out.
alexcrichton
on 21 Jun 2017
Is there a quick fix for this? This stops me from building a few (probably many) crates..
johalun
on 8 Jul 2017
Here's a temporary fix for those who can't wait
https://github.com/johalun/rust-ino64
Just re-apply each time you update nightly with rustup.
johalun
on 13 Jul 2017
@johalun Thanks for a workaround ! :beer:
kirillrdy
on 29 Jul 2017
Hi!
I prepared a patch to support the two breaking changes in FreeBSD 12.x (see rust-lang/libc#721). I tested them successfully on FreeBSD 10.3 and FreeBSD 12.0, using the testsuites of mio and hyper.
However, I'm not sure my solution around struct kevent is correct and I would love to hear from others. The pull request gives all the details about the patch and what I had in mind.
dumbbell
on 10 Aug 2017
@dumbbell there's an ongoing discussion about the fact that Rust has inadvertently baked in UBs on !Linux platforms by creating its own private copies of system headers, that are essentially an arbitrary version. https://github.com/rust-lang/libc/issues/570#issuecomment-331475043
Cargo needs to grow support for doing the equivalent of parsing uname -K and passing that as target_os_version.
For the time being I've accepted that when compiling I need to just append the following 3 lines to the Cargo.toml of any root crate when building on HEAD:
[patch.crates-io]
mio = { git = "https://github.com/FreeBSDRust/mio" }
libc = { git = "https://github.com/FreeBSDRust/rust-libc" }
mattmacy
on 22 Sep 2017
Actually - although I managed to get jsonrpc_http_server bits to work yesterday by patching some of the dependent ports and then using [patch.crates-io] overrides, today I tried building netmap_sys with user libs which failed in the gcc crate, presumably because of something in libstd. I tried installing the FreeBSD port, but the cargo in that version wouldn't let me override libc (if that's something only available in nightlies you really need to make it standard) so although it compiled the binary didn't actually work. I've lost too much time on this to consider Rust a viable option for any near-term deadlines and have concluded that Rust really isn't cross-platform outside of tier 1 targets right now.
mattmacy
on 23 Sep 2017
Here's a fun thing. @johalun's amazing method of patching old_fstat.c into an already installed (rustup) std lib messes with…
Mesa OpenGL driver loading. Running glutin or webrender examples compiled with an unpatched compiler, my GPU driver loads fine. With a patched compiler:
libGL error: MESA-LOADER: failed to retrieve device information
[…] libGL error: failed to load driver: amdgpu
[…] libGL: OpenDriver: trying /usr/local/lib/dri/swrast_dri.so
And no, MESA_LOADER_DRIVER_OVERRIDE=radeonsi doesn't help — if it loads the right driver via manual override, that driver can't communicate to the GPU device either:
amdgpu: drmGetDevice2 failed.
do_winsys_init: DRM version is 3.8.0 but this driver is only compatible with 2.12.0 (kernel 3.2) or later.
libGL error: failed to create dri screen
libGL error: failed to load driver: radeonsi
The ports/pkg rustc, compiled normally with the same old_fstat changes patched in at the source level (UPD: used on the bootstrap compiler only; actual rust patch is used instead on the resulting build), does not have this problem. I guess I'll have to resurrect the lang/rust-nightly port…
myfreeweb
on 2 Oct 2017
Here's a working nightly Rust toolchain for FreeBSD 12-CURRENT: rustc std cargo rls analysis docs. Don't ask me how I built it :D
(plus fully updated libc)
myfreeweb
on 3 Oct 2017
@myfreeweb It actually isn't my method, I took it from the rust freebsd port. It did work back when I made it but seem to cause runtime errors when using with newer nightlies.
Thanks for the nightly builds!! Is it 1.22? Does this mean that the rust-nightly port is alive again?
johalun
on 4 Oct 2017
Yep, it's 1.22 I built by copy-pasting the rust port as rust-nightly (I'll push it to ports-dank today) and changing version numbers. And restarting the build a couple times because of the deadlock issue on the bootstrap (beta) compiler :D
myfreeweb
on 4 Oct 2017
I extracted your builds to $HOME/.rustup/toolchains/nightly-x86_64-unknown-freebsd replacing the broken nightly installed by rustup and it works perfectly combined with
[patch.crates-io]
mio = { git = "https://github.com/FreeBSDRust/mio" }
libc = { git = "https://github.com/FreeBSDRust/rust-libc" }
Now I can build and run my new favorite tool exa :)
johalun
on 6 Oct 2017
You can actually tell rustup to add a custom directory as a toolchain so it won't overwrite :)
That one doesn't fully match the actual layout of the structs in 12! I think it's made for that live patched version. Use https://github.com/myfreeweb/libc instead.
myfreeweb
on 6 Oct 2017
I'm getting error: toolchain 'custom' does not support components
Is there anyway to get custom toolchain to support components? (at least rust-src)
Edit: copying rust src into the toolchain manually solved the problem for now...
johalun
on 6 Oct 2017
@myfreeweb can you build cargo with openssl which seems to be standard for ports? i can't install anything from ports since it will delete libressl and cargo stops working :(
johalun
on 12 Oct 2017
delete libressl? ports should build everything with libressl if you tell them to!
echo 'DEFAULT_VERSIONS+=ssl=libressl' >> /etc/make.conf
pkg install xxx where xxx depends on openssl will delete libressl in order to install openssl as dependency...
I rather not build all the ports myself when I can install from pkg...
johalun
on 12 Oct 2017
and all prebuilt ports seem to depend on openssl and not libressl...
edit: maybe rebuilding cargo is easier with the rustc and std you built?
johalun
on 12 Oct 2017
huh, odd — all the pkg installed stuff i have uses openssl from base, does not require openssl from ports
myfreeweb
on 12 Oct 2017
Shared object "libssl.so.43" not found, required by "cargo"
that's libressl...
johalun
on 12 Oct 2017
not cargo! I said the stuff you install from pkg…
$ ldd /usr/local/bin/wget
libssl.so.8 => /usr/lib/libssl.so.8 (0x801074000)
libcrypto.so.8 => /lib/libcrypto.so.8 (0x801400000)
For pkg install we have for example: grub2 -> gcc6 -> openssl dependency. there are more but I don't remember now...
I tried rebuilding cargo with all kinds of OPENSSL_* env variables but couldn't get it to link only to libssl from base or do static linking. Anyhow, I could rebuild cargo to use openssl instead of libressl so at least solved that problem.
johalun
on 12 Oct 2017
@johalun you can't mix OpenSSL and LibreSSL from ports on FreeBSD. Don't focus on the OPENSSL_ env vars (defined via USES=ssl and Mk/Uses/ssl.mk). Set your ssl provider as indicated by @myfreeweb
echo 'DEFAULT_VERSIONS+=ssl=libressl' >> /etc/make.conf
and rebuild all ports that depend on ssl.
Sp1l
on 6 Dec 2017
Exactly, you can't mix and freebsd official pkg repo uses openssl. That means the package discussed here is not compatible with the official repo since it require that openssl is replaced with libressl and thus a bunch of installed pkgs depending on openssl will be uninstalled. That's the problem. Wouldn't it be better to release a package that is compatible with the official repo so users won't have to rebuild all dependencies from source when installing something from 3rd party source? (which will break next time you pkg update the system...)
johalun
on 6 Dec 2017
The official rust port is now at 1.22.1, newer than this :) though not nightly channel.
Also, official packages usually were built with base openssl (/usr/lib)… maybe that changed recently I guess
myfreeweb
on 6 Dec 2017
Also, official packages usually were built with base openssl (/usr/lib)… maybe that changed recently I guess
Yeah that would make more sense if there's no actual difference between base and port openssl. But some ports do have ports openssl as dependency so...
johalun
on 6 Dec 2017
The default in FreeBSD is (still) to use OpenSSL libraries from base (i.e. /lib/libcrypto.so and /usr/lib/libssl.so). Official packages are always built like that. Users can change that behavior using the ports collection and building their own packages. Especially when switching to OpenSSL (or LibreSSL) from ports you must recompile all package that depend on libcrypto and libssl or you will experience conflicts (pkg, pam, gss are known problems).
Note: I am the maintainer of the OpenSSL and LibreSSL ports for FreeBSD. More info on the FreeBSD wiki https://wiki.freebsd.org/OpenSSL
Sp1l
on 6 Dec 2017
This really needs to be fixed.
bdrewery
on 21 Feb 2018
Agreed. It's a horrible situation that need a solution ASAP!
If you're on post-ino64 12-current you can at least get stable rust from pkg install rust (which unfortunately is not compatible with rustup), and you can build 1.25-nightly from my ports tree here https://github.com/johalun/freebsd-ports/tree/rust-nightly
It fails Q&A but at least you can extract the toolchain and put in ~/.rustup/ if you're in need of a nightly like I am.
johalun
on 21 Feb 2018
@johalun Have you submitted a PR for getting your fixed rust-nightly port committed? What is failing in it?
bdrewery
on 21 Feb 2018
passing QA is not necessary to get an installable package, I have rust-nightly installed system-wide
myfreeweb
on 21 Feb 2018
@johalun I'm working on taking your port and bringing it up-to-date with lang/rust updates, and other QA checks.
bdrewery
on 21 Feb 2018
@bdrewery I think 1.26-nightly is scheduled for release any day now so maybe we should go for that instead. Hopefully similar patching can build it! I didn't investigate further about the Q&A problem since I only needed a toolchain and is -ENOTIME.
johalun
on 22 Feb 2018
Does this also break building from git? I am getting a crash with /root/git/rust/rust/build/bootstrap/debug/bootstrap build. I'm thinking yes.
fstatat(AT_FDCWD,"config.toml",{ mode=-rw-r--r-- ,inode=1304216,size=518,blksize=4096 },0x0) = 0 (0x0)
SIGNAL 11 (SIGSEGV) code=SEGV_MAPERR trapno=12 addr=0x0
Yeah it seems that the FreeBSD bootstrap files are needed in build/cache, as well as the libstd patching
that the port does.
I'll have to make a blog post about how to build from git from 12-CURRENT because it's quite complicated.
bdrewery
on 23 Feb 2018
FYI I've updated the lang/rust-nightly port with 1.25.0 2018-02-29.
bdrewery
on 24 Feb 2018
Technically this is not fixed yet as I need to update the submodule and the bootstrap needs to get the fix. Will be a few days.
bdrewery
on 2 Mar 2018
The last piece for this should be updating the src/stage0.txt beta used. I am testing that now.
bdrewery
on 19 Mar 2018
Related issues
behnam
·
3Comments
pedrohjordao
·
3Comments
cuviper
·
3Comments
modsec
·
3Comments
eddyb
·
3Comments
Most helpful comment
@dumbbell there's an ongoing discussion about the fact that Rust has inadvertently baked in UBs on !Linux platforms by creating its own private copies of system headers, that are essentially an arbitrary version. https://github.com/rust-lang/libc/issues/570#issuecomment-331475043
Cargo needs to grow support for doing the equivalent of parsing uname -K and passing that as target_os_version.
For the time being I've accepted that when compiling I need to just append the following 3 lines to the Cargo.toml of any root crate when building on HEAD:
[patch.crates-io]
mio = { git = "https://github.com/FreeBSDRust/mio" }
libc = { git = "https://github.com/FreeBSDRust/rust-libc" }