Rust: Undefined symbol _fltused when compiling to x86_64-unknown-uefi

[IsaacWoods]

Recent nightlies give the following linker error when trying to compile with the x86_64-unknown-uefi target:

error: linking with `rust-lld` failed: exit code: 1
  |
  = note: "rust-lld" "-flavor" "link" "/NOLOGO" "/NXCOMPAT" "/nodefaultlib" "/entry:efi_main" "/subsystem:efi_applic
ation" "/LIBPATH:/home/isaac/Documents/pebble/bootloader/target/sysroot/lib/rustlib/x86_64-unknown-uefi/lib" "/home/
isaac/Documents/pebble/bootloader/target/x86_64-unknown-uefi/release/deps/bootloader-cb388c983ab7c215.bootloader.17p
m2976-cgu.0.rcgu.o" "/home/isaac/Documents/pebble/bootloader/target/x86_64-unknown-uefi/release/deps/bootloader-cb38
8c983ab7c215.bootloader.17pm2976-cgu.1.rcgu.o" "/home/isaac/Documents/pebble/bootloader/target/x86_64-unknown-uefi/r
elease/deps/bootloader-cb388c983ab7c215.bootloader.17pm2976-cgu.10.rcgu.o" "/home/isaac/Documents/pebble/bootloader/
target/x86_64-unknown-uefi/release/deps/bootloader-cb388c983ab7c215.bootloader.17pm2976-cgu.11.rcgu.o" "/home/isaac/
Documents/pebble/bootloader/target/x86_64-unknown-uefi/release/deps/bootloader-cb388c983ab7c215.bootloader.17pm2976-
cgu.12.rcgu.o" "/home/isaac/Documents/pebble/bootloader/target/x86_64-unknown-uefi/release/deps/bootloader-cb388c983
ab7c215.bootloader.17pm2976-cgu.13.rcgu.o" "/home/isaac/Documents/pebble/bootloader/target/x86_64-unknown-uefi/relea
se/deps/bootloader-cb388c983ab7c215.bootloader.17pm2976-cgu.14.rcgu.o" "/home/isaac/Documents/pebble/bootloader/targ
et/x86_64-unknown-uefi/release/deps/bootloader-cb388c983ab7c215.bootloader.17pm2976-cgu.15.rcgu.o" "/home/isaac/Docu
ments/pebble/bootloader/target/x86_64-unknown-uefi/release/deps/bootloader-cb388c983ab7c215.bootloader.17pm2976-cgu.
2.rcgu.o" "/home/isaac/Documents/pebble/bootloader/target/x86_64-unknown-uefi/release/deps/bootloader-cb388c983ab7c2
15.bootloader.17pm2976-cgu.3.rcgu.o" "/home/isaac/Documents/pebble/bootloader/target/x86_64-unknown-uefi/release/dep
s/bootloader-cb388c983ab7c215.bootloader.17pm2976-cgu.4.rcgu.o" "/home/isaac/Documents/pebble/bootloader/target/x86_
64-unknown-uefi/release/deps/bootloader-cb388c983ab7c215.bootloader.17pm2976-cgu.5.rcgu.o" "/home/isaac/Documents/pe
bble/bootloader/target/x86_64-unknown-uefi/release/deps/bootloader-cb388c983ab7c215.bootloader.17pm2976-cgu.6.rcgu.o
" "/home/isaac/Documents/pebble/bootloader/target/x86_64-unknown-uefi/release/deps/bootloader-cb388c983ab7c215.bootl
oader.17pm2976-cgu.7.rcgu.o" "/home/isaac/Documents/pebble/bootloader/target/x86_64-unknown-uefi/release/deps/bootlo
ader-cb388c983ab7c215.bootloader.17pm2976-cgu.8.rcgu.o" "/home/isaac/Documents/pebble/bootloader/target/x86_64-unkno
wn-uefi/release/deps/bootloader-cb388c983ab7c215.bootloader.17pm2976-cgu.9.rcgu.o" "/OUT:/home/isaac/Documents/pebbl
e/bootloader/target/x86_64-unknown-uefi/release/deps/bootloader-cb388c983ab7c215.efi" "/OPT:REF,ICF" "/DEBUG" "/LIBP
ATH:/home/isaac/Documents/pebble/bootloader/target/x86_64-unknown-uefi/release/deps" "/LIBPATH:/home/isaac/Documents
/pebble/bootloader/target/release/deps" "/LIBPATH:/home/isaac/Documents/pebble/bootloader/target/sysroot/lib/rustlib
/x86_64-unknown-uefi/lib" "/home/isaac/Documents/pebble/bootloader/target/x86_64-unknown-uefi/release/deps/libx86_64
-e73f78284a1860d7.rlib" "/home/isaac/Documents/pebble/bootloader/target/x86_64-unknown-uefi/release/deps/libbitflags
-094ae181fdaa32fd.rlib" "/home/isaac/Documents/pebble/bootloader/target/x86_64-unknown-uefi/release/deps/libmer-507c
9f2f8b4d808b.rlib" "/home/isaac/Documents/pebble/bootloader/target/x86_64-unknown-uefi/release/deps/libscroll-363628
fffd45fc02.rlib" "/home/isaac/Documents/pebble/bootloader/target/x86_64-unknown-uefi/release/deps/libbit_field-0de0b
ae1c29c3197.rlib" "/home/isaac/Documents/pebble/bootloader/target/x86_64-unknown-uefi/release/deps/liblog-7dddffb9af
6bf12e.rlib" "/home/isaac/Documents/pebble/bootloader/target/x86_64-unknown-uefi/release/deps/libcfg_if-6874da1a7a9e
b04a.rlib" "/home/isaac/Documents/pebble/bootloader/target/sysroot/lib/rustlib/x86_64-unknown-uefi/lib/librustc_std_
workspace_core-34dfaa25be8aa12b.rlib" "/home/isaac/Documents/pebble/bootloader/target/sysroot/lib/rustlib/x86_64-unk
nown-uefi/lib/libcore-6847ba0a79455c40.rlib" "/home/isaac/Documents/pebble/bootloader/target/sysroot/lib/rustlib/x86
_64-unknown-uefi/lib/libcompiler_builtins-0eac2e6da53df529.rlib"
  = note: rust-lld: error: undefined symbol: _fltused
          >>> referenced by libcore-6847ba0a79455c40.rlib(core-6847ba0a79455c40.core.e2k58cht-cgu.12.rcgu.o)
          >>> referenced by libcore-6847ba0a79455c40.rlib(core-6847ba0a79455c40.core.e2k58cht-cgu.15.rcgu.o)
          >>> referenced by libcore-6847ba0a79455c40.rlib(core-6847ba0a79455c40.core.e2k58cht-cgu.5.rcgu.o)
          >>> referenced by libcore-6847ba0a79455c40.rlib(core-6847ba0a79455c40.core.e2k58cht-cgu.6.rcgu.o)
          >>> referenced by libcore-6847ba0a79455c40.rlib(core-6847ba0a79455c40.core.e2k58cht-cgu.3.rcgu.o)
          >>> referenced by libcore-6847ba0a79455c40.rlib(core-6847ba0a79455c40.core.e2k58cht-cgu.10.rcgu.o)
          >>> referenced by libcompiler_builtins-0eac2e6da53df529.rlib(compiler_builtins-0eac2e6da53df529.compiler_b
uiltins.6cufwy9q-cgu.0.rcgu.o)

My guess at likely causes is this commit to compiler-builtins. The project itself does not use any floating-point operations so I'm surprised that any code that refers to this symbol is included in the final executable.

I'm not sure what the best fix for this would be (assuming this isn't intended behaviour), but adding this somewhere in the project works in the meantime:

#[used]
#[no_mangle]
pub static _fltused: i32 = 0;

[nikomatsakis]

Update from compiler-team:

Marking as high priority but the most obvious next step would be to bisect this and validate which PR caused the problem. The cargo-bisect-rustc tool could be very helpful here.

cc @varkor, since the commit cited above was from them, though it's not clear to me if that is the true cause

[pnkfelix]

triage: Assigning to self to at least handle the bisection, since this seems like it fell off our radar over past month+

[oli-obk]

I'll bisect tomorrow. I need to learn the bisection ropes anyway for sth else.

[nikomatsakis]

It seems like the first step is to reproduce the problem at all! I guess maybe I need to do some research into how this works, but I was not able to install the x86_64-unknown-uefi toolchain:

> rustup target install x86_64-unknown-uefi
error: toolchain 'nightly-x86_64-unknown-linux-gnu' does not contain component 'rust-std' for target 'x86_64-unknown-uefi'

I was also not able to build a project, even a #![no_std] one, due to not having the standard library available. Not sure what the steps are? Maybe @IsaacWoods could post a few more details as to what steps to take to reproduce the problem? (Or a link to their project?)

[joshtriplett]

At the moment, the uefi target requires cargo xbuild as it doesn't have a pre-built libcore.

@jyao1, can you take a look at this?

[12101111]

#![no_std]
#![no_main]

#[no_mangle]
pub extern "C" fn efi_main() -> f32 {
    2.0*3.0
}

#[panic_handler]
fn panic(_info: &core::panic::PanicInfo) -> ! {
    loop{}
}

This isn't a valid UEFI application, but it will triger this link error.
We can build it using cargo xbuild or using cargo build -Z build-std=core In recent nightly,

[12101111]

modify cargo-bisect-rustc to download rust-src: https://github.com/rust-lang-nursery/cargo-bisect-rustc/pull/35

Run:env RUST_SRC_REPO=~/rust/ cargo-bisect-rustc --test-dir=./ --with-src -v --prompt --start=4b65a86ebace8600c8e269e8bfe3365cdc460e68 --end=bc2e84ca0939b73fcf1768209044432f6a15c2e5 -- xbuild

Result:
searched toolchains 4b65a86ebace8600c8e269e8bfe3365cdc460e68 through bc2e84ca0939b73fcf1768209044432f6a15c2e5
regression in 38798c6d68394874686dfa3d03e56e12a3ff3d54

The problem is in https://github.com/rust-lang/rust/pull/62592

[12101111]

All commits related to symbol _fltused is
https://github.com/rust-lang/llvm-project/commit/2c36240a820c27450c0626a7161646e2d20d3f6d
https://github.com/rust-lang/llvm-project/commit/5cf66653736ba2265152ae5e8a409056d8337a06
https://github.com/rust-lang/llvm-project/commit/c20a5767f935e55ba2040d48c6d262dcde17c631
One comment is:

// Define _fltused, since we're not linking against the MS C runtime, but use
// floats.
extern "C" int _fltused = 0;

If I understand correctly, rust with target x86_64-unknown-uefi don't link to msvcrt, but use msvc ABI, so symbol _fltused should provided by libcore when target is x86_64-unknown-uefi.

[joshtriplett]

@12101111 Thanks for the analysis!

It sounds like _fltused serves as a marker to make sure people don't use floating-point without initialization. I don't think core should be using floating point internally; shouldn't we only emit a reference to _fltused if the program uses floating-point? (Your example used f32 as the return type of efi_main; if you drop that, does the build still fail?)

[12101111]

This code don't use floating point and build successfully.

#![no_std]
#![no_main]

#[no_mangle]
pub extern "C" fn efi_main() -> i32 {
    0
}

#[panic_handler]
fn panic(_info: &core::panic::PanicInfo) -> ! {
    loop{}
}

But core::fmt does use floating point.
This code don't build:

#![no_std]
#![no_main]
use core::fmt::{Write,Error};

struct Dummy;

impl Write for Dummy {
    fn write_str(&mut self, _s: &str) -> Result<(), Error>{
        Ok(())
    }
}

#[no_mangle]
pub extern "C" fn efi_main() -> i32 {
    write!(Dummy,"Hello, world!").unwrap();
    0
}

#[panic_handler]
fn panic(_info: &core::panic::PanicInfo) -> ! {
    loop{}
}

Edit: lto can remore some floating point usage in libcore.
without lto:

  = note: rust-lld: error: undefined symbol: _fltused
          >>> referenced by libcore-387d67bcba869bf3.rlib(core-387d67bcba869bf3.core.eqirtc3x-cgu.12.rcgu.o)
          >>> referenced by libcore-387d67bcba869bf3.rlib(core-387d67bcba869bf3.core.eqirtc3x-cgu.15.rcgu.o)
          >>> referenced by libcore-387d67bcba869bf3.rlib(core-387d67bcba869bf3.core.eqirtc3x-cgu.5.rcgu.o)
          >>> referenced by libcore-387d67bcba869bf3.rlib(core-387d67bcba869bf3.core.eqirtc3x-cgu.3.rcgu.o)
          >>> referenced by libcore-387d67bcba869bf3.rlib(core-387d67bcba869bf3.core.eqirtc3x-cgu.6.rcgu.o)
          >>> referenced by libcore-387d67bcba869bf3.rlib(core-387d67bcba869bf3.core.eqirtc3x-cgu.10.rcgu.o)
          >>> referenced by libcompiler_builtins-637c246e97e0523a.rlib(compiler_builtins-637c246e97e0523a.compiler_builtins.1b68nlie-cgu.0.rcgu.o)

with lto=true: build successfully.

In another big project, without lto:

= note: rust-lld: error: undefined symbol: _fltused
          >>> referenced by libcore-f5b7a77eb654a8d8.rlib(core-f5b7a77eb654a8d8.core.em6utcdz-cgu.12.rcgu.o)
          >>> referenced by libcore-f5b7a77eb654a8d8.rlib(core-f5b7a77eb654a8d8.core.em6utcdz-cgu.4.rcgu.o)
          >>> referenced by libcore-f5b7a77eb654a8d8.rlib(core-f5b7a77eb654a8d8.core.em6utcdz-cgu.5.rcgu.o)
          >>> referenced by libcore-f5b7a77eb654a8d8.rlib(core-f5b7a77eb654a8d8.core.em6utcdz-cgu.6.rcgu.o)
          >>> referenced by libcompiler_builtins-a3650b8e38116036.rlib(compiler_builtins-a3650b8e38116036.compiler_builtins.6dh2mb61-cgu.15.rcgu.o)
          >>> referenced by libcore-f5b7a77eb654a8d8.rlib(core-f5b7a77eb654a8d8.core.em6utcdz-cgu.3.rcgu.o)
          >>> referenced by libcore-f5b7a77eb654a8d8.rlib(core-f5b7a77eb654a8d8.core.em6utcdz-cgu.10.rcgu.o)
          >>> referenced by libcompiler_builtins-a3650b8e38116036.rlib(compiler_builtins-a3650b8e38116036.compiler_builtins.6dh2mb61-cgu.1.rcgu.o)
          >>> referenced by libcompiler_builtins-a3650b8e38116036.rlib(compiler_builtins-a3650b8e38116036.compiler_builtins.6dh2mb61-cgu.0.rcgu.o)

with lto=true:

  = note: rust-lld: error: undefined symbol: _fltused
          >>> referenced by libcompiler_builtins-a3650b8e38116036.rlib(compiler_builtins-a3650b8e38116036.compiler_builtins.6dh2mb61-cgu.15.rcgu.o)
          >>> referenced by libcompiler_builtins-a3650b8e38116036.rlib(compiler_builtins-a3650b8e38116036.compiler_builtins.6dh2mb61-cgu.1.rcgu.o)

[jyao1]

It is caused by LLVM fixing emission of _fltused for MSVC.
Reference: https://reviews.llvm.org/D56548

The current UEFI BIOS (EDKII - written in C) meets similar issue when it is built with LLVM10.
The solution in UEFI BIOS is to define _fltused symbol in the UEFI base library code.

[jyao1]

https://reviews.llvm.org/D56548 changed the policy to generate _fltused symbol - "It should be emitted when any floating-point operations (including calls) are present in the object, not just when calls to printf/scanf with floating point args are made."

To resolve the x86_64-unknown-uefi issue, we may have below possible options:
1) Let LLVM not generate _fltused.
1.1) Add option to force LLVM not to generate _fltused in MSVC, because we use MSVC tool chain only but not link MSVC library. It seems need update in LLVM project. The benefit is that we can use LLVM MSVC to build other embedded project with NODEFAULTLIB option.
1.2) Use non-MSVC target for UEFI. Not sure if GNU is an option. The default X64 calling convention between MSVC and GNU is different. Not sure if it is configurable.
1.3) Write UEFI RUST program carefully to avoid _fltused. That is very hard. Even simple core::fmt::{write}, *.as_ptr(), or core::ptr::write_bytes() will generate _fltused. Not a practical option.

2) Add _fltused symbol.
2.1) Add target_os specific symbol in rust. (rust repo such as libcore, or compiler-builtins repo.). But I am not sure how to expose it for uefi specific target_os. [cfg(all(target_os = “uefi”))] seems not work because we still use linux or windows toolchain, to build uefi target.
2.2) Add UEFI Intrinsic lib as standalone crate. That seems an easy option. The good thing is that it is extensible, if there are other compiler intrinsics needed in the future. But it may bring some confusing because the intrinsic is not really for UEFI, but for all embedded project without default lib.
2.3) Add symbol in each UEFI rust program, if it does not link to UEFI instrinsic lib.

[joshtriplett]

Ah, it seems unfortunate that using a format function triggers this even if not trying to format floats. (Though the LTO results seem encouraging.) In that case, yeah, I would suggest adding the LLVM option long-term, and until then, I think it makes sense to just define this in core internally.

target_os should work given that you use the uefi target; it defines the target OS as "uefi". (Also, you can just do so for x86_64.)

[oli-obk]

I opened https://github.com/rust-lang/rust/pull/64990 with the _fltused symbol added to libcore