Rust: Comparing to infinity is buggy on x87

[eduardosm]

When code is compiled using the X87 FPU, comparing to infinity can lead to unexpected behavior.

I tried this code:

#[inline(never)]
fn get_num() -> f64 {
    let num: f64 = 1.0e300;
    // volatile is to avoid optimizations
    unsafe { std::ptr::read_volatile(&num) }
}

fn main() {
    let x = get_num();
    let y = get_num();
    let z = x * y;
    if z != f64::INFINITY && z != f64::NEG_INFINITY && !z.is_nan() {
        let exp = (z.to_bits() >> 52) & 0x7FF;
        assert!(exp != 0x7FF);
        println!("is finite, exp = {}", exp);
    } else {
        println!("is not finite");
    }
}

I expected to see this happen:

This should either print "infinite or nan" or "is finite, exp = ...". The assert! should never fail because the exponent bits are 0x7FF if and only if the number is infinity or nan, so the if condition should have been false.

Instead, this happened:

When compiled for X86 without SSE, the assert will fail. This could also cause safety issues if unsafe code relies on it.

The following command can be used to compile without SSE:

RUSTFLAGS="-C target-cpu=pentium" cargo run --target i686-unknown-linux-gnu

Meta

rustc --version --verbose:

rustc 1.43.1 (8d69840ab 2020-05-04)
binary: rustc
commit-hash: 8d69840ab92ea7f4d323420088dd8c9775f180cd
commit-date: 2020-05-04
host: x86_64-unknown-linux-gnu
release: 1.43.1
LLVM version: 9.0

and

rustc 1.45.0-nightly (a74d1862d 2020-05-14)
binary: rustc
commit-hash: a74d1862d4d87a56244958416fd05976c58ca1a8
commit-date: 2020-05-14
host: x86_64-unknown-linux-gnu
release: 1.45.0-nightly
LLVM version: 9.0

[hanna-kruppe]

For reference, can you post the machine code this results in? I'm particularly curious if there's any calls to functions already built in the distributed libstd, since those would potentially suffer from ABI mismatches (see https://github.com/rust-lang/rust/issues/63597) that could explain the weird behavior.

[eduardosm]

For reference, can you post the machine code this results in? I'm particularly curious if there's any calls to functions already built in the distributed libstd, since those would potentially suffer from ABI mismatches (see #63597) that could explain the weird behavior.

Sure, the assembly of the main function is (after removing the println!s to simplify it):

_ZN10float_test4main17h3d6b0b215bf83903E:
    .cfi_startproc
    pushl   %ebx
    .cfi_def_cfa_offset 8
    subl    $24, %esp
    .cfi_def_cfa_offset 32
    .cfi_offset %ebx, -8
    calll   .L9$pb
    .cfi_adjust_cfa_offset 4
.L9$pb:
    popl    %ebx
    .cfi_adjust_cfa_offset -4
.Ltmp6:
    addl    $_GLOBAL_OFFSET_TABLE_+(.Ltmp6-.L9$pb), %ebx
    calll   _ZN10float_test7get_num17hedf1e4650ddf3052E
    fstpl   16(%esp)
    calll   _ZN10float_test7get_num17hedf1e4650ddf3052E
    fldl    16(%esp)
    fmulp   %st, %st(1)
    fucom   %st(0)
    fnstsw  %ax
    sahf
    jp  .LBB9_5
    flds    .LCPI9_0@GOTOFF(%ebx)
    fucomp  %st(1)
    fnstsw  %ax
    sahf
    jae .LBB9_5
    flds    .LCPI9_1@GOTOFF(%ebx)
    fxch    %st(1)
    fucom   %st(1)
    fstp    %st(1)
    fnstsw  %ax
    sahf
    jae .LBB9_5
    fstpl   8(%esp)
    movl    12(%esp), %eax
    notl    %eax
    testl   $2146435072, %eax
    fldz
    je  .LBB9_4
.LBB9_5:
    fstp    %st(0)
    addl    $24, %esp
    .cfi_def_cfa_offset 8
    popl    %ebx
    .cfi_def_cfa_offset 4
    retl
.LBB9_4:
    .cfi_def_cfa_offset 32
    fstp    %st(0)
    calll   _ZN3std9panicking11begin_panic17h261cc8b487132e56E
    ud2

...

.LCPI9_0:
    .long   4286578688
.LCPI9_1:
    .long   2139095040

Generated with:

RUSTFLAGS="-C target-cpu=pentium" cargo rustc --release --target i686-unknown-linux-gnu -- --emit asm

I think I forgot to mention in the OP that this happens in both debug and release mode.

[RalfJung]

Uh-oh... should this be labelled "unsound"? Any time codegen'd behavior does not match the spec, that can be considered a soundness issue as well.

[eduardosm]

I just found out that this can be reproduced without RUSTFLAGS="-C target-cpu=pentium" using --target i586-unknown-linux-gnu .

[ecstatic-morse]

I was searching for open floating point issues related to #73328 and stumbled upon this one, which seems to have slipped through the cracks. Marking as "unsound" and "needs prioritization". I'm still not sure exactly what causes this.

cc @rust-lang/wg-prioritization

[ranma42]

I believe this happens because x87 internally works on 80-bits numbers.
This format can handle numbers up to about 10^4932, which means that the result of the multiplication is not infinity (when it is on the FP stack), hence the comparisons do not succeed.
On the other hand, the code that extracts the exponent casts it to 64-bits, so the number is rounded (truncated?) to the positive infinity.

[ranma42]

In a way, this is what is happening behind the scenes:

#[inline(never)]
fn get_num() -> f64 {
    let num: f64 = 1.0e30;
    // volatile is to avoid optimizations
    unsafe { std::ptr::read_volatile(&num) }
}

fn main() {
    let x = get_num();
    let y = get_num();
    let z = x * y;
    if z != f64::INFINITY && z != f64::NEG_INFINITY && !z.is_nan() {
        let exp = ((z as f32).to_bits() >> 23) & 0x7F;
        assert!(exp != 0x7F);
        println!("is finite, exp = {}", exp);
    } else {
        println!("is not finite");
    }
}

Note that in this case the downcast as f32 is explicit.

[ecstatic-morse]

Indeed. The underlying cause is clear. I wonder what we should do here, though? Does Rust currently guarantee that extended precision is not used for operations on f64? If so, this is technically a miscompilation. However, I don't know whether it's worth fixing. Maybe we should just document the status quo and move on?

[RalfJung]

ecstatic-morse added I-prioritize I-unsound boom labels

Shouldn't we rather tag https://github.com/rust-lang/rust/issues/73328 instead of tagging the 4 issues that are all instances of the same problem?

[ecstatic-morse]

This is wholly distinct from #73328. There's no NaN anywhere in this program.

[RalfJung]

Oh, good point. I had missed that.

So this is truly an i686-only artifact caused by using the x87 instructions, and has nothing to do with LLVM's sloppiness around NaNs?

[RalfJung]

Could we argue that this is a bug in LLVM, or is there some statement in LLVM's LangRef that actually makes this a correct compilation of the LLVM IR rustc produces?

[ecstatic-morse]

Not sure. The LLVM reference says:

The binary format of half, float, double, and fp128 correspond to the IEEE-754-2008 specifications for binary16, binary32, binary64, and binary128 respectively.

I don't have a copy of the IEEE spec, so I don't know if sharing a "binary format" implies that they must not use a higher precision for intermediate results. I think that the IEEE-754 may explicitly allow for this, however? In any case, it's unlikely that LLVM will guarantee the same precision everywhere as long as x87 instructions are supported. C has had the same issue for 20 years now. The GCC wiki discusses what would be required to make floating point math for double predictably 64-bit on x87. Presumably this kind of approach was also considered by LLVM and rejected.

As long as 32-bit x86 is a tier 1 target, I believe we won't be able to guarantee at the language level that floating point math is always performed at 64-bit precision for f64 (same goes for f32). Instead, we will have to say something like "the precision of intermediate floating point calculations is platform-defined but rustc guarantees that all tier 1 platforms besides i686 use the same precision as the floating-point type for all computations".

[RalfJung]

As long as 32-bit x86 is a tier 1 target, I believe we won't be able to guarantee at the language level that floating point math is always performed at 64-bit precision for f64 (same goes for f32).

We could require SSE. (No idea if that is even remotely realistic, but it felt worth mentioning.)

[ecstatic-morse]

The original SSE only supported single-precision floating point arithmetic. You need at least SSE2 for the semantics we want. I believe that all i686 processors have SSE but not all have SSE2. Not sure what we do by default here?

But yes, I think that would be fine.

[hanna-kruppe]

FWIW, the target triples i686-* are a misnomer, they include SSE2 already.

[comex]

See also: https://github.com/rust-lang/rfcs/pull/2686 (“Allow floating-point operations to provide extra precision than specified, as an optimization”)

[RalfJung]

@hanna-kruppe Ah, that's why the OP had to do RUSTFLAGS="-C target-cpu=pentium".
Can we declare non-SSE2-i686 "unsupported" to "fix" the problem here? I am not sure what that would actually mean in practice, maybe error or at least warn when SSE2 is disabled, or so. Without proper hardware support, it seems hard to do anything better, and if it's a warning people can still proceed with caution.

@comex there were many concerns in that RFC about enabling such optimizations by default (https://github.com/rust-lang/rfcs/pull/2686#discussion_r276418997, https://github.com/rust-lang/rfcs/pull/2686#discussion_r276421551). Also, would that RFC really lead to problems such as this? f64 is still a 64-bit type, so the assertion here should still hold, right? The issue arises because f64 is actually represented as 80bits on x87 and thus there are more possible bit patterns than there should be?

[RalfJung]

Is this related to or a duplicate of https://github.com/rust-lang/rust/issues/73288? Both are about x87 floating point weirdness, from what I can tell.