Rust: ICE on nightly: "Forcing query with already existing DepNode."

[oconnor663]

Cargo/rustc version:

$ rustc +nightly --version
rustc 1.37.0-nightly (7cdaffd79 2019-06-05)
$ cargo +nightly --version
cargo 1.37.0-nightly (545f35425 2019-05-23)

Repro:

$ git clone https://github.com/oconnor663/blake2_simd
$ cd blake2_simd/blake2b
$ git rev-parse HEAD
4734a5eb5dad99b54c83bad176b3decdc081bdb1
$ cargo +nightly build
    Updating crates.io index
   Compiling arrayvec v0.4.10
   Compiling nodrop v0.1.13
   Compiling arrayref v0.3.5
   Compiling constant_time_eq v0.1.3
   Compiling blake2b_simd v0.5.1 (/tmp/blake2_simd/blake2b)
thread 'rustc' panicked at 'Forcing query with already existing DepNode.
- query-key: ParamEnvAnd { param_env: ParamEnv { caller_bounds: [], reveal: All, def_id: None }, value: (Const { ty: [u32; 8], val: ByRef(AllocId(1397).0x0, Allocation { bytes: [0, 0, 0, 0, 1, 0, 0, 0, 2, 0, 0, 0, 3, 0, 0, 0, 4, 0, 0, 0, 5, 0, 0, 0, 6, 0, 0, 0, 7, 0, 0, 0], relocations: Relocations(SortedMap { data: [] }), undef_mask: UndefMask { blocks: [4294967295], len: Size { raw: 32 } }, align: Align { pow2: 2 }, mutability: Immutable, extra: () }) }, field[0]) }
- dep-node: const_field(eb99858e4874f291-9a55d263e1a1abdf)', src/librustc/ty/query/plumbing.rs:548:9
note: Run with `RUST_BACKTRACE=1` environment variable to display a backtrace.
error: internal compiler error: unexpected panic
note: the compiler unexpectedly panicked. this is a bug.
note: we would appreciate a bug report: https://github.com/rust-lang/rust/blob/master/CONTRIBUTING.md#bug-reports
note: rustc 1.37.0-nightly (7cdaffd79 2019-06-05) running on x86_64-unknown-linux-gnu
note: compiler flags: -C debuginfo=2 -C incremental --crate-type lib
note: some of the compiler flags provided by cargo are hidden
error: Could not compile `blake2b_simd`.
To learn more, run the command again with --verbose.

[sfackler]

I just ran into this as well.

[mati865]

It's the same as https://github.com/rust-lang/rust/issues/61530
See https://github.com/rust-lang/rust/issues/61530#issuecomment-499522171 for details

[gnzlbg]

Similar error on many stdsimd targets, e.g., aarch64: https://travis-ci.com/rust-lang-nursery/stdsimd/jobs/206228395#L941

thread 'rustc' panicked at 'Forcing query with already existing DepNode.
- query-key: ParamEnvAnd { param_env: ParamEnv { caller_bounds: [], reveal: All, def_id: None }, value: (Const { ty: [u32; 16], val: ByRef(AllocId(3181).0x0, Allocation { bytes: [0, 0, 0, 0, 1, 0, 0, 0, 2, 0, 0, 0, 3, 0, 0, 0, 4, 0, 0, 0, 5, 0, 0, 0, 6, 0, 0, 0, 7, 0, 0, 0, 8, 0, 0, 0, 9, 0, 0, 0, 10, 0, 0, 0, 11, 0, 0, 0, 12, 0, 0, 0, 13, 0, 0, 0, 14, 0, 0, 0, 15, 0, 0, 0], relocations: Relocations(SortedMap { data: [] }), undef_mask: UndefMask { blocks: [18446744073709551615, 0], len: Size { raw: 64 } }, align: Align { pow2: 2 }, mutability: Immutable, extra: () }) }, field[0]) }
- dep-node: const_field(449c8050006f9d1b-207c1d76bc38c220)', src/librustc/ty/query/plumbing.rs:548:9

[pnkfelix]

(It doesn't seem like this is an exact duplicate of #61530, at least not as titled, since #61530 describes a bug that is exposed by using #![feature(stdsimd)], but the crate here on #61594 claims to target stable rust.)

[pnkfelix]

ugh and @eddyb's notes on #61530 indicate that bug is somehow dependent on incremental compilation. fun...

[pnkfelix]

triage: P-high, leaving nominated to discuss relationship with #61530, assigning to self for initial investigation.

[mati865]

@pnkfelix #61530 title is wrong. The issue comes down to using LLVM intrinsics, blake2b_simd works on stable because it's using core::arch which leads to stdsimd.
In #61530 it's done directly but underlying issue remains the same.

[oconnor663]

Re: incremental compilation, the ICE does seem to go away when I build with --release. Didn't notice that before.

[novacrazy]

I'm also getting this when using packed_simd on the latest nightly. Can confirm it goes away with --release, and it is also present with cargo test.

Here is my full stack trace:

thread 'rustc' panicked at 'Forcing query with already existing DepNode.
- query-key: ParamEnvAnd { param_env: ParamEnv { caller_bounds: [], reveal: All, def_id: None }, value: (Const { ty: [u32; 4], val: ByRef(AllocId(4849).0x0, Allocation { bytes: [0, 0, 0, 0, 1, 0, 0, 0, 4, 0, 0, 0, 5, 0, 0, 0], relocations: Relocations(SortedMap { data: [] }), undef_mask: UndefMask { blocks: [65535], len: Size { raw: 16 } }, align: Align { pow2: 2 }, mutability: Immutable, extra: () }) }, field[0]) }
- dep-node: const_field(fa069fcc6f4deb7b-95c6e5e89865a5b4)', src\librustc\ty\query\plumbing.rs:538:9
stack backtrace:
   0: std::sys_common::alloc::realloc_fallback
   1: std::panicking::take_hook
   2: std::panicking::take_hook
   3: <(rustc::hir::def_id::DefId, rustc::hir::def_id::DefId) as rustc::ty::query::keys::Key>::default_span
   4: std::panicking::rust_panic_with_hook
   5: std::panicking::begin_panic_fmt
   6: std::panicking::begin_panic_fmt
   7: <rustc_codegen_llvm::llv$u6d$_..ffi..debuginfo..DISPFlags$u20$as$u20$core..fmt..Debug$GT$::fmt
   8: <rustc_codegen_llvm::debuginfo::metadata::MemberDescription as core::fmt::Debug>::fmt
   9: <rustc_codegen_llvm::llv$u6d$_..ffi..debuginfo..DISPFlags$u20$as$u20$core..fmt..Debug$GT$::fmt
  10: <rustc_codegen_llvm::debuginfo::CrateDebugContext as core::ops::drop::Drop>::drop
  11: <rustc_codegen_llvm::debuginfo::metadata::MemberDescription as core::fmt::Debug>::fmt
  12: <rustc_codegen_llvm::llv$u6d$_..ffi..debuginfo..DISPFlags$u20$as$u20$core..fmt..Debug$GT$::fmt
  13: <rustc_codegen_llvm::debuginfo::CrateDebugContext as core::ops::drop::Drop>::drop
  14: <rustc_codegen_llvm::debuginfo::CrateDebugContext as core::ops::drop::Drop>::drop
  15: <rustc_codegen_llvm::llv$u6d$_..ffi..PassKind$u20$as$u20$core..fmt..Debug$GT$::fmt
  16: <rustc_codegen_llvm::back::lto::Linker as core::ops::drop::Drop>::drop
  17: <rustc_codegen_llvm::base::ValueIter as core::iter::traits::iterator::Iterator>::next
  18: <rustc_codegen_llvm::llv$u6d$_..OperandBundleDef$u20$as$u20$core..ops..drop..Drop$GT$::drop
  19: <rustc_codegen_llvm::base::ValueIter as core::iter::traits::iterator::Iterator>::next
  20: <rustc_codegen_llvm::llv$u6d$_..ffi..PassKind$u20$as$u20$core..fmt..Debug$GT$::fmt
  21: <rustc_codegen_llvm::LlvmCodegenBackend as rustc_codegen_utils::codegen_backend::CodegenBackend>::codegen_crate
  22: <rustc_interface::profile::trace::Query as core::fmt::Debug>::fmt
  23: rustc_interface::passes::BoxedResolver::to_expansion_result
  24: <rustc_interface::proc_macro_decls::Finder as rustc::hir::itemlikevisit::ItemLikeVisitor>::visit_trait_item
  25: rustc_interface::passes::BoxedResolver::complete
  26: <rustc_interface::profile::trace::Query as core::fmt::Debug>::fmt
  27: rustc_interface::passes::BoxedResolver::to_expansion_result
  28: <rustc_interface::proc_macro_decls::Finder as rustc::hir::itemlikevisit::ItemLikeVisitor>::visit_trait_item
  29: rustc_interface::queries::<impl rustc_interface::interface::Compiler>::ongoing_codegen
  30: <env_logger::filter::inner::Filter as core::fmt::Display>::fmt
  31: <rustc_driver::pretty::UserIdentifiedItem as core::fmt::Debug>::fmt
  32: <env_logger::filter::inner::Filter as core::fmt::Display>::fmt
  33: <rustc_driver::pretty::UserIdentifiedItem as core::fmt::Debug>::fmt
  34: _rust_maybe_catch_panic
  35: <env_logger::fmt::WriteStyle as core::default::Default>::default
  36: ZN244_$LT$std..error..$LT$impl$u20$core..convert..From$LT$alloc..string..String$GT$$u20$for$u20$alloc..boxed..Box$LT$dyn$u20$std..error..Error$u2b$core..marker..Sync$u2b$core..marker..Send$GT$$GT$..from..StringError$u20$as$u20$core..fmt..Display$GT$3fmt17
  37: std::sys::windows::thread::Thread::new
  38: BaseThreadInitThunk
  39: RtlUserThreadStart
query stack during panic:
end of query stack

error: internal compiler error: unexpected panic

note: the compiler unexpectedly panicked. this is a bug.
note: we would appreciate a bug report: https://github.com/rust-lang/rust/blob/master/CONTRIBUTING.md#bug-reports

note: rustc 1.37.0-nightly (b25ee6449 2019-06-17) running on x86_64-pc-windows-msvc
note: compiler flags: -C debuginfo=2 -C incremental --crate-type lib
note: some of the compiler flags provided by cargo are hidden

[novacrazy]

Addendum: The bytes: [0, 0, 0, 0, 1, 0, 0, 0, 4, 0, 0, 0, 5, 0, 0, 0] part appears to correspond to a shuffle!(value, [0, 1, 4, 5] packed_simd operation.

[novacrazy]

The matrix inverse example from packed_simd triggers this. In fact, this code is what seems to trigger it in my application.

Isolated mixed source from mine and packed_simd's:

#[macro_use]
extern crate packed_simd;

use packed_simd::{f32x4, f32x16, f32x8};

pub struct Matrix4(pub f32x16);

impl Matrix4 {
    pub fn inverse(&self) -> Option<Matrix4> {
        let m_0: f32x4 = shuffle!(self.0, [0, 1, 2, 3]);
        let m_1: f32x4 = shuffle!(self.0, [4, 5, 6, 7]);
        let m_2: f32x4 = shuffle!(self.0, [8, 9, 10, 11]);
        let m_3: f32x4 = shuffle!(self.0, [12, 13, 14, 15]);

        let tmp1: f32x4 = shuffle!(m_0, m_1, [0, 1, 4, 5]);
        let row1: f32x4 = shuffle!(m_2, m_3, [0, 1, 4, 5]);

        let row0 = shuffle!(tmp1, row1, [0, 2, 4, 6]);
        let row1: f32x4 = shuffle!(row1, tmp1, [1, 3, 5, 7]);

        let tmp1: f32x4 = shuffle!(m_0, m_1, [2, 3, 6, 7]);
        let row3: f32x4 = shuffle!(m_2, m_3, [2, 3, 6, 7]);
        let row2 = shuffle!(tmp1, row3, [0, 2, 4, 6]);
        let row3 = shuffle!(row3, tmp1, [1, 3, 5, 7]);

        let tmp1: f32x4 = row2 * row3;
        let tmp1 = shuffle!(tmp1, [1, 0, 3, 2]);

        let minor0 = row1 * tmp1;
        let minor1 = row0 * tmp1;
        let tmp1 = shuffle!(tmp1, [2, 3, 0, 1]);
        let minor0 = (row1 * tmp1) - minor0;
        let minor1 = (row0 * tmp1) - minor1;
        let minor1 = shuffle!(minor1, [2, 3, 0, 1]);

        let tmp1 = row1 * row2;
        let tmp1 = shuffle!(tmp1, [1, 0, 3, 2]);
        let minor0 = (row3 * tmp1) + minor0;
        let minor3 = row0 * tmp1;
        let tmp1 = shuffle!(tmp1, [2, 3, 0, 1]);

        let minor0 = minor0 - row3 * tmp1;
        let minor3 = row0 * tmp1 - minor3;
        let minor3 = shuffle!(minor3, [2, 3, 0, 1]);

        let tmp1 = row3 * shuffle!(row1, [2, 3, 0, 1]);
        let tmp1 = shuffle!(tmp1, [1, 0, 3, 2]);
        let row2 = shuffle!(row2, [2, 3, 0, 1]);
        let minor0 = row2 * tmp1 + minor0;
        let minor2 = row0 * tmp1;
        let tmp1 = shuffle!(tmp1, [2, 3, 0, 1]);
        let minor0 = minor0 - row2 * tmp1;
        let minor2 = row0 * tmp1 - minor2;
        let minor2 = shuffle!(minor2, [2, 3, 0, 1]);

        let tmp1 = row0 * row1;
        let tmp1 = shuffle!(tmp1, [1, 0, 3, 2]);
        let minor2 = minor2 + row3 * tmp1;
        let minor3 = row2 * tmp1 - minor3;
        let tmp1 = shuffle!(tmp1, [2, 3, 0, 1]);
        let minor2 = row3 * tmp1 - minor2;
        let minor3 = minor3 - row2 * tmp1;

        let tmp1 = row0 * row3;
        let tmp1 = shuffle!(tmp1, [1, 0, 3, 2]);
        let minor1 = minor1 - row2 * tmp1;
        let minor2 = row1 * tmp1 + minor2;
        let tmp1 = shuffle!(tmp1, [2, 3, 0, 1]);
        let minor1 = row2 * tmp1 + minor1;
        let minor2 = minor2 - row1 * tmp1;

        let tmp1 = row0 * row2;
        let tmp1 = shuffle!(tmp1, [1, 0, 3, 2]);
        let minor1 = row3 * tmp1 + minor1;
        let minor3 = minor3 - row1 * tmp1;
        let tmp1 = shuffle!(tmp1, [2, 3, 0, 1]);
        let minor1 = minor1 - row3 * tmp1;
        let minor3 = row1 * tmp1 + minor3;

        let det = row0 * minor0;
        let det = shuffle!(det, [2, 3, 0, 1]) + det;
        let det = shuffle!(det, [1, 0, 3, 2]) + det;

        if det.sum() == 0.0 {
            return None;
        }

        let tmp1 = det.recpre();
        let det = tmp1 + tmp1 - det * tmp1 * tmp1;

        let res0 = minor0 * det;
        let res1 = minor1 * det;
        let res2 = minor2 * det;
        let res3 = minor3 * det;

        let m0: f32x8 = shuffle!(res0, res1, [0, 1, 2, 3, 4, 5, 6, 7]);
        let m1: f32x8 = shuffle!(res2, res3, [0, 1, 2, 3, 4, 5, 6, 7]);

        Some(Matrix4(shuffle!(m0, m1, [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15])))
    }
}

fn main() {
    let _v = Matrix4(f32x16::splat(0.0)).inverse();
}

[eddyb]

Pretty sure this is a dupe of #61530 (despite its unfortunate name).
@oli-obk has opened a tentative fix at #61959.

[oconnor663]

Fix a hash collision issue

Under the circumstances (an ICE in the blake2_simd repo) that's pretty funny.

[eddyb]

(I guess the title could be better, this is not an actual "hash collision", just Eq and StableHash not agreeing on what is compared/hashed)

[oli-obk]

The fix-PR has been merged, can you try again with tomorrow's nightly?

[oconnor663]

Working for me now! 929b48ec9 2019-06-21