Rust: Recompiling causes panic

[mattrutherford]

Making changes to code in modules that compile to WASM and running cargo build --release results in a panic.

This is the project: https://github.com/paritytech/substrate

Immediately running cargo clean && cargo build --release will allow successful compilation. Also upgrading to a new version of nightly will allow recompilation without cargo clean, but only the first recompile.

Meta

rustc --version --verbose:

rustc 1.38.0-nightly (6a91782b7 2019-08-06) running on x86_64-apple-darwin

Backtrace:

Fresh transaction-factory v0.0.1 (/Users/matt/projects/substrate/test-utils/transaction-factory)
error: failed to run custom build command for `node-runtime v2.0.0 (/Users/matt/projects/substrate/node/runtime)`

Caused by:
  process didn't exit successfully: `/Users/matt/projects/substrate/target/release/build/node-runtime-777af245baf20274/build-script-build` (exit code: 1)
--- stdout
Executing build command: "rustup" "run" "nightly" "cargo" "build" "--target=wasm32-unknown-unknown" "--manifest-path=/Users/matt/projects/substrate/target/release/wbuild/node-runtime/Cargo.toml" "--release"

--- stderr
   Compiling wasm-build-runner-impl v1.0.0 (/Users/matt/projects/substrate/target/release/build/node-runtime-3547c858aedc1441/out/wasm_build_runner)
    Finished release [optimized] target(s) in 0.25s
     Running `/Users/matt/projects/substrate/target/release/build/node-runtime-3547c858aedc1441/out/wasm_build_runner/target/release/wasm-build-runner-impl`
   Compiling node-runtime-wasm v1.0.0 (/Users/matt/projects/substrate/target/release/wbuild/node-runtime)
thread '<unnamed>' panicked at 'must have at least one serialized module', src/libcore/option.rs:1166:5
stack backtrace:
   0:        0x109957f12 - std::panicking::default_hook::{{closure}}::hbefba8356e31122b
   1:        0x109957bdd - std::panicking::default_hook::hb85a284cef21dddc
   2:        0x1085b55a3 - rustc::util::common::panic_hook::h8e13af8cece077e3
   3:        0x1099587d1 - std::panicking::rust_panic_with_hook::h8868b29516eb168e
   4:        0x10995822d - std::panicking::continue_panic_fmt::h8f85da70e3d42ed7
   5:        0x109958119 - rust_begin_unwind
   6:        0x109983e82 - core::panicking::panic_fmt::ha880b2fe6921e68e
   7:        0x109983f9e - core::option::expect_failed::hfea1a2f203365264
   8:        0x10bb851f8 - rustc_codegen_llvm::back::lto::run_fat::h0e6fc34b8aa41a0c
   9:        0x10bc04aa8 - rustc_codegen_ssa::back::write::start_executing_work::{{closure}}::h1c543b9ea0992759
  10:        0x10bc0b7d0 - std::sys_common::backtrace::__rust_begin_short_backtrace::h6b423e28f14e4e74
  11:        0x10bb5279c - std::panicking::try::do_call::h6fca1ea505f089a9
  12:        0x109967b4f - __rust_maybe_catch_panic
  13:        0x10bb52ba6 - core::ops::function::FnOnce::call_once{{vtable.shim}}::h6eb73fcc38df565a
  14:        0x10993bb1e - <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once::h2a90a69cfe4e95fa
  15:        0x10996695e - std::sys::unix::thread::Thread::new::thread_start::h99953f013e830b09
  16:     0x7fff7c8392eb - _pthread_body
  17:     0x7fff7c83c249 - _pthread_start
query stack during panic:
end of query stack
thread 'rustc' panicked at 'src/librustc_codegen_ssa/back/write.rs:1826: panic during codegen/LLVM phase', src/librustc/util/bug.rs:37:26
stack backtrace:
   0:        0x109957f12 - std::panicking::default_hook::{{closure}}::hbefba8356e31122b
   1:        0x109957bdd - std::panicking::default_hook::hb85a284cef21dddc
   2:        0x1085b55a3 - rustc::util::common::panic_hook::h8e13af8cece077e3
   3:        0x1099587d1 - std::panicking::rust_panic_with_hook::h8868b29516eb168e
   4:        0x1085f9b25 - std::panicking::begin_panic::he8bde94d32f67181
   5:        0x10828fbb4 - rustc::util::bug::opt_span_bug_fmt::{{closure}}::h79b523f376a07816
   6:        0x10828f9ea - rustc::ty::context::tls::with_opt::{{closure}}::heef7fe9845124b11
   7:        0x10828f777 - rustc::ty::context::tls::with_context_opt::h61634ca6ab55ac5a
   8:        0x10828f7c2 - rustc::ty::context::tls::with_opt::h5ca54a02c4c0bb9b
   9:        0x10828faf8 - rustc::util::bug::opt_span_bug_fmt::h8f8a983ed3203bff
  10:        0x10828fa4b - rustc::util::bug::bug_fmt::h7fff681b74edf248
  11:        0x10bc09e05 - rustc_codegen_ssa::back::write::OngoingCodegen<B>::join::hd601e779fa22708f
  12:        0x10bc43d98 - <rustc_codegen_llvm::LlvmCodegenBackend as rustc_codegen_utils::codegen_backend::CodegenBackend>::join_codegen_and_link::hb5084914450219e0
  13:        0x1070296d2 - rustc_interface::queries::Query<T>::compute::h79dc64dd1d20cdc1
  14:        0x1070ede3c - rustc_interface::queries::<impl rustc_interface::interface::Compiler>::link::hc53a9984fbe3c770
  15:        0x106f5d224 - rustc_interface::interface::run_compiler_in_existing_thread_pool::hbcb4433730475c4a
  16:        0x106f90744 - std::thread::local::LocalKey<T>::with::hfc60cc57f753f9b0
  17:        0x106fa6228 - syntax::with_globals::h974a79eedc8d66fd
  18:        0x106f381a7 - std::sys_common::backtrace::__rust_begin_short_backtrace::ha8d874bf2a6ac95b
  19:        0x109967b4f - __rust_maybe_catch_panic
  20:        0x106f5eeb7 - core::ops::function::FnOnce::call_once{{vtable.shim}}::hf166e26f2620097b
  21:        0x10993bb1e - <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once::h2a90a69cfe4e95fa
  22:        0x10996695e - std::sys::unix::thread::Thread::new::thread_start::h99953f013e830b09
  23:     0x7fff7c8392eb - _pthread_body
  24:     0x7fff7c83c249 - _pthread_start
query stack during panic:
end of query stack

error: internal compiler error: unexpected panic

note: the compiler unexpectedly panicked. this is a bug.

note: we would appreciate a bug report: https://github.com/rust-lang/rust/blob/master/CONTRIBUTING.md#bug-reports

note: rustc 1.38.0-nightly (6a91782b7 2019-08-06) running on x86_64-apple-darwin

note: compiler flags: -C opt-level=3 -C panic=abort -C lto -C incremental -C link-arg=--export-table -C link-arg=--export=__heap_base --crate-type cdylib

note: some of the compiler flags provided by cargo are hidden

error: Could not compile `node-runtime-wasm`.

[lzutao]

Did you use sccache or anything similar?

[mattrutherford]

not locally where I'm experiencing the problem

[nagisa]

Assigning @michaelwoerister to investigate, there is very little else to do here.

[michaelwoerister]

I have not been able to reproduce this yet. @mattrutherford, would you mind providing the exact commands you are running and the set of source files you are modifying?

[mattrutherford]

yes, ok steps I can take to reproduce:

1. make a fresh clone of: https://github.com/paritytech/substrate
2. cargo build --release
3. Make change in some rust file in srml, such as: https://github.com/paritytech/substrate/blob/master/srml/support/src/dispatch.rs
4. cargo build --release

gives the error

[hellow554]

@mattrutherford "Make change " can you specify it please? Where did you change what? A diff would be good (e.g. if you happen to have a cloned repo, then you can do git diff and show it to us).

[michaelwoerister]

I can reproduce on Fedora 30 when also setting CARGO_INCREMENTAL=1 in the environment. Thanks, @mattrutherford.

It's probably a buggy interaction incremental compilation and LTO (cc @alexcrichton). Not sure if it is wasm specific.

[alexcrichton]

Hm I was getting build errors locally and wasn't able to reproduce. I'm not really sure what compiler is necessary to fix errors like:

error[E0432]: unresolved import `alloc_crate::collections::CollectionAllocErr`
  --> /home/alex/.cargo/registry/src/github.com-1ecc6299db9ec823/hashmap_core-0.1.11/src/lib.rs:32:13
   |
32 |     pub use alloc_crate::collections::CollectionAllocErr;
   |             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ no `CollectionAllocErr` in `collections`

   Compiling srml-support-procedural-tools v2.0.0 (/home/alex/code/substrate/srml/support/procedural/tools)

[mattrutherford]

Hm I was getting build errors locally and wasn't able to reproduce. I'm not really sure what compiler is necessary to fix errors like:

error[E0432]: unresolved import `alloc_crate::collections::CollectionAllocErr`
  --> /home/alex/.cargo/registry/src/github.com-1ecc6299db9ec823/hashmap_core-0.1.11/src/lib.rs:32:13
   |
32 |     pub use alloc_crate::collections::CollectionAllocErr;
   |             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ no `CollectionAllocErr` in `collections`

   Compiling srml-support-procedural-tools v2.0.0 (/home/alex/code/substrate/srml/support/procedural/tools)

I think unfortunately you tried at a time where master was broken! If you can try again, maybe it's possible to reproduce the original problem

[alexcrichton]

Ok looks like this is actually pretty trivial to reproduce:

$ rustc +nightly foo.rs --crate-type cdylib -Copt-level=3 -C lto -C incremental=wut
$ rustc +nightly foo.rs --crate-type cdylib -Copt-level=3 -C lto -C incremental=wut
thread '<unnamed>' panicked at 'must have at least one serialized module', src/libcore/option.rs:1166:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace.
thread 'rustc' panicked at 'src/librustc_codegen_ssa/back/write.rs:1823: panic during codegen/LLVM phase', src/librustc/util/bug.rs:37:26

error: internal compiler error: unexpected panic

note: the compiler unexpectedly panicked. this is a bug.

note: we would appreciate a bug report: https://github.com/rust-lang/rust/blob/master/CONTRIBUTING.md#bug-reports

note: rustc 1.39.0-nightly (521d78407 2019-08-25) running on x86_64-unknown-linux-gnu

note: compiler flags: -C opt-level=3 -C lto -C incremental --crate-type cdylib

[alexcrichton]

Should be fixed by https://github.com/rust-lang/rust/pull/63956