rust 🚀 - Segfault compiling libc on armv7-unknown-linux-gnueabihf

@jonas-schievink this is also a regression with respect to stable Rust, so we probably need to avoid shipping a beta with this issue.

gnzlbg on 23 Jul 2019

Can you obtain backtrace using gdb or lldb?

mati865 on 23 Jul 2019

cc @nikic, this might also be fallout from the LLVM update

jonas-schievink on 23 Jul 2019

@mati865 is there a guide I can use for this?

fenhl on 24 Jul 2019

@fenhl

$ gdb --args cargo check --release

r (enter)

bt (enter)

q (enter)

and paste the output.

mati865 on 24 Jul 2019

GNU gdb (Raspbian 8.2.1-2) 8.2.1
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "arm-linux-gnueabihf".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from cargo...done.
(gdb) r
Starting program: /home/fenhl/.cargo/bin/cargo check --release
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/arm-linux-gnueabihf/libthread_db.so.1".
process 16341 is executing new program: /home/fenhl/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/bin/cargo
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/arm-linux-gnueabihf/libthread_db.so.1".
[New Thread 0x76d512f0 (LWP 16344)]
   Compiling libc v0.2.60
[New Thread 0x769fe2f0 (LWP 16345)]
[Detaching after fork from child process 16346]                          ] 0/4: libc(build.rs)                                                                                              
[Thread 0x769fe2f0 (LWP 16345) exited]

Thread 2 "cargo" received signal SIGUSR1, User defined signal 1.
[Switching to Thread 0x76d512f0 (LWP 16344)]
futex_wait_cancelable (private=0, expected=0, futex_word=0xe77938) at ../sysdeps/unix/sysv/linux/futex-internal.h:88
88  ../sysdeps/unix/sysv/linux/futex-internal.h: No such file or directory.
(gdb) bt
#0  futex_wait_cancelable (private=0, expected=0, futex_word=0xe77938) at ../sysdeps/unix/sysv/linux/futex-internal.h:88
#1  __pthread_cond_wait_common (abstime=0x0, mutex=0x0, cond=0xe77910) at pthread_cond_wait.c:502
#2  __pthread_cond_wait (cond=0xe77910, mutex=0x0) at pthread_cond_wait.c:655
#3  0x00bd0900 in std::sys::unix::condvar::Condvar::wait () at src/libstd/sys/unix/condvar.rs:69
#4  std::sys_common::condvar::Condvar::wait () at src/libstd/sys_common/condvar.rs:41
#5  std::sync::condvar::Condvar::wait () at src/libstd/sync/condvar.rs:204
#6  std::thread::park () at src/libstd/thread/mod.rs:911
#7  0x00bde4b4 in std::sync::mpsc::blocking::WaitToken::wait () at src/libstd/sync/mpsc/blocking.rs:71
#8  0x00baf508 in std::sync::mpsc::oneshot::Packet<T>::recv ()
#9  0x00baddb8 in std::sync::mpsc::Receiver<T>::recv ()
#10 0x00bb2d38 in std::sys_common::backtrace::__rust_begin_short_backtrace ()
#11 0x00bb1b44 in std::panicking::try::do_call ()
#12 0x00bebf88 in __rust_maybe_catch_panic () at src/libpanic_unwind/lib.rs:82
#13 0x00bb2110 in core::ops::function::FnOnce::call_once{{vtable-shim}} ()
#14 0x00bcfb74 in <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once () at /rustc/a7f28678bbf4e16893bb6a718e427504167a9494/src/liballoc/boxed.rs:766
#15 0x00beb48c in <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once () at /rustc/a7f28678bbf4e16893bb6a718e427504167a9494/src/liballoc/boxed.rs:766
#16 std::sys_common::thread::start_thread () at src/libstd/sys_common/thread.rs:13
#17 std::sys::unix::thread::Thread::new::thread_start () at src/libstd/sys/unix/thread.rs:79
#18 0x76f56494 in start_thread (arg=0x76d512f0) at pthread_create.c:486
Backtrace stopped: Cannot access memory at address 0x1ec9a
(gdb) q
A debugging session is active.

    Inferior 1 [process 16341] will be killed.

Quit anyway? (y or n) y

fenhl on 24 Jul 2019

cc @nikic - suspicious that this was reported right after the llvm 9 upgrade

gnzlbg on 24 Jul 2019

@fenhl gdb haven't detected failed thread. Could you try again with this?

$ gdb --args cargo check --release

r (enter)

set pagination off (enter)

thread apply all bt (enter)

mati865 on 24 Jul 2019

Thread 2 (Thread 0x76d512f0 (LWP 19622)):
#0  __GI___tls_get_addr (ti=0xdc38b4) at dl-tls.c:828
#1  0x00bd0a64 in core::option::Option<T>::as_ref () at /rustc/a7f28678bbf4e16893bb6a718e427504167a9494/src/libcore/option.rs:266
#2  std::thread::local::lazy::LazyKeyInner<T>::get () at src/libstd/thread/local.rs:279
#3  std::thread::local::fast::Key<T>::get () at src/libstd/thread/local.rs:409
#4  std::panicking::update_panic_count::PANIC_COUNT::__getit () at src/libstd/thread/local.rs:176
#5  std::thread::local::LocalKey<T>::try_with () at src/libstd/thread/local.rs:254
#6  std::thread::local::LocalKey<T>::with () at src/libstd/thread/local.rs:234
#7  std::panicking::update_panic_count () at src/libstd/panicking.rs:226
#8  std::panicking::panicking () at src/libstd/panicking.rs:303
#9  std::thread::panicking () at src/libstd/thread/mod.rs:724
#10 std::sys_common::poison::Flag::done () at src/libstd/sys_common/poison.rs:36
#11 <std::sync::mutex::MutexGuard<T> as core::ops::drop::Drop>::drop () at src/libstd/sync/mutex.rs:448
#12 core::ptr::real_drop_in_place () at /rustc/a7f28678bbf4e16893bb6a718e427504167a9494/src/libcore/ptr/mod.rs:175
#13 std::thread::park () at src/libstd/thread/mod.rs:917
#14 0x00bde4b4 in std::sync::mpsc::blocking::WaitToken::wait () at src/libstd/sync/mpsc/blocking.rs:71
#15 0x00baf508 in std::sync::mpsc::oneshot::Packet<T>::recv ()
#16 0x00baddb8 in std::sync::mpsc::Receiver<T>::recv ()
#17 0x00bb2d38 in std::sys_common::backtrace::__rust_begin_short_backtrace ()
#18 0x00bb1b44 in std::panicking::try::do_call ()
#19 0x00bebf88 in __rust_maybe_catch_panic () at src/libpanic_unwind/lib.rs:82
#20 0x00bb2110 in core::ops::function::FnOnce::call_once{{vtable-shim}} ()
#21 0x00bcfb74 in <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once () at /rustc/a7f28678bbf4e16893bb6a718e427504167a9494/src/liballoc/boxed.rs:766
#22 0x00beb48c in <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once () at /rustc/a7f28678bbf4e16893bb6a718e427504167a9494/src/liballoc/boxed.rs:766
#23 std::sys_common::thread::start_thread () at src/libstd/sys_common/thread.rs:13
#24 std::sys::unix::thread::Thread::new::thread_start () at src/libstd/sys/unix/thread.rs:79
#25 0x76f56494 in start_thread (arg=0x76d512f0) at pthread_create.c:486
Backtrace stopped: Cannot access memory at address 0x21046

Thread 1 (Thread 0x76ff4010 (LWP 19619)):
#0  futex_reltimed_wait_cancelable (private=0, reltime=0x7effb8a0, expected=0, futex_word=0xdcd3b8) at ../sysdeps/unix/sysv/linux/futex-internal.h:142
#1  __pthread_cond_wait_common (abstime=0x7effb910, mutex=0x2, cond=0xdcd390) at pthread_cond_wait.c:533
#2  __pthread_cond_timedwait (cond=0xdcd390, mutex=0x2, abstime=0x7effb910) at pthread_cond_wait.c:667
#3  0x00be44bc in std::sys::unix::condvar::Condvar::wait_timeout () at src/libstd/sys/unix/condvar.rs:100
#4  0x00bd0df8 in std::sys_common::condvar::Condvar::wait_timeout () at src/libstd/sys_common/condvar.rs:51
#5  std::sync::condvar::Condvar::wait_timeout () at src/libstd/sync/condvar.rs:405
#6  std::thread::park_timeout () at src/libstd/thread/mod.rs:1006
#7  0x00bde5f4 in std::sync::mpsc::blocking::WaitToken::wait_max_until () at src/libstd/sync/mpsc/blocking.rs:82
#8  0x00baf4d8 in std::sync::mpsc::oneshot::Packet<T>::recv ()
#9  0x00badb40 in std::sync::mpsc::Receiver<T>::recv_timeout ()
#10 0x00bad3d0 in jobserver::imp::Helper::join ()
#11 0x00bb0d70 in <jobserver::HelperThread as core::ops::drop::Drop>::drop ()
#12 0x006f5218 in core::ptr::real_drop_in_place ()
#13 0x00719b20 in cargo::core::compiler::job_queue::JobQueue::execute ()
#14 0x007d9050 in cargo::core::compiler::context::Context::compile ()
#15 0x00550a20 in cargo::ops::cargo_compile::compile_ws ()
#16 0x0054f878 in cargo::ops::cargo_compile::compile ()
#17 0x00490a44 in cargo::commands::check::exec ()
#18 0x0045a184 in cargo::cli::main ()
#19 0x0049b270 in cargo::main ()
#20 0x004787cc in std::rt::lang_start::{{closure}} ()
#21 0x00be2350 in std::rt::lang_start_internal::{{closure}} () at src/libstd/rt.rs:49
#22 std::panicking::try::do_call () at src/libstd/panicking.rs:296
#23 0x00bebf88 in __rust_maybe_catch_panic () at src/libpanic_unwind/lib.rs:82
#24 0x00be2e7c in std::panicking::try () at src/libstd/panicking.rs:275
#25 std::panic::catch_unwind () at src/libstd/panic.rs:394
#26 std::rt::lang_start_internal () at src/libstd/rt.rs:48
#27 0x0049cff8 in main ()

fenhl on 24 Jul 2019

Report from compiler-team pre-triage:

It seems like a good next step would be to bisect the regression to a specific nightly or (better) PR. The cargo-bisect-rustc tool may be of use here!

nikomatsakis on 25 Jul 2019

I'm going to need some help with that because it just said “regression in nightly-2019-07-25” which is obviously wrong. Doing some further testing, it seems the segfault doesn't occur every time cargo check --release is run:

shell session $ while command sleep 5; do cargo clean; if cargo check --release |& grep SIGSEGV; then echo segfault; else echo no segfault; fi; done segfault no segfault segfault segfault segfault segfault segfault segfault segfault segfault segfault no segfault

fenhl on 25 Jul 2019

@fenhl can you manually test if the nightlies before the one from 2019-07-21 also have this issue ? Nailing the first nightly with this problem would be very helpful.

gnzlbg on 26 Jul 2019

Looks like the issue first appears on nightly-2019-07-18 (bc2e84ca0 2019-07-17). On nightly-2019-07-17 (07e0c3651 2019-07-16), the command ran successfully 100 times.

fenhl on 26 Jul 2019

So we have the winner: https://github.com/rust-lang/rust/pull/62592

mati865 on 27 Jul 2019

The most suspicious winner. We should give a “bravest dev of the quarter”
award to whoever updates the LLVM version. Alex would often get this award,
but this time it would be @nikic and i’d Bet Alex would love to not have to
do this kind of thing and instead only work on the wasm stuff that interest
them. Or maybe we should have an LLVM working group whose job is to
regularly keep rustc following the latest LLVM trunk so that bugs in LLVM
upstream are discovered quickly as they happen. Idk.

gnzlbg on 27 Jul 2019

Typically SIGSEGV indicates an LLVM assertion firing. Would be great if a reproduction was attempted with rustc that has LLVM assertions enabled.

Furthermore the backtraces posted above are for cargo, not for the rustc process it spawns. Would be nice to see a backtrace from rustc with LLVM assertions enabled.

nagisa on 1 Aug 2019

@nagisa The stack trace in https://github.com/rust-lang/rust/issues/62896#issuecomment-514766586 indicates a segfault during __GI___tls_get_addr, so I'd assume this is a TLS miscompile.

nikic on 1 Aug 2019

@nagisa I don't really know how to use plain rustc, but I'd be happy to help if instructions are provided.

fenhl on 1 Aug 2019

@nikic from what I can tell that backtrace is taken after the process broke on a signal other than SIGSEGV.

nagisa on 1 Aug 2019

@fenhl run cargo with --verbose (possibly also -j1 to limit concurrency). It will print out rustc commands that it executes. Copy the failing command and run it under gdb with gdb --args <whatever you copied>.

Make sure that when gdb breaks, it breaks on SIGSEGV or some other abnormal signal and not SIGUSR1 or other signals that are otherwise ignored or potentially handled.

Furthermore, it would be ideal that rustc you used would have assertions enabled, but compiling rust and libcore for crosscompilation is gonna be a major pain.

nagisa on 1 Aug 2019

Unfortunately, it looks like I ran into a gdb bug.

fenhl on 4 Aug 2019

You can try lldb as well.

The usage would look something like this:

lldb -- rustc <flags>
(lldb) run
<wait for sigsegv to occur>
(lldb) bt

nagisa on 4 Aug 2019

Seems like I keep running into different issues, but somehow it looks like gdb worked now.

gdb --args rustc --crate-name build_script_build /home/fenhl/.cargo/registry/src/github.com-1ecc6299db9ec823/libc-0.2.60/build.rs --color always --crate-type bin --emit=dep-info,link -C opt-level=3 --cfg 'feature="default"' --cfg 'feature="std"' -C metadata=60aaac3506c3c4f4 -C extra-filename=-60aaac3506c3c4f4 --out-dir /home/fenhl/compile-fail/target/release/build/libc-60aaac3506c3c4f4 -L dependency=/home/fenhl/compile-fail/target/release/deps --cap-lints allow
GNU gdb (Raspbian 8.2.1-2) 8.2.1
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "arm-linux-gnueabihf".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from rustc...done.
(gdb) r
Starting program: /home/fenhl/.cargo/bin/rustc --crate-name build_script_build /home/fenhl/.cargo/registry/src/github.com-1ecc6299db9ec823/libc-0.2.60/build.rs --color always --crate-type bin --emit=dep-info,link -C opt-level=3 --cfg feature=\"default\" --cfg feature=\"std\" -C metadata=60aaac3506c3c4f4 -C extra-filename=-60aaac3506c3c4f4 --out-dir /home/fenhl/compile-fail/target/release/build/libc-60aaac3506c3c4f4 -L dependency=/home/fenhl/compile-fail/target/release/deps --cap-lints allow
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/arm-linux-gnueabihf/libthread_db.so.1".
process 22509 is executing new program: /home/fenhl/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/bin/rustc
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/arm-linux-gnueabihf/libthread_db.so.1".
[New Thread 0x74b471e0 (LWP 22524)]
[New Thread 0x6d7fe1e0 (LWP 22549)]
[New Thread 0x6d3fe1e0 (LWP 22550)]
[New Thread 0x6cffe1e0 (LWP 22551)]
[Thread 0x6cffe1e0 (LWP 22551) exited]
[New Thread 0x6cffe1e0 (LWP 22554)]
[New Thread 0x6c9fe1e0 (LWP 22555)]
[New Thread 0x6c7fd1e0 (LWP 22556)]
[New Thread 0x6c3fe1e0 (LWP 22557)]
[New Thread 0x6bffe1e0 (LWP 22558)]
[New Thread 0x6bdfd1e0 (LWP 22559)]
[New Thread 0x6bbfc1e0 (LWP 22560)]
[New Thread 0x6b9fb1e0 (LWP 22561)]
[Thread 0x6bffe1e0 (LWP 22558) exited]
[Thread 0x6c9fe1e0 (LWP 22555) exited]
[New Thread 0x6bffe1e0 (LWP 22562)]
[Thread 0x6bdfd1e0 (LWP 22559) exited]
[Thread 0x6c7fd1e0 (LWP 22556) exited]
[Thread 0x6b9fb1e0 (LWP 22561) exited]
[New Thread 0x6b9fb1e0 (LWP 22563)]
[Thread 0x6bffe1e0 (LWP 22562) exited]
[Thread 0x6bbfc1e0 (LWP 22560) exited]
[New Thread 0x6bbfc1e0 (LWP 22564)]
[Thread 0x6bbfc1e0 (LWP 22564) exited]
[Thread 0x6b9fb1e0 (LWP 22563) exited]
[New Thread 0x6b9fb1e0 (LWP 22565)]
[New Thread 0x6bbfc1e0 (LWP 22566)]
[New Thread 0x6bffe1e0 (LWP 22567)]
[Thread 0x6b9fb1e0 (LWP 22565) exited]
[Thread 0x6bbfc1e0 (LWP 22566) exited]
[Thread 0x6bffe1e0 (LWP 22567) exited]
[Thread 0x6cffe1e0 (LWP 22554) exited]
[Thread 0x6c3fe1e0 (LWP 22557) exited]
[New Thread 0x6c3fe1e0 (LWP 22568)]
[New Thread 0x6cffe1e0 (LWP 22569)]
[New Thread 0x6bffe1e0 (LWP 22570)]
[New Thread 0x6bbfc1e0 (LWP 22571)]
[New Thread 0x6b9fb1e0 (LWP 22572)]
[New Thread 0x6c7fd1e0 (LWP 22573)]
[New Thread 0x6bdfd1e0 (LWP 22574)]
[New Thread 0x6c9fe1e0 (LWP 22575)]
[New Thread 0x6b7fa1e0 (LWP 22576)]
[New Thread 0x6b5f91e0 (LWP 22577)]
[New Thread 0x6b3f81e0 (LWP 22578)]
malloc(): invalid next size (unsorted)
[New Thread 0x6b1f71e0 (LWP 22579)]
rustc: malloc.c:4028: _int_malloc: Assertion `(unsigned long) (size) >= (unsigned long) (nb)' failed.
[New Thread 0x6aff61e0 (LWP 22580)]

Thread 29 "rustc" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x6b5f91e0 (LWP 22577)]
0x74d1ab7c in _int_malloc (av=av@entry=0x6c400010, bytes=bytes@entry=1616) at malloc.c:4033
4033    malloc.c: No such file or directory.
(gdb) set pagination off
(gdb) thread apply all bt

Thread 32 (Thread 0x6aff61e0 (LWP 22580)):
#0  clone () at ../sysdeps/unix/sysv/linux/arm/clone.S:58
#1  0x74c1d180 in create_thread (pd=0x6aff6248, attr=0x6d3fced8, stopped_start=0x6d3fce82, stackaddr=<optimized out>, thread_ran=0x0) at ../sysdeps/unix/sysv/linux/createthread.c:101
#2  0x00000000 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

Thread 31 (Thread 0x6b1f71e0 (LWP 22579)):
#0  0x70b45cb6 in void std::__insertion_sort<__gnu_cxx::__normal_iterator<std::pair<unsigned long long, llvm::Function*>*, std::vector<std::pair<unsigned long long, llvm::Function*>, std::allocator<std::pair<unsigned long long, llvm::Function*> > > >, __gnu_cxx::__ops::_Iter_comp_iter<llvm::less_first> >(__gnu_cxx::__normal_iterator<std::pair<unsigned long long, llvm::Function*>*, std::vector<std::pair<unsigned long long, llvm::Function*>, std::allocator<std::pair<unsigned long long, llvm::Function*> > > >, __gnu_cxx::__normal_iterator<std::pair<unsigned long long, llvm::Function*>*, std::vector<std::pair<unsigned long long, llvm::Function*>, std::allocator<std::pair<unsigned long long, llvm::Function*> > > >, __gnu_cxx::__ops::_Iter_comp_iter<llvm::less_first>) () from /home/fenhl/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/codegen-backends/librustc_codegen_llvm-llvm.so
#1  0x6d04dcf8 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

Thread 30 (Thread 0x6b3f81e0 (LWP 22578)):
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
#1  0x74cc0230 in __GI_abort () at abort.c:79
#2  0x74d1051c in __libc_message (action=action@entry=do_abort, fmt=<optimized out>) at ../sysdeps/posix/libc_fatal.c:181
#3  0x74d17044 in malloc_printerr (str=<optimized out>) at malloc.c:5341
#4  0x74d1a6d8 in _int_malloc (av=av@entry=0x6cc00010, bytes=bytes@entry=320) at malloc.c:3735
#5  0x74d1b2fc in tcache_init () at malloc.c:2979
#6  0x74d1c1f8 in tcache_init () at malloc.c:3034
#7  __GI___libc_malloc (bytes=32) at malloc.c:3034
#8  0x74c20308 in pthread_getattr_np (thread_id=<optimized out>, attr=0x6b3f7930) at pthread_getattr_np.c:176
#9  0x74ea9298 in std::sys::unix::thread::guard::current () at src/libstd/sys/unix/thread.rs:371
#10 0x7035e3f4 in core::ops::function::FnOnce::call_once{{vtable-shim}} () from /home/fenhl/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/codegen-backends/librustc_codegen_llvm-llvm.so
#11 0x74e7fea4 in <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once () at /rustc/6e0d27d9368e2982bef8e1c4ac14d622c5ad018e/src/liballoc/boxed.rs:770
#12 0x74ea90f0 in <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once () at /rustc/6e0d27d9368e2982bef8e1c4ac14d622c5ad018e/src/liballoc/boxed.rs:770
#13 std::sys_common::thread::start_thread () at src/libstd/sys_common/thread.rs:13
#14 std::sys::unix::thread::Thread::new::thread_start () at src/libstd/sys/unix/thread.rs:79
#15 0x74c1e494 in start_thread (arg=0x6b3f81e0) at pthread_create.c:486
#16 0x74d80578 in ?? () at ../sysdeps/unix/sysv/linux/arm/clone.S:73 from /lib/arm-linux-gnueabihf/libc.so.6
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

Thread 29 (Thread 0x6b5f91e0 (LWP 22577)):
#0  0x74d1ab7c in _int_malloc (av=av@entry=0x6c400010, bytes=bytes@entry=1616) at malloc.c:4033
#1  0x74d1c318 in __GI___libc_malloc (bytes=1616) at malloc.c:3057
#2  0x71b5eedc in operator new (sz=1616) at /tmp/build/.build/src/gcc-5.2.0/libstdc++-v3/libsupc++/new_op.cc:50
#3  0x718f3a2c in llvm::LLVMContext::LLVMContext() () from /home/fenhl/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/codegen-backends/librustc_codegen_llvm-llvm.so
#4  0x704b0404 in LLVMRustContextCreate () from /home/fenhl/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/codegen-backends/librustc_codegen_llvm-llvm.so
#5  0x7039a958 in rustc_codegen_llvm::back::lto::optimize_thin_module () from /home/fenhl/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/codegen-backends/librustc_codegen_llvm-llvm.so
#6  0x703457b4 in rustc_codegen_ssa::back::lto::LtoModuleCodegen<B>::optimize () from /home/fenhl/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/codegen-backends/librustc_codegen_llvm-llvm.so
#7  0x7040f198 in std::sys_common::backtrace::__rust_begin_short_backtrace () from /home/fenhl/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/codegen-backends/librustc_codegen_llvm-llvm.so
#8  0x7035dab4 in std::panicking::try::do_call () from /home/fenhl/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/codegen-backends/librustc_codegen_llvm-llvm.so
#9  0x74eaa2ec in __rust_maybe_catch_panic () at src/libpanic_unwind/lib.rs:80
#10 0x7035e438 in core::ops::function::FnOnce::call_once{{vtable-shim}} () from /home/fenhl/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/codegen-backends/librustc_codegen_llvm-llvm.so
#11 0x74e7fea4 in <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once () at /rustc/6e0d27d9368e2982bef8e1c4ac14d622c5ad018e/src/liballoc/boxed.rs:770
#12 0x74ea90f0 in <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once () at /rustc/6e0d27d9368e2982bef8e1c4ac14d622c5ad018e/src/liballoc/boxed.rs:770
#13 std::sys_common::thread::start_thread () at src/libstd/sys_common/thread.rs:13
#14 std::sys::unix::thread::Thread::new::thread_start () at src/libstd/sys/unix/thread.rs:79
#15 0x74c1e494 in start_thread (arg=0x6b5f91e0) at pthread_create.c:486
#16 0x74d80578 in ?? () at ../sysdeps/unix/sysv/linux/arm/clone.S:73 from /lib/arm-linux-gnueabihf/libc.so.6
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

Thread 28 (Thread 0x6b7fa1e0 (LWP 22576)):
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
#1  0x74cc0230 in __GI_abort () at abort.c:79
#2  0x74d16fb0 in __malloc_assert (assertion=<optimized out>, file=<optimized out>, line=line@entry=4028, function=0x74dd7514 <__PRETTY_FUNCTION__.17241> "_int_malloc") at malloc.c:298
#3  0x74d1ac6c in _int_malloc (av=av@entry=0x6cb00010, bytes=bytes@entry=252) at malloc.c:4028
#4  0x74d1c318 in __GI___libc_malloc (bytes=252) at malloc.c:3057
#5  0x71b5eedc in operator new (sz=252) at /tmp/build/.build/src/gcc-5.2.0/libstdc++-v3/libsupc++/new_op.cc:50
#6  0x70e0c3e6 in createARMMCAsmInfo(llvm::MCRegisterInfo const&, llvm::Triple const&) () from /home/fenhl/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/codegen-backends/librustc_codegen_llvm-llvm.so
#7  0x711d6c1c in llvm::LLVMTargetMachine::initAsmInfo() () from /home/fenhl/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/codegen-backends/librustc_codegen_llvm-llvm.so
#8  0x70d9d8e4 in llvm::ARMBaseTargetMachine::ARMBaseTargetMachine(llvm::Target const&, llvm::Triple const&, llvm::StringRef, llvm::StringRef, llvm::TargetOptions const&, llvm::Optional<llvm::Reloc::Model>, llvm::Optional<llvm::CodeModel::Model>, llvm::CodeGenOpt::Level, bool) () from /home/fenhl/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/codegen-backends/librustc_codegen_llvm-llvm.so
#9  0x70d9dac0 in llvm::ARMLETargetMachine::ARMLETargetMachine(llvm::Target const&, llvm::Triple const&, llvm::StringRef, llvm::StringRef, llvm::TargetOptions const&, llvm::Optional<llvm::Reloc::Model>, llvm::Optional<llvm::CodeModel::Model>, llvm::CodeGenOpt::Level, bool) () from /home/fenhl/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/codegen-backends/librustc_codegen_llvm-llvm.so
#10 0x70d9db62 in llvm::RegisterTargetMachine<llvm::ARMLETargetMachine>::Allocator(llvm::Target const&, llvm::Triple const&, llvm::StringRef, llvm::StringRef, llvm::TargetOptions const&, llvm::Optional<llvm::Reloc::Model>, llvm::Optional<llvm::CodeModel::Model>, llvm::CodeGenOpt::Level, bool) () from /home/fenhl/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/codegen-backends/librustc_codegen_llvm-llvm.so
#11 0x704ab25c in LLVMRustCreateTargetMachine () from /home/fenhl/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/codegen-backends/librustc_codegen_llvm-llvm.so
#12 0x70473ab0 in rustc_codegen_llvm::back::write::target_machine_factory::{{closure}} () from /home/fenhl/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/codegen-backends/librustc_codegen_llvm-llvm.so
#13 0x7039a900 in rustc_codegen_llvm::back::lto::optimize_thin_module () from /home/fenhl/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/codegen-backends/librustc_codegen_llvm-llvm.so
#14 0x703457b4 in rustc_codegen_ssa::back::lto::LtoModuleCodegen<B>::optimize () from /home/fenhl/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/codegen-backends/librustc_codegen_llvm-llvm.so
#15 0x7040f198 in std::sys_common::backtrace::__rust_begin_short_backtrace () from /home/fenhl/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/codegen-backends/librustc_codegen_llvm-llvm.so
#16 0x7035dab4 in std::panicking::try::do_call () from /home/fenhl/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/codegen-backends/librustc_codegen_llvm-llvm.so
#17 0x74eaa2ec in __rust_maybe_catch_panic () at src/libpanic_unwind/lib.rs:80
#18 0x7035e438 in core::ops::function::FnOnce::call_once{{vtable-shim}} () from /home/fenhl/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/codegen-backends/librustc_codegen_llvm-llvm.so
#19 0x74e7fea4 in <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once () at /rustc/6e0d27d9368e2982bef8e1c4ac14d622c5ad018e/src/liballoc/boxed.rs:770
#20 0x74ea90f0 in <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once () at /rustc/6e0d27d9368e2982bef8e1c4ac14d622c5ad018e/src/liballoc/boxed.rs:770
#21 std::sys_common::thread::start_thread () at src/libstd/sys_common/thread.rs:13
#22 std::sys::unix::thread::Thread::new::thread_start () at src/libstd/sys/unix/thread.rs:79
#23 0x74c1e494 in start_thread (arg=0x6b7fa1e0) at pthread_create.c:486
#24 0x74d80578 in ?? () at ../sysdeps/unix/sysv/linux/arm/clone.S:73 from /lib/arm-linux-gnueabihf/libc.so.6
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

Thread 27 (Thread 0x6c9fe1e0 (LWP 22575)):
#0  0x74d1ab7c in _int_malloc (av=av@entry=0x6c000010, bytes=bytes@entry=89) at malloc.c:4033
#1  0x74d1b980 in _int_realloc (av=av@entry=0x6c000010, oldp=oldp@entry=0x6c02eb78, oldsize=oldsize@entry=48, nb=nb@entry=96) at malloc.c:4597
#2  0x74d1cde8 in __GI___libc_realloc (oldmem=0x6c02eb80, bytes=92) at malloc.c:3222
#3  0x71b02216 in llvm::SmallVectorBase::grow_pod(void*, unsigned int, unsigned int) () from /home/fenhl/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/codegen-backends/librustc_codegen_llvm-llvm.so
#4  0x7191a8ec in (anonymous namespace)::GetCFGOnlyPasses::passEnumerate(llvm::PassInfo const*) () from /home/fenhl/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/codegen-backends/librustc_codegen_llvm-llvm.so
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

Thread 26 (Thread 0x6bdfd1e0 (LWP 22574)):
#0  0x70b45cc8 in void std::__insertion_sort<__gnu_cxx::__normal_iterator<std::pair<unsigned long long, llvm::Function*>*, std::vector<std::pair<unsigned long long, llvm::Function*>, std::allocator<std::pair<unsigned long long, llvm::Function*> > > >, __gnu_cxx::__ops::_Iter_comp_iter<llvm::less_first> >(__gnu_cxx::__normal_iterator<std::pair<unsigned long long, llvm::Function*>*, std::vector<std::pair<unsigned long long, llvm::Function*>, std::allocator<std::pair<unsigned long long, llvm::Function*> > > >, __gnu_cxx::__normal_iterator<std::pair<unsigned long long, llvm::Function*>*, std::vector<std::pair<unsigned long long, llvm::Function*>, std::allocator<std::pair<unsigned long long, llvm::Function*> > > >, __gnu_cxx::__ops::_Iter_comp_iter<llvm::less_first>) () from /home/fenhl/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/codegen-backends/librustc_codegen_llvm-llvm.so
#1  0x00439338 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

Thread 25 (Thread 0x6c7fd1e0 (LWP 22573)):
#0  0x70b45cc4 in void std::__insertion_sort<__gnu_cxx::__normal_iterator<std::pair<unsigned long long, llvm::Function*>*, std::vector<std::pair<unsigned long long, llvm::Function*>, std::allocator<std::pair<unsigned long long, llvm::Function*> > > >, __gnu_cxx::__ops::_Iter_comp_iter<llvm::less_first> >(__gnu_cxx::__normal_iterator<std::pair<unsigned long long, llvm::Function*>*, std::vector<std::pair<unsigned long long, llvm::Function*>, std::allocator<std::pair<unsigned long long, llvm::Function*> > > >, __gnu_cxx::__normal_iterator<std::pair<unsigned long long, llvm::Function*>*, std::vector<std::pair<unsigned long long, llvm::Function*>, std::allocator<std::pair<unsigned long long, llvm::Function*> > > >, __gnu_cxx::__ops::_Iter_comp_iter<llvm::less_first>) () from /home/fenhl/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/codegen-backends/librustc_codegen_llvm-llvm.so
#1  0x6dc63830 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

Thread 24 (Thread 0x6b9fb1e0 (LWP 22572)):
#0  0x70b45c48 in void std::__insertion_sort<__gnu_cxx::__normal_iterator<std::pair<unsigned long long, llvm::Function*>*, std::vector<std::pair<unsigned long long, llvm::Function*>, std::allocator<std::pair<unsigned long long, llvm::Function*> > > >, __gnu_cxx::__ops::_Iter_comp_iter<llvm::less_first> >(__gnu_cxx::__normal_iterator<std::pair<unsigned long long, llvm::Function*>*, std::vector<std::pair<unsigned long long, llvm::Function*>, std::allocator<std::pair<unsigned long long, llvm::Function*> > > >, __gnu_cxx::__normal_iterator<std::pair<unsigned long long, llvm::Function*>*, std::vector<std::pair<unsigned long long, llvm::Function*>, std::allocator<std::pair<unsigned long long, llvm::Function*> > > >, __gnu_cxx::__ops::_Iter_comp_iter<llvm::less_first>) () from /home/fenhl/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/codegen-backends/librustc_codegen_llvm-llvm.so
#1  0x6d42d000 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

Thread 23 (Thread 0x6bbfc1e0 (LWP 22571)):
#0  0x70b45cc0 in void std::__insertion_sort<__gnu_cxx::__normal_iterator<std::pair<unsigned long long, llvm::Function*>*, std::vector<std::pair<unsigned long long, llvm::Function*>, std::allocator<std::pair<unsigned long long, llvm::Function*> > > >, __gnu_cxx::__ops::_Iter_comp_iter<llvm::less_first> >(__gnu_cxx::__normal_iterator<std::pair<unsigned long long, llvm::Function*>*, std::vector<std::pair<unsigned long long, llvm::Function*>, std::allocator<std::pair<unsigned long long, llvm::Function*> > > >, __gnu_cxx::__normal_iterator<std::pair<unsigned long long, llvm::Function*>*, std::vector<std::pair<unsigned long long, llvm::Function*>, std::allocator<std::pair<unsigned long long, llvm::Function*> > > >, __gnu_cxx::__ops::_Iter_comp_iter<llvm::less_first>) () from /home/fenhl/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/codegen-backends/librustc_codegen_llvm-llvm.so
Backtrace stopped: Cannot access memory at address 0x506dad52

Thread 22 (Thread 0x6bffe1e0 (LWP 22570)):
#0  0x70b45cc8 in void std::__insertion_sort<__gnu_cxx::__normal_iterator<std::pair<unsigned long long, llvm::Function*>*, std::vector<std::pair<unsigned long long, llvm::Function*>, std::allocator<std::pair<unsigned long long, llvm::Function*> > > >, __gnu_cxx::__ops::_Iter_comp_iter<llvm::less_first> >(__gnu_cxx::__normal_iterator<std::pair<unsigned long long, llvm::Function*>*, std::vector<std::pair<unsigned long long, llvm::Function*>, std::allocator<std::pair<unsigned long long, llvm::Function*> > > >, __gnu_cxx::__normal_iterator<std::pair<unsigned long long, llvm::Function*>*, std::vector<std::pair<unsigned long long, llvm::Function*>, std::allocator<std::pair<unsigned long long, llvm::Function*> > > >, __gnu_cxx::__ops::_Iter_comp_iter<llvm::less_first>) () from /home/fenhl/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/codegen-backends/librustc_codegen_llvm-llvm.so
#1  0x6cce2d90 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

Thread 21 (Thread 0x6cffe1e0 (LWP 22569)):
#0  0x70b45cc8 in void std::__insertion_sort<__gnu_cxx::__normal_iterator<std::pair<unsigned long long, llvm::Function*>*, std::vector<std::pair<unsigned long long, llvm::Function*>, std::allocator<std::pair<unsigned long long, llvm::Function*> > > >, __gnu_cxx::__ops::_Iter_comp_iter<llvm::less_first> >(__gnu_cxx::__normal_iterator<std::pair<unsigned long long, llvm::Function*>*, std::vector<std::pair<unsigned long long, llvm::Function*>, std::allocator<std::pair<unsigned long long, llvm::Function*> > > >, __gnu_cxx::__normal_iterator<std::pair<unsigned long long, llvm::Function*>*, std::vector<std::pair<unsigned long long, llvm::Function*>, std::allocator<std::pair<unsigned long long, llvm::Function*> > > >, __gnu_cxx::__ops::_Iter_comp_iter<llvm::less_first>) () from /home/fenhl/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/codegen-backends/librustc_codegen_llvm-llvm.so
#1  0x6c431428 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

Thread 20 (Thread 0x6c3fe1e0 (LWP 22568)):
#0  0x70b45cc8 in void std::__insertion_sort<__gnu_cxx::__normal_iterator<std::pair<unsigned long long, llvm::Function*>*, std::vector<std::pair<unsigned long long, llvm::Function*>, std::allocator<std::pair<unsigned long long, llvm::Function*> > > >, __gnu_cxx::__ops::_Iter_comp_iter<llvm::less_first> >(__gnu_cxx::__normal_iterator<std::pair<unsigned long long, llvm::Function*>*, std::vector<std::pair<unsigned long long, llvm::Function*>, std::allocator<std::pair<unsigned long long, llvm::Function*> > > >, __gnu_cxx::__normal_iterator<std::pair<unsigned long long, llvm::Function*>*, std::vector<std::pair<unsigned long long, llvm::Function*>, std::allocator<std::pair<unsigned long long, llvm::Function*> > > >, __gnu_cxx::__ops::_Iter_comp_iter<llvm::less_first>) () from /home/fenhl/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/codegen-backends/librustc_codegen_llvm-llvm.so
#1  0x6c043c40 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

Thread 4 (Thread 0x6d3fe1e0 (LWP 22550)):
#0  clone () at ../sysdeps/unix/sysv/linux/arm/clone.S:58
#1  0x74c1d180 in create_thread (pd=0x6aff6248, attr=0x6d3fced8, stopped_start=0x6d3fce82, stackaddr=<optimized out>, thread_ran=0x6aff61e0) at ../sysdeps/unix/sysv/linux/createthread.c:101
#2  0x6aff6250 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

Thread 3 (Thread 0x6d7fe1e0 (LWP 22549)):
#0  futex_wait_cancelable (private=0, expected=0, futex_word=0x6dc89ffc) at ../sysdeps/unix/sysv/linux/futex-internal.h:88
#1  __pthread_cond_wait_common (abstime=0x0, mutex=0x0, cond=0x6dc89fd0) at pthread_cond_wait.c:502
#2  __pthread_cond_wait (cond=0x6dc89fd0, mutex=0x0) at pthread_cond_wait.c:655
#3  0x74e80e98 in std::sys::unix::condvar::Condvar::wait () at src/libstd/sys/unix/condvar.rs:69
#4  std::sys_common::condvar::Condvar::wait () at src/libstd/sys_common/condvar.rs:41
#5  std::sync::condvar::Condvar::wait () at src/libstd/sync/condvar.rs:204
#6  std::thread::park () at src/libstd/thread/mod.rs:911
#7  0x74e964f8 in std::sync::mpsc::blocking::WaitToken::wait () at src/libstd/sync/mpsc/blocking.rs:71
#8  0x76c7c0ac in std::sync::mpsc::stream::Packet<T>::recv () from /home/fenhl/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/bin/../lib/librustc_driver-b121c769d316ad56.so
#9  0x76c7fdb4 in std::sync::mpsc::Receiver<T>::recv () from /home/fenhl/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/bin/../lib/librustc_driver-b121c769d316ad56.so
#10 0x76c7b088 in std::sys_common::backtrace::__rust_begin_short_backtrace () from /home/fenhl/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/bin/../lib/librustc_driver-b121c769d316ad56.so
#11 0x76c79b1c in std::panicking::try::do_call () from /home/fenhl/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/bin/../lib/librustc_driver-b121c769d316ad56.so
#12 0x74eaa2ec in __rust_maybe_catch_panic () at src/libpanic_unwind/lib.rs:80
#13 0x76c7a274 in core::ops::function::FnOnce::call_once{{vtable-shim}} () from /home/fenhl/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/bin/../lib/librustc_driver-b121c769d316ad56.so
#14 0x74e7fea4 in <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once () at /rustc/6e0d27d9368e2982bef8e1c4ac14d622c5ad018e/src/liballoc/boxed.rs:770
#15 0x74ea90f0 in <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once () at /rustc/6e0d27d9368e2982bef8e1c4ac14d622c5ad018e/src/liballoc/boxed.rs:770
#16 std::sys_common::thread::start_thread () at src/libstd/sys_common/thread.rs:13
#17 std::sys::unix::thread::Thread::new::thread_start () at src/libstd/sys/unix/thread.rs:79
#18 0x74c1e494 in start_thread (arg=0x6d7fe1e0) at pthread_create.c:486
#19 0x74d80578 in ?? () at ../sysdeps/unix/sysv/linux/arm/clone.S:73 from /lib/arm-linux-gnueabihf/libc.so.6
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

Thread 2 (Thread 0x74b471e0 (LWP 22524)):
#0  futex_wait_cancelable (private=0, expected=0, futex_word=0x4142cc) at ../sysdeps/unix/sysv/linux/futex-internal.h:88
#1  __pthread_cond_wait_common (abstime=0x0, mutex=0x0, cond=0x4142a0) at pthread_cond_wait.c:502
#2  __pthread_cond_wait (cond=0x4142a0, mutex=0x0) at pthread_cond_wait.c:655
#3  0x74e80e98 in std::sys::unix::condvar::Condvar::wait () at src/libstd/sys/unix/condvar.rs:69
#4  std::sys_common::condvar::Condvar::wait () at src/libstd/sys_common/condvar.rs:41
#5  std::sync::condvar::Condvar::wait () at src/libstd/sync/condvar.rs:204
#6  std::thread::park () at src/libstd/thread/mod.rs:911
#7  0x74e964f8 in std::sync::mpsc::blocking::WaitToken::wait () at src/libstd/sync/mpsc/blocking.rs:71
#8  0x75da7c94 in std::sync::mpsc::shared::Packet<T>::recv () from /home/fenhl/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/bin/../lib/librustc_driver-b121c769d316ad56.so
#9  0x75d9b4f8 in std::sync::mpsc::Receiver<T>::recv () from /home/fenhl/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/bin/../lib/librustc_driver-b121c769d316ad56.so
#10 0x75db46f0 in rustc_codegen_ssa::back::write::SharedEmitterMain::check () from /home/fenhl/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/bin/../lib/librustc_driver-b121c769d316ad56.so
#11 0x7040d31c in rustc_codegen_ssa::back::write::OngoingCodegen<B>::join () from /home/fenhl/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/codegen-backends/librustc_codegen_llvm-llvm.so
#12 0x7044a9d4 in <rustc_codegen_llvm::LlvmCodegenBackend as rustc_codegen_utils::codegen_backend::CodegenBackend>::join_codegen_and_link () from /home/fenhl/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/codegen-backends/librustc_codegen_llvm-llvm.so
#13 0x751bb610 in rustc_interface::queries::Query<T>::compute () from /home/fenhl/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/bin/../lib/librustc_driver-b121c769d316ad56.so
#14 0x752826d8 in rustc_interface::queries::<impl rustc_interface::interface::Compiler>::link () from /home/fenhl/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/bin/../lib/librustc_driver-b121c769d316ad56.so
#15 0x750e46e0 in rustc_interface::interface::run_compiler_in_existing_thread_pool () from /home/fenhl/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/bin/../lib/librustc_driver-b121c769d316ad56.so
#16 0x7511a02c in std::thread::local::LocalKey<T>::with () from /home/fenhl/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/bin/../lib/librustc_driver-b121c769d316ad56.so
#17 0x7512e424 in syntax::with_globals () from /home/fenhl/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/bin/../lib/librustc_driver-b121c769d316ad56.so
#18 0x750b9f5c in std::sys_common::backtrace::__rust_begin_short_backtrace () from /home/fenhl/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/bin/../lib/librustc_driver-b121c769d316ad56.so
#19 0x74eaa2ec in __rust_maybe_catch_panic () at src/libpanic_unwind/lib.rs:80
#20 0x750e6870 in core::ops::function::FnOnce::call_once{{vtable-shim}} () from /home/fenhl/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/bin/../lib/librustc_driver-b121c769d316ad56.so
#21 0x74e7fea4 in <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once () at /rustc/6e0d27d9368e2982bef8e1c4ac14d622c5ad018e/src/liballoc/boxed.rs:770
#22 0x74ea90f0 in <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once () at /rustc/6e0d27d9368e2982bef8e1c4ac14d622c5ad018e/src/liballoc/boxed.rs:770
#23 std::sys_common::thread::start_thread () at src/libstd/sys_common/thread.rs:13
#24 std::sys::unix::thread::Thread::new::thread_start () at src/libstd/sys/unix/thread.rs:79
#25 0x74c1e494 in start_thread (arg=0x74b471e0) at pthread_create.c:486
#26 0x74d80578 in ?? () at ../sysdeps/unix/sysv/linux/arm/clone.S:73 from /lib/arm-linux-gnueabihf/libc.so.6
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

Thread 1 (Thread 0x76ff4020 (LWP 22509)):
#0  0x74c1fa3c in __GI___pthread_timedjoin_ex (threadid=1957982688, thread_return=0x0, abstime=<optimized out>, block=<optimized out>) at pthread_join_common.c:89
#1  0x74ea91c8 in std::sys::unix::thread::Thread::join () at src/libstd/sys/unix/thread.rs:178
#2  0x750e5e9c in std::thread::JoinHandle<T>::join () from /home/fenhl/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/bin/../lib/librustc_driver-b121c769d316ad56.so
#3  0x750dc3d0 in rustc_interface::util::spawn_thread_pool () from /home/fenhl/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/bin/../lib/librustc_driver-b121c769d316ad56.so
#4  0x7513da98 in rustc_driver::run_compiler () from /home/fenhl/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/bin/../lib/librustc_driver-b121c769d316ad56.so
#5  0x75121e84 in <std::panic::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once () from /home/fenhl/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/bin/../lib/librustc_driver-b121c769d316ad56.so
#6  0x750de3a8 in std::panicking::try::do_call () from /home/fenhl/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/bin/../lib/librustc_driver-b121c769d316ad56.so
#7  0x74eaa2ec in __rust_maybe_catch_panic () at src/libpanic_unwind/lib.rs:80
#8  0x75143924 in rustc_driver::report_ices_to_stderr_if_any () from /home/fenhl/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/bin/../lib/librustc_driver-b121c769d316ad56.so
#9  0x751440b8 in rustc_driver::main () from /home/fenhl/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/bin/../lib/librustc_driver-b121c769d316ad56.so
#10 0x00400988 in std::rt::lang_start::{{closure}} ()
#11 0x74e9bb98 in std::rt::lang_start_internal::{{closure}} () at src/libstd/rt.rs:49
#12 std::panicking::try::do_call () at src/libstd/panicking.rs:296
#13 0x74eaa2ec in __rust_maybe_catch_panic () at src/libpanic_unwind/lib.rs:80
#14 0x74e9c6c4 in std::panicking::try () at src/libstd/panicking.rs:275
#15 std::panic::catch_unwind () at src/libstd/panic.rs:394
#16 std::rt::lang_start_internal () at src/libstd/rt.rs:48
#17 0x0040096c in main ()
(gdb)

fenhl on 4 Aug 2019

Based on the backtraces, something likely went wrong earlier and corrupted malloc structures, eventually leading to a crash. None of the traces look like the root cause though.

Without getting an assertion-enabled rustc build, the other possibility I can think of is trying to run rustc under valgrind and seeing if it produces any useful warning.

nikic on 9 Aug 2019

I'm seeing a segfault without any dependency on the libc crate. Is this the same issue?

$ uname -a
Linux rpi2 4.19.65-1-ARCH #1 SMP PREEMPT Fri Aug 9 23:29:04 UTC 2019 armv7l GNU/Linux
$ rustc +stable --version
rustc 1.36.0 (a53f9df32 2019-07-03)
$ rustc +nightly --version
rustc 1.38.0-nightly (00ee1b47f 2019-08-11)

# A trivial hello world program.
$ echo 'fn main() { println!("hello world"); }' > example.rs

# These all work fine.
$ rustc +stable example.rs
$ rustc +stable example.rs -O
$ rustc +nightly example.rs

# This one fails.
$ rustc +nightly example.rs -O
zsh: segmentation fault (core dumped)  rustc +nightly example.rs -O

However, with my hello world example I can't reproduce the segfault with cargo check, release or otherwise. It has to be cargo build --release, or the rustc command above.

oconnor663 on 12 Aug 2019

Have you tried -C codegen-units=1?

msizanoen on 13 Aug 2019

@msizanoen with -C codegen-units=1 all the rustc commands above succeed.

oconnor663 on 13 Aug 2019

👍1

This also seems to be a workaround for the original issue.

fenhl on 16 Aug 2019

The GCC ARM C++ ABI is not very stable (sometimes you will see the "parameter passing of argument of type ... changed in GCC ..." warning) so can somebody check if flushing sccache fixes this?

msizanoen on 30 Aug 2019

Trying to bootstrap rustc beta, building stage 1 without errors so far, so I guess we only need to clear sccache.

msizanoen on 30 Aug 2019

Compiler output (upstream nightly):

sirius@localhost:~$ rustc +nightly -Vv
rustc 1.39.0-nightly (9eae1fc0e 2019-08-23)
binary: rustc
commit-hash: 9eae1fc0ea9b00341b8fe191582c4decb5cb86a3
commit-date: 2019-08-23
host: armv7-unknown-linux-gnueabihf
release: 1.39.0-nightly
LLVM version: 9.0
sirius@localhost:~$ rustc +nightly -C opt-level=3 hello.rs
Segmentation fault
sirius@localhost:~$

Compiler output (local build stage1):

sirius@localhost:~$ cat hello.rs
fn main() {
    println!("Hello, world!");
}
sirius@localhost:~$ ~/rustc-beta-src/build/armv7-unknown-linux-gnueabihf/stage1/bin/rustc -Vv
rustc 1.38.0-beta
binary: rustc
commit-hash: unknown
commit-date: unknown
host: armv7-unknown-linux-gnueabihf
release: 1.38.0-beta
LLVM version: 9.0
sirius@localhost:~$ ~/rustc-beta-src/build/armv7-unknown-linux-gnueabihf/stage1/bin/rustc -C opt-level=3 hello.rs
sirius@localhost:~$ ./hello
Hello, world!

Still waiting for stage2 to finish building...

msizanoen on 30 Aug 2019

Comparing local build with upstream beta binaries:

sirius@localhost:~$ rustc +beta -Vv
rustc 1.38.0-beta.3 (72bfc3753 2019-08-27)
binary: rustc
commit-hash: 72bfc375356b56933955e07471f91ef97dceaa94
commit-date: 2019-08-27
host: armv7-unknown-linux-gnueabihf
release: 1.38.0-beta.3
LLVM version: 9.0
sirius@localhost:~$ ./rustc-beta/bin/rustc -Vv
rustc 1.38.0-beta
binary: rustc
commit-hash: unknown
commit-date: unknown
host: armv7-unknown-linux-gnueabihf
release: 1.38.0-beta
LLVM version: 9.0
sirius@localhost:~$ rustc +beta -C opt-level=3 hello.rs
Segmentation fault
sirius@localhost:~$ ./rustc-beta/bin/rustc -C opt-level=3 hello.rs
sirius@localhost:~$ ./hello
Hello, world!
sirius@localhost:~$

Conclusion: something is wrong with the build infrastructure.

msizanoen on 31 Aug 2019

Yeah using the workaround of forcing codegen-units to 1 makes things generally but not always work.

Things are all good on stable at least.

Lokathor on 1 Sep 2019

@msizanoen I doubt it has something to do with sccache.
Official Rust binaries are built with rust.remap-debuginfo = true and debuginfo-level-std=1 and few other options. Could you try with those two changes?

mati865 on 11 Sep 2019

Assigning to self to see if I can help with the investigation, and also to try to determine what action we need to take on this, with respect to the beta branch, before the next release.

pnkfelix on 12 Sep 2019

You might want to update the bug title, as it now affects beta 1.38.0:

cargo check --release --verbose
    Updating crates.io index
   Compiling libc v0.2.60
     Running `rustc --crate-name build_script_build /root/.cargo/registry/src/github.com-1ecc6299db9ec823/libc-0.2.60/build.rs --color always --crate-type bin --emit=dep-info,link -C opt-level=3 --cfg 'feature="default"' --cfg 'feature="std"' -C metadata=dccf822f6aca4e6f -C extra-filename=-dccf822f6aca4e6f --out-dir /tmp/compile-fail/target/release/build/libc-dccf822f6aca4e6f -L dependency=/tmp/compile-fail/target/release/deps --cap-lints allow`
error: Could not compile `libc`.

Caused by:
  process didn't exit successfully: `rustc --crate-name build_script_build /root/.cargo/registry/src/github.com-1ecc6299db9ec823/libc-0.2.60/build.rs --color always --crate-type bin --emit=dep-info,link -C opt-level=3 --cfg 'feature="default"' --cfg 'feature="std"' -C metadata=dccf822f6aca4e6f -C extra-filename=-dccf822f6aca4e6f --out-dir /tmp/compile-fail/target/release/build/libc-dccf822f6aca4e6f -L dependency=/tmp/compile-fail/target/release/deps --cap-lints allow` (signal: 11, SIGSEGV: invalid memory reference)

while it's working with codegen-units = 1

cargo check --release --verbose
   Compiling libc v0.2.60
     Running `rustc --crate-name build_script_build /root/.cargo/registry/src/github.com-1ecc6299db9ec823/libc-0.2.60/build.rs --color always --crate-type bin --emit=dep-info,link -C opt-level=3 -C codegen-units=1 --cfg 'feature="default"' --cfg 'feature="std"' -C metadata=0d79353f0e17f077 -C extra-filename=-0d79353f0e17f077 --out-dir /tmp/compile-fail/target/release/build/libc-0d79353f0e17f077 -L dependency=/tmp/compile-fail/target/release/deps --cap-lints allow`
     Running `/tmp/compile-fail/target/release/build/libc-0d79353f0e17f077/build-script-build`
     Running `rustc --crate-name libc /root/.cargo/registry/src/github.com-1ecc6299db9ec823/libc-0.2.60/src/lib.rs --color always --crate-type lib --emit=dep-info,metadata -C opt-level=3 -C codegen-units=1 --cfg 'feature="default"' --cfg 'feature="std"' -C metadata=5b917b2b2b03cc42 -C extra-filename=-5b917b2b2b03cc42 --out-dir /tmp/compile-fail/target/release/deps -L dependency=/tmp/compile-fail/target/release/deps --cap-lints allow --cfg libc_priv_mod_use --cfg libc_union --cfg libc_const_size_of --cfg libc_align --cfg libc_core_cvoid --cfg libc_packedN`
    Checking compile-fail v0.1.0 (/tmp/compile-fail)
     Running `rustc --edition=2018 --crate-name compile_fail src/main.rs --color always --crate-type bin --emit=dep-info,metadata -C opt-level=3 -C codegen-units=1 -C metadata=c17c108e2956aa64 -C extra-filename=-c17c108e2956aa64 --out-dir /tmp/compile-fail/target/release/deps -L dependency=/tmp/compile-fail/target/release/deps --extern libc=/tmp/compile-fail/target/release/deps/liblibc-5b917b2b2b03cc42.rmeta`
    Finished release [optimized] target(s) in 17.30s

dmesg is clean

anything I can provide, to make debugging easier?

stefson on 14 Sep 2019

anything I can provide, to make debugging easier?

Running build with asserts could hopefully produce useful debug info.
I'm on my mobile so I cannot check right now if there are dowloadable builds with assertions available.

mati865 on 14 Sep 2019

You could always also cargo-xbuild to your own target and that would rebuild core/alloc/std which should have debug stuff when built like that.

Lokathor on 14 Sep 2019

I have a Pi2, and if you setup a swap file ahead of time to avoid potentially running out of memory you can totally do it.

I'm away from home at the moment but your steps (using Nightly) are probably something like this:

rustup component add rust-src
cargo install cargo-xbuild
cargo xbuild --target armv7-unknown-linux-gnueabihf

It'll certainly take some time to build the cargo-xbuild tool, but after that it's actually pretty fast to build core and std.

Lokathor on 14 Sep 2019

Note that the CI configuration used for testing and distribution is different. The CI configuration for testing is Linux armhf-gnu but the configuration used for distribution is Linux dist-armv7-linux. The former uses the cross tools from Ubuntu but the latter uses cross compilation tools compiled using crosstools-ng.
This explains how this got pass CI testing.

msizanoen1 on 15 Sep 2019

👀1

My main option for trying to test this scenario would be on QEMU, and its been a while since I messed around with that.

So, before I invest the time in figuring out how to get that going on my machine (which may be trivial, since historically the project has used QEMU as a way to test ARM targets): Can someone easily check whether this problem does or does not reproduce on a QEMU target? In other words, does anyone here know if you really need to be using a hardware Raspberry Pi or something similar in order to observe a segfault or similar failure here?

Update: Given the dependence on codegen-units to observe the problem, I would not be shocked if it fails to reproduce on QEMU. Nonetheless, if someone already has such an environment set up and can check, that would be useful.

pnkfelix on 16 Sep 2019

The missmatch of toolchains between release and CI seems to make the output of CI doubtfull to me, so is there a way to fix this missmatch of CI configuration then? @msizanoen1

I tried to run this in gdb, but need rustc+cargo with debug symbols for helpfull output. It's not possible for me to build or run that, maybe someone owning a rpi4 with 2 or even 4gb of ram can jump in to help.

stefson on 16 Sep 2019

@stefson I have a 4GB RPi4, how can I help?

fenhl on 16 Sep 2019

No offense, but I meant to find someone running my own distro (gentoo), since I can't simply copy over a binary from debian.

stefson on 16 Sep 2019

@pnkfelix If you are using debian:

sudo dpkg --add-architecture armhf
sudo apt-get update
sudo apt-get install libc6:armhf qemu-user

After then download the armv7 rustc manually, binfmt_misc will make it execure transparently under QEMU.
Also if it is an atomics-related problem then it will fail to reproduce on QEMU running on x86/amd64 platforms because such architectures implies acquire/release ordering for all atomics operation.
However this is not likely the case as local build works fine.
Also it fails in Valgrind so it will probably fail in QEMU too.

@stefson The binaries should be mostly compatible as rustc doesn't have much dependencies on libraries provided by linux distributions.

Edit: somebody reproduced it in QEMU running on x86/amd64

msizanoen1 on 16 Sep 2019

This was brought up transitively through https://github.com/rust-lang/rust/issues/62933 at the infra team meeting today. I noticed there was a request to use a compiler that has debug and LLVM assertions enabled to hopefully weed out this segfault in LLVM, and I was thinking I could try to help out here.

For those of you who are able to reproduce this segfault on hardware, I've used our CI to produce an arm-host rustc which has LLVM and debug assertions enabled, hosted at http://rust-lang-ci2.s3.amazonaws.com/rustc-builds/9f4890317aa093c03f5e8b8a9dcd9d0bac9dde8a/rust-nightly-armv7-unknown-linux-gnueabihf.tar.xz (note that this will disappear in ~90 days). Would those of you with hardware be able to run that compiler? That should hopefully help us see if we're tripping an LLVM assertion, but the hidden assertion in the nightly channel may be leading to a segfault in LLVM.

alexcrichton on 17 Sep 2019

So download, unpack, then use that rustc to try and build any project that currently segfaults?

Lokathor on 17 Sep 2019

Indeed! The install.sh script inside can also be used to assemble everything together in a desired directory. If the compiler still segfaults without any helpful messages or doesn't look much different than this trace then it's probably not related to LLVM assertions and this new compiler won't help much.

alexcrichton on 17 Sep 2019

Will try this with my Raspberry Pi 4 within the next 24 hours.

Lokathor on 17 Sep 2019

Hopefully I did it right.
https://gist.github.com/xTachyon/fe13814490464ad7535604ccad47a8fc

Looks like there's more information than the previous trace, but I can't tell if it does help.

xTachyon on 18 Sep 2019

error: failed to parse bitcode for LTO module: file doesn't start with bitcode header

appears to be new from that trace, but it also seems quite odd -- I would not expect that error...

Mark-Simulacrum on 18 Sep 2019

@xTachyon judging by this line:

process 22004 is executing new program: /home/andrei/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/bin/rustc

I'm not sure that you're using the custom compiler? That looks like it's using a standard rustup nightly compiler.

alexcrichton on 18 Sep 2019

Well, I ran the install.sh ..
I don't know, would it be better to give the full path to rustc ?

xTachyon on 18 Sep 2019

Okay, I'm pretty sure I got it this time. I updated the gist with a new file named new.

xTachyon on 18 Sep 2019

@alexcrichton This reproduces in QEMU.
Which means that this is supposed to fail in CI.
However the configuration for testing uses Ubuntu cross compilers while configuration for distribution uses crosstool-ng generated toolchain (which may have bugs).

msizanoen1 on 18 Sep 2019

No assertion triggered, still segfault (on real hardware):

sirius@localhost:~$ uname -a
Linux localhost 3.18.71-perf-gff32bee #1 SMP PREEMPT Mon Jun 3 11:38:58 CEST 2019 armv7l GNU/Linux
sirius@localhost:~$ gdb -q ./rustc-test/bin/rustc core -batch -ex 'thread apply all bt'
[New LWP 8152]
[New LWP 8151]
[New LWP 8147]
[New LWP 8144]
[New LWP 8145]
[New LWP 8146]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/arm-linux-gnueabihf/libthread_db.so.1".
Core was generated by `rustc-test/bin/rustc -C opt-level=3 hello.rs'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x9f6efc0e in void std::__insertion_sort<__gnu_cxx::__normal_iterator<std::pair<unsigned long long, llvm::Function*>*, std::vector<std::pair<unsigned long long, llvm::Function*>, std::allocator<std::pair<unsigned long long, llvm::Function*> > > >, __gnu_cxx::__ops::_Iter_comp_iter<llvm::less_first> >(__gnu_cxx::__normal_iterator<std::pair<unsigned long long, llvm::Function*>*, std::vector<std::pair<unsigned long long, llvm::Function*>, std::allocator<std::pair<unsigned long long, llvm::Function*> > > >, __gnu_cxx::__normal_iterator<std::pair<unsigned long long, llvm::Function*>*, std::vector<std::pair<unsigned long long, llvm::Function*>, std::allocator<std::pair<unsigned long long, llvm::Function*> > > >, __gnu_cxx::__ops::_Iter_comp_iter<llvm::less_first>) () from /home/sirius/rustc-test/lib/rustlib/armv7-unknown-linux-gnueabihf/codegen-backends/librustc_codegen_llvm-llvm.so
[Current thread is 1 (Thread 0x9bbfe150 (LWP 8152))]

Thread 6 (Thread 0x9c5de150 (LWP 8146)):
#0  __libc_do_syscall () at ../sysdeps/unix/sysv/linux/arm/libc-do-syscall.S:46
#1  0xa480f66c in futex_wait_cancelable (private=0, expected=0, futex_word=0x9c78d1b8) at ../sysdeps/unix/sysv/linux/futex-internal.h:88
#2  __pthread_cond_wait_common (abstime=0x0, mutex=0x9c7a1ea8, cond=0x9c78d190) at pthread_cond_wait.c:502
#3  __pthread_cond_wait (cond=0x9c78d190, mutex=0x9c7a1ea8) at pthread_cond_wait.c:655
#4  0xa4a3743c in std::sys::unix::condvar::Condvar::wait () at src/libstd/sys/unix/condvar.rs:71
#5  std::sys_common::condvar::Condvar::wait () at src/libstd/sys_common/condvar.rs:41
#6  std::sync::condvar::Condvar::wait () at src/libstd/sync/condvar.rs:204
#7  std::thread::park () at src/libstd/thread/mod.rs:911
#8  0xa4a51730 in std::sync::mpsc::blocking::WaitToken::wait () at src/libstd/sync/mpsc/blocking.rs:71
#9  0xa7897b44 in std::sync::mpsc::stream::Packet<T>::recv () from /home/sirius/rustc-test/bin/../lib/librustc_driver-72f6e5eb69abc8b7.so
#10 0xa789e710 in std::sync::mpsc::Receiver<T>::recv () from /home/sirius/rustc-test/bin/../lib/librustc_driver-72f6e5eb69abc8b7.so
#11 0xa789a62c in std::sys_common::backtrace::__rust_begin_short_backtrace () from /home/sirius/rustc-test/bin/../lib/librustc_driver-72f6e5eb69abc8b7.so
#12 0xa7899644 in std::panicking::try::do_call () from /home/sirius/rustc-test/bin/../lib/librustc_driver-72f6e5eb69abc8b7.so
#13 0xa4a6a140 in __rust_maybe_catch_panic () at src/libpanic_unwind/lib.rs:80
#14 0xa789954c in std::panicking::try () from /home/sirius/rustc-test/bin/../lib/librustc_driver-72f6e5eb69abc8b7.so
#15 0xa7899ef8 in core::ops::function::FnOnce::call_once{{vtable-shim}} () from /home/sirius/rustc-test/bin/../lib/librustc_driver-72f6e5eb69abc8b7.so
#16 0xa4a356ec in <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once () at /rustc/9f4890317aa093c03f5e8b8a9dcd9d0bac9dde8a/src/liballoc/boxed.rs:922
#17 0xa4a68d64 in <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once () at /rustc/9f4890317aa093c03f5e8b8a9dcd9d0bac9dde8a/src/liballoc/boxed.rs:922
#18 std::sys_common::thread::start_thread () at src/libstd/sys_common/thread.rs:13
#19 std::sys::unix::thread::Thread::new::thread_start () at src/libstd/sys/unix/thread.rs:79
#20 0xa480ac40 in start_thread (arg=0x4cdbda5a) at pthread_create.c:486
#21 0xa4962a2c in ?? () at ../sysdeps/unix/sysv/linux/arm/clone.S:73 from /lib/arm-linux-gnueabihf/libc.so.6
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

Thread 5 (Thread 0xa4784150 (LWP 8145)):
#0  __libc_do_syscall () at ../sysdeps/unix/sysv/linux/arm/libc-do-syscall.S:46
#1  0xa480f66c in futex_wait_cancelable (private=0, expected=0, futex_word=0x85943a4c) at ../sysdeps/unix/sysv/linux/futex-internal.h:88
#2  __pthread_cond_wait_common (abstime=0x0, mutex=0x85942778, cond=0x85943a20) at pthread_cond_wait.c:502
#3  __pthread_cond_wait (cond=0x85943a20, mutex=0x85942778) at pthread_cond_wait.c:655
#4  0xa4a3743c in std::sys::unix::condvar::Condvar::wait () at src/libstd/sys/unix/condvar.rs:71
#5  std::sys_common::condvar::Condvar::wait () at src/libstd/sys_common/condvar.rs:41
#6  std::sync::condvar::Condvar::wait () at src/libstd/sync/condvar.rs:204
#7  std::thread::park () at src/libstd/thread/mod.rs:911
#8  0xa4a51730 in std::sync::mpsc::blocking::WaitToken::wait () at src/libstd/sync/mpsc/blocking.rs:71
#9  0xa61ba584 in std::sync::mpsc::shared::Packet<T>::recv () from /home/sirius/rustc-test/bin/../lib/librustc_driver-72f6e5eb69abc8b7.so
#10 0xa616de00 in std::sync::mpsc::Receiver<T>::recv () from /home/sirius/rustc-test/bin/../lib/librustc_driver-72f6e5eb69abc8b7.so
#11 0xa61c34ec in rustc_codegen_ssa::back::write::SharedEmitterMain::check () from /home/sirius/rustc-test/bin/../lib/librustc_driver-72f6e5eb69abc8b7.so
#12 0x9ed6fb24 in rustc_codegen_ssa::back::write::OngoingCodegen<B>::join () from /home/sirius/rustc-test/lib/rustlib/armv7-unknown-linux-gnueabihf/codegen-backends/librustc_codegen_llvm-llvm.so
#13 0x9eefe278 in <rustc_codegen_llvm::LlvmCodegenBackend as rustc_codegen_utils::codegen_backend::CodegenBackend>::join_codegen_and_link () from /home/sirius/rustc-test/lib/rustlib/armv7-unknown-linux-gnueabihf/codegen-backends/librustc_codegen_llvm-llvm.so
#14 0xa4e45964 in rustc_interface::queries::Query<T>::compute () from /home/sirius/rustc-test/bin/../lib/librustc_driver-72f6e5eb69abc8b7.so
#15 0xa4f97a90 in rustc_interface::queries::<impl rustc_interface::interface::Compiler>::link () from /home/sirius/rustc-test/bin/../lib/librustc_driver-72f6e5eb69abc8b7.so
#16 0xa4d3da8c in rustc_interface::interface::run_compiler_in_existing_thread_pool () from /home/sirius/rustc-test/bin/../lib/librustc_driver-72f6e5eb69abc8b7.so
#17 0xa4d82100 in std::thread::local::LocalKey<T>::with () from /home/sirius/rustc-test/bin/../lib/librustc_driver-72f6e5eb69abc8b7.so
#18 0xa4da5868 in syntax::with_globals () from /home/sirius/rustc-test/bin/../lib/librustc_driver-72f6e5eb69abc8b7.so
#19 0xa4cf0e60 in std::sys_common::backtrace::__rust_begin_short_backtrace () from /home/sirius/rustc-test/bin/../lib/librustc_driver-72f6e5eb69abc8b7.so
#20 0xa4d2d234 in std::panicking::try::do_call () from /home/sirius/rustc-test/bin/../lib/librustc_driver-72f6e5eb69abc8b7.so
#21 0xa4a6a140 in __rust_maybe_catch_panic () at src/libpanic_unwind/lib.rs:80
#22 0xa4d2c7dc in std::panicking::try () from /home/sirius/rustc-test/bin/../lib/librustc_driver-72f6e5eb69abc8b7.so
#23 0xa4d2d7b4 in core::ops::function::FnOnce::call_once{{vtable-shim}} () from /home/sirius/rustc-test/bin/../lib/librustc_driver-72f6e5eb69abc8b7.so
#24 0xa4a356ec in <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once () at /rustc/9f4890317aa093c03f5e8b8a9dcd9d0bac9dde8a/src/liballoc/boxed.rs:922
#25 0xa4a68d64 in <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once () at /rustc/9f4890317aa093c03f5e8b8a9dcd9d0bac9dde8a/src/liballoc/boxed.rs:922
#26 std::sys_common::thread::start_thread () at src/libstd/sys_common/thread.rs:13
#27 std::sys::unix::thread::Thread::new::thread_start () at src/libstd/sys/unix/thread.rs:79
#28 0xa480ac40 in start_thread (arg=0x4cdbda5a) at pthread_create.c:486
#29 0xa4962a2c in ?? () at ../sysdeps/unix/sysv/linux/arm/clone.S:73 from /lib/arm-linux-gnueabihf/libc.so.6
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

Thread 4 (Thread 0xa7eaf770 (LWP 8144)):
#0  __libc_do_syscall () at ../sysdeps/unix/sysv/linux/arm/libc-do-syscall.S:46
#1  0xa480bc38 in __GI___pthread_timedjoin_ex (threadid=2759344464, thread_return=0x0, abstime=<optimized out>, block=<optimized out>) at pthread_join_common.c:84
#2  0xa4a68e3c in std::sys::unix::thread::Thread::join () at src/libstd/sys/unix/thread.rs:179
#3  0xa4d2bbac in std::thread::JoinHandle<T>::join () from /home/sirius/rustc-test/bin/../lib/librustc_driver-72f6e5eb69abc8b7.so
#4  0xa4d198a0 in rustc_interface::util::spawn_thread_pool () from /home/sirius/rustc-test/bin/../lib/librustc_driver-72f6e5eb69abc8b7.so
#5  0xa4d3bcd0 in rustc_interface::interface::run_compiler () from /home/sirius/rustc-test/bin/../lib/librustc_driver-72f6e5eb69abc8b7.so
#6  0xa4d79bf4 in rustc_driver::run_compiler () from /home/sirius/rustc-test/bin/../lib/librustc_driver-72f6e5eb69abc8b7.so
#7  0xa4d89b34 in <std::panic::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once () from /home/sirius/rustc-test/bin/../lib/librustc_driver-72f6e5eb69abc8b7.so
#8  0xa4d2d4a8 in std::panicking::try::do_call () from /home/sirius/rustc-test/bin/../lib/librustc_driver-72f6e5eb69abc8b7.so
#9  0xa4a6a140 in __rust_maybe_catch_panic () at src/libpanic_unwind/lib.rs:80
#10 0xa4d2cce4 in std::panicking::try () from /home/sirius/rustc-test/bin/../lib/librustc_driver-72f6e5eb69abc8b7.so
#11 0xa4d80008 in rustc_driver::catch_fatal_errors () from /home/sirius/rustc-test/bin/../lib/librustc_driver-72f6e5eb69abc8b7.so
#12 0xa4d8101c in rustc_driver::main () from /home/sirius/rustc-test/bin/../lib/librustc_driver-72f6e5eb69abc8b7.so
#13 0x84f27964 in std::rt::lang_start::{{closure}} ()
#14 0xa4a59b8c in std::rt::lang_start_internal::{{closure}} () at src/libstd/rt.rs:49
#15 std::panicking::try::do_call () at src/libstd/panicking.rs:292
#16 0xa4a6a140 in __rust_maybe_catch_panic () at src/libpanic_unwind/lib.rs:80
#17 0xa4a5a920 in std::panicking::try () at src/libstd/panicking.rs:271
#18 std::panic::catch_unwind () at src/libstd/panic.rs:394
#19 std::rt::lang_start_internal () at src/libstd/rt.rs:48
#20 0x84f27948 in main ()

Thread 3 (Thread 0x9c3dd150 (LWP 8147)):
#0  __libc_do_syscall () at ../sysdeps/unix/sysv/linux/arm/libc-do-syscall.S:46
#1  0xa480f66c in futex_wait_cancelable (private=0, expected=0, futex_word=0x9c762868) at ../sysdeps/unix/sysv/linux/futex-internal.h:88
#2  __pthread_cond_wait_common (abstime=0x0, mutex=0x9c7a2788, cond=0x9c762840) at pthread_cond_wait.c:502
#3  __pthread_cond_wait (cond=0x9c762840, mutex=0x9c7a2788) at pthread_cond_wait.c:655
#4  0xa4a3743c in std::sys::unix::condvar::Condvar::wait () at src/libstd/sys/unix/condvar.rs:71
#5  std::sys_common::condvar::Condvar::wait () at src/libstd/sys_common/condvar.rs:41
#6  std::sync::condvar::Condvar::wait () at src/libstd/sync/condvar.rs:204
#7  std::thread::park () at src/libstd/thread/mod.rs:911
#8  0xa4a51730 in std::sync::mpsc::blocking::WaitToken::wait () at src/libstd/sync/mpsc/blocking.rs:71
#9  0x9eed7f28 in std::sync::mpsc::shared::Packet<T>::recv () from /home/sirius/rustc-test/lib/rustlib/armv7-unknown-linux-gnueabihf/codegen-backends/librustc_codegen_llvm-llvm.so
#10 0x9eebcae4 in std::sync::mpsc::Receiver<T>::recv () from /home/sirius/rustc-test/lib/rustlib/armv7-unknown-linux-gnueabihf/codegen-backends/librustc_codegen_llvm-llvm.so
#11 0x9eda9588 in rustc_codegen_ssa::back::write::start_executing_work::{{closure}} () from /home/sirius/rustc-test/lib/rustlib/armv7-unknown-linux-gnueabihf/codegen-backends/librustc_codegen_llvm-llvm.so
#12 0x9edaca64 in std::sys_common::backtrace::__rust_begin_short_backtrace () from /home/sirius/rustc-test/lib/rustlib/armv7-unknown-linux-gnueabihf/codegen-backends/librustc_codegen_llvm-llvm.so
#13 0x9edadc98 in std::panicking::try::do_call () from /home/sirius/rustc-test/lib/rustlib/armv7-unknown-linux-gnueabihf/codegen-backends/librustc_codegen_llvm-llvm.so
#14 0xa4a6a140 in __rust_maybe_catch_panic () at src/libpanic_unwind/lib.rs:80
#15 0x9edadad0 in std::panicking::try () from /home/sirius/rustc-test/lib/rustlib/armv7-unknown-linux-gnueabihf/codegen-backends/librustc_codegen_llvm-llvm.so
#16 0x9edae828 in core::ops::function::FnOnce::call_once{{vtable-shim}} () from /home/sirius/rustc-test/lib/rustlib/armv7-unknown-linux-gnueabihf/codegen-backends/librustc_codegen_llvm-llvm.so
#17 0xa4a356ec in <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once () at /rustc/9f4890317aa093c03f5e8b8a9dcd9d0bac9dde8a/src/liballoc/boxed.rs:922
#18 0xa4a68d64 in <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once () at /rustc/9f4890317aa093c03f5e8b8a9dcd9d0bac9dde8a/src/liballoc/boxed.rs:922
#19 std::sys_common::thread::start_thread () at src/libstd/sys_common/thread.rs:13
#20 std::sys::unix::thread::Thread::new::thread_start () at src/libstd/sys/unix/thread.rs:79
#21 0xa480ac40 in start_thread (arg=0x4cdbda5a) at pthread_create.c:486
#22 0xa4962a2c in ?? () at ../sysdeps/unix/sysv/linux/arm/clone.S:73 from /lib/arm-linux-gnueabihf/libc.so.6
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

Thread 2 (Thread 0x9b5ff150 (LWP 8151)):
#0  0x9f6efc1e in void std::__insertion_sort<__gnu_cxx::__normal_iterator<std::pair<unsigned long long, llvm::Function*>*, std::vector<std::pair<unsigned long long, llvm::Function*>, std::allocator<std::pair<unsigned long long, llvm::Function*> > > >, __gnu_cxx::__ops::_Iter_comp_iter<llvm::less_first> >(__gnu_cxx::__normal_iterator<std::pair<unsigned long long, llvm::Function*>*, std::vector<std::pair<unsigned long long, llvm::Function*>, std::allocator<std::pair<unsigned long long, llvm::Function*> > > >, __gnu_cxx::__normal_iterator<std::pair<unsigned long long, llvm::Function*>*, std::vector<std::pair<unsigned long long, llvm::Function*>, std::allocator<std::pair<unsigned long long, llvm::Function*> > > >, __gnu_cxx::__ops::_Iter_comp_iter<llvm::less_first>) () from /home/sirius/rustc-test/lib/rustlib/armv7-unknown-linux-gnueabihf/codegen-backends/librustc_codegen_llvm-llvm.so
#1  0x9b7547a8 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

Thread 1 (Thread 0x9bbfe150 (LWP 8152)):
#0  0x9f6efc0e in void std::__insertion_sort<__gnu_cxx::__normal_iterator<std::pair<unsigned long long, llvm::Function*>*, std::vector<std::pair<unsigned long long, llvm::Function*>, std::allocator<std::pair<unsigned long long, llvm::Function*> > > >, __gnu_cxx::__ops::_Iter_comp_iter<llvm::less_first> >(__gnu_cxx::__normal_iterator<std::pair<unsigned long long, llvm::Function*>*, std::vector<std::pair<unsigned long long, llvm::Function*>, std::allocator<std::pair<unsigned long long, llvm::Function*> > > >, __gnu_cxx::__normal_iterator<std::pair<unsigned long long, llvm::Function*>*, std::vector<std::pair<unsigned long long, llvm::Function*>, std::allocator<std::pair<unsigned long long, llvm::Function*> > > >, __gnu_cxx::__ops::_Iter_comp_iter<llvm::less_first>) () from /home/sirius/rustc-test/lib/rustlib/armv7-unknown-linux-gnueabihf/codegen-backends/librustc_codegen_llvm-llvm.so
#1  0x9b84dff8 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
sirius@localhost:~$ ./rustc-test/bin/rustc -C llvm-args=--version
LLVM (http://llvm.org/):
  LLVM version 9.0.0-rust-1.39.0-nightly
  Optimized build with assertions.
  Default target: armv7-unknown-linux-gnueabihf
  Host CPU: cortex-a53

msizanoen1 on 18 Sep 2019

this is my attempt to get a backtrace:

Starting program: /usr/bin/cargo check --release --verbose
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/libthread_db.so.1".
[Detaching after vfork from child process 2540]
[Detaching after vfork from child process 2543]
[Detaching after vfork from child process 2545]
[Detaching after vfork from child process 2547]
[New Thread 0x7422a210 (LWP 2549)]
   Compiling libc v0.2.60
[New Thread 0x74029210 (LWP 2550)]
     Running `rustc --crate-name build_script_build /root/.cargo/registry/src/github.com-1ecc6299db9ec823/libc-0.2.60/build.rs --color always --crate-type bin --emit=dep-info,link -C opt-level=3 --cfg 'feature="default"' --cfg 'feature="std"' -C metadata=c4e99ef4230eee51 -C extra-filename=-c4e99ef4230eee51 --out-dir /tmp/compile-fail/target/release/build/libc-c4e99ef4230eee51 -L dependency=/tmp/compile-fail/target/release/deps --cap-lints allow`
[Detaching after fork from child process 2551]                           ] 0/4: libc(build.rs)                                                                                        
[Thread 0x74029210 (LWP 2550) exited]

Thread 2 "cargo" received signal SIGUSR1, User defined signal 1.
[Switching to Thread 0x7422a210 (LWP 2549)]
futex_wait_cancelable (private=0, expected=0, futex_word=0x1254a88) at ../sysdeps/unix/sysv/linux/futex-internal.h:88
88  ../sysdeps/unix/sysv/linux/futex-internal.h: No such file or directory.
(gdb) bt
#0  futex_wait_cancelable (private=0, expected=0, futex_word=0x1254a88) at ../sysdeps/unix/sysv/linux/futex-internal.h:88
#1  __pthread_cond_wait_common (abstime=0x0, mutex=0x0, cond=0x1254a60) at pthread_cond_wait.c:502
#2  __pthread_cond_wait (cond=0x1254a60, mutex=0x0) at pthread_cond_wait.c:655
#3  0x00ebc3f4 in std::sys::unix::condvar::Condvar::wait () at src/libstd/sys/unix/condvar.rs:71
#4  std::sys_common::condvar::Condvar::wait () at src/libstd/sys_common/condvar.rs:41
#5  std::sync::condvar::Condvar::wait () at src/libstd/sync/condvar.rs:204
#6  std::thread::park () at src/libstd/thread/mod.rs:911
#7  0x00ecc5c0 in std::sync::mpsc::blocking::WaitToken::wait () at src/libstd/sync/mpsc/blocking.rs:71
#8  0x00e8cfa8 in std::sync::mpsc::oneshot::Packet<T>::recv ()
#9  0x00e8a5f8 in std::sync::mpsc::Receiver<T>::recv ()
#10 0x00e927e0 in std::sys_common::backtrace::__rust_begin_short_backtrace ()
#11 0x00e91960 in std::panicking::try::do_call ()
#12 0x00edeb5c in __rust_maybe_catch_panic () at src/libpanic_unwind/lib.rs:80
#13 0x00e91868 in std::panicking::try ()
#14 0x00e920dc in core::ops::function::FnOnce::call_once{{vtable-shim}} ()
#15 0x00ebab84 in <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once () at /rustc/9f4890317aa093c03f5e8b8a9dcd9d0bac9dde8a/src/liballoc/boxed.rs:922
#16 0x00edde80 in <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once () at /rustc/9f4890317aa093c03f5e8b8a9dcd9d0bac9dde8a/src/liballoc/boxed.rs:922
#17 std::sys_common::thread::start_thread () at src/libstd/sys_common/thread.rs:13
#18 std::sys::unix::thread::Thread::new::thread_start () at src/libstd/sys/unix/thread.rs:79
#19 0x76f64b7c in start_thread (arg=0x7422a210) at pthread_create.c:486
#20 0x76ecbb68 in ?? () at ../sysdeps/unix/sysv/linux/arm/clone.S:73 from /lib/libc.so.6
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

this is propably another bug showing here, I've never heard of a thing called futex-internal.h

stefson on 18 Sep 2019

@stefson Thread 2 "cargo" received signal SIGUSR1, unfortunately this trace not useful.

mati865 on 18 Sep 2019

😕1

Well I tried with that .tar.gz file, and I got this https://gist.github.com/rust-play/e4f74a89ecc145a2cb95164b28f452e4, which has output for cargo build --release as well as RUSTBACKTRACE=1 cargo build --release

I'm not entirely sure if I did it right, because there is no backtrace that's apparent, but there is a difference between the two runs at least.

Lokathor on 18 Sep 2019

Alas! Looks like this is unrelated to LLVM assertions in that case given the new traces.

alexcrichton on 18 Sep 2019

Looking at the distribution configuration, rustc is compiled against a very old glibc version (2.16) which may have bugs affecting LLVM 9.
(For anyone wondering how you can have old bugs on new distributions: https://stackoverflow.com/questions/2856438/how-can-i-link-to-a-specific-glibc-version)

msizanoen1 on 22 Sep 2019

If we set the env var MALLOC_CHECK_ to 1 then glibc will perform some extra malloc checks and we could have clearer backtraces.

msizanoen1 on 22 Sep 2019

After enabling malloc check:

[New LWP 6875]
[New LWP 6874]
[New LWP 6870]
[New LWP 6869]
[New LWP 6868]
[New LWP 6867]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/arm-linux-gnueabihf/libthread_db.so.1".
Core was generated by `/home/sirius/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/bin/rustc'.
Program terminated with signal SIGABRT, Aborted.
#0  __libc_do_syscall () at ../sysdeps/unix/sysv/linux/arm/libc-do-syscall.S:47
[Current thread is 1 (Thread 0xab0e6100 (LWP 6875))]

Thread 6 (Thread 0xb4535ac0 (LWP 6867)):
#0  __libc_do_syscall () at ../sysdeps/unix/sysv/linux/arm/libc-do-syscall.S:46
#1  0xb2109c38 in __GI___pthread_timedjoin_ex (threadid=2986877184, thread_return=0x0, abstime=<optimized out>, block=<optimized out>) at pthread_join_common.c:84
#2  0xb233d14c in std::sys::unix::thread::Thread::join () at src/libstd/sys/unix/thread.rs:179
#3  0xb257eadc in std::thread::JoinHandle<T>::join () from /home/sirius/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/bin/../lib/librustc_driver-34889fee6d6688d4.so
#4  0xb2572d4c in rustc_interface::util::spawn_thread_pool () from /home/sirius/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/bin/../lib/librustc_driver-34889fee6d6688d4.so
#5  0xb25b6ebc in rustc_driver::run_compiler () from /home/sirius/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/bin/../lib/librustc_driver-34889fee6d6688d4.so
#6  0xb25c2b90 in <std::panic::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once () from /home/sirius/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/bin/../lib/librustc_driver-34889fee6d6688d4.so
#7  0xb257f764 in std::panicking::try::do_call () from /home/sirius/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/bin/../lib/librustc_driver-34889fee6d6688d4.so
#8  0xb233e264 in __rust_maybe_catch_panic () at src/libpanic_unwind/lib.rs:80
#9  0xb25bc1cc in rustc_driver::catch_fatal_errors () from /home/sirius/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/bin/../lib/librustc_driver-34889fee6d6688d4.so
#10 0xb25bd0b4 in rustc_driver::main () from /home/sirius/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/bin/../lib/librustc_driver-34889fee6d6688d4.so
#11 0x82ed0978 in std::rt::lang_start::{{closure}} ()
#12 0xb232fcf0 in std::rt::lang_start_internal::{{closure}} () at src/libstd/rt.rs:49
#13 std::panicking::try::do_call () at src/libstd/panicking.rs:292
#14 0xb233e264 in __rust_maybe_catch_panic () at src/libpanic_unwind/lib.rs:80
#15 0xb23307d8 in std::panicking::try () at src/libstd/panicking.rs:271
#16 std::panic::catch_unwind () at src/libstd/panic.rs:394
#17 std::rt::lang_start_internal () at src/libstd/rt.rs:48
#18 0x82ed095c in main ()

Thread 5 (Thread 0xb2082100 (LWP 6868)):
#0  __libc_do_syscall () at ../sysdeps/unix/sysv/linux/arm/libc-do-syscall.S:46
#1  0xb210d66c in futex_wait_cancelable (private=0, expected=0, futex_word=0x83734dd4) at ../sysdeps/unix/sysv/linux/futex-internal.h:88
#2  __pthread_cond_wait_common (abstime=0x0, mutex=0x837346d0, cond=0x83734da8) at pthread_cond_wait.c:502
#3  __pthread_cond_wait (cond=0x83734da8, mutex=0x837346d0) at pthread_cond_wait.c:655
#4  0xb2313420 in std::sys::unix::condvar::Condvar::wait () at src/libstd/sys/unix/condvar.rs:71
#5  std::sys_common::condvar::Condvar::wait () at src/libstd/sys_common/condvar.rs:41
#6  std::sync::condvar::Condvar::wait () at src/libstd/sync/condvar.rs:204
#7  std::thread::park () at src/libstd/thread/mod.rs:911
#8  0xb2329af0 in std::sync::mpsc::blocking::WaitToken::wait () at src/libstd/sync/mpsc/blocking.rs:71
#9  0xb325cd70 in std::sync::mpsc::shared::Packet<T>::recv () from /home/sirius/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/bin/../lib/librustc_driver-34889fee6d6688d4.so
#10 0xb3220420 in std::sync::mpsc::Receiver<T>::recv () from /home/sirius/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/bin/../lib/librustc_driver-34889fee6d6688d4.so
#11 0xb3262cf8 in rustc_codegen_ssa::back::write::SharedEmitterMain::check () from /home/sirius/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/bin/../lib/librustc_driver-34889fee6d6688d4.so
#12 0xad9e0854 in rustc_codegen_ssa::back::write::OngoingCodegen<B>::join () from /home/sirius/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/codegen-backends/librustc_codegen_llvm-llvm.so
#13 0xadae9830 in <rustc_codegen_llvm::LlvmCodegenBackend as rustc_codegen_utils::codegen_backend::CodegenBackend>::join_codegen_and_link () from /home/sirius/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/codegen-backends/librustc_codegen_llvm-llvm.so
#14 0xb2649494 in rustc_interface::queries::Query<T>::compute () from /home/sirius/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/bin/../lib/librustc_driver-34889fee6d6688d4.so
#15 0xb271d730 in rustc_interface::queries::<impl rustc_interface::interface::Compiler>::link () from /home/sirius/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/bin/../lib/librustc_driver-34889fee6d6688d4.so
#16 0xb258bdc0 in rustc_interface::interface::run_compiler_in_existing_thread_pool () from /home/sirius/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/bin/../lib/librustc_driver-34889fee6d6688d4.so
#17 0xb25be328 in std::thread::local::LocalKey<T>::with () from /home/sirius/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/bin/../lib/librustc_driver-34889fee6d6688d4.so
#18 0xb25d41d8 in syntax::with_globals () from /home/sirius/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/bin/../lib/librustc_driver-34889fee6d6688d4.so
#19 0xb2559628 in std::sys_common::backtrace::__rust_begin_short_backtrace () from /home/sirius/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/bin/../lib/librustc_driver-34889fee6d6688d4.so
#20 0xb233e264 in __rust_maybe_catch_panic () at src/libpanic_unwind/lib.rs:80
#21 0xb257fb30 in core::ops::function::FnOnce::call_once{{vtable-shim}} () from /home/sirius/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/bin/../lib/librustc_driver-34889fee6d6688d4.so
#22 0xb23123fc in <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once () at /rustc/7efe1c6e678a263b9464f2e7f06f552b4d4db5a5/src/liballoc/boxed.rs:922
#23 0xb233d074 in <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once () at /rustc/7efe1c6e678a263b9464f2e7f06f552b4d4db5a5/src/liballoc/boxed.rs:922
#24 std::sys_common::thread::start_thread () at src/libstd/sys_common/thread.rs:13
#25 std::sys::unix::thread::Thread::new::thread_start () at src/libstd/sys/unix/thread.rs:79
#26 0xb2108c40 in start_thread (arg=0xf225bd73) at pthread_create.c:486
#27 0xb2248a2c in ?? () at ../sysdeps/unix/sysv/linux/arm/clone.S:73 from /lib/arm-linux-gnueabihf/libc.so.6
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

Thread 4 (Thread 0xab6e9100 (LWP 6869)):
#0  __libc_do_syscall () at ../sysdeps/unix/sysv/linux/arm/libc-do-syscall.S:46
#1  0xb210d66c in futex_wait_cancelable (private=0, expected=0, futex_word=0x83ac6800) at ../sysdeps/unix/sysv/linux/futex-internal.h:88
#2  __pthread_cond_wait_common (abstime=0x0, mutex=0x83ac67b8, cond=0x83ac67d8) at pthread_cond_wait.c:502
#3  __pthread_cond_wait (cond=0x83ac67d8, mutex=0x83ac67b8) at pthread_cond_wait.c:655
#4  0xb2313420 in std::sys::unix::condvar::Condvar::wait () at src/libstd/sys/unix/condvar.rs:71
#5  std::sys_common::condvar::Condvar::wait () at src/libstd/sys_common/condvar.rs:41
#6  std::sync::condvar::Condvar::wait () at src/libstd/sync/condvar.rs:204
#7  std::thread::park () at src/libstd/thread/mod.rs:911
#8  0xb2329af0 in std::sync::mpsc::blocking::WaitToken::wait () at src/libstd/sync/mpsc/blocking.rs:71
#9  0xb41b5fc4 in std::sync::mpsc::stream::Packet<T>::recv () from /home/sirius/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/bin/../lib/librustc_driver-34889fee6d6688d4.so
#10 0xb41bb8fc in std::sync::mpsc::Receiver<T>::recv () from /home/sirius/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/bin/../lib/librustc_driver-34889fee6d6688d4.so
#11 0xb41b800c in std::sys_common::backtrace::__rust_begin_short_backtrace () from /home/sirius/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/bin/../lib/librustc_driver-34889fee6d6688d4.so
#12 0xb41b733c in std::panicking::try::do_call () from /home/sirius/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/bin/../lib/librustc_driver-34889fee6d6688d4.so
#13 0xb233e264 in __rust_maybe_catch_panic () at src/libpanic_unwind/lib.rs:80
#14 0xb41b7924 in core::ops::function::FnOnce::call_once{{vtable-shim}} () from /home/sirius/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/bin/../lib/librustc_driver-34889fee6d6688d4.so
#15 0xb23123fc in <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once () at /rustc/7efe1c6e678a263b9464f2e7f06f552b4d4db5a5/src/liballoc/boxed.rs:922
#16 0xb233d074 in <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once () at /rustc/7efe1c6e678a263b9464f2e7f06f552b4d4db5a5/src/liballoc/boxed.rs:922
#17 std::sys_common::thread::start_thread () at src/libstd/sys_common/thread.rs:13
#18 std::sys::unix::thread::Thread::new::thread_start () at src/libstd/sys/unix/thread.rs:79
#19 0xb2108c40 in start_thread (arg=0xf225bd73) at pthread_create.c:486
#20 0xb2248a2c in ?? () at ../sysdeps/unix/sysv/linux/arm/clone.S:73 from /lib/arm-linux-gnueabihf/libc.so.6
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

Thread 3 (Thread 0xab4e8100 (LWP 6870)):
#0  __libc_do_syscall () at ../sysdeps/unix/sysv/linux/arm/libc-do-syscall.S:46
#1  0xb210d66c in futex_wait_cancelable (private=0, expected=0, futex_word=0x83ac7900) at ../sysdeps/unix/sysv/linux/futex-internal.h:88
#2  __pthread_cond_wait_common (abstime=0x0, mutex=0x83ac77b0, cond=0x83ac78d8) at pthread_cond_wait.c:502
#3  __pthread_cond_wait (cond=0x83ac78d8, mutex=0x83ac77b0) at pthread_cond_wait.c:655
#4  0xb2313420 in std::sys::unix::condvar::Condvar::wait () at src/libstd/sys/unix/condvar.rs:71
#5  std::sys_common::condvar::Condvar::wait () at src/libstd/sys_common/condvar.rs:41
#6  std::sync::condvar::Condvar::wait () at src/libstd/sync/condvar.rs:204
#7  std::thread::park () at src/libstd/thread/mod.rs:911
#8  0xb2329af0 in std::sync::mpsc::blocking::WaitToken::wait () at src/libstd/sync/mpsc/blocking.rs:71
#9  0xadacfaf8 in std::sync::mpsc::shared::Packet<T>::recv () from /home/sirius/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/codegen-backends/librustc_codegen_llvm-llvm.so
#10 0xadab4860 in std::sync::mpsc::Receiver<T>::recv () from /home/sirius/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/codegen-backends/librustc_codegen_llvm-llvm.so
#11 0xada0eac0 in std::sys_common::backtrace::__rust_begin_short_backtrace () from /home/sirius/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/codegen-backends/librustc_codegen_llvm-llvm.so
#12 0xada11510 in std::panicking::try::do_call () from /home/sirius/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/codegen-backends/librustc_codegen_llvm-llvm.so
#13 0xb233e264 in __rust_maybe_catch_panic () at src/libpanic_unwind/lib.rs:80
#14 0xada11ce4 in core::ops::function::FnOnce::call_once{{vtable-shim}} () from /home/sirius/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/codegen-backends/librustc_codegen_llvm-llvm.so
#15 0xb23123fc in <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once () at /rustc/7efe1c6e678a263b9464f2e7f06f552b4d4db5a5/src/liballoc/boxed.rs:922
#16 0xb233d074 in <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once () at /rustc/7efe1c6e678a263b9464f2e7f06f552b4d4db5a5/src/liballoc/boxed.rs:922
#17 std::sys_common::thread::start_thread () at src/libstd/sys_common/thread.rs:13
#18 std::sys::unix::thread::Thread::new::thread_start () at src/libstd/sys/unix/thread.rs:79
#19 0xb2108c40 in start_thread (arg=0xf225bd73) at pthread_create.c:486
#20 0xb2248a2c in ?? () at ../sysdeps/unix/sysv/linux/arm/clone.S:73 from /lib/arm-linux-gnueabihf/libc.so.6
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

Thread 2 (Thread 0xaaee3100 (LWP 6874)):
#0  0xae1cc1e0 in void std::__insertion_sort<__gnu_cxx::__normal_iterator<std::pair<unsigned long long, llvm::Function*>*, std::vector<std::pair<unsigned long long, llvm::Function*>, std::allocator<std::pair<unsigned long long, llvm::Function*> > > >, __gnu_cxx::__ops::_Iter_comp_iter<llvm::less_first> >(__gnu_cxx::__normal_iterator<std::pair<unsigned long long, llvm::Function*>*, std::vector<std::pair<unsigned long long, llvm::Function*>, std::allocator<std::pair<unsigned long long, llvm::Function*> > > >, __gnu_cxx::__normal_iterator<std::pair<unsigned long long, llvm::Function*>*, std::vector<std::pair<unsigned long long, llvm::Function*>, std::allocator<std::pair<unsigned long long, llvm::Function*> > > >, __gnu_cxx::__ops::_Iter_comp_iter<llvm::less_first>) () from /home/sirius/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/codegen-backends/librustc_codegen_llvm-llvm.so
#1  0x83af1cd0 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

Thread 1 (Thread 0xab0e6100 (LWP 6875)):
#0  __libc_do_syscall () at ../sysdeps/unix/sysv/linux/arm/libc-do-syscall.S:47
#1  0xb21d2916 in __libc_signal_restore_set (set=0xab0e4e6c) at ../sysdeps/unix/sysv/linux/internal-signals.h:84
#2  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:48
#3  0xb21c4732 in __GI_abort () at abort.c:79
#4  0xb21ffc54 in __malloc_assert (assertion=<optimized out>, file=<optimized out>, line=line@entry=4050, function=0xb2289520 <__PRETTY_FUNCTION__.17329> "_int_malloc") at malloc.c:298
#5  0xb22024d8 in _int_malloc (av=av@entry=0xb22a17fc <main_arena>, bytes=bytes@entry=397) at malloc.c:4050
#6  0xb2202740 in malloc_check (sz=396, caller=<optimized out>) at hooks.c:238
#7  0xaf1e50a8 in operator new (sz=396) at /tmp/build/.build/src/gcc-5.2.0/libstdc++-v3/libsupc++/new_op.cc:50
#8  0xaeadac8a in llvm::createReassociatePass() () from /home/sirius/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/codegen-backends/librustc_codegen_llvm-llvm.so
#9  0xae1d560e in llvm::PassManagerBuilder::addFunctionSimplificationPasses(llvm::legacy::PassManagerBase&) () from /home/sirius/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/codegen-backends/librustc_codegen_llvm-llvm.so
#10 0xae1d5ae0 in llvm::PassManagerBuilder::populateModulePassManager(llvm::legacy::PassManagerBase&) () from /home/sirius/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/codegen-backends/librustc_codegen_llvm-llvm.so
#11 0xae1d65e8 in llvm::PassManagerBuilder::populateThinLTOPassManager(llvm::legacy::PassManagerBase&) () from /home/sirius/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/codegen-backends/librustc_codegen_llvm-llvm.so
#12 0xadad9a3c in rustc_codegen_llvm::back::write::with_llvm_pmb () from /home/sirius/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/codegen-backends/librustc_codegen_llvm-llvm.so
#13 0xada205e8 in rustc_codegen_llvm::back::lto::run_pass_manager () from /home/sirius/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/codegen-backends/librustc_codegen_llvm-llvm.so
#14 0xada20de0 in rustc_codegen_llvm::back::lto::optimize_thin_module () from /home/sirius/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/codegen-backends/librustc_codegen_llvm-llvm.so
#15 0xad9da86c in rustc_codegen_ssa::back::write::execute_work_item () from /home/sirius/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/codegen-backends/librustc_codegen_llvm-llvm.so
#16 0xada0dbe8 in std::sys_common::backtrace::__rust_begin_short_backtrace () from /home/sirius/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/codegen-backends/librustc_codegen_llvm-llvm.so
#17 0xada1154c in std::panicking::try::do_call () from /home/sirius/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/codegen-backends/librustc_codegen_llvm-llvm.so
#18 0xb233e264 in __rust_maybe_catch_panic () at src/libpanic_unwind/lib.rs:80
#19 0xada11e60 in core::ops::function::FnOnce::call_once{{vtable-shim}} () from /home/sirius/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/codegen-backends/librustc_codegen_llvm-llvm.so
#20 0xb23123fc in <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once () at /rustc/7efe1c6e678a263b9464f2e7f06f552b4d4db5a5/src/liballoc/boxed.rs:922
#21 0xb233d074 in <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once () at /rustc/7efe1c6e678a263b9464f2e7f06f552b4d4db5a5/src/liballoc/boxed.rs:922
#22 std::sys_common::thread::start_thread () at src/libstd/sys_common/thread.rs:13
#23 std::sys::unix::thread::Thread::new::thread_start () at src/libstd/sys/unix/thread.rs:79
#24 0xb2108c40 in start_thread (arg=0xf225bd73) at pthread_create.c:486
#25 0xb2248a2c in ?? () at ../sysdeps/unix/sysv/linux/arm/clone.S:73 from /lib/arm-linux-gnueabihf/libc.so.6
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

msizanoen1 on 22 Sep 2019

It still sometime segfaults even with malloc checking enabled. No idea wether this is related to malloc or not.

msizanoen1 on 22 Sep 2019

The CI also statically link libstdc++ so it may also be caused by old toolchain version.

msizanoen1 on 22 Sep 2019

Compilation still fails with codegen-units = 1 when LTO is enabled.

jedisct1 on 22 Sep 2019

You use -C lto=fat or -C lto=thin?

msizanoen1 on 22 Sep 2019

Looks like ThinLTO causes this:

[INFO  jobserver::imp] created a jobserver: Client { read: File { fd: 3, path: "pipe:[676339]", read: true, write: false }, write: File { fd: 4, path: "pipe:[676339]", read: false, write: true } }
[INFO  rustc_interface::util] codegen backend candidate: /home/sirius/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/codegen-backends
[INFO  rustc_interface::util] probing /home/sirius/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/codegen-backends for a codegen backend
[INFO  rustc_metadata::creader] resolving crate `extern crate std as std`
[INFO  rustc_metadata::creader] falling back to a load
[INFO  rustc_metadata::locator] lib candidate: /home/sirius/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/lib/libstd-a6cd0d047274cc3c.rlib
[INFO  rustc_metadata::locator] lib candidate: /home/sirius/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/lib/libstd-a6cd0d047274cc3c.so
[INFO  rustc_metadata::locator] rlib reading metadata from: /home/sirius/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/lib/libstd-a6cd0d047274cc3c.rlib
[INFO  rustc_metadata::locator] reading "libstd-a6cd0d047274cc3c.rlib" => 259.635µs
[INFO  rustc_metadata::creader] register crate `extern crate std as std` (private_dep = false)
[INFO  rustc_metadata::creader] resolving dep crate core hash: `1a4aa887887e9e3f` extra filename: `-21f7450fb1cc2820`
[INFO  rustc_metadata::creader] resolving crate `extern crate core as core`
[INFO  rustc_metadata::creader] falling back to a load
[INFO  rustc_metadata::locator] lib candidate: /home/sirius/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/lib/libcore-21f7450fb1cc2820.rlib
[INFO  rustc_metadata::locator] rlib reading metadata from: /home/sirius/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/lib/libcore-21f7450fb1cc2820.rlib
[INFO  rustc_metadata::locator] reading "libcore-21f7450fb1cc2820.rlib" => 102.135µs
[INFO  rustc_metadata::creader] register crate `extern crate core as core` (private_dep = false)
[INFO  rustc_metadata::creader] resolving dep crate compiler_builtins hash: `3f14276f9d4fd15c` extra filename: `-8bc556f374887155`
[INFO  rustc_metadata::creader] resolving crate `extern crate compiler_builtins as compiler_builtins`
[INFO  rustc_metadata::creader] falling back to a load
[INFO  rustc_metadata::locator] lib candidate: /home/sirius/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/lib/libcompiler_builtins-8bc556f374887155.rlib
[INFO  rustc_metadata::locator] rlib reading metadata from: /home/sirius/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/lib/libcompiler_builtins-8bc556f374887155.rlib
[INFO  rustc_metadata::locator] reading "libcompiler_builtins-8bc556f374887155.rlib" => 622.916µs
[INFO  rustc_metadata::creader] register crate `extern crate compiler_builtins as compiler_builtins` (private_dep = false)
[INFO  rustc_metadata::creader] resolving dep crate rustc_std_workspace_core hash: `944df03f3ebec504` extra filename: `-979bf1a1fa55f0f0`
[INFO  rustc_metadata::creader] resolving crate `extern crate rustc_std_workspace_core as rustc_std_workspace_core`
[INFO  rustc_metadata::creader] falling back to a load
[INFO  rustc_metadata::locator] lib candidate: /home/sirius/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/lib/librustc_std_workspace_core-979bf1a1fa55f0f0.rlib
[INFO  rustc_metadata::locator] rlib reading metadata from: /home/sirius/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/lib/librustc_std_workspace_core-979bf1a1fa55f0f0.rlib
[INFO  rustc_metadata::locator] reading "librustc_std_workspace_core-979bf1a1fa55f0f0.rlib" => 149.636µs
[INFO  rustc_metadata::creader] register crate `extern crate rustc_std_workspace_core as rustc_std_workspace_core` (private_dep = false)
[INFO  rustc_metadata::creader] resolving dep crate core hash: `1a4aa887887e9e3f` extra filename: `-21f7450fb1cc2820`
[INFO  rustc_metadata::creader] resolving crate `extern crate core as core`
[INFO  rustc_metadata::creader] resolving dep crate core hash: `1a4aa887887e9e3f` extra filename: `-21f7450fb1cc2820`
[INFO  rustc_metadata::creader] resolving crate `extern crate core as core`
[INFO  rustc_metadata::creader] resolving dep crate rustc_std_workspace_core hash: `944df03f3ebec504` extra filename: `-979bf1a1fa55f0f0`
[INFO  rustc_metadata::creader] resolving crate `extern crate rustc_std_workspace_core as rustc_std_workspace_core`
[INFO  rustc_metadata::creader] resolving dep crate alloc hash: `ef1fb6ae7c545ce0` extra filename: `-2aac016ad2f1d216`
[INFO  rustc_metadata::creader] resolving crate `extern crate alloc as alloc`
[INFO  rustc_metadata::creader] falling back to a load
[INFO  rustc_metadata::locator] lib candidate: /home/sirius/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/lib/liballoc-2aac016ad2f1d216.rlib
[INFO  rustc_metadata::locator] rlib reading metadata from: /home/sirius/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/lib/liballoc-2aac016ad2f1d216.rlib
[INFO  rustc_metadata::locator] reading "liballoc-2aac016ad2f1d216.rlib" => 90.781µs
[INFO  rustc_metadata::creader] register crate `extern crate alloc as alloc` (private_dep = false)
[INFO  rustc_metadata::creader] resolving dep crate core hash: `1a4aa887887e9e3f` extra filename: `-21f7450fb1cc2820`
[INFO  rustc_metadata::creader] resolving crate `extern crate core as core`
[INFO  rustc_metadata::creader] resolving dep crate compiler_builtins hash: `3f14276f9d4fd15c` extra filename: `-8bc556f374887155`
[INFO  rustc_metadata::creader] resolving crate `extern crate compiler_builtins as compiler_builtins`
[INFO  rustc_metadata::creader] resolving dep crate rustc_std_workspace_core hash: `944df03f3ebec504` extra filename: `-979bf1a1fa55f0f0`
[INFO  rustc_metadata::creader] resolving crate `extern crate rustc_std_workspace_core as rustc_std_workspace_core`
[INFO  rustc_metadata::creader] resolving dep crate libc hash: `ad2793ca06518ba7` extra filename: `-c7908363f1addb30`
[INFO  rustc_metadata::creader] resolving crate `extern crate libc as libc`
[INFO  rustc_metadata::creader] falling back to a load
[INFO  rustc_metadata::locator] lib candidate: /home/sirius/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/lib/liblibc-c7908363f1addb30.rlib
[INFO  rustc_metadata::locator] rlib reading metadata from: /home/sirius/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/lib/liblibc-c7908363f1addb30.rlib
[INFO  rustc_metadata::locator] reading "liblibc-c7908363f1addb30.rlib" => 106.146µs
[INFO  rustc_metadata::creader] register crate `extern crate libc as libc` (private_dep = false)
[INFO  rustc_metadata::creader] resolving dep crate rustc_std_workspace_core hash: `944df03f3ebec504` extra filename: `-979bf1a1fa55f0f0`
[INFO  rustc_metadata::creader] resolving crate `extern crate rustc_std_workspace_core as rustc_std_workspace_core`
[INFO  rustc_metadata::creader] resolving dep crate core hash: `1a4aa887887e9e3f` extra filename: `-21f7450fb1cc2820`
[INFO  rustc_metadata::creader] resolving crate `extern crate core as core`
[INFO  rustc_metadata::creader] resolving dep crate unwind hash: `5365e3015c945582` extra filename: `-274f6d830714f60e`
[INFO  rustc_metadata::creader] resolving crate `extern crate unwind as unwind`
[INFO  rustc_metadata::creader] falling back to a load
[INFO  rustc_metadata::locator] lib candidate: /home/sirius/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/lib/libunwind-274f6d830714f60e.rlib
[INFO  rustc_metadata::locator] rlib reading metadata from: /home/sirius/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/lib/libunwind-274f6d830714f60e.rlib
[INFO  rustc_metadata::locator] reading "libunwind-274f6d830714f60e.rlib" => 123.594µs
[INFO  rustc_metadata::creader] register crate `extern crate unwind as unwind` (private_dep = false)
[INFO  rustc_metadata::creader] resolving dep crate core hash: `1a4aa887887e9e3f` extra filename: `-21f7450fb1cc2820`
[INFO  rustc_metadata::creader] resolving crate `extern crate core as core`
[INFO  rustc_metadata::creader] resolving dep crate compiler_builtins hash: `3f14276f9d4fd15c` extra filename: `-8bc556f374887155`
[INFO  rustc_metadata::creader] resolving crate `extern crate compiler_builtins as compiler_builtins`
[INFO  rustc_metadata::creader] resolving dep crate rustc_std_workspace_core hash: `944df03f3ebec504` extra filename: `-979bf1a1fa55f0f0`
[INFO  rustc_metadata::creader] resolving crate `extern crate rustc_std_workspace_core as rustc_std_workspace_core`
[INFO  rustc_metadata::creader] resolving dep crate cfg_if hash: `575dafe8896f1788` extra filename: `-152d9389be090d3c`
[INFO  rustc_metadata::creader] resolving crate `extern crate cfg_if as cfg_if`
[INFO  rustc_metadata::creader] falling back to a load
[INFO  rustc_metadata::locator] lib candidate: /home/sirius/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/lib/libcfg_if-152d9389be090d3c.rlib
[INFO  rustc_metadata::locator] rlib reading metadata from: /home/sirius/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/lib/libcfg_if-152d9389be090d3c.rlib
[INFO  rustc_metadata::locator] reading "libcfg_if-152d9389be090d3c.rlib" => 97.032µs
[INFO  rustc_metadata::creader] register crate `extern crate cfg_if as cfg_if` (private_dep = false)
[INFO  rustc_metadata::creader] resolving dep crate rustc_std_workspace_core hash: `944df03f3ebec504` extra filename: `-979bf1a1fa55f0f0`
[INFO  rustc_metadata::creader] resolving crate `extern crate rustc_std_workspace_core as rustc_std_workspace_core`
[INFO  rustc_metadata::creader] resolving dep crate core hash: `1a4aa887887e9e3f` extra filename: `-21f7450fb1cc2820`
[INFO  rustc_metadata::creader] resolving crate `extern crate core as core`
[INFO  rustc_metadata::creader] resolving dep crate compiler_builtins hash: `3f14276f9d4fd15c` extra filename: `-8bc556f374887155`
[INFO  rustc_metadata::creader] resolving crate `extern crate compiler_builtins as compiler_builtins`
[INFO  rustc_metadata::creader] resolving dep crate libc hash: `ad2793ca06518ba7` extra filename: `-c7908363f1addb30`
[INFO  rustc_metadata::creader] resolving crate `extern crate libc as libc`
[INFO  rustc_metadata::creader] resolving dep crate cfg_if hash: `575dafe8896f1788` extra filename: `-152d9389be090d3c`
[INFO  rustc_metadata::creader] resolving crate `extern crate cfg_if as cfg_if`
[INFO  rustc_metadata::creader] resolving dep crate backtrace hash: `a2a14f655022d6e7` extra filename: `-7117e551bea779c4`
[INFO  rustc_metadata::creader] resolving crate `extern crate backtrace as backtrace`
[INFO  rustc_metadata::creader] falling back to a load
[INFO  rustc_metadata::locator] lib candidate: /home/sirius/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/lib/libbacktrace-7117e551bea779c4.rlib
[INFO  rustc_metadata::locator] rlib reading metadata from: /home/sirius/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/lib/libbacktrace-7117e551bea779c4.rlib
[INFO  rustc_metadata::locator] reading "libbacktrace-7117e551bea779c4.rlib" => 85.312µs
[INFO  rustc_metadata::creader] register crate `extern crate backtrace as backtrace` (private_dep = false)
[INFO  rustc_metadata::creader] resolving dep crate rustc_std_workspace_core hash: `944df03f3ebec504` extra filename: `-979bf1a1fa55f0f0`
[INFO  rustc_metadata::creader] resolving crate `extern crate rustc_std_workspace_core as rustc_std_workspace_core`
[INFO  rustc_metadata::creader] resolving dep crate core hash: `1a4aa887887e9e3f` extra filename: `-21f7450fb1cc2820`
[INFO  rustc_metadata::creader] resolving crate `extern crate core as core`
[INFO  rustc_metadata::creader] resolving dep crate compiler_builtins hash: `3f14276f9d4fd15c` extra filename: `-8bc556f374887155`
[INFO  rustc_metadata::creader] resolving crate `extern crate compiler_builtins as compiler_builtins`
[INFO  rustc_metadata::creader] resolving dep crate rustc_demangle hash: `df7e991960a0ea4a` extra filename: `-cb52712bbcc60725`
[INFO  rustc_metadata::creader] resolving crate `extern crate rustc_demangle as rustc_demangle`
[INFO  rustc_metadata::creader] falling back to a load
[INFO  rustc_metadata::locator] lib candidate: /home/sirius/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/lib/librustc_demangle-cb52712bbcc60725.rlib
[INFO  rustc_metadata::locator] rlib reading metadata from: /home/sirius/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/lib/librustc_demangle-cb52712bbcc60725.rlib
[INFO  rustc_metadata::locator] reading "librustc_demangle-cb52712bbcc60725.rlib" => 75.833µs
[INFO  rustc_metadata::creader] register crate `extern crate rustc_demangle as rustc_demangle` (private_dep = false)
[INFO  rustc_metadata::creader] resolving dep crate rustc_std_workspace_core hash: `944df03f3ebec504` extra filename: `-979bf1a1fa55f0f0`
[INFO  rustc_metadata::creader] resolving crate `extern crate rustc_std_workspace_core as rustc_std_workspace_core`
[INFO  rustc_metadata::creader] resolving dep crate core hash: `1a4aa887887e9e3f` extra filename: `-21f7450fb1cc2820`
[INFO  rustc_metadata::creader] resolving crate `extern crate core as core`
[INFO  rustc_metadata::creader] resolving dep crate compiler_builtins hash: `3f14276f9d4fd15c` extra filename: `-8bc556f374887155`
[INFO  rustc_metadata::creader] resolving crate `extern crate compiler_builtins as compiler_builtins`
[INFO  rustc_metadata::creader] resolving dep crate cfg_if hash: `575dafe8896f1788` extra filename: `-152d9389be090d3c`
[INFO  rustc_metadata::creader] resolving crate `extern crate cfg_if as cfg_if`
[INFO  rustc_metadata::creader] resolving dep crate backtrace_sys hash: `73c9b43f26cbe5b1` extra filename: `-74b555b5f020d0f0`
[INFO  rustc_metadata::creader] resolving crate `extern crate backtrace_sys as backtrace_sys`
[INFO  rustc_metadata::creader] falling back to a load
[INFO  rustc_metadata::locator] lib candidate: /home/sirius/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/lib/libbacktrace_sys-74b555b5f020d0f0.rlib
[INFO  rustc_metadata::locator] rlib reading metadata from: /home/sirius/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/lib/libbacktrace_sys-74b555b5f020d0f0.rlib
[INFO  rustc_metadata::locator] reading "libbacktrace_sys-74b555b5f020d0f0.rlib" => 110.886µs
[INFO  rustc_metadata::creader] register crate `extern crate backtrace_sys as backtrace_sys` (private_dep = false)
[INFO  rustc_metadata::creader] resolving dep crate rustc_std_workspace_core hash: `944df03f3ebec504` extra filename: `-979bf1a1fa55f0f0`
[INFO  rustc_metadata::creader] resolving crate `extern crate rustc_std_workspace_core as rustc_std_workspace_core`
[INFO  rustc_metadata::creader] resolving dep crate core hash: `1a4aa887887e9e3f` extra filename: `-21f7450fb1cc2820`
[INFO  rustc_metadata::creader] resolving crate `extern crate core as core`
[INFO  rustc_metadata::creader] resolving dep crate compiler_builtins hash: `3f14276f9d4fd15c` extra filename: `-8bc556f374887155`
[INFO  rustc_metadata::creader] resolving crate `extern crate compiler_builtins as compiler_builtins`
[INFO  rustc_metadata::creader] resolving dep crate libc hash: `ad2793ca06518ba7` extra filename: `-c7908363f1addb30`
[INFO  rustc_metadata::creader] resolving crate `extern crate libc as libc`
[INFO  rustc_metadata::creader] resolving dep crate libc hash: `ad2793ca06518ba7` extra filename: `-c7908363f1addb30`
[INFO  rustc_metadata::creader] resolving crate `extern crate libc as libc`
[INFO  rustc_metadata::creader] resolving dep crate rustc_demangle hash: `df7e991960a0ea4a` extra filename: `-cb52712bbcc60725`
[INFO  rustc_metadata::creader] resolving crate `extern crate rustc_demangle as rustc_demangle`
[INFO  rustc_metadata::creader] resolving dep crate backtrace_sys hash: `73c9b43f26cbe5b1` extra filename: `-74b555b5f020d0f0`
[INFO  rustc_metadata::creader] resolving crate `extern crate backtrace_sys as backtrace_sys`
[INFO  rustc_metadata::creader] resolving dep crate hashbrown hash: `db6d6af3b5b969ab` extra filename: `-65f8983efb136987`
[INFO  rustc_metadata::creader] resolving crate `extern crate hashbrown as hashbrown`
[INFO  rustc_metadata::creader] falling back to a load
[INFO  rustc_metadata::locator] lib candidate: /home/sirius/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/lib/libhashbrown-65f8983efb136987.rlib
[INFO  rustc_metadata::locator] rlib reading metadata from: /home/sirius/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/lib/libhashbrown-65f8983efb136987.rlib
[INFO  rustc_metadata::locator] reading "libhashbrown-65f8983efb136987.rlib" => 72.344µs
[INFO  rustc_metadata::creader] register crate `extern crate hashbrown as hashbrown` (private_dep = false)
[INFO  rustc_metadata::creader] resolving dep crate rustc_std_workspace_core hash: `944df03f3ebec504` extra filename: `-979bf1a1fa55f0f0`
[INFO  rustc_metadata::creader] resolving crate `extern crate rustc_std_workspace_core as rustc_std_workspace_core`
[INFO  rustc_metadata::creader] resolving dep crate core hash: `1a4aa887887e9e3f` extra filename: `-21f7450fb1cc2820`
[INFO  rustc_metadata::creader] resolving crate `extern crate core as core`
[INFO  rustc_metadata::creader] resolving dep crate compiler_builtins hash: `3f14276f9d4fd15c` extra filename: `-8bc556f374887155`
[INFO  rustc_metadata::creader] resolving crate `extern crate compiler_builtins as compiler_builtins`
[INFO  rustc_metadata::creader] resolving dep crate rustc_std_workspace_alloc hash: `af90ff35903f91a5` extra filename: `-b229420cedcf2ee4`
[INFO  rustc_metadata::creader] resolving crate `extern crate rustc_std_workspace_alloc as rustc_std_workspace_alloc`
[INFO  rustc_metadata::creader] falling back to a load
[INFO  rustc_metadata::locator] lib candidate: /home/sirius/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/lib/librustc_std_workspace_alloc-b229420cedcf2ee4.rlib
[INFO  rustc_metadata::locator] rlib reading metadata from: /home/sirius/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/lib/librustc_std_workspace_alloc-b229420cedcf2ee4.rlib
[INFO  rustc_metadata::locator] reading "librustc_std_workspace_alloc-b229420cedcf2ee4.rlib" => 69.896µs
[INFO  rustc_metadata::creader] register crate `extern crate rustc_std_workspace_alloc as rustc_std_workspace_alloc` (private_dep = false)
[INFO  rustc_metadata::creader] resolving dep crate alloc hash: `ef1fb6ae7c545ce0` extra filename: `-2aac016ad2f1d216`
[INFO  rustc_metadata::creader] resolving crate `extern crate alloc as alloc`
[INFO  rustc_metadata::creader] resolving dep crate core hash: `1a4aa887887e9e3f` extra filename: `-21f7450fb1cc2820`
[INFO  rustc_metadata::creader] resolving crate `extern crate core as core`
[INFO  rustc_metadata::creader] resolving dep crate compiler_builtins hash: `3f14276f9d4fd15c` extra filename: `-8bc556f374887155`
[INFO  rustc_metadata::creader] resolving crate `extern crate compiler_builtins as compiler_builtins`
[INFO  rustc_metadata::creader] resolving dep crate rustc_std_workspace_core hash: `944df03f3ebec504` extra filename: `-979bf1a1fa55f0f0`
[INFO  rustc_metadata::creader] resolving crate `extern crate rustc_std_workspace_core as rustc_std_workspace_core`
[INFO  rustc_metadata::creader] resolving dep crate alloc hash: `ef1fb6ae7c545ce0` extra filename: `-2aac016ad2f1d216`
[INFO  rustc_metadata::creader] resolving crate `extern crate alloc as alloc`
[INFO  rustc_metadata::creader] resolving dep crate rustc_std_workspace_alloc hash: `af90ff35903f91a5` extra filename: `-b229420cedcf2ee4`
[INFO  rustc_metadata::creader] resolving crate `extern crate rustc_std_workspace_alloc as rustc_std_workspace_alloc`
[INFO  rustc_metadata::creader] resolving dep crate panic_unwind hash: `bcbfcfd00b91a521` extra filename: `-870bf9d53373dc66`
[INFO  rustc_metadata::creader] resolving crate `extern crate panic_unwind as panic_unwind`
[INFO  rustc_metadata::creader] falling back to a load
[INFO  rustc_metadata::locator] lib candidate: /home/sirius/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/lib/libpanic_unwind-870bf9d53373dc66.rlib
[INFO  rustc_metadata::locator] rlib reading metadata from: /home/sirius/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/lib/libpanic_unwind-870bf9d53373dc66.rlib
[INFO  rustc_metadata::locator] reading "libpanic_unwind-870bf9d53373dc66.rlib" => 77.604µs
[INFO  rustc_metadata::creader] register crate `extern crate panic_unwind as panic_unwind` (private_dep = false)
[INFO  rustc_metadata::creader] resolving dep crate core hash: `1a4aa887887e9e3f` extra filename: `-21f7450fb1cc2820`
[INFO  rustc_metadata::creader] resolving crate `extern crate core as core`
[INFO  rustc_metadata::creader] resolving dep crate compiler_builtins hash: `3f14276f9d4fd15c` extra filename: `-8bc556f374887155`
[INFO  rustc_metadata::creader] resolving crate `extern crate compiler_builtins as compiler_builtins`
[INFO  rustc_metadata::creader] resolving dep crate rustc_std_workspace_core hash: `944df03f3ebec504` extra filename: `-979bf1a1fa55f0f0`
[INFO  rustc_metadata::creader] resolving crate `extern crate rustc_std_workspace_core as rustc_std_workspace_core`
[INFO  rustc_metadata::creader] resolving dep crate alloc hash: `ef1fb6ae7c545ce0` extra filename: `-2aac016ad2f1d216`
[INFO  rustc_metadata::creader] resolving crate `extern crate alloc as alloc`
[INFO  rustc_metadata::creader] resolving dep crate cfg_if hash: `575dafe8896f1788` extra filename: `-152d9389be090d3c`
[INFO  rustc_metadata::creader] resolving crate `extern crate cfg_if as cfg_if`
[INFO  rustc_metadata::creader] resolving dep crate unwind hash: `5365e3015c945582` extra filename: `-274f6d830714f60e`
[INFO  rustc_metadata::creader] resolving crate `extern crate unwind as unwind`
[INFO  rustc_metadata::creader] resolving dep crate libc hash: `ad2793ca06518ba7` extra filename: `-c7908363f1addb30`
[INFO  rustc_metadata::creader] resolving crate `extern crate libc as libc`
[INFO  rustc_metadata::creader] injecting a dep from 1 to 14
[INFO  rustc_metadata::creader] panic runtime not found -- loading panic_unwind
[INFO  rustc_metadata::creader] resolving crate `extern crate panic_unwind as panic_unwind`
[INFO  rustc_metadata::creader] injecting a dep from 1 to 14
[INFO  rustc_metadata::creader] resolved crates:
[INFO  rustc_metadata::creader]   name: std
[INFO  rustc_metadata::creader]   cnum: 1
[INFO  rustc_metadata::creader]   hash: 84bcd27f2b336682
[INFO  rustc_metadata::creader]   reqd: Explicit
[INFO  rustc_metadata::creader]   dylib: /home/sirius/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/lib/libstd-a6cd0d047274cc3c.so
[INFO  rustc_metadata::creader]    rlib: /home/sirius/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/lib/libstd-a6cd0d047274cc3c.rlib
[INFO  rustc_metadata::creader]   name: core
[INFO  rustc_metadata::creader]   cnum: 2
[INFO  rustc_metadata::creader]   hash: 1a4aa887887e9e3f
[INFO  rustc_metadata::creader]   reqd: Explicit
[INFO  rustc_metadata::creader]    rlib: /home/sirius/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/lib/libcore-21f7450fb1cc2820.rlib
[INFO  rustc_metadata::creader]   name: compiler_builtins
[INFO  rustc_metadata::creader]   cnum: 3
[INFO  rustc_metadata::creader]   hash: 3f14276f9d4fd15c
[INFO  rustc_metadata::creader]   reqd: Explicit
[INFO  rustc_metadata::creader]    rlib: /home/sirius/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/lib/libcompiler_builtins-8bc556f374887155.rlib
[INFO  rustc_metadata::creader]   name: rustc_std_workspace_core
[INFO  rustc_metadata::creader]   cnum: 4
[INFO  rustc_metadata::creader]   hash: 944df03f3ebec504
[INFO  rustc_metadata::creader]   reqd: Explicit
[INFO  rustc_metadata::creader]    rlib: /home/sirius/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/lib/librustc_std_workspace_core-979bf1a1fa55f0f0.rlib
[INFO  rustc_metadata::creader]   name: alloc
[INFO  rustc_metadata::creader]   cnum: 5
[INFO  rustc_metadata::creader]   hash: ef1fb6ae7c545ce0
[INFO  rustc_metadata::creader]   reqd: Explicit
[INFO  rustc_metadata::creader]    rlib: /home/sirius/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/lib/liballoc-2aac016ad2f1d216.rlib
[INFO  rustc_metadata::creader]   name: libc
[INFO  rustc_metadata::creader]   cnum: 6
[INFO  rustc_metadata::creader]   hash: ad2793ca06518ba7
[INFO  rustc_metadata::creader]   reqd: Explicit
[INFO  rustc_metadata::creader]    rlib: /home/sirius/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/lib/liblibc-c7908363f1addb30.rlib
[INFO  rustc_metadata::creader]   name: unwind
[INFO  rustc_metadata::creader]   cnum: 7
[INFO  rustc_metadata::creader]   hash: 5365e3015c945582
[INFO  rustc_metadata::creader]   reqd: Explicit
[INFO  rustc_metadata::creader]    rlib: /home/sirius/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/lib/libunwind-274f6d830714f60e.rlib
[INFO  rustc_metadata::creader]   name: cfg_if
[INFO  rustc_metadata::creader]   cnum: 8
[INFO  rustc_metadata::creader]   hash: 575dafe8896f1788
[INFO  rustc_metadata::creader]   reqd: Explicit
[INFO  rustc_metadata::creader]    rlib: /home/sirius/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/lib/libcfg_if-152d9389be090d3c.rlib
[INFO  rustc_metadata::creader]   name: backtrace
[INFO  rustc_metadata::creader]   cnum: 9
[INFO  rustc_metadata::creader]   hash: a2a14f655022d6e7
[INFO  rustc_metadata::creader]   reqd: Explicit
[INFO  rustc_metadata::creader]    rlib: /home/sirius/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/lib/libbacktrace-7117e551bea779c4.rlib
[INFO  rustc_metadata::creader]   name: rustc_demangle
[INFO  rustc_metadata::creader]   cnum: 10
[INFO  rustc_metadata::creader]   hash: df7e991960a0ea4a
[INFO  rustc_metadata::creader]   reqd: Explicit
[INFO  rustc_metadata::creader]    rlib: /home/sirius/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/lib/librustc_demangle-cb52712bbcc60725.rlib
[INFO  rustc_metadata::creader]   name: backtrace_sys
[INFO  rustc_metadata::creader]   cnum: 11
[INFO  rustc_metadata::creader]   hash: 73c9b43f26cbe5b1
[INFO  rustc_metadata::creader]   reqd: Explicit
[INFO  rustc_metadata::creader]    rlib: /home/sirius/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/lib/libbacktrace_sys-74b555b5f020d0f0.rlib
[INFO  rustc_metadata::creader]   name: hashbrown
[INFO  rustc_metadata::creader]   cnum: 12
[INFO  rustc_metadata::creader]   hash: db6d6af3b5b969ab
[INFO  rustc_metadata::creader]   reqd: Explicit
[INFO  rustc_metadata::creader]    rlib: /home/sirius/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/lib/libhashbrown-65f8983efb136987.rlib
[INFO  rustc_metadata::creader]   name: rustc_std_workspace_alloc
[INFO  rustc_metadata::creader]   cnum: 13
[INFO  rustc_metadata::creader]   hash: af90ff35903f91a5
[INFO  rustc_metadata::creader]   reqd: Explicit
[INFO  rustc_metadata::creader]    rlib: /home/sirius/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/lib/librustc_std_workspace_alloc-b229420cedcf2ee4.rlib
[INFO  rustc_metadata::creader]   name: panic_unwind
[INFO  rustc_metadata::creader]   cnum: 14
[INFO  rustc_metadata::creader]   hash: bcbfcfd00b91a521
[INFO  rustc_metadata::creader]   reqd: Implicit
[INFO  rustc_metadata::creader]    rlib: /home/sirius/.rustup/toolchains/nightly-armv7-unknown-linux-gnueabihf/lib/rustlib/armv7-unknown-linux-gnueabihf/lib/libpanic_unwind-870bf9d53373dc66.rlib
[INFO  rustc_interface::passes] 0 parse sess buffered_lints
[INFO  rustc_mir::build] fn_id DefId(0:12 ~ hello[317d]::main[0]) has attrs Borrowed([])
Pre-codegen
Ty interner             total           ty lt ct all
    Adt               :    432 61.2%,  0.0%   3.5%  0.0%  0.0%
    Array             :     34  4.8%,  0.4%   3.3%  0.0%  0.0%
    Slice             :     28  4.0%,  0.0%   1.8%  0.0%  0.0%
    RawPtr            :      6  0.8%,  0.3%   0.0%  0.0%  0.0%
    Ref               :    147 20.8%,  6.1%  13.9%  0.0%  0.0%
    FnDef             :     12  1.7%,  0.0%   0.7%  0.0%  0.0%
    FnPtr             :      2  0.3%,  0.0%   0.0%  0.0%  0.0%
    Placeholder       :      0  0.0%,  0.0%   0.0%  0.0%  0.0%
    Generator         :      0  0.0%,  0.0%   0.0%  0.0%  0.0%
    GeneratorWitness  :      0  0.0%,  0.0%   0.0%  0.0%  0.0%
    Dynamic           :      0  0.0%,  0.0%   0.0%  0.0%  0.0%
    Closure           :      0  0.0%,  0.0%   0.0%  0.0%  0.0%
    Tuple             :      1  0.1%,  0.0%   0.0%  0.0%  0.0%
    Bound             :      0  0.0%,  0.0%   0.0%  0.0%  0.0%
    Param             :     28  4.0%,  0.0%   0.0%  0.0%  0.0%
    Infer             :     16  2.3%,  2.3%   0.0%  0.0%  0.0%
    UnnormalizedProjection:      0  0.0%,  0.0%   0.0%  0.0%  0.0%
    Projection        :      0  0.0%,  0.0%   0.0%  0.0%  0.0%
    Opaque            :      0  0.0%,  0.0%   0.0%  0.0%  0.0%
    Foreign           :      0  0.0%,  0.0%   0.0%  0.0%  0.0%
                  total    706         9.1%  23.2%  0.0%  0.0%
InternalSubsts interner: #238
Region interner: #125
Stability interner: #4
Allocation interner: #1
Layout interner: #1
[INFO  rustc_mir::interpret::eval_context] ENTERING(0) main
[INFO  rustc_mir::interpret::eval_context] ENTERING(0) main
[INFO  rustc_mir::interpret::step] _3 = const "Hello, world\n"
[INFO  rustc_mir::interpret::step] _2 = _3
[INFO  rustc_mir::interpret::step] _1 = [move _2]
[INFO  rustc_mir::interpret::step] _0 = move _1
[INFO  rustc_mir::interpret::step] return
[INFO  rustc_mir::interpret::eval_context] LEAVING(0) main
[INFO  rustc_mir::interpret::eval_context] ENTERING(0) main
[INFO  rustc_mir::interpret::step] _1 = []
[INFO  rustc_mir::interpret::step] _0 = move _1
[INFO  rustc_mir::interpret::step] return
[INFO  rustc_mir::interpret::eval_context] LEAVING(0) main
[INFO  rustc::traits::codegen] Cache miss: Binder(<[closure@DefId(1:6024 ~ std[ed44]::rt[0]::lang_start[0]::{{closure}}[0]) 0:fn()] as std::ops::Fn<()>>) => VtableClosureData(closure_def_id=DefId(1:6024 ~ std[ed44]::rt[0]::lang_start[0]::{{closure}}[0]), substs=ClosureSubsts { substs: [(), i8, extern "rust-call" fn(()) -> i32, fn()] }, nested=[()])
[INFO  rustc::traits::codegen] Cache miss: Binder(<[closure@DefId(1:6024 ~ std[ed44]::rt[0]::lang_start[0]::{{closure}}[0]) 0:fn()] as std::ops::FnMut<()>>) => VtableClosureData(closure_def_id=DefId(1:6024 ~ std[ed44]::rt[0]::lang_start[0]::{{closure}}[0]), substs=ClosureSubsts { substs: [(), i8, extern "rust-call" fn(()) -> i32, fn()] }, nested=[()])
[INFO  rustc::traits::codegen] Cache miss: Binder(<() as std::process::Termination>) => VtableImplData(impl_def_id=DefId(1:4238 ~ std[ed44]::process[0]::{{impl}}[37]), substs=[], nested=[])
[INFO  rustc::traits::codegen] Cache miss: Binder(<std::process::ExitCode as std::process::Termination>) => VtableImplData(impl_def_id=DefId(1:4248 ~ std[ed44]::process[0]::{{impl}}[41]), substs=[], nested=[])
[INFO  rustc_mir::interpret::eval_context] ENTERING(0) std::process::ExitCode::SUCCESS
[INFO  rustc_mir::interpret::step] (_0.0: std::sys::unix::process::process_common::ExitCode) = const std::sys::unix::process::process_common::ExitCode::SUCCESS
[INFO  rustc_mir::interpret::eval_context] ENTERING(0) std::sys::unix::process::process_common::ExitCode::SUCCESS
[INFO  rustc_mir::interpret::step] StorageLive(_1)
[INFO  rustc_mir::interpret::step] StorageLive(_2)
[INFO  rustc_mir::interpret::step] _2 = const libc::unix::linux_like::EXIT_SUCCESS as u8 (Misc)
[INFO  rustc_mir::interpret::eval_context] ENTERING(0) libc::unix::linux_like::EXIT_SUCCESS
[INFO  rustc_mir::interpret::step] _0 = const 0i32
[INFO  rustc_mir::interpret::step] return
[INFO  rustc_mir::interpret::eval_context] LEAVING(0) libc::unix::linux_like::EXIT_SUCCESS
[INFO  rustc_mir::interpret::step] _1 = _2
[INFO  rustc_mir::interpret::step] (_0.0: u8) = move _1
[INFO  rustc_mir::interpret::step] StorageDead(_2)
[INFO  rustc_mir::interpret::step] return
[INFO  rustc_mir::interpret::eval_context] LEAVING(0) std::sys::unix::process::process_common::ExitCode::SUCCESS
[INFO  rustc_mir::interpret::step] return
[INFO  rustc_mir::interpret::eval_context] LEAVING(0) std::process::ExitCode::SUCCESS
[INFO  rustc::traits::codegen] Cache miss: Binder(<[closure@DefId(1:6024 ~ std[ed44]::rt[0]::lang_start[0]::{{closure}}[0]) 0:fn()] as std::ops::FnOnce<()>>) => VtableClosureData(closure_def_id=DefId(1:6024 ~ std[ed44]::rt[0]::lang_start[0]::{{closure}}[0]), substs=ClosureSubsts { substs: [(), i8, extern "rust-call" fn(()) -> i32, fn()] }, nested=[()])
[INFO  rustc_codegen_ssa::base] codegen_instance(std::fmt::Arguments::new_v1)
[INFO  rustc_codegen_ssa::base] codegen_instance(main)
[INFO  rustc_codegen_ssa::base] codegen_instance(std::rt::lang_start::<()>)
[INFO  rustc_codegen_ssa::base] codegen_instance(std::rt::lang_start::<()>::{{closure}}#0)
[INFO  rustc_codegen_ssa::base] codegen_instance(std::sys::unix::process::process_common::ExitCode::as_i32)
[INFO  rustc_codegen_ssa::base] codegen_instance(<[closure@DefId(1:6024 ~ std[ed44]::rt[0]::lang_start[0]::{{closure}}[0]) 0:fn()] as std::ops::FnOnce<()>>::call_once - shim(vtable))
[INFO  rustc_codegen_ssa::base] codegen_instance(<[closure@DefId(1:6024 ~ std[ed44]::rt[0]::lang_start[0]::{{closure}}[0]) 0:fn()] as std::ops::FnOnce<()>>::call_once - shim)
[INFO  rustc_mir::interpret::eval_context] ENTERING(0) unwind::libunwind::_Unwind_Reason_Code::_URC_NO_REASON::{{constant}}#0
[INFO  rustc_mir::interpret::step] _0 = const 0isize
[INFO  rustc_mir::interpret::step] return
[INFO  rustc_mir::interpret::eval_context] LEAVING(0) unwind::libunwind::_Unwind_Reason_Code::_URC_NO_REASON::{{constant}}#0
[INFO  rustc_mir::interpret::eval_context] ENTERING(0) unwind::libunwind::_Unwind_Reason_Code::_URC_FOREIGN_EXCEPTION_CAUGHT::{{constant}}#0
[INFO  rustc_mir::interpret::step] _0 = const 1isize
[INFO  rustc_mir::interpret::step] return
[INFO  rustc_mir::interpret::eval_context] LEAVING(0) unwind::libunwind::_Unwind_Reason_Code::_URC_FOREIGN_EXCEPTION_CAUGHT::{{constant}}#0
[INFO  rustc_mir::interpret::eval_context] ENTERING(0) unwind::libunwind::_Unwind_Reason_Code::_URC_FATAL_PHASE2_ERROR::{{constant}}#0
[INFO  rustc_mir::interpret::step] _0 = const 2isize
[INFO  rustc_mir::interpret::step] return
[INFO  rustc_mir::interpret::eval_context] LEAVING(0) unwind::libunwind::_Unwind_Reason_Code::_URC_FATAL_PHASE2_ERROR::{{constant}}#0
[INFO  rustc_mir::interpret::eval_context] ENTERING(0) unwind::libunwind::_Unwind_Reason_Code::_URC_FATAL_PHASE1_ERROR::{{constant}}#0
[INFO  rustc_mir::interpret::step] _0 = const 3isize
[INFO  rustc_mir::interpret::step] return
[INFO  rustc_mir::interpret::eval_context] LEAVING(0) unwind::libunwind::_Unwind_Reason_Code::_URC_FATAL_PHASE1_ERROR::{{constant}}#0
[INFO  rustc_mir::interpret::eval_context] ENTERING(0) unwind::libunwind::_Unwind_Reason_Code::_URC_NORMAL_STOP::{{constant}}#0
[INFO  rustc_mir::interpret::step] _0 = const 4isize
[INFO  rustc_mir::interpret::step] return
[INFO  rustc_mir::interpret::eval_context] LEAVING(0) unwind::libunwind::_Unwind_Reason_Code::_URC_NORMAL_STOP::{{constant}}#0
[INFO  rustc_mir::interpret::eval_context] ENTERING(0) unwind::libunwind::_Unwind_Reason_Code::_URC_END_OF_STACK::{{constant}}#0
[INFO  rustc_mir::interpret::step] _0 = const 5isize
[INFO  rustc_mir::interpret::step] return
[INFO  rustc_mir::interpret::eval_context] LEAVING(0) unwind::libunwind::_Unwind_Reason_Code::_URC_END_OF_STACK::{{constant}}#0
[INFO  rustc_mir::interpret::eval_context] ENTERING(0) unwind::libunwind::_Unwind_Reason_Code::_URC_HANDLER_FOUND::{{constant}}#0
[INFO  rustc_mir::interpret::step] _0 = const 6isize
[INFO  rustc_mir::interpret::step] return
[INFO  rustc_mir::interpret::eval_context] LEAVING(0) unwind::libunwind::_Unwind_Reason_Code::_URC_HANDLER_FOUND::{{constant}}#0
[INFO  rustc_mir::interpret::eval_context] ENTERING(0) unwind::libunwind::_Unwind_Reason_Code::_URC_INSTALL_CONTEXT::{{constant}}#0
[INFO  rustc_mir::interpret::step] _0 = const 7isize
[INFO  rustc_mir::interpret::step] return
[INFO  rustc_mir::interpret::eval_context] LEAVING(0) unwind::libunwind::_Unwind_Reason_Code::_URC_INSTALL_CONTEXT::{{constant}}#0
[INFO  rustc_mir::interpret::eval_context] ENTERING(0) unwind::libunwind::_Unwind_Reason_Code::_URC_CONTINUE_UNWIND::{{constant}}#0
[INFO  rustc_mir::interpret::step] _0 = const 8isize
[INFO  rustc_mir::interpret::step] return
[INFO  rustc_mir::interpret::eval_context] LEAVING(0) unwind::libunwind::_Unwind_Reason_Code::_URC_CONTINUE_UNWIND::{{constant}}#0
[INFO  rustc_mir::interpret::eval_context] ENTERING(0) unwind::libunwind::_Unwind_Reason_Code::_URC_FAILURE::{{constant}}#0
[INFO  rustc_mir::interpret::step] _0 = const 9isize
[INFO  rustc_mir::interpret::step] return
[INFO  rustc_mir::interpret::eval_context] LEAVING(0) unwind::libunwind::_Unwind_Reason_Code::_URC_FAILURE::{{constant}}#0
[INFO  rustc_mir::interpret::eval_context] ENTERING(0) unwind::libunwind::_Unwind_State::_US_VIRTUAL_UNWIND_FRAME::{{constant}}#0
[INFO  rustc_mir::interpret::step] _0 = const 0isize
[INFO  rustc_mir::interpret::step] return
[INFO  rustc_mir::interpret::eval_context] LEAVING(0) unwind::libunwind::_Unwind_State::_US_VIRTUAL_UNWIND_FRAME::{{constant}}#0
[INFO  rustc_mir::interpret::eval_context] ENTERING(0) unwind::libunwind::_Unwind_State::_US_UNWIND_FRAME_STARTING::{{constant}}#0
[INFO  rustc_mir::interpret::step] _0 = const 1isize
[INFO  rustc_mir::interpret::step] return
[INFO  rustc_mir::interpret::eval_context] LEAVING(0) unwind::libunwind::_Unwind_State::_US_UNWIND_FRAME_STARTING::{{constant}}#0
[INFO  rustc_mir::interpret::eval_context] ENTERING(0) unwind::libunwind::_Unwind_State::_US_UNWIND_FRAME_RESUME::{{constant}}#0
[INFO  rustc_mir::interpret::step] _0 = const 2isize
[INFO  rustc_mir::interpret::step] return
[INFO  rustc_mir::interpret::eval_context] LEAVING(0) unwind::libunwind::_Unwind_State::_US_UNWIND_FRAME_RESUME::{{constant}}#0
[INFO  rustc_mir::interpret::eval_context] ENTERING(0) unwind::libunwind::_Unwind_State::_US_ACTION_MASK::{{constant}}#0
[INFO  rustc_mir::interpret::step] _0 = const 3isize
[INFO  rustc_mir::interpret::step] return
[INFO  rustc_mir::interpret::eval_context] LEAVING(0) unwind::libunwind::_Unwind_State::_US_ACTION_MASK::{{constant}}#0
[INFO  rustc_mir::interpret::eval_context] ENTERING(0) unwind::libunwind::_Unwind_State::_US_FORCE_UNWIND::{{constant}}#0
[INFO  rustc_mir::interpret::step] _0 = const 8isize
[INFO  rustc_mir::interpret::step] return
[INFO  rustc_mir::interpret::eval_context] LEAVING(0) unwind::libunwind::_Unwind_State::_US_FORCE_UNWIND::{{constant}}#0
[INFO  rustc_mir::interpret::eval_context] ENTERING(0) unwind::libunwind::_Unwind_State::_US_END_OF_STACK::{{constant}}#0
[INFO  rustc_mir::interpret::step] _0 = const 16isize
[INFO  rustc_mir::interpret::step] return
[INFO  rustc_mir::interpret::eval_context] LEAVING(0) unwind::libunwind::_Unwind_State::_US_END_OF_STACK::{{constant}}#0
[INFO  rustc_mir::interpret::eval_context] ENTERING(0) unwind::libunwind::_Unwind_Exception::private::{{constant}}#0
[INFO  rustc_mir::interpret::step] _0 = const unwind::libunwind::unwinder_private_data_size
[INFO  rustc_mir::interpret::eval_context] ENTERING(0) unwind::libunwind::unwinder_private_data_size
[INFO  rustc_mir::interpret::step] _0 = const 20usize
[INFO  rustc_mir::interpret::step] return
[INFO  rustc_mir::interpret::eval_context] LEAVING(0) unwind::libunwind::unwinder_private_data_size
[INFO  rustc_mir::interpret::step] return
[INFO  rustc_mir::interpret::eval_context] LEAVING(0) unwind::libunwind::_Unwind_Exception::private::{{constant}}#0
[INFO  rustc_codegen_ssa::base] codegen_instance(std::ptr::real_drop_in_place::<[closure@DefId(1:6024 ~ std[ed44]::rt[0]::lang_start[0]::{{closure}}[0]) 0:fn()]> - shim(None))
[INFO  rustc_codegen_ssa::base] codegen_instance(<() as std::process::Termination>::report)
[INFO  rustc_codegen_ssa::base] codegen_instance(<std::process::ExitCode as std::process::Termination>::report)
Post-codegen
Ty interner             total           ty lt ct all
    Adt               :    451 53.2%,  0.1%   3.1%  0.0%  0.0%
    Array             :     37  4.4%,  0.4%   2.7%  0.0%  0.0%
    Slice             :     28  3.3%,  0.0%   1.5%  0.0%  0.0%
    RawPtr            :     19  2.2%,  0.2%   0.1%  0.0%  0.0%
    Ref               :    205 24.2%,  5.5%  12.5%  0.0%  0.0%
    FnDef             :     28  3.3%,  0.0%   0.6%  0.0%  0.0%
    FnPtr             :     12  1.4%,  0.0%   0.0%  0.0%  0.0%
    Placeholder       :      0  0.0%,  0.0%   0.0%  0.0%  0.0%
    Generator         :      0  0.0%,  0.0%   0.0%  0.0%  0.0%
    GeneratorWitness  :      0  0.0%,  0.0%   0.0%  0.0%  0.0%
    Dynamic           :     11  1.3%,  0.0%   0.1%  0.0%  0.0%
    Closure           :      3  0.4%,  0.0%   0.0%  0.0%  0.0%
    Tuple             :      2  0.2%,  0.0%   0.0%  0.0%  0.0%
    Bound             :      1  0.1%,  0.0%   0.0%  0.0%  0.0%
    Param             :     33  3.9%,  0.0%   0.0%  0.0%  0.0%
    Infer             :     16  1.9%,  1.9%   0.0%  0.0%  0.0%
    UnnormalizedProjection:      0  0.0%,  0.0%   0.0%  0.0%  0.0%
    Projection        :      2  0.2%,  0.0%   0.0%  0.0%  0.0%
    Opaque            :      0  0.0%,  0.0%   0.0%  0.0%  0.0%
    Foreign           :      0  0.0%,  0.0%   0.0%  0.0%  0.0%
                  total    848         8.1%  20.6%  0.0%  0.0%
InternalSubsts interner: #322
Region interner: #137
Stability interner: #4
Allocation interner: #16
Layout interner: #36
[INFO  rustc_codegen_llvm::back::lto] 5 symbols to preserve in this crate
[INFO  rustc_codegen_llvm::back::lto] going for that thin, thin LTO
[INFO  rustc_codegen_llvm::back::lto] local module: 0 - hello.7rcbfp3g-cgu.1
[INFO  rustc_codegen_llvm::back::lto] local module: 1 - hello.7rcbfp3g-cgu.0
[INFO  rustc_codegen_llvm::back::lto] thin LTO data created
[INFO  rustc_codegen_llvm::back::lto] thin LTO import map loaded
[INFO  rustc_codegen_llvm::back::lto] checking which modules can be-reused and which have to be re-optimized.
[INFO  rustc_codegen_llvm::back::lto]  - hello.7rcbfp3g-cgu.1: re-compiled
[INFO  rustc_codegen_llvm::back::lto]  - hello.7rcbfp3g-cgu.0: re-compiled
[INFO  rustc_codegen_llvm::back::lto] running thin lto passes over hello.7rcbfp3g-cgu.0
[INFO  rustc_codegen_llvm::back::lto] running thin lto passes over hello.7rcbfp3g-cgu.1
Segmentation fault

msizanoen1 on 22 Sep 2019

I use whatever lto = true in cargo does :)

jedisct1 on 22 Sep 2019

This has now hit stable.

fenhl on 26 Sep 2019

😕8 👀4

They just don't care about non-x86_64 targets.

jedisct1 on 26 Sep 2019

👎5

They just don't care about non-x86_64 targets.

(Note There is the workaround of using codegen-units=1 (supposedly). Its not ideal by any means, but there's a lot of work to go around and sometimes things fall through the cracks.)

pnkfelix on 3 Oct 2019

(Note that work around _doesn't_ actually always work around the problem)

Lokathor on 3 Oct 2019

I'm getting inconsistent segfaults with codegen_units = 1 and lto = "fat", though if I keep trying compilation eventually succeeds.

Palladinium on 4 Oct 2019

Can reproduce the above on stable rustc 1.38.0 (2019-09-23) on an armv7l (Marvell Armada XP, installed toolchain stable-armv7-unknown-linux-gnueabihf). @fenhl initial workaround still applies. Let me know if you need more info.

monochromec on 5 Oct 2019

@rustbot modify labels: -regression-from-stable-to-beta +regression-from-stable-to-stable

mati865 on 5 Oct 2019

Error: Label regression-from-stable-to-beta can only be set by Rust team members

Please let @rust-lang/release know if you're having trouble with this bot.

rustbot on 5 Oct 2019

fun fact: cross compile seems to be fine!

stefson on 7 Oct 2019

@stefson do you mean cross compiling from armv7 to different arch?
This bug affects only the host compiler.

mati865 on 8 Oct 2019

@alexcrichton Can you push the docker image used to build Linux dist-armv7-linux on Docker Hub so I can try to reproduce this?

msizanoen1 on 9 Oct 2019

@msizanoen1 you can build it on your machine as described in https://github.com/rust-lang/rust/blob/master/src/ci/docker/README.md
To get identical results to the one from CI you need to define environment variable DEPLOY=1.

mati865 on 9 Oct 2019

Can somebody check if updating the GCC version in the CI docker images fixes the issue?

msizanoen1 on 9 Oct 2019

I have successfully reproduced this using the CI docker images. It must be somewhing wrong in the CI images, most likely the C++ cross toolchain.
The CI use GCC 5.2.0 which is released in 2015.
Maybe we should upgrade GCC?

msizanoen1 on 10 Oct 2019

The binaries can be found here: https://github.com/msizanoen1/rust-binaries/releases/tag/rustc-commit-8ee24f6ee04c1dc42ddcd99c57bb0a7a15df1309-segfault

msizanoen1 on 10 Oct 2019

Currently we use older gcc toolchains to lower our dependency on glibc to ensure we run on as many linux distributions as possible. It's sort of unfortunate :(

That being said producing working compilers is far more important than producing segfaulting compilers that run everywhere. In that sense it should be fine to upgrade gcc (configured via this file) and we probably just want to stick as old as we can that still works.

alexcrichton on 10 Oct 2019

I tested using GCC 8.3.0 with glibc 2.17.0 and the toolchain builds successfully.

msizanoen1 on 11 Oct 2019

I enabled debuginfo in LLVM and run valgrind on it. Here is the log: https://github.com/msizanoen1/rust-binaries/releases/download/rustc-commit-8c7b921feb5b100b1825ad54d46a8cca575b35b3-segfault-dbg/valgrind.txt

msizanoen1 on 11 Oct 2019

I have compiled LLVM with gcc 8.3.0 and libc 2.17.0 and I can confirm that this fixes the issue.

msizanoen1 on 11 Oct 2019

🎉3 ❤2

may I ask, which version of llvm are you refering to? Did you bootstrap the llvm-9 fork from the rust-1.38.0 release tar ball, or is it your distro's llvm?

stefson on 11 Oct 2019

@stefson The LLVM fork of rustc at https://github.com/rust-lang/llvm-project. The same as the one used in CI.

msizanoen1 on 11 Oct 2019

Confirmed the PR fixes the issue:

sirius@localhost:~$ rustup-toolchain-install-master e66a6282275802fcb0a29ba58ddc445fc64ac8ef
detecting the channel of the `e66a6282275802fcb0a29ba58ddc445fc64ac8ef` toolchain...
downloading <https://rust-lang-ci2.s3-us-west-1.amazonaws.com/rustc-builds/e66a6282275802fcb0a29ba58ddc445fc64ac8ef/rustc-nightly-armv7-unknown-linux-gnueabihf.tar.xz>...
completed                                         
downloading <https://rust-lang-ci2.s3-us-west-1.amazonaws.com/rustc-builds/e66a6282275802fcb0a29ba58ddc445fc64ac8ef/rust-std-nightly-armv7-unknown-linux-gnueabihf.tar.xz>...
completed
toolchain `e66a6282275802fcb0a29ba58ddc445fc64ac8ef` is successfully installed!
sirius@localhost:~$ rustc +e66a6282275802fcb0a29ba58ddc445fc64ac8ef -C opt-level=3 hello.rs
sirius@localhost:~$ ./hello
Hello, world
sirius@localhost:~$

msizanoen1 on 20 Oct 2019

🎉2 👍1

are the side tools like clippy and rustfmt likely to also build in the next nightly now?

Lokathor on 20 Oct 2019

It is another issue.

msizanoen1 on 20 Oct 2019

Just had this pop up for me on both stable (1.40.0) and nightly (1.42.0 f6449ba23 2020-01-21) with my Raspberry Pi Zero W (arm-unknown-linux-gnueabihf) and was able to reproduce with the steps described. setting codegen-units = 1 fixes the problem

repnop on 24 Jan 2020

Also running into this on Raspberry Pi 4 via Balena.io builds. Can confirm that setting codegen-units = 1 fixes the problem there

thebigredgeek on 26 Feb 2020

I can also still reproduce this on latest stable (1.43.1) with the same SIGSEGV. codegen-units=1 indeed does not trigger the bug.

faern on 24 May 2020

I just reproduced this with

rustc 1.43.1 (8d69840ab 2020-05-04)
cargo 1.43.0 (2cbe9048e 2020-05-03)

haslersn on 1 Jun 2020

@pnkfelix I think this should be either reopened or codegen-units should be set to 1 by default

cztomsik on 22 Aug 2020

I tried to reproduce this with rustc-1.44.1 on armv7-unknown-linux-gnueabihf, and it did ~not~ fail, neither does it with rustc-1.41.1

I used reproducer from the initial posting. If you need people to confirm the breakage, please offer a reproducer

stefson on 22 Aug 2020

Rust: Segfault compiling libc on armv7-unknown-linux-gnueabihf

Steps to reproduce

Workaround

Most helpful comment

All 99 comments

Related issues