rustc(with --enable-debuginfo) hangs for 20-45 seconds on errors(of any kind) when RUST_BACKTRACE is set
EDIT: when env var RUST_BACKTRACE is set to any value (even empty) [it was set to 1 for me here]
EDIT2: I had rust compiled with --enable-debuginfo; when compiled with --disable-debuginfo then it only takes 10 times more time than normal: that is, 0.6sec instead of 0.05 sec
$ time rustc -VV
error: Option 'version' given more than once.
real 0m47.717s
user 0m47.273s
sys 0m0.250s
$ time rustc -Vv
rustc 1.5.0-dev (04e497c00 2015-10-24)
binary: rustc
commit-hash: 04e497c0056aed899cd6edbc98e7a68a9b391c5c
commit-date: 2015-10-24
host: x86_64-unknown-linux-gnu
release: 1.5.0-dev
real 0m0.068s
user 0m0.017s
sys 0m0.030s
I thought the culprit was cargo at first: https://github.com/rust-lang/cargo/issues/2079 but nope.
T-compiler
ghost
All 27 comments
I forgot to mention that this may be related: https://github.com/rust-lang/rust/issues/11493
EDIT: however it hangs on the next line for me, next (empty) line after error: Option 'version' given more than once.
ghost
on 25 Oct 2015
Can't seem to repro this on OSX
:~/src/rust (master=)$ time rustc -VV
error: Option 'version' given more than once.
real 0m0.106s
user 0m0.054s
sys 0m0.055s
:~/src/rust(master=)$ time rustc -V
rustc 1.5.0-nightly (1210fb9bc 2015-10-23)
real 0m0.105s
user 0m0.053s
sys 0m0.053s
achanda
on 25 Oct 2015
Can't reproduce it on Linux.
dylanmckay
on 25 Oct 2015
Thanks for testing guys.
I just notice that while it's hanged, it uses 100% cpu
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
16019 zazdxscf 20 0 371032 87312 47316 S 100.0 0.6 0:09.06 rustc
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
16038 zazdxscf 20 0 395396 116692 52220 S 100.0 0.7 0:42.02 rustc
It works fine like this:
$ time sudo --user=zazdxscf /home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage2/bin/rustc -VV
error: Option 'version' given more than once.
real 0m0.109s
user 0m0.047s
sys 0m0.057s
but not like this:
$ time /home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage2/bin/rustc -VV
error: Option 'version' given more than once.
real 0m44.118s
user 0m43.787s
sys 0m0.313s
Here's a strace:
$ time strace /home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage2/bin/rustc -VV 2>/tmp/strace3.log
real 0m44.451s
user 0m44.073s
sys 0m0.360s
$ cat /tmp/strace3.log
EDIT: moved here to make scrolling here on this page easier.
ghost
on 25 Oct 2015
Here's a crazier strace, at the end we see most of the time is spend in futex stuff
https://gist.githubusercontent.com/zazdxscf/7768f1a4679ad2d60b7e/raw/0309272e682c93507321825e97e8d4cbe43abccd/strace.log
% time seconds usecs/call calls errors syscall
------ ----------- ----------- --------- --------- ----------------
98.60 1.126667 375556 3 futex
0.68 0.007794 14 558 read
0.41 0.004738 0 11064 mmap
0.24 0.002716 17 159 munmap
0.05 0.000552 7 81 13 open
0.02 0.000235 3 68 close
0.00 0.000000 0 8 write
0.00 0.000000 0 10 8 stat
0.00 0.000000 0 33 fstat
0.00 0.000000 0 24 lseek
0.00 0.000000 0 66 mprotect
0.00 0.000000 0 4 brk
0.00 0.000000 0 5 rt_sigaction
0.00 0.000000 0 1 rt_sigprocmask
0.00 0.000000 0 2 ioctl
0.00 0.000000 0 1 1 access
0.00 0.000000 0 25 madvise
0.00 0.000000 0 1 clone
0.00 0.000000 0 1 execve
0.00 0.000000 0 33 fcntl
0.00 0.000000 0 2 1 readlink
0.00 0.000000 0 2 getrlimit
0.00 0.000000 0 2 sigaltstack
0.00 0.000000 0 1 prctl
0.00 0.000000 0 1 arch_prctl
0.00 0.000000 0 2 sched_getaffinity
0.00 0.000000 0 1 set_tid_address
0.00 0.000000 0 2 set_robust_list
0.00 0.000000 0 2 getrandom
------ ----------- ----------- --------- --------- ----------------
100.00 1.142702 12162 23 total
EDIT: for comparison, here's when it works(doesn't hang)
https://gist.github.com/zazdxscf/f30352d3deffce5c1439
% time seconds usecs/call calls errors syscall
------ ----------- ----------- --------- --------- ----------------
77.45 0.026666 8889 3 futex
19.98 0.006879 12 558 read
1.38 0.000475 7 66 mprotect
0.58 0.000201 4 47 12 open
0.38 0.000131 1 99 mmap
0.23 0.000078 2 33 fstat
0.00 0.000000 0 8 write
0.00 0.000000 0 35 close
0.00 0.000000 0 10 8 stat
0.00 0.000000 0 24 lseek
0.00 0.000000 0 12 munmap
0.00 0.000000 0 4 brk
0.00 0.000000 0 5 rt_sigaction
0.00 0.000000 0 1 rt_sigprocmask
0.00 0.000000 0 2 ioctl
0.00 0.000000 0 1 1 access
0.00 0.000000 0 23 madvise
0.00 0.000000 0 1 clone
0.00 0.000000 0 1 execve
0.00 0.000000 0 2 1 readlink
0.00 0.000000 0 2 getrlimit
0.00 0.000000 0 2 sigaltstack
0.00 0.000000 0 1 prctl
0.00 0.000000 0 1 arch_prctl
0.00 0.000000 0 2 sched_getaffinity
0.00 0.000000 0 1 set_tid_address
0.00 0.000000 0 2 set_robust_list
0.00 0.000000 0 2 getrandom
------ ----------- ----------- --------- --------- ----------------
100.00 0.034430 948 22 total
Oh figured it out: it's because RUST_BACKTRACE was set in my environment (set to 1)
$ unset RUST_BACKTRACE
time /home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage2/bin/rustc -VV
error: Option 'version' given more than once.
real 0m0.106s
user 0m0.053s
sys 0m0.050s
even if I set it to 0 or empty value, the issue remains:
$ time RUST_BACKTRACE=0 /home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage2/bin/rustc -VV
error: Option 'version' given more than once.
real 0m44.758s
user 0m44.310s
sys 0m0.390s
$ time RUST_BACKTRACE= /home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage2/bin/rustc -VV
error: Option 'version' given more than once.
real 0m44.767s
user 0m44.460s
sys 0m0.283s
EDIT: This explains why rust tests were taking ages to complete... (half a day at least, but I stopped them)
EDIT2: Also realized that the value likely means nothing, and just the fact that the var is set is considered enabled.
ghost
on 25 Oct 2015
futex taking so long seems to only be indicative of some mutex waiting to get a lock for a long time.
What seems to be troubling is count of mmap calls. Even if their time/call seems to be recorded as 0, there鈥檚 syscall overhead to think about (the overhead is not big enough to cause 40 seconds though).
That being said, I can鈥檛 reproduce this (nor increase in syscall count/duration) either.
nagisa
on 25 Oct 2015
@zazdxscf like others, I unfortunately can't reproduce this (even with RUST_BACKTRACE being set). Lots of mmap syscalls may be indicative of something just going into an infinite loop of allocating, so could you also try to use gdb to capture some stack traces? That may help us pin down where this is coming from.
alexcrichton
on 25 Oct 2015
tl;dr: hang time reduced by 20 times(now at 2.6 sec instead of 40 sec) after recompiling rust(with newer latest master and unoptimized llvm) and someone please tell me how to make gdb show symbols-ridden stacktrace. Thanks.
I'm having trouble figuring out how get a stacktrace (with symbols). I need help (maybe someone could point me to an article on how to debug rustc with gdb, but not how to debug rust programs with gdb).
$ PATH="$PATH:/home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage0/bin/" RUST_BACKTRACE=1 rust-gdb -- rustc
GNU gdb (Gentoo 7.10 vanilla) 7.10
Copyright (C) 2015 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-pc-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://bugs.gentoo.org/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from rustc...done.
(gdb) run -VV
Starting program: /home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage2/bin/rustc -VV
warning: Cannot call inferior functions, Linux kernel PaX protection forbids return to non-executable pages!
[New LWP 7481]
error: Option 'version' given more than once.
^C
Program received signal SIGINT, Interrupt.
0x0000033afe21e9ad in ?? ()
(gdb) bt full
#0 0x0000033afe21e9ad in ?? ()
No symbol table info available.
#1 0x000000000000000e in ?? ()
No symbol table info available.
#2 0x00000000fbc24120 in ?? ()
No symbol table info available.
#3 0x0000033afe21e8d0 in ?? ()
No symbol table info available.
#4 0x0000033afbb6bd28 in ?? ()
No symbol table info available.
#5 0x0000033afbc00000 in ?? ()
No symbol table info available.
#6 0x0000000000000000 in ?? ()
No symbol table info available.
Hmm now that I look at this gdb from 2 days ago, I notice some have symbols and some don't https://github.com/rust-lang/rust/issues/29270#issue-113143959
Is it because the source is not c/c++ ?
On another note: I noticed that I cannot reproduce it with the snapshot rust
$ time RUST_BACKTRACE=1 /home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage0/bin/rustc -VV
error: Option 'version' given more than once.
real 0m0.401s
user 0m0.360s
sys 0m0.020s
$ time RUST_BACKTRACE=1 /home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage0/bin/rustc -Vv
rustc 1.4.0-dev (1af31d497 2015-08-11)
binary: rustc
commit-hash: 1af31d4974e33027a68126fa5a5a3c2c6491824f
commit-date: 2015-08-11
host: x86_64-unknown-linux-gnu
release: 1.4.0-dev
real 0m0.018s
user 0m0.000s
sys 0m0.010s
Although there's 0.4 sec vs 0.02 sec ! I can feel that hang a litle, but let's say it doesn't at all.
Now with non-optimized debug-enabled llvm (this --enable-debug-symbols --disable-optimized --enable-debug-runtime to llvm, and this --disable-optimize-llvm to rust's ./configure; instead of this --enable-optimized to llvm(which implies no debug symbols) because of this --enable-optimize-llvm to rust's ./configure), the time spent is 10-20 times less:
(I swear I had output that was consistently showing 4.3 seconds(like 10 times less), but now consistently 2.6 sec (like 20 times less, than I previously reported))
stage0 rust (not snapshot stage0)
$ time RUST_BACKTRACE= /home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage0/lib/rustlib/x86_64-unknown-linux-gnu/bin/rustc -VV
error: Option 'version' given more than once.
real 0m2.671s
user 0m2.407s
sys 0m0.070s
$ time RUST_BACKTRACE= /home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage0/lib/rustlib/x86_64-unknown-linux-gnu/bin/rustc -Vv
rustc 1.5.0-dev (2a418216f 2015-10-25)
binary: rustc
commit-hash: 2a418216feeb83fd3f68c725c0e5577beacff59b
commit-date: 2015-10-25
host: x86_64-unknown-linux-gnu
release: 1.5.0-dev
real 0m0.123s
user 0m0.080s
sys 0m0.030s
stage1 rust
$ time RUST_BACKTRACE= /home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage1/bin/rustc -VV
error: Option 'version' given more than once.
real 0m2.654s
user 0m2.410s
sys 0m0.090s
$ time RUST_BACKTRACE= /home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage1/bin/rustc -Vv
rustc 1.5.0-dev (2a418216f 2015-10-25)
binary: rustc
commit-hash: 2a418216feeb83fd3f68c725c0e5577beacff59b
commit-date: 2015-10-25
host: x86_64-unknown-linux-gnu
release: 1.5.0-dev
real 0m0.150s
user 0m0.070s
sys 0m0.037s
stage2 is still compiling... Good god! :)) (but I don't want go, I want rust) EDIT: my bad for having set cpu governor to conservative!! (it was stuck at 800Mhz even thought rustc was using 100% cpu, aka 1 thread, instead of 4) switched back to ondemand. cpupower frequency-set -g ondemand (which was is the default). The thing is, it was actually pretty fast until after stage1 got compiled; but then, it's been some serious hours since! (likely because of that 800Mhz)
EDIT2: here's stage2 (which takes normal time, compared to stage1)
$ time RUST_BACKTRACE= /home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage2/bin/rustc -VV
error: Option 'version' given more than once.
real 0m46.847s
user 0m45.843s
sys 0m0.313s
$ time RUST_BACKTRACE= /home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage2/bin/rustc -Vv
rustc 1.5.0-dev (2a418216f 2015-10-25)
binary: rustc
commit-hash: 2a418216feeb83fd3f68c725c0e5577beacff59b
commit-date: 2015-10-25
host: x86_64-unknown-linux-gnu
release: 1.5.0-dev
real 0m0.181s
user 0m0.080s
sys 0m0.030s
$ time /home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage2/bin/rustc -VV
error: Option 'version' given more than once.
real 0m0.116s
user 0m0.083s
sys 0m0.033s
(I did rerun stage1 after this, and it's still 2.5 sec)
$ RUST_BACKTRACE= gdb -- /home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage1/bin/rustc
GNU gdb (Gentoo 7.10 vanilla) 7.10
Copyright (C) 2015 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-pc-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://bugs.gentoo.org/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage1/bin/rustc...done.
(gdb) run -VV
Starting program: /home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage1/bin/rustc -VV
warning: Cannot call inferior functions, Linux kernel PaX protection forbids return to non-executable pages!
[New LWP 31140]
error: Option 'version' given more than once.
^C
Program received signal SIGINT, Interrupt.
0x0000035c616709ad in ?? ()
(gdb) run -VVQuit
(gdb) thread apply all bt full
Thread 2 (LWP 31140):
#0 0x0000035c694cc986 in ?? ()
No symbol table info available.
#1 0x0000035c5fb20ca0 in ?? ()
No symbol table info available.
#2 0x0000035c694cc461 in ?? ()
No symbol table info available.
#3 0x0000035c694cc95e in ?? ()
No symbol table info available.
#4 0x0000000000000018 in ?? ()
No symbol table info available.
#5 0x0000000000000685 in ?? ()
No symbol table info available.
#6 0x0000035c5ce548e8 in ?? ()
No symbol table info available.
#7 0x0000035c5ce548e8 in ?? ()
No symbol table info available.
#8 0x000000000000011b in ?? ()
No symbol table info available.
#9 0x000000000000031f in ?? ()
No symbol table info available.
#10 0x0000035c5ce548e8 in ?? ()
---Type <return> to continue, or q <return> to quit---
No symbol table info available.
#11 0x0000035c5fb20cf0 in ?? ()
No symbol table info available.
#12 0x0000035c694cc56a in ?? ()
No symbol table info available.
#13 0x0000035c694cc95e in ?? ()
No symbol table info available.
#14 0x0000000000000018 in ?? ()
No symbol table info available.
#15 0x00000000000016ee in ?? ()
No symbol table info available.
#16 0x0000035c5ce3c708 in ?? ()
No symbol table info available.
#17 0x0000035c5ce64830 in ?? ()
No symbol table info available.
#18 0x0000000000001013 in ?? ()
No symbol table info available.
#19 0x00000000000016ee in ?? ()
No symbol table info available.
#20 0x0000035c5ce3c708 in ?? ()
No symbol table info available.
#21 0x0000035c5fb20d40 in ?? ()
No symbol table info available.
---Type <return> to continue, or q <return> to quit---
#22 0x0000035c694cc50d in ?? ()
No symbol table info available.
#23 0x0000035c694cc95e in ?? ()
No symbol table info available.
#24 0x0000000000000018 in ?? ()
No symbol table info available.
#25 0x0000000000004c32 in ?? ()
No symbol table info available.
#26 0x0000035c5cdb7e80 in ?? ()
No symbol table info available.
#27 0x0000035c5cdb7e80 in ?? ()
No symbol table info available.
#28 0x0000000000001ab6 in ?? ()
No symbol table info available.
#29 0x0000000000004c32 in ?? ()
No symbol table info available.
#30 0x0000035c5ce3c708 in ?? ()
No symbol table info available.
#31 0x0000035c5fb20d90 in ?? ()
No symbol table info available.
#32 0x0000035c694cc56a in ?? ()
No symbol table info available.
#33 0x0000035c694cc95e in ?? ()
---Type <return> to continue, or q <return> to quit---
No symbol table info available.
#34 0x0000000000000018 in ?? ()
No symbol table info available.
#35 0x000000000001d455 in ?? ()
No symbol table info available.
#36 0x0000035c5c96b000 in ?? ()
No symbol table info available.
#37 0x0000000000000000 in ?? ()
No symbol table info available.
Thread 1 (LWP 31020):
#0 0x0000035c616709ad in ?? ()
No symbol table info available.
#1 0x00000000697213a8 in ?? ()
No symbol table info available.
#2 0x0000000000000010 in ?? ()
No symbol table info available.
#3 0x0000035c616708d0 in ?? ()
No symbol table info available.
#4 0x0000035c5fb26d28 in ?? ()
No symbol table info available.
#5 0x0000000000000000 in ?? ()
No symbol table info available.
(gdb) quit
A debugging session is active.
Inferior 1 [process 31020] will be killed.
Quit anyway? (y or n) y
And dmesg shows:
[282001.448603] grsec: exec of /bin/bash (/bin/bash -c exec /home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage1/bin/rustc -VV ) by /bin/bash[gdb:31020] uid/euid:1000/1000 gid/egid:1000/1000, parent /usr/bin/gdb[gdb:30613] uid/euid:1000/1000 gid/egid:1000/1000
[282001.454206] grsec: exec of /home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage1/bin/rustc (/home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage1/bin/rustc -VV ) by /home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage1/bin/rustc[bash:31020] uid/euid:1000/1000 gid/egid:1000/1000, parent /usr/bin/gdb[gdb:30613] uid/euid:1000/1000 gid/egid:1000/1000
[282001.472263] PAX: execution attempt in: <anonymous mapping>, 351e56b2000-351e56b3000 351e56b2000
[282001.472282] PAX: terminating task: /usr/bin/gdb(gdb):31031, uid/euid: 1000/1000, PC: 00000351e56b2000, SP: 0000038f2575db10
[282001.472294] PAX: bytes at PC: cc 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
[282001.472322] PAX: bytes at SP-8: 00000351e56b2000 0000038f2575db50 62869905ee6e2300 0000000000000000 0000006fd7b21760 0000006fd74918a0 000000000000792c 0000006fd8e80270 0000006fd8dce0d0 0000038f2575dbe0 0000006fd72f4409
(NOTE however that there are no errors of any kind on dmesg while executing rust -VV normally, only through gdb)
The above pax terminating task may be explained by this: https://forums.gentoo.org/viewtopic-p-7706666.html#7706666
In other words, it's just a test by gdb to detect stuff, after which I presume it displays that warning.
Perhaps gdb needs this in order to show function names and such? in the stacktrace. I don't see a way to disable this temporarily(tried some paxctl-ng flags on gdb with no effect)
Hmm that must be it, I mean, the No symbol table info available. may always be there regardless, but the ?? () should show the right things, if I manage to relax PaX so that gdb doesn't output that warning. Right? It seems to make sense.
Oh wait, what did I just find? https://wiki.gentoo.org/wiki/Hardened/Debugging
Ok this is stupid:)
/home/zazdxscf $ sudo paxctl -v /home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage1/bin/rustc
Password:
PaX control v0.9
Copyright 2004,2005,2006,2007,2009,2010,2011,2012,2014 PaX Team <[email protected]>
- PaX flags: -------x-e-- [/home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage1/bin/rustc]
RANDEXEC is disabled
EMUTRAMP is disabled
-----------
zazdxscf@gobaby 2015/10/26 01:12:24 -bash4.3.42 t:14 j:0 d:5 pp:9467 p:30555
!12306 8 0 4.2.3-hardened-r4-g45b4b78 #2 SMP Tue Oct 20 14:39:20 CEST 2015
/home/zazdxscf
/home/zazdxscf $ sudo paxctl -vr /home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage1/bin/rustc
PaX control v0.9
Copyright 2004,2005,2006,2007,2009,2010,2011,2012,2014 PaX Team <[email protected]>
/home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage1/bin/rustc: Text file busy
-----------
zazdxscf@gobaby 2015/10/26 01:12:46 -bash4.3.42 t:14 j:0 d:5 pp:9467 p:30555
!12307 9 1 4.2.3-hardened-r4-g45b4b78 #2 SMP Tue Oct 20 14:39:20 CEST 2015
/home/zazdxscf
/home/zazdxscf $ sudo paxctl -v /home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage1/bin/rustc
PaX control v0.9
Copyright 2004,2005,2006,2007,2009,2010,2011,2012,2014 PaX Team <[email protected]>
- PaX flags: -------x-e-- [/home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage1/bin/rustc]
RANDEXEC is disabled
EMUTRAMP is disabled
-----------
zazdxscf@gobaby 2015/10/26 01:12:53 -bash4.3.42 t:14 j:0 d:5 pp:9467 p:30555
!12308 10 0 4.2.3-hardened-r4-g45b4b78 #2 SMP Tue Oct 20 14:39:20 CEST 2015
/home/zazdxscf
/home/zazdxscf $ sudo paxctl -vr /home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage1/bin/rustc
PaX control v0.9
Copyright 2004,2005,2006,2007,2009,2010,2011,2012,2014 PaX Team <[email protected]>
/home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage1/bin/rustc: Text file busy
-----------
zazdxscf@gobaby 2015/10/26 01:12:56 -bash4.3.42 t:14 j:0 d:5 pp:9467 p:30555
!12309 11 1 4.2.3-hardened-r4-g45b4b78 #2 SMP Tue Oct 20 14:39:20 CEST 2015
/home/zazdxscf
/home/zazdxscf $ sudo paxctl -v /home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage1/bin/rustc
PaX control v0.9
Copyright 2004,2005,2006,2007,2009,2010,2011,2012,2014 PaX Team <[email protected]>
- PaX flags: -------x-e-- [/home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage1/bin/rustc]
RANDEXEC is disabled
EMUTRAMP is disabled
-----------
zazdxscf@gobaby 2015/10/26 01:13:11 -bash4.3.42 t:14 j:0 d:5 pp:9467 p:30555
!12310 12 0 4.2.3-hardened-r4-g45b4b78 #2 SMP Tue Oct 20 14:39:20 CEST 2015
/home/zazdxscf
/home/zazdxscf $ sudo paxctl-ng -v /home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage1/bin/rustc
/home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage1/bin/rustc:
open(O_RDWR) failed: cannot change PT_PAX flags
PT_PAX : -e---
XATTR_PAX : not found
-----------
zazdxscf@gobaby 2015/10/26 01:13:21 -bash4.3.42 t:14 j:0 d:5 pp:9467 p:30555
!12311 13 0 4.2.3-hardened-r4-g45b4b78 #2 SMP Tue Oct 20 14:39:20 CEST 2015
/home/zazdxscf
/home/zazdxscf $ sudo paxctl-ng -v -l -r /home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage1/bin/rustc
/home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage1/bin/rustc:
open(O_RDWR) failed: cannot change PT_PAX flags
PT_PAX : -e---
XATTR_PAX : -e-r-
-----------
zazdxscf@gobaby 2015/10/26 01:13:55 -bash4.3.42 t:14 j:0 d:5 pp:9467 p:30555
!12312 14 0 4.2.3-hardened-r4-g45b4b78 #2 SMP Tue Oct 20 14:39:20 CEST 2015
/home/zazdxscf
/home/zazdxscf $ sudo paxctl-ng -v -rl /home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage1/bin/rustc
/home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage1/bin/rustc:
open(O_RDWR) failed: cannot change PT_PAX flags
PT_PAX : -e---
XATTR_PAX : -e-r-
-----------
zazdxscf@gobaby 2015/10/26 01:14:23 -bash4.3.42 t:14 j:0 d:5 pp:9467 p:30555
!12313 15 0 4.2.3-hardened-r4-g45b4b78 #2 SMP Tue Oct 20 14:39:20 CEST 2015
/home/zazdxscf
/home/zazdxscf $ sudo paxctl-ng -vrl /home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage1/bin/rustc
/home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage1/bin/rustc:
open(O_RDWR) failed: cannot change PT_PAX flags
PT_PAX : -e---
XATTR_PAX : -e-r-
-----------
zazdxscf@gobaby 2015/10/26 01:14:27 -bash4.3.42 t:14 j:0 d:5 pp:9467 p:30555
!12314 16 0 4.2.3-hardened-r4-g45b4b78 #2 SMP Tue Oct 20 14:39:20 CEST 2015
/home/zazdxscf
/home/zazdxscf $ sudo paxctl-ng -vrl /home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage1/bin/rustc
/home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage1/bin/rustc:
open(O_RDWR) failed: cannot change PT_PAX flags
PT_PAX : -e---
XATTR_PAX : -e-r-
-----------
zazdxscf@gobaby 2015/10/26 01:14:41 -bash4.3.42 t:14 j:0 d:5 pp:9467 p:30555
!12315 17 0 4.2.3-hardened-r4-g45b4b78 #2 SMP Tue Oct 20 14:39:20 CEST 2015
/home/zazdxscf
/home/zazdxscf $ time RUST_BACKTRACE= gdb --args /home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage1/bin/rustc -VV
GNU gdb (Gentoo 7.10 vanilla) 7.10
Copyright (C) 2015 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-pc-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://bugs.gentoo.org/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage1/bin/rustc...done.
(gdb) r
Starting program: /home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage1/bin/rustc -VV
During startup program terminated with signal SIGSEGV, Segmentation fault.
(gdb) quit
real 0m6.921s
user 0m0.090s
sys 0m0.077s
-----------
zazdxscf@gobaby 2015/10/26 01:14:58 -bash4.3.42 t:14 j:0 d:5 pp:9467 p:30555
!12316 18 0 4.2.3-hardened-r4-g45b4b78 #2 SMP Tue Oct 20 14:39:20 CEST 2015
/home/zazdxscf
/home/zazdxscf $ sudo paxctl-ng -vRl /home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage1/bin/rustc
/home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage1/bin/rustc:
open(O_RDWR) failed: cannot change PT_PAX flags
PT_PAX : -e---
XATTR_PAX : -e-R-
-----------
zazdxscf@gobaby 2015/10/26 01:15:03 -bash4.3.42 t:14 j:0 d:5 pp:9467 p:30555
!12317 19 0 4.2.3-hardened-r4-g45b4b78 #2 SMP Tue Oct 20 14:39:20 CEST 2015
/home/zazdxscf
/home/zazdxscf $ time RUST_BACKTRACE= gdb --args /home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage1/bin/rustc -VV
GNU gdb (Gentoo 7.10 vanilla) 7.10
Copyright (C) 2015 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-pc-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://bugs.gentoo.org/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage1/bin/rustc...done.
(gdb) r
Starting program: /home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage1/bin/rustc -VV
warning: Cannot call inferior functions, Linux kernel PaX protection forbids return to non-executable pages!
[New LWP 20518]
error: Option 'version' given more than once.
^C
Program received signal SIGINT, Interrupt.
0x000003bd5507b9ad in ?? ()
(gdb) bt full
#0 0x000003bd5507b9ad in ?? ()
No symbol table info available.
#1 0x000000005d12c3a8 in ?? ()
No symbol table info available.
#2 0x0000000000000010 in ?? ()
No symbol table info available.
#3 0x000003bd5507b8d0 in ?? ()
No symbol table info available.
#4 0x000003bd53512d28 in ?? ()
No symbol table info available.
#5 0x0000000000000000 in ?? ()
No symbol table info available.
(gdb) quit
A debugging session is active.
Inferior 1 [process 20514] will be killed.
Quit anyway? (y or n) EOF [assumed Y]
real 0m10.945s
user 0m1.257s
sys 0m0.860s
-----------
zazdxscf@gobaby 2015/10/26 01:15:16 -bash4.3.42 t:14 j:0 d:5 pp:9467 p:30555
!12318 20 0 4.2.3-hardened-r4-g45b4b78 #2 SMP Tue Oct 20 14:39:20 CEST 2015
/home/zazdxscf
/home/zazdxscf $ sudo paxctl-ng -vrsmpl /home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage1/bin/rustc
/home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage1/bin/rustc:
open(O_RDWR) failed: cannot change PT_PAX flags
PT_PAX : -e---
XATTR_PAX : pemrs
-----------
zazdxscf@gobaby 2015/10/26 01:15:38 -bash4.3.42 t:14 j:0 d:5 pp:9467 p:30555
!12319 21 0 4.2.3-hardened-r4-g45b4b78 #2 SMP Tue Oct 20 14:39:20 CEST 2015
/home/zazdxscf
/home/zazdxscf $ time RUST_BACKTRACE= gdb --args /home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage1/bin/rustc -VV
GNU gdb (Gentoo 7.10 vanilla) 7.10
Copyright (C) 2015 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-pc-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://bugs.gentoo.org/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage1/bin/rustc...done.
(gdb) r
Starting program: /home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage1/bin/rustc -VV
During startup program terminated with signal SIGSEGV, Segmentation fault.
(gdb) quit
real 0m3.143s
user 0m0.093s
sys 0m0.070s
-----------
zazdxscf@gobaby 2015/10/26 01:15:44 -bash4.3.42 t:14 j:0 d:5 pp:9467 p:30555
!12320 22 0 4.2.3-hardened-r4-g45b4b78 #2 SMP Tue Oct 20 14:39:20 CEST 2015
/home/zazdxscf
/home/zazdxscf $ sudo paxctl-ng -vRsmpl /home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage1/bin/rustc
/home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage1/bin/rustc:
open(O_RDWR) failed: cannot change PT_PAX flags
PT_PAX : -e---
XATTR_PAX : pemRs
-----------
zazdxscf@gobaby 2015/10/26 01:15:50 -bash4.3.42 t:14 j:0 d:5 pp:9467 p:30555
!12321 23 0 4.2.3-hardened-r4-g45b4b78 #2 SMP Tue Oct 20 14:39:20 CEST 2015
/home/zazdxscf
/home/zazdxscf $ time RUST_BACKTRACE= gdb --args /home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage1/bin/rustc -VV
GNU gdb (Gentoo 7.10 vanilla) 7.10
Copyright (C) 2015 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-pc-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://bugs.gentoo.org/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage1/bin/rustc...done.
(gdb) r
Starting program: /home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage1/bin/rustc -VV
During startup program terminated with signal SIGSEGV, Segmentation fault.
(gdb) quit
real 0m5.499s
user 0m0.107s
sys 0m0.053s
-----------
zazdxscf@gobaby 2015/10/26 01:15:58 -bash4.3.42 t:14 j:0 d:5 pp:9467 p:30555
!12322 24 0 4.2.3-hardened-r4-g45b4b78 #2 SMP Tue Oct 20 14:39:20 CEST 2015
/home/zazdxscf
/home/zazdxscf $ sudo paxctl-ng -vzl /home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage1/bin/rustc
/home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage1/bin/rustc:
open(O_RDWR) failed: cannot change PT_PAX flags
PT_PAX : -e---
XATTR_PAX : -----
-----------
zazdxscf@gobaby 2015/10/26 01:16:10 -bash4.3.42 t:14 j:0 d:5 pp:9467 p:30555
!12323 25 0 4.2.3-hardened-r4-g45b4b78 #2 SMP Tue Oct 20 14:39:20 CEST 2015
/home/zazdxscf
/home/zazdxscf $ sudo paxctl-ng -vzle /home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage1/bin/rustc
Package Name : elfix 0.9.1
Bug Reports : http://bugs.gentoo.org/
Program Name : paxctl-ng
Description : Get or set pax flags on an ELF object
Usage : paxctl-ng -PpEeMmRrSs|-Z|-z [-L|-l] [-v] ELF
: paxctl-ng -C|-c|-d [-v] ELF
: paxctl-ng -F|-f [-v] ELF
: paxctl-ng -v ELF
: paxctl-ng -L|-l
: paxctl-ng [-h]
Options : -P enable PAGEEXEC -p disable PAGEEXEC
: -E enable EMUTRAMP -e disable EMUTRAMP
: -M enable MPROTECT -m disable MPROTECT
: -R enable RANDMMAP -r disable RANDMMAP
: -S enable SEGMEXEC -s disable SEGMEXEC
: -Z all secure settings -z all default settings
: -L set only PT_PAX flags -l set only XATTR_PAX flags
:
: -C create XATTR_PAX with most secure setting
: -c create XATTR_PAX all default settings
: -d delete XATTR_PAX field
: -F copy PT_PAX to XATTR_PAX
: -f copy XATTR_PAX to PT_PAX
: -L when given alone, EXIT_SUCCESS (PT_PAX is supported)
: -l when given alone, EXIT_SUCCESS (XATTR_PAX is supported)
: -v view the flags, along with any accompanying operation
: -h print out this help
Note : If both enabling and disabling flags are set, the default - is used
-----------
zazdxscf@gobaby 2015/10/26 01:16:14 -bash4.3.42 t:14 j:0 d:5 pp:9467 p:30555
!12324 26 0 4.2.3-hardened-r4-g45b4b78 #2 SMP Tue Oct 20 14:39:20 CEST 2015
/home/zazdxscf
/home/zazdxscf $ sudo paxctl-ng -vel /home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage1/bin/rustc
/home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage1/bin/rustc:
open(O_RDWR) failed: cannot change PT_PAX flags
PT_PAX : -e---
XATTR_PAX : -e---
-----------
zazdxscf@gobaby 2015/10/26 01:16:19 -bash4.3.42 t:14 j:0 d:5 pp:9467 p:30555
!12325 27 0 4.2.3-hardened-r4-g45b4b78 #2 SMP Tue Oct 20 14:39:20 CEST 2015
/home/zazdxscf
/home/zazdxscf $ sudo paxctl-ng -vc /home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage1/bin/rustc
/home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage1/bin/rustc:
open(O_RDWR) failed: cannot change PT_PAX flags
PT_PAX : -e---
XATTR_PAX : -e---
-----------
zazdxscf@gobaby 2015/10/26 01:16:26 -bash4.3.42 t:14 j:0 d:5 pp:9467 p:30555
!12326 28 1 4.2.3-hardened-r4-g45b4b78 #2 SMP Tue Oct 20 14:39:20 CEST 2015
/home/zazdxscf
/home/zazdxscf $ sudo paxctl-ng -vd /home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage1/bin/rustc
/home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage1/bin/rustc:
open(O_RDWR) failed: cannot change PT_PAX flags
PT_PAX : -e---
XATTR_PAX : not found
-----------
zazdxscf@gobaby 2015/10/26 01:16:28 -bash4.3.42 t:14 j:0 d:5 pp:9467 p:30555
!12327 29 0 4.2.3-hardened-r4-g45b4b78 #2 SMP Tue Oct 20 14:39:20 CEST 2015
/home/zazdxscf
/home/zazdxscf $
dmesg:
[284431.059807] grsec: exec of /usr/bin/gdb (gdb --args /home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage1/bin/rustc -VV ) by /usr/bin/gdb[bash:20489] uid/euid:1000/1000 gid/egid:1000/1000, parent /bin/bash[bash:30555] uid/euid:1000/1000 gid/egid:1000/1000
[284431.081685] grsec: exec of /usr/bin/iconv (iconv -l ) by /usr/bin/iconv[gdb:20490] uid/euid:1000/1000 gid/egid:1000/1000, parent /usr/bin/gdb[gdb:20489] uid/euid:1000/1000 gid/egid:1000/1000
[284431.901487] grsec: exec of /bin/bash (/bin/bash -c exec /home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage1/bin/rustc -VV ) by /bin/bash[gdb:20491] uid/euid:1000/1000 gid/egid:1000/1000, parent /usr/bin/gdb[gdb:20489] uid/euid:1000/1000 gid/egid:1000/1000
[284431.909694] grsec: exec of /home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage1/bin/rustc (/home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage1/bin/rustc -VV ) by /home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage1/bin/rustc[bash:20491] uid/euid:1000/1000 gid/egid:1000/1000, parent /usr/bin/gdb[gdb:20489] uid/euid:1000/1000 gid/egid:1000/1000
[284431.911916] grsec: Segmentation fault occurred at (nil) in /home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage1/bin/rustc[bash:20491] uid/euid:1000/1000 gid/egid:1000/1000, parent /usr/bin/gdb[gdb:20489] uid/euid:1000/1000 gid/egid:1000/1000
...
[284447.041691] grsec: exec of /home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage1/bin/rustc (/home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage1/bin/rustc -VV ) by /home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage1/bin/rustc[bash:20514] uid/euid:1000/1000 gid/egid:1000/1000, parent /usr/bin/gdb[gdb:20512] uid/euid:1000/1000 gid/egid:1000/1000
[284447.065900] PAX: execution attempt in: <anonymous mapping>, 383691f5000-383691f6000 383691f5000
[284447.065917] PAX: terminating task: /usr/bin/gdb(gdb):20517, uid/euid: 1000/1000, PC: 00000383691f5000, SP: 000003dc07db2ce0
[284447.065927] PAX: bytes at PC: cc 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
[284447.065955] PAX: bytes at SP-8: 00000383691f5000 000003dc07db2d20 bbee2a0d7dc77d00 0000000000000000 0000007af2533760 0000007af1ea38a0 0000000000005022 0000007af5893a80 0000007af57d31e0 000003dc07db2db0 0000007af1d06409
...
[284483.000550] grsec: Segmentation fault occurred at (nil) in /home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage1/bin/rustc[bash:20543] uid/euid:1000/1000 gid/egid:1000/1000, parent /usr/bin/gdb[gdb:20541] uid/euid:1000/1000 gid/egid:1000/1000
...
[284496.186188] grsec: Segmentation fault occurred at (nil) in /home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage1/bin/rustc[bash:20566] uid/euid:1000/1000 gid/egid:1000/1000, parent /usr/bin/gdb[gdb:20564] uid/euid:1000/1000 gid/egid:1000/1000
...
I forced it to dump core (by sending it signal 11, twice! apparently once had no effect!)
$ time RUST_BACKTRACE= /home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage2/bin/rustc -VV
error: Option 'version' given more than once.
Segmentation fault (core dumped)
real 0m15.874s
user 0m13.973s
sys 0m1.537s
$ time LD_LIBRARY_PATH=/home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage2/lib coredump_to_backtrace /home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage2/bin/rustc ./core all >~/logs/rustc2_crash_backtrace_full.log 2>&1
real 0m26.339s
user 0m24.480s
sys 0m1.377s
EDIT: Skip to rust-gdb output: https://github.com/rust-lang/rust/issues/29293#issuecomment-151046394
The following is with plain gdb (as seen with the above command):
2 -ex thread apply all bt full
3 -ex quit --batch
[New LWP 16104]
[New LWP 16105]
warning: Could not load shared library symbols for linux-vdso.so.1.
Do you need "set solib-search-path" or "set sysroot"?
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
Core was generated by `/home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage2/bin/rus'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0 0x000003761d0289ad in pthread_join (threadid=3805789603584, thread_return=0x0) at pthread_join.c:90
90 lll_wait_tid (pd->tid);
[Current thread is 1 (Thread 0x37625654a00 (LWP 16104))]
warning: Missing auto-load script at offset 0 in section .debug_gdb_scripts
of file /home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage2/bin/rustc.
Use `info auto-load python-scripts [REGEXP]' to list them.
Thread 2 (Thread 0x3761abcc700 (LWP 16105)):
#0 0x0000037624c3724b in backtrace_alloc (state=0x3762567a000, size=384, error_callback=0x37624c25e60 <sys_common::gnu::libbacktrace::print::error_cb::h2189b2356f8e6008het>, data=0x3761abc75c0) at ../../../src/libbacktrace/mmap.c:110
ret = 0x0
locked = 1
pp = 0x375fc349da8
pagesize = 3805957782718
asksize = 3805789579536
page = 0x3761abc70d0
#1 0x0000037624c37558 in backtrace_vector_grow (state=0x3762567a000, size=24, error_callback=0x37624c25e60 <sys_common::gnu::libbacktrace::print::error_cb::h2189b2356f8e6008het>, data=0x3761abc75c0, vec=0x3761abc6bd0) at ../../../src/libbacktrace/mmap.c:236
pagesize = 4096
alc = 384
base = 0x3761abc6a20
ret = 0x8
#2 0x0000037624c3386c in add_function_range (state=0x3762567a000, ddata=0x37618082970, function=0x375fad7e230, lowpc=3805919552193, highpc=3805919552197, error_callback=0x37624c25e60 <sys_common::gnu::libbacktrace::print::error_cb::h2189b2356f8e6008het>, data=0x3761abc75c0, vec=0x3761abc6bd0) at ../../../src/libbacktrace/dwarf.c:2175
p = 0x37624c30173 <read_address+126>
#3 0x0000037624c33a26 in add_function_ranges (state=0x3762567a000, ddata=0x37618082970, u=0x3761a110998, function=0x375fad7e230, ranges=64, base=0, error_callback=0x37624c25e60 <sys_common::gnu::libbacktrace::print::error_cb::h2189b2356f8e6008het>, data=0x3761abc75c0, vec=0x3761abc6bd0) at ../../../src/libbacktrace/dwarf.c:2233
low = 553665
high = 553669
ranges_buf = {name = 0x37624cc5999 ".debug_ranges", start = 0x3761288e030 <error: Cannot access memory at address 0x3761288e030>, buf = 0x3761288e080 <error: Cannot access memory at address 0x3761288e080>, left = 5360352, is_bigendian = 0, error_callback = 0x37624c25e60 <sys_common::gnu::libbacktrace::print::error_cb::h2189b2356f8e6008het>, data = 0x3761abc75c0, reported_underflow = 0}
#4 0x0000037624c33fd0 in read_function_entry (state=0x3762567a000, ddata=0x37618082970, u=0x3761a110998, base=0, unit_buf=0x3761abc70d0, lhdr=0x3761abc71b0, error_callback=0x37624c25e60 <sys_common::gnu::libbacktrace::print::error_cb::h2189b2356f8e6008het>, data=0x3761abc75c0, vec_function=0x376180829d8, vec_inlined=0x3761abc6bd0) at ../../../src/libbacktrace/dwarf.c:2443
is_function = 1
vec = 0x3761abc6bd0
lowpc = 0
highpc = 0
highpc_is_relative = 0
have_ranges = 1
function = 0x375fad7e230
have_lowpc = 0
ranges = 64
code = 10
abbrev = 0x3761a125de8
i = 4
have_highpc = 0
#5 0x0000037624c34136 in read_function_entry (state=0x3762567a000, ddata=0x37618082970, u=0x3761a110998, base=0, unit_buf=0x3761abc70d0, lhdr=0x3761abc71b0, error_callback=0x37624c25e60 <sys_common::gnu::libbacktrace::print::error_cb::h2189b2356f8e6008het>, data=0x3761abc75c0, vec_function=0x376180829d8, vec_inlined=0x376180829d8) at ../../../src/libbacktrace/dwarf.c:2481
fvec = {vec = {base = 0x0, size = 0, alc = 0}, count = 0}
is_function = 1
vec = 0x376180829d8
lowpc = 553600
highpc = 553957
highpc_is_relative = 1
have_ranges = 0
function = 0x375fad7e208
have_lowpc = 1
ranges = 0
code = 46
abbrev = 0x3761a126268
i = 9
have_highpc = 1
#6 0x0000037624c340c4 in read_function_entry (state=0x3762567a000, ddata=0x37618082970, u=0x3761a110998, base=0, unit_buf=0x3761abc70d0, lhdr=0x3761abc71b0, error_callback=0x37624c25e60 <sys_common::gnu::libbacktrace::print::error_cb::h2189b2356f8e6008het>, data=0x3761abc75c0, vec_function=0x376180829d8, vec_inlined=0x376180829d8) at ../../../src/libbacktrace/dwarf.c:2467
is_function = 0
vec = 0x376180829d8
lowpc = 0
highpc = 0
highpc_is_relative = 0
have_ranges = 0
function = 0x0
have_lowpc = 0
ranges = 0
code = 2
abbrev = 0x3761a125ce8
i = 1
have_highpc = 0
#7 0x0000037624c340c4 in read_function_entry (state=0x3762567a000, ddata=0x37618082970, u=0x3761a110998, base=0, unit_buf=0x3761abc70d0, lhdr=0x3761abc71b0, error_callback=0x37624c25e60 <sys_common::gnu::libbacktrace::print::error_cb::h2189b2356f8e6008het>, data=0x3761abc75c0, vec_function=0x376180829d8, vec_inlined=0x376180829d8) at ../../../src/libbacktrace/dwarf.c:2467
is_function = 0
vec = 0x376180829d8
lowpc = 0
highpc = 0
highpc_is_relative = 0
have_ranges = 0
function = 0x0
have_lowpc = 0
ranges = 0
code = 2
abbrev = 0x3761a125ce8
i = 1
have_highpc = 0
#8 0x0000037624c340c4 in read_function_entry (state=0x3762567a000, ddata=0x37618082970, u=0x3761a110998, base=0, unit_buf=0x3761abc70d0, lhdr=0x3761abc71b0, error_callback=0x37624c25e60 <sys_common::gnu::libbacktrace::print::error_cb::h2189b2356f8e6008het>, data=0x3761abc75c0, vec_function=0x376180829d8, vec_inlined=0x376180829d8) at ../../../src/libbacktrace/dwarf.c:2467
is_function = 0
vec = 0x376180829d8
lowpc = 0
highpc = 0
highpc_is_relative = 0
have_ranges = 0
function = 0x0
have_lowpc = 0
ranges = 0
code = 2
abbrev = 0x3761a125ce8
i = 1
have_highpc = 0
#9 0x0000037624c340c4 in read_function_entry (state=0x3762567a000, ddata=0x37618082970, u=0x3761a110998, base=0, unit_buf=0x3761abc70d0, lhdr=0x3761abc71b0, error_callback=0x37624c25e60 <sys_common::gnu::libbacktrace::print::error_cb::h2189b2356f8e6008het>, data=0x3761abc75c0, vec_function=0x376180829d8, vec_inlined=0x376180829d8) at ../../../src/libbacktrace/dwarf.c:2467
is_function = 0
vec = 0x376180829d8
lowpc = 0
highpc = 0
highpc_is_relative = 0
have_ranges = 0
function = 0x0
have_lowpc = 0
ranges = 0
code = 1
abbrev = 0x3761a125cc8
i = 8
have_highpc = 0
#10 0x0000037624c342fa in read_function_info (state=0x3762567a000, ddata=0x37618082970, lhdr=0x3761abc71b0, error_callback=0x37624c25e60 <sys_common::gnu::libbacktrace::print::error_cb::h2189b2356f8e6008het>, data=0x3761abc75c0, u=0x3761a110998, fvec=0x376180829d8, ret_addrs=0x3761abc7210, ret_addrs_count=0x3761abc71f8) at ../../../src/libbacktrace/dwarf.c:2546
lvec = {vec = {base = 0x3761abc75c0, size = 3805341024256, alc = 3805261574144}, count = 6350376}
pfvec = 0x376180829d8
unit_buf = {name = 0x37624cc59a7 ".debug_info", start = 0x376116e0fff <error: Cannot access memory at address 0x376116e0fff>, buf = 0x376119642ff <error: Cannot access memory at address 0x376119642ff>, left = 4604877, is_bigendian = 0, error_callback = 0x37624c25e60 <sys_common::gnu::libbacktrace::print::error_cb::h2189b2356f8e6008het>, data = 0x3761abc75c0, reported_underflow = 0}
addrs = 0x37624f2d401 <panic_loc53072+33>
addrs_count = 264598
#11 0x0000037624c347f5 in dwarf_lookup_pc (state=0x3762567a000, ddata=0x37618082970, pc=3805923426506, callback=0x37624c25e70 <sys_common::gnu::libbacktrace::print::pcinfo_cb::ha768a72a949356196et>, error_callback=0x37624c25e60 <sys_common::gnu::libbacktrace::print::error_cb::h2189b2356f8e6008het>, data=0x3761abc75c0, found=0x3761abc72d4) at ../../../src/libbacktrace/dwarf.c:2731
pfvec = 0x376180829d8
function_addrs_count = 0
lhdr = {version = 2, min_insn_len = 1, max_ops_per_insn = 1, line_base = -5, line_range = 14, opcode_base = 13, opcode_lengths = 0x37611dc8e84 <error: Cannot access memory at address 0x37611dc8e84>, dirs_count = 52, dirs = 0x375fed75d20, filenames_count = 225, filenames = 0x375fc430000}
count = 264598
entry = 0x3761807f5e0
u = 0x3761a110998
new_data = 0
lines = 0x375fb43b000
ln = 0x0
function_addrs = 0x0
function = 0x37625449048 <vtable34401>
filename = 0x37612e05a82 <error: Cannot access memory at address 0x37612e05a82>
lineno = 0
ret = 886
#12 0x0000037624c34cbf in dwarf_fileline (state=0x3762567a000, pc=3805923426506, callback=0x37624c25e70 <sys_common::gnu::libbacktrace::print::pcinfo_cb::ha768a72a949356196et>, error_callback=0x37624c25e60 <sys_common::gnu::libbacktrace::print::error_cb::h2189b2356f8e6008het>, data=0x3761abc75c0) at ../../../src/libbacktrace/dwarf.c:2876
ddata = 0x37618082970
found = 1
ret = 0
#13 0x0000037624c35269 in backtrace_pcinfo (state=0x3762567a000, pc=3805923426506, callback=0x37624c25e70 <sys_common::gnu::libbacktrace::print::pcinfo_cb::ha768a72a949356196et>, error_callback=0x37624c25e60 <sys_common::gnu::libbacktrace::print::error_cb::h2189b2356f8e6008het>, data=0x3761abc75c0) at ../../../src/libbacktrace/fileline.c:176
No locals.
#14 0x0000037624c267c8 in sys::backtrace::tracing::imp::write::trace_fn::he8e626ec33150cf8Npt () at src/libstd/sys/common/gnu/libbacktrace.rs:191
No symbol table info available.
#15 0x000003761cb1b769 in _Unwind_Backtrace (trace=0x37624c26300 <sys::backtrace::tracing::imp::write::trace_fn::he8e626ec33150cf8Npt>, trace_argument=0x3761abc79c8) at /usr/src/debug/sys-devel/gcc-5.2.0/gcc-5.2.0/libgcc/unwind.inc:295
fs = {regs = {reg = {{loc = {reg = 0, offset = 0, exp = 0x0}, how = REG_UNSAVED}, {loc = {reg = 0, offset = 0, exp = 0x0}, how = REG_UNSAVED}, {loc = {reg = 0, offset = 0, exp = 0x0}, how = REG_UNSAVED}, {loc = {reg = 0, offset = 0, exp = 0x0}, how = REG_UNSAVED}, {loc = {reg = 0, offset = 0, exp = 0x0}, how = REG_UNSAVED}, {loc = {reg = 0, offset = 0, exp = 0x0}, how = REG_UNSAVED}, {loc = {reg = 18446744073709551600, offset = -16, exp = 0xfffffffffffffff0 <error: Cannot access memory at address 0xfffffffffffffff0>}, how = REG_SAVED_OFFSET}, {loc = {reg = 0, offset = 0, exp = 0x0}, how = REG_UNSAVED}, {loc = {reg = 0, offset = 0, exp = 0x0}, how = REG_UNSAVED}, {loc = {reg = 0, offset = 0, exp = 0x0}, how = REG_UNSAVED}, {loc = {reg = 0, offset = 0, exp = 0x0}, how = REG_UNSAVED}, {loc = {reg = 0, offset = 0, exp = 0x0}, how = REG_UNSAVED}, {loc = {reg = 0, offset = 0, exp = 0x0}, how = REG_UNSAVED}, {loc = {reg = 0, offset = 0, exp = 0x0}, how = REG_UNSAVED}, {loc = {reg = 0, offset = 0, exp = 0x0}, how = REG_UNSAVED}, {loc = {reg = 0, offset = 0, exp = 0x0}, how = REG_UNSAVED}, {loc = {reg = 18446744073709551608, offset = -8, exp = 0xfffffffffffffff8 <error: Cannot access memory at address 0xfffffffffffffff8>}, how = REG_SAVED_OFFSET}, {loc = {reg = 0, offset = 0, exp = 0x0}, how = REG_UNSAVED}}, prev = 0x0, cfa_offset = 16, cfa_reg = 6, cfa_exp = 0x0, cfa_how = CFA_REG_OFFSET}, pc = 0x37622b6c0b4 <rustc::sys_common::unwind::begin_unwind<syntax::diagnostic::FatalError>+4>, personality = 0x0, data_align = -8, code_align = 1, retaddr_column = 16, fde_encoding = 27 '\033', lsda_encoding = 255 '\377', saw_z = 1 '\001', signal_frame = 0 '\000', eh_ptr = 0x0}
context = {reg = {0x0, 0x0, 0x0, 0x3761abc7d58, 0x0, 0x0, 0x3761abc7d80, 0x0, 0x0, 0x0, 0x0, 0x0, 0x3761abc7d60, 0x3761abc7d68, 0x3761abc7d70, 0x3761abc7d78, 0x3761abc7d88, 0x0}, cfa = 0x3761abc7d90, ra = 0x37622b6c0cb, lsda = 0x0, bases = {tbase = 0x0, dbase = 0x0, func = 0x37622b6c0b0 <rustc::sys_common::unwind::begin_unwind<syntax::diagnostic::FatalError>>}, flags = 4611686018427387904, version = 0, args_size = 0, by_value = '\000' <repeats 17 times>}
code = _URC_NO_REASON
#16 0x0000037624c26017 in std::sys::backtrace::tracing::imp::write (w=...) at src/libstd/sys/unix/backtrace/tracing/gcc_s.rs:40
No locals.
#17 0x0000037624c2e0dc in fnfn (err=...) at src/libstd/panicking.rs:48
No locals.
#18 std::panicking::log_panic (obj=..., file=..., line=<optimized out>, log_backtrace=true) at src/libstd/panicking.rs:55
thread = <optimized out>
#19 0x0000037624be445e in std::panicking::on_panic (file=..., line=486, obj=...) at src/libstd/panicking.rs:87
No locals.
#20 std::sys_common::unwind::begin_unwind_inner (msg=..., file_line=<optimized out>) at src/libstd/sys/common/unwind/mod.rs:242
file = {data_ptr = 0x37622c0f840 <str110789> "src/librustc/session/mod.rs", length = 27}
line = 486
#21 0x0000037622b6c0cb in rustc::sys_common::unwind::begin_unwind<syntax::diagnostic::FatalError> (msg=..., file_line=<optimized out>) at src/libstd/sys/common/unwind/mod.rs:224
No locals.
#22 0x0000037622b9853b in rustc::session::early_error (color=<optimized out>, msg=...) at <std macros>:3
emitter = <optimized out>
#23 0x00000376251f9a5b in rustc_driver::handle_options::parse_all_options (args=<optimized out>) at src/librustc_driver/lib.rs:724
all_groups = <optimized out>
#24 0x00000376251ec738 in rustc_driver::handle_options (args=...) at src/librustc_driver/lib.rs:742
matches = <optimized out>
#25 0x00000376251e9f84 in rustc_driver::run_compiler (args=..., callbacks=...) at src/librustc_driver/lib.rs:117
sess = <optimized out>
control = <optimized out>
sopts = <optimized out>
matches = <optimized out>
cfg = <optimized out>
#26 0x00000376251e7d5b in fnfn () at src/librustc_driver/lib.rs:102
No locals.
#27 fnfn () at src/librustc_driver/lib.rs:824
No locals.
#28 fnfn () at src/libstd/thread/mod.rs:271
No locals.
#29 rustc_driver::sys_common::unwind::try::try_fn<closure> (opt_closure=<optimized out>) at src/libstd/sys/common/unwind/mod.rs:156
No locals.
#30 0x0000037624c23529 in __rust_try () at src/libcore/macros.rs:20
No symbol table info available.
#31 0x0000037624c13671 in fnfn (s=<optimized out>) at src/libstd/sys/common/unwind/mod.rs:144
ep = <optimized out>
prev = <optimized out>
#32 std::thread::local::LocalKey<T>::with<closure,core::result::Result<(), Box<Any>>> (self=<optimized out>, f=...) at src/libstd/thread/local.rs:217
No locals.
#33 std::sys_common::unwind::try::inner_try (f=0x376251e7b80 <rustc_driver::sys_common::unwind::try::try_fn<closure>>, data=0x3761abcbb70 "") at src/libstd/sys/common/unwind/mod.rs:141
No locals.
#34 0x00000376251e822d in rustc_driver::sys_common::unwind::try<closure> (f=...) at src/libstd/sys/common/unwind/mod.rs:126
No locals.
#35 fnfn () at src/libstd/thread/mod.rs:271
ptr = <optimized out>
output = <optimized out>
their_packet = <optimized out>
#36 rustc_driver::boxed::F.FnBox<A>::call_box (self=0x3761ac24120, args=<optimized out>) at src/liballoc/boxed.rs:521
No locals.
#37 0x0000037624c2ca56 in sys::thread::_$LT$impl$GT$::new::thread_start::h29e6fdb9ec3a6fccfGw () at src/liballoc/boxed.rs:529
No symbol table info available.
#38 0x000003761d02764c in start_thread (arg=0x3761abcc700) at pthread_create.c:334
__res = <optimized out>
pd = 0x3761abcc700
now = <optimized out>
unwind_buf = {cancel_jmp_buf = {{jmp_buf = {3805789603584, 300255978620416884, 4161273009680, 4161273009135, 0, 4161273009680, 300255978599445364, 300252734095455092}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
not_first_call = <optimized out>
pagesize_m1 = <optimized out>
sp = <optimized out>
freesize = <optimized out>
__PRETTY_FUNCTION__ = "start_thread"
#39 0x00000376248a59bd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
No locals.
Thread 1 (Thread 0x37625654a00 (LWP 16104)):
#0 0x000003761d0289ad in pthread_join (threadid=3805789603584, thread_return=0x0) at pthread_join.c:90
__tid = 16105
_buffer = {__routine = 0x3761d0288d0 <cleanup>, __arg = 0x3761abccd28, __canceltype = 448790528, __prev = 0x0}
oldtype = 0
pd = 0x3761abcc700
self = 0x37625654a00
result = 0
#1 0x0000037624c2cb3f in std::sys::thread::Thread::join (self=...) at src/libstd/sys/unix/thread.rs:160
ret = <optimized out>
#2 0x00000376251e6bfb in rustc_driver::thread::JoinInner<T>::join (self=<optimized out>) at src/libstd/thread/mod.rs:586
No locals.
#3 rustc_driver::thread::JoinHandle<T>::join (self=...) at src/libstd/thread/mod.rs:616
No locals.
#4 rustc_driver::monitor<closure> (f=...) at src/librustc_driver/lib.rs:824
cfg = {name = {RUST$ENCODED$ENUM$0$0$0$0$0$0$None = {__0 = {vec = {buf = {ptr = {pointer = {__0 = 0x3c8df331e60 '\035' <repeats 40 times>, "\001"}, _marker = {<No data fields>}}, cap = 8388608}, len = 2097865012304223517}}}}, stack_size = {{RUST$ENUM$DISR = (Some | unknown: 2097865012304223516)}, {RUST$ENUM$DISR = (Some | unknown: 2097865012304223516), __0 = 36}}}
err = <optimized out>
#5 rustc_driver::run (args=...) at src/librustc_driver/lib.rs:102
No locals.
#6 0x00000376251fa55b in rustc_driver::main () at src/librustc_driver/lib.rs:879
No locals.
#7 0x0000037624c2dc0a in fnfn () at src/libstd/thread/mod.rs:381
No locals.
#8 std::sys_common::unwind::try::try_fn<closure> (opt_closure=<optimized out>) at src/libstd/sys/common/unwind/mod.rs:156
No locals.
#9 0x0000037624c23529 in __rust_try () at src/libcore/macros.rs:20
No symbol table info available.
#10 0x0000037624c2da0b in fnfn (s=<optimized out>) at src/libstd/sys/common/unwind/mod.rs:144
ep = <optimized out>
prev = <optimized out>
#11 std::thread::local::LocalKey<T>::with<closure,core::result::Result<(), Box<Any>>> (self=<optimized out>, f=...) at src/libstd/thread/local.rs:217
No locals.
#12 std::sys_common::unwind::try::inner_try (f=0x3761abcc9d0, data=<optimized out>) at src/libstd/sys/common/unwind/mod.rs:141
No locals.
#13 std::sys_common::unwind::try<closure> (f=...) at src/libstd/sys/common/unwind/mod.rs:126
No locals.
#14 std::thread::catch_panic<fn(),()> (f=<optimized out>) at src/libstd/thread/mod.rs:381
result = <optimized out>
result = <optimized out>
#15 std::rt::lang_start (main=0x7f7cc32910 <driver::main> "UH\211\345]\351v\376\377\377f\017\037D", argc=<optimized out>, argv=<optimized out>) at src/libstd/rt.rs:55
No locals.
#16 0x00000376247d06a4 in __libc_start_main (main=0x7f7cc32920 <main>, argc=2, argv=0x3c8df332838, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x3c8df332828) at libc-start.c:289
result = <optimized out>
unwind_buf = {cancel_jmp_buf = {{jmp_buf = {0, 384831440184636276, 547554011472, 4161273014320, 0, 0, 384831440344019828, 300268635666477940}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x37625687148, 0x3c8df332820}, data = {prev = 0x0, cleanup = 0x0, canceltype = 627601736}}}
not_first_call = <optimized out>
#17 0x0000007f7cc32809 in _start () at ../sysdeps/x86_64/start.S:108
No locals.
Here's with rust-gdb
EDIT: Here's raw (since the below is hard to horizontally scroll through): https://gist.githubusercontent.com/zazdxscf/56623981eb212b4066e5/raw/4fdfaa3acafbfc87568f3a554b930c64c687b258/gistfile1.txt
$ time rust-gdb /home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage2/bin/rustc --core ./core --quiet -ex 'thread apply all bt full' -ex quit --batch >~/logs/rustc2_crash_backtrace_via_rust-gdb_full.log 2>&1
real 0m25.916s
user 0m24.437s
sys 0m1.357s
$ cat ~/logs/rustc2_crash_backtrace_via_rust-gdb_full.log
[New LWP 16104]
[New LWP 16105]
warning: Could not load shared library symbols for linux-vdso.so.1.
Do you need "set solib-search-path" or "set sysroot"?
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
Core was generated by `/home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage2/bin/rus'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0 0x000003761d0289ad in pthread_join (threadid=3805789603584, thread_return=0x0) at pthread_join.c:90
90 lll_wait_tid (pd->tid);
[Current thread is 1 (Thread 0x37625654a00 (LWP 16104))]
Thread 2 (Thread 0x3761abcc700 (LWP 16105)):
#0 0x0000037624c3724b in backtrace_alloc (state=0x3762567a000, size=384, error_callback=0x37624c25e60 <sys_common::gnu::libbacktrace::print::error_cb::h2189b2356f8e6008het>, data=0x3761abc75c0) at ../../../src/libbacktrace/mmap.c:110
ret = 0x0
locked = 1
pp = 0x375fc349da8
pagesize = 3805957782718
asksize = 3805789579536
page = 0x3761abc70d0
#1 0x0000037624c37558 in backtrace_vector_grow (state=0x3762567a000, size=24, error_callback=0x37624c25e60 <sys_common::gnu::libbacktrace::print::error_cb::h2189b2356f8e6008het>, data=0x3761abc75c0, vec=0x3761abc6bd0) at ../../../src/libbacktrace/mmap.c:236
pagesize = 4096
alc = 384
base = 0x3761abc6a20
ret = 0x8
#2 0x0000037624c3386c in add_function_range (state=0x3762567a000, ddata=0x37618082970, function=0x375fad7e230, lowpc=3805919552193, highpc=3805919552197, error_callback=0x37624c25e60 <sys_common::gnu::libbacktrace::print::error_cb::h2189b2356f8e6008het>, data=0x3761abc75c0, vec=0x3761abc6bd0) at ../../../src/libbacktrace/dwarf.c:2175
p = 0x37624c30173 <read_address+126>
#3 0x0000037624c33a26 in add_function_ranges (state=0x3762567a000, ddata=0x37618082970, u=0x3761a110998, function=0x375fad7e230, ranges=64, base=0, error_callback=0x37624c25e60 <sys_common::gnu::libbacktrace::print::error_cb::h2189b2356f8e6008het>, data=0x3761abc75c0, vec=0x3761abc6bd0) at ../../../src/libbacktrace/dwarf.c:2233
low = 553665
high = 553669
ranges_buf = dwarf_buf = {name = 0x37624cc5999 ".debug_ranges", start = 0x3761288e030 <error: Cannot access memory at address 0x3761288e030>, buf = 0x3761288e080 <error: Cannot access memory at address 0x3761288e080>, left = 5360352, is_bigendian = 0, error_callback = 0x37624c25e60 <sys_common::gnu::libbacktrace::print::error_cb::h2189b2356f8e6008het>, data = 0x3761abc75c0, reported_underflow = 0}
#4 0x0000037624c33fd0 in read_function_entry (state=0x3762567a000, ddata=0x37618082970, u=0x3761a110998, base=0, unit_buf=0x3761abc70d0, lhdr=0x3761abc71b0, error_callback=0x37624c25e60 <sys_common::gnu::libbacktrace::print::error_cb::h2189b2356f8e6008het>, data=0x3761abc75c0, vec_function=0x376180829d8, vec_inlined=0x3761abc6bd0) at ../../../src/libbacktrace/dwarf.c:2443
is_function = 1
vec = 0x3761abc6bd0
lowpc = 0
highpc = 0
highpc_is_relative = 0
have_ranges = 1
function = 0x375fad7e230
have_lowpc = 0
ranges = 64
code = 10
abbrev = 0x3761a125de8
i = 4
have_highpc = 0
#5 0x0000037624c34136 in read_function_entry (state=0x3762567a000, ddata=0x37618082970, u=0x3761a110998, base=0, unit_buf=0x3761abc70d0, lhdr=0x3761abc71b0, error_callback=0x37624c25e60 <sys_common::gnu::libbacktrace::print::error_cb::h2189b2356f8e6008het>, data=0x3761abc75c0, vec_function=0x376180829d8, vec_inlined=0x376180829d8) at ../../../src/libbacktrace/dwarf.c:2481
fvec = function_vector = {vec = backtrace_vector = {base = 0x0, size = 0, alc = 0}, count = 0}
is_function = 1
vec = 0x376180829d8
lowpc = 553600
highpc = 553957
highpc_is_relative = 1
have_ranges = 0
function = 0x375fad7e208
have_lowpc = 1
ranges = 0
code = 46
abbrev = 0x3761a126268
i = 9
have_highpc = 1
#6 0x0000037624c340c4 in read_function_entry (state=0x3762567a000, ddata=0x37618082970, u=0x3761a110998, base=0, unit_buf=0x3761abc70d0, lhdr=0x3761abc71b0, error_callback=0x37624c25e60 <sys_common::gnu::libbacktrace::print::error_cb::h2189b2356f8e6008het>, data=0x3761abc75c0, vec_function=0x376180829d8, vec_inlined=0x376180829d8) at ../../../src/libbacktrace/dwarf.c:2467
is_function = 0
vec = 0x376180829d8
lowpc = 0
highpc = 0
highpc_is_relative = 0
have_ranges = 0
function = 0x0
have_lowpc = 0
ranges = 0
code = 2
abbrev = 0x3761a125ce8
i = 1
have_highpc = 0
#7 0x0000037624c340c4 in read_function_entry (state=0x3762567a000, ddata=0x37618082970, u=0x3761a110998, base=0, unit_buf=0x3761abc70d0, lhdr=0x3761abc71b0, error_callback=0x37624c25e60 <sys_common::gnu::libbacktrace::print::error_cb::h2189b2356f8e6008het>, data=0x3761abc75c0, vec_function=0x376180829d8, vec_inlined=0x376180829d8) at ../../../src/libbacktrace/dwarf.c:2467
is_function = 0
vec = 0x376180829d8
lowpc = 0
highpc = 0
highpc_is_relative = 0
have_ranges = 0
function = 0x0
have_lowpc = 0
ranges = 0
code = 2
abbrev = 0x3761a125ce8
i = 1
have_highpc = 0
#8 0x0000037624c340c4 in read_function_entry (state=0x3762567a000, ddata=0x37618082970, u=0x3761a110998, base=0, unit_buf=0x3761abc70d0, lhdr=0x3761abc71b0, error_callback=0x37624c25e60 <sys_common::gnu::libbacktrace::print::error_cb::h2189b2356f8e6008het>, data=0x3761abc75c0, vec_function=0x376180829d8, vec_inlined=0x376180829d8) at ../../../src/libbacktrace/dwarf.c:2467
is_function = 0
vec = 0x376180829d8
lowpc = 0
highpc = 0
highpc_is_relative = 0
have_ranges = 0
function = 0x0
have_lowpc = 0
ranges = 0
code = 2
abbrev = 0x3761a125ce8
i = 1
have_highpc = 0
#9 0x0000037624c340c4 in read_function_entry (state=0x3762567a000, ddata=0x37618082970, u=0x3761a110998, base=0, unit_buf=0x3761abc70d0, lhdr=0x3761abc71b0, error_callback=0x37624c25e60 <sys_common::gnu::libbacktrace::print::error_cb::h2189b2356f8e6008het>, data=0x3761abc75c0, vec_function=0x376180829d8, vec_inlined=0x376180829d8) at ../../../src/libbacktrace/dwarf.c:2467
is_function = 0
vec = 0x376180829d8
lowpc = 0
highpc = 0
highpc_is_relative = 0
have_ranges = 0
function = 0x0
have_lowpc = 0
ranges = 0
code = 1
abbrev = 0x3761a125cc8
i = 8
have_highpc = 0
#10 0x0000037624c342fa in read_function_info (state=0x3762567a000, ddata=0x37618082970, lhdr=0x3761abc71b0, error_callback=0x37624c25e60 <sys_common::gnu::libbacktrace::print::error_cb::h2189b2356f8e6008het>, data=0x3761abc75c0, u=0x3761a110998, fvec=0x376180829d8, ret_addrs=0x3761abc7210, ret_addrs_count=0x3761abc71f8) at ../../../src/libbacktrace/dwarf.c:2546
lvec = function_vector = {vec = backtrace_vector = {base = 0x3761abc75c0, size = 3805341024256, alc = 3805261574144}, count = 6350376}
pfvec = 0x376180829d8
unit_buf = dwarf_buf = {name = 0x37624cc59a7 ".debug_info", start = 0x376116e0fff <error: Cannot access memory at address 0x376116e0fff>, buf = 0x376119642ff <error: Cannot access memory at address 0x376119642ff>, left = 4604877, is_bigendian = 0, error_callback = 0x37624c25e60 <sys_common::gnu::libbacktrace::print::error_cb::h2189b2356f8e6008het>, data = 0x3761abc75c0, reported_underflow = 0}
addrs = 0x37624f2d401 <panic_loc53072+33>
addrs_count = 264598
#11 0x0000037624c347f5 in dwarf_lookup_pc (state=0x3762567a000, ddata=0x37618082970, pc=3805923426506, callback=0x37624c25e70 <sys_common::gnu::libbacktrace::print::pcinfo_cb::ha768a72a949356196et>, error_callback=0x37624c25e60 <sys_common::gnu::libbacktrace::print::error_cb::h2189b2356f8e6008het>, data=0x3761abc75c0, found=0x3761abc72d4) at ../../../src/libbacktrace/dwarf.c:2731
pfvec = 0x376180829d8
function_addrs_count = 0
lhdr = line_header = {version = 2, min_insn_len = 1, max_ops_per_insn = 1, line_base = -5, line_range = 14, opcode_base = 13, opcode_lengths = 0x37611dc8e84 <error: Cannot access memory at address 0x37611dc8e84>, dirs_count = 52, dirs = 0x375fed75d20, filenames_count = 225, filenames = 0x375fc430000}
count = 264598
entry = 0x3761807f5e0
u = 0x3761a110998
new_data = 0
lines = 0x375fb43b000
ln = 0x0
function_addrs = 0x0
function = 0x37625449048 <vtable34401>
filename = 0x37612e05a82 <error: Cannot access memory at address 0x37612e05a82>
lineno = 0
ret = 886
#12 0x0000037624c34cbf in dwarf_fileline (state=0x3762567a000, pc=3805923426506, callback=0x37624c25e70 <sys_common::gnu::libbacktrace::print::pcinfo_cb::ha768a72a949356196et>, error_callback=0x37624c25e60 <sys_common::gnu::libbacktrace::print::error_cb::h2189b2356f8e6008het>, data=0x3761abc75c0) at ../../../src/libbacktrace/dwarf.c:2876
ddata = 0x37618082970
found = 1
ret = 0
#13 0x0000037624c35269 in backtrace_pcinfo (state=0x3762567a000, pc=3805923426506, callback=0x37624c25e70 <sys_common::gnu::libbacktrace::print::pcinfo_cb::ha768a72a949356196et>, error_callback=0x37624c25e60 <sys_common::gnu::libbacktrace::print::error_cb::h2189b2356f8e6008het>, data=0x3761abc75c0) at ../../../src/libbacktrace/fileline.c:176
No locals.
#14 0x0000037624c267c8 in sys::backtrace::tracing::imp::write::trace_fn::he8e626ec33150cf8Npt () at src/libstd/sys/common/gnu/libbacktrace.rs:191
No symbol table info available.
#15 0x000003761cb1b769 in _Unwind_Backtrace (trace=0x37624c26300 <sys::backtrace::tracing::imp::write::trace_fn::he8e626ec33150cf8Npt>, trace_argument=0x3761abc79c8) at /usr/src/debug/sys-devel/gcc-5.2.0/gcc-5.2.0/libgcc/unwind.inc:295
Python Exception <type 'exceptions.AttributeError'> 'NoneType' object has no attribute 'startswith':
Python Exception <type 'exceptions.TypeError'> Type is not a structure, union, or enum type.:
fs = {regs = frame_state_reg_info = {reg = {{loc = {reg = 0, offset = 0, exp = 0x0}, how = REG_UNSAVED}Python Exception <type 'exceptions.AttributeError'> 'NoneType' object has no attribute 'startswith':
Python Exception <type 'exceptions.TypeError'> Type is not a structure, union, or enum type.:
, {loc = {reg = 0, offset = 0, exp = 0x0}, how = REG_UNSAVED}Python Exception <type 'exceptions.AttributeError'> 'NoneType' object has no attribute 'startswith':
Python Exception <type 'exceptions.TypeError'> Type is not a structure, union, or enum type.:
, {loc = {reg = 0, offset = 0, exp = 0x0}, how = REG_UNSAVED}Python Exception <type 'exceptions.AttributeError'> 'NoneType' object has no attribute 'startswith':
Python Exception <type 'exceptions.TypeError'> Type is not a structure, union, or enum type.:
, {loc = {reg = 0, offset = 0, exp = 0x0}, how = REG_UNSAVED}Python Exception <type 'exceptions.AttributeError'> 'NoneType' object has no attribute 'startswith':
Python Exception <type 'exceptions.TypeError'> Type is not a structure, union, or enum type.:
, {loc = {reg = 0, offset = 0, exp = 0x0}, how = REG_UNSAVED}Python Exception <type 'exceptions.AttributeError'> 'NoneType' object has no attribute 'startswith':
Python Exception <type 'exceptions.TypeError'> Type is not a structure, union, or enum type.:
, {loc = {reg = 0, offset = 0, exp = 0x0}, how = REG_UNSAVED}Python Exception <type 'exceptions.AttributeError'> 'NoneType' object has no attribute 'startswith':
Python Exception <type 'exceptions.TypeError'> Type is not a structure, union, or enum type.:
, {loc = {reg = 18446744073709551600, offset = -16, exp = 0xfffffffffffffff0 <error: Cannot access memory at address 0xfffffffffffffff0>}, how = REG_SAVED_OFFSET}Python Exception <type 'exceptions.AttributeError'> 'NoneType' object has no attribute 'startswith':
Python Exception <type 'exceptions.TypeError'> Type is not a structure, union, or enum type.:
, {loc = {reg = 0, offset = 0, exp = 0x0}, how = REG_UNSAVED}Python Exception <type 'exceptions.AttributeError'> 'NoneType' object has no attribute 'startswith':
Python Exception <type 'exceptions.TypeError'> Type is not a structure, union, or enum type.:
, {loc = {reg = 0, offset = 0, exp = 0x0}, how = REG_UNSAVED}Python Exception <type 'exceptions.AttributeError'> 'NoneType' object has no attribute 'startswith':
Python Exception <type 'exceptions.TypeError'> Type is not a structure, union, or enum type.:
, {loc = {reg = 0, offset = 0, exp = 0x0}, how = REG_UNSAVED}Python Exception <type 'exceptions.AttributeError'> 'NoneType' object has no attribute 'startswith':
Python Exception <type 'exceptions.TypeError'> Type is not a structure, union, or enum type.:
, {loc = {reg = 0, offset = 0, exp = 0x0}, how = REG_UNSAVED}Python Exception <type 'exceptions.AttributeError'> 'NoneType' object has no attribute 'startswith':
Python Exception <type 'exceptions.TypeError'> Type is not a structure, union, or enum type.:
, {loc = {reg = 0, offset = 0, exp = 0x0}, how = REG_UNSAVED}Python Exception <type 'exceptions.AttributeError'> 'NoneType' object has no attribute 'startswith':
Python Exception <type 'exceptions.TypeError'> Type is not a structure, union, or enum type.:
, {loc = {reg = 0, offset = 0, exp = 0x0}, how = REG_UNSAVED}Python Exception <type 'exceptions.AttributeError'> 'NoneType' object has no attribute 'startswith':
Python Exception <type 'exceptions.TypeError'> Type is not a structure, union, or enum type.:
, {loc = {reg = 0, offset = 0, exp = 0x0}, how = REG_UNSAVED}Python Exception <type 'exceptions.AttributeError'> 'NoneType' object has no attribute 'startswith':
Python Exception <type 'exceptions.TypeError'> Type is not a structure, union, or enum type.:
, {loc = {reg = 0, offset = 0, exp = 0x0}, how = REG_UNSAVED}Python Exception <type 'exceptions.AttributeError'> 'NoneType' object has no attribute 'startswith':
Python Exception <type 'exceptions.TypeError'> Type is not a structure, union, or enum type.:
, {loc = {reg = 0, offset = 0, exp = 0x0}, how = REG_UNSAVED}Python Exception <type 'exceptions.AttributeError'> 'NoneType' object has no attribute 'startswith':
Python Exception <type 'exceptions.TypeError'> Type is not a structure, union, or enum type.:
, {loc = {reg = 18446744073709551608, offset = -8, exp = 0xfffffffffffffff8 <error: Cannot access memory at address 0xfffffffffffffff8>}, how = REG_SAVED_OFFSET}Python Exception <type 'exceptions.AttributeError'> 'NoneType' object has no attribute 'startswith':
Python Exception <type 'exceptions.TypeError'> Type is not a structure, union, or enum type.:
, {loc = {reg = 0, offset = 0, exp = 0x0}, how = REG_UNSAVED}}, prev = 0x0, cfa_offset = 16, cfa_reg = 6, cfa_exp = 0x0, cfa_how = CFA_REG_OFFSET}, pc = 0x37622b6c0b4 <rustc::sys_common::unwind::begin_unwind<syntax::diagnostic::FatalError>+4>, personality = 0x0, data_align = -8, code_align = 1, retaddr_column = 16, fde_encoding = 27 '\033', lsda_encoding = 255 '\377', saw_z = 1 '\001', signal_frame = 0 '\000', eh_ptr = 0x0}
context = _Unwind_Context = {reg = {0x0, 0x0, 0x0, 0x3761abc7d58, 0x0, 0x0, 0x3761abc7d80, 0x0, 0x0, 0x0, 0x0, 0x0, 0x3761abc7d60, 0x3761abc7d68, 0x3761abc7d70, 0x3761abc7d78, 0x3761abc7d88, 0x0}, cfa = 0x3761abc7d90, ra = 0x37622b6c0cb, lsda = 0x0, bases = dwarf_eh_bases = {tbase = 0x0, dbase = 0x0, func = 0x37622b6c0b0 <rustc::sys_common::unwind::begin_unwind<syntax::diagnostic::FatalError>>}, flags = 4611686018427387904, version = 0, args_size = 0, by_value = '\000' <repeats 17 times>}
code = _URC_NO_REASON
#16 0x0000037624c26017 in std::sys::backtrace::tracing::imp::write (w=&mut Write) at src/libstd/sys/unix/backtrace/tracing/gcc_s.rs:40
No locals.
#17 0x0000037624c2e0dc in fnfn (err=...) at src/libstd/panicking.rs:48
No locals.
#18 std::panicking::log_panic (obj=..., file=..., line=<optimized out>, log_backtrace=true) at src/libstd/panicking.rs:55
thread = <optimized out>
#19 0x0000037624be445e in std::panicking::on_panic (file="src/librustc/session/mod.rs", line=486, obj=...) at src/libstd/panicking.rs:87
No locals.
#20 std::sys_common::unwind::begin_unwind_inner (msg=..., file_line=<optimized out>) at src/libstd/sys/common/unwind/mod.rs:242
file = "src/librustc/session/mod.rs"
line = 486
#21 0x0000037622b6c0cb in rustc::sys_common::unwind::begin_unwind<syntax::diagnostic::FatalError> (msg=..., file_line=<optimized out>) at src/libstd/sys/common/unwind/mod.rs:224
No locals.
#22 0x0000037622b9853b in rustc::session::early_error (color=<optimized out>, msg=...) at <std macros>:3
emitter = <optimized out>
#23 0x00000376251f9a5b in rustc_driver::handle_options::parse_all_options (args=<optimized out>) at src/librustc_driver/lib.rs:724
all_groups = <optimized out>
#24 0x00000376251ec738 in rustc_driver::handle_options (args=...) at src/librustc_driver/lib.rs:742
Python Exception <type 'exceptions.MemoryError'> :
matches = <optimized out>
#25 0x00000376251e9f84 in rustc_driver::run_compiler (args=&[collections::string::String](len: 3805957742176), callbacks=&mut CompilerCalls) at src/librustc_driver/lib.rs:117
sess = <optimized out>
control = <optimized out>
sopts = <optimized out>
matches = <optimized out>
cfg = <optimized out>
#26 0x00000376251e7d5b in fnfn () at src/librustc_driver/lib.rs:102
No locals.
#27 fnfn () at src/librustc_driver/lib.rs:824
No locals.
#28 fnfn () at src/libstd/thread/mod.rs:271
No locals.
#29 rustc_driver::sys_common::unwind::try::try_fn<closure> (opt_closure=<optimized out>) at src/libstd/sys/common/unwind/mod.rs:156
No locals.
#30 0x0000037624c23529 in __rust_try () at src/libcore/macros.rs:20
No symbol table info available.
#31 0x0000037624c13671 in fnfn (s=<optimized out>) at src/libstd/sys/common/unwind/mod.rs:144
ep = <optimized out>
prev = <optimized out>
#32 std::thread::local::LocalKey<T>::with<closure,core::result::Result<(), Box<Any>>> (self=<optimized out>, f=...) at src/libstd/thread/local.rs:217
No locals.
#33 std::sys_common::unwind::try::inner_try (f=0x376251e7b80 <rustc_driver::sys_common::unwind::try::try_fn<closure>>, data=0x3761abcbb70 "") at src/libstd/sys/common/unwind/mod.rs:141
No locals.
#34 0x00000376251e822d in rustc_driver::sys_common::unwind::try<closure> (f=...) at src/libstd/sys/common/unwind/mod.rs:126
No locals.
#35 fnfn () at src/libstd/thread/mod.rs:271
ptr = <optimized out>
output = <optimized out>
their_packet = <optimized out>
#36 rustc_driver::boxed::F.FnBox<A>::call_box (self=0x3761ac24120, args=<optimized out>) at src/liballoc/boxed.rs:521
No locals.
#37 0x0000037624c2ca56 in sys::thread::_$LT$impl$GT$::new::thread_start::h29e6fdb9ec3a6fccfGw () at src/liballoc/boxed.rs:529
No symbol table info available.
#38 0x000003761d02764c in start_thread (arg=0x3761abcc700) at pthread_create.c:334
Python Exception <type 'exceptions.AttributeError'> 'NoneType' object has no attribute 'startswith':
__res = <optimized out>
pd = 0x3761abcc700
now = <optimized out>
unwind_buf = pthread_unwind_buf = {cancel_jmp_buf = {{jmp_buf = {3805789603584, 300255978620416884, 4161273009680, 4161273009135, 0, 4161273009680, 300255978599445364, 300252734095455092}, mask_was_saved = 0}}Python Exception <type 'exceptions.TypeError'> Invalid lookup for a field not contained in the value.:
, priv = {pad = {0x0, 0x0, 0x0, 0x0}Python Exception <type 'exceptions.AttributeError'> 'NoneType' object has no attribute 'startswith':
, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
not_first_call = <optimized out>
pagesize_m1 = <optimized out>
sp = <optimized out>
freesize = <optimized out>
__PRETTY_FUNCTION__ = "start_thread"
#39 0x00000376248a59bd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
No locals.
Thread 1 (Thread 0x37625654a00 (LWP 16104)):
#0 0x000003761d0289ad in pthread_join (threadid=3805789603584, thread_return=0x0) at pthread_join.c:90
__tid = 16105
_buffer = _pthread_cleanup_buffer = {__routine = 0x3761d0288d0 <cleanup>, __arg = 0x3761abccd28, __canceltype = 448790528, __prev = 0x0}
oldtype = 0
pd = 0x3761abcc700
self = 0x37625654a00
result = 0
#1 0x0000037624c2cb3f in std::sys::thread::Thread::join (self=...) at src/libstd/sys/unix/thread.rs:160
ret = <optimized out>
#2 0x00000376251e6bfb in rustc_driver::thread::JoinInner<T>::join (self=<optimized out>) at src/libstd/thread/mod.rs:586
No locals.
#3 rustc_driver::thread::JoinHandle<T>::join (self=...) at src/libstd/thread/mod.rs:616
No locals.
#4 rustc_driver::monitor<closure> (f=...) at src/librustc_driver/lib.rs:824
Python Exception <class 'gdb.error'> Cannot convert value to int.:
Python Exception <type 'exceptions.OverflowError'> signed integer is greater than maximum:
cfg = Builder = {name = {RUST$ENCODED$ENUM$0$0$0$0$0$0$None = Some = {}}Python Exception <type 'exceptions.TypeError'> Could not convert Python object: None.:
, stack_size = {(Some | unknown: 2097865012304223516), Some = {36}}}
err = <optimized out>
#5 rustc_driver::run (args=...) at src/librustc_driver/lib.rs:102
No locals.
#6 0x00000376251fa55b in rustc_driver::main () at src/librustc_driver/lib.rs:879
No locals.
#7 0x0000037624c2dc0a in fnfn () at src/libstd/thread/mod.rs:381
No locals.
#8 std::sys_common::unwind::try::try_fn<closure> (opt_closure=<optimized out>) at src/libstd/sys/common/unwind/mod.rs:156
No locals.
#9 0x0000037624c23529 in __rust_try () at src/libcore/macros.rs:20
No symbol table info available.
#10 0x0000037624c2da0b in fnfn (s=<optimized out>) at src/libstd/sys/common/unwind/mod.rs:144
ep = <optimized out>
prev = <optimized out>
#11 std::thread::local::LocalKey<T>::with<closure,core::result::Result<(), Box<Any>>> (self=<optimized out>, f=...) at src/libstd/thread/local.rs:217
No locals.
#12 std::sys_common::unwind::try::inner_try (f=0x3761abcc9d0, data=<optimized out>) at src/libstd/sys/common/unwind/mod.rs:141
No locals.
#13 std::sys_common::unwind::try<closure> (f=...) at src/libstd/sys/common/unwind/mod.rs:126
No locals.
#14 std::thread::catch_panic<fn(),()> (f=<optimized out>) at src/libstd/thread/mod.rs:381
result = <optimized out>
result = <optimized out>
#15 std::rt::lang_start (main=0x7f7cc32910 <driver::main> "UH\211\345]\351v\376\377\377f\017\037D", argc=<optimized out>, argv=<optimized out>) at src/libstd/rt.rs:55
No locals.
#16 0x00000376247d06a4 in __libc_start_main (main=0x7f7cc32920 <main>, argc=2, argv=0x3c8df332838, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x3c8df332828) at libc-start.c:289
Python Exception <type 'exceptions.AttributeError'> 'NoneType' object has no attribute 'startswith':
result = <optimized out>
unwind_buf = pthread_unwind_buf = {cancel_jmp_buf = {{jmp_buf = {0, 384831440184636276, 547554011472, 4161273014320, 0, 0, 384831440344019828, 300268635666477940}, mask_was_saved = 0}}Python Exception <type 'exceptions.TypeError'> Invalid lookup for a field not contained in the value.:
, priv = {pad = {0x0, 0x0, 0x37625687148, 0x3c8df332820}Python Exception <type 'exceptions.AttributeError'> 'NoneType' object has no attribute 'startswith':
, data = {prev = 0x0, cleanup = 0x0, canceltype = 627601736}}}
not_first_call = <optimized out>
#17 0x0000007f7cc32809 in _start () at ../sysdeps/x86_64/start.S:108
No locals.
I guess the question is: why take 10 times more time when RUST_BACKTRACE is set in the environment AND not display any backtrace? (or, just forget the AND part)
$ unset RUST_BACKTRACE
$ which rustc
/home/zazdxscf/build/1nonpkgs/rust/rust/x86_64-unknown-linux-gnu/stage2/bin/rustc
$ time rustc -VV
error: Option 'version' given more than once.
real 0m0.055s
user 0m0.027s
sys 0m0.023s
$ time RUST_BACKTRACE=1 rustc -VV
error: Option 'version' given more than once.
real 0m0.619s
user 0m0.563s
sys 0m0.050s
$ time RUST_BACKTRACE=1 rustc -Vv
rustc 1.6.0-dev (cc8d398e2 2015-10-30)
binary: rustc
commit-hash: cc8d398e28b6b1918ef85479c2d040dfd0fe582d
commit-date: 2015-10-30
host: x86_64-unknown-linux-gnu
release: 1.6.0-dev
real 0m0.055s
user 0m0.027s
sys 0m0.027s
Can someone confirm, by doing the above steps, that their rustc takes like 10 times more (even though only half a second total) when RUST_BACKTRACE is set? Please. So that I know whether it's my other unusual ./configure options causing this or not.
Apparently only when compiling with --enable-debuginfo then, only stage2 rustc (whose files inside /lib directory(excluding rustlib/ directory) is 100meg bigger than stage1 - that is, like 180meg vs 80meg) does take like 45 seconds to complete on errors. Anything else (any stage0 or stage1; or a stage2 with --disable-debuginfo) takes 0.6sec on errors when RUST_BACKTRACE is set, and 0.05 sec when it's not set).
But, even with --enable-debuginfo, stage2 rustc will still do 0.6sec if the stage2 compilation is not finished(eg. rust is still being compiled, but stage2 rustc is already available at some point during this compilation). So when compilation is all done, stage2 rustc (with debuginfo) will always take around 44sec on errors when RUST_BACKTRACE is set.
In other words, the 10 times delay that occurs when RUST_BACKTRACE is set is exacerbated by having built rust with --enable-debuginfo (so it now takes like 44 seconds instead of 0.6 seconds)
This also means that stage1 is not built with --enable-debuginfo yet. Only stage2 is. (not only because stage2's lib files weigh 100 meg more than stage1's, but also because stage1 takes 0.6s and stage2 takes 44s - see above). Although it looks like it should be: https://github.com/rust-lang/rust/commit/92276dc616195f327ff793235fd81f498b565b8b#diff-f40c1cf0c004abfc517b43348f11ab48R137
ghost
on 31 Oct 2015
Closing, as I've decided, as a lifestyle choice, to avoid focusing on bugs(so passionately) and rather just work around them, mainly because that is just wasting everybody's time needlessly.
For current issue, the workaround is: either not use --enable-debuginfo when compiling rust, or/and make sure RUST_BACKTRACE is unset in environment.
Peace out.
ghost
on 1 Nov 2015
I can confirm this and would like to see it fixed.
arielb1
on 1 Nov 2015
I did a bit of testing here. As suggested by the previous posts it seems to be spending all its time in src/libbacktrace/mmap.c:backtrace_alloc (traversing the linked list, apparently). I removed the statements suppressing the backtrace output and found that most of the time is being spent between these two lines:
4: 0x7f864c947cea - sys_common::unwind::begin_unwind::begin_unwind::h127295331395719567
at src/libstd/sys/common/unwind/mod.rs:224
(with another reasonable chunk two frames further out).
The basic cause of this seems to be that backtrace_alloc is slow and is being called a lot of times (roughly 300,000). On my machine, RUST_BACKTRACE unset takes under 0.1 second and RUST_BACKTRACE=1 takes about a minute. If I swap out the backtrace allocators for malloc and similar, that drops to around 1.2 seconds (this isn't a solution because it's not safe to call malloc in some places you'd want a backtrace). At that point we seem to be dominated by the processing in src/libbacktrace/dwarf.c:read_function_entry.
wthrowe
on 5 Nov 2015
❤2
🎉2
👍2
@wthrowe that's... weird! Do you know why backtrace wants to allocate so much memory? Is something accidentally being called a huge number of times?
alexcrichton
on 6 Nov 2015
read_function_entry is being called a huge number of times, mostly by recursively calling itself apparently. I'm not quite sure what that is doing, but I think it may be examining every symbol in...something? I don't have a good feeling for whether what is going on there is reasonable or not.
In the experimenting direction, I tried moving the struct function defined near the top of read_function_entry onto the stack and storing the one in struct function_addrs by value. This reduced the number of calls to the allocator by a factor of three but apparently somehow made them even slower so the runtime actually increased (still apparently allocation-dominated). My current hypothesis is that the backtrace allocator is getting into some pathological state. I will investigate further.
wthrowe
on 6 Nov 2015
❤2
🎉2
👍2
Hypothesis confirmed. The linked list of freed chunks of memory available for reuse accumulates a huge number of chunks that are too small to fulfill most allocation requests, so some requests have to check tens of thousands of chunks for sufficient space before giving up and mmaping more memory.
As an experiment, I modified the allocator to only check the first (most recently added) 10 chunks on the list. This dropped the runtime to a similar value to my previous malloc tests (a bit over a second). After 100,000 allocations the modified version has a resident memory size of 67.5 MB, as opposed to the original version which uses 66.6 MB, so while this does decrease them memory efficiency of the allocator the effect is not large. I'm not sure if this is the correct path to fixing the problem; it may make more sense to add logic to the memory freeing code to drop small chunks (but the definition of "small" is not obvious).
I think I'll file an upstream bug with gcc (which appears to be the main repository for libbacktrace) to see if they have any ideas.
wthrowe
on 6 Nov 2015
🎉3
❤2
👍2
Upstream bug: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=68239
wthrowe
on 6 Nov 2015
❤3
👍2
Impressive investigation @wthrowe! Thanks for filing the upstream bug!
alexcrichton
on 6 Nov 2015
🎉1
👍1
I think this has since been fixed with an update to libbacktrace, so I'm gonna close, feel free to reopen though if it still happens!
alexcrichton
on 24 May 2016
👎1
Nope, still an issue (and the upstream bug seems still open/unaddressed):
$ time rustc -vv
!! executing '/home/z/build/1nonpkgs/rust/rust//x86_64-unknown-linux-gnu/stage2/bin//rustc' with args: '-vv'
error: Option 'verbose' given more than once.
real 0m0.058s
user 0m0.021s
sys 0m0.035s
$ time rustc -vv
!! executing '/home/z/build/1nonpkgs/rust/rust//x86_64-unknown-linux-gnu/stage2/bin//rustc' with args: '-vv'
error: Option 'verbose' given more than once.
real 0m0.058s
user 0m0.020s
sys 0m0.037s
$ time RUST_BACKTRACE=1 rustc -vv
!! executing '/home/z/build/1nonpkgs/rust/rust//x86_64-unknown-linux-gnu/stage2/bin//rustc' with args: '-vv'
error: Option 'verbose' given more than once.
real 0m0.372s
user 0m0.320s
sys 0m0.050s
$ time RUST_BACKTRACE=1 rustc -vv
!! executing '/home/z/build/1nonpkgs/rust/rust//x86_64-unknown-linux-gnu/stage2/bin//rustc' with args: '-vv'
error: Option 'verbose' given more than once.
real 0m0.371s
user 0m0.315s
sys 0m0.053s
$ rustc -Vv
!! executing '/home/z/build/1nonpkgs/rust/rust//x86_64-unknown-linux-gnu/stage2/bin//rustc' with args: '-Vv'
rustc 1.11.0-dev (8d8a88f4a 2016-06-19)
binary: rustc
commit-hash: 8d8a88f4a5bac0038a621bfb195db13d9ae00cb8
commit-date: 2016-06-19
host: x86_64-unknown-linux-gnu
release: 1.11.0-dev
(It looks like )I had compiled rust with --disable-debuginfo and definitely did not have --enable-debuginfo . I also had --disable-debug and no --enable-debug.
So the only workaround(of the two) currently is via unset or non '1' RUST_BACKTRACE.
EDIT: Oh wait, I forgot! It's due to not using --enable-debuginfo that it takes under half a second instead of 44seconds. And when not using a set RUST_BACKTRACE is when it takes normal time. So basically, if I were to compile with --enable-debuginfo it would take 40+ seconds above, but without it, it only takes 6.4 times more timeas when no backtrace env. var is set. (that's 0.371sec vs 0.058sec)
ActualizeInMaterial
on 23 Jun 2016
Still happening:
0m0.062s vs 0m0.454s
/home/z $ time RUST_BACKTRACE=0 rustc -vv
!! executing '/home/z/build/1nonpkgs/rust/rust//x86_64-unknown-linux-gnu/stage2/bin//rustc' with args: '-vv'
error: Option 'verbose' given more than once.
real 0m0.062s
user 0m0.020s
sys 0m0.041s
-----------
z@myzee 2016/08/06 15:04:23 -bash4.3.46 t:3 j:0 d:3 pp:1119 p:4019 ut1603
!13231 19 101 4.7.0-ga157b3a #6 SMP PREEMPT Sat Aug 6 12:11:16 EEST 2016
/home/z
/home/z $ time RUST_BACKTRACE=1 rustc -vv
!! executing '/home/z/build/1nonpkgs/rust/rust//x86_64-unknown-linux-gnu/stage2/bin//rustc' with args: '-vv'
error: Option 'verbose' given more than once.
real 0m0.454s
user 0m0.341s
sys 0m0.080s
$ rustc -Vv
!! executing '/home/z/build/1nonpkgs/rust/rust//x86_64-unknown-linux-gnu/stage2/bin//rustc' with args: '-Vv'
rustc 1.12.0-dev (2c1612c62 2016-08-01)
binary: rustc
commit-hash: 2c1612c62aa59e40cf1a4bddde943938e0147eec
commit-date: 2016-08-01
host: x86_64-unknown-linux-gnu
release: 1.12.0-dev
icanrealizeum
on 6 Aug 2016
There's this similar-looking bug for the linux kernel: https://bugzilla.kernel.org/show_bug.cgi?id=151621
whereby umount-ing something(probably not just btrfs) will spend quite some time freeing pagecache pages, usually between 7 and 127 seconds. So if your shutdown takes a longer than usual time(no? just me then? okay) then this might be it.
Sorry, just had to mention it.
icanrealizeum
on 6 Aug 2016
I am also seeing this with rustc 1.15.0-nightly (ac635aa95 2016-11-18). Unfortunately can't post the source, but behavior is exactly as described above in this post, and #37477 -- a long delay after compiling when backtraces are enabled. gdb confirms that time is spent in backtrace_alloc:
(gdb) t 2.2
[Switching to thread 2.2 (Thread 0x7fffee1ff700 (LWP 20241))]
#0 0x00007ffff77b207d in backtrace_alloc (state=0x7ffff7fc1000, size=384, error_callback=0x7ffff7798e40 <std::sys_common::gnu::libbacktrace::print::error_cb>, data=0x7fffee1fb4f0)
at /buildslave/rust-buildbot/slave/nightly-dist-rustc-linux/build/src/libbacktrace/mmap.c:115
115 in /buildslave/rust-buildbot/slave/nightly-dist-rustc-linux/build/src/libbacktrace/mmap.c
(gdb) bt
#0 0x00007ffff77b207d in backtrace_alloc (state=0x7ffff7fc1000, size=384, error_callback=0x7ffff7798e40 <std::sys_common::gnu::libbacktrace::print::error_cb>, data=0x7fffee1fb4f0)
at /buildslave/rust-buildbot/slave/nightly-dist-rustc-linux/build/src/libbacktrace/mmap.c:115
#1 0x00007ffff77b2378 in backtrace_vector_grow (state=0x7ffff7fc1000, size=24, error_callback=0x7ffff7798e40 <std::sys_common::gnu::libbacktrace::print::error_cb>, data=0x7fffee1fb4f0, vec=0x7fffee1fa840)
at /buildslave/rust-buildbot/slave/nightly-dist-rustc-linux/build/src/libbacktrace/mmap.c:244
#2 0x00007ffff77ae782 in add_function_range (state=0x7ffff7fc1000, ddata=0x7fffe73b6138, function=0x7fffe47d2fa0, lowpc=140737299864630, highpc=140737299864692,
error_callback=0x7ffff7798e40 <std::sys_common::gnu::libbacktrace::print::error_cb>, data=0x7fffee1fb4f0, vec=0x7fffee1fa840)
at /buildslave/rust-buildbot/slave/nightly-dist-rustc-linux/build/src/libbacktrace/dwarf.c:2175
#3 0x00007ffff77aef42 in read_function_entry (state=0x7ffff7fc1000, ddata=0x7fffe73b6138, u=0x7fffe7c1a700, base=0, unit_buf=0x7fffee1fb040, lhdr=0x7fffee1fb120,
error_callback=0x7ffff7798e40 <std::sys_common::gnu::libbacktrace::print::error_cb>, data=0x7fffee1fb4f0, vec_function=0x7fffe73b61a0, vec_inlined=0x7fffee1fa840)
at /buildslave/rust-buildbot/slave/nightly-dist-rustc-linux/build/src/libbacktrace/dwarf.c:2451
#4 0x00007ffff77af067 in read_function_entry (state=0x7ffff7fc1000, ddata=0x7fffe73b6138, u=0x7fffe7c1a700, base=0, unit_buf=0x7fffee1fb040, lhdr=0x7fffee1fb120,
error_callback=0x7ffff7798e40 <std::sys_common::gnu::libbacktrace::print::error_cb>, data=0x7fffee1fb4f0, vec_function=0x7fffe73b61a0, vec_inlined=0x7fffee1fa940)
at /buildslave/rust-buildbot/slave/nightly-dist-rustc-linux/build/src/libbacktrace/dwarf.c:2481
#5 0x00007ffff77af067 in read_function_entry (state=0x7ffff7fc1000, ddata=0x7fffe73b6138, u=0x7fffe7c1a700, base=0, unit_buf=0x7fffee1fb040, lhdr=0x7fffee1fb120,
error_callback=0x7ffff7798e40 <std::sys_common::gnu::libbacktrace::print::error_cb>, data=0x7fffee1fb4f0, vec_function=0x7fffe73b61a0, vec_inlined=0x7fffee1faa40)
at /buildslave/rust-buildbot/slave/nightly-dist-rustc-linux/build/src/libbacktrace/dwarf.c:2481
#6 0x00007ffff77af067 in read_function_entry (state=0x7ffff7fc1000, ddata=0x7fffe73b6138, u=0x7fffe7c1a700, base=0, unit_buf=0x7fffee1fb040, lhdr=0x7fffee1fb120,
error_callback=0x7ffff7798e40 <std::sys_common::gnu::libbacktrace::print::error_cb>, data=0x7fffee1fb4f0, vec_function=0x7fffe73b61a0, vec_inlined=0x7fffe73b61a0)
at /buildslave/rust-buildbot/slave/nightly-dist-rustc-linux/build/src/libbacktrace/dwarf.c:2481
#7 0x00007ffff77aefe4 in read_function_entry (state=0x7ffff7fc1000, ddata=0x7fffe73b6138, u=0x7fffe7c1a700, base=0, unit_buf=0x7fffee1fb040, lhdr=0x7fffee1fb120,
error_callback=0x7ffff7798e40 <std::sys_common::gnu::libbacktrace::print::error_cb>, data=0x7fffee1fb4f0, vec_function=0x7fffe73b61a0, vec_inlined=0x7fffe73b61a0)
at /buildslave/rust-buildbot/slave/nightly-dist-rustc-linux/build/src/libbacktrace/dwarf.c:2467
#8 0x00007ffff77aefe4 in read_function_entry (state=0x7ffff7fc1000, ddata=0x7fffe73b6138, u=0x7fffe7c1a700, base=0, unit_buf=0x7fffee1fb040, lhdr=0x7fffee1fb120,
error_callback=0x7ffff7798e40 <std::sys_common::gnu::libbacktrace::print::error_cb>, data=0x7fffee1fb4f0, vec_function=0x7fffe73b61a0, vec_inlined=0x7fffe73b61a0)
at /buildslave/rust-buildbot/slave/nightly-dist-rustc-linux/build/src/libbacktrace/dwarf.c:2467
#9 0x00007ffff77aefe4 in read_function_entry (state=0x7ffff7fc1000, ddata=0x7fffe73b6138, u=0x7fffe7c1a700, base=0, unit_buf=0x7fffee1fb040, lhdr=0x7fffee1fb120,
error_callback=0x7ffff7798e40 <std::sys_common::gnu::libbacktrace::print::error_cb>, data=0x7fffee1fb4f0, vec_function=0x7fffe73b61a0, vec_inlined=0x7fffe73b61a0)
at /buildslave/rust-buildbot/slave/nightly-dist-rustc-linux/build/src/libbacktrace/dwarf.c:2467
#10 0x00007ffff77aefe4 in read_function_entry (state=0x7ffff7fc1000, ddata=0x7fffe73b6138, u=0x7fffe7c1a700, base=0, unit_buf=0x7fffee1fb040, lhdr=0x7fffee1fb120,
error_callback=0x7ffff7798e40 <std::sys_common::gnu::libbacktrace::print::error_cb>, data=0x7fffee1fb4f0, vec_function=0x7fffe73b61a0, vec_inlined=0x7fffe73b61a0)
at /buildslave/rust-buildbot/slave/nightly-dist-rustc-linux/build/src/libbacktrace/dwarf.c:2467
#11 0x00007ffff77aefe4 in read_function_entry (state=0x7ffff7fc1000, ddata=0x7fffe73b6138, u=0x7fffe7c1a700, base=0, unit_buf=0x7fffee1fb040, lhdr=0x7fffee1fb120,
error_callback=0x7ffff7798e40 <std::sys_common::gnu::libbacktrace::print::error_cb>, data=0x7fffee1fb4f0, vec_function=0x7fffe73b61a0, vec_inlined=0x7fffe73b61a0)
at /buildslave/rust-buildbot/slave/nightly-dist-rustc-linux/build/src/libbacktrace/dwarf.c:2467
#12 0x00007ffff77af230 in read_function_info (state=0x7ffff7fc1000, ddata=0x7fffe73b6138, lhdr=0x7fffee1fb120, error_callback=0x7ffff7798e40 <std::sys_common::gnu::libbacktrace::print::error_cb>,
data=0x7fffee1fb4f0, u=0x7fffe7c1a700, fvec=0x7fffe73b61a0, ret_addrs=0x7fffee1fb180, ret_addrs_count=0x7fffee1fb168)
at /buildslave/rust-buildbot/slave/nightly-dist-rustc-linux/build/src/libbacktrace/dwarf.c:2546
#13 0x00007ffff77af714 in dwarf_lookup_pc (state=0x7ffff7fc1000, ddata=0x7fffe73b6138, pc=140737297503162, callback=0x7ffff7798e60 <std::sys_common::gnu::libbacktrace::print::pcinfo_cb>,
error_callback=0x7ffff7798e40 <std::sys_common::gnu::libbacktrace::print::error_cb>, data=0x7fffee1fb4f0, found=0x7fffee1fb244)
at /buildslave/rust-buildbot/slave/nightly-dist-rustc-linux/build/src/libbacktrace/dwarf.c:2731
#14 0x00007ffff77afbc5 in dwarf_fileline (state=0x7ffff7fc1000, pc=140737297503162, callback=0x7ffff7798e60 <std::sys_common::gnu::libbacktrace::print::pcinfo_cb>,
error_callback=0x7ffff7798e40 <std::sys_common::gnu::libbacktrace::print::error_cb>, data=0x7fffee1fb4f0) at /buildslave/rust-buildbot/slave/nightly-dist-rustc-linux/build/src/libbacktrace/dwarf.c:2876
#15 0x00007ffff77b0179 in backtrace_pcinfo (state=0x7ffff7fc1000, pc=140737297503162, callback=0x7ffff7798e60 <std::sys_common::gnu::libbacktrace::print::pcinfo_cb>,
error_callback=0x7ffff7798e40 <std::sys_common::gnu::libbacktrace::print::error_cb>, data=0x7fffee1fb4f0) at /buildslave/rust-buildbot/slave/nightly-dist-rustc-linux/build/src/libbacktrace/fileline.c:176
#16 0x00007ffff7799808 in std::sys_common::gnu::libbacktrace::print () at /buildslave/rust-buildbot/slave/nightly-dist-rustc-linux/build/obj/../src/libstd/sys_common/gnu/libbacktrace.rs:166
#17 std::sys::imp::backtrace::tracing::imp::write::trace_fn () at /buildslave/rust-buildbot/slave/nightly-dist-rustc-linux/build/obj/../src/libstd/sys/unix/backtrace/tracing/gcc_s.rs:101
#18 0x00007fffef544679 in _Unwind_Backtrace (trace=0x7ffff7799220 <std::sys::imp::backtrace::tracing::imp::write::trace_fn>, trace_argument=0x7fffee1fb8b0) at /build/gcc-multilib/src/gcc/libgcc/unwind.inc:295
#19 0x00007ffff779917b in std::sys::imp::backtrace::tracing::imp::write () at /buildslave/rust-buildbot/slave/nightly-dist-rustc-linux/build/obj/../src/libstd/sys/unix/backtrace/tracing/gcc_s.rs:42
#20 0x00007ffff77a8020 in std::panicking::default_hook::{{closure}} () at /buildslave/rust-buildbot/slave/nightly-dist-rustc-linux/build/obj/../src/libstd/panicking.rs:247
#21 0x00007ffff77a7bbe in std::panicking::default_hook () at /buildslave/rust-buildbot/slave/nightly-dist-rustc-linux/build/obj/../src/libstd/panicking.rs:257
#22 0x00007ffff77a84c8 in std::panicking::rust_panic_with_hook () at /buildslave/rust-buildbot/slave/nightly-dist-rustc-linux/build/obj/../src/libstd/panicking.rs:451
#23 0x00007ffff49fd3bb in std::panicking::begin_panic<rustc_errors::FatalError> () at /buildslave/rust-buildbot/slave/nightly-dist-rustc-linux/build/obj/../src/libstd/panicking.rs:413
#24 0x00007ffff4b8acac in rustc::session::Session::fatal () at /buildslave/rust-buildbot/slave/nightly-dist-rustc-linux/build/obj/../src/librustc/session/mod.rs:191
#25 0x00007ffff7a88f94 in rustc_driver::run::{{closure}}<closure> () at /buildslave/rust-buildbot/slave/nightly-dist-rustc-linux/build/obj/../src/librustc_driver/lib.rs:142
jonhoo
on 19 Nov 2016
With RUST_BACKTRACE=1 (for a rustc compiled with debuginfo = true), it currently takes, for me, 2min 20sec with 100% CPU usage (one full core(@2.4Ghz fixed)), out of 4 cores) to continue from a panic:
$ time rustc -vv
!! LD_LIBRARY_PATH=/home/xftroxgpx/build/2nonpkgs/rust.stuff/rust/rust/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib:/home/xftroxgpx/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib:/home/xftroxgpx/build/2nonpkgs/rust.stuff/rust/rust/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib:/home/xftroxgpx/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib:
!! Executing '/home/xftroxgpx/build/2nonpkgs/rust.stuff/rust/rust/build//x86_64-unknown-linux-gnu/stage2/bin//rustc' with args: '-vv'
error: Option 'verbose' given more than once.
real 2m20.208s
user 2m19.662s
sys 0m0.441s
That's 3115 times more time than it takes when RUST_BACKTRACE=0 :) (2m20.208s vs 0m0.045s)
rustc version used
$ rustc -vV
!! LD_LIBRARY_PATH=/home/xftroxgpx/build/2nonpkgs/rust.stuff/rust/rust/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib:/home/xftroxgpx/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib:/home/xftroxgpx/build/2nonpkgs/rust.stuff/rust/rust/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib:/home/xftroxgpx/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib:
!! Executing '/home/xftroxgpx/build/2nonpkgs/rust.stuff/rust/rust/build//x86_64-unknown-linux-gnu/stage2/bin//rustc' with args: '-vV'
rustc 1.21.0-dev (7f397bdb0 2017-08-19)
binary: rustc
commit-hash: 7f397bdb062fe13a4707219a2f32486c5294f642
commit-date: 2017-08-19
host: x86_64-unknown-linux-gnu
release: 1.21.0-dev
LLVM version: 4.0
config.toml used to compile rustc
[llvm]
optimize = true
release-debuginfo = false
assertions = false
ccache = "/usr/bin/ccache"
static-libstdcpp = false
ninja = true
targets = "X86"
link-jobs = 4
[build]
build = "x86_64-unknown-linux-gnu"
host = ["x86_64-unknown-linux-gnu"]
target = ["x86_64-unknown-linux-gnu"]
cargo = "/home/xftroxgpx/.cargo/bin/cargo"
rustc = "/home/xftroxgpx/.cargo/bin/rustc"
docs = true
compiler-docs = true
submodules = true
locked-deps = false
full-bootstrap = false
extended = false
verbose = 0
sanitizers = false
profiler = false
openssl-static = false
low-priority = true
[install]
[rust]
optimize = true
codegen-units = 1 #We currently have the capability to do multiple codegen units in parallel. Unfortunately, one drawback of using this functionality is that using multiple codegen units loses optimization opportunities, like inlining, between the units. src: https://internals.rust-lang.org/t/towards-a-second-edition-of-the-compiler/5582
debug-assertions = false
debuginfo = true
debuginfo-lines = true
debuginfo-only-std = false
debug-jemalloc = false
backtrace = true
channel = "dev"
rpath = true
optimize-tests = false
debuginfo-tests = false
codegen-tests = false
ignore-git = false
[target.x86_64-unknown-linux-gnu]
[dist]
src-tarball = false
Rust nightly (via rustup) takes only under 1 second:
$ time ~/.cargo/bin/rustc -vv
error: Option 'verbose' given more than once.
real 0m1.161s
user 0m0.773s
sys 0m0.082s
$ time ~/.cargo/bin/rustc -vv
error: Option 'verbose' given more than once.
real 0m0.814s
user 0m0.775s
sys 0m0.040s
$ time ~/.cargo/bin/rustc -vv
error: Option 'verbose' given more than once.
real 0m0.790s
user 0m0.754s
sys 0m0.038s
time with RUST_BACKTRACE=0
$ time RUST_BACKTRACE=0 rustc -vv
!! LD_LIBRARY_PATH=/home/xftroxgpx/build/2nonpkgs/rust.stuff/rust/rust/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib:/home/xftroxgpx/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib:/home/xftroxgpx/build/2nonpkgs/rust.stuff/rust/rust/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib:/home/xftroxgpx/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib:
!! Executing '/home/xftroxgpx/build/2nonpkgs/rust.stuff/rust/rust/build//x86_64-unknown-linux-gnu/stage2/bin//rustc' with args: '-vv'
error: Option 'verbose' given more than once.
real 0m0.045s
user 0m0.033s
sys 0m0.012s
$ time RUST_BACKTRACE=0 rustc -vv
!! LD_LIBRARY_PATH=/home/xftroxgpx/build/2nonpkgs/rust.stuff/rust/rust/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib:/home/xftroxgpx/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib:/home/xftroxgpx/build/2nonpkgs/rust.stuff/rust/rust/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib:/home/xftroxgpx/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib:
!! Executing '/home/xftroxgpx/build/2nonpkgs/rust.stuff/rust/rust/build//x86_64-unknown-linux-gnu/stage2/bin//rustc' with args: '-vv'
error: Option 'verbose' given more than once.
real 0m0.046s
user 0m0.035s
sys 0m0.011s
$ time RUST_BACKTRACE=0 ~/.cargo/bin/rustc -vv
error: Option 'verbose' given more than once.
real 0m0.146s
user 0m0.131s
sys 0m0.017s
$ time RUST_BACKTRACE=0 ~/.cargo/bin/rustc -vv
error: Option 'verbose' given more than once.
real 0m0.138s
user 0m0.120s
sys 0m0.020s
(sidenote: looks like nightly is "significantly" slower.)
ghost
on 20 Aug 2017
Was this page helpful?
0 / 5 - 0 ratings
Related issues
nikomatsakis
路
210Comments
cramertj
路
512Comments
Centril
路
382Comments
nikomatsakis
路
236Comments
nikomatsakis
路
340Comments
Most helpful comment
Hypothesis confirmed. The linked list of freed chunks of memory available for reuse accumulates a huge number of chunks that are too small to fulfill most allocation requests, so some requests have to check tens of thousands of chunks for sufficient space before giving up and
mmaping more memory.As an experiment, I modified the allocator to only check the first (most recently added) 10 chunks on the list. This dropped the runtime to a similar value to my previous
malloctests (a bit over a second). After 100,000 allocations the modified version has a resident memory size of 67.5 MB, as opposed to the original version which uses 66.6 MB, so while this does decrease them memory efficiency of the allocator the effect is not large. I'm not sure if this is the correct path to fixing the problem; it may make more sense to add logic to the memory freeing code to drop small chunks (but the definition of "small" is not obvious).I think I'll file an upstream bug with gcc (which appears to be the main repository for libbacktrace) to see if they have any ideas.