When a Rust program run with RUST_BACKTRACE=1
panics it outputs backtrace which only has function names (with noisy hashes), but doesn't include file names and line numbers.
RUST_BACKTRACE=1 cargo run --verbose --example test
12: 0x1088db5d9 - main::hed092b4e13c9cca7Iaa
During development (not necessarily in release builds), I'd prefer something like:
12: 0x1088db5d9 - main() at test.rs:10
Sometimes line numbers are very important, e.g. if the panic is some generic out-of-bounds assertion, and the function uses vectors and slices in many places, just the name is not enough to pinpoint the offending expression.
This does work on some platforms for binaries compiled with debug-info (which cargo run
does by default, unless you've edited profiles in the Cargo.toml
); what are you on?
E.g. on x86-64 linux, I get:
fn foo() {
panic!()
}
fn bar() {
foo()
}
fn main() {
bar()
}
thread '<main>' panicked at 'explicit panic', panic.rs:2
stack backtrace:
1: 0x7fbdbdb2d5f9 - sys::backtrace::write::h6bf89e75ba594056cRC
2: 0x7fbdbdb308f9 - panicking::on_panic::hbf9e41a2aefa0bc6ghJ
3: 0x7fbdbdb2ad62 - rt::unwind::begin_unwind_inner::h41f8448b15c8a007oWI
4: 0x7fbdbdb2a16e - rt::unwind::begin_unwind::h17882462255754141092
at /home/rustbuild/src/rust-buildbot/slave/nightly-dist-rustc-linux/build/src/libstd/rt/unwind.rs:522
5: 0x7fbdbdb2a0ba - foo::h846a49565d377979eaa
at /home/huon/projects/test-rust/<std macros>:3
6: 0x7fbdbdb2a4bd - bar::h20543e655da0f511Eaa
at /home/huon/projects/test-rust/panic.rs:5
7: 0x7fbdbdb2a4ed - main::h32ccc23e635fe73eJaa
at /home/huon/projects/test-rust/panic.rs:8
8: 0x7fbdbdb34148 - rust_try_inner
9: 0x7fbdbdb34135 - rust_try
10: 0x7fbdbdb31f9b - rt::lang_start::hba3ed105db79dd3fJbJ
11: 0x7fbdbdb2a634 - main
12: 0x7fbdbcd3bb44 - __libc_start_main
13: 0x7fbdbdb29f58 - <unknown>
14: 0x0 - <unknown>
I'm using rustc 1.0.0-nightly (6436e348e 2015-04-08) (built 2015-04-08) on OS X 10.10.3.
rustc 1.2.0-nightly (0cc99f9cc 2015-05-17) (built 2015-05-17) in OS X 10.9.4 won't print line number.
But same version on Linux x86_64 can.
:+1: on this one, sorely needed!
Any news on this?
Looks like this has gotten even worse lately; type names seem to get mangled now. For instance I get this:
9: 0x10d924b37 - _<impl>::on_cycle::h0c7dcf14c18c4ee9Jpa
for a regular struct
with an impl
(not implementing any trait or anything).
The results right now in 1.5 are utterly useless:
> cargo run
Running `target\debug\day21bin.exe`
thread '<main>' panicked at 'called `Option::unwrap()` on a `None` value', ../src/libcore\option.rs:366
stack backtrace:
0: 0x4517a7 - <unknown>
1: 0x45915e - <unknown>
2: 0x415d1c - <unknown>
3: 0x41648b - <unknown>
4: 0x44e9c2 - <unknown>
5: 0x473cd8 - <unknown>
6: 0x46e130 - <unknown>
7: 0x410530 - <unknown>
8: 0x4107df - <unknown>
9: 0x410a9a - <unknown>
10: 0x410dd4 - <unknown>
11: 0x40153e - <unknown>
12: 0x4587e8 - <unknown>
13: 0x441da1 - <unknown>
14: 0x4586f7 - <unknown>
15: 0x402a0a - <unknown>
16: 0x4013b4 - <unknown>
17: 0x4014e7 - <unknown>
18: 0x7ffaf70a8101 - <unknown>
Process didn't exit successfully: `target\debug\day21bin.exe` (exit code: 101)
This is on Windows, but both Powershell and Cygwin64 produce identical results.
Can confirm on macos 10.10.5
$ rustc --version
rustc 1.5.0 (3d7cd77e4 2015-12-04)
$ rustc -g file_embed.rs
$ RUST_BACKTRACE=1 ./file_embed
thread '<main>' panicked at 'called `Result::unwrap()` on an `Err` value: Error { repr: Os { code: 9, message: "Bad file descriptor" } }', ../src/libcore/result.rs:738
stack backtrace:
1: 0x100ff9b88 - sys::backtrace::tracing::imp::write::h06df28c2a8d245f9t9s
2: 0x100ffb1ef - panicking::log_panic::_<closure>::closure.39451
3: 0x100ffac92 - panicking::log_panic::h9348b1b0af36ae6aCYw
4: 0x100ff52c6 - sys_common::unwind::begin_unwind_inner::h005d402245ebd8560cs
5: 0x100ff53fe - sys_common::unwind::begin_unwind_fmt::h440d2eb18c439fa86bs
6: 0x100ff9217 - rust_begin_unwind
7: 0x1010166b0 - panicking::panic_fmt::h4c8d12e3c05f3b8cZEK
8: 0x100ff3566 - result::_<impl>::unwrap::unwrap::h5984419141872088150
9: 0x100ff2a3d - write_cpp::hf673fd381a899287Gca
10: 0x100ff4340 - main::h35317f0a7a25286cpia
11: 0x100ffaa12 - sys_common::unwind::try::try_fn::h4733637608193038088
12: 0x100ff9058 - __rust_try
13: 0x100ffa8b9 - rt::lang_start::h44a8548d0ff91511KVw
14: 0x100ff4789 - main
cc @alexcrichton @lifthrasiir
From what I can tell you are the original authors of the line number in backtraces functionality (or at least familiar with the code) so I was wondering if one of you could chime in?
I'm most curios about whether the current situation is expected (ie. it is a known deficit of the current implementation) or if things should be working on os x (and thus there may be bug in the implementation)?
I can confirm on Linux, too, so it's not exclusive to mac.
rustc 1.7.0-nightly (0c7c94e8a 2015-12-26)
Correction: I do get line numbers on Linux. I no longer have the exact test-case I had before, but an attempt at reproducing it does show line numbers.
@TimNN OSX support for filename and line numbers is unimplemented - the dladdr-based implementation here calls to output
but not output_fileline
which includes that information.
Bump, also not working for me on OS X 10.11 with yesterday's nightly. The nightly on Gentoo Linux properly displays the line numbers however. Unfortunately I do most of my dev work on my Macbook so this makes debugging very difficult.
After some investigating this seems to be a limitation of dladdr. dladdr is, as far as I can tell, unable to retrieve the source filename or line number (the filename it gets is the name of the compiled binary, which is not what we're looking for). And, since OS X uses clang by default, GCC's libbacktrace is unavailable on that platform.
I'm not sure if there is an implementation similar to libbacktrace that could be used to accomplish this on OS X, but I think that would most likely need to be the approach taken here.
@TimNN sure, here's a summary of what I know:
-g
flag to the compiler). If no debug info is present they won't be printed. This information is somewhat resilient to inlining as well.@shssoichiro
Yes it's intentional there is no file/line information on OSX. If there's a library we could use, however, we could consider linking to it!
I have the same issue on Mac. My work-around is like that this:
lldb <executable>
breakpoint set -b rust_begin_unwind
run
to start the executable. bt
to see a backtrace with filenamesNot as nice as to see the filenames directly but better than nothing.
So what is the plan with this? Is there any ETA?
@korczis Someone who feels sufficiently motivated will have to track down the appropriate libraries/APIs on Windows, OSX, BSD, etc and integrate them.
As @emoon saying, lldb can do it. Thanks. https://github.com/rust-lang/rfcs/pull/1669 is relatived.
So I think that at this point this bug is basically about OSX. I've seen really good backtrace line numbers on MSVC and Linux, so OSX is the only major desktop platform at least left that needs this.
@sujayakar has done some awesome work and discovered that the atos
tool shipped on OSX actually already can do the conversion from address to filename/line number, we just need to call it! Unfortunately we probably can't shell out during a backtrace in the standard library, so we'll probably need to figure out how to do the parsing ourselves to figure out this kind of information.
The atos
tool may be closed source, but Facebook apparently has an implementation for ARM which may be a great place to start with this perhaps? I'd also be quite interested in adding support immediately to the backtrace crate which shells out to a tool, as it could easily go behind a feature flag or something like that.
This might be interesting to look at also https://github.com/google/breakpad/tree/master/src/common/mac
Now I'm not sure if that supports file/line but I think it does so it may be good to look at as a reference also.
@alexcrichton This issue is present on windows-gnu as well.
@alexcrichton typically instead of using atos one can also use llvm-symbolizer which is open source.
Source to reference for the symbolizer: https://github.com/llvm-mirror/llvm/blob/master/tools/llvm-symbolizer/llvm-symbolizer.cpp
Just a quick brain dump: I discovered that /System/Library/PrivateFrameworks/CoreSymbolication.framework/Versions/A/CoreSymbolication
exists on OS X. It's a private framework but there are some reverse engineered headers: https://github.com/mountainstorm/CoreSymbolication
Seems reasonably straightforward: https://github.com/mitsuhiko/CoreSymbolication/blob/master/CoreSymbolication_test/CoreSymbolication/CoreSymbolication.m
Main issue is that this thing is clearly not intentionally supported by Apple (which is a shame, the API looks very friendly) so this would have to be used with caution.
Implemented against backtrace-rs
: https://github.com/alexcrichton/backtrace-rs/pull/15
@emoon oh nice, thanks for the link!
@kdar ah yeah good point, although we're trying to shift more towards MSVC nowadays so the problem may be lessening there.
@mitsuhiko awesome! I wonder if llvm-symbolizer
could be extracted to a small C interface, or if it's a little too entangled in LLVM's runtime right now?
@alexcrichton I looked more into that now and I the amount of stuff that pulls in is insane. Like in the many megabytes. I think given that CoreSymbolication is a thing we might not want to pursue this unless there are no options.
I looked a bit more into CoreSymbolizer now and it looks like Mozilla is using it somewhere in Firefox. Maybe someone could figure out what the stability of that thing is through them :)
What makes you think that? I don't see any references to it in the source
code: http://searchfox.org/mozilla-central/search?q=coresymbollizer&path=
Afaik, the SPS profiler uses hand written stack walking code and if you're
using the gecko profiler add-on to get C++ names, it is opening a shell and
calling atos, otherwise you end up with just js names.
For the crash reports, those use Google breakpad to unwind and
symbolication happens offline, once the crash report is submitted.
On Aug 16, 2016 12:56 AM, "Armin Ronacher" [email protected] wrote:
I looked a bit more into CoreSymbolizer now and it looks like Mozilla is
using it somewhere in Firefox. Maybe someone could figure out what the
stability of that thing is through them :)—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/rust-lang/rust/issues/24346#issuecomment-240029701,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAEjS1oLab_qke_Au9VGnvKU2t-STCxlks5qgW02gaJpZM4D-4iX
.
Oh interesting:
http://searchfox.org/mozilla-central/source/widget/cocoa/nsCocoaDebugUtils.h#11
On Aug 16, 2016 7:57 AM, "Nick Fitzgerald" [email protected] wrote:
What makes you think that? I don't see any references to it in the source
code: http://searchfox.org/mozilla-central/search?q=coresymbollizer&path=Afaik, the SPS profiler uses hand written stack walking code and if you're
using the gecko profiler add-on to get C++ names, it is opening a shell and
calling atos, otherwise you end up with just js names.For the crash reports, those use Google breakpad to unwind and
symbolication happens offline, once the crash report is submitted.On Aug 16, 2016 12:56 AM, "Armin Ronacher" [email protected]
wrote:I looked a bit more into CoreSymbolizer now and it looks like Mozilla is
using it somewhere in Firefox. Maybe someone could figure out what the
stability of that thing is through them :)—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/rust-lang/rust/issues/24346#issuecomment-240029701,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAEjS1oLab_qke_Au9VGnvKU2t-STCxlks5qgW02gaJpZM4D-4iX
.
What is the status on this? I see that @mitsuhiko's pull request to backtrace-rs was merged. When can one expect some improvement?
@alexbool unlikely that this would land in rust before the new cargo based build process lands.
So what's the status of this issue? Is there any plan for displaying line number on backtrace for OS X?
I think this is mostly blocked on rust not being able to depend on backtrace-rs yet.
Out of curiosity, what does rust depend on currently (if not backtrace-rs) to get line numbers?
For those in the thread running into this issue, there is this workaround:
RUST_BACKTRACE=1 rust-lldb cargo -- test
break set --name rust_panic
run
bt
By running under lldb, you'll get the symbolication from lldb's backtrace.
@nipunn1313 the only way to symbolicate on osx is the CoreSymbolication framework. rust core does not have an implementation for that on OS X so it does not use anything.
@nipunn1313 right now the way line numbers work is by parsing encoded DWARF information in the executable that's running. That is, line numbers require compiling with -g
. Unfortunately the library we're using, libbacktrace, doesn't have support for mach-o files (the executable file format on OSX). As a result, by default, there's no line numbers on OSX.
The strategy taken in backtrace-rs is to use an unstable API from OSX which isn't documented and uses private headers if you compile it from C. I at least wasn't quite comfortable adding that to the standard library yet due to the instability.
Is this at
all helpful https://github.com/gimli-rs/addr2line?
I know this is not the highest pressing issue on the list but I really would love to see this fixed. How much work would it be to switch this over to backtrace-rs
? It's very frustrating to debug test cases on osx currently.
I would be wary of integrating backtrace-rs in the standard library due to the usage of private and not necessarily stable APIs on OSX (or at least, so I am told about CoreSymbolication). My personal hope for integration here is to actually get gimli to the point where we can integrate it into the standard library which would provide quite robust parsing of the line tables and whatnot.
Note that you can swap out the default implementation with your own based off of the crates.io backtrace crate with the std::panic::set_hook
function.
Even though core symbolication is a private framework i still think that it's the best of all options because it will also do the right thing for loaded dylibs. Is there a particular thing you're worried about here?
My specific worry is that we'd be embedding basically all Rust executables with the logic of loading core symbolilcation. If core symbolication then changes an API (e.g. changes the ABI of one of the functions we use) it'd cause all Rust executables to start segfaulting on OSX by default basically. With such a large impact (all Rust executables) I'd prefer to ensure that we'll continue working into the future
But this only ever gets executed if Rust is already crashing. So realistically even if this were to break in the future it only breaks if stuff already breaks.
Uh, a SIGSEGV is very different than a panic in a long running process.
Well. I'm assuming that any panic outside of testsuites better shut down the process anyways. That said if you do thinkt his is no path forward I wonder what is. Because OSX does not guarantee much about debug symbols either so if one does not use coresymbolication one needs to keep updating a custom implementation with all apple changes as they come in.
embedding basically all Rust executables
Can this be done only in debug Rust executables?
I would even prefer it that way, because line numbers are inaccurate in optimized builds anyway, and for distribution I'd rather have smaller executables.
@mitsuhiko I'd consider any SIGSEGV from the standard library as unacceptable, requiring an immediate fix. I could also imagine that some changes in ABI could even lead to exploitations perhaps. In that sense I'd just want to shield us as much as possible.
If changes in format periodically broke line numbers (e.g. they're just not displayed) then that's totally fine by me at least. We can always fix that promptly, and it'd be a non critical thing to fix.
I'm the author of atosl, so I can speak a bit on that front. It does it's own Mach-O parsing, so there's no dependency on that part. For the DWARF parsing it uses libdwarf, but I'm not so happy with that library. I see that someone already a port of an alternative dwarf library, gimli, to Rust, at https://github.com/gimli-rs/gimli. And it uses the 'object' crate which already implements objective-c support. But I'm not sure what the dependencies are on that, and if anyone would want to pull them in.
I'll try to take a look at this in a bit more detail if I get a chance.
Taking a look at libbacktrace, it looks like it already does the dwarf parsing, so likely it just needs the Mach-O loader support. After looking at elf.c and pecoff.c, I think it'd be relatively easy to add Mach-O support.
The slightly tricky bit is that the dwarf symbols are often in in a separate .dSYM file. CoreSymbolication knows how to find them, I believe by using spotlight (you can do a similar lookup by using mdfind). But I think in the case of Rust it's probably good enough to just look for the .dSYM directory in the same location as the main binary, at least for a start.
@zlandau2 one thing to consider here:
But I think in the case of Rust it's probably good enough to just look for the .dSYM directory in the same location as the main binary, at least for a start.
For a start probably but definitely not in general. The reason for this is that sooner or later Rust programs will start linking against things (at least during development) that are third party libraries which are not installed through cargo and then you need to find the debug symbols.
We could at least symbolicate the function name, but yeah we wouldn't have line number information unless that third party library also used the same structure of having
There are already function names on OS X.
Would it be possible maybe to add a simplified hook to rust in the meantime so one plug backtrace-rs
in? I tried to build a panic handler that uses it's quite hard to replace the default panic handler since it uses some internals that are neither documented nor exposed.
@mitsuhiko Take https://github.com/sfackler/rust-log-panics/blob/master/src/lib.rs#L67 and replace error!
with println!
.
@mitsuhiko
Fortunately when a dSYM is generated the executable and the dSYM are given matching UUIDs, so simply scanning the directory containing the executable can be made robust against extra dSYM files. I'm working on adding Mach-O support to libbacktrace, hopefully I'll be done by the end of the weekend.
Any luck? Rust is really shaping up but no line numbers in stacktraces on OSX is a ball dropped.
I've submitted a pull request (https://github.com/ianlancetaylor/libbacktrace/pull/2) adding Mach-O support to libbacktrace. If/when that's accepted it can be added to Rust.
@JohnColanduoni holy cow that's awesome!
I think we'd definitely be up to merge that as soon as it's ready!
Also if you're feeling inspired you could try upstreaming a local soundness fix as well :)
upstreaming a local soundness fix
And this one as well https://github.com/rust-lang/rust/commit/55e2b7e1b4606ae0bc684293f011b7006d1f1258#diff-29b9b4427b4e6fd7450b277c2bc4fed5
libbacktrace accepting GitHub pull requests is something new (?)
I believe they required sending patches through GCC mailing lists previously.
This discussion has mostly been about MacOSX, but does anyone know where to start looking on FreeBSD (and probably other BSDs)? I use FreeBSD for my production servers, and it would be very nice to have this working on that platform.
I'll poke around, but it would be nice if someone could give mea hint as to where to start.
@beatgammit
IIRC the issue with the BSDs is that there are some memory safety concerns about how libbacktrace handles malformed DWARF data, and the ability for an attacker to manipulate current_exe on those platforms (see here). libbacktrace supports ELF already, so it's not an issue on that side of things.
Line numbers are important to my development process. If I didn't have a linux box, I'd be dead in the water with Rust right now (particularly given my use of closures).
I'm confused as to why this is labeled I-papercut. Is anyone here still using OS X, and if so, what tools are you using to mitigate this? How do you debug? I must be missing something.
I use a debugger to debug. The third party backtrace crate can produce line numbers on OSX, though it seems to be a bit unreliable on whether that actually ends up working.
Specifically, break on rust_panic
.
Also, prefer expect("useful message here")
to unwrap()
(although that is just one specific kind of panic).
That's not to say that this isn't a large drain on productivity.
Thanks! I stopped trying to make lldb work with Rust back in October... but
lldb stack traces via b rust_panic
and bt
are working again :)
I'm shipping rust binaries to win/max/linux, so bug reports need good stack traces. Do you mean I shoul use backtrace-rs from std::panic::set_hook
?
Yep - here's a hook that should be equivalent to the standard library's format-wise for reference: https://github.com/sfackler/rust-log-panics/blob/master/src/lib.rs#L51
@sfackler backtrace-rs (after a few seconds) adds line numbers for the outer crate, but can't seem to resolve line numbers for other (local, debug) crates where most frames are at. Using OS X, nightly-2017-02-04, backtrace 0.3.0.
To find the dSYM files you can use the DebugSymbols framework. That's what gdb/lldb use. You can find some information about using it here: https://github.com/jrmuizel/dsym-find/blob/master/dsym-find.c
Is it necessary that we wait for upstream acceptance of libbacktrace patches? OS X has been waiting a while to get good runtime errors, and the workarounds are unreliable.
@jrmuizel DebugSymbols is a private framework. At that point, we might as well use the private CoreSymbolication framework which will handle the whole process.
@nathanaeljones
I've fixed the issues for that pull request. I'd be happy to put together one for the Rust codebase (should be pretty minimal, there's no API changes) if that's acceptable.
@JohnColanduoni I think such a PR would be great.
I'd also love to see this be upgraded from "papercut". I ship server software that other people run; backtraces are crucial.
I'd also love to see this be upgraded from "papercut".
+1
Another workaround I'm currently using is the trace-error crate. There are caveats though:
?
operator)Triage: there hasn't been any comments here in a long time, but there has been a ton of changes in the past 18 months. Is this still an issue?
I've haven't been hit by this since a long time on two macos machines.
-> cat src/main.rs
fn main() {
panic!("Hello, world!");
}
-> cargo run
Finished dev [unoptimized + debuginfo] target(s) in 0.21s
Running `target/debug/issue24346`
thread 'main' panicked at 'Hello, world!', src/main.rs:2:5
stack backtrace:
0: std::sys::unix::backtrace::tracing::imp::unwind_backtrace
at libstd/sys/unix/backtrace/tracing/gcc_s.rs:49
1: std::sys_common::backtrace::print
at libstd/sys_common/backtrace.rs:71
at libstd/sys_common/backtrace.rs:59
2: std::panicking::default_hook::{{closure}}
at libstd/panicking.rs:211
3: std::panicking::default_hook
at libstd/panicking.rs:227
4: <std::panicking::begin_panic::PanicPayload<A> as core::panic::BoxMeUp>::get
at libstd/panicking.rs:476
5: std::panicking::begin_panic
at libstd/panicking.rs:410
6: issue24346::main
at src/main.rs:2
7: std::rt::lang_start::{{closure}}
at libstd/rt.rs:74
8: std::panicking::try::do_call
at libstd/rt.rs:59
at libstd/panicking.rs:310
9: macho_symbol_search
at libpanic_unwind/lib.rs:102
10: std::alloc::default_alloc_error_hook
at libstd/panicking.rs:289
at libstd/panic.rs:392
at libstd/rt.rs:58
11: std::rt::lang_start
at libstd/rt.rs:74
12: issue24346::main
Yeah, I think this has been fixed for a while.
Backtraces still do not have line numbers for me (as of 1.36.0-nightly), on macOS 10.12.6.
fn main() {
panic!();
}
thread 'main' panicked at 'explicit panic', panic.rs:2:5
stack backtrace:
0: std::sys::unix::backtrace::tracing::imp::unwind_backtrace
1: std::sys_common::backtrace::_print
2: std::panicking::default_hook::{{closure}}
3: std::panicking::default_hook
4: std::panicking::rust_panic_with_hook
5: std::panicking::begin_panic
6: panic::main
7: std::rt::lang_start::{{closure}}
8: std::panicking::try::do_call
9: __rust_maybe_catch_panic
10: std::rt::lang_start_internal
11: std::rt::lang_start
12: main
@varkor Are you using --release
? Because the precise backtraces can work only if the debug symbols are present. In the debug mode, it works for me on rustc 1.36.0-nightly (6d599337f 2019-04-22) on macOS 10.14.
@kornelski: ah, I'm not using --release
, but I was not using -g
either. I get line numbers when I use -g
. For whatever reason I was assuming line numbers would be included by default. (Maybe adding a comment to the non-debug info message could be helpful here?) Thanks!
This is torture.
I don't get line numbers or filenames in tracebacks from my [lib] crate on Mac 10.14.6, Rust 1.42.0. I am building the crate with cargo build
, so as I understand things the debuginfo should be included. I am not modifying the profiles in Cargo.toml in any way, so cargo build
should use the dev profile, which has debug=true
by default, so debuginfo=2
is passed to rustc, which is the same as -g
mentioned above.
There are no .dSYM files in /target/debug/deps/
, but there are .dSYM files in /target/debug/build/[package-name]-[hash]/
.
With the same code-
I get debugging info on two separate Ubuntu machines
I lack debugging info on two separate Mac machines
Perhaps this issue should be reopened? Happy to provide more information to help track this down.
Most helpful comment
I have the same issue on Mac. My work-around is like that this:
lldb <executable>
breakpoint set -b rust_begin_unwind
run
to start the executable.bt
to see a backtrace with filenamesNot as nice as to see the filenames directly but better than nothing.