Bazel: Debugging C++ on MacOS does not work

[Teivaz]

Description of the problem / feature request:

Debugging C++ on MacOS does not work. The built binary file contains debug hints which point to the location in the sandbox (which is removed after compilation).

Feature requests: what underlying problem are you trying to solve with this feature?

Replace this line with your answer.

Bugs: what's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

BUILD file

cc_binary(
    name = "sample",
    srcs = ["main.cpp"],
    visibility = ["//main:__pkg__"],
)

main.cpp file

#include <iostream>

int main() {
    std::cout << "tests" << std::endl;
    return 0;
}

Build with bazel build --compilation_mode=dbg sample
And debug

lldb bazel-bin/sample
(lldb) breakpoint set -n main
Breakpoint 1: 13 locations.
(lldb) r
Process 24391 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = breakpoint 1.1
    frame #0: 0x00000001000021b0 sample`main
sample`main:
->  0x1000021b0 <+0>: pushq  %rbp
    0x1000021b1 <+1>: movq   %rsp, %rbp
    0x1000021b4 <+4>: subq   $0x20, %rsp
    0x1000021b8 <+8>: movq   0xe51(%rip), %rdi         ; (void *)0x00007fff9c93a660: std::__1::cout
Target 0: (sample) stopped.

Debug hints in the executable:

nm -pa bazel-bin/sample | grep OSO
000000005bb7c96b - 03 0001   OSO /private/var/tmp/_bazel_teivaz/1e731ee5ae5a3ce6177976984318ec76/bazel-sandbox/1688134534644271312/execroot/__main__/bazel-out/darwin-dbg/bin/_objs/sample/main.o

While file built with clang++ -g main.cpp -o sample can be debugged normally:

lldb sample
(lldb) breakpoint set -n main
Breakpoint 1: where = sample`main + 29 at main.cpp:4, address = 0x0000000100000f9d
(lldb) r
Process 24410 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = breakpoint 1.1
    frame #0: 0x0000000100000f9d sample`main at main.cpp:4
   1    #include <iostream>
   2    
   3    int main() {
-> 4        std::cout << "tests" << std::endl;
   5        return 0;
   6    }
Target 0: (sample) stopped.

What operating system are you running Bazel on?

MacOS High Sierra 10.13.2

What's the output of bazel info release?

release 0.17.2-homebrew

Have you found anything relevant by searching the web?

I have found related issue 5545 but the proposed solution was already merged to the version of Bazel I am using.

Any other information, logs, or outputs that you want to share?

I have also asked a question on Stackoverflow

[steeve]

I think we can add https://github.com/bazelbuild/bazel/issues/1000 to the list of related issues

[steeve]

And https://github.com/bazelbuild/bazel/issues/5031

[kastiglione]

As a potential temporary work around, does --sandbox_debug (which leaves the sandbox directory intact) get around the problem?

[Teivaz]

@kastiglione Yes it is another workaround though I would like not to use it. I have found another workaround - disabling sandbox with --spawn_strategy=standalone. But none of them _solves_ the issue.

[hlopko]

Yup this never worked, and I don't think we'll have time to look into this in 2018. Bringing apple expertise @sergiocampama just in case the fix is simpler than it sounds :).

[sergiocampama]

I would defer to the C++ language team, I don't have much experience with debugging C++ apps.

[jwnimmer-tri]

Also related to #2537 perhaps.

[George35mk]

I have the same problem on Windows 10, I get

No symbol table is loaded. error message in the console when I try to debug

https://stackoverflow.com/questions/53840959/how-to-build-and-debug-a-c-executable-using-bazel-on-windows-10-x64

[steeve]

For the record, it's working for apple targets via --apple_generate_dsym flag. Perhaps cc_binary could consume it too ?

[slonka]

@steeve for me --apple_generate_dsym does not work. I'm trying to debug https://github.com/envoyproxy/envoy. I have the version of the bazyl clion extension linked here: https://github.com/bazelbuild/intellij/issues/494#issuecomment-498830916 .

My setup is:
CLion 2019.1.3 Build #CL-191.7141.37, built on May 6, 2019
macOS 10.14.4
Build label: 0.26.0-homebrew
Build target: bazel-out/darwin-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Tue May 28 14:16:39 2019 (1559052999)
Build timestamp: 1559052999
Build timestamp as int: 1559052999

I'm putting a breakpoint the log line you see in the output and it does not catch it.

My run configuration:

When I pause the debugger it seems to know where it's paused (it can read the names of the functions, for example: Envoy::MainCommonBase::run() 0x000000010081e52e, but it does not seem to be able to map it in the editor - it jumps to disassembly)

Can you help with that or show me what to do so I can fix it myself?

[steeve]

@slonka can you try to disable the sandbox? --spawn_strategy=standalone
The reason is that paths in DWARF can reference the sandbox, not the actual path.
You can either disable sandboxing, or use a sourcemap.

Also, apple_generate_dsym works only for macos_application, but not cc_binary (which was my comment).

As for CLion, I don't know how to make it work, but I managed to do it in VSCode.

[slonka]

@steeve I tried with "--spawn_strategy=standalone" and "--sandbox_debug" they did not work in clion. Do you think it's an issue for bazel or the clion plugin?

[bofeiw]

Also experiencing this issue, I got this error when I debug the C++ program:

com.jetbrains.cidr.execution.debugger.backend.gdb.GDBDriver$GDBCommandException: "/private/var/tmp/_bazel_bofeiwang/856f12db26bfbe04ae52bb24ae3db2e3/execroot/__main__/bazel-out/darwin-dbg/bin/assignments/wl/main": not in executable format: File format not recognized

[depp]

While --spawn_strategy=standalone worked for me, there are some drawbacks to running all rules standalone. I added the following line to my bazelrc:

build --strategy_regexp=^Linking=local

This means that only the linking steps will be run without the sandbox.

[depp]

If changing the spawn strategy doesn’t work, try running bazel clean and then running bazel build --strategy_regexp=^Linking=local. Bazel will not rebuild targets just because the spawn strategy changes!

[TheButlah]

@depp using local for the linking strategy doesn't seem to work for me. I did the following to test my executable:

bazel clean
bazel run -c dbg @spybot//realsense/programs:server --run_under=lldb --strategy_regexp=^Linking=local

Now while in lldb:

break set -n main
run

This gives something like this:

Process 33280 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = breakpoint 1.1
    frame #0: 0x000000010000432b server`main(argc=1, argv=0x00007ffeefbff228) at server.cpp:60:15
Target 0: (server) stopped.

server.cpp is my file that contains the main() function. I wanted to see the location that the debug symbols claim this file is located at, so I did:

source info

This gives:

Lines found in module `server
[0x000000010000432b-0x0000000100004337): /private/var/tmp/_bazel_ryan.butler/14e00cfc95af169a4eb8b5156cec6517/sandbox/darwin-sandbox/369/execroot/saicny_robotics/external/spybot/realsense/programs/server.cpp:60:15

Note that the path is under the sandbox. And as feared, that file doesn't exist.

Also, it really would be good to get this fixed, its been a year and a half :(

[kjteske]

+1 - would love to see this fixed

[philwo]

I just tried to debug an issue in Bazel's own C++ code using lldb on my Mac and ... I couldn't get the line numbers or code to show up. 🤦

I'll try to find someone who can help fix this. Thanks everyone for documenting the workaround with disabling sandboxing!

Edit: Yay, it works with --spawn_strategy=local and my usual load of debug mode flags (-c dbg --javacopt="-g" --copt="-g" --strip="never") :)