Delve: Some externally linked binaries have .eh_frame section instead of .debug_frame

[eungjun-yi]

1. What version of Delve are you using (dlv version)?

   Delve Debugger
   Version: 0.11.0-alpha
   Build:

2. What version of Go are you using? (go version)?

   go version go1.6 linux/amd64

3. What operating system and processor architecture are you using?

   OS: Linux 4.4.5-1-ARCH
   processor arch: x86_64

4. What did you do?

   I ran dlv attach 7741 to debug a running docker daemon process.

5. What did you expect to see?

   debug console of delve

6. What did you see instead?

   "could not find .debug_frame section in binary" error

[eungjun-yi]

Do I need to recompile the binary to be debugged by delve instead of go compiler?

[aarzilli]

What's the output of ls -l /proc/7741/exe (where 7741 is the pid you are trying to attach to)?

Do I need to recompile the binary to be debugged by delve instead of go compiler?

not necessary.

[eungjun-yi]

@aarzilli

lrwxrwxrwx 1 nori users 0 3월 21 18:06 /proc/7741/exe -> /usr/bin/docker

[aarzilli]

It seems that the docker binary has a .eh_frame section instead of .debug_frame.

More information:

http://www.airs.com/blog/archives/460
https://refspecs.linuxfoundation.org/LSB_3.0.0/LSB-PDA/LSB-PDA/ehframechpt.html

[eungjun-yi]

workaround: Build docker binary by DOCKER_DEBUG=1 make binary. It adds "-ldflags -w" option to go compiler so that the docker binary has .debug_frame.

[dotNetDR]

@npcode
I got the same problem(dlv exec docker ... -> could not find .debug_frame section in binary)
can i see your go build docker args?

my go build

go build -v -o bundles/1.8.3/binary/docker-1.8.3  -a -tags 'netgo static_build apparmor selinux daemon btrfs_noversion libdm_no_deferred_remove' -installsuffix netgo -ldflags '
                -w

        -linkmode external
        -extldflags "-static -lpthread -Wl,--unresolved-symbols=ignore-in-object-files"

        ' ./docker

[dotNetDR]

1. What version of Delve are you using (dlv version)?

   Delve version: 0.10.0-alpha

2. What version of Go are you using? (go version)?

   go version go1.4.2 linux/amd64

3. What operating system and processor architecture are you using?

   OS: CentOS Linux release 7.1.1503 (Core)
   Kernel: 3.10.0-229.el7.x86_64

4. What did you do?

   [root@debugdocker docker]# dlv exec /usr/bin/docker --log -- daemon -H tcp://0.0.0.0:2375 -H unix:///var/run/docker.sock
   2016/03/25 09:51:50 debugger.go:56: launching process with args: [/usr/bin/docker daemon -H tcp://0.0.0.0:2375 -H unix:///var/run/docker.sock]
   could not find .debug_frame section in binary
   [root@debugdocker docker]# WARN[0000] /!\ DON'T BIND ON ANY IP ADDRESS WITHOUT setting -tlsverify IF YOU DON'T KNOW WHAT YOU'RE DOING /!\
   INFO[0000] Listening for HTTP on tcp (0.0.0.0:2375)
   INFO[0000] Listening for HTTP on unix (/var/run/docker.sock)
   WARN[0000] Udev sync is not supported. This will lead to unexpected behavior, data loss and errors. For more information, see https://docs.docker.com/reference/commandline/cli/#daemon-storage-driver-option
   INFO[0000] [graphdriver] using prior storage driver "devicemapper"
   INFO[0000] Option DefaultDriver: bridge
   INFO[0000] Option DefaultNetwork: bridge
   WARN[0000] Running modprobe bridge nf_nat br_netfilter failed with message: modprobe: WARNING: Module br_netfilter not found.
   , error: exit status 1
   INFO[0000] Firewalld running: false
   INFO[0000] Loading containers: start.
   
   INFO[0000] Loading containers: done.
   INFO[0000] Daemon has completed initialization
   INFO[0000] Docker daemon                                 commit=2ecfbd7-dirty execdriver=native-0.2 graphdriver=devicemapper version=1.8.3
   

5. What did you expect to see?

   enter dlv debug mode console.

6. What did you see instead?

   could not find .debug_frame section in binary

[aarzilli]

@dotNetDR -ldflags -w disables DWARF generation, I'm not surprised we can't find .debug_frame, not much we can do about that (the binary would also lack .debug_info anyway). I bet -lptrace is responsible for the placement of g struct in FS.

[eungjun-yi]

@dotNetDR Sorry, I don't know how do I get the arguments used while running go build. I just ran DOCKER_DEBUG=1 make binary to compile the docker binary which contains .debug_frame.

[dotNetDR]

@aarzilli
i remove -ldflags -w args, it's working.

thank you for your help.

[aarzilli]

@dotNetDR you should be running into the same problem that @npcode had in issue #468. Did you not?

[dotNetDR]

@aarzilli

my docker version is 1.8.3
Does not exist that code

ipam, err := n.getController().getIPAM(n.ipamType)

(dlv) ls
> github.com/docker/libnetwork.(*network).CreateEndpoint() /go/src/github.com/docker/docker/vendor/src/github.com/docker/libnetwork/network.go:299
   294:     return nil
   295: }
   296: 
   297: func (n *network) CreateEndpoint(name string, options ...EndpointOption) (Endpoint, error) {
   298:     var err error
=> 299:     if !config.IsValidName(name) {
   300:         return nil, ErrInvalidName(name)
   301:     }
   302: 
   303:     if _, err = n.EndpointByName(name); err == nil {
   304:         return nil, types.ForbiddenErrorf("service endpoint with name %s already exists", name)
(dlv) n
2016/03/28 15:54:35 debugger.go:246: nexting
Command failed: Breakpoint exists at /go/src/github.com/docker/docker/vendor/src/github.com/godbus/dbus/conn.go:370 at cdd2fd
(dlv) bp
Breakpoint 1 at 0x958eb5 /go/src/github.com/docker/docker/vendor/src/github.com/docker/libnetwork/network.go:298
Breakpoint 2 at 0x958ecd /go/src/github.com/docker/docker/vendor/src/github.com/docker/libnetwork/network.go:299
Breakpoint 3 at 0x958f93 /go/src/github.com/docker/docker/vendor/src/github.com/docker/libnetwork/network.go:303

go build:

go build -v -o bundles/1.8.3/binary/docker-1.8.3  -a -tags 'netgo static_build apparmor selinux daemon btrfs_noversion libdm_no_deferred_remove' -installsuffix netgo -ldflags '

        -linkmode external
        -extldflags "-static -lpthread -Wl,--unresolved-symbols=ignore-in-object-files"

        ' ./docker

I do not understand go build compilation parameters(-linkmode, -extldflags, -tags, -installsuffix)
update:
go build compilation parameters
Linking golang statically

[dotNetDR]

@npcode
at /hack/make/binary

you can print it on build time. (e.g. binary)