Bpftrace: Build broken with latest bcc

[pzakha]

bpftrace fails to build with the latest bcc bits with the following error:

[ 30%] Building CXX object src/ast/CMakeFiles/ast.dir/irbuilderbpf.cpp.o
/home/ubuntu/linux-pkg/packages/bpftrace/tmp/repo/src/ast/irbuilderbpf.cpp: In member function ‘llvm::CallInst* bpftrace::ast::IRBuilderBPF::CreateGetCurrentCgroupId()’:
/home/ubuntu/linux-pkg/packages/bpftrace/tmp/repo/src/ast/irbuilderbpf.cpp:422:16: error: ‘BPF_FUNC_get_current_cgroup_id’ was not declared in this scope
       getInt64(BPF_FUNC_get_current_cgroup_id),
                ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/ubuntu/linux-pkg/packages/bpftrace/tmp/repo/src/ast/irbuilderbpf.cpp:422:16: note: suggested alternative: ‘BPF_FUNC_get_current_comm’
       getInt64(BPF_FUNC_get_current_cgroup_id),
                ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
                BPF_FUNC_get_current_comm
src/ast/CMakeFiles/ast.dir/build.make:110: recipe for target 'src/ast/CMakeFiles/ast.dir/irbuilderbpf.cpp.o' failed

I've bisected the issue to this bcc commit: https://github.com/iovisor/bcc/commit/dc1254edbf85ad07c732fca3bb1b4c114a12d3d2.

[brendangregg]

Thanks; yes, bpftrace needs updating.

[danobi]

I think when bcc tags a new release, it'd be easier for us to ifdef things.

[jvnn]

Any suggested good workarounds for this? I had to build bcc myself due to #335 (btw, llvm 7 is already available in Ubuntu so I think using the default packages for that should be okay now) and ran into this issue instead. I'm still very unfamiliar with the whole BPF business, so I'm not sure if I can / should simply select an older tag from bcc and rebuild.

[tyroguru]

Any suggested good workarounds for this? I had to build bcc myself due to #335 (btw, llvm 7 is already available in Ubuntu so I think using the default packages for that should be okay now) and ran into this issue instead. I'm still very unfamiliar with the whole BPF business, so I'm not sure if I can / should simply select an older tag from bcc and rebuild.

@jvnn - Yeah, I have worked around this by branching off the "tag_v0.8.0" bcc branch. This really needs sorting out.

[brendangregg]

I thought we had a PR that fixed it? #374 not enough?

[tyroguru]

It doesn't appear to. I've been away from bpftrace for a month or two so needed to install a local libbcc in order to build. I pulled from the master bcc branch yesterday and bpftrace won't build with that (because of Yongong's changes for #2149 I presume - use kernel libbpf in bcc). I therefore had to go back to the last tagged build.

I don't think this is because of any extra hoops I have to jump through at FB but you never know.

[sdht0]

I tried building bcc+bpftrace on Archlinux. Both git master and v0.8.0 gives me this compilation error.

[brendangregg]

I took a look, it's because BPF_FUNC_get_current_cgroup_id was added in 4.18, and we're building on an older kernel.

If BPF_FUNC_get_current_cgroup_id is not defined, we need to print an error from semanticanalyzer. Plus we need to set it to zero or something so that the build succeeds.

[brendangregg]

So if I work past that error, I hit

/mnt/src/bpftrace/src/attached_probe.cpp: In member function ‘void bpftrace::AttachedProbe::attach_kprobe()’:
/mnt/src/bpftrace/src/attached_probe.cpp:375:58: error: too few arguments to function ‘void* bpf_attach_kprobe(int, bpf_probe_attach_type, const char*, const char*, pid_t, int, int, perf_reader_cb, void*)’
       eventname().c_str(), probe_.attach_point.c_str(), 0);
                                                          ^
In file included from /mnt/src/bpftrace/src/attached_probe.h:5:0,
                 from /mnt/src/bpftrace/src/attached_probe.cpp:13:
/usr/include/bcc/libbpf.h:73:8: note: declared here
 void * bpf_attach_kprobe(int progfd, enum bpf_probe_attach_type attach_type,
        ^~~~~~~~~~~~~~~~~
/mnt/src/bpftrace/src/attached_probe.cpp: In member function ‘void bpftrace::AttachedProbe::attach_uprobe()’:
/mnt/src/bpftrace/src/attached_probe.cpp:396:62: error: too few arguments to function ‘void* bpf_attach_uprobe(int, bpf_probe_attach_type, const char*, const char*, uint64_t, pid_t, int, int, perf_reader_cb, void*)’
       eventname().c_str(), probe_.path.c_str(), offset(), pid);

I think we need more of this to finish supporting both old and newer bcc: https://github.com/iovisor/bpftrace/pull/374/files

[sdht0]

I took a look, it's because BPF_FUNC_get_current_cgroup_id was added in 4.18, and we're building on an older kernel.

I am running Linux 4.20.6, but my build still fails with the same error.

[brendangregg]

Ok, I should have looked more closely. My problem is the libbpfcc-dev version, which is 0.5.0, and it doesn't include get_current_cgroup_id in /usr/include/bcc/compat/linux/bpf.h.

Version 0.8.0 does, and if I install it (from https://packages.debian.org/sid/libs/libbpfcc), the build almost works -- I had to create a symlink from libbcc.so to libbcc.so.0, not sure why, and then it works.

So I think the problem is: how do we get bpftrace to build with both libbpfcc{-dev} 0.5.0 and 0.8.0?

We could make 0.8.0 a requirement and for those with 0.5.0 require them to do a bcc install from source. Then we're not adding ifdefs and the problem fixes itself down the road.

[williangaspar]

I Solved the build problem doing this on /src/ast/CMakeLists.txt:

if(HAVE_BCC_CREATE_MAP)
  target_compile_definitions(ast PUBLIC BPF_FUNC_get_current_cgroup_id=0)
endif(HAVE_BCC_CREATE_MAP)

But I still don't know why bcc isn't finding BPF_FUNC_get_current_cgroup_id even when it is present in the headers. In fact, if I set it to 80 instead of 0 cgroup will work when running in kernel 4.18 and above

[jonhoo]

@brendangregg any chance we could get a new point release with the fix from https://github.com/iovisor/bpftrace/pull/427? Without it, the Arch Linux package doesn't build: https://aur.archlinux.org/packages/bpftrace/

[jasonk000]

@jonhoo I think there are two issues to fix here before it will work on arch:
1) a fix to bpftrace following williangaspar's lead: https://github.com/iovisor/bpftrace/pull/484
2) arch specifically has another issue with linux-api-headers being out of date, currently it's at 4.17, I posted a patch to arch-general but I'm not a maintaner: https://lists.archlinux.org/pipermail/arch-general/2019-March/046241.html