Bcc: Failed to trace Round Trip Time (RTT) in kernel functions

[alvenwong]

The kernel version is 4.14.62-65.117.amzn1.x86_64. I used attach_kprobe() to inject code into three kernel functions about RTT. But it failed in two of them.
tcp_ack_update_rtt(): cannot attach kprobe, probe entry may not exist.
tcp_update_rtt_min(): ERROR: tcp_update_rtt_min() kernel function not found or traceable.
tcp_synack_rtt_meas(): it works.
Could anybody explain why attach_kprobe() fails for the first two functions? Thanks very much for the help.

[palmtenor]

Does your /proc/kallsyms contain the two functions you are attaching to?

[alvenwong]

$ cat /proc/kallsyms | grep "rtt"
ffffffff8150e3b0 t tcp_ack_update_rtt.isra.34
ffffffff81511a90 T tcp_synack_rtt_meas

It contains the first and the third ones. However, the first one in /proc/kallsyms is tcp_ack_update_rtt.isra.34 instead of tcp_ack_update_rtt. Do you have any suggestion to tackle it?@palmtenor

[yonghong-song]

Could you use function name tcp_ack_update_rtt.isra.34 rather than tcp_ack_update_rtt in your script? Currently, bcc is not trying to relate tcp_ack_update_rtt to tcp_ack_update_rtt.isra.34. I need to double check in the compiler whether there is a possibility a local static function may have more than one .isra.<..> variants.

[yonghong-song]

The .isra function is an optimization to replace the struct parameter with a pointer to struct parameter. So I suppose that only one such function could happen for the original function.
It is possible for bcc to be smart to check if a kprobe function function, a function with
the .isra.<num> suffix can be tried.

Do you want to try with a pull request to implement this functionality?

[alvenwong]

Thanks very much! @yonghong-song As you suggested, I use tcp_ack_update_rtt.isra.34 in my script and it works.

Do you ask me to implement this functionality? @yonghong-song

[yonghong-song]

@alvenwong Right. Since you have this issue, other people may have similar issues. Maybe you want to help automate this in bcc? so that when people try to kprobe a function <func> and it is not available but a <func>.isra.,<num> is available, maybe that function can be used instead. We can print a warning to the user about this implicit trying of a different function so user can correct themselves with right function name if they choose to do so.

[alvenwong]

BCC is an amazing project. I will try to help, though I am just a green hand.