Bcc: Clear perf buffers from python

[banh-gao]

Hi
I'm using the python library to run a bpf program. I use the perf ring buffer to send events from bpf to userspace.

At some point I want to stop calling perf_buffer_poll() and I need to clear all events still in the buffer (I need to reuse the same buffer later).

Is there a way to clear all those elements?

I've tried to use the clear method but it throws the following exeption (with bcc v0.6.0 on python3 and with kernel 4.15):

perf_table.clear()
  File "/usr/lib/python3/dist-packages/bcc/table.py", line 252, in clear
    self.__delitem__(k)
  File "/usr/lib/python3/dist-packages/bcc/table.py", line 543, in __delitem__
    if key_id in self.bpf.perf_buffers:
TypeError: unhashable type

[yonghong-song]

This is a ring buffer, not a regular map. So we cannot just delete items from it. To clear all elements, you need:

1. in kernel, stop sending data to this ring buffer,
2. in user space, drain the buffer. That is, poll on the buffer until all data are drained.
   If you really want to ensure, in kernel send a special message and in user space once
   you received the special message, stop polling.

[banh-gao]

Thank you @yonghong-song,
Do you think is there a reasonable way to implement the clear method to behave as expected (i.e. by draining all data from the buffer)?

[palmtenor]

The clear method would be exactly the same as polling the buffer until nothing returns (after certain timeout), which we already have in perf_buffer_poll. Do you have any specific use case that a clearing method would do differently?

[banh-gao]

As far as I understand, the only way to determine if a call to perf_buffer_poll(timeout=1) successfully reads data from buffer rather than timing out is to test if the callback function was called.

This adds implementation-specific complexity that I would prefer to embed inside the library rather than to be exposed to the user. Implementing it in the clear method is in my opinion the most reasonable place as there it does what the user expects.

[banh-gao]

A more convenient way to test the outcome of the call to perf_buffer_poll would be to intercept the return value of the poll call in https://github.com/iovisor/bcc/blob/18d3814ca9f6bf36f3768a2d5fee0fbd55c11798/src/cc/perf_reader.c#L221-L237

In particular returning something different from 0 when poll returns 0 allows to report the timeout condition to the caller.

[palmtenor]

I see. In C++ we return the number of events being polled so it's very easy to just do things like while(buf_.poll(0) > 0) {}. We can definitely do the same to return the value in Python.

[banh-gao]

I agree. Returning the number of polled events would be very useful.

[yonghong-song]

@banh-gao @palmtenor Are you willing to work on a pull request for this feature, basically, providing a return value for python perf_buffer_poll() API to indicate whether it is returned due to timeout or not?