Bcc: opensnoop: can't get the filename in some specific cases

[iaguis]

If the filename string happens to be, for example, near the border of a page and the next page is not mapped, we'll try to read the full buffer and bpf_probe_read() will return -EFAULT.

Here's an example (courtesy of @alban):

open_buffer_border.c:

#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <unistd.h>
#include <string.h>
#include <stdio.h>
#include <stdlib.h>
#include <sys/mman.h>


char *big_buffer;
#define FILENAME "/tmp/test1.txt"
char *filename1;
char *filename2;

int main() {
  int fd;

  printf("pid\n%d\n", getpid());

  big_buffer = mmap((void*)0x05000000, 4096, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
  memset(big_buffer, 0, sizeof(4096));

  filename1 = big_buffer;
  strcpy(filename1, FILENAME);
  filename2 = big_buffer + 4096 - strlen(FILENAME) - 2;
  strcpy(filename2, FILENAME);

  printf("filename1: %s\n", filename1);
  printf("filename2: %s\n", filename2);

  sleep(3);

  fd = open(filename1, O_RDONLY, 0);
  printf("open %s: fd %d\n", filename1, fd);
  close(fd);

  fd = open(filename2, O_RDONLY, 0);
  printf("open %s: fd %d\n", filename2, fd);
  close(fd);

  sleep(1);
  return 0;
}

Let's run it:

$ ./open_buffer_border 
pid
31188
filename1: /tmp/test1.txt
filename2: /tmp/test1.txt
open /tmp/test1.txt: fd 3
open /tmp/test1.txt: fd 3

But running opensnoop:

$ sudo ./tools/opensnoop.py
...
31188  open_buffer_bor     3   0 /tmp/test1.txt
31188  open_buffer_bor     3   0 

The filename in the second open doesn't appear.

[yonghong-song]

Looks like this is related kernel implementation.

Inside the kernel, bpf_probe_read will read specified "size" bytes.

In opensnoop.py:
bpf_probe_read(&data.fname, sizeof(data.fname), (void )valp->fname);
struct data_t {
u64 id;
u64 ts;
int ret;
char comm[TASK_COMM_LEN];
char fname[NAME_MAX];
};
uapi/linux/limits.h:#define NAME_MAX 255 / # chars in
a file name */

In this test case, the bpf_probe_read will read 255 bytes and will
read into unmapped area.
Kernel will have fault and clear the buffer to the bpf program.

Not sure how to solve this problem though.

On Tue, Aug 22, 2017 at 9:51 AM, Iago López Galeiras
[e-mail] wrote:
>

If the filename string happens to be, for example, near the border of a page and the next page is not mapped, we'll try to read the full buffer and bpf_probe_read() will return -EFAULT.

Here's an example (courtesy of @alban):

open_buffer_border.c:

include

include

include

include

include

include

include

include

char *big_buffer;

define FILENAME "/tmp/test1.txt"

char *filename1;
char *filename2;

int main() {
int fd;

printf("pid\n%d\n", getpid());

big_buffer = mmap((void*)0x05000000, 4096, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
memset(big_buffer, 0, sizeof(4096));

filename1 = big_buffer;
strcpy(filename1, FILENAME);
filename2 = big_buffer + 4096 - strlen(FILENAME) - 2;
strcpy(filename2, FILENAME);

printf("filename1: %s\n", filename1);
printf("filename2: %s\n", filename2);

sleep(3);

fd = open(filename1, O_RDONLY, 0);
printf("open %s: fd %d\n", filename1, fd);
close(fd);

fd = open(filename2, O_RDONLY, 0);
printf("open %s: fd %d\n", filename2, fd);
close(fd);

sleep(1);
return 0;
}

Let's run it:

$ ./open_buffer_border
pid
31188
filename1: /tmp/test1.txt
filename2: /tmp/test1.txt
open /tmp/test1.txt: fd 3
open /tmp/test1.txt: fd 3

But running opensnoop:

$ sudo ./tools/opensnoop.py
...
31188 open_buffer_bor 3 0 /tmp/test1.txt
31188 open_buffer_bor 3 0

The filename in the second open doesn't appear.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.

[4ast]

@yonghong-song is correct
opensnoop.py will try to probe_read the arguments before syscall is executed,
so 2nd page can be easily not mapped and since bpf program cannot take faults
the only thing bpf_probe_read() can do is to return EFAULT.
The workaround is to remember the arguments and snoop them
somewhere in the middle of the open() syscall or via kretprobe.
Then page faults will be taken and pointers will be ready for probe_read to access.
Long term we may introduce some special case of bpf programs that can only
be attached to syscalls and can only execute in user context then
we'll be able to take faults from the program, but there are plenty
of other headaches with such approach.

[brendangregg]

Very interesting bug.

Isn't this why Gianluca added BPF_FUNC_probe_read_str? He used open() in the example: https://github.com/torvalds/linux/commit/a5e8c07059d0f0b31737408711d44794928ac218

I haven't used it yet (what'd that make it into? 4.11?), but I'd imagine it reads until the NULL, and we can assume everything to the NULL should already be paged in by open() by the time opensnoop calls kretprobe.

Of course, we then need to have opensnoop use bpf_probe_read_str(), and maybe conditionally so that it works on older kernels.

[yonghong-song]

Agreed. bpf_probe_read_str should fix the issue.

On Tue, Aug 22, 2017 at 8:56 PM, Brendan Gregg [e-mail]
wrote:

Very interesting bug.

Isn't this why Gianluca added BPF_FUNC_probe_read_str? He used open() in
the example: torvalds/linux@a5e8c07
https://github.com/torvalds/linux/commit/a5e8c07059d0f0b31737408711d44794928ac218

I haven't used it yet (what'd that make it into? 4.11?), but I'd imagine
it reads until the NULL, and we can assume everything to the NULL should
already be paged in by open() by the time opensnoop calls kretprobe.

Of course, we then need to have opensnoop use bpf_probe_read_str(), and
maybe conditionally so that it works on older kernels.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/iovisor/bcc/issues/1307#issuecomment-324214279, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ALq6ouD3gmplguFDSyTm2ih4-hQx7-ZOks5sa6L_gaJpZM4O-5kT
.