Async-profiler: Perf events unavailble. See stderr of the target process.

[billythach]

Hello!

I try to use Async Profiler to analyze the off-heap memory : when i profiled malloc, mmap or mprotect events, i faced to the same error Perf events unavailble. See stderr of the target process.

Environnement :

• Hypervisor
  
  
  
  • qemu-kvm-0.12.1.2-2.503.el6_9.3.x86_64
  
  
  • qemu-kvm-tools-0.12.1.2-2.503.el6_9.3.x86_64
  
  
• VM :
  
  
  
  • CentOS release 6.9 : Linux version 2.6.32-696.23.1.el6.x86_64 ([email protected]) (gcc version 4.4.7 20120313 (Red Hat 4.4.7-18) (GCC) ) #1 SMP Tue Mar 13 22:44:18 UTC 2018
  
  
  • OpenJDK Runtime Environment (build 1.8.0_191-debug-b12)
  
  
  • Configuration
  
  

$ cat /proc/sys/kernel/kptr_restrict                                                                                                                                                                               
0
$ cat /proc/sys/kernel/perf_event_paranoid                                                                                                                                                                         
-1

Testing :

• Java Class : your implementation of AllocatingTarget.java

• CPU profiling : OK

Script :

#!/bin/bash

set -e  # exit on any failure
set -x  # print all executed lines

(
  cd $(dirname $0)

  if [ "AllocatingTarget.class" -ot "AllocatingTarget.java" ]; then
     javac AllocatingTarget.java
  fi

  java AllocatingTarget &

  JAVAPID=$!

  sleep 5     # allow the Java runtime to initialize
  /opt/lyra/async_profiler/profiler.sh -d 10 -e cpu $JAVAPID

  kill $JAVAPID
)

Result :

++ dirname cpu-smoke-test.sh
+ cd .
+ '[' AllocatingTarget.class -ot AllocatingTarget.java ']'
+ JAVAPID=9727
+ sleep 5
+ java AllocatingTarget
+ /opt/lyra/async_profiler/profiler.sh -d 10 -e cpu 9727
Started [cpu] profiling
--- Execution profile ---
Total samples:         1394
Non-Java:              644 (46.20%)
Unknown (native):      8 (0.57%)

Frame buffer usage:    0.0349%

--- 6010787418 ns (43.11%), 601 samples
  [ 0] Copy::pd_fill_to_words(HeapWord*, unsigned long, unsigned int)

--- 3490823110 ns (25.03%), 349 samples
  [ 0] Copy::pd_fill_to_words(HeapWord*, unsigned long, unsigned int)
  [ 1] java.util.Random.next
  [ 2] java.util.Random.nextBoolean
  [ 3] AllocatingTarget.allocate
  [ 4] AllocatingTarget.run
  [ 5] java.lang.Thread.run

   ...

• Allocation Off-Heap profiling : KO

Script :

#!/bin/bash

set -e  # exit on any failure
set -x  # print all executed lines

(
  cd $(dirname $0)

  if [ "AllocatingTarget.class" -ot "AllocatingTarget.java" ]; then
     javac AllocatingTarget.java
  fi

  java AllocatingTarget &

  JAVAPID=$!

  sleep 5     # allow the Java runtime to initialize
  /opt/lyra/async_profiler/profiler.sh -d 10 -e malloc $JAVAPID
  /opt/lyra/async_profiler/profiler.sh -d 10 -e mmap -t $JAVAPID
  /opt/lyra/async_profiler/profiler.sh -d 10 -e mprotect -t $JAVAPID

  kill $JAVAPID
)

Result :

++ dirname alloc-smoke-test.sh
+ cd .
+ '[' AllocatingTarget.class -ot AllocatingTarget.java ']'
+ JAVAPID=13896
+ sleep 5
+ java AllocatingTarget
+ /opt/lyra/async_profiler/profiler.sh -d 10 -e malloc 13896
perf_event_open failed: No such file or directory
perf_event_open failed: No such file or directory
    ....
perf_event_open failed: No such file or directory
Perf events unavailble. See stderr of the target process.
--- Execution profile ---
Total samples:         0

Frame buffer usage:    0%

      events  percent  samples  top
  ----------  -------  -------  ---
+ /opt/lyra/async_profiler/profiler.sh -d 10 -e mmap -t 13896
perf_event_open failed: No such file or directory
perf_event_open failed: No such file or directory
    ....
perf_event_open failed: No such file or directory
Perf events unavailble. See stderr of the target process.
--- Execution profile ---
Total samples:         0

Frame buffer usage:    0%

      events  percent  samples  top
  ----------  -------  -------  ---
+ /opt/lyra/async_profiler/profiler.sh -d 10 -e mprotect -t 13896
perf_event_open failed: No such file or directory
perf_event_open failed: No such file or directory
    ....
perf_event_open failed: No such file or directory
Perf events unavailble. See stderr of the target process.
--- Execution profile ---
Total samples:         0

Frame buffer usage:    0%

      events  percent  samples  top
  ----------  -------  -------  ---

Has anyone faced to the same issue ? I tried to launch theses scripts with root user and the result is the same ..
Thanks!

[apangin]

perf_event_open syscall returns ENOENT, which suggests that selected profiling event type (PERF_TYPE_BREAKPOINT) is not supported on your system.

According to the man-page, PERF_TYPE_BREAKPOINT is supported since Linux 2.6.33, while you are running the kernel 2.6.32.

2.6.32 is way too old. I'm afraid native function profiling won't work on your system unless the kernel is upgraded.

[billythach]

Thanks for your answer @apangin .
It's explain why async-profiler works on my developpement environment (4.19.9-300.fc29.x86_64) but not in Integration environment.
Maybe, you could add this requirement to the documentation : this element could help some people.