perf probe and structs

perf probe and structs

[Arnaldo Carvalho de Melo]

Hi Masami,

	I tried somethig with perf probe today, namely to ask for a
variable that is a struct perf_event_attr to be collected after
the perf_event_open syscall copies it from userspace, and got this
message:

[root@zoo ~]# perf probe SYSC_perf_event_open:23 args
Failed to find 'args' in this function.
  Error: Failed to add events.
[root@zoo ~]#

Ok, I guess it should be instead:

[root@zoo ~]# perf probe SYSC_perf_event_open:23 args
The 'args' variable is a struct, this is not supported yet.
  Error: Failed to add events.
[root@zoo ~]#

:-)

That said, can't we just go ahead and collect sizeof(args) into the
buffer and let userspace cast the result to the right type, etc, i.e.
kinda like what is done now for a string.

Alexei, is this already possible with eBPF?

This is all in the context of:

  [root@zoo ~]# perf trace -e perf_event_open &
  [1] 20775
  [root@zoo ~]# perf stat -e cycles usleep 1
    7304.425 ( 0.519 ms): perf/20776 perf_event_open(attr_uptr: 0x222b220, pid: 20777, cpu: -1, group_fd: -1, flags: FD_CLOEXEC) = 3

   Performance counter stats for 'usleep 1':

         2,024,026      cycles                                                      

       0.005277428 seconds time elapsed

  [root@zoo ~]#

I want to decode that attr_uptr thing :-)

- Arnaldo

Re: perf probe and structs

[Masami Hiramatsu]

On 2015/06/13 4:27, Arnaldo Carvalho de Melo wrote:
> Hi Masami,
> 
> 	I tried somethig with perf probe today, namely to ask for a
> variable that is a struct perf_event_attr to be collected after
> the perf_event_open syscall copies it from userspace, and got this
> message:
> 
> [root@zoo ~]# perf probe SYSC_perf_event_open:23 args
> Failed to find 'args' in this function.
>   Error: Failed to add events.
> [root@zoo ~]#
> 
> Ok, I guess it should be instead:
> 
> [root@zoo ~]# perf probe SYSC_perf_event_open:23 args
> The 'args' variable is a struct, this is not supported yet.
>   Error: Failed to add events.
> [root@zoo ~]#
> 
> :-)

Hmm, I need to look into it. BTW, could you tried to run
`perf probe --vars SYSC_perf_event_open:23` ?


> 
> That said, can't we just go ahead and collect sizeof(args) into the
> buffer and let userspace cast the result to the right type, etc, i.e.
> kinda like what is done now for a string.

I'd like to add dump data struct feature instead of dumping binary.
Or, at least suggesting which field you can specify :)

Thank you,


> 
> Alexei, is this already possible with eBPF?
> 
> This is all in the context of:
> 
>   [root@zoo ~]# perf trace -e perf_event_open &
>   [1] 20775
>   [root@zoo ~]# perf stat -e cycles usleep 1
>     7304.425 ( 0.519 ms): perf/20776 perf_event_open(attr_uptr: 0x222b220, pid: 20777, cpu: -1, group_fd: -1, flags: FD_CLOEXEC) = 3
> 
>    Performance counter stats for 'usleep 1':
> 
>          2,024,026      cycles                                                      
> 
>        0.005277428 seconds time elapsed
> 
>   [root@zoo ~]#
> 
> I want to decode that attr_uptr thing :-)
> 
> - Arnaldo
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 


-- 
Masami HIRAMATSU
Linux Technology Research Center, System Productivity Research Dept.
Center for Technology Innovation - Systems Engineering
Hitachi, Ltd., Research & Development Group
E-mail: masami.hiramatsu.pt@hitachi.com

Re: perf probe and structs

[Alexei Starovoitov]

On 6/12/15 12:27 PM, Arnaldo Carvalho de Melo wrote:
> Alexei, is this already possible with eBPF?
> I want to decode that attr_uptr thing :-)

yes, it's already possible :)

Here is working example from our experimental c+python thingy:
#!/usr/bin/env python

from bpf import BPF
from subprocess import call

prog = """
#include <uapi/linux/ptrace.h>
#include <uapi/linux/perf_event.h>
int hello(struct pt_regs *ctx)
{
   struct perf_event_attr attr = {};
   bpf_probe_read(&attr, sizeof(attr), (void *) ctx->di);
   char fmt[] = "type %x size %d config %d\\n";
   bpf_trace_printk(fmt, sizeof(fmt), attr.type, attr.size, attr.config);
   return 0;
}
"""
b = BPF(text=prog)
fn = b.load_func("hello", BPF.KPROBE)
BPF.attach_kprobe(fn, "SYSC_perf_event_open")
try:
     call(["cat", "/sys/kernel/debug/tracing/trace_pipe"])
except KeyboardInterrupt:
     pass

running above gives me output:
# ./example.py
       perf_4.1.0-5544  [001] d.h3  3818.231428: : type 1 size 0 config 0
       perf_4.1.0-5544  [001] d.h3  3818.231494: : type 0 size 112 config 0
       perf_4.1.0-5544  [001] d.h3  3818.231530: : type 0 size 112 config 0
       perf_4.1.0-5544  [001] d.h3  3818.231554: : type 0 size 112 config 0
       perf_4.1.0-5544  [001] d.h3  3818.231564: : type 0 size 112 config 0

Re: perf probe and structs

[Masami Hiramatsu]

On 2015/06/13 4:27, Arnaldo Carvalho de Melo wrote:
> Hi Masami,
> 
> 	I tried somethig with perf probe today, namely to ask for a
> variable that is a struct perf_event_attr to be collected after
> the perf_event_open syscall copies it from userspace, and got this
> message:
> 
> [root@zoo ~]# perf probe SYSC_perf_event_open:23 args
> Failed to find 'args' in this function.
>   Error: Failed to add events.
> [root@zoo ~]#

BTW, would you sure that the args exists at the given place?
I couldn't find it in kernel/events/core.c.

And also, could you make sure to run below command to clarify
the existed variables and its valid ranges?
 # perf probe -V --range SYSC_perf_event_open

For example, I got below outouts.

# perf probe -V SYSC_perf_event_open:23
Available variables at SYSC_perf_event_open:23
        @<SYSC_perf_event_open+502>
                int     cpu
                int     group_fd
                long unsigned int       flags
                pid_t   pid
                struct perf_event_attr  attr
# perf probe SYSC_perf_event_open:23 attr
perf_event_attr exceeds max-bitwidth. Cut down to 64 bits.
Added new event:
  probe:SYSC_perf_event_open (on SYSC_perf_event_open:23 with attr)

You can now use it in all perf tools, such as:

        perf record -e probe:SYSC_perf_event_open -aR sleep 1

Of course, this is still not enough, I must fix this as you suggested.

> 
> Ok, I guess it should be instead:
> 
> [root@zoo ~]# perf probe SYSC_perf_event_open:23 args
> The 'args' variable is a struct, this is not supported yet.
>   Error: Failed to add events.
> [root@zoo ~]#
> 
> :-)
> 
-- 
Masami HIRAMATSU
Linux Technology Research Center, System Productivity Research Dept.
Center for Technology Innovation - Systems Engineering
Hitachi, Ltd., Research & Development Group
E-mail: masami.hiramatsu.pt@hitachi.com