perf regression: failed to stat syscalls events

perf regression: failed to stat syscalls events

[Wu Fengguang]

Greetings,

I noticed that

        # perf stat -e syscalls:sys_enter_mmap true

refuse to work on 2.6.35-rc1. The last working kernel is 2.6.34.

Any suggestions before me jumping into debug?

Thanks,
Fengguang

Re: perf regression: failed to stat syscalls events

[Wu Fengguang]

On Thu, Jun 24, 2010 at 09:39:15AM +0800, Wu Fengguang wrote:
> Greetings,
> 
> I noticed that
> 
>         # perf stat -e syscalls:sys_enter_mmap true
> 
> refuse to work on 2.6.35-rc1. The last working kernel is 2.6.34.

Other events like lock:* or block:* or whatever continue to work.
So this is a problem specific to syscalls:*.

Thanks,
Fengguang

Re: perf regression: failed to stat syscalls events

[Peter Zijlstra]

On Thu, 2010-06-24 at 09:44 +0800, Wu Fengguang wrote:
> On Thu, Jun 24, 2010 at 09:39:15AM +0800, Wu Fengguang wrote:
> > Greetings,
> > 
> > I noticed that
> > 
> >         # perf stat -e syscalls:sys_enter_mmap true
> > 
> > refuse to work on 2.6.35-rc1. The last working kernel is 2.6.34.
> 
> Other events like lock:* or block:* or whatever continue to work.
> So this is a problem specific to syscalls:*.

Does your tree contain:

---
commit a8fb2608053547bc3152ea61a5ec7cdfce5d942c
Author: Steven Rostedt <srostedt@redhat.com>
Date:   Thu Jun 10 14:53:16 2010 -0400

    perf/tracing: Fix regression of perf losing kprobe events
    
    With the addition of the code to shrink the kernel tracepoint
    infrastructure, we lost kprobes being traced by perf. The reason
    is that I tested if the "tp_event->class->perf_probe" existed before
    enabling it. This prevents "ftrace only" events (like the function
    trace events) from being enabled by perf.
    
    Unfortunately, kprobe events do not use perf_probe. This causes
    kprobes to be missed by perf. To fix this, we add the test to
    see if "tp_event->class->reg" exists as well as perf_probe.
    
    Normal trace events have only "perf_probe" but no "reg" function,
    and kprobes and syscalls have the "reg" but no "perf_probe".
    The ftrace unique events do not have either, so this is a valid
    test. If a kprobe or syscall is not to be probed by perf, the
    "reg" function is called anyway, and will return a failure and
    prevent perf from probing it.
    
    Reported-by: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
    Tested-by: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
    Acked-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
    Signed-off-by: Steven Rostedt <rostedt@goodmis.org>

diff --git a/kernel/trace/trace_event_perf.c b/kernel/trace/trace_event_perf.c
index e6f6588..8a2b73f 100644
--- a/kernel/trace/trace_event_perf.c
+++ b/kernel/trace/trace_event_perf.c
@@ -96,7 +96,9 @@ int perf_trace_init(struct perf_event *p_event)
 	mutex_lock(&event_mutex);
 	list_for_each_entry(tp_event, &ftrace_events, list) {
 		if (tp_event->event.type == event_id &&
-		    tp_event->class && tp_event->class->perf_probe &&
+		    tp_event->class &&
+		    (tp_event->class->perf_probe ||
+		     tp_event->class->reg) &&
 		    try_module_get(tp_event->mod)) {
 			ret = perf_trace_event_init(tp_event, p_event);
 			break;

Re: perf regression: failed to stat syscalls events

[Wu Fengguang]

On Thu, Jun 24, 2010 at 01:58:45PM +0800, Peter Zijlstra wrote:
> On Thu, 2010-06-24 at 09:44 +0800, Wu Fengguang wrote:
> > On Thu, Jun 24, 2010 at 09:39:15AM +0800, Wu Fengguang wrote:
> > > Greetings,
> > > 
> > > I noticed that
> > > 
> > >         # perf stat -e syscalls:sys_enter_mmap true
> > > 
> > > refuse to work on 2.6.35-rc1. The last working kernel is 2.6.34.
> > 
> > Other events like lock:* or block:* or whatever continue to work.
> > So this is a problem specific to syscalls:*.
> 
> Does your tree contain:

Thanks, it worked!  This patch is missing from linux-2.6.git for now.

Thanks,
Fengguang


> ---
> commit a8fb2608053547bc3152ea61a5ec7cdfce5d942c
> Author: Steven Rostedt <srostedt@redhat.com>
> Date:   Thu Jun 10 14:53:16 2010 -0400
> 
>     perf/tracing: Fix regression of perf losing kprobe events
>     
>     With the addition of the code to shrink the kernel tracepoint
>     infrastructure, we lost kprobes being traced by perf. The reason
>     is that I tested if the "tp_event->class->perf_probe" existed before
>     enabling it. This prevents "ftrace only" events (like the function
>     trace events) from being enabled by perf.
>     
>     Unfortunately, kprobe events do not use perf_probe. This causes
>     kprobes to be missed by perf. To fix this, we add the test to
>     see if "tp_event->class->reg" exists as well as perf_probe.
>     
>     Normal trace events have only "perf_probe" but no "reg" function,
>     and kprobes and syscalls have the "reg" but no "perf_probe".
>     The ftrace unique events do not have either, so this is a valid
>     test. If a kprobe or syscall is not to be probed by perf, the
>     "reg" function is called anyway, and will return a failure and
>     prevent perf from probing it.
>     
>     Reported-by: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
>     Tested-by: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
>     Acked-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
>     Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
> 
> diff --git a/kernel/trace/trace_event_perf.c b/kernel/trace/trace_event_perf.c
> index e6f6588..8a2b73f 100644
> --- a/kernel/trace/trace_event_perf.c
> +++ b/kernel/trace/trace_event_perf.c
> @@ -96,7 +96,9 @@ int perf_trace_init(struct perf_event *p_event)
>  	mutex_lock(&event_mutex);
>  	list_for_each_entry(tp_event, &ftrace_events, list) {
>  		if (tp_event->event.type == event_id &&
> -		    tp_event->class && tp_event->class->perf_probe &&
> +		    tp_event->class &&
> +		    (tp_event->class->perf_probe ||
> +		     tp_event->class->reg) &&
>  		    try_module_get(tp_event->mod)) {
>  			ret = perf_trace_event_init(tp_event, p_event);
>  			break;
>