* [PATCH 14/14] tracing: fix regression of perf function tracing
2013-03-27 9:48 [PATCH 00/14] event tracing expose change and bugfix/cleanup zhangwei(Jovi)
@ 2013-03-27 9:48 ` zhangwei(Jovi)
2013-03-27 9:48 ` zhangwei(Jovi)
1 sibling, 0 replies; 3+ messages in thread
From: zhangwei(Jovi) @ 2013-03-27 9:48 UTC (permalink / raw)
To: Steven Rostedt, Frederic Weisbecker, Ingo Molnar, LKML; +Cc: zhangwei(Jovi)
From: "zhangwei(Jovi)" <jovi.zhangwei@huawei.com>
Using perf command: perf stat -e ftrace:function ls
this will cause kernel warning and oops.
[ 797.828904] ------------[ cut here ]------------
[ 797.828946] WARNING: at include/linux/ftrace.h:209 ftrace_ops_control_func+0xb1/0xc0()
[ 797.829065] Pid: 6086, comm: perf Not tainted 3.8.0-rc4+ #88
[ 797.829066] Call Trace:
[ 797.829078] [<c0447a42>] warn_slowpath_common+0x72/0xa0
[ 797.829080] [<c04bd6d1>] ? ftrace_ops_control_func+0xb1/0xc0
[ 797.829082] [<c04bd6d1>] ? ftrace_ops_control_func+0xb1/0xc0
[ 797.829083] [<c04b7453>] ? synchronize_sched+0x3/0x50
[ 797.829085] [<c04be50f>] ? __unregister_ftrace_function+0x8f/0x170
[ 797.829087] [<c0447a92>] warn_slowpath_null+0x22/0x30
[ 797.829089] [<c04bd6d1>] ftrace_ops_control_func+0xb1/0xc0
[ 797.829099] [<c08eba6b>] ftrace_call+0x5/0xb
[ 797.829100] [<c04b7458>] ? synchronize_sched+0x8/0x50
[ 797.829102] [<c04be50f>] __unregister_ftrace_function+0x8f/0x170
[ 797.829104] [<c04c00cf>] unregister_ftrace_function+0x1f/0x50
[ 797.829109] [<c04d31bd>] perf_ftrace_event_register+0x9d/0x140
[ 797.829111] [<c04d304b>] perf_trace_destroy+0x2b/0x50
[ 797.829117] [<c04db3c8>] tp_perf_event_destroy+0x8/0x10
[ 797.829119] [<c04dd672>] free_event+0x42/0x110
[ 797.829121] [<c04de446>] perf_event_release_kernel+0x56/0x90
[ 797.829122] [<c04de4fc>] put_event+0x7c/0xa0
[ 797.829124] [<c04de5cb>] perf_release+0xb/0x10
[ 797.829128] [<c0532ae6>] __fput+0xc6/0x1f0
[ 797.829130] [<c0532c1d>] ____fput+0xd/0x10
[ 797.829134] [<c04645b1>] task_work_run+0x81/0xa0
[ 797.829142] [<c0412819>] do_notify_resume+0x59/0x90
[ 797.829150] [<c08e4745>] work_notifysig+0x30/0x37
[ 797.829152] ---[ end trace 4dbd63f12b55163f ]---
This bug was introduced by below commit(in 3.8-rc4):
commit 0a016409e42f273415f8225ddf2c58eb2df88034
Author: Steven Rostedt <srostedt@redhat.com>
Date: Fri Nov 2 17:03:03 2012 -0400
ftrace: Optimize the function tracer list loop
When variable op is ftrace_list_end, it cannot pass control ops checking,
so that loop optimize is not suit for ftrace_control_list, change it back.
Signed-off-by: zhangwei(Jovi) <jovi.zhangwei@huawei.com>
---
kernel/trace/ftrace.c | 7 +++++--
1 file changed, 5 insertions(+), 2 deletions(-)
diff --git a/kernel/trace/ftrace.c b/kernel/trace/ftrace.c
index 2577082..2899974 100644
--- a/kernel/trace/ftrace.c
+++ b/kernel/trace/ftrace.c
@@ -4185,11 +4185,14 @@ ftrace_ops_control_func(unsigned long ip, unsigned long parent_ip,
*/
preempt_disable_notrace();
trace_recursion_set(TRACE_CONTROL_BIT);
- do_for_each_ftrace_op(op, ftrace_control_list) {
+ op = rcu_dereference_raw(ftrace_control_list);
+ while (op != &ftrace_list_end) {
if (!ftrace_function_local_disabled(op) &&
ftrace_ops_test(op, ip))
op->func(ip, parent_ip, op, regs);
- } while_for_each_ftrace_op(op);
+
+ op = rcu_dereference_raw(op->next);
+ }
trace_recursion_clear(TRACE_CONTROL_BIT);
preempt_enable_notrace();
}
--
1.7.9.7
^ permalink raw reply related [flat|nested] 3+ messages in thread
* [PATCH 14/14] tracing: fix regression of perf function tracing
2013-03-27 9:48 [PATCH 00/14] event tracing expose change and bugfix/cleanup zhangwei(Jovi)
2013-03-27 9:48 ` [PATCH 14/14] tracing: fix regression of perf function tracing zhangwei(Jovi)
@ 2013-03-27 9:48 ` zhangwei(Jovi)
1 sibling, 0 replies; 3+ messages in thread
From: zhangwei(Jovi) @ 2013-03-27 9:48 UTC (permalink / raw)
To: Steven Rostedt, Frederic Weisbecker, Ingo Molnar, LKML
Cc: zhangwei(Jovi), stable
From: "zhangwei(Jovi)" <jovi.zhangwei@huawei.com>
Using perf command: perf stat -e ftrace:function ls
this will cause kernel warning and oops.
[ 797.828904] ------------[ cut here ]------------
[ 797.828946] WARNING: at include/linux/ftrace.h:209 ftrace_ops_control_func+0xb1/0xc0()
[ 797.829065] Pid: 6086, comm: perf Not tainted 3.8.0-rc4+ #88
[ 797.829066] Call Trace:
[ 797.829078] [<c0447a42>] warn_slowpath_common+0x72/0xa0
[ 797.829080] [<c04bd6d1>] ? ftrace_ops_control_func+0xb1/0xc0
[ 797.829082] [<c04bd6d1>] ? ftrace_ops_control_func+0xb1/0xc0
[ 797.829083] [<c04b7453>] ? synchronize_sched+0x3/0x50
[ 797.829085] [<c04be50f>] ? __unregister_ftrace_function+0x8f/0x170
[ 797.829087] [<c0447a92>] warn_slowpath_null+0x22/0x30
[ 797.829089] [<c04bd6d1>] ftrace_ops_control_func+0xb1/0xc0
[ 797.829099] [<c08eba6b>] ftrace_call+0x5/0xb
[ 797.829100] [<c04b7458>] ? synchronize_sched+0x8/0x50
[ 797.829102] [<c04be50f>] __unregister_ftrace_function+0x8f/0x170
[ 797.829104] [<c04c00cf>] unregister_ftrace_function+0x1f/0x50
[ 797.829109] [<c04d31bd>] perf_ftrace_event_register+0x9d/0x140
[ 797.829111] [<c04d304b>] perf_trace_destroy+0x2b/0x50
[ 797.829117] [<c04db3c8>] tp_perf_event_destroy+0x8/0x10
[ 797.829119] [<c04dd672>] free_event+0x42/0x110
[ 797.829121] [<c04de446>] perf_event_release_kernel+0x56/0x90
[ 797.829122] [<c04de4fc>] put_event+0x7c/0xa0
[ 797.829124] [<c04de5cb>] perf_release+0xb/0x10
[ 797.829128] [<c0532ae6>] __fput+0xc6/0x1f0
[ 797.829130] [<c0532c1d>] ____fput+0xd/0x10
[ 797.829134] [<c04645b1>] task_work_run+0x81/0xa0
[ 797.829142] [<c0412819>] do_notify_resume+0x59/0x90
[ 797.829150] [<c08e4745>] work_notifysig+0x30/0x37
[ 797.829152] ---[ end trace 4dbd63f12b55163f ]---
This bug was introduced by below commit(in 3.8-rc4):
commit 0a016409e42f273415f8225ddf2c58eb2df88034
Author: Steven Rostedt <srostedt@redhat.com>
Date: Fri Nov 2 17:03:03 2012 -0400
ftrace: Optimize the function tracer list loop
When variable op is ftrace_list_end, it cannot pass control ops checking,
so that loop optimize is not suit for ftrace_control_list, change it back.
Signed-off-by: zhangwei(Jovi) <jovi.zhangwei@huawei.com>
Cc: stable@vger.kernel.org
---
kernel/trace/ftrace.c | 7 +++++--
1 file changed, 5 insertions(+), 2 deletions(-)
diff --git a/kernel/trace/ftrace.c b/kernel/trace/ftrace.c
index 2577082..2899974 100644
--- a/kernel/trace/ftrace.c
+++ b/kernel/trace/ftrace.c
@@ -4185,11 +4185,14 @@ ftrace_ops_control_func(unsigned long ip, unsigned long parent_ip,
*/
preempt_disable_notrace();
trace_recursion_set(TRACE_CONTROL_BIT);
- do_for_each_ftrace_op(op, ftrace_control_list) {
+ op = rcu_dereference_raw(ftrace_control_list);
+ while (op != &ftrace_list_end) {
if (!ftrace_function_local_disabled(op) &&
ftrace_ops_test(op, ip))
op->func(ip, parent_ip, op, regs);
- } while_for_each_ftrace_op(op);
+
+ op = rcu_dereference_raw(op->next);
+ }
trace_recursion_clear(TRACE_CONTROL_BIT);
preempt_enable_notrace();
}
--
1.7.9.7
^ permalink raw reply related [flat|nested] 3+ messages in thread
* Re: [PATCH 14/14] tracing: fix regression of perf function tracing
[not found] ` <1364389775.6345.210.camel@gandalf.local.home>
@ 2013-03-28 0:17 ` zhangwei(Jovi)
0 siblings, 0 replies; 3+ messages in thread
From: zhangwei(Jovi) @ 2013-03-28 0:17 UTC (permalink / raw)
To: Steven Rostedt
Cc: Frederic Weisbecker, Ingo Molnar, stable,
linux-kernel@vger.kernel.org
On 2013/3/27 21:09, Steven Rostedt wrote:
> On Wed, 2013-03-27 at 17:44 +0800, zhangwei(Jovi) wrote:
>> From: "zhangwei(Jovi)" <jovi.zhangwei@huawei.com>
>>
>> Using perf command: perf stat -e ftrace:function ls
>>
>> this will cause kernel warning and oops.
>>
>> [ 797.828904] ------------[ cut here ]------------
>> [ 797.828946] WARNING: at include/linux/ftrace.h:209 ftrace_ops_control_func+0xb1/0xc0()
>> [ 797.829065] Pid: 6086, comm: perf Not tainted 3.8.0-rc4+ #88
>> [ 797.829066] Call Trace:
>> [ 797.829078] [<c0447a42>] warn_slowpath_common+0x72/0xa0
>> [ 797.829080] [<c04bd6d1>] ? ftrace_ops_control_func+0xb1/0xc0
>> [ 797.829082] [<c04bd6d1>] ? ftrace_ops_control_func+0xb1/0xc0
>> [ 797.829083] [<c04b7453>] ? synchronize_sched+0x3/0x50
>> [ 797.829085] [<c04be50f>] ? __unregister_ftrace_function+0x8f/0x170
>> [ 797.829087] [<c0447a92>] warn_slowpath_null+0x22/0x30
>> [ 797.829089] [<c04bd6d1>] ftrace_ops_control_func+0xb1/0xc0
>> [ 797.829099] [<c08eba6b>] ftrace_call+0x5/0xb
>> [ 797.829100] [<c04b7458>] ? synchronize_sched+0x8/0x50
>> [ 797.829102] [<c04be50f>] __unregister_ftrace_function+0x8f/0x170
>> [ 797.829104] [<c04c00cf>] unregister_ftrace_function+0x1f/0x50
>> [ 797.829109] [<c04d31bd>] perf_ftrace_event_register+0x9d/0x140
>> [ 797.829111] [<c04d304b>] perf_trace_destroy+0x2b/0x50
>> [ 797.829117] [<c04db3c8>] tp_perf_event_destroy+0x8/0x10
>> [ 797.829119] [<c04dd672>] free_event+0x42/0x110
>> [ 797.829121] [<c04de446>] perf_event_release_kernel+0x56/0x90
>> [ 797.829122] [<c04de4fc>] put_event+0x7c/0xa0
>> [ 797.829124] [<c04de5cb>] perf_release+0xb/0x10
>> [ 797.829128] [<c0532ae6>] __fput+0xc6/0x1f0
>> [ 797.829130] [<c0532c1d>] ____fput+0xd/0x10
>> [ 797.829134] [<c04645b1>] task_work_run+0x81/0xa0
>> [ 797.829142] [<c0412819>] do_notify_resume+0x59/0x90
>> [ 797.829150] [<c08e4745>] work_notifysig+0x30/0x37
>> [ 797.829152] ---[ end trace 4dbd63f12b55163f ]---
>>
>> This bug was introduced by below commit(in 3.8-rc4):
>> commit 0a016409e42f273415f8225ddf2c58eb2df88034
>> Author: Steven Rostedt <srostedt@redhat.com>
>> Date: Fri Nov 2 17:03:03 2012 -0400
>>
>> ftrace: Optimize the function tracer list loop
>>
>> When variable op is ftrace_list_end, it cannot pass control ops checking,
>> so that loop optimize is not suit for ftrace_control_list, change it back.
>>
>
> Thanks for finding the bug, but this isn't the fix.
>
>> Signed-off-by: zhangwei(Jovi) <jovi.zhangwei@huawei.com>
>> Cc: stable@vger.kernel.org
>> ---
>> kernel/trace/ftrace.c | 7 +++++--
>> 1 file changed, 5 insertions(+), 2 deletions(-)
>>
>> diff --git a/kernel/trace/ftrace.c b/kernel/trace/ftrace.c
>> index 2577082..2899974 100644
>> --- a/kernel/trace/ftrace.c
>> +++ b/kernel/trace/ftrace.c
>> @@ -4185,11 +4185,14 @@ ftrace_ops_control_func(unsigned long ip, unsigned long parent_ip,
>> */
>> preempt_disable_notrace();
>> trace_recursion_set(TRACE_CONTROL_BIT);
>> - do_for_each_ftrace_op(op, ftrace_control_list) {
>
> I'd much rather keep the do_for_each_op() as it keeps everything
> consistent, and I don't have to worry about this if I change things in
> the future.
I changed this on thought of this would save a duplicate checking in hot path.
But never mind, it's a only tiny change, either way don't have big issue.
>
> I'll fix this so that it wont process stub functions.
>
> Thanks!
>
> -- Steve
>
>
>> + op = rcu_dereference_raw(ftrace_control_list);
>> + while (op != &ftrace_list_end) {
>> if (!ftrace_function_local_disabled(op) &&
>> ftrace_ops_test(op, ip))
>> op->func(ip, parent_ip, op, regs);
>> - } while_for_each_ftrace_op(op);
>> +
>> + op = rcu_dereference_raw(op->next);
>> + }
>> trace_recursion_clear(TRACE_CONTROL_BIT);
>> preempt_enable_notrace();
>> }
>
>
>
> .
>
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2013-03-28 0:17 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
[not found] <1364377499-1900-1-git-send-email-jovi.zhangwei@huawei.com>
[not found] ` <1364377499-1900-15-git-send-email-jovi.zhangwei@huawei.com>
[not found] ` <1364389775.6345.210.camel@gandalf.local.home>
2013-03-28 0:17 ` [PATCH 14/14] tracing: fix regression of perf function tracing zhangwei(Jovi)
2013-03-27 9:48 [PATCH 00/14] event tracing expose change and bugfix/cleanup zhangwei(Jovi)
2013-03-27 9:48 ` [PATCH 14/14] tracing: fix regression of perf function tracing zhangwei(Jovi)
2013-03-27 9:48 ` zhangwei(Jovi)
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox