* Re: [PATCH 14/14] tracing: fix regression of perf function tracing [not found] ` <1364389775.6345.210.camel@gandalf.local.home> @ 2013-03-28 0:17 ` zhangwei(Jovi) 0 siblings, 0 replies; 3+ messages in thread From: zhangwei(Jovi) @ 2013-03-28 0:17 UTC (permalink / raw) To: Steven Rostedt Cc: Frederic Weisbecker, Ingo Molnar, stable, linux-kernel@vger.kernel.org On 2013/3/27 21:09, Steven Rostedt wrote: > On Wed, 2013-03-27 at 17:44 +0800, zhangwei(Jovi) wrote: >> From: "zhangwei(Jovi)" <jovi.zhangwei@huawei.com> >> >> Using perf command: perf stat -e ftrace:function ls >> >> this will cause kernel warning and oops. >> >> [ 797.828904] ------------[ cut here ]------------ >> [ 797.828946] WARNING: at include/linux/ftrace.h:209 ftrace_ops_control_func+0xb1/0xc0() >> [ 797.829065] Pid: 6086, comm: perf Not tainted 3.8.0-rc4+ #88 >> [ 797.829066] Call Trace: >> [ 797.829078] [<c0447a42>] warn_slowpath_common+0x72/0xa0 >> [ 797.829080] [<c04bd6d1>] ? ftrace_ops_control_func+0xb1/0xc0 >> [ 797.829082] [<c04bd6d1>] ? ftrace_ops_control_func+0xb1/0xc0 >> [ 797.829083] [<c04b7453>] ? synchronize_sched+0x3/0x50 >> [ 797.829085] [<c04be50f>] ? __unregister_ftrace_function+0x8f/0x170 >> [ 797.829087] [<c0447a92>] warn_slowpath_null+0x22/0x30 >> [ 797.829089] [<c04bd6d1>] ftrace_ops_control_func+0xb1/0xc0 >> [ 797.829099] [<c08eba6b>] ftrace_call+0x5/0xb >> [ 797.829100] [<c04b7458>] ? synchronize_sched+0x8/0x50 >> [ 797.829102] [<c04be50f>] __unregister_ftrace_function+0x8f/0x170 >> [ 797.829104] [<c04c00cf>] unregister_ftrace_function+0x1f/0x50 >> [ 797.829109] [<c04d31bd>] perf_ftrace_event_register+0x9d/0x140 >> [ 797.829111] [<c04d304b>] perf_trace_destroy+0x2b/0x50 >> [ 797.829117] [<c04db3c8>] tp_perf_event_destroy+0x8/0x10 >> [ 797.829119] [<c04dd672>] free_event+0x42/0x110 >> [ 797.829121] [<c04de446>] perf_event_release_kernel+0x56/0x90 >> [ 797.829122] [<c04de4fc>] put_event+0x7c/0xa0 >> [ 797.829124] [<c04de5cb>] perf_release+0xb/0x10 >> [ 797.829128] [<c0532ae6>] __fput+0xc6/0x1f0 >> [ 797.829130] [<c0532c1d>] ____fput+0xd/0x10 >> [ 797.829134] [<c04645b1>] task_work_run+0x81/0xa0 >> [ 797.829142] [<c0412819>] do_notify_resume+0x59/0x90 >> [ 797.829150] [<c08e4745>] work_notifysig+0x30/0x37 >> [ 797.829152] ---[ end trace 4dbd63f12b55163f ]--- >> >> This bug was introduced by below commit(in 3.8-rc4): >> commit 0a016409e42f273415f8225ddf2c58eb2df88034 >> Author: Steven Rostedt <srostedt@redhat.com> >> Date: Fri Nov 2 17:03:03 2012 -0400 >> >> ftrace: Optimize the function tracer list loop >> >> When variable op is ftrace_list_end, it cannot pass control ops checking, >> so that loop optimize is not suit for ftrace_control_list, change it back. >> > > Thanks for finding the bug, but this isn't the fix. > >> Signed-off-by: zhangwei(Jovi) <jovi.zhangwei@huawei.com> >> Cc: stable@vger.kernel.org >> --- >> kernel/trace/ftrace.c | 7 +++++-- >> 1 file changed, 5 insertions(+), 2 deletions(-) >> >> diff --git a/kernel/trace/ftrace.c b/kernel/trace/ftrace.c >> index 2577082..2899974 100644 >> --- a/kernel/trace/ftrace.c >> +++ b/kernel/trace/ftrace.c >> @@ -4185,11 +4185,14 @@ ftrace_ops_control_func(unsigned long ip, unsigned long parent_ip, >> */ >> preempt_disable_notrace(); >> trace_recursion_set(TRACE_CONTROL_BIT); >> - do_for_each_ftrace_op(op, ftrace_control_list) { > > I'd much rather keep the do_for_each_op() as it keeps everything > consistent, and I don't have to worry about this if I change things in > the future. I changed this on thought of this would save a duplicate checking in hot path. But never mind, it's a only tiny change, either way don't have big issue. > > I'll fix this so that it wont process stub functions. > > Thanks! > > -- Steve > > >> + op = rcu_dereference_raw(ftrace_control_list); >> + while (op != &ftrace_list_end) { >> if (!ftrace_function_local_disabled(op) && >> ftrace_ops_test(op, ip)) >> op->func(ip, parent_ip, op, regs); >> - } while_for_each_ftrace_op(op); >> + >> + op = rcu_dereference_raw(op->next); >> + } >> trace_recursion_clear(TRACE_CONTROL_BIT); >> preempt_enable_notrace(); >> } > > > > . > ^ permalink raw reply [flat|nested] 3+ messages in thread
* [PATCH 00/14] event tracing expose change and bugfix/cleanup
@ 2013-03-27 9:48 zhangwei(Jovi)
2013-03-27 9:48 ` [PATCH 14/14] tracing: fix regression of perf function tracing zhangwei(Jovi)
2013-03-27 9:48 ` zhangwei(Jovi)
0 siblings, 2 replies; 3+ messages in thread
From: zhangwei(Jovi) @ 2013-03-27 9:48 UTC (permalink / raw)
To: Steven Rostedt, Frederic Weisbecker, Ingo Molnar, LKML; +Cc: zhangwei(Jovi)
From: "zhangwei(Jovi)" <jovi.zhangwei@huawei.com>
Hi steven,
This patchset contain:
1) event tracing expose work (v2)
this expose work rewrited compare with v1, new implementation
is based on multi-instances buffer work, it also integrate syscall
tracing code to use same event backend store mechanism.
The change include patch 1-7(patch 2 also fix a long-term minor bug)
2) some cleanup
This include patch 8-12.
3) patch 13 fix libtraceevent warning
4) patch 14 fix a regression bug of perf function tracing
Note that these patches is based on latest linux-trace git tree:
(on top of multi-instances buffer implementation)
git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-trace.git
tip/perf/core
All patches pass basic testing.
zhangwei(Jovi) (14):
tracing: move trace_array definition into include/linux/trace_array.h
tracing: fix irqs-off tag display in syscall tracing
tracing: expose event tracing infrastructure
tracing: add private data field into struct ftrace_event_file
tracing: switch syscall tracing to use event_trace_ops backend
tracing: export syscall metadata
tracing: expose structure ftrace_event_field
tracing: remove TRACE_EVENT_TYPE enum definition
tracing: remove obsolete macro guard _TRACE_PROFILE_INIT
tracing: remove ftrace(...) function
tracing: use per trace_array clock_id instead of global
trace_clock_id
tracing: guard tracing_selftest_disabled by
CONFIG_FTRACE_STARTUP_TEST
libtraceevent: add libtraceevent prefix in warning message
tracing: fix regression of perf function tracing
include/linux/ftrace_event.h | 38 ++++++++++
include/linux/trace_array.h | 118 +++++++++++++++++++++++++++++
include/trace/ftrace.h | 71 ++++++------------
include/trace/syscall.h | 1 +
kernel/trace/ftrace.c | 7 +-
kernel/trace/trace.c | 27 +++----
kernel/trace/trace.h | 143 +-----------------------------------
kernel/trace/trace_events.c | 55 ++++++++++++++
kernel/trace/trace_syscalls.c | 51 ++++++-------
tools/lib/traceevent/event-parse.c | 2 +-
10 files changed, 277 insertions(+), 236 deletions(-)
create mode 100644 include/linux/trace_array.h
--
1.7.9.7
^ permalink raw reply [flat|nested] 3+ messages in thread* [PATCH 14/14] tracing: fix regression of perf function tracing 2013-03-27 9:48 [PATCH 00/14] event tracing expose change and bugfix/cleanup zhangwei(Jovi) @ 2013-03-27 9:48 ` zhangwei(Jovi) 2013-03-27 9:48 ` zhangwei(Jovi) 1 sibling, 0 replies; 3+ messages in thread From: zhangwei(Jovi) @ 2013-03-27 9:48 UTC (permalink / raw) To: Steven Rostedt, Frederic Weisbecker, Ingo Molnar, LKML; +Cc: zhangwei(Jovi) From: "zhangwei(Jovi)" <jovi.zhangwei@huawei.com> Using perf command: perf stat -e ftrace:function ls this will cause kernel warning and oops. [ 797.828904] ------------[ cut here ]------------ [ 797.828946] WARNING: at include/linux/ftrace.h:209 ftrace_ops_control_func+0xb1/0xc0() [ 797.829065] Pid: 6086, comm: perf Not tainted 3.8.0-rc4+ #88 [ 797.829066] Call Trace: [ 797.829078] [<c0447a42>] warn_slowpath_common+0x72/0xa0 [ 797.829080] [<c04bd6d1>] ? ftrace_ops_control_func+0xb1/0xc0 [ 797.829082] [<c04bd6d1>] ? ftrace_ops_control_func+0xb1/0xc0 [ 797.829083] [<c04b7453>] ? synchronize_sched+0x3/0x50 [ 797.829085] [<c04be50f>] ? __unregister_ftrace_function+0x8f/0x170 [ 797.829087] [<c0447a92>] warn_slowpath_null+0x22/0x30 [ 797.829089] [<c04bd6d1>] ftrace_ops_control_func+0xb1/0xc0 [ 797.829099] [<c08eba6b>] ftrace_call+0x5/0xb [ 797.829100] [<c04b7458>] ? synchronize_sched+0x8/0x50 [ 797.829102] [<c04be50f>] __unregister_ftrace_function+0x8f/0x170 [ 797.829104] [<c04c00cf>] unregister_ftrace_function+0x1f/0x50 [ 797.829109] [<c04d31bd>] perf_ftrace_event_register+0x9d/0x140 [ 797.829111] [<c04d304b>] perf_trace_destroy+0x2b/0x50 [ 797.829117] [<c04db3c8>] tp_perf_event_destroy+0x8/0x10 [ 797.829119] [<c04dd672>] free_event+0x42/0x110 [ 797.829121] [<c04de446>] perf_event_release_kernel+0x56/0x90 [ 797.829122] [<c04de4fc>] put_event+0x7c/0xa0 [ 797.829124] [<c04de5cb>] perf_release+0xb/0x10 [ 797.829128] [<c0532ae6>] __fput+0xc6/0x1f0 [ 797.829130] [<c0532c1d>] ____fput+0xd/0x10 [ 797.829134] [<c04645b1>] task_work_run+0x81/0xa0 [ 797.829142] [<c0412819>] do_notify_resume+0x59/0x90 [ 797.829150] [<c08e4745>] work_notifysig+0x30/0x37 [ 797.829152] ---[ end trace 4dbd63f12b55163f ]--- This bug was introduced by below commit(in 3.8-rc4): commit 0a016409e42f273415f8225ddf2c58eb2df88034 Author: Steven Rostedt <srostedt@redhat.com> Date: Fri Nov 2 17:03:03 2012 -0400 ftrace: Optimize the function tracer list loop When variable op is ftrace_list_end, it cannot pass control ops checking, so that loop optimize is not suit for ftrace_control_list, change it back. Signed-off-by: zhangwei(Jovi) <jovi.zhangwei@huawei.com> --- kernel/trace/ftrace.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/kernel/trace/ftrace.c b/kernel/trace/ftrace.c index 2577082..2899974 100644 --- a/kernel/trace/ftrace.c +++ b/kernel/trace/ftrace.c @@ -4185,11 +4185,14 @@ ftrace_ops_control_func(unsigned long ip, unsigned long parent_ip, */ preempt_disable_notrace(); trace_recursion_set(TRACE_CONTROL_BIT); - do_for_each_ftrace_op(op, ftrace_control_list) { + op = rcu_dereference_raw(ftrace_control_list); + while (op != &ftrace_list_end) { if (!ftrace_function_local_disabled(op) && ftrace_ops_test(op, ip)) op->func(ip, parent_ip, op, regs); - } while_for_each_ftrace_op(op); + + op = rcu_dereference_raw(op->next); + } trace_recursion_clear(TRACE_CONTROL_BIT); preempt_enable_notrace(); } -- 1.7.9.7 ^ permalink raw reply related [flat|nested] 3+ messages in thread
* [PATCH 14/14] tracing: fix regression of perf function tracing 2013-03-27 9:48 [PATCH 00/14] event tracing expose change and bugfix/cleanup zhangwei(Jovi) 2013-03-27 9:48 ` [PATCH 14/14] tracing: fix regression of perf function tracing zhangwei(Jovi) @ 2013-03-27 9:48 ` zhangwei(Jovi) 1 sibling, 0 replies; 3+ messages in thread From: zhangwei(Jovi) @ 2013-03-27 9:48 UTC (permalink / raw) To: Steven Rostedt, Frederic Weisbecker, Ingo Molnar, LKML Cc: zhangwei(Jovi), stable From: "zhangwei(Jovi)" <jovi.zhangwei@huawei.com> Using perf command: perf stat -e ftrace:function ls this will cause kernel warning and oops. [ 797.828904] ------------[ cut here ]------------ [ 797.828946] WARNING: at include/linux/ftrace.h:209 ftrace_ops_control_func+0xb1/0xc0() [ 797.829065] Pid: 6086, comm: perf Not tainted 3.8.0-rc4+ #88 [ 797.829066] Call Trace: [ 797.829078] [<c0447a42>] warn_slowpath_common+0x72/0xa0 [ 797.829080] [<c04bd6d1>] ? ftrace_ops_control_func+0xb1/0xc0 [ 797.829082] [<c04bd6d1>] ? ftrace_ops_control_func+0xb1/0xc0 [ 797.829083] [<c04b7453>] ? synchronize_sched+0x3/0x50 [ 797.829085] [<c04be50f>] ? __unregister_ftrace_function+0x8f/0x170 [ 797.829087] [<c0447a92>] warn_slowpath_null+0x22/0x30 [ 797.829089] [<c04bd6d1>] ftrace_ops_control_func+0xb1/0xc0 [ 797.829099] [<c08eba6b>] ftrace_call+0x5/0xb [ 797.829100] [<c04b7458>] ? synchronize_sched+0x8/0x50 [ 797.829102] [<c04be50f>] __unregister_ftrace_function+0x8f/0x170 [ 797.829104] [<c04c00cf>] unregister_ftrace_function+0x1f/0x50 [ 797.829109] [<c04d31bd>] perf_ftrace_event_register+0x9d/0x140 [ 797.829111] [<c04d304b>] perf_trace_destroy+0x2b/0x50 [ 797.829117] [<c04db3c8>] tp_perf_event_destroy+0x8/0x10 [ 797.829119] [<c04dd672>] free_event+0x42/0x110 [ 797.829121] [<c04de446>] perf_event_release_kernel+0x56/0x90 [ 797.829122] [<c04de4fc>] put_event+0x7c/0xa0 [ 797.829124] [<c04de5cb>] perf_release+0xb/0x10 [ 797.829128] [<c0532ae6>] __fput+0xc6/0x1f0 [ 797.829130] [<c0532c1d>] ____fput+0xd/0x10 [ 797.829134] [<c04645b1>] task_work_run+0x81/0xa0 [ 797.829142] [<c0412819>] do_notify_resume+0x59/0x90 [ 797.829150] [<c08e4745>] work_notifysig+0x30/0x37 [ 797.829152] ---[ end trace 4dbd63f12b55163f ]--- This bug was introduced by below commit(in 3.8-rc4): commit 0a016409e42f273415f8225ddf2c58eb2df88034 Author: Steven Rostedt <srostedt@redhat.com> Date: Fri Nov 2 17:03:03 2012 -0400 ftrace: Optimize the function tracer list loop When variable op is ftrace_list_end, it cannot pass control ops checking, so that loop optimize is not suit for ftrace_control_list, change it back. Signed-off-by: zhangwei(Jovi) <jovi.zhangwei@huawei.com> Cc: stable@vger.kernel.org --- kernel/trace/ftrace.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/kernel/trace/ftrace.c b/kernel/trace/ftrace.c index 2577082..2899974 100644 --- a/kernel/trace/ftrace.c +++ b/kernel/trace/ftrace.c @@ -4185,11 +4185,14 @@ ftrace_ops_control_func(unsigned long ip, unsigned long parent_ip, */ preempt_disable_notrace(); trace_recursion_set(TRACE_CONTROL_BIT); - do_for_each_ftrace_op(op, ftrace_control_list) { + op = rcu_dereference_raw(ftrace_control_list); + while (op != &ftrace_list_end) { if (!ftrace_function_local_disabled(op) && ftrace_ops_test(op, ip)) op->func(ip, parent_ip, op, regs); - } while_for_each_ftrace_op(op); + + op = rcu_dereference_raw(op->next); + } trace_recursion_clear(TRACE_CONTROL_BIT); preempt_enable_notrace(); } -- 1.7.9.7 ^ permalink raw reply related [flat|nested] 3+ messages in thread
end of thread, other threads:[~2013-03-28 0:17 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
[not found] <1364377499-1900-1-git-send-email-jovi.zhangwei@huawei.com>
[not found] ` <1364377499-1900-15-git-send-email-jovi.zhangwei@huawei.com>
[not found] ` <1364389775.6345.210.camel@gandalf.local.home>
2013-03-28 0:17 ` [PATCH 14/14] tracing: fix regression of perf function tracing zhangwei(Jovi)
2013-03-27 9:48 [PATCH 00/14] event tracing expose change and bugfix/cleanup zhangwei(Jovi)
2013-03-27 9:48 ` [PATCH 14/14] tracing: fix regression of perf function tracing zhangwei(Jovi)
2013-03-27 9:48 ` zhangwei(Jovi)
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox