[RFC] ftrace: support tracing functions in one module

[RFC] ftrace: support tracing functions in one module

[Ming Lei]

Hi,

IMO, ftrace is a very good tools, which can monitor almost all
functions calling
in the running kernel. The traced result is very complete and intact.
But it seems
too large to grasp the interested content.  For example, one may only
have interest in
functions calling in usbcore.ko, but he must trace all the functions
calling in the
kernel, so the tracing result is too large to use it.

Could you add the support of tracing functions in one module only to ftrace?

Thanks,

Re: [RFC] ftrace: support tracing functions in one module

[Frank Ch. Eigler]

"Ming Lei" <tom.leiming@gmail.com> writes:

> [...]  For example, one may only have interest in functions calling
> in usbcore.ko, but he must trace all the functions calling in the
> kernel, so the tracing result is too large to use it. [...]

Until ftrace gains this ability, you could use systemtap:

# cat callret.stp
probe $1.call { printf ("%s <- %s\n", thread_indent(1), probefunc()) }
probe $1.return { printf ("%s -> %s\n", thread_indent(-1), probefunc()) }
# stap callret.stp 'module("usbcore").function("*")'
[...]
^C
#

(One can also add conditions based on function parameters, processes,
whatever; soon it'll be easy to trace all function parameters.)


- FChE

Re: [RFC] ftrace: support tracing functions in one module

[Steven Rostedt]

On Sun, 3 Aug 2008, Ming Lei wrote:

> Hi,
> 
> IMO, ftrace is a very good tools, which can monitor almost all
> functions calling
> in the running kernel. The traced result is very complete and intact.
> But it seems
> too large to grasp the interested content.  For example, one may only
> have interest in
> functions calling in usbcore.ko, but he must trace all the functions
> calling in the
> kernel, so the tracing result is too large to use it.
> 
> Could you add the support of tracing functions in one module only to ftrace?

Look at the set_ftrace_filter in ftrace.txt. You can pick and choose which 
functions to trace.  All the functions that can be traced is in 
available_filter_functions in the debugfs/tracing directory.

Note, the functions must execute at least once before they can be 
recorded.

-- Steve

Re: [RFC] ftrace: support tracing functions in one module

[Abhishek Sagar]

On Sun, Aug 3, 2008 at 1:20 PM, Ming Lei <tom.leiming@gmail.com> wrote:
> Could you add the support of tracing functions in one module only to ftrace?

You can do that at compile time only by selective compilation of
files/modules which need to be traced with the -pg CFLAG. For
instance, the inlined patch enables tracing of all functions in
softirq.o only.

Regards,
Abhishek Sagar

--
## For testing only

diff --git a/Makefile b/Makefile
index 40f2481..09d216b 100644
--- a/Makefile
+++ b/Makefile
@@ -532,10 +532,6 @@ KBUILD_CFLAGS      += -g
 KBUILD_AFLAGS  += -gdwarf-2
 endif

-ifdef CONFIG_FTRACE
-KBUILD_CFLAGS  += -pg
-endif
-
 # We trigger additional mismatches with less inlining
 ifdef CONFIG_DEBUG_SECTION_MISMATCH
 KBUILD_CFLAGS += $(call cc-option, -fno-inline-functions-called-once)
diff --git a/kernel/Makefile b/kernel/Makefile
index 82f1f3f..c099be3 100644
--- a/kernel/Makefile
+++ b/kernel/Makefile
@@ -14,14 +14,8 @@ obj-y     = sched.o fork.o exec_domain.o panic.o printk.o \
 CFLAGS_REMOVE_sched.o = -mno-spe

 ifdef CONFIG_FTRACE
-# Do not trace debug files and internal ftrace files
-CFLAGS_REMOVE_lockdep.o = -pg
-CFLAGS_REMOVE_lockdep_proc.o = -pg
-CFLAGS_REMOVE_mutex-debug.o = -pg
-CFLAGS_REMOVE_rtmutex-debug.o = -pg
-CFLAGS_REMOVE_cgroup-debug.o = -pg
-CFLAGS_REMOVE_sched_clock.o = -pg
-CFLAGS_REMOVE_sched.o = -mno-spe -pg
+CFLAGS_softirq.o = -pg
+CFLAGS_REMOVE_sched.o = -mno-spe
 endif

 obj-$(CONFIG_PROFILING) += profile.o

Re: [RFC] ftrace: support tracing functions in one module

[Ming Lei]

2008/8/3 Steven Rostedt <rostedt@goodmis.org>:
>
> On Sun, 3 Aug 2008, Ming Lei wrote:
>
>> Hi,
>>
>> IMO, ftrace is a very good tools, which can monitor almost all
>> functions calling
>> in the running kernel. The traced result is very complete and intact.
>> But it seems
>> too large to grasp the interested content.  For example, one may only
>> have interest in
>> functions calling in usbcore.ko, but he must trace all the functions
>> calling in the
>> kernel, so the tracing result is too large to use it.
>>
>> Could you add the support of tracing functions in one module only to ftrace?
>
> Look at the set_ftrace_filter in ftrace.txt. You can pick and choose which
> functions to trace.  All the functions that can be traced is in

It seems not ver easy to opearte to trace all functions in a module. You need to
write all function names to set_ftrace_filter. Also some functions have
same names in kernel and modules. This can lead to some messed trace result.

Do you have the plan to support tracing functions in one module or in one kernel
address range?

> available_filter_functions in the debugfs/tracing directory.
>
> Note, the functions must execute at least once before they can be
> recorded.
>
> -- Steve
>
>



-- 
Lei Ming

Re: [RFC] ftrace: support tracing functions in one module

[Arnaldo Carvalho de Melo]

Em Mon, Aug 04, 2008 at 10:11:17AM +0800, Ming Lei escreveu:
> 2008/8/3 Steven Rostedt <rostedt@goodmis.org>:
> >
> > On Sun, 3 Aug 2008, Ming Lei wrote:
> >
> >> Hi,
> >>
> >> IMO, ftrace is a very good tools, which can monitor almost all
> >> functions calling
> >> in the running kernel. The traced result is very complete and intact.
> >> But it seems
> >> too large to grasp the interested content.  For example, one may only
> >> have interest in
> >> functions calling in usbcore.ko, but he must trace all the functions
> >> calling in the
> >> kernel, so the tracing result is too large to use it.
> >>
> >> Could you add the support of tracing functions in one module only to ftrace?
> >
> > Look at the set_ftrace_filter in ftrace.txt. You can pick and choose which
> > functions to trace.  All the functions that can be traced is in
> 
> It seems not ver easy to opearte to trace all functions in a module. You need to
> write all function names to set_ftrace_filter. Also some functions have
> same names in kernel and modules. This can lead to some messed trace result.
> 
> Do you have the plan to support tracing functions in one module or in one kernel
> address range?

What about using:

[acme@doppio pahole]$ nm --defined-only /usr/lib/debug/lib/modules/2.6.24.7-75.el5rt/kernel/drivers/net/tg3.ko.debug | grep ' t ' | cut -d' ' -f3 | head 
__tg3_set_coalesce
__tg3_set_mac_addr
__tg3_set_rx_mode
_tw32_flush
tg3_5700_link_polarity
tg3_abort_hw
tg3_alloc_rx_skb
tg3_ape_driver_state_change
tg3_bmcr_reset
tg3_change_mtu
[acme@doppio pahole]$

Feed this to /sys/kernel/debug/tracing/set_ftrace_filter and you should
be set, no?

Ok, you need to have the kernel-debuginfo package installed, and I guess
we can get away with that, but for now, isn't that enough?

- Arnaldo

Re: [RFC] ftrace: support tracing functions in one module

[Steven Rostedt]

On Mon, 4 Aug 2008, Arnaldo Carvalho de Melo wrote:
> > 
> > Do you have the plan to support tracing functions in one module or in one kernel
> > address range?
> 
> What about using:
> 
> [acme@doppio pahole]$ nm --defined-only /usr/lib/debug/lib/modules/2.6.24.7-75.el5rt/kernel/drivers/net/tg3.ko.debug | grep ' t ' | cut -d' ' -f3 | head 
> __tg3_set_coalesce
> __tg3_set_mac_addr
> __tg3_set_rx_mode
> _tw32_flush
> tg3_5700_link_polarity
> tg3_abort_hw
> tg3_alloc_rx_skb
> tg3_ape_driver_state_change
> tg3_bmcr_reset
> tg3_change_mtu
> [acme@doppio pahole]$
> 
> Feed this to /sys/kernel/debug/tracing/set_ftrace_filter and you should
> be set, no?
> 
> Ok, you need to have the kernel-debuginfo package installed, and I guess
> we can get away with that, but for now, isn't that enough?


Or simply (again for tg3 :


# awk '/\[tg3\]/ { print $3 }' /proc/kallsyms  \
   > /debug/tracing/set_ftrace_filter

-- Steve

Re: [RFC] ftrace: support tracing functions in one module

[Arnaldo Carvalho de Melo]

Em Mon, Aug 04, 2008 at 12:28:12PM -0400, Steven Rostedt escreveu:
> 
> On Mon, 4 Aug 2008, Arnaldo Carvalho de Melo wrote:
> > > 
> > > Do you have the plan to support tracing functions in one module or in one kernel
> > > address range?
> > 
> > What about using:
> > 
> > [acme@doppio pahole]$ nm --defined-only /usr/lib/debug/lib/modules/2.6.24.7-75.el5rt/kernel/drivers/net/tg3.ko.debug | grep ' t ' | cut -d' ' -f3 | head 
> > __tg3_set_coalesce
> > __tg3_set_mac_addr
> > __tg3_set_rx_mode
> > _tw32_flush
> > tg3_5700_link_polarity
> > tg3_abort_hw
> > tg3_alloc_rx_skb
> > tg3_ape_driver_state_change
> > tg3_bmcr_reset
> > tg3_change_mtu
> > [acme@doppio pahole]$
> > 
> > Feed this to /sys/kernel/debug/tracing/set_ftrace_filter and you should
> > be set, no?
> > 
> > Ok, you need to have the kernel-debuginfo package installed, and I guess
> > we can get away with that, but for now, isn't that enough?
> 
> 
> Or simply (again for tg3 :
> 
> 
> # awk '/\[tg3\]/ { print $3 }' /proc/kallsyms  \
>    > /debug/tracing/set_ftrace_filter

For the drivers that do proper namespacing, yes, that should be enough
in most cases :)

- Arnaldo