Re: [PATCH][GIT PULL] tracing: add function profiler

[Ingo Molnar <mingo@elte.hu>]

* Peter Zijlstra <peterz@infradead.org> wrote:

> On Sat, 2009-03-21 at 04:26 -0700, Andrew Morton wrote:
> > On Sat, 21 Mar 2009 00:37:59 -0400 (EDT) Steven Rostedt <rostedt@goodmis.org> wrote:
> > 
> > >    This patch adds a function profiler. In debugfs/tracing/ two new
> > >     files are created.
> > >     
> > >       function_profile_enabled  - to enable or disable profiling
> > >     
> > >       trace_stat/functions   - the profiled functions.
> > >     
> > >     For example:
> > >     
> > >       echo 1 > /debugfs/tracing/function_profile_enabled
> > >       ./hackbench 50
> > >       echo 0 > /debugfs/tracing/function_profile_enabled
> > >     
> > >     yields:
> > >     
> > >       cat /debugfs/tracing/trace_stat/functions
> > >     
> > >       Function                               Hit
> > >       --------                               ---
> > >       _spin_lock                        10106442
> > >       _spin_unlock                      10097492
> > >       kfree                              6013704
> > >       _spin_unlock_irqrestore            4423941
> > >       _spin_lock_irqsave                 4406825
> > >       __phys_addr                        4181686
> > >       __slab_free                        4038222
> > >       dput                               4030130
> > >       path_put                           4023387
> > >       unroll_tree_refs                   4019532
> > >     [...]
> > >     
> > >     The most hit functions are listed first. Functions that are not
> > >     hit are not listed.
> > 
> > Why is this useful?
> > 
> > Can we think of any scenarios where kernel developers would get
> > useful-to-them results from this?  Results which couldn't be 
> > obtained by other similarly-accessible means?
> > 
> > <strains a bit>
> > 
> > I guess that one could run workload A, look at
> > /debugfs/tracing/trace_stat/functions changes, then run worklaod B, then
> > look at its /debugfs/tracing/trace_stat/functions changes, then somehow
> > glean some information about the differences between the effects of the two
> > workloads on the kernel.  Or something.
> > 
> > But in this rather fake example and, I suspect, in many others, 
> > the result will be less useful than using oprofile/etc in the 
> > same fashion.
> 
> I have to agree with Andrew here, my plan is to remove all the 
> profiling stuff from kernel/trace in favour of perf counters.

i agree with that - but still it would be useful to also have the 
ability to do in-kernel histograms and the likely/unlikely profiler 
(which is using the same histogram code) is already using that 
facility.

But instead of the single-purpose likely/unlikely and now function 
histogram code, we should allow individual counters/events to be 
linked up with an in-kernel histogram - or route it to user-space 
via perfcounters.

> If you want exact function count profiling we could try to do 
> something perf counter based, eg. stick a software counter in the 
> mcount thingy.

while i agree in general - lets realize that function (and branch) 
software events _are_ special due to their sheer mass. I dont think 
it's feasible to route that much information to user-space - just to 
collapse it into a histogram there. Even doing it in the kernel is 
already straining the performance envelope quite a bit.

	Ingo