Re: [patch 10/10] Scheduler profiling - Use immediate values

[Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>]

* Adrian Bunk (bunk@stusta.de) wrote:
> On Fri, Jul 06, 2007 at 07:43:15PM -0400, Mathieu Desnoyers wrote:
> > * Adrian Bunk (bunk@stusta.de) wrote:
> > > On Fri, Jul 06, 2007 at 06:14:10PM -0400, Chuck Ebbert wrote:
> > > > On 07/06/2007 07:44 AM, Andi Kleen wrote:
> > > > > I think the optimization is a good idea, although i dislike it
> > > > > that it is complicated for the dynamic markers. If it was just
> > > > > static it would be much simpler.
> > > > 
> > > > Another thing to consider is that there might be hundreds of these
> > > > probes/tracepoints active in an instrumented kernel. The overhead
> > > > adds up fast, so the gain may be worth all the pain.
> > > 
> > > Only if you want to squeeze the last bit of performance out of
> > > _debugging_ functionality.
> > > 
> > > You avoid all the pain if you simply don't use debugging functionality 
> > > on production systems.
> > > 
> > 
> > Adrian,
> > 
> > Please have a look at my markers posts, especially:
> > 
> > http://www.ussg.iu.edu/hypermail/linux/kernel/0707.0/0669.html
> > 
> > And also look into OLS 2007 proceedings for Martin Bligh's paper on
> > Debugging Google sized clusters. It basically makes the case for adding
> > functionnality to debug _user space_ problems on production systems that
> > can be turned on dynamically.
> 
> Using a different kernel for tracing still fulfills all the requirements 
> listed in section 5 of your paper...
> 

Not exactly. I assume you understand that rebooting 1000 live production
servers to find the source of a rare bug or the cause of a performance
issue is out of question.

Moreover, strategies like enabling flight recorder traces on a few nodes
on demand to detect performance problems can only be deployed in
production environment if they are part of the standard production
kernel.

Also, managing two different kernels is often out of question. Not only
is it a maintainance burden, but just switching to the "debug" kernel
can impact the system's behavior so badly that it could make the problem
disappear.

Mathieu

-- 
Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68