Re: [PATCH 0/11] LTTng-core (basic tracing infrastructure) 0.5.108

[Ingo Molnar <mingo@elte.hu>]

* Roman Zippel <zippel@linux-m68k.org> wrote:

> > > Why don't you leave the choice to the users? Why do you constantly 
> > > make it an exclusive choice? [...]
> > 
> > as i outlined it tons of times before: once we add markups for static 
> > tracers, we cannot remove them. That is a constant kernel maintainance 
> > drag that i feel uncomfortable about.
> 
> As many, many people have already said, any tracepoints have an 
> maintainance overhead, which is barely different between dynamic and 
> static tracing and only increases the further away the tracepoints are 
> from the source.

i have demonstrated that with dynamic tracers it's possible to have: 
"half the number of tracepoints" or "no tracepoints at all", right in 
the traced kernel source. That way we are able to shift away the 
maintainance overhead from the subsystem which is being traced to the 
person who _wants_ to do the tracing (instead of on the person who 
maintains the code that is being traced), in a finegrained way.

But even the secondary metric, the "sum of all maintainance, including 
the maintanance of tracepoints" can become lower with dynamic tracers: 
if a subsystem changes with a much higher frequency than the tracing 
scripts follow.

Let me try to explain it to you with other words: if all tracing is done 
via scripts and no in-source tracepoints at all, then we could for 
example update the tracing scripts only once per release. A subsystem 
might undergo a heavy cycle of updates, changing functions that are 
traced many times: i call this a "high frequency update to the source 
code".

If tracing is done via tracepoints for static tracers, then such "high 
frequency updates to the source code" have to "carry with them" all the 
markups. It might be zero overhead if a subsystem has no tracepoints, 
but it might be alot more complex too.

For example, I can tell you that the -rt tree has a number of very 
useful scheduling tracepoints but which are also a constant maintainance 
hindrance. For example i even have a separate _function_ that is a 
helper to one of the tracepoints. And this was the _bare minimum_ of 
static tracepoints i needed for the purposes of visualizing and 
analyzing scheduling patterns in the -rt tree (either on my boxes or on 
users' boxes). Occasionally users needed alot more tracepoints. So i am 
talking from first-hand experience. This maintainance overhead occured 
(and still occurs) to /me/, so please dont try to tell me that the 
maintainance overhead is minimal. Even "half the tracepoints" would be 
great. And i only have a dozen tracepoints, not hundreds!

	Ingo