Re: [PATCH 0/11] LTTng-core (basic tracing infrastructure) 0.5.108

Ingo Molnar
2006-09-16T08:32:26Z

* Roman Zippel <zippel@linux-m68k.org> wrote:

> > > Why don't you leave the choice to the users? Why do you constantly
> > > make it an exclusive choice? [...]
> >
> > as i outlined it tons of times before: once we add markups for static
> > tracers, we cannot remove them. That is a constant kernel maintainance
> > drag that i feel uncomfortable about.
>
> As many, many people have already said, any tracepoints have an
> maintainance overhead, which is barely different between dynamic and
> static tracing and only increases the further away the tracepoints are
> from the source.

i have demonstrated that with dynamic tracers it's possible to have:
"half the number of tracepoints" or "no tracepoints at all", right in
the traced kernel source. That way we are able to shift away the
maintainance overhead from the subsystem which is being traced to the
person who _wants_ to do the tracing (instead of on the person who
maintains the code that is being traced), in a finegrained way.

But even the secondary metric, the "sum of all maintainance, including
the maintanance of tracepoints" can become lower with dynamic tracers:
if a subsystem changes with a much higher frequency than the tracing
scripts follow.

Let me try to explain it to you with other words: if all tracing is done
via scripts and no in-source tracepoints at all, then we could for
example update the tracing scripts only once per release. A subsystem
might undergo a heavy cycle of updates, changing functions that are
traced many times: i call this a "high frequency update to the source
code".

If tracing is done via tracepoints for static tracers, then such "high
frequency updates to the source code" have to "carry with them" all the
markups. It might be zero overhead if a subsystem has no tracepoints,
but it might be alot more complex too.

For example, I can tell you that the -rt tree has a number of very
useful scheduling tracepoints but which are also a constant maintainance
hindrance. For example i even have a separate _function_ that is a
helper to one of the tracepoints. And this was the _bare minimum_ of
static tracepoints i needed for the purposes of visualizing and
analyzing scheduling patterns in the -rt tree (either on my boxes or on
users' boxes). Occasionally users needed alot more tracepoints. So i am
talking from first-hand experience. This maintainance overhead occured
(and still occurs) to /me/, so please dont try to tell me that the
maintainance overhead is minimal. Even "half the tracepoints" would be
great. And i only have a dozen tracepoints, not hundreds!

Ingo