Re: [PATCH 0/11] LTTng-core (basic tracing infrastructure) 0.5.108

Ingo Molnar
2006-09-14T17:51:48Z

* Martin J. Bligh <mbligh@mbligh.org> wrote:

> >>Comments and reviews are very welcome.
> >
> > i have one very fundamental question: why should we do this
> > source-intrusive method of adding tracepoints instead of the
> > dynamic, unintrusive (and thus zero-overhead) KProbes+SystemTap
> > method?
>
> Because:
>
> 1. Kprobes are more overhead when they *are* being used.

minimally so - at least on i386 and x86_64. In that sense tracing is a
_slowpath_, and it _will_ slow things down if done excessively. I dont
care about the tracepoint being slower by a few instructions as long as
it has _zero effect_ on normal code, be that source code or binary code.

> 2. You can get zero overhead by CONFIG'ing things out.

but that's not how a fair chunk of people want to use tracing. People
(enterprise customers trying to figure out performance problems,
engineers trying to debug things on a live, production system) want to
be able to insert a tracepoint anywhere and anytime - and also they want
to have zero overhead from tracing if no tracepoints are used on a
system.

> 3. (most importantly) it's a bitch to maintain tracepoints out
> of-tree on a rapidly moving kernel

wrong: the original demo tracepoints that came with SystemTap still work
on the current kernel, because the 'coupling' is loose: based on
function names.

Static tracepoints on the other hand, if added via an external patch, do
depend on the target function not moving around and the context of the
tracepoint not being changed. (and static tracepoints if in the source
all the time are a constant hindrance to development and code
readability.)

and of course the big advantage of dynamic probing is its flexibility:
you can add add-hoc tracepoints to thousands of functions, instead of
having to maintain hundreds (or thousands) of static tracepoints all the
time. (and if we wont end up with hundreds/thousands of static
tracepoints then it wont be usable enough as a generic solution.)

> 4. I believe kprobes still doesn't have full access to local
> variables.

wrong: with SystemTap you can probe local variables too (via
jprobes/kretprobes, all in the upstream kernel already).

> Now (3) is possibly solvable by putting the points in as no-ops
> (either insert a few nops or just a marker entry in the symbol
> table?), but full dynamic just isn't sustainable. [...]

i'm not sure i follow. Could you explain where SystemTap has this
difficulty?

Ingo