Re: [PATCH 0/11] LTTng-core (basic tracing infrastructure) 0.5.108

[Ingo Molnar <mingo@elte.hu>]

* Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca> wrote:

> > > > sorry, but i disagree. There _is_ a solution that is superior in 
> > > > every aspect: kprobes + SystemTap. (or any other equivalent 
> > > > dynamic tracer)
> > > > 
> > > 
> > > I am sorry to have to repeat myself, but this is not true for 
> > > heavy loads.
> > 
> > djprobes?
> > 
> 
> I am fully aware of djprobes limitations towards fully preemptible 
> kernel [...]

i dont see any fundamental limitation with a preemptible kernel. 
(preemptability was never a showstopper for any kernel feature in the 
past, and i dont expect it to be a showstopper for anything in the 
future either.)

> [...] (and around branches instructions ? I don't remember if they 
> solved this one). Oh, yes, and if a trap happen to come at the wrong 
> spot, then the thread gets scheduled out... well, it cannot be applied 
> everywhere, eh ?

i expect the number of places where dynamic tracers have problems to 
gradually shrink. It has shrunk significantly already. Hence i'm 
supportive of static markers (as i stated it numerous times), as long as 
it's there to ease dynamic probing - _and as long as these static 
markers shrink in number as the capabilities of dynamic tracers 
improve_. With static tracers i just dont see that possibility: a static 
tracer needs all its static tracepoints forever or otherwise it just 
wont work.

> >  $ grep "\<trace_.*(" * | wc -l
> >  359
> > 
> 
> This count includes the inline trace functions definitions.

yes, as i stated:

> > some of those are not true tracepoints, but there's at least this many 
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> > of them:
> > 
> >  $ grep "\<trace_.*(" *instrumentation* | wc -l
> >  235
> > 
> 
> 1 - This counts per architecture trace points. It quickly adds up 
> considering that we support ARM, MIPS, i386, powerpc, ppc and x86_64.

yes. That's my point: overhead of static tracepoints "quickly adds up". 
The cost goes up linearly, as you grow into more subsystems and into 
more architectures.

btw., an observation: that's 6 LTT architectures in 7 years, while 
kprobes are now on 5 architectures in 2 years.

> 2 - It also counts some experimental trace points that I do not want 
> to submit.
> 3 - Most of these are instrumentation of the traps handlers, which is 
> conceptually only one event.

i counted the number of tracepoints, not the number of unique types of 
events, because:

> > when judging kernel maintainance overhead, the sum of all patches 
> > matters. And i considered all the other patches too (the ones that 
> > add actual tracepoints) that will come after the currently offered 
> > ones, not just the ones you submitted to lkml.
> 
> I plan to rework the instrumentation patches before submitting them to 
> LKML, don't worry. I just hasn't been my focus until now. Too bad that 
> you take those as arguments.

the static tracer patches make little sense without instrumentation, so 
sure i considered them. I also clearly declared that you didnt submit 
them yet:

>>> Let me quote from the latest LTT patch (patch-2.6.17-lttng-0.5.108, 
>>> which is the same version submitted to lkml - although no specific 
                                                  ^^^^^^^^^^^^^^^^^^^^
>>> tracepoints were submitted):
    ^^^^^^^^^^^^^^^^^^^^^^^^^^

	Ingo