Re: tracepoint maintainance models

[Ingo Molnar <mingo@elte.hu>]

* Nicholas Miell <nmiell@comcast.net> wrote:

> On my system, Solaris has 49 "real" static probes (with actual 
> documentation[1]). They are as follows:

yeah, _some_ static markers are OK, as long as they are within a dynamic 
tracing framework! (and are thus constantly "kept in check" by the easy 
availability of dynamic probes)

what is being proposed here is entirely different from dprobes though: 
Roman suggests that he doesnt want to implement kprobes on his arch, and 
he wants LTT to remain an _all-static_ tracer. That's the point where i 
beg to differ: static markers are fine (but they should be kept to a 
minimum), but generic static /tracers/ need alot more than just a few 
static markers to be meaningful.

So if we accepted static tracers into the kernel, we'd automatically 
commit (for a long period of time) to a much larger body of static 
markers - and i'm highly uncomfortable about that. (for the many reasons 
outlined before)

Even if the LTT folks proposed to "compromise" to 50 tracepoints - users 
of static tracers would likely _not_ be willing to compromise, so there 
would be a constant (and I say unnecessary) battle going on for the 
increase of the number of static markers. Static markers, if done for 
static tracers, have "viral" (Roman: here i mean "auto-spreading", not 
"disease") properties in that sense - they want to spread to alot larger 
area of code than they start out from.

While if we only have a dynamic tracing framework (which is a mix of 
static markers and dynamic probes) then pretty much the only user 
pressure would be: "implement kprobes!". (which is already implemented 
for 5 major arches and takes only between 500 and 1000 lines of per-arch 
code for most of them.)

( furthermore, from what you've described it seems to me that 
  kprobes/kretprobes/djprobes+SystemTap is already more capable than 
  dprobes is - hence the number of static markes needed in Linux might 
  in fact be lower in the end than in Solaris. )

> This is the important part: In a dynamic tracing system, the number of 
> static probes necessary for the tracing system to be useful is 
> drastically, dramatically, absurdly lower than in a purely static 
> tracing system. Hell, you don't even need the static probes for it to 
> be useful, they're just a convenience for events which happen in 
> multiple places or a high-level name for a low-level implementation 
> detail.

yeah, precisely my point.

> In order for the static tracing system to be as useful as the dynamic 
> system, all of those dynamically generated probe points would have to 
> be manually added to the kernel. The maintenance burden of this number 
> of probes is stupidly high. In reality, no static system would ever 
> reach that level of coverage.

yeah, agreed.

	Ingo