Re: [RFC] Create instrumentation directory (git repository)

[Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>]

* Jeff Garzik (jeff@garzik.org) wrote:
> Mathieu Desnoyers wrote:
> >I see no good reason to have so many different adhoc instrumentation
> >mechanisms for profiling (sched, vm, oprofile) and tracing (blktrace,
> >suspend/resume tracing) all over the place. Merging them in a single
> >directory seems like a good step towards a more generic
> >instrumentation/profiling/tracing infrastructure.
> 
> Moving files about in directories should be at the /lowest/ end of the 
> priority scale.  It makes diffs unreadable, file histories and diffing 
> difficult, and a host of other problems.
> 
> Please solve the /real/ problems, and then come back and clean up the 
> file structure after that is done.  Massive file renaming to satisfying 
> some imagined future everything-is-golden scheme is the /last/ step.  It 
> is the last step taken because the previous steps inevitably give you 
> guidance that you otherwise would not have had at the start of the task.
> 
> When I try to diff between old and new alpha oprofile code, I really 
> want to know that the reason why diffing is a pain in the ass is more 
> than "it seemed like a good first step."
> 

And how is this confirmed by the way the i386-x86_64 -> x86 merge is
done ? It seems like a good current counter-example of what you just
affirmed.

First organizing the functionally similar existing code into a single
placeholder will just help finding code duplication, just like two very
similar architectures such as i386 and x86_64.

Talking about solving "real" problems, this is what I have been working
on for about 3 years in the kernel tracing area, writing the LTTng
tracer. What I see at this point is that there is a strong interest for
collaboration between the instrumentation projects (LTTng, SystemTAP,
DTI), but since the code ends up being sprinkled all across the kernel,
it's rather hard to spot duplicates. Actually, I just ran into Linus's
suspend/resume tracer _today_.

Talking about solving real problems, this is also what I did with the
Linux Kernel Markers patch, which can now be used to instrument the
kernel code. But it only deals with one aspect of instrumentation: the
markup itself.

I would categorize what we need for instrumentation in the following
categories :

- Data identification
  * static markup, enabled dynamically, very low impact
  * dynamic markup
  * oprofile (especially for the performance counters)
  * stack traces
- Control
  * Tracing management
  * Profiling management
  * PMC management
- Data extraction
  * relay
  * debugfs
  * serial port output
  * LKCD

What I consider to fit into the instrumentation directory is the data
identification and the control mechanisms. The data extraction should be
done be generic pieces of infrastructure already present in the kernel.

Your suggestion of "first fixing the real problems" (do you mean by
this : add new code ?) and later bother about the file structure just
seems to go against most suggestions I have received from kernel
developers in the past years. Getting something new in the kernel is
much more straightforward if someone is willing to first clean up the
mess (I am quoting Thomas Gleixner here).  So, in this particular case,
addressing the real problem : people out there want a tracer in the
Linux kernel, will first require to clean up the currently existing
mess : overlapping instrumentation code sprinkled all over the place.


> 
> >Back to "profile" and "probes" directory names, they might be short, but
> >they do not represent the whole markup-profiling-tracing trio,
> >"profile" lacks the tracing part and "probe" lacks the markup part.
> 
> You can always add more letters (and words) to even reach the desired 
> level of specificity.  That does nothing to help readability though.
> 
> Anyway, it should be clear from existing precedent -- existing pathnames 
> -- that "instrumentation" is too long, and really IMO too vague anyway.
> 

I guess you don't use the Documentation/ directory often then. ;)

How about i13n ? :) Jokes aside, I could live with "probe", although it
doesn't fit the purpose exactly. Getting the perfect name, to me, come
second after the need to group those files.

Mathieu


-- 
Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68