Re: 2.6.22 -mm merge plans

[Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>]

Hi Andi,

* Andi Kleen (andi@firstfloor.org) wrote:
> Andrew Morton <akpm@linux-foundation.org> writes:
> 
> 
> > Static markers.  Will merge.
> There don't seem to be any users of this. How do you know it hasn't
> already bitrotted?
> 

See the detailed explanation at :
http://kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.21-rc7/2.6.21-rc7-mm2/broken-out/linux-kernel-markers-kconfig-menus.patch

Major points :

It is currently used as an instrumentation infrastructure for the LTTng
tracer at IBM, Google, Autodesk, Sony, MontaVista and deployed in
WindRiver products.  The SystemTAP project also plan to use this type of
infrastructure to trace sites hard to instrument. The Linux Kernel
Markers has the support of Frank C. Eigler, author of their current
marker alternative (which he wishes to drop in order to adopt the
markers infrastructure as soon as it hits mainline).

Quoting Jim Keniston <jkenisto@us.ibm.com> :

"kprobes remains a vital foundation for SystemTap.  But markers are
attactive as an alternate source of trace/debug info.  Here's why:
[...]"

> It seems quite overcomplicated to me. Has the complexity been justified?
> 

To summarize the document pointed at the URL above, where the full
the key goals of the markers, showing the rationale being the most
important design choices :
- Almost non perceivable impact on production machines when compiled in
  but markers are "disabled".
  - Use a separate section to keep the data to minimize d-cache
    trashing.
  - Put the code (stack setup and function call) in unlikely branches of the
    if() condition to minimize i-cache impact.
  - Since it is required to allow instrumentation of variables within
    the body of a function, accept the impact on compiler's
    optimizations and let it keep the variables "live" sometimes longer
    than required. It is up to the person who puts the marker in the
    code to choose the location that will have a small impact in this
    aspect.
  - Allow per-architecture optimized versions which removes the need for
    a d-cache based branch (patch a "load immediate" instruction
    instead). It minimized the d-cache impact of the disabled markers.
  - Accept the cost of an unlikely branch at the marker site because the
    gcc compiler does not give the ability to put "nops" instead of a
    branch generated from C code. Keep this in mind for future
    per-architecture optimizations.
- Instrumentation of challenging kernel sites
  - Instrumentation such as the one provided in the already existing
    Lock dependency checker (lockdep) and instrumentation of trap
    handlers implies being reentrant for such context. Therefore, the
    implementation must be lock-free and update the state in an atomic
    fashion (rcu-style). It must also let the programmer who describes
    a marker site the ability to specify what is forbidden in the probe
    that will be connected to the marker : can it generate a trap ? Can
    it call lockdep (irq disable, take any type of lock), can it call
    printk ? This is why flags can be passed to the _MARK() marker,
    while the MARK() marker has the default flags.

Please tell me if I forgot to explain the rationale behind some
implementation detail and I will be happy to explain in more depth.

Regards,

Mathieu


-- 
Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68