Re: Performance analysis of Linux Kernel Markers 0.20 for 2.6.17

["Jose R. Santos" <jrs@us.ibm.com>]

Mathieu Desnoyers wrote:
> However, the performance impact for using a kprobe is non negligible when
> activated. Assuming that kprobes would have a mechanism to get the variables
> from the caller's stack, it would perform the same task in at least 4178.23
> cycles vs 55.74 for a marker and a probe (ratio : 75). While kprobes are very
> useful for the reason explained earlier, the high event rate paths in the kernel
> would clearly benefit from a marker mechanism when the are probed.
>   

The problem now is how do we define "high event rate".  This is 
something that is highly dependent on the workload being run as well as 
the system configuration for such workload.  There are a lot of places 
in the kernel that can be turned into high event rates with with the 
right workload even though the may not represent 99% of most user cases. 

I would guess that anything above 500 event/s per-CPU on several 
realistic workloads is a good place to start.


> On the macro-benchmark side, no significant difference in performance has been
> found between the vanilla kernel and a kernel "marked" with the standard LTTng
> instrumentation.
>   

Out of curiosity,  how many cycles does it take to process a complete 
LTTng event up until the point were it has been completely stored into 
the trace buffer.  Since this should take a lot more than 55.74 cycles, 
it would be interesting to know at what event rate would a static marker 
stop showing as big of a performance advantage compared to dynamic probing.

-JRS