Re: [ANNOUNCE] New tools: lttngtrace and lttngreport

[Ingo Molnar <mingo@elte.hu>]

* Mathieu Desnoyers <mathieu.desnoyers@efficios.com> wrote:

> Well, I'm afraid the collection approach "trace" is currently taking won't allow 
> this kind of dependency wakeup chain tracking, because they focus on tracing 
> operations happening on a thread and its children, but the reality is that the 
> wakeup chains often spread outside of this scope.
>
> This is why lttngtrace gathers a system-wide trace even though we're mostly 
> intested in the wait/wakeups of a specific PID.

Mathieu, what you say is factually wrong - of course perf events allows system-wide 
tracing, it always did.

This is available in the 'trace' tool as well, try:

  $ trace record --all sleep 1          # do system-wide tracing
  
  # trace recorded [1.928 MB] - try 'trace summary' to get an overview

  $ trace summary

   .-----------------------------------.
 __)    Summary of 'sleep 1' events    (__

              [ task - pid ]     [ events ] [ ratio ]  [ runtime ]
 _____________________________________________________________________

             swapper - 0     :         64   [  0.4% ]      0.000 ms
         ksoftirqd/0 - 3     :          6   [  0.0% ]      0.063 ms
         migration/1 - 7     :          2   [  0.0% ]      0.000 ms
         kworker/0:1 - 10    :          6   [  0.0% ]      0.139 ms
         kworker/6:1 - 53    :          3   [  0.0% ]      0.014 ms
         kworker/5:1 - 54    :          3   [  0.0% ]      0.014 ms
         kworker/4:1 - 55    :          3   [  0.0% ]      0.014 ms
         kworker/3:1 - 56    :          3   [  0.0% ]      0.015 ms
         kworker/2:1 - 57    :          3   [  0.0% ]      0.015 ms
         kworker/1:1 - 58    :          3   [  0.0% ]      0.015 ms
        kworker/15:1 - 318   :          6   [  0.0% ]      0.045 ms
        kworker/14:1 - 319   :          6   [  0.0% ]      0.033 ms
        kworker/13:1 - 320   :          6   [  0.0% ]      0.036 ms
        kworker/12:1 - 321   :          6   [  0.0% ]      0.035 ms
        kworker/11:1 - 322   :          6   [  0.0% ]      0.041 ms
        kworker/10:1 - 323   :          6   [  0.0% ]      0.037 ms
         kworker/9:1 - 324   :          6   [  0.0% ]      0.042 ms
         kworker/7:1 - 326   :          6   [  0.0% ]      0.032 ms
         kworker/8:2 - 1047  :          6   [  0.0% ]      0.039 ms
            sendmail - 2327  :         18   [  0.1% ]      0.111 ms
               trace - 32281 :      15728   [ 96.2% ]     86.478 ms
               sleep - 32282 :        460   [  2.8% ]      1.305 ms
 _____________________________________________________________________

As you can see all those kworker events - it's a system-wide trace.

This is in addition to per process and per workflow (child hierarchy) tracing - 
which we expect to be more commonly used.

Thanks,

	Ingo