Re: [RFD] Future tracing/instrumentation directions

[Ingo Molnar <mingo@elte.hu>]

* Frederic Weisbecker <fweisbec@gmail.com> wrote:

> On Thu, May 20, 2010 at 11:31:31AM +0200, Ingo Molnar wrote:
> >  - [ While it's still a long way off, if this trend continues
> >      we eventually might even be able to get rid of the 
> >      /debug/tracing/ temporary debug API and get rid of 
> >      the ugly in-kernel pretty-printing bits. This is 
> >      good: it may make Andrew very happy for a change ;-)
> > 
> >      The main detail here to be careful of is that lots of
> >      people are fond of the simplicity of the 
> >      /debug/tracing/ debug UI, so when we replace it we 
> >      want to do it by keeping that simple workflow (or 
> >      best by making it even simpler). I have a few ideas 
> >      how to do this.
> 
> How? We can emulate the /debug/tracing result with 
> something like perf trace, still that won't replace the 
> immediate availability of the result of any trace, which 
> makes it valuable for any simplest workflows.

Firstly, one thing is sure: until there's no full 
replacement we obviously dont want to phase out 
/sys/kernel/debug/tracing. This was more of a 'our future' 
email (as i see it), the process that will lead to solve 
some of our more strategic problems in tracing land.

Regarding the issues you raised, there are several 
solutions that dont need /sys/kernel/debug/tracing but 
still support the very useful and usable 'immediate 
tracing' workflow that ftrace prototyped. We can have a 
combination of several things:

 - Have a simple 'ftrace' command aliased to perf trace.

   Means less typing, and it also allows a much more 
   finegrained tracing workflow: per user and per task/job 
   workflows, instead of the global/exclusive tracing mode 
   that /sys/kernel/debug/tracing. There would be ready 
   equivalents:

     ftrace --available-tracers
     ftrace --current-tracer
     ftrace --start
     ftrace --stop

     ... etc ...

 - Immediate availability of a trace: persistent events
   combined with roundrobin ('flight-recorder') recording
   would solve this.

   If events are active then if type 'ftrace' you get the 
   current trace. Simple to scrip and simple to use - no 
   need to have access to /sys/kernel/debug/tracing, also 
   can evidently be turned into a per user facility, 
   supports multiple plugins active at once, etc.

 - For lightweight embedded tracing there are two separate
   solutions that would work:

    - we already have perf 'minimal builds' (when certain 
      libraries are not available), we could push that 
      concept some more to create a 'lightweight' command
      that embedded systems can run just fine.

    - extend our proxy recording and proxy execution/analysis
      concepts. We can already run a perf trace recording 
      through a pipe and netcat, and we have perf archive
      and cross-arch analysis code.

      If there's interest, then this could be made more 
      convenient and the functionality around this could 
      be collected into a handy proxy tool:

        ftrace --proxy smallbox --start
        ftrace --proxy smallbox --stop
        ftrace --proxy smallbox           # prints trace

        ... etc ...

      Thus the recording is done on the small box, while 
      all the analysis (and even all the commands) is 
      typed/executed on the bigger box.

So there are lots of possibilities - and there are other 
options as well.

Does this address your worries?

Thanks,

	Ingo