Re: [PATCH] Enhance perf to collect KVM guest os statistics from host side

["Zhang, Yanmin" <yanmin_zhang@linux.intel.com>]

On Wed, 2010-03-17 at 17:26 +0800, Zhang, Yanmin wrote:
> On Tue, 2010-03-16 at 10:47 +0100, Ingo Molnar wrote:
> > * Zhang, Yanmin <yanmin_zhang@linux.intel.com> wrote:
> > 
> > > On Tue, 2010-03-16 at 15:48 +0800, Zhang, Yanmin wrote:
> > > > On Tue, 2010-03-16 at 07:41 +0200, Avi Kivity wrote:
> > > > > On 03/16/2010 07:27 AM, Zhang, Yanmin wrote:
> > > > > > From: Zhang, Yanmin<yanmin_zhang@linux.intel.com>
> > > > > >
> > > > > > Based on the discussion in KVM community, I worked out the patch to support
> > > > > > perf to collect guest os statistics from host side. This patch is implemented
> > > > > > with Ingo, Peter and some other guys' kind help. Yang Sheng pointed out a
> > > > > > critical bug and provided good suggestions with other guys. I really appreciate
> > > > > > their kind help.
> > > > > >
> > > > > > The patch adds new subcommand kvm to perf.
> > > > > >
> > > > > >    perf kvm top
> > > > > >    perf kvm record
> > > > > >    perf kvm report
> > > > > >    perf kvm diff
> > > > > >
> > > > > > The new perf could profile guest os kernel except guest os user space, but it
> > > > > > could summarize guest os user space utilization per guest os.
> > > > > >
> > > > > > Below are some examples.
> > > > > > 1) perf kvm top
> > > > > > [root@lkp-ne01 norm]# perf kvm --host --guest --guestkallsyms=/home/ymzhang/guest/kallsyms
> > > > > > --guestmodules=/home/ymzhang/guest/modules top
> > > > > >
> > > > > >    
> > > > > 
> > > > Thanks for your kind comments.
> > > > 
> > > > > Excellent, support for guest kernel != host kernel is critical (I can't 
> > > > > remember the last time I ran same kernels).
> > > > > 
> > > > > How would we support multiple guests with different kernels?
> > > > With the patch, 'perf kvm report --sort pid" could show
> > > > summary statistics for all guest os instances. Then, use
> > > > parameter --pid of 'perf kvm record' to collect single problematic instance data.
> > > Sorry. I found currently --pid isn't process but a thread (main thread).
> > > 
> > > Ingo,
> > > 
> > > Is it possible to support a new parameter or extend --inherit, so 'perf 
> > > record' and 'perf top' could collect data on all threads of a process when 
> > > the process is running?
> > > 
> > > If not, I need add a new ugly parameter which is similar to --pid to filter 
> > > out process data in userspace.
> > 
> > Yeah. For maximum utility i'd suggest to extend --pid to include this, and 
> > introduce --tid for the previous, limited-to-a-single-task functionality.
> > 
> > Most users would expect --pid to work like a 'late attach' - i.e. to work like 
> > strace -f or like a gdb attach.
> 
> Thanks Ingo, Avi.
> 
> I worked out below patch against tip/master of March 15th.
> 
> Subject: [PATCH] Change perf's parameter --pid to process-wide collection
> From: Zhang, Yanmin <yanmin_zhang@linux.intel.com>
> 
> Change parameter -p (--pid) to real process pid and add -t (--tid) meaning
> thread id. Now, --pid means perf collects the statistics of all threads of
> the process, while --tid means perf just collect the statistics of that thread.
> 
> BTW, the patch fixes a bug of 'perf stat -p'. 'perf stat' always configures
> attr->disabled=1 if it isn't a system-wide collection. If there is a '-p'
> and no forks, 'perf stat -p' doesn't collect any data. In addition, the
> while(!done) in run_perf_stat consumes 100% single cpu time which has bad impact
> on running workload. I added a sleep(1) in the loop.
> 
> Signed-off-by: Zhang Yanmin <yanmin_zhang@linux.intel.com>
Ingo,

Sorry, the patch has bugs.  I need do a better job and will work out 2
separate patches against the 2 issues.

Yanmin