Re: [ANNOUNCE] New utility: 'trace'

[Avi Kivity <avi@redhat.com>]

On 11/17/2010 05:58 PM, Thomas Gleixner wrote:
> >
> >  Filtering is also useful for fine-grained perf events; this cannot be done in
> >  userspace.
>
> Could you explain that a bit more detailed please ?

I have a tracepoint kvm:kvm_exit which is called every time a guest 
exits to the host (for whatever reason).  The first step in profiling 
kvm is checking the rate at which this tracepoint is triggered.

(I have a small tool kvm_stat which shows a 'perf top' like display, 
except that instead of showing functions, it shows kvm tracepoints 
sorted by triggering rate)

If you see a high exit rate, you may also be interested in what exit 
reasons triggered those exits.  So kvm_stat creates additional events, 
one per possible exit_reason (which is a field in the kvm_exit 
tracepoint), with a filter matching exit_reason to the enum describing 
the possible exit reasons.  We add that to the display and get a sort of 
histogram showing which exit_reason is responsible for how many exits.

A screenshot:

kvm statistics

  kvm_exit                                     40075   10293
  kvm_entry                                    40075   10293
  kvm_exit(EXCEPTION_NMI)                      29824    7712
  kvm_page_fault                               29642    7672
  kvm_emulate_insn                              9117    2205
  kvm_inj_exception                             8052    2134
  kvm_mmio                                      6160    1514
  kvm_apic                                      6143    1509
  kvm_exit(CR_ACCESS)                           2307     635
  kvm_cr                                        2307     635
  kvm_exit(INVLPG)                              1998     599
  kvm_userspace_exit                            2956     582
  kvm_exit(IO_INSTRUCTION)                      2954     582
  kvm_pio                                       2954     582
  kvm_inj_virq                                  2194     551
  kvm_apic_accept_irq                           2181     547
  kvm_exit(HLT)                                 1523     372
  kvm_exit(EXTERNAL_INTERRUPT)                  1131     304
  kvm_fpu                                        706     176
  kvm_set_irq                                    802     156
  kvm_pic_set_irq                                802     156
  kvm_ioapic_set_irq                             802     156
  kvm_apic_ipi                                   454     132
  kvm_exit(PENDING_INTERRUPT)                    175      44
  kvm_cpuid                                      163      42
  kvm_exit(CPUID)                                163      42
  kvm_ack_irq                                    152      29


So there are 10K exits/sec, of which 7.7K are EXCEPTION_NMI and 635 are 
CR_ACCESS.

Tool source http://tinyurl.com/2ue5tev; one day I'll make it a 
tools/perf script.

-- 
error compiling committee.c: too many arguments to function