Re: Performance monitoring units and KVM

[Markus Armbruster <armbru-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>]

Avi Kivity <avi-atKUWr5tajBWk0Htik3J/w@public.gmane.org> writes:

> Markus Armbruster wrote:
>>
>> System-wide profiling of the *virtual* machine is related to profiling
>> just a process.  That's hard.  I guess building on Perfmon2 would make
>> sense there, but as long as it's out of tree...  Can we wait for it?
>> If not, what then?
>>
>>   
>
> Give the guest access to the real PMU.  Save them on every exit
> (switching profiling off), and restore them on every entry (switching
> profiling on).  The only problem with this is that it is very cpu
> model dependent, losing the hardware independence that virtual
> machines have.  If you are satisfied with the architectural
> performance counters, then we even have hardware independence.

Saving and restoring the PMU is *slow* on most machines.  Especially
bad on machines where reading / writing a PMU register involves
serializing instructions.

Want to try anyway?

I hope hardware vendors will eventually make PMUs friendlier to
virtualization.

>> The same ideas should work for KVM.  The whole hypervisor headache
>> just evaporates, of course.  What remains is the host kernel routing
>> samples to active guests (over virtio, I guess), and guests kernels
>> receiving samples from there instead of the hardware PMU.  In other
>> words, the sample channel from the host becomes our virtual PMU for
>> the guest.  Which needs a driver for it.  It's a weird PMU, because
>> you can't program its performance counters.  That's left to the host.
>>   
>
> Is there really a requirement to profile several userspace programs,
> on several guests, simultaneously?  If not, passing through the PMU
> will work best, with the additional advantage that guests will not
> need modification (so you can run Windows with VTune, for example).

There are uses for both kinds of system-wide profiling.

> If this three-tier profiling is actually needed, perhaps we can do all
> recording on the host, but have an interface to let the guest
> translate rip samples to something more meaningful.  This might work
> in this way:
>
> - oprofile on the host receives the pmu nmi
> - oprofile calls a hook (placed there by kvm) when it sees that the
> task is actually a virtual machine, instead of the usual translation
> process
> - kvm injects an interrupt into the guest
> - the guest converts the pmu rip value into a meaningful string and
> writes it into memory
> - (later) kvm picks this up and passes it back to oprofile
>
> The advantage here is that besides a fairly simple driver that needs
> to be loaded into the guest (and can be loaded automatically),
> everything is controlled from the host.  All the information is
> available on the host, so that sorting by counter occurences, for
> example, works.

Yep.

Problems include:

* OProfile user space receives dcookies from the kernel, which it
  passes to lookup_dcookie().  We'd have to delegate that to the
  appropriate guest.

* OProfile user space needs to be taught where do find each guest's
  debug info.

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/