Re: Performance monitoring units and KVM

Markus Armbruster
2008-01-30T19:55:40Z

Avi Kivity <avi-atKUWr5tajBWk0Htik3J/w@public.gmane.org> writes:

> Markus Armbruster wrote:
>>
>> System-wide profiling of the *virtual* machine is related to profiling
>> just a process. That's hard. I guess building on Perfmon2 would make
>> sense there, but as long as it's out of tree... Can we wait for it?
>> If not, what then?
>>
>>
>
> Give the guest access to the real PMU. Save them on every exit
> (switching profiling off), and restore them on every entry (switching
> profiling on). The only problem with this is that it is very cpu
> model dependent, losing the hardware independence that virtual
> machines have. If you are satisfied with the architectural
> performance counters, then we even have hardware independence.

Saving and restoring the PMU is *slow* on most machines. Especially
bad on machines where reading / writing a PMU register involves
serializing instructions.

Want to try anyway?

I hope hardware vendors will eventually make PMUs friendlier to
virtualization.

>> The same ideas should work for KVM. The whole hypervisor headache
>> just evaporates, of course. What remains is the host kernel routing
>> samples to active guests (over virtio, I guess), and guests kernels
>> receiving samples from there instead of the hardware PMU. In other
>> words, the sample channel from the host becomes our virtual PMU for
>> the guest. Which needs a driver for it. It's a weird PMU, because
>> you can't program its performance counters. That's left to the host.
>>
>
> Is there really a requirement to profile several userspace programs,
> on several guests, simultaneously? If not, passing through the PMU
> will work best, with the additional advantage that guests will not
> need modification (so you can run Windows with VTune, for example).

There are uses for both kinds of system-wide profiling.

> If this three-tier profiling is actually needed, perhaps we can do all
> recording on the host, but have an interface to let the guest
> translate rip samples to something more meaningful. This might work
> in this way:
>
> - oprofile on the host receives the pmu nmi
> - oprofile calls a hook (placed there by kvm) when it sees that the
> task is actually a virtual machine, instead of the usual translation
> process
> - kvm injects an interrupt into the guest
> - the guest converts the pmu rip value into a meaningful string and
> writes it into memory
> - (later) kvm picks this up and passes it back to oprofile
>
> The advantage here is that besides a fairly simple driver that needs
> to be loaded into the guest (and can be loaded automatically),
> everything is controlled from the host. All the information is
> available on the host, so that sorting by counter occurences, for
> example, works.

Yep.

Problems include:

* OProfile user space receives dcookies from the kernel, which it
passes to lookup_dcookie(). We'd have to delegate that to the
appropriate guest.

* OProfile user space needs to be taught where do find each guest's
debug info.