[RFC] Performance monitoring units and KVM

[RFC] Performance monitoring units and KVM

[Balaji Rao]

Hi all!

Earlier it was suggested that we go ahead with emulating Perf Mon Events in 
exposing it to the guest. The serious limitation in this approach is that we 
end up exposing only a small number of events to the guest, even though the 
host hardware is capable of much more. The only benefit this approach offers is 
that, it doesn't break live migration.

The other option is to pass through the real PMU to the guest. I believe this 
approach is far better in the sense that,

1. All the available events in the host hardware can be passed on to the guest, 
which can be used by oprofile to profile the guest and trackdown slowdowns 
introduced due to virtualization.

2. Its much cleaner and easier to pass through the PMU.

Yes, this approach breaks live migration. Migration should not be possible 
*only* when the PMU is being used by oprofile. We can mark the guest as 
unmigratable in such situations. Once the PMU is not being used, migration can 
be performed normally.

Note, this requires a small change to oprofile source. Upon migration, oprofile 
should be made to re-identify the CPU and use the perf mon events appropriate 
to that CPU. I think this could be done by having a migrate_notifier, or 
something like that..

Please provide comments on this.

-- 
regards,
balaji rao
NITK

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

Re: [RFC] Performance monitoring units and KVM

[Anthony Liguori]

Balaji Rao wrote:
> Hi all!
>
> Earlier it was suggested that we go ahead with emulating Perf Mon Events in 
> exposing it to the guest. The serious limitation in this approach is that we 
> end up exposing only a small number of events to the guest, even though the 
> host hardware is capable of much more. The only benefit this approach offers is 
> that, it doesn't break live migration.
>   

I think performance monitors are no different than anything else in 
KVM.  We should virtualize as much as possible and by default provide 
only the common subset to the guest supported by the majority of hardware.

Then we can use mechanisms like QEMU's CPU support to enable additional 
features that may be available and unique to the underlying hardware.  
It's then up to the management tools to deal with migratability since 
they've explicitly enabled the feature.

Regards,

Anthony Liguori

> The other option is to pass through the real PMU to the guest. I believe this 
> approach is far better in the sense that,
>
> 1. All the available events in the host hardware can be passed on to the guest, 
> which can be used by oprofile to profile the guest and trackdown slowdowns 
> introduced due to virtualization.
>
> 2. Its much cleaner and easier to pass through the PMU.
>
> Yes, this approach breaks live migration. Migration should not be possible 
> *only* when the PMU is being used by oprofile. We can mark the guest as 
> unmigratable in such situations. Once the PMU is not being used, migration can 
> be performed normally.
>
> Note, this requires a small change to oprofile source. Upon migration, oprofile 
> should be made to re-identify the CPU and use the perf mon events appropriate 
> to that CPU. I think this could be done by having a migrate_notifier, or 
> something like that..
>
> Please provide comments on this.
>
>   


-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

Re: [RFC] Performance monitoring units and KVM

[Balaji Rao]

On Sunday 17 February 2008 03:34:43 am Anthony Liguori wrote:
> Balaji Rao wrote:
> > Hi all!
> >
> > Earlier it was suggested that we go ahead with emulating Perf Mon Events
> > in exposing it to the guest. The serious limitation in this approach is
> > that we end up exposing only a small number of events to the guest, even
> > though the host hardware is capable of much more. The only benefit this
> > approach offers is that, it doesn't break live migration.
>
> I think performance monitors are no different than anything else in
> KVM.  We should virtualize as much as possible and by default provide
> only the common subset to the guest supported by the majority of hardware.
>
> Then we can use mechanisms like QEMU's CPU support to enable additional
> features that may be available and unique to the underlying hardware.
> It's then up to the management tools to deal with migratability since
> they've explicitly enabled the feature.

Sorry, I don't understand how it can done through QEMU, but according to what I 
understand, it makes migration very difficult/impossible. So, why should we go 
for this approach at all ? Its the very reason direct access to PMU was thought 
of as a bad idea.

Do you see any other problem in directly exposing the PMU ?

-- 
regards,

balaji rao
NITK

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

Re: [RFC] Performance monitoring units and KVM

[Markus Armbruster]

Balaji Rao <balajirrao@gmail.com> writes:

> Hi all!
>
> Earlier it was suggested that we go ahead with emulating Perf Mon Events in 
> exposing it to the guest. The serious limitation in this approach is that we 
> end up exposing only a small number of events to the guest, even though the 
> host hardware is capable of much more. The only benefit this approach offers is 
> that, it doesn't break live migration.
>
> The other option is to pass through the real PMU to the guest. I believe this 
> approach is far better in the sense that,

Do we really have an either/or alternative here?

> 1. All the available events in the host hardware can be passed on to the guest, 
> which can be used by oprofile to profile the guest and trackdown slowdowns 
> introduced due to virtualization.
>
> 2. Its much cleaner and easier to pass through the PMU.
>
> Yes, this approach breaks live migration. Migration should not be possible 
> *only* when the PMU is being used by oprofile. We can mark the guest as 
> unmigratable in such situations. Once the PMU is not being used, migration can 
> be performed normally.
>
> Note, this requires a small change to oprofile source. Upon migration, oprofile 
> should be made to re-identify the CPU and use the perf mon events appropriate 
> to that CPU. I think this could be done by having a migrate_notifier, or 
> something like that..
>
> Please provide comments on this.

Different implementations of the same processor architecture have
different PMUs.  Existing software using the PMU (directly) knows
exactly what PMU to expect with a particular CPU.

If we want to run such software in a guest (say VTune under Windows),
we need to provide a virtual CPU that is sufficiently complete,
including the PMU.  This will be *costly* on most CPUs.  Bad vmexit
latencies.

Sometimes it doesn't matter when profiling slows down your system, as
long as the profile is still sufficiently accurate, e.g. when a
developer examines a program in a test bed.  At other times, such
overhead is simply unacceptable, e.g. when you examine a real system
in the field, to figure out why it misbehaves.

There are ways to use the PMU in guests that don't require costly
virtualization of the real PMU.  They put the guest's performance
monitoring interface at a level higher than hardware PMU.

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

Re: [RFC] Performance monitoring units and KVM

[Avi Kivity]

Markus Armbruster wrote:
> Balaji Rao <balajirrao@gmail.com> writes:
>
>   
>> Hi all!
>>
>> Earlier it was suggested that we go ahead with emulating Perf Mon Events in 
>> exposing it to the guest. The serious limitation in this approach is that we 
>> end up exposing only a small number of events to the guest, even though the 
>> host hardware is capable of much more. The only benefit this approach offers is 
>> that, it doesn't break live migration.
>>
>> The other option is to pass through the real PMU to the guest. I believe this 
>> approach is far better in the sense that,
>>     
>
> Do we really have an either/or alternative here?
>
>   

I think that PMU pass through makes more sense, due to the nature of 
existing non-paravirtualized tools.

> There are ways to use the PMU in guests that don't require costly
> virtualization of the real PMU.  They put the guest's performance
> monitoring interface at a level higher than hardware PMU.
>   

On the other hand, they will work only on very new guests.  Paravirt pmu 
makes sense, but it cannot replace the hardware pmu interface.


-- 
error compiling committee.c: too many arguments to function


-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/