Re: [PATCH] Always save/restore performance counters when HVM guest switching VCPU

[George Dunlap <george.dunlap@eu.citrix.com>]

On 11/03/13 14:53, Konrad Rzeszutek Wilk wrote:
> On Mon, Mar 11, 2013 at 11:11:02AM +0000, George Dunlap wrote:
>> On 08/03/13 15:11, Boris Ostrovsky wrote:
>>> ----- george.dunlap@eu.citrix.com wrote:
>>>
>>>> On 08/03/13 14:50, Boris Ostrovsky wrote:
>>>>> ----- JBeulich@suse.com wrote:
>>>>>
>>>>>>>>> On 04.03.13 at 13:42, George Dunlap
>>>> <George.Dunlap@eu.citrix.com>
>>>>>> wrote:
>>>>>>> On Fri, Mar 1, 2013 at 8:49 PM,  <suravee.suthikulpanit@amd.com>
>>>>>> wrote:
>>>>>>>> From: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
>>>>>>>>
>>>>>>>> Currently, the performance counter registers are saved/restores
>>>>>>>> when the HVM guest switchs VCPUs only if they are running.
>>>>>>>> However, PERF has one check where it writes the MSR and read
>>>> back
>>>>>>>> the value to check if the MSR is working.  This has shown to
>>>> fails
>>>>>>>> the check if the VCPU is moved in between rdmsr and wrmsr and
>>>>>>>> resulting in the values are different.
>>>>>>> Many moons ago (circa 2005) when I used performance counters, I
>>>>>> found
>>>>>>> that adding them to the save/restore path added a non-neligible
>>>>>>> overhead -- something like 5% slow-down.  Do you have any reason
>>>> to
>>>>>>> believe this is no longer the case?  Have you done any benchmarks
>>>>>>> before and after?
>>>>> I was doing some VPMU tracing a couple of weeks ago and by looking
>>>> at
>>>>> trace timestamps I think I saw about 4000 cycles on VPMU save and
>>>>> ~9000 cycles on restore. Don't remember what it was percentage-wise
>>>> of
>>>>> a whole context switch.
>>>>>
>>>>> This was on Intel.
>>>> That's a really hefty expense to make all users pay on every context
>>>> switch, on behalf of a random check in a piece of software that only a
>>>> handful of people are going to be actually using.
>>> I believe Linux uses perf infrastructure to implement the watchdog.
> And by default it won't work as for Intel you need these flags:
>
> cpuid=['0xa:eax=0x07300403,ebx=0x00000004,ecx=0x00000000,edx=0x00000603' ]
>
> What we get right now when booting PVHVM under Intel is:
>
> [    0.160989] Performance Events: unsupported p6 CPU model 45 no PMU driver, software events only.
> [    0.168098] NMI watchdog disabled (cpu0): hardware events not enabled
>
> Unless said above CPUID flag is provided.
>> Hmm -- well if it is the case that adding performance counters to
>> the vcpu context switch path will add a measurable overhead, then we
>> probably don't want them enabled for typical guests anyway.  If
>> people are actually using the performance counters to measure
>> performance, that makes sense; but for watchdogs it seems like Xen
>> should be able to provide something that is useful for a watchdog
>> without the extra overhead of saving and restoring performance
>> counters.
>>
>> Konrad, any thoughts?
> The other thing is that there is an Xen watchdog. The one that Jan Beulich
> wrote which should also work under PVHVM:
>
> drivers/watchdog/xen_wdt.c

But my main question is: If the Linux perf system successfully detects a 
vpmu, will it use the Xen watchdog, or will it try to use the vpmu?  Do 
we need to do anything to make sure that when running under Xen, Linux 
will *not* try to use the vpmu for the watchdog?

  -George