Re: [PATCH] Always save/restore performance counters when HVM guest switching VCPU

[Boris Ostrovsky <boris.ostrovsky@oracle.com>]

On 03/11/2013 10:59 AM, George Dunlap wrote:
> On 11/03/13 14:53, Konrad Rzeszutek Wilk wrote:
>> On Mon, Mar 11, 2013 at 11:11:02AM +0000, George Dunlap wrote:
>>> On 08/03/13 15:11, Boris Ostrovsky wrote:
>>>> ----- george.dunlap@eu.citrix.com wrote:
>>>>
>>>>> On 08/03/13 14:50, Boris Ostrovsky wrote:
>>>>>> ----- JBeulich@suse.com wrote:
>>>>>>
>>>>>>>>>> On 04.03.13 at 13:42, George Dunlap
>>>>> <George.Dunlap@eu.citrix.com>
>>>>>>> wrote:
>>>>>>>> On Fri, Mar 1, 2013 at 8:49 PM,  <suravee.suthikulpanit@amd.com>
>>>>>>> wrote:
>>>>>>>>> From: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
>>>>>>>>>
>>>>>>>>> Currently, the performance counter registers are saved/restores
>>>>>>>>> when the HVM guest switchs VCPUs only if they are running.
>>>>>>>>> However, PERF has one check where it writes the MSR and read
>>>>> back
>>>>>>>>> the value to check if the MSR is working.  This has shown to
>>>>> fails
>>>>>>>>> the check if the VCPU is moved in between rdmsr and wrmsr and
>>>>>>>>> resulting in the values are different.
>>>>>>>> Many moons ago (circa 2005) when I used performance counters, I
>>>>>>> found
>>>>>>>> that adding them to the save/restore path added a non-neligible
>>>>>>>> overhead -- something like 5% slow-down.  Do you have any reason
>>>>> to
>>>>>>>> believe this is no longer the case?  Have you done any benchmarks
>>>>>>>> before and after?
>>>>>> I was doing some VPMU tracing a couple of weeks ago and by looking
>>>>> at
>>>>>> trace timestamps I think I saw about 4000 cycles on VPMU save and
>>>>>> ~9000 cycles on restore. Don't remember what it was percentage-wise
>>>>> of
>>>>>> a whole context switch.
>>>>>>
>>>>>> This was on Intel.
>>>>> That's a really hefty expense to make all users pay on every context
>>>>> switch, on behalf of a random check in a piece of software that 
>>>>> only a
>>>>> handful of people are going to be actually using.
>>>> I believe Linux uses perf infrastructure to implement the watchdog.
>> And by default it won't work as for Intel you need these flags:
>>
>> cpuid=['0xa:eax=0x07300403,ebx=0x00000004,ecx=0x00000000,edx=0x00000603' 
>> ]
>>
>> What we get right now when booting PVHVM under Intel is:
>>
>> [    0.160989] Performance Events: unsupported p6 CPU model 45 no PMU 
>> driver, software events only.
>> [    0.168098] NMI watchdog disabled (cpu0): hardware events not enabled
>>
>> Unless said above CPUID flag is provided.
>>> Hmm -- well if it is the case that adding performance counters to
>>> the vcpu context switch path will add a measurable overhead, then we
>>> probably don't want them enabled for typical guests anyway. If
>>> people are actually using the performance counters to measure
>>> performance, that makes sense; but for watchdogs it seems like Xen
>>> should be able to provide something that is useful for a watchdog
>>> without the extra overhead of saving and restoring performance
>>> counters.
>>>
>>> Konrad, any thoughts?
>> The other thing is that there is an Xen watchdog. The one that Jan 
>> Beulich
>> wrote which should also work under PVHVM:
>>
>> drivers/watchdog/xen_wdt.c
>
> But my main question is: If the Linux perf system successfully detects 
> a vpmu, will it use the Xen watchdog, or will it try to use the vpmu?  
> Do we need to do anything to make sure that when running under Xen, 
> Linux will *not* try to use the vpmu for the watchdog?


It looks to me that both watchdogs are running. Perf's counter 0 (which 
is what watchdog uses) is definitely enabled.

-boris