From mboxrd@z Thu Jan 1 00:00:00 1970 From: George Dunlap Subject: Re: [PATCH] Always save/restore performance counters when HVM guest switching VCPU Date: Mon, 11 Mar 2013 14:59:17 +0000 Message-ID: <513DF145.5010905@eu.citrix.com> References: <69a6d98c-8200-48c6-adc7-fcc40347cffc@default> <513DBBC6.8030806@eu.citrix.com> <20130311145349.GA26394@phenom.dumpdata.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; Format="flowed" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <20130311145349.GA26394@phenom.dumpdata.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Konrad Rzeszutek Wilk Cc: Boris Ostrovsky , "JBeulich@suse.com" , "suravee.suthikulpanit@amd.com" , "xen-devel@lists.xen.org" List-Id: xen-devel@lists.xenproject.org On 11/03/13 14:53, Konrad Rzeszutek Wilk wrote: > On Mon, Mar 11, 2013 at 11:11:02AM +0000, George Dunlap wrote: >> On 08/03/13 15:11, Boris Ostrovsky wrote: >>> ----- george.dunlap@eu.citrix.com wrote: >>> >>>> On 08/03/13 14:50, Boris Ostrovsky wrote: >>>>> ----- JBeulich@suse.com wrote: >>>>> >>>>>>>>> On 04.03.13 at 13:42, George Dunlap >>>> >>>>>> wrote: >>>>>>> On Fri, Mar 1, 2013 at 8:49 PM, >>>>>> wrote: >>>>>>>> From: Suravee Suthikulpanit >>>>>>>> >>>>>>>> Currently, the performance counter registers are saved/restores >>>>>>>> when the HVM guest switchs VCPUs only if they are running. >>>>>>>> However, PERF has one check where it writes the MSR and read >>>> back >>>>>>>> the value to check if the MSR is working. This has shown to >>>> fails >>>>>>>> the check if the VCPU is moved in between rdmsr and wrmsr and >>>>>>>> resulting in the values are different. >>>>>>> Many moons ago (circa 2005) when I used performance counters, I >>>>>> found >>>>>>> that adding them to the save/restore path added a non-neligible >>>>>>> overhead -- something like 5% slow-down. Do you have any reason >>>> to >>>>>>> believe this is no longer the case? Have you done any benchmarks >>>>>>> before and after? >>>>> I was doing some VPMU tracing a couple of weeks ago and by looking >>>> at >>>>> trace timestamps I think I saw about 4000 cycles on VPMU save and >>>>> ~9000 cycles on restore. Don't remember what it was percentage-wise >>>> of >>>>> a whole context switch. >>>>> >>>>> This was on Intel. >>>> That's a really hefty expense to make all users pay on every context >>>> switch, on behalf of a random check in a piece of software that only a >>>> handful of people are going to be actually using. >>> I believe Linux uses perf infrastructure to implement the watchdog. > And by default it won't work as for Intel you need these flags: > > cpuid=['0xa:eax=0x07300403,ebx=0x00000004,ecx=0x00000000,edx=0x00000603' ] > > What we get right now when booting PVHVM under Intel is: > > [ 0.160989] Performance Events: unsupported p6 CPU model 45 no PMU driver, software events only. > [ 0.168098] NMI watchdog disabled (cpu0): hardware events not enabled > > Unless said above CPUID flag is provided. >> Hmm -- well if it is the case that adding performance counters to >> the vcpu context switch path will add a measurable overhead, then we >> probably don't want them enabled for typical guests anyway. If >> people are actually using the performance counters to measure >> performance, that makes sense; but for watchdogs it seems like Xen >> should be able to provide something that is useful for a watchdog >> without the extra overhead of saving and restoring performance >> counters. >> >> Konrad, any thoughts? > The other thing is that there is an Xen watchdog. The one that Jan Beulich > wrote which should also work under PVHVM: > > drivers/watchdog/xen_wdt.c But my main question is: If the Linux perf system successfully detects a vpmu, will it use the Xen watchdog, or will it try to use the vpmu? Do we need to do anything to make sure that when running under Xen, Linux will *not* try to use the vpmu for the watchdog? -George