From mboxrd@z Thu Jan 1 00:00:00 1970 From: Boris Ostrovsky Subject: Re: [PATCH] Always save/restore performance counters when HVM guest switching VCPU Date: Mon, 11 Mar 2013 11:54:10 -0400 Message-ID: <513DFE22.3070301@oracle.com> References: <69a6d98c-8200-48c6-adc7-fcc40347cffc@default> <513DBBC6.8030806@eu.citrix.com> <20130311145349.GA26394@phenom.dumpdata.com> <513DF145.5010905@eu.citrix.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; Format="flowed" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <513DF145.5010905@eu.citrix.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: George Dunlap Cc: "xen-devel@lists.xen.org" , "suravee.suthikulpanit@amd.com" , "JBeulich@suse.com" , Konrad Rzeszutek Wilk List-Id: xen-devel@lists.xenproject.org On 03/11/2013 10:59 AM, George Dunlap wrote: > On 11/03/13 14:53, Konrad Rzeszutek Wilk wrote: >> On Mon, Mar 11, 2013 at 11:11:02AM +0000, George Dunlap wrote: >>> On 08/03/13 15:11, Boris Ostrovsky wrote: >>>> ----- george.dunlap@eu.citrix.com wrote: >>>> >>>>> On 08/03/13 14:50, Boris Ostrovsky wrote: >>>>>> ----- JBeulich@suse.com wrote: >>>>>> >>>>>>>>>> On 04.03.13 at 13:42, George Dunlap >>>>> >>>>>>> wrote: >>>>>>>> On Fri, Mar 1, 2013 at 8:49 PM, >>>>>>> wrote: >>>>>>>>> From: Suravee Suthikulpanit >>>>>>>>> >>>>>>>>> Currently, the performance counter registers are saved/restores >>>>>>>>> when the HVM guest switchs VCPUs only if they are running. >>>>>>>>> However, PERF has one check where it writes the MSR and read >>>>> back >>>>>>>>> the value to check if the MSR is working. This has shown to >>>>> fails >>>>>>>>> the check if the VCPU is moved in between rdmsr and wrmsr and >>>>>>>>> resulting in the values are different. >>>>>>>> Many moons ago (circa 2005) when I used performance counters, I >>>>>>> found >>>>>>>> that adding them to the save/restore path added a non-neligible >>>>>>>> overhead -- something like 5% slow-down. Do you have any reason >>>>> to >>>>>>>> believe this is no longer the case? Have you done any benchmarks >>>>>>>> before and after? >>>>>> I was doing some VPMU tracing a couple of weeks ago and by looking >>>>> at >>>>>> trace timestamps I think I saw about 4000 cycles on VPMU save and >>>>>> ~9000 cycles on restore. Don't remember what it was percentage-wise >>>>> of >>>>>> a whole context switch. >>>>>> >>>>>> This was on Intel. >>>>> That's a really hefty expense to make all users pay on every context >>>>> switch, on behalf of a random check in a piece of software that >>>>> only a >>>>> handful of people are going to be actually using. >>>> I believe Linux uses perf infrastructure to implement the watchdog. >> And by default it won't work as for Intel you need these flags: >> >> cpuid=['0xa:eax=0x07300403,ebx=0x00000004,ecx=0x00000000,edx=0x00000603' >> ] >> >> What we get right now when booting PVHVM under Intel is: >> >> [ 0.160989] Performance Events: unsupported p6 CPU model 45 no PMU >> driver, software events only. >> [ 0.168098] NMI watchdog disabled (cpu0): hardware events not enabled >> >> Unless said above CPUID flag is provided. >>> Hmm -- well if it is the case that adding performance counters to >>> the vcpu context switch path will add a measurable overhead, then we >>> probably don't want them enabled for typical guests anyway. If >>> people are actually using the performance counters to measure >>> performance, that makes sense; but for watchdogs it seems like Xen >>> should be able to provide something that is useful for a watchdog >>> without the extra overhead of saving and restoring performance >>> counters. >>> >>> Konrad, any thoughts? >> The other thing is that there is an Xen watchdog. The one that Jan >> Beulich >> wrote which should also work under PVHVM: >> >> drivers/watchdog/xen_wdt.c > > But my main question is: If the Linux perf system successfully detects > a vpmu, will it use the Xen watchdog, or will it try to use the vpmu? > Do we need to do anything to make sure that when running under Xen, > Linux will *not* try to use the vpmu for the watchdog? It looks to me that both watchdogs are running. Perf's counter 0 (which is what watchdog uses) is definitely enabled. -boris