From mboxrd@z Thu Jan 1 00:00:00 1970 From: Dave Winchell Subject: Re: [PATCH] Add a timer mode that disables pending missed ticks Date: Thu, 14 Feb 2008 12:55:57 -0500 Message-ID: <47B480AD.4050308@virtualiron.com> References: <20080214092114625.00000001516@djm-pc> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <20080214092114625.00000001516@djm-pc> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: "dan.magenheimer@oracle.com" Cc: "xen-devel@lists.xensource.com" , Deepak Patel , Dave Winchell List-Id: xen-devel@lists.xenproject.org Dan, Here are some boot snipets for rh4u564 on xen 3.2. #1: Feb 14 10:44:59 vs076 kernel: Bootdata ok (command line is ro root=LABEL=/ console=ttyS0 clocksource=pit nohpet) Feb 14 10:44:59 vs076 kernel: Linux version 2.6.9-55.ELsmp (brewbuilder@hs20-bc2-4.build.redhat.com) (gcc version 3.4.6 20060404 (Red Hat 3.4.6-3)) #1 SMP Fri Apr 20 16:36:54 EDT 2007 ... Feb 14 10:44:59 vs076 kernel: Kernel command line: ro root=LABEL=/ console=ttyS0 clocksource=pit nohpet Feb 14 10:44:59 vs076 kernel: Initializing CPU#0 Feb 14 10:44:59 vs076 kernel: PID hash table entries: 2048 (order: 11, 65536 bytes) Feb 14 10:44:59 vs076 kernel: time.c: Using 3.579545 MHz PM timer. Feb 14 10:44:59 vs076 kernel: time.c: Detected 1992.050 MHz processor. ... Feb 14 10:45:00 vs076 kernel: checking TSC synchronization across 8 CPUs: passed. Feb 14 10:45:00 vs076 kernel: Brought up 8 CPUs Feb 14 10:45:00 vs076 kernel: Disabling vsyscall due to use of PM timer Feb 14 10:45:00 vs076 kernel: time.c: Using PM based timekeeping. #2: Feb 14 10:47:57 vs076 kernel: Bootdata ok (command line is ro root=LABEL=/ console=ttyS0 clocksource=pit nohpet nopmtimer) Feb 14 10:47:57 vs076 kernel: Linux version 2.6.9-55.ELsmp (brewbuilder@hs20-bc2-4.build.redhat.com) (gcc version 3.4.6 20060404 (Red Hat 3.4.6-3)) #1 SMP Fri Apr 20 16:36:54 EDT 2007 ... Feb 14 10:47:58 vs076 kernel: Kernel command line: ro root=LABEL=/ console=ttyS0 clocksource=pit nohpet nopmtimer Feb 14 10:47:58 vs076 kernel: Initializing CPU#0 Feb 14 10:47:58 vs076 kernel: PID hash table entries: 2048 (order: 11, 65536 bytes) Feb 14 10:47:58 vs076 kernel: time.c: Using 1.193182 MHz PIT timer. Feb 14 10:47:58 vs076 kernel: time.c: Detected 1991.959 MHz processor. ... Feb 14 10:47:59 vs076 kernel: checking TSC synchronization across 8 CPUs: passed. Feb 14 10:47:59 vs076 kernel: Brought up 8 CPUs Feb 14 10:47:59 vs076 kernel: time.c: Using PIT/TSC based timekeeping. As you can see, I only get the pit if I specify nopmtimer. Dan Magenheimer wrote: >Hi Dave -- > >Thanks for continuing to run tests! > >Hmmm... I thought I had noticed that even though Linux will acknowledge >the existence of the pmtimer, it still prints: > >time.c: Using PIT/TSC based timekeeping. > >I will check again, but assuming the clocksource for our tests is >indeed pit, the huge difference in the results (yours vs ours) is >baffling. I wonder if the difference may be the underlying hardware. >Maybe we will try to ensure we can duplicate the results on a different >box. > > >So your testing was with stock 3.2.0 xen bits (what cset?) without >any of your [quote from below] "clock related tweaks that I haven't >submitted, because I'm still characterizing them"? > > None of the tweaks I mentioned are in this test. It was stock with some patches. However, none of the patches are time related to my knowledge and I checked vpt.c to make sure that it is the same as what's in unstable. The only difference is in pt_intr_post, where I set the timer mode. I don't have timer mode tied into our config process yet, which is different than official xen method. (In pt_intr_post) else { + if(v->arch.paging.mode->guest_levels == 4) + v->domain->arch.hvm_domain.params[HVM_PARAM_TIMER_MODE] = HVMPTM_no_missed_ticks_pending; + else + v->domain->arch.hvm_domain.params[HVM_PARAM_TIMER_MODE] = HVMPTM_delay_for_missed_ticks; if ( mode_is(v->domain, one_missed_tick_pending) || mode_is(v->domain, no_missed_ticks_pending) ) { >Could you also send detail on the rhel4u4-64 kernel you >are testing with, just to ensure we are not comparing apples >and oranges? (Perhaps there's some way we can even share the >identical disk image and vm.cfg file?) > >And if our problem is indeed the pmtimer, I will need to submit >another patch to Keir to add an hvm pmtimer platform variable. >(Hmmm... I don't think he's even accepted the hpet variable patch >yet. I'll have to check.) > >Thanks, >Dan > > > > >>-----Original Message----- >>From: Dave Winchell [mailto:dwinchell@virtualiron.com] >>Sent: Thursday, February 14, 2008 9:00 AM >>To: dan.magenheimer@oracle.com >>Cc: Dave Winchell; Keir Fraser; xen-devel@lists.xensource.com; Deepak >>Patel >>Subject: Re: [Xen-devel] [PATCH] Add a timer mode that >>disables pending >>missed ticks >> >> >>Hi Dan, >> >>I ran the ltp tests with 3.2 and found the errors >>for a 16 hour run to be: >> >>rh4u564 -9.9 sec (-.017%) >>rh4u464 -7.3 sec (-.013%) >> >>There were no cliffs and the drift was linear. >> >>I think the problem you had may be due to the use of the >>pm timer. If you still have the boot log, it would tell you. >> >>When I first tried a guest on 3.2 with "clocksource=pit nohpet" >>I noticed that it picked the pm timer. Adding "nopmtimer", the >>guest will pick the pit. >> >>The reason I didn't have the problem with our 3.1 base is that >>I had disabled the hpet and the pmtimer by not advertising them >>in the acpi tables. I did this so long ago, I forgot that I had to >>disable pmtimer as well as hpet. >> >>So, can you re-run your test with "clocksource=pit nohpet nopmtimer"? >>You should see this in the boot messages: >> >>time.c: Using PIT/TSC based timekeeping. >> >>Thanks, >>Dave >> >> >>Dave Winchell wrote: >> >> >> >>>Hi Dan, >>> >>>Over the weekend the drift was +18 seconds for each guest (no ntp). >>>The duration was 3900 minutes, so the error for each was +.0077%. >>>Looking back through the data, it appears to drift linearly at >>>this rate. I've attached a plot for rh4u5-64. >>> >>>This accuracy is better than what I've seen before (.03-.05%). >>>This may be due to the different load (ltp vs usex) or to one of the >>>changes I've made recently. I'll do some experimentation to see if >>>there is >>>a fix I should propose. >>> >>>This still doesn't address the radical drift you saw. >>>The next step for me is to run 3.2 and see if I can reproduce it. >>> >>>Regards, >>>Dave >>> >>> >>> >>> >>> >>>Dave Winchell wrote: >>> >>> >>> >>>>Hi Dan, >>>> >>>>Sorry it took me so long, but I finally ran an ltp test today. >>>>Its on rh4u4-64. I'm using the defaults for ltp and using a script >>>>called runltp. I had a usex load on rh4u5-64. No ntpd. >>>>virtual processors / physical processors = 2. >>>> >>>>The clocks drifted -1 sec (4u5) and +1.5 sec (4u4) in 300 minutes >>>>for -.005% and .008%. >>>> >>>>I'm running a 3.1 based hypervisor with some clock related >>>> >>>> >>tweaks that >> >> >>>>I haven't submitted, because I'm still characterizing them. >>>> >>>>I'm stopping the usex load on 4u5-64 now and replacing it with ltp >>>>and will leave the two guests running ltp over the weekend. >>>> >>>>Regards, >>>>Dave >>>> >>>> >>>>Dave Winchell wrote: >>>> >>>> >>>> >>>>>Hi Dan, Deepak: >>>>> >>>>>Thanks for the data. Those drifts are severe - no wonder >>>>> >>>>> >>ntp couldn't >> >> >>>>>keep then in synch. I'll try to reproduce that behaviour >>>>> >>>>> >>here, with >> >> >>>>>my code base. >>>>>If I can't reproduce it, I'll try 3.2. >>>>> >>>>>If you can isolate what ltp is doing during the cliffs, that would >>>>>be very >>>>>helpful. >>>>> >>>>>thanks, >>>>>Dave >>>>> >>>>> >>>>> >>>>> >>>>>Dan Magenheimer wrote: >>>>> >>>>> >>>>> >>>>>>OK, Deepak repeated the test without ntpd and using >>>>>> >>>>>> >>ntpdate -b before >> >> >>>>>>the test. >>>>>> >>>>>>The attached graph shows his results: el5u1-64 (best=~0.07%), >>>>>>el4u5-64 (middle=~0.2%), and el4u5-32 (worst=~0.3%). >>>>>> >>>>>>We will continue to look at LTP to try to isolate. >>>>>> >>>>>>Thanks, >>>>>>Dan >>>>>> >>>>>>P.S. elXuY is essentially RHEL XuY with some patches. >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>>>-----Original Message----- >>>>>>>From: Dave Winchell [mailto:dwinchell@virtualiron.com] >>>>>>>Sent: Wednesday, January 30, 2008 2:45 PM >>>>>>>To: Deepak Patel >>>>>>>Cc: dan.magenheimer@oracle.com; Keir Fraser; >>>>>>>xen-devel@lists.xensource.com; akira.ijuin@oracle.com; >>>>>>> >>>>>>> >>Dave Winchell >> >> >>>>>>>Subject: Re: [Xen-devel] [PATCH] Add a timer mode that disables >>>>>>>pending >>>>>>>missed ticks >>>>>>> >>>>>>> >>>>>>>Dan, Deeepak, >>>>>>> >>>>>>>It may be that the underlying clock error is too great for ntp >>>>>>>to handle. It would be useful if you did not run ntpd >>>>>>>and, instead did ntpdate -b at the start >>>>>>> >>>>>>> >>of the test >> >> >>>>>>>for each guest. Then capture the data as you have been doing. >>>>>>>If the drift is greater than .05%, then we need to address that. >>>>>>> >>>>>>>Another option is, when running ntpd, to enable loop >>>>>>> >>>>>>> >>statistics in >> >> >>>>>>>/etc/ntp.conf >>>>>>>by adding this to the file: >>>>>>> >>>>>>>statistics loopstats >>>>>>>statsdir /var/lib/ntp/ >>>>>>> >>>>>>>Then you will see loop data in that directory. >>>>>>>Correlating the data in the loopstats files with the >>>>>>>peaks in skew would be interesting. You will see >>>>>>> >>>>>>> >>entries of the form >> >> >>>>>>>54495 76787.701 -0.045153303 -132.569229 0.020806776 >>>>>>> >>>>>>> >>239.735511 10 >> >> >>>>>>>Where the second to last column is the Allan Deviation. >>>>>>> >>>>>>> >>When that >> >> >>>>>>>gets over 1000, ntpd is working pretty hard. However, I have not >>>>>>>seen ntpd >>>>>>>completely loose it like you have. >>>>>>> >>>>>>>I'm on vacation until Monday, and won't be reading >>>>>>>email. >>>>>>> >>>>>>>Thanks for all your work on this! >>>>>>> >>>>>>>-Dave >>>>>>> >>>>>>>Deepak Patel wrote: >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>>>Is the graph for RHEL5u1-64? (I've never tested this one.) >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>>I do not know which graph was attached with this. But >>>>>>>> >>>>>>>> >>I saw this >> >> >>>>>>>>behavior in EL4u5 - 32, EL4U5 - 64 and EL5U1 - 64 hvm >>>>>>>> >>>>>>>> >>guests when I >> >> >>>>>>>>was running ltp tests continuously. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>>What was the behaviour of the other guests running? >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>>All pvm guests are fine. But behavior of most of the >>>>>>>> >>>>>>>> >>hvm guests were >> >> >>>>>>>>as described. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>>If they had spikes, were they at the same wall time? >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>>No. They are not at the same wall time. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>>Were the other guests running ltp as well? >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>>Yes all 6 guests (4 hvm and 2 pvm) the guests are running ltp >>>>>>>>continuously. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>>How are you measuring skew? >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>>I was collecting output of "ntpdate -q every >>>>>>>> >>>>>>>> >>>>>>> >>>>>>>300 seconds >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>>(5 minutes) and have created graph based on that. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>>Are you running ntpd? >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>>Yes. ntp was running on all the guests. >>>>>>>> >>>>>>>>I am investigating what causes this spikes and let everyone >>>>>>>> >>>>>>>> >>>>>>> >>>>>>>know what >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>>are my findings. >>>>>>>> >>>>>>>>Thanks, >>>>>>>>Deepak >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>>Anything that you can discover that would be in sync with >>>>>>>>>the spikes would be very helpful! >>>>>>>>> >>>>>>>>>The code that I test with is our product code, which is based >>>>>>>>>on 3.1. So it is possible that something in 3.2 other >>>>>>>>> >>>>>>>>> >>than vpt.c >> >> >>>>>>>>>is the cause. I can test with 3.2, if necessary. >>>>>>>>> >>>>>>>>>thanks, >>>>>>>>>Dave >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>>Dan Magenheimer wrote: >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>>>Hi Dave (Keir, see suggestion below) -- >>>>>>>>>> >>>>>>>>>>Thanks! >>>>>>>>>> >>>>>>>>>>Turning off vhpet certainly helps a lot (though see below). >>>>>>>>>> >>>>>>>>>>I wonder if timekeeping with vhpet is so bad that it >>>>>>>>>> >>>>>>>>>> >>should be >> >> >>>>>>>>>>turned off by default (in 3.1, 3.2, and unstable) until it is >>>>>>>>>>fixed? (I have a patch that defaults it off, can post it if >>>>>>>>>>there is agreement on the above point.) The whole >>>>>>>>>> >>>>>>>>>> >>point of an >> >> >>>>>>>>>>HPET is to provide more precise timekeeping and if vhpet is >>>>>>>>>>worse than vpit, it can only confuse users. Comments? >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>In your testing, are you just measuring % skew over a long >>>>>>>>>>period of time? >>>>>>>>>>We are graphing the skew continuously and >>>>>>>>>>seeing periodic behavior that is unsettling, even with pit. >>>>>>>>>>See attached. Though your algorithm recovers, the "cliffs" >>>>>>>>>>could still cause real user problems. I wonder if there is >>>>>>>>>>anything that can be done to make the "recovery" more >>>>>>>>>>responsive? >>>>>>>>>> >>>>>>>>>>We are looking into what part(s) of LTP is causing >>>>>>>>>> >>>>>>>>>> >>the cliffs. >> >> >>>>>>>>>>Thanks, >>>>>>>>>>Dan >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>>-----Original Message----- >>>>>>>>>>>From: Dave Winchell [mailto:dwinchell@virtualiron.com] >>>>>>>>>>>Sent: Monday, January 28, 2008 8:21 AM >>>>>>>>>>>To: dan.magenheimer@oracle.com >>>>>>>>>>>Cc: Keir Fraser; xen-devel@lists.xensource.com; >>>>>>>>>>>deepak.patel@oracle.com; >>>>>>>>>>>akira.ijuin@oracle.com; Dave Winchell >>>>>>>>>>>Subject: Re: [Xen-devel] [PATCH] Add a timer mode >>>>>>>>>>> >>>>>>>>>>> >>that disables >> >> >>>>>>>>>>>pending >>>>>>>>>>>missed ticks >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>Dan, >>>>>>>>>>> >>>>>>>>>>>I guess I'm a bit out of date calling for clock= usage. >>>>>>>>>>>Looking at linux 2.6.20.4 sources, I think you >>>>>>>>>>> >>>>>>>>>>> >>should specify >> >> >>>>>>>>>>>"clocksource=pit nohpet" on the linux guest bootline. >>>>>>>>>>> >>>>>>>>>>>You can leave the xen and dom0 bootlines as they are. >>>>>>>>>>>The xen and guest clocksources do not need to be the same. >>>>>>>>>>>In my tests, xen is using the hpet for its timekeeping and >>>>>>>>>>>that appears to be the default. >>>>>>>>>>> >>>>>>>>>>>When you boot the guests you should see >>>>>>>>>>> time.c: Using PIT/TSC based timekeeping. >>>>>>>>>>>on the rh4u5-64 guest, and something similar on the others. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>>(xm dmesg shows 8x Xeon 3.2GHz stepping 04, Platform timer >>>>>>>>>>>>14.318MHz HPET.) >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>This appears to be the xen state, which is fine. >>>>>>>>>>>I was wrongly assuming that this was the guest state. >>>>>>>>>>>You might want to look in your guest logs and see >>>>>>>>>>> >>>>>>>>>>> >>what they were >> >> >>>>>>>>>>>picking >>>>>>>>>>>for a clock source. >>>>>>>>>>> >>>>>>>>>>>Regards, >>>>>>>>>>>Dave >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>Dan Magenheimer wrote: >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>>Thanks, I hadn't realized that! No wonder we didn't >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>see the same >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>>>>>>improvement you saw! >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>>Try specifying clock=pit on the linux boot line... >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>I'm confused... do you mean "clocksource=pit" on the Xen >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>command line or >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>>"nohpet" / "clock=pit" / "clocksource=pit" on the guest (or >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>dom0?) command >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>>line? Or both places? Since the tests take awhile, it >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>would be nice >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>>to get this right the first time. Do the Xen and guest >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>clocksources need >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>>to be the same? >>>>>>>>>>>> >>>>>>>>>>>>Thanks, >>>>>>>>>>>>Dan >>>>>>>>>>>> >>>>>>>>>>>>-----Original Message----- >>>>>>>>>>>>*From:* Dave Winchell [mailto:dwinchell@virtualiron.com] >>>>>>>>>>>>*Sent:* Sunday, January 27, 2008 2:22 PM >>>>>>>>>>>>*To:* dan.magenheimer@oracle.com; Keir Fraser >>>>>>>>>>>>*Cc:* xen-devel@lists.xensource.com; >>>>>>>>>>>> >>>>>>>>>>>> >>deepak.patel@oracle.com; >> >> >>>>>>>>>>>>akira.ijuin@oracle.com; Dave Winchell >>>>>>>>>>>>*Subject:* RE: [Xen-devel] [PATCH] Add a timer mode >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>that disables >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>>>>>>pending missed ticks >>>>>>>>>>>> >>>>>>>>>>>> Hi Dan, >>>>>>>>>>>> >>>>>>>>>>>> Hpet timer does have a fairly large error, as I >>>>>>>>>>>>was >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>trying this >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>>>>>> one recently. >>>>>>>>>>>> I don't remember what I got for error, but 1% sounds >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>about right. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> The problem is that hpet is not built on top of vpt.c, >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>the module >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> Keir and I did >>>>>>>>>>>> all the recent work in, for its periodic timer needs. Try >>>>>>>>>>>> specifying clock=pit >>>>>>>>>>>> on the linux boot line. If it still picks the >>>>>>>>>>>> >>>>>>>>>>>> >>hpet, which it >> >> >>>>>>>>>>>> might, let me know >>>>>>>>>>>> and I'll tell you how to get around this. >>>>>>>>>>>> >>>>>>>>>>>> Regards, >>>>>>>>>>>> Dave >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>-------------------------------------------------------------- >> >> >>>>>>>>>>>---------- >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> *From:* Dan Magenheimer >>>>>>>>>>>> >>>>>>>>>>>> >>[mailto:dan.magenheimer@oracle.com] >> >> >>>>>>>>>>>> *Sent:* Fri 1/25/2008 6:50 PM >>>>>>>>>>>> *To:* Dave Winchell; Keir Fraser >>>>>>>>>>>> *Cc:* xen-devel@lists.xensource.com; >>>>>>>>>>>> >>>>>>>>>>>> >>deepak.patel@oracle.com; >> >> >>>>>>>>>>>> akira.ijuin@oracle.com >>>>>>>>>>>> *Subject:* RE: [Xen-devel] [PATCH] Add a timer mode >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>that disables >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> pending missed ticks >>>>>>>>>>>> >>>>>>>>>>>> Sorry for the very late followup on this but we finally >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>were able >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> to get our testing set up again on stable 3.1 >>>>>>>>>>>> >>>>>>>>>>>> >>bits and have >> >> >>>>>>>>>>>> seen some very bad results on 3.1.3-rc1, on the >>>>>>>>>>>> >>>>>>>>>>>> >>order of 1%. >> >> >>>>>>>>>>>> Test enviroment was a 4-socket dual core machine >>>>>>>>>>>> >>>>>>>>>>>> >>with 24GB of >> >> >>>>>>>>>>>> memory running six two-vcpu 2GB domains, four hvm >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>plus two pv. >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>>>>>> All six guests were running LTP simultaneously. >>>>>>>>>>>> >>>>>>>>>>>> >>The four hvm >> >> >>>>>>>>>>>> guests were: RHEL5u1-64, RHEL4u5-32, RHEL5-64, and >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>RHEL4u5-64. >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>>>>>> Timer_mode was set to 2 for 64-bit guests and 0 for >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>32-bit guests. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> All four hvm guests experienced skew around -1%, >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>even the 32-bit >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>>>>>> guest. Less intensive testing didn't exhibit much >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>skew at all. >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>>>>>> A representative graph is attached. >>>>>>>>>>>> >>>>>>>>>>>> Dave, I wonder if some portion of your patches >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>didn't end up in >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>>>>>> the xen trees? >>>>>>>>>>>> >>>>>>>>>>>> (xm dmesg shows 8x Xeon 3.2GHz stepping 04, >>>>>>>>>>>> >>>>>>>>>>>> >>Platform timer >> >> >>>>>>>>>>>> 14.318MHz HPET.) >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> Dan >>>>>>>>>>>> >>>>>>>>>>>> P.S. Many thanks to Deepak and Akira for running tests. >>>>>>>>>>>> >>>>>>>>>>>> > -----Original Message----- >>>>>>>>>>>> > From: xen-devel-bounces@lists.xensource.com >>>>>>>>>>>> > >>>>>>>>>>>> >>>>>>>>>>>> >>[mailto:xen-devel-bounces@lists.xensource.com]On Behalf Of >> >> >>>>>>>>>>>> > Dave Winchell >>>>>>>>>>>> > Sent: Wednesday, January 09, 2008 9:53 AM >>>>>>>>>>>> > To: Keir Fraser >>>>>>>>>>>> > Cc: dan.magenheimer@oracle.com; >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>xen-devel@lists.xensource.com; Dave >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> > Winchell >>>>>>>>>>>> > Subject: Re: [Xen-devel] [PATCH] Add a timer mode that >>>>>>>>>>>> > disables pending >>>>>>>>>>>> > missed ticks >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> > Hi Keir, >>>>>>>>>>>> > >>>>>>>>>>>> > The latest change, c/s 16690, looks fine. >>>>>>>>>>>> > I agree that the code in c/s 16690 is equivalent to >>>>>>>>>>>> > the code I submitted. Also, your version is more >>>>>>>>>>>> > concise. >>>>>>>>>>>> > >>>>>>>>>>>> > The error tests confirm the equivalence. With >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>overnight cpu loads, >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> > the checked in version was accurate to +.048% for sles >>>>>>>>>>>> > and +.038% for red hat. My version was +.046% >>>>>>>>>>>>and >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>+.032% in a >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>>>>>> > 2 hour test. >>>>>>>>>>>> > I don't think the difference is significant. >>>>>>>>>>>> > >>>>>>>>>>>> > i/o loads produced errors of +.01%. >>>>>>>>>>>> > >>>>>>>>>>>> > Thanks for all your efforts on this issue. >>>>>>>>>>>> > >>>>>>>>>>>> > Regards, >>>>>>>>>>>> > Dave >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> > Keir Fraser wrote: >>>>>>>>>>>> > >>>>>>>>>>>> > >Applied as c/s 16690, although the checked-in patch is >>>>>>>>>>>> > smaller. I think the >>>>>>>>>>>> > >only important fix is to pt_intr_post() and the >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>only bit of >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>>>>>> > the patch I >>>>>>>>>>>> > >totally omitted was the change to >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>pt_process_missed_ticks(). >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>>>>>> > I don't think >>>>>>>>>>>> > >that change can be important, but let's see what >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>happens to the >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> error >>>>>>>>>>>> > >percentage... >>>>>>>>>>>> > > >>>>>>>>>>>> > > -- Keir >>>>>>>>>>>> > > >>>>>>>>>>>> > >On 4/1/08 23:24, "Dave Winchell" >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> > > >>>>>>>>>>>> > > >>>>>>>>>>>> > > >>>>>>>>>>>> > >>Hi Dan and Keir, >>>>>>>>>>>> > >> >>>>>>>>>>>> > >>Attached is a patch that fixes some issues with the >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>SYNC policy >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> > >>(no_missed_ticks_pending). >>>>>>>>>>>> > >>I have not tried to make the change the >>>>>>>>>>>> >>>>>>>>>>>> >>minimal one, but, >> >> >>>>>>>>>>>> > rather, just >>>>>>>>>>>> > >>ported into >>>>>>>>>>>> > >>the new code what I know to work well. The error for >>>>>>>>>>>> > >>no_missed_ticks_pending goes from >>>>>>>>>>>> > >>over 3% to .03% with this change according >>>>>>>>>>>> >>>>>>>>>>>> >>to my testing. >> >> >>>>>>>>>>>> > >> >>>>>>>>>>>> > >>Regards, >>>>>>>>>>>> > >>Dave >>>>>>>>>>>> > >> >>>>>>>>>>>> > >>Dan Magenheimer wrote: >>>>>>>>>>>> > >> >>>>>>>>>>>> > >> >>>>>>>>>>>> > >> >>>>>>>>>>>> > >>>Hi Dave -- >>>>>>>>>>>> > >>> >>>>>>>>>>>> > >>>Did you get your correction ported? If so, >>>>>>>>>>>> >>>>>>>>>>>> >>it would be >> >> >>>>>>>>>>>> > nice to see this get >>>>>>>>>>>> > >>>into 3.1.3. >>>>>>>>>>>> > >>> >>>>>>>>>>>> > >>>Note that I just did some very limited testing with >>>>>>>>>>>> > timer_mode=2(=SYNC=no >>>>>>>>>>>> > >>>missed ticks pending) >>>>>>>>>>>> > >>>on tip of xen-3.1-testing (64-bit Linux hv >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>guest) and the >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>>>>>> > worst error I've >>>>>>>>>>>> > >>>seen so far >>>>>>>>>>>> > >>>is 0.012%. But I haven't tried any exotic >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>loads, just LTP. >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>>>>>> > >>> >>>>>>>>>>>> > >>>Thanks, >>>>>>>>>>>> > >>>Dan >>>>>>>>>>>> > >>> >>>>>>>>>>>> > >>> >>>>>>>>>>>> > >>> >>>>>>>>>>>> > >>> >>>>>>>>>>>> > >>> >>>>>>>>>>>> > >>>>-----Original Message----- >>>>>>>>>>>> > >>>>From: Dave Winchell >>>>>>>>>>>> >>>>>>>>>>>> >>[mailto:dwinchell@virtualiron.com] >> >> >>>>>>>>>>>> > >>>>Sent: Wednesday, December 19, 2007 12:33 PM >>>>>>>>>>>> > >>>>To: dan.magenheimer@oracle.com >>>>>>>>>>>> > >>>>Cc: Keir Fraser; Shan, Haitao; >>>>>>>>>>>> > xen-devel@lists.xensource.com; Dong, >>>>>>>>>>>> > >>>>Eddie; Jiang, Yunhong; Dave Winchell >>>>>>>>>>>> > >>>>Subject: Re: [Xen-devel] [PATCH] Add a >>>>>>>>>>>> >>>>>>>>>>>> >>timer mode that >> >> >>>>>>>>>>>> > >>>>disables pending >>>>>>>>>>>> > >>>>missed ticks >>>>>>>>>>>> > >>>> >>>>>>>>>>>> > >>>> >>>>>>>>>>>> > >>>>Dan, >>>>>>>>>>>> > >>>> >>>>>>>>>>>> > >>>>I did some testing with the constant tsc offset >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>SYNC method >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> > >>>>(now called >>>>>>>>>>>> > >>>>no_missed_ticks_pending) >>>>>>>>>>>> > >>>>and found the error to be very high, much larger >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>than 1 %, as >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> > >>>>I recall. >>>>>>>>>>>> > >>>>I have not had a chance to submit a correction. I >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>will try to >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> > >>>>do it later >>>>>>>>>>>> > >>>>this week or the first week in January. My >>>>>>>>>>>> >>>>>>>>>>>> >>version of >> >> >>>>>>>>>>>> constant tsc >>>>>>>>>>>> > >>>>offset SYNC method >>>>>>>>>>>> > >>>>produces .02 % error, so I just need to port >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>that into the >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>>>>>> > >>>>current code. >>>>>>>>>>>> > >>>> >>>>>>>>>>>> > >>>>The error you got for both of those kernels is >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>what I would >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> expect >>>>>>>>>>>> > >>>>for the default mode, delay_for_missed_ticks. >>>>>>>>>>>> > >>>> >>>>>>>>>>>> > >>>>I'll let Keir answer on how to set the time mode. >>>>>>>>>>>> > >>>> >>>>>>>>>>>> > >>>>Regards, >>>>>>>>>>>> > >>>>Dave >>>>>>>>>>>> > >>>> >>>>>>>>>>>> > >>>>Dan Magenheimer wrote: >>>>>>>>>>>> > >>>> >>>>>>>>>>>> > >>>> >>>>>>>>>>>> > >>>> >>>>>>>>>>>> > >>>> >>>>>>>>>>>> > >>>> >>>>>>>>>>>> > >>>>>Anyone make measurements on the final patch? >>>>>>>>>>>> > >>>>> >>>>>>>>>>>> > >>>>>I just ran a 64-bit RHEL5.1 pvm kernel and >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>saw a loss of >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>>>>>> > >>>>> >>>>>>>>>>>> > >>>>> >>>>>>>>>>>> > >>>>> >>>>>>>>>>>> > >>>>> >>>>>>>>>>>> > >>>>about 0.2% with no load. This was >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>xen-unstable tip today >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>>>>>> > >>>>with no options specified. 32-bit was about 0.01%. >>>>>>>>>>>> > >>>> >>>>>>>>>>>> > >>>> >>>>>>>>>>>> > >>>> >>>>>>>>>>>> > >>>> >>>>>>>>>>>> > >>>>>I think I missed something... how do I >>>>>>>>>>>> >>>>>>>>>>>> >>run the various >> >> >>>>>>>>>>>> > >>>>> >>>>>>>>>>>> > >>>>> >>>>>>>>>>>> > >>>>> >>>>>>>>>>>> > >>>>> >>>>>>>>>>>> > >>>>accounting choices and which ones are known to be >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>appropriate >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> > >>>>for which kernels? >>>>>>>>>>>> > >>>> >>>>>>>>>>>> > >>>> >>>>>>>>>>>> > >>>> >>>>>>>>>>>> > >>>> >>>>>>>>>>>> > >>>>>Thanks, >>>>>>>>>>>> > >>>>>Dan >>>>>>>>>>>> > >>>>> >>>>>>>>>>>> > >>>>> >>>>>>>>>>>> > >>>>> >>>>>>>>>>>> > >>>>> >>>>>>>>>>>> > >>>>> >>>>>>>>>>>> > >>>>> >>>>>>>>>>>> > >>>>> >>>>>>>>>>>> > >>>>>>-----Original Message----- >>>>>>>>>>>> > >>>>>>From: xen-devel-bounces@lists.xensource.com >>>>>>>>>>>> > >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>[mailto:xen-devel-bounces@lists.xensource.com]On Behalf Of >> >> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>Keir Fraser >>>>>>>>>>>> > >>>> >>>>>>>>>>>> > >>>> >>>>>>>>>>>> > >>>> >>>>>>>>>>>> > >>>> >>>>>>>>>>>> > >>>>>>Sent: Thursday, December 06, 2007 4:57 AM >>>>>>>>>>>> > >>>>>>To: Dave Winchell >>>>>>>>>>>> > >>>>>>Cc: Shan, Haitao; >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>xen-devel@lists.xensource.com; Dong, >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>>>>>> > Eddie; Jiang, >>>>>>>>>>>> > >>>>>>Yunhong >>>>>>>>>>>> > >>>>>>Subject: Re: [Xen-devel] [PATCH] Add a timer >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>mode that >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>>>>>> > >>>>>>disables pending >>>>>>>>>>>> > >>>>>>missed ticks >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>>Please take a look at xen-unstable >>>>>>>>>>>> >>>>>>>>>>>> >>changeset 16545. >> >> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>>-- Keir >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>>On 26/11/07 20:57, "Dave Winchell" >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>> wrote: >>>>>>>>>>>> > >>>> >>>>>>>>>>>> > >>>> >>>>>>>>>>>> > >>>> >>>>>>>>>>>> > >>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>>>Keir, >>>>>>>>>>>> > >>>>>>> >>>>>>>>>>>> > >>>>>>>The accuracy data I've collected for i/o >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>loads for the >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>>>>>> > >>>>>>>various time protocols follows. In >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>addition, the data >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>>>>>> > >>>>>>>for cpu loads is shown. >>>>>>>>>>>> > >>>>>>> >>>>>>>>>>>> > >>>>>>>The loads labeled cpu and i/o-8 are on an 8 >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>processor AMD >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> box. >>>>>>>>>>>> > >>>>>>>Two guests, red hat and sles 64 bit, 8 >>>>>>>>>>>> >>>>>>>>>>>> >>vcpu each. >> >> >>>>>>>>>>>> > >>>>>>>The cpu load is usex -e36 on each guest. >>>>>>>>>>>> > >>>>>>>(usex is available at >>>>>>>>>>>> http://people.redhat.com/anderson/usex.) >>>>>>>>>>>> > >>>>>>>i/o load is 8 instances of dd if=/dev/hda6 >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>of=/dev/null. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> > >>>>>>> >>>>>>>>>>>> > >>>>>>>The loads labeled i/o-32 are 32 instances of dd. >>>>>>>>>>>> > >>>>>>>Also, these are run on 4 cpu AMD box. >>>>>>>>>>>> > >>>>>>>In addition, there is an idle rh-32bit guest. >>>>>>>>>>>> > >>>>>>>All three guests are 8vcpu. >>>>>>>>>>>> > >>>>>>> >>>>>>>>>>>> > >>>>>>>The loads labeled i/o-4/32 are the same >>>>>>>>>>>> >>>>>>>>>>>> >>as i/o-32 >> >> >>>>>>>>>>>> > >>>>>>>except that the redhat-64 guest has 4 >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>instances of dd. >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>>>>>> > >>>>>>> >>>>>>>>>>>> > >>>>>>>Date Duration Protocol sles, rhat error load >>>>>>>>>>>> > >>>>>>> >>>>>>>>>>>> > >>>>>>>11/07 23 hrs 40 min ASYNC -4.96 sec, >>>>>>>>>>>>+4.42 >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>sec -.006%, >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>>>>>> > +.005% cpu >>>>>>>>>>>> > >>>>>>>11/09 3 hrs 19 min ASYNC -.13 sec, +1.44 >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>sec, -.001%, >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>>>>>> > +.012% cpu >>>>>>>>>>>> > >>>>>>> >>>>>>>>>>>> > >>>>>>>11/08 2 hrs 21 min SYNC -.80 sec, -.34 >>>>>>>>>>>> >>>>>>>>>>>> >>sec, -.009%, >> >> >>>>>>>>>>>> -.004% cpu >>>>>>>>>>>> > >>>>>>>11/08 1 hr 25 min SYNC -.24 sec, -.26 sec, >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>-.005%, -.005% cpu >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> > >>>>>>>11/12 65 hrs 40 min SYNC -18 sec, -8 sec, >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>-.008%, -.003% cpu >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> > >>>>>>> >>>>>>>>>>>> > >>>>>>>11/08 28 min MIXED -.75 sec, -.67 sec -.045%, >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>-.040% cpu >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> > >>>>>>>11/08 15 hrs 39 min MIXED -19. sec,-17.4 >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>sec, -.034%, >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>>>>>> > -.031% cpu >>>>>>>>>>>> > >>>>>>> >>>>>>>>>>>> > >>>>>>> >>>>>>>>>>>> > >>>>>>>11/14 17 hrs 17 min ASYNC -6.1 >>>>>>>>>>>> >>>>>>>>>>>> >>sec,-55.7 sec, -.01%, >> >> >>>>>>>>>>>> > -.09% i/o-8 >>>>>>>>>>>> > >>>>>>>11/15 2 hrs 44 min ASYNC -1.47 >>>>>>>>>>>> >>>>>>>>>>>> >>sec,-14.0 sec, -.015% >> >> >>>>>>>>>>>> > -.14% i/o-8 >>>>>>>>>>>> > >>>>>>> >>>>>>>>>>>> > >>>>>>>11/13 15 hrs 38 min SYNC -9.7 sec,-12.3 >>>>>>>>>>>> >>>>>>>>>>>> >>sec, -.017%, >> >> >>>>>>>>>>>> > -.022% i/o-8 >>>>>>>>>>>> > >>>>>>>11/14 48 min SYNC - .46 sec, - .48 sec, >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>-.017%, -.018% i/o-8 >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> > >>>>>>> >>>>>>>>>>>> > >>>>>>>11/14 4 hrs 2 min MIXED -2.9 sec, -4.15 >>>>>>>>>>>> >>>>>>>>>>>> >>sec, -.020%, >> >> >>>>>>>>>>>> > -.029% i/o-8 >>>>>>>>>>>> > >>>>>>>11/20 16 hrs 2 min MIXED -13.4 sec,-18.1 >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>sec, -.023%, >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>>>>>> > -.031% i/o-8 >>>>>>>>>>>> > >>>>>>> >>>>>>>>>>>> > >>>>>>> >>>>>>>>>>>> > >>>>>>> >>>>>>>>>>>> > >>>>>>>11/21 28 min MIXED -2.01 sec, -.67 sec, -.12%, >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>-.04% i/o-32 >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> > >>>>>>>11/21 2 hrs 25 min SYNC -.96 sec, -.43 >>>>>>>>>>>> >>>>>>>>>>>> >>sec, -.011%, >> >> >>>>>>>>>>>> > -.005% i/o-32 >>>>>>>>>>>> > >>>>>>>11/21 40 min ASYNC -2.43 sec, -2.77 sec -.10%, >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>-.11% i/o-32 >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> > >>>>>>> >>>>>>>>>>>> > >>>>>>>11/26 113 hrs 46 min MIXED -297. sec, >>>>>>>>>>>> >>>>>>>>>>>> >>13. sec -.07%, >> >> >>>>>>>>>>>> > .003% i/o-4/32 >>>>>>>>>>>> > >>>>>>>11/26 4 hrs 50 min SYNC -3.21 sec, 1.44 >>>>>>>>>>>> >>>>>>>>>>>> >>sec, -.017%, >> >> >>>>>>>>>>>> > .01% i/o-4/32 >>>>>>>>>>>> > >>>>>>> >>>>>>>>>>>> > >>>>>>> >>>>>>>>>>>> > >>>>>>>Overhead measurements: >>>>>>>>>>>> > >>>>>>> >>>>>>>>>>>> > >>>>>>>Progress in terms of number of passes >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>through a fixed >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>>>>>> > >>>>>>> >>>>>>>>>>>> > >>>>>>> >>>>>>>>>>>> > >>>>>>> >>>>>>>>>>>> > >>>>>>> >>>>>>>>>>>> > >>>>>>> >>>>>>>>>>>> > >>>>>>> >>>>>>>>>>>> > >>>>>>system workload >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>>>on an 8 vcpu red hat with an 8 vcpu sles idle. >>>>>>>>>>>> > >>>>>>>The workload was usex -b48. >>>>>>>>>>>> > >>>>>>> >>>>>>>>>>>> > >>>>>>> >>>>>>>>>>>> > >>>>>>>ASYNC 167 min 145 passes .868 passes/min >>>>>>>>>>>> > >>>>>>>SYNC 167 min 144 passes .862 passes/min >>>>>>>>>>>> > >>>>>>>SYNC 1065 min 919 passes .863 passes/min >>>>>>>>>>>> > >>>>>>>MIXED 221 min 196 passes .887 passes/min >>>>>>>>>>>> > >>>>>>> >>>>>>>>>>>> > >>>>>>> >>>>>>>>>>>> > >>>>>>>Conclusions: >>>>>>>>>>>> > >>>>>>> >>>>>>>>>>>> > >>>>>>>The only protocol which meets the .05% accuracy >>>>>>>>>>>> > requirement for ntp >>>>>>>>>>>> > >>>>>>>tracking under the loads >>>>>>>>>>>> > >>>>>>>above is the SYNC protocol. The worst case >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>accuracies for >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> > >>>>>>> >>>>>>>>>>>> > >>>>>>> >>>>>>>>>>>> > >>>>>>> >>>>>>>>>>>> > >>>>>>> >>>>>>>>>>>> > >>>>>>> >>>>>>>>>>>> > >>>>>>> >>>>>>>>>>>> > >>>>>>SYNC, MIXED, >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>>>and ASYNC >>>>>>>>>>>> > >>>>>>>are .022%, .12%, and .14%, respectively. >>>>>>>>>>>> > >>>>>>> >>>>>>>>>>>> > >>>>>>>We could reduce the cost of the SYNC >>>>>>>>>>>> >>>>>>>>>>>> >>method by only >> >> >>>>>>>>>>>> > >>>>>>> >>>>>>>>>>>> > >>>>>>> >>>>>>>>>>>> > >>>>>>> >>>>>>>>>>>> > >>>>>>> >>>>>>>>>>>> > >>>>>>> >>>>>>>>>>>> > >>>>>>> >>>>>>>>>>>> > >>>>>>scheduling the extra >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>>>wakeups if a certain number >>>>>>>>>>>> > >>>>>>>of ticks are missed. >>>>>>>>>>>> > >>>>>>> >>>>>>>>>>>> > >>>>>>>Regards, >>>>>>>>>>>> > >>>>>>>Dave >>>>>>>>>>>> > >>>>>>> >>>>>>>>>>>> > >>>>>>>Keir Fraser wrote: >>>>>>>>>>>> > >>>>>>> >>>>>>>>>>>> > >>>>>>> >>>>>>>>>>>> > >>>>>>> >>>>>>>>>>>> > >>>>>>> >>>>>>>>>>>> > >>>>>>> >>>>>>>>>>>> > >>>>>>> >>>>>>>>>>>> > >>>>>>> >>>>>>>>>>>> > >>>>>>>>On 9/11/07 19:22, "Dave Winchell" >>>>>>>>>>>> > >>>>>>>> >>>>>>>>>>>> > >>>>>>>> >>>>>>>>>>>> > >>>>>>>> >>>>>>>>>>>> > >>>>>>>> >>>>>>>>>>>> > >>>>>>>> >>>>>>>>>>>> > >>>>>>>> >>>>>>>>>>>> > >>>>>> wrote: >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>>>> >>>>>>>>>>>> > >>>>>>>> >>>>>>>>>>>> > >>>>>>>> >>>>>>>>>>>> > >>>>>>>> >>>>>>>>>>>> > >>>>>>>>>Since I had a high error (~.03%) for the >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>ASYNC method a >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> > >>>>>>>>> >>>>>>>>>>>> > >>>>>>>>> >>>>>>>>>>>> > >>>>>>>>> >>>>>>>>>>>> > >>>>>>>>> >>>>>>>>>>>> > >>>>>>>>> >>>>>>>>>>>> > >>>>>>>>> >>>>>>>>>>>> > >>>>>>couple of days ago, >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>>>>>I ran another ASYNC test. I think >>>>>>>>>>>> >>>>>>>>>>>> >>there may have >> >> >>>>>>>>>>>> > been something >>>>>>>>>>>> > >>>>>>>>>wrong with the code I used a couple of >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>days ago for >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>>>>>> > >>>>>>>>> >>>>>>>>>>>> > >>>>>>>>> >>>>>>>>>>>> > >>>>>>>>> >>>>>>>>>>>> > >>>>>>>>> >>>>>>>>>>>> > >>>>>>>>> >>>>>>>>>>>> > >>>>>>>>> >>>>>>>>>>>> > >>>>>>ASYNC. It may have been >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>>>>>missing the immediate delivery of interrupt >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>after context >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> > >>>>>>>>> >>>>>>>>>>>> > >>>>>>>>> >>>>>>>>>>>> > >>>>>>>>> >>>>>>>>>>>> > >>>>>>>>> >>>>>>>>>>>> > >>>>>>>>> >>>>>>>>>>>> > >>>>>>>>> >>>>>>>>>>>> > >>>>>>switch in. >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>>>>>My results indicate that either SYNC >>>>>>>>>>>> >>>>>>>>>>>> >>or ASYNC give >> >> >>>>>>>>>>>> > >>>>>>>>> >>>>>>>>>>>> > >>>>>>>>> >>>>>>>>>>>> > >>>>>>>>> >>>>>>>>>>>> > >>>>>>>>> >>>>>>>>>>>> > >>>>>>>>> >>>>>>>>>>>> > >>>>>>>>> >>>>>>>>>>>> > >>>>>>acceptable accuracy, >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>>>>>each running consistently around or under >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>.01%. MIXED has >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> > >>>>>>>>> >>>>>>>>>>>> > >>>>>>>>> >>>>>>>>>>>> > >>>>>>>>> >>>>>>>>>>>> > >>>>>>>>> >>>>>>>>>>>> > >>>>>>>>> >>>>>>>>>>>> > >>>>>>>>> >>>>>>>>>>>> > >>>>>>a fairly high >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>>>>>error of >>>>>>>>>>>> > >>>>>>>>>greater than .03%. Probably too close >>>>>>>>>>>> >>>>>>>>>>>> >>to .05% ntp >> >> >>>>>>>>>>>> > >>>>>>>>> >>>>>>>>>>>> > >>>>>>>>> >>>>>>>>>>>> > >>>>>>>>> >>>>>>>>>>>> > >>>>>>>>> >>>>>>>>>>>> > >>>>>>>>> >>>>>>>>>>>> > >>>>>>>>> >>>>>>>>>>>> > >>>>>>threshold for comfort. >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>>>>>I don't have an overnight run with SYNC. I >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>plan to leave >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> > >>>>>>>>> >>>>>>>>>>>> > >>>>>>>>> >>>>>>>>>>>> > >>>>>>>>> >>>>>>>>>>>> > >>>>>>>>> >>>>>>>>>>>> > >>>>>>>>> >>>>>>>>>>>> > >>>>>>>>> >>>>>>>>>>>> > >>>>>>SYNC running >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>>>>>over the weekend. If you'd rather I can >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>leave MIXED >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>>>>>> > >>>>>>>>> >>>>>>>>>>>> > >>>>>>>>> >>>>>>>>>>>> > >>>>>>>>> >>>>>>>>>>>> > >>>>>>>>> >>>>>>>>>>>> > >>>>>>>>> >>>>>>>>>>>> > >>>>>>>>> >>>>>>>>>>>> > >>>>>>running instead. >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>>>>>It may be too early to pick the protocol and >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>I can run >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> > >>>>>>>>> >>>>>>>>>>>> > >>>>>>>>> >>>>>>>>>>>> > >>>>>>>>> >>>>>>>>>>>> > >>>>>>>>> >>>>>>>>>>>> > >>>>>>>>> >>>>>>>>>>>> > >>>>>>>>> >>>>>>>>>>>> > >>>>>>more overnight tests >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>>>>>next week. >>>>>>>>>>>> > >>>>>>>>> >>>>>>>>>>>> > >>>>>>>>> >>>>>>>>>>>> > >>>>>>>>> >>>>>>>>>>>> > >>>>>>>>> >>>>>>>>>>>> > >>>>>>>>> >>>>>>>>>>>> > >>>>>>>>> >>>>>>>>>>>> > >>>>>>>>> >>>>>>>>>>>> > >>>>>>>>> >>>>>>>>>>>> > >>>>>>>>I'm a bit worried about any unwanted side >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>effects of the >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> > >>>>>>>> >>>>>>>>>>>> > >>>>>>>> >>>>>>>>>>>> > >>>>>>>> >>>>>>>>>>>> > >>>>>>>> >>>>>>>>>>>> > >>>>>>>> >>>>>>>>>>>> > >>>>>>>> >>>>>>>>>>>> > >>>>>>SYNC+run_timer >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>>>>approach -- e.g., whether timer wakeups will >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>cause higher >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> > >>>>>>>> >>>>>>>>>>>> > >>>>>>>> >>>>>>>>>>>> > >>>>>>>> >>>>>>>>>>>> > >>>>>>>> >>>>>>>>>>>> > >>>>>>>> >>>>>>>>>>>> > >>>>>>>> >>>>>>>>>>>> > >>>>>>system-wide CPU >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>>>>contention. I find it easier to think >>>>>>>>>>>> >>>>>>>>>>>> >>through the >> >> >>>>>>>>>>>> > >>>>>>>> >>>>>>>>>>>> > >>>>>>>> >>>>>>>>>>>> > >>>>>>>> >>>>>>>>>>>> > >>>>>>>> >>>>>>>>>>>> > >>>>>>>> >>>>>>>>>>>> > >>>>>>>> >>>>>>>>>>>> > >>>>>>implications of ASYNC. I'm >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>>>>surprised that MIXED loses time, and is less >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>accurate than >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> > >>>>>>>> >>>>>>>>>>>> > >>>>>>>> >>>>>>>>>>>> > >>>>>>>> >>>>>>>>>>>> > >>>>>>>> >>>>>>>>>>>> > >>>>>>>> >>>>>>>>>>>> > >>>>>>>> >>>>>>>>>>>> > >>>>>>ASYNC. Perhaps it >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>>>>delivers more timer interrupts than the other >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>approaches, >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> > >>>>>>>> >>>>>>>>>>>> > >>>>>>>> >>>>>>>>>>>> > >>>>>>>> >>>>>>>>>>>> > >>>>>>>> >>>>>>>>>>>> > >>>>>>>> >>>>>>>>>>>> > >>>>>>>> >>>>>>>>>>>> > >>>>>>and each interrupt >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>>>>event causes a small accumulated error? >>>>>>>>>>>> > >>>>>>>> >>>>>>>>>>>> > >>>>>>>>Overall I would consider MIXED and ASYNC as >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>favourites and >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> > >>>>>>>> >>>>>>>>>>>> > >>>>>>>> >>>>>>>>>>>> > >>>>>>>> >>>>>>>>>>>> > >>>>>>>> >>>>>>>>>>>> > >>>>>>>> >>>>>>>>>>>> > >>>>>>>> >>>>>>>>>>>> > >>>>>>if the latter is >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>>>>actually more accurate then I can >>>>>>>>>>>> >>>>>>>>>>>> >>simply revert the >> >> >>>>>>>>>>>> > changeset that >>>>>>>>>>>> > >>>>>>>>implemented MIXED. >>>>>>>>>>>> > >>>>>>>> >>>>>>>>>>>> > >>>>>>>>Perhaps rather than running more of the same >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>workloads you >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> > >>>>>>>> >>>>>>>>>>>> > >>>>>>>> >>>>>>>>>>>> > >>>>>>>> >>>>>>>>>>>> > >>>>>>>> >>>>>>>>>>>> > >>>>>>>> >>>>>>>>>>>> > >>>>>>>> >>>>>>>>>>>> > >>>>>>could try idle >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>>>>VCPUs and I/O bound VCPUs (e.g., repeated >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>large disc reads >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> > >>>>>>>> >>>>>>>>>>>> > >>>>>>>> >>>>>>>>>>>> > >>>>>>>> >>>>>>>>>>>> > >>>>>>>> >>>>>>>>>>>> > >>>>>>>> >>>>>>>>>>>> > >>>>>>>> >>>>>>>>>>>> > >>>>>>to /dev/null)? We >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>>>>don't have any data on workloads that aren't >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>CPU bound, so >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> > >>>>>>>> >>>>>>>>>>>> > >>>>>>>> >>>>>>>>>>>> > >>>>>>>> >>>>>>>>>>>> > >>>>>>>> >>>>>>>>>>>> > >>>>>>>> >>>>>>>>>>>> > >>>>>>>> >>>>>>>>>>>> > >>>>>>that's really an >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>>>>obvious place to put any further effort imo. >>>>>>>>>>>> > >>>>>>>> >>>>>>>>>>>> > >>>>>>>>-- Keir >>>>>>>>>>>> > >>>>>>>> >>>>>>>>>>>> > >>>>>>>> >>>>>>>>>>>> > >>>>>>>> >>>>>>>>>>>> > >>>>>>>> >>>>>>>>>>>> > >>>>>>>> >>>>>>>>>>>> > >>>>>>>> >>>>>>>>>>>> > >>>>>>>> >>>>>>>>>>>> > >>>>>>>> >>>>>>>>>>>> > >>>>>>>> >>>>>>>>>>>> > >>>>>>>> >>>>>>>>>>>> > >>>>>>_______________________________________________ >>>>>>>>>>>> > >>>>>>Xen-devel mailing list >>>>>>>>>>>> > >>>>>>Xen-devel@lists.xensource.com >>>>>>>>>>>> > >>>>>>http://lists.xensource.com/xen-devel >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>>> >>>>>>>>>>>> > >>>>> >>>>>>>>>>>> > >>>>> >>>>>>>>>>>> > >>>>> >>>>>>>>>>>> > >>>>> >>>>>>>>>>>> > >>>> >>>>>>>>>>>> > >>>> >>>>>>>>>>>> > >>>> >>>>>>>>>>>> > >>>> >>>>>>>>>>>> > >>> >>>>>>>>>>>> > >>> >>>>>>>>>>>> > >>> >>>>>>>>>>>> > >>> >>>>>>>>>>>> > >>diff -r cfdbdca5b831 xen/arch/x86/hvm/vpt.c >>>>>>>>>>>> > >>--- a/xen/arch/x86/hvm/vpt.c Thu Dec 06 15:36:07 >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>2007 +0000 >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>>>>>> > >>+++ b/xen/arch/x86/hvm/vpt.c Fri Jan 04 17:58:16 >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>2008 -0500 >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>>>>>> > >>@@ -58,7 +58,7 @@ static void >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>pt_process_missed_ticks(stru >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>>>>>> > >> >>>>>>>>>>>> > >> missed_ticks = missed_ticks / (s_time_t) >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>pt->period + 1; >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> > >> if ( mode_is(pt->vcpu->domain, >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>no_missed_ticks_pending) ) >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> > >>- pt->do_not_freeze = !pt->pending_intr_nr; >>>>>>>>>>>> > >>+ pt->do_not_freeze = 1; >>>>>>>>>>>> > >> else >>>>>>>>>>>> > >> pt->pending_intr_nr += missed_ticks; >>>>>>>>>>>> > >> pt->scheduled += missed_ticks * pt->period; >>>>>>>>>>>> > >>@@ -127,7 +127,12 @@ static void >>>>>>>>>>>> >>>>>>>>>>>> >>pt_timer_fn(void *data) >> >> >>>>>>>>>>>> > >> >>>>>>>>>>>> > >> pt_lock(pt); >>>>>>>>>>>> > >> >>>>>>>>>>>> > >>- pt->pending_intr_nr++; >>>>>>>>>>>> > >>+ if ( mode_is(pt->vcpu->domain, >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>no_missed_ticks_pending) ) { >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> > >>+ pt->pending_intr_nr = 1; >>>>>>>>>>>> > >>+ pt->do_not_freeze = 0; >>>>>>>>>>>> > >>+ } >>>>>>>>>>>> > >>+ else >>>>>>>>>>>> > >>+ pt->pending_intr_nr++; >>>>>>>>>>>> > >> >>>>>>>>>>>> > >> if ( !pt->one_shot ) >>>>>>>>>>>> > >> { >>>>>>>>>>>> > >>@@ -221,8 +226,6 @@ void pt_intr_post(struct >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>vcpu *v, struct >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>>>>>> > >> return; >>>>>>>>>>>> > >> } >>>>>>>>>>>> > >> >>>>>>>>>>>> > >>- pt->do_not_freeze = 0; >>>>>>>>>>>> > >>- >>>>>>>>>>>> > >> if ( pt->one_shot ) >>>>>>>>>>>> > >> { >>>>>>>>>>>> > >> pt->enabled = 0; >>>>>>>>>>>> > >>@@ -235,6 +238,10 @@ void pt_intr_post(struct vcpu >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>*v, struct >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> > >> pt->last_plt_gtime = >>>>>>>>>>>> >>>>>>>>>>>> >>hvm_get_guest_time(v); >> >> >>>>>>>>>>>> > >> pt->pending_intr_nr = 0; /* >>>>>>>>>>>> >>>>>>>>>>>> >>'collapse' all >> >> >>>>>>>>>>>> > missed ticks */ >>>>>>>>>>>> > >> } >>>>>>>>>>>> > >>+ else if ( mode_is(v->domain, >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>no_missed_ticks_pending) ) { >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>>>>>> > >>+ pt->pending_intr_nr--; >>>>>>>>>>>> > >>+ pt->last_plt_gtime = hvm_get_guest_time(v); >>>>>>>>>>>> > >>+ } >>>>>>>>>>>> > >> else >>>>>>>>>>>> > >> { >>>>>>>>>>>> > >> pt->last_plt_gtime += pt->period_cycles; >>>>>>>>>>>> > >> >>>>>>>>>>>> > >> >>>>>>>>>>>> > > >>>>>>>>>>>> > > >>>>>>>>>>>> > > >>>>>>>>>>>> > > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> > _______________________________________________ >>>>>>>>>>>> > Xen-devel mailing list >>>>>>>>>>>> > Xen-devel@lists.xensource.com >>>>>>>>>>>> > http://lists.xensource.com/xen-devel >>>>>>>>>>>> > >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>> >>> >>> >>-------------------------------------------------------------- >>---------- >> >> >> >> > > >