From mboxrd@z Thu Jan 1 00:00:00 1970 From: Keir Fraser Subject: Re: [PATCH] rendezvous-based local time calibration WOW! Date: Sun, 03 Aug 2008 18:24:46 +0100 Message-ID: References: <20080803105029796.00000008444@djm-pc> Mime-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <20080803105029796.00000008444@djm-pc> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: "dan.magenheimer@oracle.com" , "Xen-Devel (E-mail)" Cc: Ian Pratt , Dave Winchell List-Id: xen-devel@lists.xenproject.org It's not safe to poke a new timestamp record from an interrupt handler (which is what the smp_call_function() callback functions are). Users of the timestamp records (e.g., get_s_time) need local_irq_save/restore() or an equivalent of the Linux seqlock. The latter is likely faster. I'm dubious about update_vcpu_system_time() from an interrupt handler too. It needs thought about how it might race with a context switch (change of 'current') or if it interrupts an existing invocation of update_vcpu_system_time(). -- Keir On 3/8/08 17:50, "Dan Magenheimer" wrote: > The synchronization of local_time_calibration (l_t_c) via > round-to-nearest-epoch provided some improvement, but I was > still seeing skew up to 16usec and higher. I measured the > temporal distance between the rounded-epoch vs when ltc > was actually running to ensure there wasn't some kind of > bug and found that l_t_c was running up to 150us after the > round-epoch and sometimes up to 50us before. I guess this > is the granularity of setting a Xen timer. While it seemed > that +/- 100us shouldn't cause that much skew, I finally > decided to try synchronization-via-rendezvous, as suggested > by Ian here: > > http://lists.xensource.com/archives/html/xen-devel/2008-07/msg01074.html > http://lists.xensource.com/archives/html/xen-devel/2008-07/msg01080.html > > The result is phenomenal... using this approach (in attached > patch), I have yet to see a skew exceed 1usec!!! So this is > about a 10-fold increase in accuracy vs the rounded-epoch > method and about 20-fold over the one-epoch-from-NOW() method. > > The platform time is now read once for all processors rather > than once per processor. (Actually, it is read once again > in platform_time_calibration()... by "inlining" that routine > into master_local_time_calibration() that extra read can > be -- and probably should be -- avoided too.) > > It may be too late to get this into 3.3.0 but, if so, please > consider it asap for 3.3.1 rather than just xen-unstable/3.4. > > Dan > > =================================== > Thanks... for the memory > I really could use more / My throughput's on the floor > The balloon is flat / My swap disk's fat / I've OOM's in store > Overcommitted so much > (with apologies to the late great Bob Hope)