From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Dan Magenheimer" Subject: RE: Xen system skew MUCH worse than tsc skew (was RE: RE: [PATCH] record max stime skew (was RE: [PATCH] strictly increasing hvm guest time)) Date: Fri, 11 Jul 2008 14:53:39 -0600 Message-ID: <20080711145339796.00000003744@djm-pc> References: Reply-To: "dan.magenheimer@oracle.com" Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Return-path: In-Reply-To: List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: Keir Fraser , "Xen-Devel (E-mail)" Cc: Dave Winchell List-Id: xen-devel@lists.xenproject.org > I didn't measure skew across CPUs. I measured jitter between = > one local TSC > and the chosen platform timer for calibration (in my case I = > think this was > the HPET). I did this because getting a consistent tick rate from the > platform timer, and from each local TSC, is the basis for the = > calibration > algorithm. The more jitter there is between them, the less = > well it will > work. > = > I implemented a user-space program to collect the required = > stats. It used > CLI/STI to prevent getting interrupted when reading the timer pair. Hmmm... if the TSC is known to be stable*, is there any reason to do the calibration vs the platform timer? If TSC is stable, could we instead just do essentially a divide by cpu_ghz in get_s_time() and be done, no periodic local_time_calibration() necessary? Since TSC is stable on many newer platforms, it would be nice to use this feature to decrease skew for guests (both PV and HV). * stable is the term used by Linux to mean that there's no skew between the different TSC's in an SMP system I gave this a try and it seems to work so far. (Fortunately, my CPU is 3GHz so I just had to divide by 3... I'm not sure how to divide by a non-integer.) Max skew for stime is holding steady at 270nsec, >40x better than periodic calibration w/hpet. If this sounds good, a design question: Should this be controlled: 1) by a boot option, or 2) by the TSC_CONSTANT cpu flag, or 3) when determined dynamically to be safe using code similar to arch/x86/tsc_sync.c in recent Linux kernels (1) is by far the easiest (perhaps not too late for 3.3?) while (3) is clearly the best for users but adds lots of code (bloat/untested) Thanks, Dan