RE: [PATCH] Add a timer mode that disables pending missed ticks

["Dan Magenheimer" <dan.magenheimer@oracle.com>]

(One more time on this old thread...)

Hi Dave --

We have a strange but alarming result:  Running rhel5ga-64 with
timer_mode=0 has no skew but with timer_mode=2 has very bad skew.
With rhel5u1-64 (and all rhel4-64 guests) the opposite is true.
We repeated the tests to verify.  In all cases, dmesg shows:

time.c: Using 1.193182 MHz WALL PIT GTOD PIT/TSC timer.

Could you try your heavy load tests with all four combinations
of rhel5ga-64/rhel5u1-64 and timer_mode=0/2 to see if you
can reproduce our results.

Thanks,
Dan

> -----Original Message-----
> From: Dan Magenheimer [mailto:dan.magenheimer@oracle.com]
> Sent: Monday, February 25, 2008 9:42 AM
> To: 'Dave Winchell'
> Cc: 'Keir Fraser'; 'xen-devel@lists.xensource.com'; 'Deepak Patel'
> Subject: RE: [Xen-devel] [PATCH] Add a timer mode that 
> disables pending
> missed ticks
> 
> 
> Hi Dave --
> 
> I've looked into RHEL5-64 a bit more and would appreciate
> any thoughts you might have:
> 
> > >So, some open questions:
> > >[2] In RHEL5, I *think* it is the WALL source that we care about?
> > >
> > >
> > I'll have to check on this too.
> 
> On second look, I may be wrong.  The GTOD clock seems to be
> the one associated with vxtime.mode and vxtime.mode is used in
> linux-2.6.18.8/arch/x86_64/kernel/time.c:main_timer_handler() to
> determine if a tick is lost or not.  Is this the code that we
> want timer_mode=2 to influence?
> 
> > >[1] Is notsc necessary for proper ticks for RHEL4-32/RHEL5-64?
> > >    (I *think* not as it has never come up in any email.)
> > >
> > >
> > I have not investigated this yet.
> 
> Well for RHEL5-64, looking at 
> linux-2.6.18.8/arch/x86_64/kernel/time.c,
> it appears whether notsc is required or not depends in part on the
> underlying virtual AND physical system.
> 
> notsc definitely is involved in the selection of GTOD time
> but notsc can get set not only by kernel command line parameter
> but also by the result of unsynchronized_tsc():
> 
> if (apic_is_clustered_box) notsc=1
> if (box_is_Intel) {
> 	/* C3 delay is worst case HW latency to enter/exit C3 state */
> 	if (C3 delay < 1000) notsc=1
> }
> else { /* e.g. AMD */
> 	if (num_present_cpus() > 1) notsc=1
> }
> 
> I'm not sure what constitutes a clustered HVM guest or how that
> C3 state latency is determined under Xen, but its clear that the
> clocksource can be influenced not only by what clock hardware is
> present and command-line parameters but also by the physical CPU
> and number of guest vcpus.
> 
> Yuck!
> 
> Thanks,
> Dan
> 
> 
>