Concern about c/s 21404: Remove special-case paths for start-of-day SMP bringup

Concern about c/s 21404: Remove special-case paths for start-of-day SMP bringup

[Dan Magenheimer]

"Notably ganged TSC calibration is removed"

Really?  We have recently seen machines which generated
the removed message "CPUX had N usecs TSC skew, fixed it up."

I'm not sure I fully understand the scope or the purpose
of all the patches, but since the TSC is the foundation for
Xen system time which is the foundation for virtually all time
handling in all guests (because even the non-TSC emulated
timers are built on Xen system time), I think you are introducing
a big risk here for an unknown number of existing systems.
Some of the problems may be obvious because a guest often
won't boot when clocks are screwy, but other problems may
not manifest until "later" when under a heavy load that
causes more frequent vcpu->pcpu switching and may not
be at all obviously related to clock problems.  (We've
recently seen that also with a Solaris HV guest running
on a pre-4.0 Xen where rdtsc is not yet emulated.)

Note that Linux is *adding* code to attempt to make TSC more
reliable, while it appears this Xen patch is taking code away.

If I understand, the purpose is to provide a better
way to online CPUs so as to handle CPU hotplug better?
While hotplug is a nice feature, am I correct that the
feature is supported in a very small number of
systems?  And is actually used in an even smaller
number?

My two cents: Doesn't seem like a good tradeoff to me...

Re: Concern about c/s 21404: Remove special-case paths for start-of-day SMP bringup

[Keir Fraser]

I'll add an __initcall() to rendezvous the CPUs and whack their TSCs.

 K.

On 19/05/2010 17:24, "Dan Magenheimer" <dan.magenheimer@oracle.com> wrote:

> "Notably ganged TSC calibration is removed"
> 
> Really?  We have recently seen machines which generated
> the removed message "CPUX had N usecs TSC skew, fixed it up."
> 
> I'm not sure I fully understand the scope or the purpose
> of all the patches, but since the TSC is the foundation for
> Xen system time which is the foundation for virtually all time
> handling in all guests (because even the non-TSC emulated
> timers are built on Xen system time), I think you are introducing
> a big risk here for an unknown number of existing systems.
> Some of the problems may be obvious because a guest often
> won't boot when clocks are screwy, but other problems may
> not manifest until "later" when under a heavy load that
> causes more frequent vcpu->pcpu switching and may not
> be at all obviously related to clock problems.  (We've
> recently seen that also with a Solaris HV guest running
> on a pre-4.0 Xen where rdtsc is not yet emulated.)
> 
> Note that Linux is *adding* code to attempt to make TSC more
> reliable, while it appears this Xen patch is taking code away.
> 
> If I understand, the purpose is to provide a better
> way to online CPUs so as to handle CPU hotplug better?
> While hotplug is a nice feature, am I correct that the
> feature is supported in a very small number of
> systems?  And is actually used in an even smaller
> number?
> 
> My two cents: Doesn't seem like a good tradeoff to me...