Re: The SMP RHEL 5.1 PAE guest can't boot up issue

Dong, Eddie
2008-02-25T23:46:25Z

>> I don't know if the patch was still needed now, since it was posted
>> long ago(I don't know which issue it solved). I'd like to post a
>> revert patch if necessary.
>>
>
> I believe the patch is still necessary, since we still need to
> guarantee that a vcpu's tsc is monotonous. I think there are three
> issues to be addressed:
>
> 1. The majority of intel machines don't need the offset adjustment
> since they already have a constant rate tsc that is synchronized on
> all cpus. I think this is indicated by X86_FEATURE_CONSTANT_TSC
> (though I'm not 100% certain if it means that the rate is the same
> for all cpus, Thomas can you clarify?)

So why not make the TSC_OFFSET adjustment conditional?
The original patch doesn't bring any benefit for those platforms
with CONSTANT TSC, especially if it is majority,
while the accumurated difference due to the patch will be very big
which makes guest timer worse.

>
> This will improve tsc quality for those machines, but we can't depend
> on it, since some machines don't have constant tsc. Further, I don't
> think really large machines can have constant tsc since clock
> distribution becomes difficult or impossible.

For NUMA machines, this is an issue, but depend on how you support
NUMA. One way is to bind VCPUs of a guest to same node if guest is not
NUMA, if this is the model, then we don't have issue.
I think Xen is planning in this way and it is same for KVM.

>
> 2. We should implement round robin and lowest priority like qemu does.
> Xen does the same thing:
>
>> /* HACK: Route IRQ0 only to VCPU0 to prevent time jumps. */
>> #define IRQ0_SPECIAL_ROUTING 1
> in arch/x86/hvm/vioapic.c, at least for irq 0.

We did same thing in Xen long time ago to avoid this issue.
It helps but not perfect.

>
> 3. The extra migrations on vcpu 0 are likely due to its role servicing
> I/O on behalf of the entire virtual machine. We should move this
> extra work to an independent thread. I have done some work in this
> area. It is becoming more important as kvm becomes more scalable.