Re: [PATCH 0/7] KVM: Suspend and cpu hotplug fixes

[Avi Kivity <avi@qumranet.com>]

Shaohua Li wrote:
> On Thu, 2007-05-24 at 20:10 +0800, Avi Kivity wrote:
>   
>> The following patchset makes kvm more robust wrt cpu hotunplug, and
>> makes suspend-to-ram actually work.  Suspend-to-disk benefits from
>> the cpu hotunplug improvements as well.
>>
>> The major issue is that KVM wants to disable the virtualization
>> extensions at a point in time when no user processes are schedulable
>> on the victim cpu.  No current notifier exists, so a new one,
>> CPU_DYING,
>> is added for the purpose.
>>
>> Should there be no objections, I will submit this patchset for
>> inclusion
>> in 2.6.22, and backport it to 2.6.21.stable.
>>     
> Is it possible disabling kvm can be done at the begining of play_dead?
> take_cpu_done is designed to run fast.
>
>   

It's possible, but I have issues with play_dead():

- it is arch specific, so we need to modify i386, x86_64, and ia64 (when
we have an ia64 kvm port)
- there is no hook available here to call modules like the hotplug notifier

I estimate that that take_cpu_down will run for about a millisecond if
there are a few hundred vcpus which have last run on the dying cpu (and
that's an extreme case, which is not expected in normal operation).  If
that's too much, it can be reduced as follows:

- add a per-cpu list of vcpus that have last run on a cpu, maintained at
runtime
- on CPU_DOWN_PREPARE, walk the list and vmclear any vcpus that last ran
on the dying cpu
- on CPU_DYING (take_cpu_down), walk the list again an vmclear any vcpus
that managed to get scheduled to the dying vcpu again.  the list should
not have more than 1-2 entries in normal operation.


-- 
Do not meddle in the internals of kernels, for they are subtle and quick to panic.