Re: Re: [PATCH 1/4] CPU online/offline support in Xen

[Frank van der Linden <Frank.Vanderlinden@Sun.COM>]

Haitao Shan wrote:
> Agree. Placing migration in stop_machine context will definitely make
> our jobs easier. I will start making a new patch tomorrow. :)
> I place the migraton code outside the stop_machine_run context, partly
> because I am not quite sure how long it will take to migrate all the
> vcpus away. If it takes too much time, all useful works are blocked
> since all cpus are in the stop_machine context. Of course, I borrowed
> the ideas from kernel, which also let me made the desicion.
>
> 2008/9/10 Keir Fraser <keir.fraser@eu.citrix.com>:
>   
>> I feel this is more complicated than it needs to be.
>>
>> How about clearing VCPUs from the offlined CPU's runqueue from the very end
>> of __cpu_disable()? At that point all other CPUs are safely in softirq
>> context with IRQs disabled, and we are running on the correct CPU (being
>> offlined). We could have a hook into the scheduler subsystem at that point
>> to break affinities, assign to different runqueues, etc. We would just need
>> to be careful not to try an IPI. :-) This approach would not need a
>> cpu_schedule_map (which is really increasing code fragility imo, by creating
>> possible extra confusion about which cpumask is the wright one to use in a
>> given situation).
>>
>> My feeling, unless I've missed something, is that this would make the patch
>> quite a bit smaller and with a smaller spread of code changes.
>>
>>  -- Keir
>>     
This would also address some problems I saw with the patch: race 
conditions regarding migration of VCPUs, because other CPUs may call 
runq_tickle. Or a hypercall may come in changing the VCPU affinity, 
since things are done in 2 stages.

The changes I have are more complicated, because I was working off 
3.1.4, which is our current Xen version. It doesn't have things like 
stop_machine_run. But if the patch is simplified in this manner, it is 
easier for us to use, and we can just backport things like 
stop_machine_run for the time being.

The other issue I was seeing was that cpu_up sometimes did not succeed 
in actually getting a CPU to boot. But there have been a few fixes to 
smpboot.c, so I'll have to see if that always works now.

- Frank