Re: [PATCH] sched: Simplify cpu-hot-unplug task migration

[Peter Zijlstra <peterz@infradead.org>]

On Thu, 2010-11-18 at 15:05 +0100, Oleg Nesterov wrote:
> > -     /* Wait for it to sleep (leaving idle task). */
> > -     while (!idle_cpu(cpu))
> > -             yield();
> > +     /*
> > +      * The migration_call() CPU_DYING callback will have removed all
> > +      * runnable tasks from the cpu, there's only the idle task left now
> > +      * that the migration thread is done doing the stop_machine thing.
> > +      */
> > +     BUG_ON(!idle_cpu(cpu));
> 
> I am not sure.
> 
> Yes, we know for sure rhat the only runnable task is rq->idle.
> But only after migration thread calls schedule() and switches to the
> idle thread.
> 
> However, I see nothing which can guarantee this. Migration thread
> running on the dead cpu wakes up the caller of stop_cpus() before
> it calls schedule(), _cpu_down() can check rq->curr before it was
> changed.
> 
> No?
> 
> 
> 
> Hmm. In fact, I think it is possible that cpu_stopper_thread() can
> have more cpu_stop_work's queued when __stop_machine() returns.
> This has nothing to do with this patch, but I think it makes sense
> to clear stopper->enabled at CPU_DYING stage as well (of course,
> this needs a separate patch). 

Hmm, I think you're right, although I haven't hit that case during
testing.

There is no firm guarantee the dying cpu actually got to running the
idle thread (there's a guarantee it will at some point), so we ought to
maintain that wait-loop, possibly using cpu_relax(), I don't see the
point in calling yield() here.