Re: [PATCH 4/7] sched: implement force_cpus_allowed()

[Peter Zijlstra <peterz@infradead.org>]

On Mon, 2009-12-07 at 19:34 +0900, Tejun Heo wrote:
> Hello,
> 
> On 12/07/2009 05:35 PM, Peter Zijlstra wrote:
> >> * PF_THREAD_BOUND.  This is used to mark tasks which are bound to a
> >>   cpu using kthread_bind() to be bound permanently.  However, new
> >>   trustee based workqueue hotplugging decouples per-cpu workqueue
> >>   flushing with cpu hot plug/unplugging.  This is necessary because
> >>   with cmwq, long running works can be served by regular workqueues,
> >>   so delaying completion of hot plug/unplugging till certain works are
> >>   flushed isn't feasible.  So, what becomes necessary is the ability
> >>   to re-bind tasks which has PF_THREAD_BOUND set but unbound from its
> >>   now offline cpu which is coming online again.
> > 
> > I'm not at all sure I like that. I'd be perfectly happy with delaying
> > the hot-unplug.
> > 
> > The whole cpu hotplug mess is tricky enough as it is and I see no
> > compelling reason to further complicate it. If people are really going
> > to enqueue strict per-cpu worklets (queue_work_on()) that takes seconds
> > to complete, then they get to keep the results of that, which includes
> > slow hot unplug.
> > 
> > Having an off-line cpu still process code like it was online is asking
> > for trouble, don't go there.
> 
> We're already there.  Users of workqueue which require strict CPU
> affinity are required to flush respective works from CPU down
> notifiers and fire them as necessary on up notifiers; otherwise, works
> will continue to run until they're done after the cpu went down
> regardless of whether explicit queue_work_on() was used or not.

So we seem to do cleanup_workqueue_thread() from CPU_POST_DEAD, but at
that time any thread that might still be around will most certainly not
be running on the offlined cpu anymore.

If you really want to ensure you remain on the cpu, you have to complete
from CPU_DOWN_PREPARE.

We're not running things from offline CPUs.