Re: [PATCH 4/7] sched: implement force_cpus_allowed()

[Peter Zijlstra <peterz@infradead.org>]

On Tue, 2009-12-08 at 20:56 +0900, Tejun Heo wrote:
> Hello,
> 
> On 12/08/2009 08:48 PM, Peter Zijlstra wrote:
> > Why bother with that?
> > 
> > workqueue's CPU_POST_DEAD will flush the workqueue and destroy all
> > threads under cpu_add_remove_lock, which excludes the cpu from coming
> > back up before its fully destroyed.
> > 
> > So there's no remaining tasks to be migrated back.
> > 
> > Changing that semantics is not worthwhile.
> 
> It is worthwhile because the goal is to unify all worker pool
> mechanisms.  So, slow_work or whatnot (scsi EHs, FS specific worker
> pools, async workers used for parallel probing) will all be converted
> to use workqueue instead and with that we can't afford to wait for all
> works to flush to down a cpu.  All that's necessary to implement that
> is migrating back the unbound workers which can be implemented as a
> separate piece of code apart from regular operation.  It would even
> benefit the current implementation as it makes cpu up/down operations
> much more deterministic.

Hotplug and deterministic are not to be used in the same sentence, its
an utter slow path and I'd much rather have simple code than clever code
there -- there's been way too many 'interesting' hotplug problems.

If there is work being enqueued that takes more than a few seconds to
complete then I'm thinking there's something seriously wrong and up to
that point its perfectly fine to simply wait for it.

Furthermore if it's objective is to cater to generic thread pools then I
think its an utter fail simply because it mandates strict cpu affinity,
that basically requires you to write a work scheduler to balance work
load etc.. Much easier is a simple unbounded thread pool that gets
balanced by the regular scheduler.

/me liking this stuff less and less :/