Re: [PATCH 16/19] workqueue: reimplement workqueue flushing using color coded works

[Peter Zijlstra <peterz@infradead.org>]

On Fri, 2009-11-20 at 13:46 +0900, Tejun Heo wrote:
> Reimplement workqueue flushing using color coded works.  wq has the
> current work color which is painted on the works being issued via
> cwqs.  Flushing a workqueue is achieved by advancing the current work
> colors of cwqs and waiting for all the works which have any of the
> previous colors to drain.
> 
> Currently there are 16 colors allowing 15 concurrent flushes.  When
> color space gets full, flush attempts are batched up and processed
> together when color frees up, so even with many concurrent flushers,
> the new implementation won't build up huge queue of flushers which has
> to be processed one after another.
> 
> This new implementation leaves only cleanup_workqueue_thread() as the
> user of flush_cpu_workqueue().  Just make its users use
> flush_workqueue() and kthread_stop() directly and kill
> cleanup_workqueue_thread().  As workqueue flushing doesn't use barrier
> request anymore, the comment describing the complex synchronization
> around it in cleanup_workqueue_thread() is removed together with the
> function.
> 
> This new implementation is to allow having and sharing multiple
> workers per cpu.

Hmm, a long while back I did a PI aware workqueue implementation, that
used nested work-lists to implement barriers and flushing.

This colour thing seems very prone to starvation when you want to extend
worklets with priority.