Re: [patch 1/3] raid5: offload stripe handle to workqueue

[Shaohua Li <shli@kernel.org>]

On Tue, Jul 30, 2013 at 09:57:51AM -0400, Tejun Heo wrote:
> Hello,
> 
> On Tue, Jul 30, 2013 at 09:07:08PM +0800, Shaohua Li wrote:
> > Ok, I should explain here. I can't add a work_struct for each stripe, because
> > this will stress workqueue very hard. My system handles > 1M/s stripes, which
> > makes workqueue pool lock contended very hard.
> 
> It doesn't have to be embedding work_struct in each stripe and
> schduling them altogether.  It's more about scheduling "work units"
> rather than "workers" - ie. letting each scheduled work item handle
> single work unit rather than making it dispatch multiple work items.
> It may make controlling concurrency a bit more interesting but you can
> always do it with workqueue_set_max_active(), which is the intended
> usage anyway.

stripe is the work unit actually. As I said, if I queue a work for each stripe,
just queue_work() will make the system blast because of the pwq->pool->lock
contention. dispatching one work has another side effect that I can't add block
plug. Since this is the queue stage concurrency problem,
workqueue_set_max_active() doesn't help.

Thanks,
Shaohua