Re: [patch 1/3] raid5: offload stripe handle to workqueue

[Shaohua Li <shli@kernel.org>]

On Tue, Jul 30, 2013 at 08:53:06AM -0400, Tejun Heo wrote:
> Hello,
> 
> On Tue, Jul 30, 2013 at 01:52:08PM +0800, shli@kernel.org wrote:
> > +static void raid5_wakeup_stripe_thread(struct stripe_head *sh)
> > +{
> > +	struct r5conf *conf = sh->raid_conf;
> > +	struct r5worker_group *group;
> > +	int i;
> > +
> > +	if (conf->worker_cnt_per_group == 0) {
> > +		md_wakeup_thread(conf->mddev->thread);
> > +		return;
> > +	}
> > +
> > +	group = conf->worker_groups + cpu_to_group(sh->cpu);
> > +
> > +	for (i = 0; i < conf->worker_cnt_per_group; i++)
> > +		queue_work_on(sh->cpu, raid5_wq, &group->workers[i].work);
> > +}
> 
> Another general suggestion.  Using workqueue mechanism simply as
> thread dispatching mechanism like above and then buliding your own
> work dispatching code on top is usually a poor form.  It usually is
> much better to assign a single unit of work to a single work item as
> it allows things like per work unit flushing and much easier
> implementation of freezing.  It's possible that you have some
> overriding constraints here but if so it'd be nice if you can explain
> it.

Ok, I should explain here. I can't add a work_struct for each stripe, because
this will stress workqueue very hard. My system handles > 1M/s stripes, which
makes workqueue pool lock contended very hard.

Thanks,
Shaohua