Re: [PATCH 04/10] block: initial patch for on-stack per-task plugging

[Jens Axboe <jaxboe@fusionio.com>]

On 2011-01-24 20:36, Jeff Moyer wrote:
> Jens Axboe <jaxboe@fusionio.com> writes:
> 
> This looks mostly good.  I just have a couple of questions, listed below.
> 
>> +/*
>> + * Attempts to merge with the plugged list in the current process. Returns
>> + * true if merge was succesful, otherwise false.
>> + */
>> +static bool check_plug_merge(struct task_struct *tsk, struct request_queue *q,
>> +			     struct bio *bio)
>> +{
> 
> Would a better name for this function be attempt_plug_merge?

Most likely :-). I'll change it.

>> +	plug = current->plug;
>> +	if (plug && !sync) {
>> +		if (!plug->should_sort && !list_empty(&plug->list)) {
>> +			struct request *__rq;
>> +
>> +			__rq = list_entry_rq(plug->list.prev);
>> +			if (__rq->q != q)
>> +				plug->should_sort = 1;
> 
> [snip]
> 
>> +static int plug_rq_cmp(void *priv, struct list_head *a, struct list_head *b)
>> +{
>> +	struct request *rqa = container_of(a, struct request, queuelist);
>> +	struct request *rqb = container_of(b, struct request, queuelist);
>> +
>> +	return !(rqa->q == rqb->q);
>> +}
> 
> 
>> +static void __blk_finish_plug(struct task_struct *tsk, struct blk_plug *plug)
>> +{
> 
> [snip]
> 
>> +	if (plug->should_sort)
>> +		list_sort(NULL, &plug->list, plug_rq_cmp);
> 
> The other way to do this is to just keep track of which queues you need
> to run after exhausting the plug list.  Is it safe to assume you've done
> things this way to keep each request queue's data structures cache hot
> while working on it?

But then you get into memory problems as well, as the number of
different queues could (potentially) be huge. In reality they will not
be, but it's something that has to be handled. And if you track those
queues, that still means you have to grab each queue lock twice instead
of just once.

There are probably areas where the double lock approach may be faster
than spending cycles on a sort, but in practice I think the sort will be
faster. It's something we can play with, though.

>> +static inline void blk_flush_plug(struct task_struct *tsk)
>> +{
>> +	struct blk_plug *plug = tsk->plug;
>> +
>> +	if (unlikely(plug))
>> +		__blk_flush_plug(tsk, plug);
>> +}
> 
> Why is that unlikely?

Main caller is the CPU scheduler, when someone is scheduled out. So the
logic is that unless you're very IO intensive, you'll be more likely to
go to sleep without anything waiting on the plug list than not. This
puts it out-of-line in the CPU scheduler, to keep the cost to a minimum.

-- 
Jens Axboe