Re: [PATCH v1 5/9] block: loop: convert to blk-mq

[Jens Axboe <axboe@kernel.dk>]

On 2014-08-18 06:53, Ming Lei wrote:
> On Mon, Aug 18, 2014 at 9:22 AM, Ming Lei <ming.lei@canonical.com> wrote:
>> On Mon, Aug 18, 2014 at 1:48 AM, Jens Axboe <axboe@kernel.dk> wrote:
>>> On 2014-08-16 02:06, Ming Lei wrote:
>>>>
>>>> On 8/16/14, Jens Axboe <axboe@kernel.dk> wrote:
>>>>>
>>>>> On 08/15/2014 10:36 AM, Jens Axboe wrote:
>>>>>>
>>>>>> On 08/15/2014 10:31 AM, Christoph Hellwig wrote:
>>>>>>>>
>>>>>>>> +static void loop_queue_work(struct work_struct *work)
>>>>>>>
>>>>>>>
>>>>>>> Offloading work straight to a workqueue dosn't make much sense
>>>>>>> in the blk-mq model as we'll usually be called from one.  If you
>>>>>>> need to avoid the cases where we are called directly a flag for
>>>>>>> the blk-mq code to always schedule a workqueue sounds like a much
>>>>>>> better plan.
>>>>>>
>>>>>>
>>>>>> That's a good point - would clean up this bit, and be pretty close to a
>>>>>> one-liner to support in blk-mq for the drivers that always need blocking
>>>>>> context.
>>>>>
>>>>>
>>>>> Something like this should do the trick - totally untested. But with
>>>>> that, loop would just need to add BLK_MQ_F_WQ_CONTEXT to it's tag set
>>>>> flags and it could always do the work inline from ->queue_rq().
>>>>
>>>>
>>>> I think it is a good idea.
>>>>
>>>> But for loop, there may be two problems:
>>>>
>>>> - default max_active for bound workqueue is 256, which means several slow
>>>> loop devices might slow down whole block system. With kernel AIO, it won't
>>>> be a big deal, but some block/fs may not support direct I/O and still
>>>> fallback to
>>>> workqueue
>>>>
>>>> - 6. Guidelines of Documentation/workqueue.txt
>>>> If there is dependency among multiple work items used during memory
>>>> reclaim, they should be queued to separate wq each with WQ_MEM_RECLAIM.
>>>
>>>
>>> Both are good points. But I think this mainly means that we should support
>>> this through a potentially per-dispatch queue workqueue, separate from
>>> kblockd. There's no reason blk-mq can't support this with a per-hctx
>>> workqueue, for drivers that need it.
>>
>> Good idea, and per-device workqueue should be enough if
>> BLK_MQ_F_WQ_CONTEXT flag is set.
>
> Maybe for most of cases per-device class(driver) workqueue should be
> enough since dependency between devices driven by same driver
> isn't common, for example, loop over loop is absolutely insane.

It's insane, but it can happen. And given how cheap it is to do a 
workqueue, I don't see a reason why we should not. Loop over loop might 
seem nutty, but it's not that far out into the realm of nutty things 
that people end up doing.

> I will keep the work queue in loop-mq V2, and it should be easy to switch
> to the mechanism once it is ready.

Reworked a bit more:

http://git.kernel.dk/?p=linux-block.git;a=commit;h=a323185a761b9a54dc340d383695b4205ea258b6

Lets base loop-mq on the blk-mq workqueues, it would simplify it quite a 
bit and I don't think there's much point in doing v1 and then ripping it 
out for v2. Especially since it isn't queued up for 3.18 yet.

-- 
Jens Axboe

--
To unsubscribe, send a message with 'unsubscribe linux-aio' in
the body to majordomo@kvack.org.  For more info on Linux AIO,
see: http://www.kvack.org/aio/
Don't email: <a href=mailto:"aart@kvack.org">aart@kvack.org</a>