Re: [RFC PATCH] blk-mq: implement queue quiesce via percpu_ref for BLK_MQ_F_BLOCKING

Sagi Grimberg
2020-07-29T22:37:36Z

>>>> In case of BLK_MQ_F_BLOCKING, blk-mq uses SRCU to mark read critical
>>>> section during dispatching request, then request queue quiesce is based on
>>>> SRCU. What we want to get is low cost added in fast path.
>>>>
>>>> However, from srcu_read_lock/srcu_read_unlock implementation, not see
>>>> it is quicker than percpu refcount, so use percpu_ref to implement
>>>> queue quiesce. This usage is cleaner and simpler & enough for implementing
>>>> queue quiesce. The main requirement is to make sure all read sections to observe
>>>> QUEUE_FLAG_QUIESCED once blk_mq_quiesce_queue() returns.
>>>>
>>>> Also it becomes much easier to add interface of async queue quiesce.
>>>
>>> BTW, no obvious IOPS difference is observed with this patch applied when running
>>> io on null_blk(blocking, submit_queues=32) in one dual-socket, 32cores system.
>>
>> Thanks Ming, can you test for non-blocking on the same setup?
>
> OK, I can do that, but this patch supposes to not affect non-blocking,
> care to share your motivation for testing non-blocking?

I think it will be a significant improvement to have a single code path.
The code will be more robust and we won't need to face issues that are
specific for blocking.

If the cost is negligible, I think the upside is worth it.