Re: [RFC PATCH] blk-mq: implement queue quiesce via percpu_ref for BLK_MQ_F_BLOCKING

Sagi Grimberg
2020-07-29T15:42:35Z

>> In case of BLK_MQ_F_BLOCKING, blk-mq uses SRCU to mark read critical
>> section during dispatching request, then request queue quiesce is based on
>> SRCU. What we want to get is low cost added in fast path.
>>
>> However, from srcu_read_lock/srcu_read_unlock implementation, not see
>> it is quicker than percpu refcount, so use percpu_ref to implement
>> queue quiesce. This usage is cleaner and simpler & enough for implementing
>> queue quiesce. The main requirement is to make sure all read sections to observe
>> QUEUE_FLAG_QUIESCED once blk_mq_quiesce_queue() returns.
>>
>> Also it becomes much easier to add interface of async queue quiesce.
>
> BTW, no obvious IOPS difference is observed with this patch applied when running
> io on null_blk(blocking, submit_queues=32) in one dual-socket, 32cores system.

Thanks Ming, can you test for non-blocking on the same setup?

I can test some reset storms during traffic.