Re: [PATCH v4 1/2] blk-mq: add async quiesce interface

[Jens Axboe <axboe@kernel.dk>]

On 7/27/20 4:07 PM, Sagi Grimberg wrote:
> Drivers that may have to quiesce a large amount of request queues at once
> (e.g. controller or adapter reset). These drivers would benefit from an
> async quiesce interface such that the can trigger quiesce asynchronously
> and wait for all in parallel.
> 
> This leaves the synchronization responsibility to the driver, but adds
> a convenient interface to quiesce async and wait in a single pass.
> 
> Signed-off-by: Sagi Grimberg <sagi@grimberg.me>
> ---
>  block/blk-mq.c         | 46 ++++++++++++++++++++++++++++++++++++++++++
>  include/linux/blk-mq.h |  4 ++++
>  2 files changed, 50 insertions(+)
> 
> diff --git a/block/blk-mq.c b/block/blk-mq.c
> index abcf590f6238..d913924117d2 100644
> --- a/block/blk-mq.c
> +++ b/block/blk-mq.c
> @@ -209,6 +209,52 @@ void blk_mq_quiesce_queue_nowait(struct request_queue *q)
>  }
>  EXPORT_SYMBOL_GPL(blk_mq_quiesce_queue_nowait);
>  
> +void blk_mq_quiesce_queue_async(struct request_queue *q)
> +{
> +	struct blk_mq_hw_ctx *hctx;
> +	unsigned int i;
> +	int rcu = false;
> +
> +	blk_mq_quiesce_queue_nowait(q);
> +
> +	queue_for_each_hw_ctx(q, hctx, i) {
> +		hctx->rcu_sync = kmalloc(sizeof(*hctx->rcu_sync), GFP_KERNEL);
> +		if (!hctx->rcu_sync) {
> +			/* fallback to serial rcu sync */
> +			if (hctx->flags & BLK_MQ_F_BLOCKING)
> +				synchronize_srcu(hctx->srcu);
> +			else
> +				rcu = true;
> +		} else {
> +			init_completion(&hctx->rcu_sync->completion);
> +			init_rcu_head(&hctx->rcu_sync->head);
> +			if (hctx->flags & BLK_MQ_F_BLOCKING)
> +				call_srcu(hctx->srcu, &hctx->rcu_sync->head,
> +					wakeme_after_rcu);
> +			else
> +				call_rcu(&hctx->rcu_sync->head,
> +					wakeme_after_rcu);
> +		}
> +	}
> +	if (rcu)
> +		synchronize_rcu();
> +}
> +EXPORT_SYMBOL_GPL(blk_mq_quiesce_queue_async);

This won't always be async, and that might matter to some users. I think
it'd be better to put the fallback path into the _wait() part instead,
since the caller should expect that to be blocking/waiting as the name
implies.

Nit picking, but...

-- 
Jens Axboe