Re: [PATCH 2/4] block: Fix a race between request queue freezing and running queues

[Bart Van Assche <bvanassche@acm.org>]

On Tue, 2019-04-02 at 08:53 +0800, Ming Lei wrote:
> On Mon, Apr 01, 2019 at 02:20:12PM -0700, Bart Van Assche wrote:
> > diff --git a/block/blk-mq.c b/block/blk-mq.c
> > index 3ff3d7b49969..652d0c6d5945 100644
> > --- a/block/blk-mq.c
> > +++ b/block/blk-mq.c
> > @@ -1499,12 +1499,20 @@ void blk_mq_run_hw_queues(struct request_queue *q, bool async)
> >  	struct blk_mq_hw_ctx *hctx;
> >  	int i;
> >  
> > +	/*
> > +	 * Do not run any hardware queues if the queue is frozen or if a
> > +	 * concurrent blk_cleanup_queue() call is removing any data
> > +	 * structures used by this function.
> > +	 */
> > +	if (!percpu_ref_tryget(&q->q_usage_counter))
> > +		return;
> >  	queue_for_each_hw_ctx(q, hctx, i) {
> >  		if (blk_mq_hctx_stopped(hctx))
> >  			continue;
> >  
> >  		blk_mq_run_hw_queue(hctx, async);
> >  	}
> > +	percpu_ref_put(&q->q_usage_counter);
> >  }
> >  EXPORT_SYMBOL(blk_mq_run_hw_queues);
> 
> I don't see it is necessary to add percpu_ref_tryget()/percpu_ref_put()
> in the fast path if we simply release all hctx resource in hctx's
> release handler by the following patch:
> 
> https://lore.kernel.org/linux-block/20190401044247.29881-2-ming.lei@redhat.com/T/#u
> 
> Even we can kill the percpu_ref_tryget_live()/percpu_ref_put() in
> scsi_end_request().

The above approach has the advantages of being easy to review and to maintain.

Patch "[PATCH V2 1/3] blk-mq: free hw queue's resource in hctx's release handler"
makes the block layer more complicated because it introduces a new state for
hardware queues: block driver cleanup has happened (set->ops->exit_hctx(...)) but
the hardware queues are still in use by the block layer core.

Let's see what other reviewers think.

Bart.