[PATCH 0/4] some random cleanups for blk-mq.c

[PATCH 0/4] some random cleanups for blk-mq.c

[Jinlong Chen]

Patch 1 updates the outdated comment of blk_mq_quiesce_queue_nowait().
Patch 2 improves the error handling blk_mq_alloc_rq_map(). Patch 3 and 4
improve readability of request alloc routines.

Jinlong Chen (4):
  blk-mq: update comment for blk_mq_quiesce_queue_nowait()
  blk-mq: improve error handling in blk_mq_alloc_rq_map()
  blk-mq: use if-else instead of goto in blk_mq_alloc_cached_request()
  blk-mq: improve readability of blk_mq_alloc_request()

 block/blk-mq.c | 105 ++++++++++++++++++++++++++++---------------------
 1 file changed, 61 insertions(+), 44 deletions(-)

-- 
2.31.1

[PATCH 1/4] blk-mq: update comment for blk_mq_quiesce_queue_nowait()

[Jinlong Chen]

blk_mq_quiesce_queue_nowait() is now reasonably used by scsi_host_block()
to avoid calling synchronize_rcu() for each LUN. We should not consider to
remove it.

See commit f983622ae60516d634008c7b1ff9ffff4f7bb8ae ("scsi: core: Avoid
calling synchronize_rcu() for each device in scsi_host_block()")

Signed-off-by: Jinlong Chen <nickyc975@zju.edu.cn>
---
 block/blk-mq.c | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/block/blk-mq.c b/block/blk-mq.c
index 623e8a506539..7ceceea91b3b 100644
--- a/block/blk-mq.c
+++ b/block/blk-mq.c
@@ -237,9 +237,12 @@ void blk_mq_unfreeze_queue(struct request_queue *q)
 }
 EXPORT_SYMBOL_GPL(blk_mq_unfreeze_queue);
 
-/*
- * FIXME: replace the scsi_internal_device_*block_nowait() calls in the
- * mpt3sas driver such that this function can be removed.
+/**
+ * blk_mq_quiesce_queue_nowait() - start the quiesce of the queue
+ * @q: request queue.
+ *
+ * Note: synchronize_rcu() or synchronize_srcu(q->srcu) needs to
+ * be called to ensure the quiesce is done.
  */
 void blk_mq_quiesce_queue_nowait(struct request_queue *q)
 {
-- 
2.31.1

Re: [PATCH 1/4] blk-mq: update comment for blk_mq_quiesce_queue_nowait()

[Christoph Hellwig]

On Tue, Nov 01, 2022 at 11:11:34PM +0800, Jinlong Chen wrote:
> blk_mq_quiesce_queue_nowait() is now reasonably used by scsi_host_block()
> to avoid calling synchronize_rcu() for each LUN. We should not consider to
> remove it.

I strongly disagree about this being a reasonable use.  What SCSI did
there is a horrible hack that should have never been merged.  The
right thing to do is the per-tag_set quiesce that we've been working
on.

Re: Re: [PATCH 1/4] blk-mq: update comment for blk_mq_quiesce_queue_nowait()

[Jinlong Chen]

> On Tue, Nov 01, 2022 at 11:11:34PM +0800, Jinlong Chen wrote:
> > blk_mq_quiesce_queue_nowait() is now reasonably used by scsi_host_block()
> > to avoid calling synchronize_rcu() for each LUN. We should not consider to
> > remove it.
> 
> I strongly disagree about this being a reasonable use.  What SCSI did
> there is a horrible hack that should have never been merged.  The
> right thing to do is the per-tag_set quiesce that we've been working
> on.

Ok, I guess I need to read the per-tag_set quiesce patch series.

Thanks!
Jinlong Chen

[PATCH 2/4] blk-mq: improve error handling in blk_mq_alloc_rq_map()

[Jinlong Chen]

Use goto-style error handling like we do elsewhere in the kernel.

Signed-off-by: Jinlong Chen <nickyc975@zju.edu.cn>
---
 block/blk-mq.c | 22 ++++++++++++----------
 1 file changed, 12 insertions(+), 10 deletions(-)

diff --git a/block/blk-mq.c b/block/blk-mq.c
index 7ceceea91b3b..408a929be90e 100644
--- a/block/blk-mq.c
+++ b/block/blk-mq.c
@@ -3272,26 +3272,28 @@ static struct blk_mq_tags *blk_mq_alloc_rq_map(struct blk_mq_tag_set *set,
 	tags = blk_mq_init_tags(nr_tags, reserved_tags, node,
 				BLK_MQ_FLAG_TO_ALLOC_POLICY(set->flags));
 	if (!tags)
-		return NULL;
+		goto err_out;
 
 	tags->rqs = kcalloc_node(nr_tags, sizeof(struct request *),
 				 GFP_NOIO | __GFP_NOWARN | __GFP_NORETRY,
 				 node);
-	if (!tags->rqs) {
-		blk_mq_free_tags(tags);
-		return NULL;
-	}
+	if (!tags->rqs)
+		goto err_free_tags;
 
 	tags->static_rqs = kcalloc_node(nr_tags, sizeof(struct request *),
 					GFP_NOIO | __GFP_NOWARN | __GFP_NORETRY,
 					node);
-	if (!tags->static_rqs) {
-		kfree(tags->rqs);
-		blk_mq_free_tags(tags);
-		return NULL;
-	}
+	if (!tags->static_rqs)
+		goto err_free_rqs;
 
 	return tags;
+
+err_free_rqs:
+	kfree(tags->rqs);
+err_free_tags:
+	blk_mq_free_tags(tags);
+err_out:
+	return NULL;
 }
 
 static int blk_mq_init_request(struct blk_mq_tag_set *set, struct request *rq,
-- 
2.31.1

Re: [PATCH 2/4] blk-mq: improve error handling in blk_mq_alloc_rq_map()

[Christoph Hellwig]

>  	if (!tags)
> -		return NULL;
> +		goto err_out;

Using a label just for the direct error return here is a bit silly.

The rest looks fine.

Re: [PATCH 2/4] blk-mq: improve error handling in blk_mq_alloc_rq_map()

[Jens Axboe]

On 11/1/22 11:34 AM, Christoph Hellwig wrote:
>>  	if (!tags)
>> -		return NULL;
>> +		goto err_out;
> 
> Using a label just for the direct error return here is a bit silly.

Plus that's not idiomatic code for cleanup, just keep the NULL
return if there's no cleanup to be done.

-- 
Jens Axboe

[PATCH 3/4] blk-mq: use if-else instead of goto in blk_mq_alloc_cached_request()

[Jinlong Chen]

if-else is more readable than goto here.

Signed-off-by: Jinlong Chen <nickyc975@zju.edu.cn>
---
 block/blk-mq.c | 27 ++++++++++++++-------------
 1 file changed, 14 insertions(+), 13 deletions(-)

diff --git a/block/blk-mq.c b/block/blk-mq.c
index 408a929be90e..87a6348a0d0a 100644
--- a/block/blk-mq.c
+++ b/block/blk-mq.c
@@ -547,25 +547,26 @@ static struct request *blk_mq_alloc_cached_request(struct request_queue *q,
 
 	if (!plug)
 		return NULL;
+
 	if (rq_list_empty(plug->cached_rq)) {
 		if (plug->nr_ios == 1)
 			return NULL;
 		rq = blk_mq_rq_cache_fill(q, plug, opf, flags);
-		if (rq)
-			goto got_it;
-		return NULL;
-	}
-	rq = rq_list_peek(&plug->cached_rq);
-	if (!rq || rq->q != q)
-		return NULL;
+		if (!rq)
+			return NULL;
+	} else {
+		rq = rq_list_peek(&plug->cached_rq);
+		if (!rq || rq->q != q)
+			return NULL;
 
-	if (blk_mq_get_hctx_type(opf) != rq->mq_hctx->type)
-		return NULL;
-	if (op_is_flush(rq->cmd_flags) != op_is_flush(opf))
-		return NULL;
+		if (blk_mq_get_hctx_type(opf) != rq->mq_hctx->type)
+			return NULL;
+		if (op_is_flush(rq->cmd_flags) != op_is_flush(opf))
+			return NULL;
+
+		plug->cached_rq = rq_list_next(rq);
+	}
 
-	plug->cached_rq = rq_list_next(rq);
-got_it:
 	rq->cmd_flags = opf;
 	INIT_LIST_HEAD(&rq->queuelist);
 	return rq;
-- 
2.31.1

Re: [PATCH 3/4] blk-mq: use if-else instead of goto in blk_mq_alloc_cached_request()

[Christoph Hellwig]

On Tue, Nov 01, 2022 at 11:11:36PM +0800, Jinlong Chen wrote:
> if-else is more readable than goto here.

Looks good:

Reviewed-by: Christoph Hellwig <hch@lst.de>

[PATCH 4/4] blk-mq: improve readability of blk_mq_alloc_request()

[Jinlong Chen]

Add a helper blk_mq_alloc_request_nocache() to alloc request without
cache. This makes blk_mq_alloc_request() more readable.

Signed-off-by: Jinlong Chen <nickyc975@zju.edu.cn>
---
 block/blk-mq.c | 47 +++++++++++++++++++++++++++++------------------
 1 file changed, 29 insertions(+), 18 deletions(-)

diff --git a/block/blk-mq.c b/block/blk-mq.c
index 87a6348a0d0a..2fae111a42c8 100644
--- a/block/blk-mq.c
+++ b/block/blk-mq.c
@@ -572,36 +572,47 @@ static struct request *blk_mq_alloc_cached_request(struct request_queue *q,
 	return rq;
 }
 
-struct request *blk_mq_alloc_request(struct request_queue *q, blk_opf_t opf,
-		blk_mq_req_flags_t flags)
+static struct request *blk_mq_alloc_request_nocache(struct request_queue *q,
+		blk_opf_t opf, blk_mq_req_flags_t flags)
 {
-	struct request *rq;
-
-	rq = blk_mq_alloc_cached_request(q, opf, flags);
-	if (!rq) {
-		struct blk_mq_alloc_data data = {
+	struct blk_mq_alloc_data data = {
 			.q		= q,
 			.flags		= flags,
 			.cmd_flags	= opf,
 			.nr_tags	= 1,
 		};
-		int ret;
+	struct request *rq;
+	int ret;
 
-		ret = blk_queue_enter(q, flags);
-		if (ret)
-			return ERR_PTR(ret);
+	ret = blk_queue_enter(q, flags);
+	if (ret)
+		return ERR_PTR(ret);
 
-		rq = __blk_mq_alloc_requests(&data);
-		if (!rq)
-			goto out_queue_exit;
+	rq = __blk_mq_alloc_requests(&data);
+	if (!rq) {
+		rq = ERR_PTR(-EWOULDBLOCK);
+		blk_queue_exit(q);
 	}
+
+	return rq;
+}
+
+struct request *blk_mq_alloc_request(struct request_queue *q, blk_opf_t opf,
+		blk_mq_req_flags_t flags)
+{
+	struct request *rq;
+
+	rq = blk_mq_alloc_cached_request(q, opf, flags);
+	if (!rq) {
+		rq = blk_mq_alloc_request_nocache(q, opf, flags);
+		if (IS_ERR(rq))
+			return rq;
+	}
+
 	rq->__data_len = 0;
 	rq->__sector = (sector_t) -1;
 	rq->bio = rq->biotail = NULL;
-	return rq;
-out_queue_exit:
-	blk_queue_exit(q);
-	return ERR_PTR(-EWOULDBLOCK);
+	return rq;
 }
 EXPORT_SYMBOL(blk_mq_alloc_request);
 
-- 
2.31.1

Re: [PATCH 4/4] blk-mq: improve readability of blk_mq_alloc_request()

[Christoph Hellwig]

On Tue, Nov 01, 2022 at 11:11:37PM +0800, Jinlong Chen wrote:
> Add a helper blk_mq_alloc_request_nocache() to alloc request without
> cache. This makes blk_mq_alloc_request() more readable.
> 
> Signed-off-by: Jinlong Chen <nickyc975@zju.edu.cn>
> ---
>  block/blk-mq.c | 47 +++++++++++++++++++++++++++++------------------
>  1 file changed, 29 insertions(+), 18 deletions(-)
> 
> diff --git a/block/blk-mq.c b/block/blk-mq.c
> index 87a6348a0d0a..2fae111a42c8 100644
> --- a/block/blk-mq.c
> +++ b/block/blk-mq.c
> @@ -572,36 +572,47 @@ static struct request *blk_mq_alloc_cached_request(struct request_queue *q,
>  	return rq;
>  }
>  
> +static struct request *blk_mq_alloc_request_nocache(struct request_queue *q,
> +		blk_opf_t opf, blk_mq_req_flags_t flags)

The name is a bit odd, but I can't think off a better one.

> +	struct blk_mq_alloc_data data = {
>  			.q		= q,
>  			.flags		= flags,
>  			.cmd_flags	= opf,
>  			.nr_tags	= 1,
>  		};

And this now has superflous indenation.  Overall, while the separate
helper looks marginally nicer, I'm not really sure it is worth the
churn.

Re: [PATCH 4/4] blk-mq: improve readability of blk_mq_alloc_request()

[Chaitanya Kulkarni]

>> +static struct request *blk_mq_alloc_request_nocache(struct request_queue *q,
>> +		blk_opf_t opf, blk_mq_req_flags_t flags)
> 
> The name is a bit odd, but I can't think off a better one.
> 
>> +	struct blk_mq_alloc_data data = {
>>   			.q		= q,
>>   			.flags		= flags,
>>   			.cmd_flags	= opf,
>>   			.nr_tags	= 1,
>>   		};
> 
> And this now has superflous indenation.  Overall, while the separate
> helper looks marginally nicer, I'm not really sure it is worth the
> churn.

existing code is fine here than adding this indentation which
is not worth a churn...

-ck

Re: Re: [PATCH 4/4] blk-mq: improve readability of blk_mq_alloc_request()

[Jinlong Chen]

> On Tue, Nov 01, 2022 at 11:11:37PM +0800, Jinlong Chen wrote:
> > Add a helper blk_mq_alloc_request_nocache() to alloc request without
> > cache. This makes blk_mq_alloc_request() more readable.
> > 
> > Signed-off-by: Jinlong Chen <nickyc975@zju.edu.cn>
> > ---
> >  block/blk-mq.c | 47 +++++++++++++++++++++++++++++------------------
> >  1 file changed, 29 insertions(+), 18 deletions(-)
> > 
> > diff --git a/block/blk-mq.c b/block/blk-mq.c
> > index 87a6348a0d0a..2fae111a42c8 100644
> > --- a/block/blk-mq.c
> > +++ b/block/blk-mq.c
> > @@ -572,36 +572,47 @@ static struct request *blk_mq_alloc_cached_request(struct request_queue *q,
> >  	return rq;
> >  }
> >  
> > +static struct request *blk_mq_alloc_request_nocache(struct request_queue *q,
> > +		blk_opf_t opf, blk_mq_req_flags_t flags)
> 
> The name is a bit odd, but I can't think off a better one.
> 
> > +	struct blk_mq_alloc_data data = {
> >  			.q		= q,
> >  			.flags		= flags,
> >  			.cmd_flags	= opf,
> >  			.nr_tags	= 1,
> >  		};
> 
> And this now has superflous indenation.  Overall, while the separate
> helper looks marginally nicer, I'm not really sure it is worth the
> churn.

I'll drop the patch if you think it is not worth the churn. But I
started doing this because of the following goto statement:

	rq = blk_mq_alloc_cached_request(q, opf, flags);
	if (!rq) {
		[...]
		ret = blk_queue_enter(q, flags);
		[...]
		rq = __blk_mq_alloc_requests(&data);
		if (!rq)
			goto out_queue_exit;
	}
	[...]
out_queue_exit:
	blk_queue_exit(q);
	return ERR_PTR(-EWOULDBLOCK);

Queue entering has been moved into the fallback path, left queue exiting
outside. Should I just eliminate the goto statement and move the error
handling into the fallback path too? Like:

	rq = blk_mq_alloc_cached_request(q, opf, flags);
	if (!rq) {
		[...]
		ret = blk_queue_enter(q, flags);
		[...]
		rq = __blk_mq_alloc_requests(&data);
		if (!rq) {
			blk_queue_exit(q);
			return ERR_PTR(-EWOULDBLOCK);
		}
	}

Thanks!
Jinlong Chen

Re: [PATCH 4/4] blk-mq: improve readability of blk_mq_alloc_request()

[Jens Axboe]

On 11/1/22 8:19 PM, Jinlong Chen wrote:
>> On Tue, Nov 01, 2022 at 11:11:37PM +0800, Jinlong Chen wrote:
>>> Add a helper blk_mq_alloc_request_nocache() to alloc request without
>>> cache. This makes blk_mq_alloc_request() more readable.
>>>
>>> Signed-off-by: Jinlong Chen <nickyc975@zju.edu.cn>
>>> ---
>>>  block/blk-mq.c | 47 +++++++++++++++++++++++++++++------------------
>>>  1 file changed, 29 insertions(+), 18 deletions(-)
>>>
>>> diff --git a/block/blk-mq.c b/block/blk-mq.c
>>> index 87a6348a0d0a..2fae111a42c8 100644
>>> --- a/block/blk-mq.c
>>> +++ b/block/blk-mq.c
>>> @@ -572,36 +572,47 @@ static struct request *blk_mq_alloc_cached_request(struct request_queue *q,
>>>  	return rq;
>>>  }
>>>  
>>> +static struct request *blk_mq_alloc_request_nocache(struct request_queue *q,
>>> +		blk_opf_t opf, blk_mq_req_flags_t flags)
>>
>> The name is a bit odd, but I can't think off a better one.
>>
>>> +	struct blk_mq_alloc_data data = {
>>>  			.q		= q,
>>>  			.flags		= flags,
>>>  			.cmd_flags	= opf,
>>>  			.nr_tags	= 1,
>>>  		};
>>
>> And this now has superflous indenation.  Overall, while the separate
>> helper looks marginally nicer, I'm not really sure it is worth the
>> churn.
> 
> I'll drop the patch if you think it is not worth the churn. But I
> started doing this because of the following goto statement:

Please just drop it, I don't think it's an improvement.

-- 
Jens Axboe

Re: Re: [PATCH 4/4] blk-mq: improve readability of blk_mq_alloc_request()

[Jinlong Chen]

> -----原始邮件-----
> 发件人: "Jens Axboe" <axboe@kernel.dk>
> 发送时间: 2022-11-02 10:25:27 (星期三)
> 收件人: "Jinlong Chen" <nickyc975@zju.edu.cn>, "Christoph Hellwig" <hch@lst.de>
> 抄送: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org
> 主题: Re: [PATCH 4/4] blk-mq: improve readability of blk_mq_alloc_request()
> 
> On 11/1/22 8:19 PM, Jinlong Chen wrote:
> >> On Tue, Nov 01, 2022 at 11:11:37PM +0800, Jinlong Chen wrote:
> >>> Add a helper blk_mq_alloc_request_nocache() to alloc request without
> >>> cache. This makes blk_mq_alloc_request() more readable.
> >>>
> >>> Signed-off-by: Jinlong Chen <nickyc975@zju.edu.cn>
> >>> ---
> >>>  block/blk-mq.c | 47 +++++++++++++++++++++++++++++------------------
> >>>  1 file changed, 29 insertions(+), 18 deletions(-)
> >>>
> >>> diff --git a/block/blk-mq.c b/block/blk-mq.c
> >>> index 87a6348a0d0a..2fae111a42c8 100644
> >>> --- a/block/blk-mq.c
> >>> +++ b/block/blk-mq.c
> >>> @@ -572,36 +572,47 @@ static struct request *blk_mq_alloc_cached_request(struct request_queue *q,
> >>>  	return rq;
> >>>  }
> >>>  
> >>> +static struct request *blk_mq_alloc_request_nocache(struct request_queue *q,
> >>> +		blk_opf_t opf, blk_mq_req_flags_t flags)
> >>
> >> The name is a bit odd, but I can't think off a better one.
> >>
> >>> +	struct blk_mq_alloc_data data = {
> >>>  			.q		= q,
> >>>  			.flags		= flags,
> >>>  			.cmd_flags	= opf,
> >>>  			.nr_tags	= 1,
> >>>  		};
> >>
> >> And this now has superflous indenation.  Overall, while the separate
> >> helper looks marginally nicer, I'm not really sure it is worth the
> >> churn.
> > 
> > I'll drop the patch if you think it is not worth the churn. But I
> > started doing this because of the following goto statement:
> 
> Please just drop it, I don't think it's an improvement.

Ok, then I'll just resend patch 2 without the silly goto return NULL and
patch 3.

Thanks!
Jinlong Chen