[PATCH] block: fix blk_mq_attempt_bio_merge and rq_qos_throttle protection

[PATCH] block: fix blk_mq_attempt_bio_merge and rq_qos_throttle protection

[Shin'ichiro Kawasaki]

Commit 9d497e2941c3 ("block: don't protect submit_bio_checks by
q_usage_counter") moved blk_mq_attempt_bio_merge and rq_qos_throttle
calls out of q_usage_counter protection. However, these functions require
q_usage_counter protection. The blk_mq_attempt_bio_merge call without
the protection resulted in blktests block/005 failure with KASAN null-
ptr-deref or use-after-free at bio merge. The rq_qos_throttle call
without the protection caused kernel hang at qos throttle.

To fix the failures, move the blk_mq_attempt_bio_merge and
rq_qos_throttle calls back to q_usage_counter protection.

Fixes: 9d497e2941c3 ("block: don't protect submit_bio_checks by q_usage_counter")
Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
---
 block/blk-mq.c | 35 +++++++++++++++++++++++------------
 1 file changed, 23 insertions(+), 12 deletions(-)

diff --git a/block/blk-mq.c b/block/blk-mq.c
index d69ca91fbc8b..9a9185a0a2d1 100644
--- a/block/blk-mq.c
+++ b/block/blk-mq.c
@@ -2718,7 +2718,8 @@ static bool blk_mq_attempt_bio_merge(struct request_queue *q,
 
 static struct request *blk_mq_get_new_requests(struct request_queue *q,
 					       struct blk_plug *plug,
-					       struct bio *bio)
+					       struct bio *bio,
+					       unsigned int nsegs)
 {
 	struct blk_mq_alloc_data data = {
 		.q		= q,
@@ -2730,6 +2731,11 @@ static struct request *blk_mq_get_new_requests(struct request_queue *q,
 	if (unlikely(bio_queue_enter(bio)))
 		return NULL;
 
+	if (blk_mq_attempt_bio_merge(q, bio, nsegs))
+		goto queue_exit;
+
+	rq_qos_throttle(q, bio);
+
 	if (plug) {
 		data.nr_tags = plug->nr_ios;
 		plug->nr_ios = 1;
@@ -2742,12 +2748,13 @@ static struct request *blk_mq_get_new_requests(struct request_queue *q,
 	rq_qos_cleanup(q, bio);
 	if (bio->bi_opf & REQ_NOWAIT)
 		bio_wouldblock_error(bio);
+queue_exit:
 	blk_queue_exit(q);
 	return NULL;
 }
 
 static inline struct request *blk_mq_get_cached_request(struct request_queue *q,
-		struct blk_plug *plug, struct bio *bio)
+		struct blk_plug *plug, struct bio **bio, unsigned int nsegs)
 {
 	struct request *rq;
 
@@ -2757,12 +2764,19 @@ static inline struct request *blk_mq_get_cached_request(struct request_queue *q,
 	if (!rq || rq->q != q)
 		return NULL;
 
-	if (blk_mq_get_hctx_type(bio->bi_opf) != rq->mq_hctx->type)
+	if (blk_mq_attempt_bio_merge(q, *bio, nsegs)) {
+		*bio = NULL;
+		return NULL;
+	}
+
+	rq_qos_throttle(q, *bio);
+
+	if (blk_mq_get_hctx_type((*bio)->bi_opf) != rq->mq_hctx->type)
 		return NULL;
-	if (op_is_flush(rq->cmd_flags) != op_is_flush(bio->bi_opf))
+	if (op_is_flush(rq->cmd_flags) != op_is_flush((*bio)->bi_opf))
 		return NULL;
 
-	rq->cmd_flags = bio->bi_opf;
+	rq->cmd_flags = (*bio)->bi_opf;
 	plug->cached_rq = rq_list_next(rq);
 	INIT_LIST_HEAD(&rq->queuelist);
 	return rq;
@@ -2800,14 +2814,11 @@ void blk_mq_submit_bio(struct bio *bio)
 	if (!bio_integrity_prep(bio))
 		return;
 
-	if (blk_mq_attempt_bio_merge(q, bio, nr_segs))
-		return;
-
-	rq_qos_throttle(q, bio);
-
-	rq = blk_mq_get_cached_request(q, plug, bio);
+	rq = blk_mq_get_cached_request(q, plug, &bio, nr_segs);
 	if (!rq) {
-		rq = blk_mq_get_new_requests(q, plug, bio);
+		if (!bio)
+			return;
+		rq = blk_mq_get_new_requests(q, plug, bio, nr_segs);
 		if (unlikely(!rq))
 			return;
 	}
-- 
2.34.1

Re: [PATCH] block: fix blk_mq_attempt_bio_merge and rq_qos_throttle protection

[Ming Lei]

On Tue, Mar 08, 2022 at 05:09:15PM +0900, Shin'ichiro Kawasaki wrote:
> Commit 9d497e2941c3 ("block: don't protect submit_bio_checks by
> q_usage_counter") moved blk_mq_attempt_bio_merge and rq_qos_throttle
> calls out of q_usage_counter protection. However, these functions require
> q_usage_counter protection. The blk_mq_attempt_bio_merge call without
> the protection resulted in blktests block/005 failure with KASAN null-
> ptr-deref or use-after-free at bio merge. The rq_qos_throttle call
> without the protection caused kernel hang at qos throttle.
> 
> To fix the failures, move the blk_mq_attempt_bio_merge and
> rq_qos_throttle calls back to q_usage_counter protection.
> 
> Fixes: 9d497e2941c3 ("block: don't protect submit_bio_checks by q_usage_counter")
> Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>

Looks fine,

Reviewed-by: Ming Lei <ming.lei@redhat.com>

Thanks,
Ming

Re: [PATCH] block: fix blk_mq_attempt_bio_merge and rq_qos_throttle protection

[Jens Axboe]

On Tue, 8 Mar 2022 17:09:15 +0900, Shin'ichiro Kawasaki wrote:
> Commit 9d497e2941c3 ("block: don't protect submit_bio_checks by
> q_usage_counter") moved blk_mq_attempt_bio_merge and rq_qos_throttle
> calls out of q_usage_counter protection. However, these functions require
> q_usage_counter protection. The blk_mq_attempt_bio_merge call without
> the protection resulted in blktests block/005 failure with KASAN null-
> ptr-deref or use-after-free at bio merge. The rq_qos_throttle call
> without the protection caused kernel hang at qos throttle.
> 
> [...]

Applied, thanks!

[1/1] block: fix blk_mq_attempt_bio_merge and rq_qos_throttle protection
      commit: 0a5aa8d161d19a1b12fd25b434b32f7c885c73bb

Best regards,
-- 
Jens Axboe