[PATCH] block: protect hctx attributes/params using q->elevator_lock

[PATCH] block: protect hctx attributes/params using q->elevator_lock

[Nilay Shroff]

Currently, hctx attributes (nr_tags, nr_reserved_tags, and cpu_list)
are protected using `q->sysfs_lock`. However, these attributes can be
updated in multiple scenarios:
  - During the driver's probe method.
  - When updating nr_hw_queues.
  - When writing to the sysfs attribute nr_requests,
    which can modify nr_tags.
The nr_requests attribute is already protected using q->elevator_lock,
but none of the update paths actually use q->sysfs_lock to protect hctx
attributes. So to ensure proper synchronization, replace q->sysfs_lock
with q->elevator_lock when reading hctx attributes through sysfs.

Additionally, blk_mq_update_nr_hw_queues allocates and updates hctx.
The allocation of hctx is protected using q->elevator_lock, however,
updating hctx params happens without any protection, so safeguard hctx
param update path by also using q->elevator_lock.

Signed-off-by: Nilay Shroff <nilay@linux.ibm.com>
---
Please note that this patch was unit tested against blktests and 
quick xfstest with lockdep enabled.

This patch should go on top of the previous patchset:
https://lore.kernel.org/all/20250304102551.2533767-1-nilay@linux.ibm.com/
---
 block/blk-mq-sysfs.c   |  4 ++--
 block/blk-mq.c         |  4 ++++
 include/linux/blkdev.h | 14 ++++++++------
 3 files changed, 14 insertions(+), 8 deletions(-)

diff --git a/block/blk-mq-sysfs.c b/block/blk-mq-sysfs.c
index 3feeeccf8a99..24656980f443 100644
--- a/block/blk-mq-sysfs.c
+++ b/block/blk-mq-sysfs.c
@@ -61,9 +61,9 @@ static ssize_t blk_mq_hw_sysfs_show(struct kobject *kobj,
 	if (!entry->show)
 		return -EIO;
 
-	mutex_lock(&q->sysfs_lock);
+	mutex_lock(&q->elevator_lock);
 	res = entry->show(hctx, page);
-	mutex_unlock(&q->sysfs_lock);
+	mutex_unlock(&q->elevator_lock);
 	return res;
 }
 
diff --git a/block/blk-mq.c b/block/blk-mq.c
index 5a2d63927525..b9550a127c8e 100644
--- a/block/blk-mq.c
+++ b/block/blk-mq.c
@@ -4094,6 +4094,8 @@ static void blk_mq_map_swqueue(struct request_queue *q)
 	struct blk_mq_ctx *ctx;
 	struct blk_mq_tag_set *set = q->tag_set;
 
+	mutex_lock(&q->elevator_lock);
+
 	queue_for_each_hw_ctx(q, hctx, i) {
 		cpumask_clear(hctx->cpumask);
 		hctx->nr_ctx = 0;
@@ -4198,6 +4200,8 @@ static void blk_mq_map_swqueue(struct request_queue *q)
 		hctx->next_cpu = blk_mq_first_mapped_cpu(hctx);
 		hctx->next_cpu_batch = BLK_MQ_CPU_WORK_BATCH;
 	}
+
+	mutex_unlock(&q->elevator_lock);
 }
 
 /*
diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h
index 4daef0408683..b6f893e1741b 100644
--- a/include/linux/blkdev.h
+++ b/include/linux/blkdev.h
@@ -563,12 +563,14 @@ struct request_queue {
 	struct list_head	flush_list;
 
 	/*
-	 * Protects against I/O scheduler switching, particularly when
-	 * updating q->elevator. Since the elevator update code path may
-	 * also modify q->nr_requests and wbt latency, this lock also
-	 * protects the sysfs attributes nr_requests and wbt_lat_usec.
-	 * To ensure proper locking order during an elevator update, first
-	 * freeze the queue, then acquire ->elevator_lock.
+	 * Protects against I/O scheduler switching, particularly when updating
+	 * q->elevator. Since the elevator update code path may also modify q->
+	 * nr_requests and wbt latency, this lock also protects the sysfs attrs
+	 * nr_requests and wbt_lat_usec. Additionally the nr_hw_queues update may
+	 * modify hctx tags, reserved-tags and cpumask, so this lock also helps
+	 * protect the hctx attrs.
+	 * To ensure proper locking order during an elevator or nr_hw_queue
+	 * update, first freeze the queue, then acquire ->elevator_lock.
 	 */
 	struct mutex		elevator_lock;
 
-- 
2.47.1

Re: [PATCH] block: protect hctx attributes/params using q->elevator_lock

[Christoph Hellwig]

> +	 * nr_requests and wbt_lat_usec. Additionally the nr_hw_queues update may

This adds an overly long line.

Otherwise the change looks good to me.

Re: [PATCH] block: protect hctx attributes/params using q->elevator_lock

[Jens Axboe]

On Thu, 06 Mar 2025 15:09:53 +0530, Nilay Shroff wrote:
> Currently, hctx attributes (nr_tags, nr_reserved_tags, and cpu_list)
> are protected using `q->sysfs_lock`. However, these attributes can be
> updated in multiple scenarios:
>   - During the driver's probe method.
>   - When updating nr_hw_queues.
>   - When writing to the sysfs attribute nr_requests,
>     which can modify nr_tags.
> The nr_requests attribute is already protected using q->elevator_lock,
> but none of the update paths actually use q->sysfs_lock to protect hctx
> attributes. So to ensure proper synchronization, replace q->sysfs_lock
> with q->elevator_lock when reading hctx attributes through sysfs.
> 
> [...]

Applied, thanks!

[1/1] block: protect hctx attributes/params using q->elevator_lock
      commit: 5abba4cebec0a591ca7e7f55701e42cd5dc059af

Best regards,
-- 
Jens Axboe