From: Guixin Liu <kanie@linux.alibaba.com>
To: Tao Cui <cui.tao@linux.dev>, Jens Axboe <axboe@kernel.dk>,
Christoph Hellwig <hch@lst.de>, Tejun Heo <tj@kernel.org>,
Josef Bacik <josef@toxicpanda.com>,
Nilay Shroff <nilay@linux.ibm.com>,
Ming Lei <ming.lei@redhat.com>, Hannes Reinecke <hare@suse.de>
Cc: linux-block@vger.kernel.org
Subject: Re: [PATCH 3/7] blk-iocost: don't take rq_qos_mutex before the queue is frozen in cost model write
Date: Fri, 20 Nov 2026 10:14:01 +0800 [thread overview]
Message-ID: <70f7faf8-e362-4a1c-ae5d-33c168337755@linux.alibaba.com> (raw)
In-Reply-To: <473ff2e1-329f-49c0-a124-b9e2194d7bcf@linux.dev>
在 2026/11/19 21:40, Tao Cui 写道:
> Hello, Guixin,
>
> 在 2026/11/19 15:05, Guixin Liu 写道:
>> ioc_cost_model_write() freezes the queue while holding rq_qos_mutex,
>> which blkg_conf_open_bdev() acquired first. Every other path takes
>> rq_qos_mutex only after the queue is frozen: ioc_qos_write() releases
>> and re-acquires the mutex around blk_mq_freeze_queue(), and rq_qos_add()
>> documents that its caller holds the mutex with the queue frozen.
>> Waiting for in-flight IO while holding rq_qos_mutex breaks that rule,
>> and on the first write to a queue without iocost the path even calls
>> blk_iocost_init() -> rq_qos_add() -> blk_mq_freeze_queue() while
>> nothing has frozen the queue yet.
>>
>> Mirror the release-freeze-reacquire sequence of ioc_qos_write(). The
>> unfreeze moves to the close_bdev label so the error exits after the
>> freeze don't leave the queue frozen, and it runs before
>> blkg_conf_close_bdev() drops the bdev reference, while the queue is
>> still pinned.
>>
> I tried the same change in cost model write recently and it turned out flawed, these two reviews explain it better than I can [1][2].
>
> In short: the disk_live() check in blkg_conf_open_bdev() only holds while rq_qos_mutex is kept, and dropping it around the freeze lets the disk and the ioc go away; Tejun reproduced a use-after-free that way.
>
> I've been busy with another thread lately, so I'm glad to see you proposing a fix for this. Hope the links help.
>
> Thanks,
> Tao
>
> [1] https://lore.kernel.org/all/20261111163306.GA17267@lst.de/
> [2] https://lore.kernel.org/all/bfe5007b2758bb06504b6966abaf1f7e@kernel.org/
>
Hi Tao,
Thanks for the review, right, this patch enter the same issue, I will drop this
in next version.
And about the ioc_qos_write()'s disk_live race window, My patch 6 fixed the issue,
could you please take a look?
Best Regards,
Guixin Liu
>> Fixes: 2b2da2f6dc80 ("blk-iocost: prevent configuration update concurrent with io throttling")
>> Cc: stable@vger.kernel.org
>> Signed-off-by: Guixin Liu <kanie@linux.alibaba.com>
>> ---
>> block/blk-iocost.c | 12 ++++++++++--
>> 1 file changed, 10 insertions(+), 2 deletions(-)
>>
>> diff --git a/block/blk-iocost.c b/block/blk-iocost.c
>> index 1c1025157d8e..8e06b83691aa 100644
>> --- a/block/blk-iocost.c
>> +++ b/block/blk-iocost.c
>> @@ -3589,6 +3589,15 @@ static ssize_t ioc_cost_model_write(struct kernfs_open_file *of, char *input,
>> ret = blkg_conf_open_bdev(&ctx);
>> if (ret)
>> return ret;
>> + /*
>> + * Nothing is protecting QoS state yet, so release q->rq_qos_mutex,
>> + * which was first acquired in blkg_conf_open_bdev, and re-acquire it
>> + * after freezing the queue to maintain the correct locking order.
>> + */
>> + mutex_unlock(&ctx.bdev->bd_queue->rq_qos_mutex);
>> +
>> + memflags = blk_mq_freeze_queue(ctx.bdev->bd_queue);
>> + mutex_lock(&ctx.bdev->bd_queue->rq_qos_mutex);
>>
>> body = ctx.body;
>> q = bdev_get_queue(ctx.bdev);
>> @@ -3605,7 +3614,6 @@ static ssize_t ioc_cost_model_write(struct kernfs_open_file *of, char *input,
>> ioc = q_to_ioc(q);
>> }
>>
>> - memflags = blk_mq_freeze_queue(q);
>> blk_mq_quiesce_queue(q);
>>
>> spin_lock_irq(&ioc->lock);
>> @@ -3663,9 +3671,9 @@ static ssize_t ioc_cost_model_write(struct kernfs_open_file *of, char *input,
>> spin_unlock_irq(&ioc->lock);
>>
>> blk_mq_unquiesce_queue(q);
>> - blk_mq_unfreeze_queue(q, memflags);
>>
>> close_bdev:
>> + blk_mq_unfreeze_queue(q, memflags);
>> blkg_conf_close_bdev(&ctx);
>> return ret ?: nbytes;
>> }
next prev parent reply other threads:[~2026-11-20 2:14 UTC|newest]
Thread overview: 12+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-19 7:05 [PATCH 0/7] block fixes found by code inspection Guixin Liu
2026-11-19 7:05 ` [PATCH 1/7] blk-mq: free the rqs arrays of tags in the SRCU callback Guixin Liu
2026-11-19 7:05 ` [PATCH 2/7] blk-mq: clear stale stopped flag on hctx reuse Guixin Liu
2026-11-19 7:05 ` [PATCH 3/7] blk-iocost: don't take rq_qos_mutex before the queue is frozen in cost model write Guixin Liu
2026-11-19 13:40 ` Tao Cui
2026-11-20 2:14 ` Guixin Liu [this message]
2026-11-19 7:05 ` [PATCH 4/7] blk-iocost: unfreeze the queue before dropping the bdev reference in qos write Guixin Liu
2026-11-19 7:05 ` [PATCH 5/7] blk-stat: account for the mean of the source in blk_rq_stat_sum() Guixin Liu
2026-11-19 13:32 ` Tao Cui
2026-11-20 2:04 ` Guixin Liu
2026-11-19 7:05 ` [PATCH 6/7] blk-cgroup: don't attach a qos policy to a dead disk Guixin Liu
2026-11-19 7:05 ` [PATCH 7/7] blk-mq: unpublish the tags before deferring their free to SRCU Guixin Liu
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=70f7faf8-e362-4a1c-ae5d-33c168337755@linux.alibaba.com \
--to=kanie@linux.alibaba.com \
--cc=axboe@kernel.dk \
--cc=cui.tao@linux.dev \
--cc=hare@suse.de \
--cc=hch@lst.de \
--cc=josef@toxicpanda.com \
--cc=linux-block@vger.kernel.org \
--cc=ming.lei@redhat.com \
--cc=nilay@linux.ibm.com \
--cc=tj@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox