From: Tao Cui <cui.tao@linux.dev>
To: Guixin Liu <kanie@linux.alibaba.com>,
Jens Axboe <axboe@kernel.dk>, Christoph Hellwig <hch@lst.de>,
Tejun Heo <tj@kernel.org>, Josef Bacik <josef@toxicpanda.com>,
Nilay Shroff <nilay@linux.ibm.com>,
Ming Lei <ming.lei@redhat.com>, Hannes Reinecke <hare@suse.de>
Cc: cui.tao@linux.dev, linux-block@vger.kernel.org
Subject: Re: [PATCH 3/7] blk-iocost: don't take rq_qos_mutex before the queue is frozen in cost model write
Date: Fri, 20 Nov 2026 10:56:17 +0800 [thread overview]
Message-ID: <7106a456-8e3d-4fe2-ae7d-b8e753e2b131@linux.dev> (raw)
In-Reply-To: <70f7faf8-e362-4a1c-ae5d-33c168337755@linux.alibaba.com>
Hello, Guixin,
在 2026/11/20 10:14, Guixin Liu 写道:
>
>
> 在 2026/11/19 21:40, Tao Cui 写道:
>> Hello, Guixin,
>>
>> 在 2026/11/19 15:05, Guixin Liu 写道:
>>> ioc_cost_model_write() freezes the queue while holding rq_qos_mutex,
>>> which blkg_conf_open_bdev() acquired first. Every other path takes
>>> rq_qos_mutex only after the queue is frozen: ioc_qos_write() releases
>>> and re-acquires the mutex around blk_mq_freeze_queue(), and rq_qos_add()
>>> documents that its caller holds the mutex with the queue frozen.
>>> Waiting for in-flight IO while holding rq_qos_mutex breaks that rule,
>>> and on the first write to a queue without iocost the path even calls
>>> blk_iocost_init() -> rq_qos_add() -> blk_mq_freeze_queue() while
>>> nothing has frozen the queue yet.
>>>
>>> Mirror the release-freeze-reacquire sequence of ioc_qos_write(). The
>>> unfreeze moves to the close_bdev label so the error exits after the
>>> freeze don't leave the queue frozen, and it runs before
>>> blkg_conf_close_bdev() drops the bdev reference, while the queue is
>>> still pinned.
>>>
>> I tried the same change in cost model write recently and it turned out flawed, these two reviews explain it better than I can [1][2].
>>
>> In short: the disk_live() check in blkg_conf_open_bdev() only holds while rq_qos_mutex is kept, and dropping it around the freeze lets the disk and the ioc go away; Tejun reproduced a use-after-free that way.
>>
>> I've been busy with another thread lately, so I'm glad to see you proposing a fix for this. Hope the links help.
>>
>> Thanks,
>> Tao
>>
>> [1] https://lore.kernel.org/all/20261111163306.GA17267@lst.de/
>> [2] https://lore.kernel.org/all/bfe5007b2758bb06504b6966abaf1f7e@kernel.org/
>>
> Hi Tao,
> Thanks for the review, right, this patch enter the same issue, I will drop this
> in next version.
>
Thanks for checking the history before reworking. I'm still busy with another thread, so if you have time, feel free to take this forward.
Thanks,
Tao
> And about the ioc_qos_write()'s disk_live race window, My patch 6 fixed the issue,
> could you please take a look?
>
> Best Regards,
> Guixin Liu
>>> Fixes: 2b2da2f6dc80 ("blk-iocost: prevent configuration update concurrent with io throttling")
>>> Cc: stable@vger.kernel.org
>>> Signed-off-by: Guixin Liu <kanie@linux.alibaba.com>
>>> ---
>>> block/blk-iocost.c | 12 ++++++++++--
>>> 1 file changed, 10 insertions(+), 2 deletions(-)
>>>
>>> diff --git a/block/blk-iocost.c b/block/blk-iocost.c
>>> index 1c1025157d8e..8e06b83691aa 100644
>>> --- a/block/blk-iocost.c
>>> +++ b/block/blk-iocost.c
>>> @@ -3589,6 +3589,15 @@ static ssize_t ioc_cost_model_write(struct kernfs_open_file *of, char *input,
>>> ret = blkg_conf_open_bdev(&ctx);
>>> if (ret)
>>> return ret;
>>> + /*
>>> + * Nothing is protecting QoS state yet, so release q->rq_qos_mutex,
>>> + * which was first acquired in blkg_conf_open_bdev, and re-acquire it
>>> + * after freezing the queue to maintain the correct locking order.
>>> + */
>>> + mutex_unlock(&ctx.bdev->bd_queue->rq_qos_mutex);
>>> +
>>> + memflags = blk_mq_freeze_queue(ctx.bdev->bd_queue);
>>> + mutex_lock(&ctx.bdev->bd_queue->rq_qos_mutex);
>>>
>>> body = ctx.body;
>>> q = bdev_get_queue(ctx.bdev);
>>> @@ -3605,7 +3614,6 @@ static ssize_t ioc_cost_model_write(struct kernfs_open_file *of, char *input,
>>> ioc = q_to_ioc(q);
>>> }
>>>
>>> - memflags = blk_mq_freeze_queue(q);
>>> blk_mq_quiesce_queue(q);
>>>
>>> spin_lock_irq(&ioc->lock);
>>> @@ -3663,9 +3671,9 @@ static ssize_t ioc_cost_model_write(struct kernfs_open_file *of, char *input,
>>> spin_unlock_irq(&ioc->lock);
>>>
>>> blk_mq_unquiesce_queue(q);
>>> - blk_mq_unfreeze_queue(q, memflags);
>>>
>>> close_bdev:
>>> + blk_mq_unfreeze_queue(q, memflags);
>>> blkg_conf_close_bdev(&ctx);
>>> return ret ?: nbytes;
>>> }
>
next prev parent reply other threads:[~2026-11-20 2:56 UTC|newest]
Thread overview: 14+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-19 7:05 [PATCH 0/7] block fixes found by code inspection Guixin Liu
2026-11-19 7:05 ` [PATCH 1/7] blk-mq: free the rqs arrays of tags in the SRCU callback Guixin Liu
2026-11-19 7:05 ` [PATCH 2/7] blk-mq: clear stale stopped flag on hctx reuse Guixin Liu
2026-11-19 7:05 ` [PATCH 3/7] blk-iocost: don't take rq_qos_mutex before the queue is frozen in cost model write Guixin Liu
2026-11-19 13:40 ` Tao Cui
2026-11-20 2:14 ` Guixin Liu
2026-11-20 2:56 ` Tao Cui [this message]
2026-11-19 7:05 ` [PATCH 4/7] blk-iocost: unfreeze the queue before dropping the bdev reference in qos write Guixin Liu
2026-11-19 7:05 ` [PATCH 5/7] blk-stat: account for the mean of the source in blk_rq_stat_sum() Guixin Liu
2026-11-19 13:32 ` Tao Cui
2026-11-20 2:04 ` Guixin Liu
2026-11-20 2:50 ` Tao Cui
2026-11-19 7:05 ` [PATCH 6/7] blk-cgroup: don't attach a qos policy to a dead disk Guixin Liu
2026-11-19 7:05 ` [PATCH 7/7] blk-mq: unpublish the tags before deferring their free to SRCU Guixin Liu
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=7106a456-8e3d-4fe2-ae7d-b8e753e2b131@linux.dev \
--to=cui.tao@linux.dev \
--cc=axboe@kernel.dk \
--cc=hare@suse.de \
--cc=hch@lst.de \
--cc=josef@toxicpanda.com \
--cc=kanie@linux.alibaba.com \
--cc=linux-block@vger.kernel.org \
--cc=ming.lei@redhat.com \
--cc=nilay@linux.ibm.com \
--cc=tj@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox