Linux block layer
 help / color / mirror / Atom feed
From: Guixin Liu <kanie@linux.alibaba.com>
To: Tao Cui <cui.tao@linux.dev>, Jens Axboe <axboe@kernel.dk>,
	Christoph Hellwig <hch@lst.de>, Tejun Heo <tj@kernel.org>,
	Josef Bacik <josef@toxicpanda.com>,
	Nilay Shroff <nilay@linux.ibm.com>,
	Ming Lei <ming.lei@redhat.com>, Hannes Reinecke <hare@suse.de>
Cc: linux-block@vger.kernel.org
Subject: Re: [PATCH 3/7] blk-iocost: don't take rq_qos_mutex before the queue is frozen in cost model write
Date: Fri, 20 Nov 2026 10:14:01 +0800	[thread overview]
Message-ID: <70f7faf8-e362-4a1c-ae5d-33c168337755@linux.alibaba.com> (raw)
In-Reply-To: <473ff2e1-329f-49c0-a124-b9e2194d7bcf@linux.dev>



在 2026/11/19 21:40, Tao Cui 写道:
> Hello, Guixin,
>
> 在 2026/11/19 15:05, Guixin Liu 写道:
>> ioc_cost_model_write() freezes the queue while holding rq_qos_mutex,
>> which blkg_conf_open_bdev() acquired first. Every other path takes
>> rq_qos_mutex only after the queue is frozen: ioc_qos_write() releases
>> and re-acquires the mutex around blk_mq_freeze_queue(), and rq_qos_add()
>> documents that its caller holds the mutex with the queue frozen.
>> Waiting for in-flight IO while holding rq_qos_mutex breaks that rule,
>> and on the first write to a queue without iocost the path even calls
>> blk_iocost_init() -> rq_qos_add() -> blk_mq_freeze_queue() while
>> nothing has frozen the queue yet.
>>
>> Mirror the release-freeze-reacquire sequence of ioc_qos_write(). The
>> unfreeze moves to the close_bdev label so the error exits after the
>> freeze don't leave the queue frozen, and it runs before
>> blkg_conf_close_bdev() drops the bdev reference, while the queue is
>> still pinned.
>>
> I tried the same change in cost model write recently and it turned out flawed, these two reviews explain it better than I can [1][2].
>
> In short: the disk_live() check in blkg_conf_open_bdev() only holds while rq_qos_mutex is kept, and dropping it around the freeze lets the disk and the ioc go away; Tejun reproduced a use-after-free that way.
>
> I've been busy with another thread lately, so I'm glad to see you proposing a fix for this. Hope the links help.
>
> Thanks,
> Tao
>
> [1] https://lore.kernel.org/all/20261111163306.GA17267@lst.de/
> [2] https://lore.kernel.org/all/bfe5007b2758bb06504b6966abaf1f7e@kernel.org/
>  
Hi Tao,
  Thanks for the review, right, this patch enter the same issue, I will drop this
in next version.

  And about the ioc_qos_write()'s disk_live race window, My patch 6 fixed the issue,
could you please take a look?

Best Regards,
Guixin Liu
>> Fixes: 2b2da2f6dc80 ("blk-iocost: prevent configuration update concurrent with io throttling")
>> Cc: stable@vger.kernel.org
>> Signed-off-by: Guixin Liu <kanie@linux.alibaba.com>
>> ---
>>  block/blk-iocost.c | 12 ++++++++++--
>>  1 file changed, 10 insertions(+), 2 deletions(-)
>>
>> diff --git a/block/blk-iocost.c b/block/blk-iocost.c
>> index 1c1025157d8e..8e06b83691aa 100644
>> --- a/block/blk-iocost.c
>> +++ b/block/blk-iocost.c
>> @@ -3589,6 +3589,15 @@ static ssize_t ioc_cost_model_write(struct kernfs_open_file *of, char *input,
>>  	ret = blkg_conf_open_bdev(&ctx);
>>  	if (ret)
>>  		return ret;
>> +	/*
>> +	 * Nothing is protecting QoS state yet, so release q->rq_qos_mutex,
>> +	 * which was first acquired in blkg_conf_open_bdev, and re-acquire it
>> +	 * after freezing the queue to maintain the correct locking order.
>> +	 */
>> +	mutex_unlock(&ctx.bdev->bd_queue->rq_qos_mutex);
>> +
>> +	memflags = blk_mq_freeze_queue(ctx.bdev->bd_queue);
>> +	mutex_lock(&ctx.bdev->bd_queue->rq_qos_mutex);
>>  
>>  	body = ctx.body;
>>  	q = bdev_get_queue(ctx.bdev);
>> @@ -3605,7 +3614,6 @@ static ssize_t ioc_cost_model_write(struct kernfs_open_file *of, char *input,
>>  		ioc = q_to_ioc(q);
>>  	}
>>  
>> -	memflags = blk_mq_freeze_queue(q);
>>  	blk_mq_quiesce_queue(q);
>>  
>>  	spin_lock_irq(&ioc->lock);
>> @@ -3663,9 +3671,9 @@ static ssize_t ioc_cost_model_write(struct kernfs_open_file *of, char *input,
>>  	spin_unlock_irq(&ioc->lock);
>>  
>>  	blk_mq_unquiesce_queue(q);
>> -	blk_mq_unfreeze_queue(q, memflags);
>>  
>>  close_bdev:
>> +	blk_mq_unfreeze_queue(q, memflags);
>>  	blkg_conf_close_bdev(&ctx);
>>  	return ret ?: nbytes;
>>  }


  reply	other threads:[~2026-11-20  2:14 UTC|newest]

Thread overview: 12+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-19  7:05 [PATCH 0/7] block fixes found by code inspection Guixin Liu
2026-11-19  7:05 ` [PATCH 1/7] blk-mq: free the rqs arrays of tags in the SRCU callback Guixin Liu
2026-11-19  7:05 ` [PATCH 2/7] blk-mq: clear stale stopped flag on hctx reuse Guixin Liu
2026-11-19  7:05 ` [PATCH 3/7] blk-iocost: don't take rq_qos_mutex before the queue is frozen in cost model write Guixin Liu
2026-11-19 13:40   ` Tao Cui
2026-11-20  2:14     ` Guixin Liu [this message]
2026-11-19  7:05 ` [PATCH 4/7] blk-iocost: unfreeze the queue before dropping the bdev reference in qos write Guixin Liu
2026-11-19  7:05 ` [PATCH 5/7] blk-stat: account for the mean of the source in blk_rq_stat_sum() Guixin Liu
2026-11-19 13:32   ` Tao Cui
2026-11-20  2:04     ` Guixin Liu
2026-11-19  7:05 ` [PATCH 6/7] blk-cgroup: don't attach a qos policy to a dead disk Guixin Liu
2026-11-19  7:05 ` [PATCH 7/7] blk-mq: unpublish the tags before deferring their free to SRCU Guixin Liu

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=70f7faf8-e362-4a1c-ae5d-33c168337755@linux.alibaba.com \
    --to=kanie@linux.alibaba.com \
    --cc=axboe@kernel.dk \
    --cc=cui.tao@linux.dev \
    --cc=hare@suse.de \
    --cc=hch@lst.de \
    --cc=josef@toxicpanda.com \
    --cc=linux-block@vger.kernel.org \
    --cc=ming.lei@redhat.com \
    --cc=nilay@linux.ibm.com \
    --cc=tj@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox