All of lore.kernel.org
 help / color / mirror / Atom feed
From: Tao Cui <cui.tao@linux.dev>
To: Guixin Liu <kanie@linux.alibaba.com>,
	Jens Axboe <axboe@kernel.dk>, Christoph Hellwig <hch@lst.de>,
	Tejun Heo <tj@kernel.org>, Josef Bacik <josef@toxicpanda.com>,
	Nilay Shroff <nilay@linux.ibm.com>,
	Ming Lei <ming.lei@redhat.com>, Hannes Reinecke <hare@suse.de>
Cc: cui.tao@linux.dev, linux-block@vger.kernel.org
Subject: Re: [PATCH 3/7] blk-iocost: don't take rq_qos_mutex before the queue is frozen in cost model write
Date: Fri, 20 Nov 2026 10:56:17 +0800	[thread overview]
Message-ID: <7106a456-8e3d-4fe2-ae7d-b8e753e2b131@linux.dev> (raw)
In-Reply-To: <70f7faf8-e362-4a1c-ae5d-33c168337755@linux.alibaba.com>

Hello, Guixin,

在 2026/11/20 10:14, Guixin Liu 写道:
> 
> 
> 在 2026/11/19 21:40, Tao Cui 写道:
>> Hello, Guixin,
>>
>> 在 2026/11/19 15:05, Guixin Liu 写道:
>>> ioc_cost_model_write() freezes the queue while holding rq_qos_mutex,
>>> which blkg_conf_open_bdev() acquired first. Every other path takes
>>> rq_qos_mutex only after the queue is frozen: ioc_qos_write() releases
>>> and re-acquires the mutex around blk_mq_freeze_queue(), and rq_qos_add()
>>> documents that its caller holds the mutex with the queue frozen.
>>> Waiting for in-flight IO while holding rq_qos_mutex breaks that rule,
>>> and on the first write to a queue without iocost the path even calls
>>> blk_iocost_init() -> rq_qos_add() -> blk_mq_freeze_queue() while
>>> nothing has frozen the queue yet.
>>>
>>> Mirror the release-freeze-reacquire sequence of ioc_qos_write(). The
>>> unfreeze moves to the close_bdev label so the error exits after the
>>> freeze don't leave the queue frozen, and it runs before
>>> blkg_conf_close_bdev() drops the bdev reference, while the queue is
>>> still pinned.
>>>
>> I tried the same change in cost model write recently and it turned out flawed, these two reviews explain it better than I can [1][2].
>>
>> In short: the disk_live() check in blkg_conf_open_bdev() only holds while rq_qos_mutex is kept, and dropping it around the freeze lets the disk and the ioc go away; Tejun reproduced a use-after-free that way.
>>
>> I've been busy with another thread lately, so I'm glad to see you proposing a fix for this. Hope the links help.
>>
>> Thanks,
>> Tao
>>
>> [1] https://lore.kernel.org/all/20261111163306.GA17267@lst.de/
>> [2] https://lore.kernel.org/all/bfe5007b2758bb06504b6966abaf1f7e@kernel.org/
>>  
> Hi Tao,
>   Thanks for the review, right, this patch enter the same issue, I will drop this
> in next version.
>

Thanks for checking the history before reworking. I'm still busy with another thread, so if you have time, feel free to take this forward.

Thanks,
Tao
 
>   And about the ioc_qos_write()'s disk_live race window, My patch 6 fixed the issue,
> could you please take a look?
> 
> Best Regards,
> Guixin Liu
>>> Fixes: 2b2da2f6dc80 ("blk-iocost: prevent configuration update concurrent with io throttling")
>>> Cc: stable@vger.kernel.org
>>> Signed-off-by: Guixin Liu <kanie@linux.alibaba.com>
>>> ---
>>>  block/blk-iocost.c | 12 ++++++++++--
>>>  1 file changed, 10 insertions(+), 2 deletions(-)
>>>
>>> diff --git a/block/blk-iocost.c b/block/blk-iocost.c
>>> index 1c1025157d8e..8e06b83691aa 100644
>>> --- a/block/blk-iocost.c
>>> +++ b/block/blk-iocost.c
>>> @@ -3589,6 +3589,15 @@ static ssize_t ioc_cost_model_write(struct kernfs_open_file *of, char *input,
>>>  	ret = blkg_conf_open_bdev(&ctx);
>>>  	if (ret)
>>>  		return ret;
>>> +	/*
>>> +	 * Nothing is protecting QoS state yet, so release q->rq_qos_mutex,
>>> +	 * which was first acquired in blkg_conf_open_bdev, and re-acquire it
>>> +	 * after freezing the queue to maintain the correct locking order.
>>> +	 */
>>> +	mutex_unlock(&ctx.bdev->bd_queue->rq_qos_mutex);
>>> +
>>> +	memflags = blk_mq_freeze_queue(ctx.bdev->bd_queue);
>>> +	mutex_lock(&ctx.bdev->bd_queue->rq_qos_mutex);
>>>  
>>>  	body = ctx.body;
>>>  	q = bdev_get_queue(ctx.bdev);
>>> @@ -3605,7 +3614,6 @@ static ssize_t ioc_cost_model_write(struct kernfs_open_file *of, char *input,
>>>  		ioc = q_to_ioc(q);
>>>  	}
>>>  
>>> -	memflags = blk_mq_freeze_queue(q);
>>>  	blk_mq_quiesce_queue(q);
>>>  
>>>  	spin_lock_irq(&ioc->lock);
>>> @@ -3663,9 +3671,9 @@ static ssize_t ioc_cost_model_write(struct kernfs_open_file *of, char *input,
>>>  	spin_unlock_irq(&ioc->lock);
>>>  
>>>  	blk_mq_unquiesce_queue(q);
>>> -	blk_mq_unfreeze_queue(q, memflags);
>>>  
>>>  close_bdev:
>>> +	blk_mq_unfreeze_queue(q, memflags);
>>>  	blkg_conf_close_bdev(&ctx);
>>>  	return ret ?: nbytes;
>>>  }
> 


  reply	other threads:[~2026-11-20  2:56 UTC|newest]

Thread overview: 15+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-19  7:05 [PATCH 0/7] block fixes found by code inspection Guixin Liu
2026-11-19  7:05 ` [PATCH 1/7] blk-mq: free the rqs arrays of tags in the SRCU callback Guixin Liu
2026-11-19  7:05 ` [PATCH 2/7] blk-mq: clear stale stopped flag on hctx reuse Guixin Liu
2026-11-19  7:05 ` [PATCH 3/7] blk-iocost: don't take rq_qos_mutex before the queue is frozen in cost model write Guixin Liu
2026-11-19 13:40   ` Tao Cui
2026-11-20  2:14     ` Guixin Liu
2026-11-20  2:56       ` Tao Cui [this message]
2026-11-19  7:05 ` [PATCH 4/7] blk-iocost: unfreeze the queue before dropping the bdev reference in qos write Guixin Liu
2026-11-19  7:05 ` [PATCH 5/7] blk-stat: account for the mean of the source in blk_rq_stat_sum() Guixin Liu
2026-11-19 13:32   ` Tao Cui
2026-11-20  2:04     ` Guixin Liu
2026-11-20  2:50       ` Tao Cui
2026-11-20  6:22         ` Guixin Liu
2026-11-19  7:05 ` [PATCH 6/7] blk-cgroup: don't attach a qos policy to a dead disk Guixin Liu
2026-11-19  7:05 ` [PATCH 7/7] blk-mq: unpublish the tags before deferring their free to SRCU Guixin Liu

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=7106a456-8e3d-4fe2-ae7d-b8e753e2b131@linux.dev \
    --to=cui.tao@linux.dev \
    --cc=axboe@kernel.dk \
    --cc=hare@suse.de \
    --cc=hch@lst.de \
    --cc=josef@toxicpanda.com \
    --cc=kanie@linux.alibaba.com \
    --cc=linux-block@vger.kernel.org \
    --cc=ming.lei@redhat.com \
    --cc=nilay@linux.ibm.com \
    --cc=tj@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.