From: Jens Axboe <axboe@kernel.dk>
To: Coly Li <colyli@fnnas.com>, mingzhe.zou@easystack.cn
Cc: colyli@kernel.org, linux-bcache@vger.kernel.org,
zoumingzhe@qq.com, zoumingzhe@outlook.com
Subject: Re: [PATCH v3] bcache: fix cached_dev.sb_bio use-after-free and crash
Date: Tue, 31 Mar 2026 10:34:19 -0600 [thread overview]
Message-ID: <99e65e33-faec-49f5-b17b-5b47e5e4754e@kernel.dk> (raw)
In-Reply-To: <acvrq_PBzn4T2TqH@studio.local>
On 3/31/26 9:46 AM, Coly Li wrote:
> On Tue, Mar 31, 2026 at 09:56:18AM +0800, Coly Li wrote:
>> On Mon, Mar 30, 2026 at 09:11:53PM +0800, mingzhe.zou@easystack.cn wrote:
>>> From: Mingzhe Zou <mingzhe.zou@easystack.cn>
>>>
>>> In our production environment, we have received multiple crash reports
>>> regarding libceph, which have caught our attention:
>>>
>>> ```
>>> [6888366.280350] Call Trace:
>>> [6888366.280452] blk_update_request+0x14e/0x370
>>> [6888366.280561] blk_mq_end_request+0x1a/0x130
>>> [6888366.280671] rbd_img_handle_request+0x1a0/0x1b0 [rbd]
>>> [6888366.280792] rbd_obj_handle_request+0x32/0x40 [rbd]
>>> [6888366.280903] __complete_request+0x22/0x70 [libceph]
>>> [6888366.281032] osd_dispatch+0x15e/0xb40 [libceph]
>>> [6888366.281164] ? inet_recvmsg+0x5b/0xd0
>>> [6888366.281272] ? ceph_tcp_recvmsg+0x6f/0xa0 [libceph]
>>> [6888366.281405] ceph_con_process_message+0x79/0x140 [libceph]
>>> [6888366.281534] ceph_con_v1_try_read+0x5d7/0xf30 [libceph]
>>> [6888366.281661] ceph_con_workfn+0x329/0x680 [libceph]
>>> ```
>>>
>>> After analyzing the coredump file, we found that the address of dc->sb_bio
>>> has been freed. We know that cached_dev is only freed when it is stopped.
>>>
>>> Since sb_bio is a part of struct cached_dev, rather than an alloc every time.
>>> If the device is stopped while writing to the superblock, the released address
>>> will be accessed at endio.
>>>
>>> This patch hopes to wait for sb_write to complete in cached_dev_free.
>>>
>>> Signed-off-by: Mingzhe Zou <mingzhe.zou@easystack.cn>
>>>
>>
>> Hi Mingzhe,
>>
>> Yeah, this patch is in better shape. Thank for the fix up again.
>>
>>> ---
>>> v2: fix the crash caused by not calling closure_init in v1
>>> v3: pair the down and up of semaphore sb_write_mutex
>>> ---
>>> drivers/md/bcache/super.c | 8 ++++++++
>>> 1 file changed, 8 insertions(+)
>>>
>>> diff --git a/drivers/md/bcache/super.c b/drivers/md/bcache/super.c
>>> index 64bb38c95895..97d9adb0bf96 100644
>>> --- a/drivers/md/bcache/super.c
>>> +++ b/drivers/md/bcache/super.c
>>> @@ -1373,6 +1373,14 @@ static CLOSURE_CALLBACK(cached_dev_free)
>>>
>>> mutex_unlock(&bch_register_lock);
>>>
>>> + /*
>>> + * Wait for any pending sb_write to complete before free.
>>> + * The sb_bio is embedded in struct cached_dev, so we must
>>> + * ensure no I/O is in progress.
>>> + */
>>> + down(&dc->sb_write_mutex);
>>> + up(&dc->sb_write_mutex);
>>> +
>>> if (dc->sb_disk)
>>> folio_put(virt_to_folio(dc->sb_disk));
>>>
>>
>> The patch itself is good. But the previous one (based on closre_sync()) is
>> in Jens block-tree. Let me ask.
>>
>> Hi Jens,
>>
>> Should Mingzhe send a incremental patch based on commit b36478a1fece in
>> block-7.0 branch of linux-block tree? Or Just use this patch to replace
>> the in-linux-block-tree one?
>>
>
> Hi Mingzhe,
>
> Considering no responds from Jens at this moment, I assume that an incremental
> patch againt the already-in-linux-block patch might be more convenient for him.
> Then he can simply apply the incremental patch without extra rebase stuffs.
>
> Can you re-compose a patch against block-7.0 branch of linux-block tree?
Yes send a new patch, I'm not going to rebase the tree just to fold in
a fix for a patch that's deep in the stack. The fixup doesn't even
mention which patch it's fixing either...
--
Jens Axboe
prev parent reply other threads:[~2026-03-31 16:34 UTC|newest]
Thread overview: 4+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-03-30 13:11 [PATCH v3] bcache: fix cached_dev.sb_bio use-after-free and crash mingzhe.zou
2026-03-31 1:56 ` Coly Li
2026-03-31 15:46 ` Coly Li
2026-03-31 16:34 ` Jens Axboe [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=99e65e33-faec-49f5-b17b-5b47e5e4754e@kernel.dk \
--to=axboe@kernel.dk \
--cc=colyli@fnnas.com \
--cc=colyli@kernel.org \
--cc=linux-bcache@vger.kernel.org \
--cc=mingzhe.zou@easystack.cn \
--cc=zoumingzhe@outlook.com \
--cc=zoumingzhe@qq.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox