From: Ming Lei <ming.lei@redhat.com>
To: Jens Axboe <axboe@kernel.dk>
Cc: linux-block@vger.kernel.org,
Caleb Sander Mateos <csander@purestorage.com>,
Uday Shankar <ushankar@purestorage.com>
Subject: Re: [PATCH V2] ublk: fix deadlock when reading partition table
Date: Wed, 17 Dec 2025 11:33:11 +0800 [thread overview]
Message-ID: <aUIkd9Nt9oSmHKKp@fedora> (raw)
In-Reply-To: <8b2d7335-fd49-4c15-87d9-0eb50e0a09a1@kernel.dk>
On Tue, Dec 16, 2025 at 08:19:15PM -0700, Jens Axboe wrote:
> On 12/16/25 8:09 PM, Ming Lei wrote:
> > On Tue, Dec 16, 2025 at 10:57:25AM -0700, Jens Axboe wrote:
> >> On 12/16/25 8:03 AM, Jens Axboe wrote:
> >>>> The issue for ublk is actually triggered by something abnormal: submit AIO
> >>>> & close(ublk disk) in client application, then fput() is called when the
> >>>> submitted AIO is done, it will cause deferred fput handler to wq for any block
> >>>> IO completed from irq handler.
> >>>
> >>> My suggested logic is something ala this in bdev_release():
> >>>
> >>> if (current->flags & PF_KTHREAD) {
> >>> mutex_lock(&disk->open_mutex);
> >>> } else {
> >>> if (!mutex_trylock(&disk->open_mutex)) {
> >>> deferred_put(file);
> >>> return;
> >>> }
> >>> }
> >>>
> >>> and that's about it.
> >>
> >> I took a look at the bug report, and now it makes more sense to me -
> >> this is an aio only issue, as it does fput() from ->bi_end_io() context.
> >> That's pretty nasty, as you don't really know what context that might
> >> be, both in terms of irq/bh state, but also in terms of locks. The
> >> former fput() does work around.
> >>
> >> Why isn't the fix something as simple as the below, with your comment
> >> added on top? I'm not aware of anyone else that would do fput off
> >> ->bi_end_io, so we migt as well treat the source of the issue rather
> >> than work around it in ublk. THAT makes a lot more sense to me.
> >
> > It doesn't matter if fput is called from ->bi_end_io() directly, it can
> > be triggered on io-uring indirectly too, in which fput() is called from
> > __io_submit_flush_completions() in case of non-registerd file.
>
> Because of the work-around in io_req_post_cqe()? Or just because of
> !DEFER_TASKRUN?
When fput() is called from __io_submit_flush_completions(), its release
handler will be deferred to run task work, where the current task
is blocked because of ->open_mutex.
It is actually one ublk specific issue which relies on the current task
for handling IO and providing forward progress, so cause deadlock since
reading partition table(with ->open_mutex) requires the task for handling IO.
>
> The real problem is holding ->open_mutex over IO, and then also
> requiring it to put the file as well. bdev_release() should be able to
> work-around that, rather than need anyone to paper around it.
deferred bdev_release is not safe, for example of suggestion:
if (current->flags & PF_KTHREAD) {
mutex_lock(&disk->open_mutex);
} else {
if (!mutex_trylock(&disk->open_mutex)) {
deferred_put(file);
return;
}
}
deferred_put(file) will cause disk released after returning to userspace.
Yes, __fput_deferred() allows that in case of in_interrupt(), which usually
means one abnormal application(close(disk) before completing/handling IO),
but it will cause normal application to release disk after returning to
userspace, it may cause -EBUSY for following syscall.
Thanks,
Ming
next prev parent reply other threads:[~2025-12-17 3:33 UTC|newest]
Thread overview: 13+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-12-12 14:34 [PATCH V2] ublk: fix deadlock when reading partition table Ming Lei
2025-12-12 16:57 ` Caleb Sander Mateos
2025-12-12 19:49 ` Jens Axboe
2025-12-13 2:28 ` Ming Lei
2025-12-14 6:41 ` Jens Axboe
2025-12-16 8:56 ` Ming Lei
2025-12-16 15:03 ` Jens Axboe
2025-12-16 17:57 ` Jens Axboe
2025-12-17 3:09 ` Ming Lei
2025-12-17 3:19 ` Jens Axboe
2025-12-17 3:33 ` Ming Lei [this message]
2025-12-18 2:37 ` Jens Axboe
2025-12-18 2:41 ` Jens Axboe
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=aUIkd9Nt9oSmHKKp@fedora \
--to=ming.lei@redhat.com \
--cc=axboe@kernel.dk \
--cc=csander@purestorage.com \
--cc=linux-block@vger.kernel.org \
--cc=ushankar@purestorage.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox