From: "Darrick J. Wong" <djwong@kernel.org>
To: Sergio Lopez <slp@redhat.com>
Cc: Miklos Szeredi <miklos@szeredi.hu>, linux-fsdevel@vger.kernel.org
Subject: Re: [PATCH] fuse: mark DAX inode releases as blocking
Date: Mon, 26 Jan 2026 10:40:15 -0800 [thread overview]
Message-ID: <20260126184015.GC5900@frogsfrogsfrogs> (raw)
In-Reply-To: <20260118232411.536710-1-slp@redhat.com>
On Mon, Jan 19, 2026 at 12:24:11AM +0100, Sergio Lopez wrote:
> Commit 26e5c67deb2e ("fuse: fix livelock in synchronous file put from
> fuseblk workers") made fputs on closing files always asynchronous.
>
> As cleaning up DAX inodes may require issuing a number of synchronous
> request for releasing the mappings, completing the release request from
> the worker thread may lead to it hanging like this:
>
> [ 21.386751] Workqueue: events virtio_fs_requests_done_work
> [ 21.386769] Call trace:
> [ 21.386770] __switch_to+0xe4/0x140
> [ 21.386780] __schedule+0x294/0x72c
> [ 21.386787] schedule+0x24/0x90
> [ 21.386794] request_wait_answer+0x184/0x298
> [ 21.386799] __fuse_simple_request+0x1f4/0x320
> [ 21.386805] fuse_send_removemapping+0x80/0xa0
> [ 21.386810] dmap_removemapping_list+0xac/0xfc
> [ 21.386814] inode_reclaim_dmap_range.constprop.0+0xd0/0x204
> [ 21.386820] fuse_dax_inode_cleanup+0x28/0x5c
> [ 21.386825] fuse_evict_inode+0x120/0x190
> [ 21.386834] evict+0x188/0x320
> [ 21.386847] iput_final+0xb0/0x20c
> [ 21.386854] iput+0xa0/0xbc
> [ 21.386862] fuse_release_end+0x18/0x2c
> [ 21.386868] fuse_request_end+0x9c/0x2c0
Ok, so this is the reply from the async FUSE_RELEASE command. But then
we iput the inode, which results in fuse issuing a new synchronous
command from within the completion for the first command.
Ouch.
> [ 21.386872] virtio_fs_request_complete+0x150/0x384
> [ 21.386879] virtio_fs_requests_done_work+0x18c/0x37c
> [ 21.386885] process_one_work+0x15c/0x2e8
> [ 21.386891] worker_thread+0x278/0x480
> [ 21.386898] kthread+0xd0/0xdc
> [ 21.386902] ret_from_fork+0x10/0x20
>
> Here, the virtio-fs worker_thread is waiting on request_wait_answer()
> for a reply from the virtio-fs server that is already in the virtqueue
> but will never be processed since it's that same worker thread the one
> in charge of consuming the elements from the virtqueue.
Yes. Ow.
> To address this issue, when relesing a DAX inode mark the operation as
> potentially blocking. Doing this will ensure these release requests are
> processed on a different worker thread.
I wonder if you've solved this problem report?
https://github.com/Nevuly/WSL2-Rolling-Kernel-Issue/issues/38
Naturally they reverted the patch, emailed me, and refused to talk about
this on the public list, which is why nobody's heard of this until now.
> Signed-off-by: Sergio Lopez <slp@redhat.com>
> ---
> fs/fuse/file.c | 6 ++++++
> 1 file changed, 6 insertions(+)
>
> diff --git a/fs/fuse/file.c b/fs/fuse/file.c
> index 3b2a171e652f..a65c5d32a34b 100644
> --- a/fs/fuse/file.c
> +++ b/fs/fuse/file.c
> @@ -117,6 +117,12 @@ static void fuse_file_put(struct fuse_file *ff, bool sync)
> fuse_simple_request(ff->fm, args);
> fuse_release_end(ff->fm, args, 0);
> } else {
> + /*
> + * DAX inodes may need to issue a number of synchronous
> + * request for clearing the mappings.
> + */
> + if (ra && ra->inode && FUSE_IS_DAX(ra->inode))
> + args->may_block = true;
There's no documentation for what may_block does, but there are so few
uses of it that I can tell that this is kicking the FUSE_RELEASE
completion to a workqueue instead of processing it directly, which
eliminates the livelock.
I wonder if fuse ought to grow the ability to whine when something is
trying to issue a synchronous fuse command while running in a command
queue completion context (aka the worker threads) but I don't know how
difficult that would *really* be.
Insofar as I understand anything in fuse, I think this is the right
thing to do. Thanks for fixing this.
Reviewed-by: "Darrick J. Wong" <djwong@kernel.org>
--D
> args->end = fuse_release_end;
> if (fuse_simple_background(ff->fm, args,
> GFP_KERNEL | __GFP_NOFAIL))
> --
> 2.52.0
>
>
next prev parent reply other threads:[~2026-01-26 18:40 UTC|newest]
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-01-18 23:24 [PATCH] fuse: mark DAX inode releases as blocking Sergio Lopez
2026-01-26 18:40 ` Darrick J. Wong [this message]
2026-01-27 2:43 ` Jingbo Xu
2026-01-27 11:18 ` Sergio Lopez Pascual
2026-01-27 11:30 ` Jingbo Xu
2026-01-27 11:12 ` Sergio Lopez Pascual
2026-01-27 23:30 ` Darrick J. Wong
2026-02-17 16:38 ` Miklos Szeredi
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20260126184015.GC5900@frogsfrogsfrogs \
--to=djwong@kernel.org \
--cc=linux-fsdevel@vger.kernel.org \
--cc=miklos@szeredi.hu \
--cc=slp@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox