Linux filesystem development
 help / color / mirror / Atom feed
From: "Darrick J. Wong" <djwong@kernel.org>
To: Sergio Lopez <slp@redhat.com>
Cc: Miklos Szeredi <miklos@szeredi.hu>, linux-fsdevel@vger.kernel.org
Subject: Re: [PATCH] fuse: mark DAX inode releases as blocking
Date: Mon, 26 Jan 2026 10:40:15 -0800	[thread overview]
Message-ID: <20260126184015.GC5900@frogsfrogsfrogs> (raw)
In-Reply-To: <20260118232411.536710-1-slp@redhat.com>

On Mon, Jan 19, 2026 at 12:24:11AM +0100, Sergio Lopez wrote:
> Commit 26e5c67deb2e ("fuse: fix livelock in synchronous file put from
> fuseblk workers") made fputs on closing files always asynchronous.
> 
> As cleaning up DAX inodes may require issuing a number of synchronous
> request for releasing the mappings, completing the release request from
> the worker thread may lead to it hanging like this:
> 
> [   21.386751] Workqueue: events virtio_fs_requests_done_work
> [   21.386769] Call trace:
> [   21.386770]  __switch_to+0xe4/0x140
> [   21.386780]  __schedule+0x294/0x72c
> [   21.386787]  schedule+0x24/0x90
> [   21.386794]  request_wait_answer+0x184/0x298
> [   21.386799]  __fuse_simple_request+0x1f4/0x320
> [   21.386805]  fuse_send_removemapping+0x80/0xa0
> [   21.386810]  dmap_removemapping_list+0xac/0xfc
> [   21.386814]  inode_reclaim_dmap_range.constprop.0+0xd0/0x204
> [   21.386820]  fuse_dax_inode_cleanup+0x28/0x5c
> [   21.386825]  fuse_evict_inode+0x120/0x190
> [   21.386834]  evict+0x188/0x320
> [   21.386847]  iput_final+0xb0/0x20c
> [   21.386854]  iput+0xa0/0xbc
> [   21.386862]  fuse_release_end+0x18/0x2c
> [   21.386868]  fuse_request_end+0x9c/0x2c0

Ok, so this is the reply from the async FUSE_RELEASE command.  But then
we iput the inode, which results in fuse issuing a new synchronous
command from within the completion for the first command.

Ouch.

> [   21.386872]  virtio_fs_request_complete+0x150/0x384
> [   21.386879]  virtio_fs_requests_done_work+0x18c/0x37c
> [   21.386885]  process_one_work+0x15c/0x2e8
> [   21.386891]  worker_thread+0x278/0x480
> [   21.386898]  kthread+0xd0/0xdc
> [   21.386902]  ret_from_fork+0x10/0x20
> 
> Here, the virtio-fs worker_thread is waiting on request_wait_answer()
> for a reply from the virtio-fs server that is already in the virtqueue
> but will never be processed since it's that same worker thread the one
> in charge of consuming the elements from the virtqueue.

Yes.  Ow.

> To address this issue, when relesing a DAX inode mark the operation as
> potentially blocking. Doing this will ensure these release requests are
> processed on a different worker thread.

I wonder if you've solved this problem report?
https://github.com/Nevuly/WSL2-Rolling-Kernel-Issue/issues/38

Naturally they reverted the patch, emailed me, and refused to talk about
this on the public list, which is why nobody's heard of this until now.

> Signed-off-by: Sergio Lopez <slp@redhat.com>
> ---
>  fs/fuse/file.c | 6 ++++++
>  1 file changed, 6 insertions(+)
> 
> diff --git a/fs/fuse/file.c b/fs/fuse/file.c
> index 3b2a171e652f..a65c5d32a34b 100644
> --- a/fs/fuse/file.c
> +++ b/fs/fuse/file.c
> @@ -117,6 +117,12 @@ static void fuse_file_put(struct fuse_file *ff, bool sync)
>  			fuse_simple_request(ff->fm, args);
>  			fuse_release_end(ff->fm, args, 0);
>  		} else {
> +			/*
> +			 * DAX inodes may need to issue a number of synchronous
> +			 * request for clearing the mappings.
> +			 */
> +			if (ra && ra->inode && FUSE_IS_DAX(ra->inode))
> +				args->may_block = true;

There's no documentation for what may_block does, but there are so few
uses of it that I can tell that this is kicking the FUSE_RELEASE
completion to a workqueue instead of processing it directly, which
eliminates the livelock.

I wonder if fuse ought to grow the ability to whine when something is
trying to issue a synchronous fuse command while running in a command
queue completion context (aka the worker threads) but I don't know how
difficult that would *really* be.

Insofar as I understand anything in fuse, I think this is the right
thing to do.  Thanks for fixing this.

Reviewed-by: "Darrick J. Wong" <djwong@kernel.org>

--D

>  			args->end = fuse_release_end;
>  			if (fuse_simple_background(ff->fm, args,
>  						   GFP_KERNEL | __GFP_NOFAIL))
> -- 
> 2.52.0
> 
> 

  reply	other threads:[~2026-01-26 18:40 UTC|newest]

Thread overview: 8+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-01-18 23:24 [PATCH] fuse: mark DAX inode releases as blocking Sergio Lopez
2026-01-26 18:40 ` Darrick J. Wong [this message]
2026-01-27  2:43   ` Jingbo Xu
2026-01-27 11:18     ` Sergio Lopez Pascual
2026-01-27 11:30       ` Jingbo Xu
2026-01-27 11:12   ` Sergio Lopez Pascual
2026-01-27 23:30     ` Darrick J. Wong
2026-02-17 16:38 ` Miklos Szeredi

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20260126184015.GC5900@frogsfrogsfrogs \
    --to=djwong@kernel.org \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=miklos@szeredi.hu \
    --cc=slp@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox