Re: [RFC] Another take at restarting FUSE servers

[Amir Goldstein <amir73il@gmail.com>]

On Wed, Nov 5, 2025 at 12:50 PM Luis Henriques <luis@igalia.com> wrote:
>
> Hi Amir,
>
> On Wed, Nov 05 2025, Amir Goldstein wrote:
>
> > On Tue, Nov 4, 2025 at 3:52 PM Luis Henriques <luis@igalia.com> wrote:
>
> <...>
>
> >> > fuse_entry_out was extended once and fuse_reply_entry()
> >> > sends the size of the struct.
> >>
> >> So, if I'm understanding you correctly, you're suggesting to extend
> >> fuse_entry_out to add the new handle (a 'size' field + the actual handle).
> >
> > Well it depends...
> >
> > There are several ways to do it.
> > I would really like to get Miklos and Bernd's opinion on the preferred way.
>
> Sure, all feedback is welcome!
>
> > So far, it looks like the client determines the size of the output args.
> >
> > If we want the server to be able to write a different file handle size
> > per inode that's going to be a bigger challenge.
> >
> > I think it's plenty enough if server and client negotiate a max file handle
> > size and then the client always reserves enough space in the output
> > args buffer.
> >
> > One more thing to ask is what is "the actual handle".
> > If "the actual handle" is the variable sized struct file_handle then
> > the size is already available in the file handle header.
>
> Actually, this is exactly what I was trying to mimic for my initial
> attempt.  However, I was not going to do any size negotiation but instead
> define a maximum size for the handle.  See below.
>
> > If it is not, then I think some sort of type or version of the file handles
> > encoding should be negotiated beyond the max handle size.
>
> In my initial stab at this I was going to take a very simple approach and
> hard-code a maximum size for the handle.  This would have the advantage of
> allowing the server to use different sizes for different inodes (though
> I'm not sure how useful that would be in practice).  So, in summary, I
> would define the new handle like this:
>
> /* Same value as MAX_HANDLE_SZ */
> #define FUSE_MAX_HANDLE_SZ 128
>
> struct fuse_file_handle {
>         uint32_t        size;
>         uint32_t        padding;

I think that the handle type is going to be relevant as well.

>         char            handle[FUSE_MAX_HANDLE_SZ];
> };
>
> and this struct would be included in fuse_entry_out.
>
> There's probably a problem with having this (big) fixed size increase to
> fuse_entry_out, but maybe that could be fixed once I have all the other
> details sorted out.  Hopefully I'm not oversimplifying the problem,
> skipping the need for negotiating a handle size.
>

Maybe this fixed size is reasonable for the first version of FUSE protocol
as long as this overhead is NOT added if the server does not opt-in for the
feature.

IOW, allow the server to negotiate FUSE_MAX_HANDLE_SZ or 0,
but keep the negotiation protocol extendable to another value later on.

> >> That's probably a good idea.  I was working towards having the
> >> LOOKUP_HANDLE to be similar to LOOKUP, but extending it so that it would
> >> include:
> >>
> >>  - An extra inarg: the parent directory handle.  (To be honest, I'm not
> >>    really sure this would be needed.)
> >
> > Yes, I think you need extra inarg.
> > Why would it not be needed?
> > The problem is that you cannot know if the parent node id in the lookup
> > command is stale after server restart.
>
> Ah, of course.  Hence the need for this extra inarg.
>
> > The thing is that the kernel fuse inode will need to store the file handle,
> > much the same as an NFS client stores the file handle provided by the
> > NFS server.
> >
> > FYI, fanotify has an optimized way to store file handles in
> > struct fanotify_fid_event - small file handles are stored inline
> > and larger file handles can use an external buffer.
> >
> > But fuse does not need to support any size of file handles.
> > For first version we could definitely simplify things by limiting the size
> > of supported file handles, because server and client need to negotiate
> > the max file handle size anyway.
>
> I'll definitely need to have a look at how fanotify does that.  But I
> guess that if my simplistic approach with a static array is acceptable for
> now, I'll stick with it for the initial attempt to implement this, and
> eventually revisit it later to do something more clever.
>

What you proposed is the extension of fuse_entry_out for fuse
protocol.

My reference to fanotify_fid_event is meant to explain how to encode
a file handle in fuse_inode in cache, because the fuse_inode_cachep
cannot have variable sized inodes and in most of the cases, a short
inline file handle should be enough.

Therefore, if you limit the support in the first version to something like
FANOTIFY_INLINE_FH_LEN, you can always store the file handle
in fuse_inode and postpone support for bigger file handles to later.

Thanks,
Amir.