Re: [RFC] Another take at restarting FUSE servers

[Bernd Schubert <bschubert@ddn.com>]

On 11/5/25 22:38, Darrick J. Wong wrote:
> On Wed, Nov 05, 2025 at 04:30:51PM +0100, Amir Goldstein wrote:
>> On Wed, Nov 5, 2025 at 12:50 PM Luis Henriques <luis@igalia.com> wrote:
>>>
>>> Hi Amir,
>>>
>>> On Wed, Nov 05 2025, Amir Goldstein wrote:
>>>
>>>> On Tue, Nov 4, 2025 at 3:52 PM Luis Henriques <luis@igalia.com> wrote:
>>>
>>> <...>
>>>
>>>>>> fuse_entry_out was extended once and fuse_reply_entry()
>>>>>> sends the size of the struct.
>>>>>
>>>>> So, if I'm understanding you correctly, you're suggesting to extend
>>>>> fuse_entry_out to add the new handle (a 'size' field + the actual handle).
>>>>
>>>> Well it depends...
>>>>
>>>> There are several ways to do it.
>>>> I would really like to get Miklos and Bernd's opinion on the preferred way.
>>>
>>> Sure, all feedback is welcome!
>>>
>>>> So far, it looks like the client determines the size of the output args.
>>>>
>>>> If we want the server to be able to write a different file handle size
>>>> per inode that's going to be a bigger challenge.
>>>>
>>>> I think it's plenty enough if server and client negotiate a max file handle
>>>> size and then the client always reserves enough space in the output
>>>> args buffer.
>>>>
>>>> One more thing to ask is what is "the actual handle".
>>>> If "the actual handle" is the variable sized struct file_handle then
>>>> the size is already available in the file handle header.
>>>
>>> Actually, this is exactly what I was trying to mimic for my initial
>>> attempt.  However, I was not going to do any size negotiation but instead
>>> define a maximum size for the handle.  See below.
>>>
>>>> If it is not, then I think some sort of type or version of the file handles
>>>> encoding should be negotiated beyond the max handle size.
>>>
>>> In my initial stab at this I was going to take a very simple approach and
>>> hard-code a maximum size for the handle.  This would have the advantage of
>>> allowing the server to use different sizes for different inodes (though
>>> I'm not sure how useful that would be in practice).  So, in summary, I
>>> would define the new handle like this:
>>>
>>> /* Same value as MAX_HANDLE_SZ */
>>> #define FUSE_MAX_HANDLE_SZ 128
>>>
>>> struct fuse_file_handle {
>>>         uint32_t        size;
>>>         uint32_t        padding;
>>
>> I think that the handle type is going to be relevant as well.
>>
>>>         char            handle[FUSE_MAX_HANDLE_SZ];
>>> };
>>>
>>> and this struct would be included in fuse_entry_out.
>>>
>>> There's probably a problem with having this (big) fixed size increase to
>>> fuse_entry_out, but maybe that could be fixed once I have all the other
>>> details sorted out.  Hopefully I'm not oversimplifying the problem,
>>> skipping the need for negotiating a handle size.
>>>
>>
>> Maybe this fixed size is reasonable for the first version of FUSE protocol
>> as long as this overhead is NOT added if the server does not opt-in for the
>> feature.
>>
>> IOW, allow the server to negotiate FUSE_MAX_HANDLE_SZ or 0,
>> but keep the negotiation protocol extendable to another value later on.
>>
>>>>> That's probably a good idea.  I was working towards having the
>>>>> LOOKUP_HANDLE to be similar to LOOKUP, but extending it so that it would
>>>>> include:
>>>>>
>>>>>  - An extra inarg: the parent directory handle.  (To be honest, I'm not
>>>>>    really sure this would be needed.)
>>>>
>>>> Yes, I think you need extra inarg.
>>>> Why would it not be needed?
>>>> The problem is that you cannot know if the parent node id in the lookup
>>>> command is stale after server restart.
>>>
>>> Ah, of course.  Hence the need for this extra inarg.
>>>
>>>> The thing is that the kernel fuse inode will need to store the file handle,
>>>> much the same as an NFS client stores the file handle provided by the
>>>> NFS server.
>>>>
>>>> FYI, fanotify has an optimized way to store file handles in
>>>> struct fanotify_fid_event - small file handles are stored inline
>>>> and larger file handles can use an external buffer.
>>>>
>>>> But fuse does not need to support any size of file handles.
>>>> For first version we could definitely simplify things by limiting the size
>>>> of supported file handles, because server and client need to negotiate
>>>> the max file handle size anyway.
>>>
>>> I'll definitely need to have a look at how fanotify does that.  But I
>>> guess that if my simplistic approach with a static array is acceptable for
>>> now, I'll stick with it for the initial attempt to implement this, and
>>> eventually revisit it later to do something more clever.
>>>
>>
>> What you proposed is the extension of fuse_entry_out for fuse
>> protocol.
>>
>> My reference to fanotify_fid_event is meant to explain how to encode
>> a file handle in fuse_inode in cache, because the fuse_inode_cachep
>> cannot have variable sized inodes and in most of the cases, a short
>> inline file handle should be enough.
>>
>> Therefore, if you limit the support in the first version to something like
>> FANOTIFY_INLINE_FH_LEN, you can always store the file handle
>> in fuse_inode and postpone support for bigger file handles to later.
> 
> I suggest that you also provide a way for the fuse server to tell the
> kernel that it can construct its own handles from {fuse_inode::nodeid,
> inode::i_generation} if they want something more efficient than
> uploading 128b blobs.

Isn't that covered by handle size defined in FUSE_INIT reply? I.e.
handle size would be 0B in this case? 

Thanks,
Bernd