Re: [PATCH v3] fuse: optional FORGET delivery over io_uring

[Li Wang <liwang@kylinos.cn>]

Hi Bernd and Joanne,

On 26/04/2026 23:48, Bernd Schubert wrote:
> 
> 
> On 4/24/26 22:38, Joanne Koong wrote:
>> On Thu, Apr 23, 2026 at 4:11 AM Li Wang <liwang@kylinos.cn> wrote:
>>>
>>> Deliver FUSE_FORGET through fuse_uring_queue_fuse_req() when the io_uring
>>> is ready and userspace has opted in by setting
>>> FUSE_IO_URING_REGISTER_FORGET_COMMIT in fuse_uring_cmd_req.flags on
>>> FUSE_IO_URING_CMD_REGISTER. Until any REGISTER
>>> carries that bit, FORGET continues to use the legacy
>>> fuse_dev_queue_forget() path even while io_uring is active, so unmodified
>>> userspace (e.g. libfuse that does not issue a completion SQE for FORGET)
>>> does not wedge ring entries.
>>>
>>> Benefits:
>>> - FORGET can share the same commit/fetch loop as other opcodes.
>>> - Reduces split transport for high-volume forgets when the ring is primary.
>>> - Reuses existing per-queue io-uring machinery and noreply/force
>>>   request setup.
>>>
>>> Signed-off-by: Li Wang <liwang@kylinos.cn>
>>
>> Hi Li,
>>
>> Thanks for sending this. To be completely honest, I'm not convinced
>> delivering forget over io-uring is worth the added complexity/cost. In
>> the /dev/fuse path we rely on forget batching/amoritizing and explicit
>> scheduling/fairness logic so forget processing makes progress and
>> doesn't get drowned out by regular requests; I think we'd likely need
>> something comparable for the io-uring path as well. Additionally,
>> routing it through io-uring makes forget behave more like a "real"
>> request on the ring (it needs per-request state to live until
>> userspace completes and the entry can be recyycled) which introduces
>> extra allocation/lifetime management on this path and it requires a
>> uapi change and corresponding libfuse changes.
>>
>> Forgets would consume ring entries but they're tiny one-way
>> notifications and imo I don't think they benefit much from io-uring's
>> main advantages (eg data-path/zero-copy). I worry theyy could contend
>> with read-write heavy traffic where ring capacity is more valuable.
> 
> I think when FORGET starts to disturb writes or reads, there also must
> be some metadata load that causes many of these requests. Going via
> /dev/fuse also includes another two syscalls and cpu task switch to the
> libfuse thread handling them - that is not for free either.
> 
> However, this currently might disturb reads/writes, if the queue depth
> is limited and memory optimized to carry reads/writes. The missing
> feature here is to have a multiple request sizes on the same queue.
> 
> Btw, syscall overhead is basically the reason why I wouldn't like to
> have multiple rings per request size, but one ring with entries of
> different sizes. Will try to respond to the other mail later today.
> 
>>
>> Given that, my preference would be to keep forget/interrupts on the
>> legacy /dev/fuse path even when io-uring is enabled. Sorry for not
>> bringing this up earlier in v2. Bernd, I’d also be curious to hear
>> your perspective on this.
> 
> It is certainly not something that I would count as priority. Though I
> don't think it would hurt either. In the end we might be able to run
> entirely without /dev/fuse IO and threads and would save a bit resources.
> 
> @Li do you see performance improvements with FORGETs through the ring or
> could you post your underlying motivation?
> 
To be honest, my initial motivation for this wasn't necessarily performance, 
but rather architectural simplicity. Maintaining two separate paths and 
thread pools for forget/interrupt and I/O requests adds a bit of complexity 
to the code and introduces relatively higher overhead, as well as some 
resource redundancy.

Furthermore, in the current implementation, forget and I/O requests are 
dispatched through different paths and handled by different sets of threads. 
This means we cannot guarantee their original priority order upon reception, 
and it potentially introduces extra context-switching overhead, which 
impacts the processing efficiency of both.

Theoretically, optimization measures such as batching forget requests can 
also be implemented via the io_uring path. This would allow us to amortize 
the state maintenance and completion overhead of forget requests, while also 
addressing the issue of uniform request sizes within the same ring. I'll 
get around to running some performance benchmarks.

Thanks,
Li