[LSF/MM/BPF TOPIC] FUSE io_uring zero copy

[David Wei <dw@davidwei.uk>]

Hi folks, I want to propose a discussion on adding zero copy to FUSE
io_uring in the kernel. The source is some userspace buffer or device
memory e.g. GPU VRAM. The destination is FUSE server in userspace, which
will then either forward it over the network or to an underlying
FS/block device. The FUSE server may want to read the data.

My goal is to eliminate copies in this entire data path, including the
initial hop between the userspace client and the kernel. I know Ming and
Keith are working on adding ublk zero copy but it does not cover this
initial hop and it does not allow the ublk/FUSE server to read the data.

My idea is to use shared memory or dma-buf, i.e. the source data is
encapsulated in an mmap()able fd. The client and FUSE server exchange
this fd through a back channel with no kernel involvement. The FUSE
server maps the fd into its address space and registers the fd with
io_uring via the io_uring_register() infra. When the client does e.g. a
DIO write, the pages are pinned and forwarded to FUSE kernel, which does
a lookup and understands that the pages belong to the fd that was
registered from the FUSE server. Then io_uring tells the FUSE server
that the data is in the fd it registered, so there is no need to copy
anything at all.

I would like to discuss this and get feedback from the community. My top
question is why do this in the kernel at all? It is entirely possible to
bypass the kernel entirely by having the client and FUSE server exchange
the fd and then do the I/O purely through IPC.

David