Re: [PATCH] fuse: Add support for fuse stacked I/O

[Nikolaus Rath <Nikolaus-BTH8mxji4b0@public.gmane.org>]

On Jan 15 2016, Antonio SJ Musumeci <trapexit-wGTF+nt6ur047o9RxwvyTQ@public.gmane.org> wrote:
> The idea is that you want to be able to reason about open, create, etc. but
> don't care about the data transfer.
>
> I have N filesystems I wish to unionize. When I create a new file I want to
> pick the drive with the most free space (or some other algo). creat is
> called, succeeds, and now the application issuing this starts writing. The
> FUSE fs doesn't care about the writes. It just wanted to pick the drive
> this file should have been created on. Anything I'd do with the FD after
> that I'm happy to short circuit. I don't need to be asked what to do when
> fstat'ing this FD or anything which in FUSE hands over the 'fh'. It's just
> a file descriptor for me and I'd simply be calling the same function.
>
> Ideally I think one would want to be able to select which functions to
> short circuit and maybe even have it so that a short circuited function
> could propagate back through FUSE on error. But the read and write short
> circuiting is probably the biggest win given the overhead.


I think you should avoid using the term "stacked" completely (which
would also make Christoph happy). There have been several discussions in
the past about adding a "fd delegation" function to FUSE. Generally, the
idea is that the FUSE userspace code tells the FUSE kernel module to
internally "delegate" to writes and reads for a given file (or even a
range in that file) to a different file descriptor provided by
userspace.

I think that function would be useful, and not just for union file
systems. There are many FUSE file systems that end up writing the data
into some other file on the disk without doing any transformations on
the data itself. Especially with the range feature, they would all
benefit from the ability to delegate reads and writes.

However, Miklos has said in the past that the performance gain from this
is very small. You can get almost as good a result by splicing from one
fd to the other in userspace. In that case this function could actually
be implemented completely in libfuse.


Do you have any benchmark results that compare a splice-in-userspace
approach with your patch?


Best,
-Nikolaus

-- 
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

             »Time flies like an arrow, fruit flies like a Banana.«