[LSF/MM/BPF TOPIC] fuse passthrough solutions and status

[LSF/MM/BPF TOPIC] fuse passthrough solutions and status

[Miklos Szeredi]

Several fuse based filesystems pass file data from an underlying
filesystem without modification.  The added value can come from
changed directory structure, changed metadata or the ability to
intercept I/O only in special cases.  This pattern is very common, so
optimizing it could be very worthwhile.

I'd like to discuss proposed solutions to enabling data passthrough.
There are several prototypes:

 - fuse2[1] (myself, very old)
 - fuse-passthrough[2] (Alessio Balsini, more recent)
 - fuse-bpf[3] (Daniel Rosenberg, new)

The scope of fuse-bpf is much wider, but it does offer conditional
passthrough behavior as well.

One of the questions is how to reference underlying files.  Passing
open file descriptors directly in the fuse messages could be
dangerous[4].  Setting up the mapping from an open file descriptor to
the kernel using an ioctl() instead should be safe.

Other open issues:

 - what shall be the lifetime of the mapping?

 - does the mapped open file need to be visible to userspace?
Remember, this is a kernel module, so there's no process involved
where you could look at /proc/PID/fd.  Adding a kernel thread for each
fuse instance that installs these mapped fds as actual file descriptor
might be the solution.

Thanks,
Miklos


[1] https://lore.kernel.org/all/CAJfpegtjEoE7H8tayLaQHG9fRSBiVuaspnmPr2oQiOZXVB1+7g@mail.gmail.com/

[2] https://lore.kernel.org/all/20210125153057.3623715-1-balsini@android.com/

[3] https://lore.kernel.org/all/20221122021536.1629178-1-drosen@google.com/

[4] https://lore.kernel.org/all/CAG48ez17uXtjCTa7xpa=JWz3iBbNDQTKO2hvn6PAZtfW3kXgcA@mail.gmail.com/

Re: [LSF/MM/BPF TOPIC] fuse passthrough solutions and status

[Daniel Rosenberg]

On Fri, Feb 10, 2023 at 7:52 AM Miklos Szeredi <miklos@szeredi.hu> wrote:
>
> Several fuse based filesystems pass file data from an underlying
> filesystem without modification.  The added value can come from
> changed directory structure, changed metadata or the ability to
> intercept I/O only in special cases.  This pattern is very common, so
> optimizing it could be very worthwhile.
>
> I'd like to discuss proposed solutions to enabling data passthrough.
> There are several prototypes:
>
>  - fuse2[1] (myself, very old)
>  - fuse-passthrough[2] (Alessio Balsini, more recent)
>  - fuse-bpf[3] (Daniel Rosenberg, new)
>
> The scope of fuse-bpf is much wider, but it does offer conditional
> passthrough behavior as well.
>
> One of the questions is how to reference underlying files.  Passing
> open file descriptors directly in the fuse messages could be
> dangerous[4].  Setting up the mapping from an open file descriptor to
> the kernel using an ioctl() instead should be safe.
>
> Other open issues:
>
>  - what shall be the lifetime of the mapping?
>
>  - does the mapped open file need to be visible to userspace?
> Remember, this is a kernel module, so there's no process involved
> where you could look at /proc/PID/fd.  Adding a kernel thread for each
> fuse instance that installs these mapped fds as actual file descriptor
> might be the solution.
>
> Thanks,
> Miklos
>
>
> [1] https://lore.kernel.org/all/CAJfpegtjEoE7H8tayLaQHG9fRSBiVuaspnmPr2oQiOZXVB1+7g@mail.gmail.com/
>
> [2] https://lore.kernel.org/all/20210125153057.3623715-1-balsini@android.com/
>
> [3] https://lore.kernel.org/all/20221122021536.1629178-1-drosen@google.com/
>
> [4] https://lore.kernel.org/all/CAG48ez17uXtjCTa7xpa=JWz3iBbNDQTKO2hvn6PAZtfW3kXgcA@mail.gmail.com/

I'd be very interested to discuss fuse-bpf, and how it aligns with the
other efforts. I recall there being some io uring effort as well,
though I haven't looked into that too much yet.

Currently there are plenty of open areas for discussion with how the
bpf side of fuse-bpf interacts with the userspace daemon, particularly
if you create a node via bpf, and later want to handle some call for
it via the daemon.

I've been working on switching fuse-bpf over to using bpf's struct_ops
and kfuncs, along with dynptrs which open up a lot more opportunities
for adding capabilities on the bpf side. For the file descriptor
passing, I'm currently using an ioctl that acts as a mirror of the
/dev node. Normal fuse responses to lookup that add a backing file are
rejected unless they take the ioctl path.

-Daniel

Re: [Lsf-pc] [LSF/MM/BPF TOPIC] fuse passthrough solutions and status

[Amir Goldstein]

On Sat, Feb 25, 2023 at 2:59 AM Daniel Rosenberg via Lsf-pc
<lsf-pc@lists.linux-foundation.org> wrote:
>
> On Fri, Feb 10, 2023 at 7:52 AM Miklos Szeredi <miklos@szeredi.hu> wrote:
> >
> > Several fuse based filesystems pass file data from an underlying
> > filesystem without modification.  The added value can come from
> > changed directory structure, changed metadata or the ability to
> > intercept I/O only in special cases.  This pattern is very common, so
> > optimizing it could be very worthwhile.
> >
> > I'd like to discuss proposed solutions to enabling data passthrough.
> > There are several prototypes:
> >
> >  - fuse2[1] (myself, very old)
> >  - fuse-passthrough[2] (Alessio Balsini, more recent)
> >  - fuse-bpf[3] (Daniel Rosenberg, new)
> >
> > The scope of fuse-bpf is much wider, but it does offer conditional
> > passthrough behavior as well.
> >
> > One of the questions is how to reference underlying files.  Passing
> > open file descriptors directly in the fuse messages could be
> > dangerous[4].  Setting up the mapping from an open file descriptor to
> > the kernel using an ioctl() instead should be safe.
> >
> > Other open issues:
> >
> >  - what shall be the lifetime of the mapping?
> >
> >  - does the mapped open file need to be visible to userspace?
> > Remember, this is a kernel module, so there's no process involved
> > where you could look at /proc/PID/fd.  Adding a kernel thread for each
> > fuse instance that installs these mapped fds as actual file descriptor
> > might be the solution.
> >
> > Thanks,
> > Miklos
> >
> >
> > [1] https://lore.kernel.org/all/CAJfpegtjEoE7H8tayLaQHG9fRSBiVuaspnmPr2oQiOZXVB1+7g@mail.gmail.com/
> >
> > [2] https://lore.kernel.org/all/20210125153057.3623715-1-balsini@android.com/
> >
> > [3] https://lore.kernel.org/all/20221122021536.1629178-1-drosen@google.com/
> >
> > [4] https://lore.kernel.org/all/CAG48ez17uXtjCTa7xpa=JWz3iBbNDQTKO2hvn6PAZtfW3kXgcA@mail.gmail.com/
>
> I'd be very interested to discuss fuse-bpf, and how it aligns with the
> other efforts. I recall there being some io uring effort as well,
> though I haven't looked into that too much yet.
>

For the record, I scheduled a (30 min) session for this with you and Miklos
co-leads.

Bernd's session on FUSE uring will follow right after.

Thanks,
Amir.