Re: [LSF/MM/BPF TOPIC] VFS idmappings support in NFS

Alexander Mikhalitsyn
2026-02-18T15:25:39Z

Am Mi., 18. Feb. 2026 um 16:08 Uhr schrieb Jeff Layton <jlayton@kernel.org>:
>
> On Wed, 2026-02-18 at 09:37 -0500, Trond Myklebust wrote:
> > On Wed, 2026-02-18 at 08:49 -0500, Jeff Layton wrote:
> > > On Wed, 2026-02-18 at 13:44 +0100, Alexander Mikhalitsyn wrote:
> > > > Dear friends,
> > > >
> > > > I would like to propose "VFS idmappings support in NFS" as a topic
> > > > for discussion at the LSF/MM/BPF Summit.
> > > >
> > > > Previously, I worked on VFS idmap support for FUSE/virtiofs [2] and
> > > > cephfs [1] with support/guidance
> > > > from Christian.
> > > >
> > > > This experience with Cephfs & FUSE has shown that VFS idmap
> > > > semantics, while being very elegant and
> > > > intuitive for local filesystems, can be quite challenging to
> > > > combine with network/network-like (e.g. FUSE)
> > > > FSes. In case of Cephfs we had to modify its protocol (!) (see [2])
> > > > as a part of our agreement with
> > > > ceph folks about the right way to support idmaps.
> > > >
> > > > One obstacle here was that cephfs has some features that are not
> > > > very Linux-wayish, I would say.
> > > > In particular, system administrator can configure path-based
> > > > UID/GID restrictions on a *server*-side (Ceph MDS).
> > > > Basically, you can say "I expect UID 1000 and GID 2000 for all
> > > > files under /stuff directory".
> > > > The problem here is that these UID/GIDs are taken from a syscall-
> > > > caller's creds (not from (struct file *)->f_cred)
> > > > which makes cephfs FDs not very transferable through unix sockets.
> > > > [3]
> > > >
> > > > These path-based UID/GID restrictions mean that server expects
> > > > client to send UID/GID with every single request,
> > > > not only for those OPs where UID/GID needs to be written to the
> > > > disk (mknod, mkdir, symlink, etc).
> > > > VFS idmaps API is designed to prevent filesystems developers from
> > > > making a mistakes when supporting FS_ALLOW_IDMAP.
> > > > For example, (struct mnt_idmap *) is not passed to every single
> > > > i_op, but instead to only those where it can be
> > > > used legitimately. Particularly, readlink/listxattr or rmdir are
> > > > not expected to use idmapping information anyhow.
> > > >
> > > > We've seen very similar challenges with FUSE. Not a long time ago
> > > > on Linux Containers project forum, there
> > > > was a discussion about mergerfs (a popular FUSE-based filesystem) &
> > > > VFS idmaps [5]. And I see that this problem
> > > > of "caller UID/GID are needed everywhere" still blocks VFS idmaps
> > > > adoption in some usecases.
> > > > Antonio Musumeci (mergerfs maintainer) claimed that in many cases
> > > > filesystems behind mergerfs may not be fully
> > > > POSIX and basically, when mergerfs does IO on the underlying FSes
> > > > it needs to do UID/GID switch to caller's UID/GID
> > > > (taken from FUSE request header).
> > > >
> > > > We don't expect NFS to be any simpler :-) I would say that
> > > > supporting NFS is a final boss. It would be great
> > > > to have a deep technical discussion with VFS/FSes maintainers and
> > > > developers about all these challenges and
> > > > make some conclusions and identify a right direction/approach to
> > > > these problems. From my side, I'm going
> > > > to get more familiar with high-level part of NFS (or even make PoC
> > > > if time permits), identify challenges,
> > > > summarize everything and prepare some slides to navigate/plan
> > > > discussion.
> > > >
> > > > [1] cephfs
> > > > https://lore.kernel.org/linux-fsdevel/20230807132626.182101-1-aleksandr.mikhalitsyn@canonical.com
> > > > [2] cephfs protocol changes https://github.com/ceph/ceph/pull/52575
> > > > [3] cephfs & f_cred
> > > > https://lore.kernel.org/lkml/CAEivzxeZ6fDgYMnjk21qXYz13tHqZa8rP-cZ2jdxkY0eX+dOjw@mail.gmail.com/
> > > > [4] fuse/virtiofs
> > > > https://lore.kernel.org/linux-fsdevel/20240903151626.264609-1-aleksandr.mikhalitsyn@canonical.com/
> > > > [5]
> > > > mergerfs
> > > > https://discuss.linuxcontainers.org/t/is-it-the-case-that-you-
> > > > cannot-use-shift-true-for-disk-devices-where-the-source-is-a-
> > > > mergerfs-mount-is-there-a-workaround/25336/11?u=amikhalitsyn
> > > >
> > > > Kind regards,
> > > > Alexander Mikhalitsyn @ futurfusion.io
> > >
> > >
> > > IIUC, people mostly use vfs-layer idmappings because they want to
> > > remap
> > > the uid/gid values of files that get stored on the backing store
> > > (disk,
> > > ceph MDS, or whatever).
> > >
> > > I've never used idmappings myself much in practice. Could you lay out
> > > an example of how you would use them with NFS in a real environment
> > > so
> > > I understand the problem better? I'd start by assuming a simple setup
> > > with AUTH_SYS and no NFSv4 idmapping involved, since that case should
> > > be fairly straightforward.
> > >
> > > Mixing in AUTH_GSS and real idmapping will be where things get
> > > harder,
> > > so let's not worry about those cases for now.
> >
> > I think you do need to worry about those cases. As the NFS and RPC
> > protocols stand today, strong authentication will defeat any client
> > side idmapping scheme, because the server can't know what uids or gids
> > the client is using on its end; it just knows about the account that
> > was used to authenticate.
> >
>
> Oh, we absolutely need to worry about them, but this is a difficult
> topic to get our arms around. We can potentially have several layers
> that are doing idmapping, so I want to understand a simple use-case
> first. Once that's clear I plan to start throwing in monkey wrenches.
>
> > I think if you do want to implement something generic, you're going to
> > have to consider how the client and server can exchange (and store) the
> > information needed to allow the client to perform the mapping of file
> > owners/group owners on its end. The client would presumably also need
> > to be in charge of enforcing permissions for such mappings.
> > It would be a very different security model than the one used by NFS
> > today, and almost certainly require protocol extensions.
>
> That may be, but I still don't fully understand the use-case here.

Please, let me know if my earlier reply doesn't clarify LXC/Incus use case.
I can prepare a more detailed explanation with command line/configuration
examples with pleasure.

> Maybe they'd be content with just shifting UIDs at a higher level
> without changing the protocol? Without understanding how they intend to
> use this, it's hard to know what's needed.

If you ask me, I have no problem or I would say more, I look positively
on the way "keep it high level & don't touch NFS protocol" ;-)
But I remember a very tight discussion (good context [1]) about Cephfs and
this way wasn't considered as acceptable back then (and we had to make
a protocol extension).
We can always go iteratively, and first version can be simple and then on-demand
we can support more tricky cases if this is acceptable for you guys.
You set the rules. ;-)

[1] https://lore.kernel.org/lkml/f3864ed6-8c97-8a7a-f268-dab29eb2fb21@redhat.com/

Kind regards,
Alex

>
> --
> Jeff Layton <jlayton@kernel.org>