Re: [LSF/MM/BPF TOPIC] VFS idmappings support in NFS

Christian Brauner
2026-02-24T08:54:37Z

On Sat, Feb 21, 2026 at 01:44:26AM -0500, Demi Marie Obenour wrote:
> On 2/18/26 07:44, Alexander Mikhalitsyn wrote:
> > Dear friends,
> >
> > I would like to propose "VFS idmappings support in NFS" as a topic for discussion at the LSF/MM/BPF Summit.
> >
> > Previously, I worked on VFS idmap support for FUSE/virtiofs [2] and cephfs [1] with support/guidance
> > from Christian.
> >
> > This experience with Cephfs & FUSE has shown that VFS idmap semantics, while being very elegant and
> > intuitive for local filesystems, can be quite challenging to combine with network/network-like (e.g. FUSE)
> > FSes. In case of Cephfs we had to modify its protocol (!) (see [2]) as a part of our agreement with
> > ceph folks about the right way to support idmaps.
> >
> > One obstacle here was that cephfs has some features that are not very Linux-wayish, I would say.
> > In particular, system administrator can configure path-based UID/GID restrictions on a *server*-side (Ceph MDS).
> > Basically, you can say "I expect UID 1000 and GID 2000 for all files under /stuff directory".
> > The problem here is that these UID/GIDs are taken from a syscall-caller's creds (not from (struct file *)->f_cred)
> > which makes cephfs FDs not very transferable through unix sockets. [3]
> >
> > These path-based UID/GID restrictions mean that server expects client to send UID/GID with every single request,
> > not only for those OPs where UID/GID needs to be written to the disk (mknod, mkdir, symlink, etc).
> > VFS idmaps API is designed to prevent filesystems developers from making a mistakes when supporting FS_ALLOW_IDMAP.
> > For example, (struct mnt_idmap *) is not passed to every single i_op, but instead to only those where it can be
> > used legitimately. Particularly, readlink/listxattr or rmdir are not expected to use idmapping information anyhow.
> >
> > We've seen very similar challenges with FUSE. Not a long time ago on Linux Containers project forum, there
> > was a discussion about mergerfs (a popular FUSE-based filesystem) & VFS idmaps [5]. And I see that this problem
> > of "caller UID/GID are needed everywhere" still blocks VFS idmaps adoption in some usecases.
> > Antonio Musumeci (mergerfs maintainer) claimed that in many cases filesystems behind mergerfs may not be fully
> > POSIX and basically, when mergerfs does IO on the underlying FSes it needs to do UID/GID switch to caller's UID/GID
> > (taken from FUSE request header).
> >
> > We don't expect NFS to be any simpler :-) I would say that supporting NFS is a final boss. It would be great
> > to have a deep technical discussion with VFS/FSes maintainers and developers about all these challenges and
> > make some conclusions and identify a right direction/approach to these problems. From my side, I'm going
> > to get more familiar with high-level part of NFS (or even make PoC if time permits), identify challenges,
> > summarize everything and prepare some slides to navigate/plan discussion.
> >
> > [1] cephfs https://lore.kernel.org/linux-fsdevel/20230807132626.182101-1-aleksandr.mikhalitsyn@canonical.com
> > [2] cephfs protocol changes https://github.com/ceph/ceph/pull/52575
> > [3] cephfs & f_cred https://lore.kernel.org/lkml/CAEivzxeZ6fDgYMnjk21qXYz13tHqZa8rP-cZ2jdxkY0eX+dOjw@mail.gmail.com/
> > [4] fuse/virtiofs https://lore.kernel.org/linux-fsdevel/20240903151626.264609-1-aleksandr.mikhalitsyn@canonical.com/
> > [5]
> > mergerfshttps://discuss.linuxcontainers.org/t/is-it-the-case-that-you-cannot-use-shift-true-for-disk-devices-where-the-source-is-a-mergerfs-mount-is-there-a-workaround/25336/11?u=amikhalitsyn
> >
> > Kind regards,
> > Alexander Mikhalitsyn @ futurfusion.io
>
> The secure case (strong authentication) has similar problems to
> In both cases, there is no way to store the files with the UID/GID/etc

It's easy to support idmapped mounts without user namespaces. They're
completely decoupled from them already for that purpose so they can
support id squashing and so on going forward. The only thing that's
needed is to extend the api so that we can specific mappings to be used
for a mount. That's not difficult and there's no need to adhere to any
inherent limit on the number of mappings that user namespaces have.

It's also useful indepent of all that for local filesystems that want to
expose files with different ownership at different locations without
getting into namespaces at all.

> that the VFS says they should have. The server (NFS) or kernel
> (virtiofsd) simply will not (and, for security reasons, *must not*)
> allow this.
>
> I proposed a workaround for virtiofsd [1] that I will also propose
> here: store the mapped UID and GID as a user.* xattr. This requires

xattrs as an ownership side-channel are an absolute clusterfuck. The
kernel implementation for POSIX ACLs and filesystem capabilities that
slap ownership information that the VFS must consume on arbitraries
inodes should be set on fire. I've burned way too many cycles getting
this into an even remotely acceptable shape and it still sucks to no
end. Permission checking is a completely nightmare because we need to go
fetch stuff from disk, cache it in a global format, then do an in-place
translation having to parse ownership out of binary data stored
alongside the inode.

Nowever, if userspace wants to consume ownership information by storing
arbitrary ownership information as user.* xattrs then I obviously
couldn't care less but it won't nest, performance will suck, and it will
be brittle to get this right imho.

> no special permissions, and so it completely solves this problem.
> It is also the only solution I know of that scales to NFS servers
> with over 2^16 users, which might well exist.
>
> The only better solution I can think of is to replace the numeric
> UID/GID with hierarchical identifier, such as a Windows-style SID.
> Those are much more complex, though.
>
> [1]: https://gitlab.com/virtio-fs/virtiofsd/-/issues/225
> --
> Sincerely,
> Demi Marie Obenour (she/her/hers)