Re: [LSF/MM/BPF TOPIC] VFS idmappings support in NFS

Alexander Mikhalitsyn
2026-02-18T14:36:20Z

Am Mi., 18. Feb. 2026 um 14:49 Uhr schrieb Jeff Layton <jlayton@kernel.org>:
>
> On Wed, 2026-02-18 at 13:44 +0100, Alexander Mikhalitsyn wrote:
> > Dear friends,
> >
> > I would like to propose "VFS idmappings support in NFS" as a topic for discussion at the LSF/MM/BPF Summit.
> >
> > Previously, I worked on VFS idmap support for FUSE/virtiofs [2] and cephfs [1] with support/guidance
> > from Christian.
> >
> > This experience with Cephfs & FUSE has shown that VFS idmap semantics, while being very elegant and
> > intuitive for local filesystems, can be quite challenging to combine with network/network-like (e.g. FUSE)
> > FSes. In case of Cephfs we had to modify its protocol (!) (see [2]) as a part of our agreement with
> > ceph folks about the right way to support idmaps.
> >
> > One obstacle here was that cephfs has some features that are not very Linux-wayish, I would say.
> > In particular, system administrator can configure path-based UID/GID restrictions on a *server*-side (Ceph MDS).
> > Basically, you can say "I expect UID 1000 and GID 2000 for all files under /stuff directory".
> > The problem here is that these UID/GIDs are taken from a syscall-caller's creds (not from (struct file *)->f_cred)
> > which makes cephfs FDs not very transferable through unix sockets. [3]
> >
> > These path-based UID/GID restrictions mean that server expects client to send UID/GID with every single request,
> > not only for those OPs where UID/GID needs to be written to the disk (mknod, mkdir, symlink, etc).
> > VFS idmaps API is designed to prevent filesystems developers from making a mistakes when supporting FS_ALLOW_IDMAP.
> > For example, (struct mnt_idmap *) is not passed to every single i_op, but instead to only those where it can be
> > used legitimately. Particularly, readlink/listxattr or rmdir are not expected to use idmapping information anyhow.
> >
> > We've seen very similar challenges with FUSE. Not a long time ago on Linux Containers project forum, there
> > was a discussion about mergerfs (a popular FUSE-based filesystem) & VFS idmaps [5]. And I see that this problem
> > of "caller UID/GID are needed everywhere" still blocks VFS idmaps adoption in some usecases.
> > Antonio Musumeci (mergerfs maintainer) claimed that in many cases filesystems behind mergerfs may not be fully
> > POSIX and basically, when mergerfs does IO on the underlying FSes it needs to do UID/GID switch to caller's UID/GID
> > (taken from FUSE request header).
> >
> > We don't expect NFS to be any simpler :-) I would say that supporting NFS is a final boss. It would be great
> > to have a deep technical discussion with VFS/FSes maintainers and developers about all these challenges and
> > make some conclusions and identify a right direction/approach to these problems. From my side, I'm going
> > to get more familiar with high-level part of NFS (or even make PoC if time permits), identify challenges,
> > summarize everything and prepare some slides to navigate/plan discussion.
> >
> > [1] cephfs https://lore.kernel.org/linux-fsdevel/20230807132626.182101-1-aleksandr.mikhalitsyn@canonical.com
> > [2] cephfs protocol changes https://github.com/ceph/ceph/pull/52575
> > [3] cephfs & f_cred https://lore.kernel.org/lkml/CAEivzxeZ6fDgYMnjk21qXYz13tHqZa8rP-cZ2jdxkY0eX+dOjw@mail.gmail.com/
> > [4] fuse/virtiofs https://lore.kernel.org/linux-fsdevel/20240903151626.264609-1-aleksandr.mikhalitsyn@canonical.com/
> > [5]
> > mergerfshttps://discuss.linuxcontainers.org/t/is-it-the-case-that-you-cannot-use-shift-true-for-disk-devices-where-the-source-is-a-mergerfs-mount-is-there-a-workaround/25336/11?u=amikhalitsyn
> >
> > Kind regards,
> > Alexander Mikhalitsyn @ futurfusion.io
>

Hi Jeff,

thanks for such a fast reply! ;)

>
> IIUC, people mostly use vfs-layer idmappings because they want to remap
> the uid/gid values of files that get stored on the backing store (disk,
> ceph MDS, or whatever).

yes, precisely.

>
> I've never used idmappings myself much in practice. Could you lay out
> an example of how you would use them with NFS in a real environment so
> I understand the problem better? I'd start by assuming a simple setup
> with AUTH_SYS and no NFSv4 idmapping involved, since that case should
> be fairly straightforward.

For me, from the point of LXC/Incus project, idmapped mounts are used as
a way to "delegate" filesystems (or subtrees) to the containers:
1. We, of course, assume that container enables user namespaces and
user can't mount a filesystem
inside because it has no FS_USERNS_MOUNT flag set (like in case of Cephfs, NFS,
CIFS and many others).
2. At the same time host's system administrator wants to avoid
remapping between container's user ns and
sb->s_user_ns (which is init_user_ns for those filesystems). [
motivation here is that in many
cases you may want to have the same subtree to be shared with other
containers and even host users too and
you want UIDs to be "compatible", i.e UID 1000 in one container and
UID 1000 in another container should
land as UID 1000 on the filesystem's inode ]

For this usecase, when we bind-mount filesystem to container, we apply
VFS idmap equal to container's
user namespace. This makes a behavior I described.

But this is just one use case. I'm pretty sure there are some more
around here :)
I know that folks from Preferred Networks (preferred.jp) are also
interested in VFS idmap support in NFS,
probably they can share some ideas/use cases too.

>
> Mixing in AUTH_GSS and real idmapping will be where things get harder,
> so let's not worry about those cases for now.
> --
> Jeff Layton <jlayton@kernel.org>

Kind regards,
Alex