Re: [LSF/MM/BPF TOPIC] VFS idmappings support in NFS

Alexander Mikhalitsyn
2026-02-18T16:40:04Z

Am Mi., 18. Feb. 2026 um 17:01 Uhr schrieb Jeff Layton <jlayton@kernel.org>:
>
> On Wed, 2026-02-18 at 15:36 +0100, Alexander Mikhalitsyn wrote:
> > Am Mi., 18. Feb. 2026 um 14:49 Uhr schrieb Jeff Layton <jlayton@kernel.org>:
> > >
> > > On Wed, 2026-02-18 at 13:44 +0100, Alexander Mikhalitsyn wrote:
> > > > Dear friends,
> > > >
> > > > I would like to propose "VFS idmappings support in NFS" as a topic for discussion at the LSF/MM/BPF Summit.
> > > >
> > > > Previously, I worked on VFS idmap support for FUSE/virtiofs [2] and cephfs [1] with support/guidance
> > > > from Christian.
> > > >
> > > > This experience with Cephfs & FUSE has shown that VFS idmap semantics, while being very elegant and
> > > > intuitive for local filesystems, can be quite challenging to combine with network/network-like (e.g. FUSE)
> > > > FSes. In case of Cephfs we had to modify its protocol (!) (see [2]) as a part of our agreement with
> > > > ceph folks about the right way to support idmaps.
> > > >
> > > > One obstacle here was that cephfs has some features that are not very Linux-wayish, I would say.
> > > > In particular, system administrator can configure path-based UID/GID restrictions on a *server*-side (Ceph MDS).
> > > > Basically, you can say "I expect UID 1000 and GID 2000 for all files under /stuff directory".
> > > > The problem here is that these UID/GIDs are taken from a syscall-caller's creds (not from (struct file *)->f_cred)
> > > > which makes cephfs FDs not very transferable through unix sockets. [3]
> > > >
> > > > These path-based UID/GID restrictions mean that server expects client to send UID/GID with every single request,
> > > > not only for those OPs where UID/GID needs to be written to the disk (mknod, mkdir, symlink, etc).
> > > > VFS idmaps API is designed to prevent filesystems developers from making a mistakes when supporting FS_ALLOW_IDMAP.
> > > > For example, (struct mnt_idmap *) is not passed to every single i_op, but instead to only those where it can be
> > > > used legitimately. Particularly, readlink/listxattr or rmdir are not expected to use idmapping information anyhow.
> > > >
> > > > We've seen very similar challenges with FUSE. Not a long time ago on Linux Containers project forum, there
> > > > was a discussion about mergerfs (a popular FUSE-based filesystem) & VFS idmaps [5]. And I see that this problem
> > > > of "caller UID/GID are needed everywhere" still blocks VFS idmaps adoption in some usecases.
> > > > Antonio Musumeci (mergerfs maintainer) claimed that in many cases filesystems behind mergerfs may not be fully
> > > > POSIX and basically, when mergerfs does IO on the underlying FSes it needs to do UID/GID switch to caller's UID/GID
> > > > (taken from FUSE request header).
> > > >
> > > > We don't expect NFS to be any simpler :-) I would say that supporting NFS is a final boss. It would be great
> > > > to have a deep technical discussion with VFS/FSes maintainers and developers about all these challenges and
> > > > make some conclusions and identify a right direction/approach to these problems. From my side, I'm going
> > > > to get more familiar with high-level part of NFS (or even make PoC if time permits), identify challenges,
> > > > summarize everything and prepare some slides to navigate/plan discussion.
> > > >
> > > > [1] cephfs https://lore.kernel.org/linux-fsdevel/20230807132626.182101-1-aleksandr.mikhalitsyn@canonical.com
> > > > [2] cephfs protocol changes https://github.com/ceph/ceph/pull/52575
> > > > [3] cephfs & f_cred https://lore.kernel.org/lkml/CAEivzxeZ6fDgYMnjk21qXYz13tHqZa8rP-cZ2jdxkY0eX+dOjw@mail.gmail.com/
> > > > [4] fuse/virtiofs https://lore.kernel.org/linux-fsdevel/20240903151626.264609-1-aleksandr.mikhalitsyn@canonical.com/
> > > > [5]
> > > > mergerfshttps://discuss.linuxcontainers.org/t/is-it-the-case-that-you-cannot-use-shift-true-for-disk-devices-where-the-source-is-a-mergerfs-mount-is-there-a-workaround/25336/11?u=amikhalitsyn
> > > >
> > > > Kind regards,
> > > > Alexander Mikhalitsyn @ futurfusion.io
> > >
> >
> > Hi Jeff,
> >
> > thanks for such a fast reply! ;)
> >
> > >
> > > IIUC, people mostly use vfs-layer idmappings because they want to remap
> > > the uid/gid values of files that get stored on the backing store (disk,
> > > ceph MDS, or whatever).
> >
> > yes, precisely.
> >
> > >
> > > I've never used idmappings myself much in practice. Could you lay out
> > > an example of how you would use them with NFS in a real environment so
> > > I understand the problem better? I'd start by assuming a simple setup
> > > with AUTH_SYS and no NFSv4 idmapping involved, since that case should
> > > be fairly straightforward.
> >
> > For me, from the point of LXC/Incus project, idmapped mounts are used as
> > a way to "delegate" filesystems (or subtrees) to the containers:
> > 1. We, of course, assume that container enables user namespaces and
> > user can't mount a filesystem
> > inside because it has no FS_USERNS_MOUNT flag set (like in case of Cephfs, NFS,
> > CIFS and many others).
> > 2. At the same time host's system administrator wants to avoid
> > remapping between container's user ns and
> > sb->s_user_ns (which is init_user_ns for those filesystems). [
> > motivation here is that in many
> > cases you may want to have the same subtree to be shared with other
> > containers and even host users too and
> > you want UIDs to be "compatible", i.e UID 1000 in one container and
> > UID 1000 in another container should
> > land as UID 1000 on the filesystem's inode ]
> >
> > For this usecase, when we bind-mount filesystem to container, we apply
> > VFS idmap equal to container's
> > user namespace. This makes a behavior I described.
> >
>
> Ok: so you have a process running in a userns as UID 2000 and you want
> to use vfs layer idmapping so that when you create a file as that user
> that it ends up being owned by UID 1000. Is that basically correct?

In our case, we have a UID 1000 (inside user namespace), which mapped to
something like 10000 + 1000 (in the init_user_ns). And then we have
NFS mount (sb->s_user_ns = init_user_ns, ofc), so if user UID 1000
(inside the container)
creates a file, it will be 11000, right? But we do bind-mount of that
NFS mount+VFS idmap,
so that once file is created it has owner_uid = 1000. (This scenario
is covered by [1] and [2])

[1] https://docs.kernel.org/filesystems/idmappings.html#example-3
[2] https://docs.kernel.org/filesystems/idmappings.html#example-3-reconsidered

>
> Typically, the RPC credentials used in an OPEN or CREATE call is what
> determines its ownership (at least until a SETATTR comes in). With
> AUTH_SYS, the credential is just a uid and set of gids.
>
> So in this case, it sounds like you would need just do that conversion
> (maybe at the RPC client layer?) when issuing an RPC. You don't really
> need a protocol extension for that case.
>
> As Trond points out though, AUTH_GSS and NFSv4 idmapping will make this
> more complex. Once you're using kerberos credentials for
> authentication, you don't have much control over what the UIDs and GIDs
> will be on newly-created files, but is that really a problem? As long
> as all of the clients have a consistent view, I wouldn't think so.

I absolutely agree.

>
> > But this is just one use case. I'm pretty sure there are some more
> > around here :)
> > I know that folks from Preferred Networks (preferred.jp) are also
> > interested in VFS idmap support in NFS,
> > probably they can share some ideas/use cases too.
> >
> >
>
> Yes, we don't want to focus too much on a single use-case, but I find
> it helpful to focus on a single simple problem first.

Yes, I could prepare RFC patches before LSF/MM/BPF for that simple case so
we have something to start with.

> --
> Jeff Layton <jlayton@kernel.org>