Re: [LSF/MM/BPF TOPIC] VFS idmappings support in NFS

[Jeff Layton <jlayton@kernel.org>]

On Wed, 2026-02-18 at 09:37 -0500, Trond Myklebust wrote:
> On Wed, 2026-02-18 at 08:49 -0500, Jeff Layton wrote:
> > On Wed, 2026-02-18 at 13:44 +0100, Alexander Mikhalitsyn wrote:
> > > Dear friends,
> > > 
> > > I would like to propose "VFS idmappings support in NFS" as a topic
> > > for discussion at the LSF/MM/BPF Summit.
> > > 
> > > Previously, I worked on VFS idmap support for FUSE/virtiofs [2] and
> > > cephfs [1] with support/guidance
> > > from Christian.
> > > 
> > > This experience with Cephfs & FUSE has shown that VFS idmap
> > > semantics, while being very elegant and
> > > intuitive for local filesystems, can be quite challenging to
> > > combine with network/network-like (e.g. FUSE)
> > > FSes. In case of Cephfs we had to modify its protocol (!) (see [2])
> > > as a part of our agreement with
> > > ceph folks about the right way to support idmaps.
> > > 
> > > One obstacle here was that cephfs has some features that are not
> > > very Linux-wayish, I would say.
> > > In particular, system administrator can configure path-based
> > > UID/GID restrictions on a *server*-side (Ceph MDS).
> > > Basically, you can say "I expect UID 1000 and GID 2000 for all
> > > files under /stuff directory".
> > > The problem here is that these UID/GIDs are taken from a syscall-
> > > caller's creds (not from (struct file *)->f_cred)
> > > which makes cephfs FDs not very transferable through unix sockets.
> > > [3]
> > > 
> > > These path-based UID/GID restrictions mean that server expects
> > > client to send UID/GID with every single request,
> > > not only for those OPs where UID/GID needs to be written to the
> > > disk (mknod, mkdir, symlink, etc).
> > > VFS idmaps API is designed to prevent filesystems developers from
> > > making a mistakes when supporting FS_ALLOW_IDMAP.
> > > For example, (struct mnt_idmap *) is not passed to every single
> > > i_op, but instead to only those where it can be
> > > used legitimately. Particularly, readlink/listxattr or rmdir are
> > > not expected to use idmapping information anyhow.
> > > 
> > > We've seen very similar challenges with FUSE. Not a long time ago
> > > on Linux Containers project forum, there
> > > was a discussion about mergerfs (a popular FUSE-based filesystem) &
> > > VFS idmaps [5]. And I see that this problem
> > > of "caller UID/GID are needed everywhere" still blocks VFS idmaps
> > > adoption in some usecases.
> > > Antonio Musumeci (mergerfs maintainer) claimed that in many cases
> > > filesystems behind mergerfs may not be fully
> > > POSIX and basically, when mergerfs does IO on the underlying FSes
> > > it needs to do UID/GID switch to caller's UID/GID
> > > (taken from FUSE request header).
> > > 
> > > We don't expect NFS to be any simpler :-) I would say that
> > > supporting NFS is a final boss. It would be great
> > > to have a deep technical discussion with VFS/FSes maintainers and
> > > developers about all these challenges and
> > > make some conclusions and identify a right direction/approach to
> > > these problems. From my side, I'm going
> > > to get more familiar with high-level part of NFS (or even make PoC
> > > if time permits), identify challenges,
> > > summarize everything and prepare some slides to navigate/plan
> > > discussion.
> > > 
> > > [1] cephfs
> > > https://lore.kernel.org/linux-fsdevel/20230807132626.182101-1-aleksandr.mikhalitsyn@canonical.com
> > > [2] cephfs protocol changes https://github.com/ceph/ceph/pull/52575
> > > [3] cephfs & f_cred
> > > https://lore.kernel.org/lkml/CAEivzxeZ6fDgYMnjk21qXYz13tHqZa8rP-cZ2jdxkY0eX+dOjw@mail.gmail.com/
> > > [4] fuse/virtiofs
> > > https://lore.kernel.org/linux-fsdevel/20240903151626.264609-1-aleksandr.mikhalitsyn@canonical.com/
> > > [5]
> > > mergerfs
> > > https://discuss.linuxcontainers.org/t/is-it-the-case-that-you-
> > > cannot-use-shift-true-for-disk-devices-where-the-source-is-a-
> > > mergerfs-mount-is-there-a-workaround/25336/11?u=amikhalitsyn
> > > 
> > > Kind regards,
> > > Alexander Mikhalitsyn @ futurfusion.io
> > 
> > 
> > IIUC, people mostly use vfs-layer idmappings because they want to
> > remap
> > the uid/gid values of files that get stored on the backing store
> > (disk,
> > ceph MDS, or whatever).
> > 
> > I've never used idmappings myself much in practice. Could you lay out
> > an example of how you would use them with NFS in a real environment
> > so
> > I understand the problem better? I'd start by assuming a simple setup
> > with AUTH_SYS and no NFSv4 idmapping involved, since that case should
> > be fairly straightforward.
> > 
> > Mixing in AUTH_GSS and real idmapping will be where things get
> > harder,
> > so let's not worry about those cases for now.
> 
> I think you do need to worry about those cases. As the NFS and RPC
> protocols stand today, strong authentication will defeat any client
> side idmapping scheme, because the server can't know what uids or gids
> the client is using on its end; it just knows about the account that
> was used to authenticate.
> 

Oh, we absolutely need to worry about them, but this is a difficult
topic to get our arms around. We can potentially have several layers
that are doing idmapping, so I want to understand a simple use-case
first. Once that's clear I plan to start throwing in monkey wrenches.

> I think if you do want to implement something generic, you're going to
> have to consider how the client and server can exchange (and store) the
> information needed to allow the client to perform the mapping of file
> owners/group owners on its end. The client would presumably also need
> to be in charge of enforcing permissions for such mappings.
> It would be a very different security model than the one used by NFS
> today, and almost certainly require protocol extensions.

That may be, but I still don't fully understand the use-case here.
Maybe they'd be content with just shifting UIDs at a higher level
without changing the protocol? Without understanding how they intend to
use this, it's hard to know what's needed.

-- 
Jeff Layton <jlayton@kernel.org>