Re: [LSF/MM/BPF TOPIC] VFS idmappings support in NFS

[Jeff Layton <jlayton@kernel.org>]

On Wed, 2026-02-18 at 13:44 +0100, Alexander Mikhalitsyn wrote:
> Dear friends,
> 
> I would like to propose "VFS idmappings support in NFS" as a topic for discussion at the LSF/MM/BPF Summit.
> 
> Previously, I worked on VFS idmap support for FUSE/virtiofs [2] and cephfs [1] with support/guidance
> from Christian.
> 
> This experience with Cephfs & FUSE has shown that VFS idmap semantics, while being very elegant and
> intuitive for local filesystems, can be quite challenging to combine with network/network-like (e.g. FUSE)
> FSes. In case of Cephfs we had to modify its protocol (!) (see [2]) as a part of our agreement with
> ceph folks about the right way to support idmaps.
> 
> One obstacle here was that cephfs has some features that are not very Linux-wayish, I would say.
> In particular, system administrator can configure path-based UID/GID restrictions on a *server*-side (Ceph MDS).
> Basically, you can say "I expect UID 1000 and GID 2000 for all files under /stuff directory".
> The problem here is that these UID/GIDs are taken from a syscall-caller's creds (not from (struct file *)->f_cred)
> which makes cephfs FDs not very transferable through unix sockets. [3]
> 
> These path-based UID/GID restrictions mean that server expects client to send UID/GID with every single request,
> not only for those OPs where UID/GID needs to be written to the disk (mknod, mkdir, symlink, etc).
> VFS idmaps API is designed to prevent filesystems developers from making a mistakes when supporting FS_ALLOW_IDMAP.
> For example, (struct mnt_idmap *) is not passed to every single i_op, but instead to only those where it can be
> used legitimately. Particularly, readlink/listxattr or rmdir are not expected to use idmapping information anyhow.
> 
> We've seen very similar challenges with FUSE. Not a long time ago on Linux Containers project forum, there
> was a discussion about mergerfs (a popular FUSE-based filesystem) & VFS idmaps [5]. And I see that this problem
> of "caller UID/GID are needed everywhere" still blocks VFS idmaps adoption in some usecases.
> Antonio Musumeci (mergerfs maintainer) claimed that in many cases filesystems behind mergerfs may not be fully
> POSIX and basically, when mergerfs does IO on the underlying FSes it needs to do UID/GID switch to caller's UID/GID
> (taken from FUSE request header).
> 
> We don't expect NFS to be any simpler :-) I would say that supporting NFS is a final boss. It would be great
> to have a deep technical discussion with VFS/FSes maintainers and developers about all these challenges and
> make some conclusions and identify a right direction/approach to these problems. From my side, I'm going
> to get more familiar with high-level part of NFS (or even make PoC if time permits), identify challenges,
> summarize everything and prepare some slides to navigate/plan discussion.
> 
> [1] cephfs https://lore.kernel.org/linux-fsdevel/20230807132626.182101-1-aleksandr.mikhalitsyn@canonical.com
> [2] cephfs protocol changes https://github.com/ceph/ceph/pull/52575
> [3] cephfs & f_cred https://lore.kernel.org/lkml/CAEivzxeZ6fDgYMnjk21qXYz13tHqZa8rP-cZ2jdxkY0eX+dOjw@mail.gmail.com/
> [4] fuse/virtiofs https://lore.kernel.org/linux-fsdevel/20240903151626.264609-1-aleksandr.mikhalitsyn@canonical.com/
> [5]
> mergerfshttps://discuss.linuxcontainers.org/t/is-it-the-case-that-you-cannot-use-shift-true-for-disk-devices-where-the-source-is-a-mergerfs-mount-is-there-a-workaround/25336/11?u=amikhalitsyn
> 
> Kind regards,
> Alexander Mikhalitsyn @ futurfusion.io


IIUC, people mostly use vfs-layer idmappings because they want to remap
the uid/gid values of files that get stored on the backing store (disk,
ceph MDS, or whatever).

I've never used idmappings myself much in practice. Could you lay out
an example of how you would use them with NFS in a real environment so
I understand the problem better? I'd start by assuming a simple setup
with AUTH_SYS and no NFSv4 idmapping involved, since that case should
be fairly straightforward.

Mixing in AUTH_GSS and real idmapping will be where things get harder,
so let's not worry about those cases for now.
-- 
Jeff Layton <jlayton@kernel.org>