Re: [LSF/MM/BPF TOPIC] VFS idmappings support in NFS

[Demi Marie Obenour <demiobenour@gmail.com>]

[-- Attachment #1.1.1: Type: text/plain, Size: 4530 bytes --]

On 2/18/26 07:44, Alexander Mikhalitsyn wrote:
> Dear friends,
> 
> I would like to propose "VFS idmappings support in NFS" as a topic for discussion at the LSF/MM/BPF Summit.
> 
> Previously, I worked on VFS idmap support for FUSE/virtiofs [2] and cephfs [1] with support/guidance
> from Christian.
> 
> This experience with Cephfs & FUSE has shown that VFS idmap semantics, while being very elegant and
> intuitive for local filesystems, can be quite challenging to combine with network/network-like (e.g. FUSE)
> FSes. In case of Cephfs we had to modify its protocol (!) (see [2]) as a part of our agreement with
> ceph folks about the right way to support idmaps.
> 
> One obstacle here was that cephfs has some features that are not very Linux-wayish, I would say.
> In particular, system administrator can configure path-based UID/GID restrictions on a *server*-side (Ceph MDS).
> Basically, you can say "I expect UID 1000 and GID 2000 for all files under /stuff directory".
> The problem here is that these UID/GIDs are taken from a syscall-caller's creds (not from (struct file *)->f_cred)
> which makes cephfs FDs not very transferable through unix sockets. [3]
> 
> These path-based UID/GID restrictions mean that server expects client to send UID/GID with every single request,
> not only for those OPs where UID/GID needs to be written to the disk (mknod, mkdir, symlink, etc).
> VFS idmaps API is designed to prevent filesystems developers from making a mistakes when supporting FS_ALLOW_IDMAP.
> For example, (struct mnt_idmap *) is not passed to every single i_op, but instead to only those where it can be
> used legitimately. Particularly, readlink/listxattr or rmdir are not expected to use idmapping information anyhow.
> 
> We've seen very similar challenges with FUSE. Not a long time ago on Linux Containers project forum, there
> was a discussion about mergerfs (a popular FUSE-based filesystem) & VFS idmaps [5]. And I see that this problem
> of "caller UID/GID are needed everywhere" still blocks VFS idmaps adoption in some usecases.
> Antonio Musumeci (mergerfs maintainer) claimed that in many cases filesystems behind mergerfs may not be fully
> POSIX and basically, when mergerfs does IO on the underlying FSes it needs to do UID/GID switch to caller's UID/GID
> (taken from FUSE request header).
> 
> We don't expect NFS to be any simpler :-) I would say that supporting NFS is a final boss. It would be great
> to have a deep technical discussion with VFS/FSes maintainers and developers about all these challenges and
> make some conclusions and identify a right direction/approach to these problems. From my side, I'm going
> to get more familiar with high-level part of NFS (or even make PoC if time permits), identify challenges,
> summarize everything and prepare some slides to navigate/plan discussion.
> 
> [1] cephfs https://lore.kernel.org/linux-fsdevel/20230807132626.182101-1-aleksandr.mikhalitsyn@canonical.com
> [2] cephfs protocol changes https://github.com/ceph/ceph/pull/52575
> [3] cephfs & f_cred https://lore.kernel.org/lkml/CAEivzxeZ6fDgYMnjk21qXYz13tHqZa8rP-cZ2jdxkY0eX+dOjw@mail.gmail.com/
> [4] fuse/virtiofs https://lore.kernel.org/linux-fsdevel/20240903151626.264609-1-aleksandr.mikhalitsyn@canonical.com/
> [5]
> mergerfshttps://discuss.linuxcontainers.org/t/is-it-the-case-that-you-cannot-use-shift-true-for-disk-devices-where-the-source-is-a-mergerfs-mount-is-there-a-workaround/25336/11?u=amikhalitsyn
> 
> Kind regards,
> Alexander Mikhalitsyn @ futurfusion.io

The secure case (strong authentication) has similar problems to
unprivileged virtiofsd on a system with user namespaces disabled.
In both cases, there is no way to store the files with the UID/GID/etc
that the VFS says they should have.  The server (NFS) or kernel
(virtiofsd) simply will not (and, for security reasons, *must not*)
allow this.

I proposed a workaround for virtiofsd [1] that I will also propose
here: store the mapped UID and GID as a user.* xattr.  This requires
no special permissions, and so it completely solves this problem.
It is also the only solution I know of that scales to NFS servers
with over 2^16 users, which might well exist.

The only better solution I can think of is to replace the numeric
UID/GID with hierarchical identifier, such as a Windows-style SID.
Those are much more complex, though.

[1]: https://gitlab.com/virtio-fs/virtiofsd/-/issues/225
-- 
Sincerely,
Demi Marie Obenour (she/her/hers)

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 7253 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]