[RFC] NFS: named client identities for mTLS mounts and a per-namespace .nfs keyring

[Chuck Lever <cel@kernel.org>]

Today, exactly one x.509 certificate and private key pair can be
used at a time for all NFS mounts. The location of that pair is
set in /etc/tlshd/config.

We currently have an awkward experimental mechanism for specifying
an alternative x.509 certificate and private key for an xprtsec=mtls
NFS mount, but it needs to be completed so it can be documented and
advertised for use.

I asked Claude to write a rough draft of a design document that
outlines what needs to be done to finish the work. I would like
input on the kernel-side mechanism in particular for the
per-network-namespace keyring and the way userspace reaches it.


Problem
=======

NFS mutual-TLS mounts (xprtsec=mtls) need the client to present an
x.509 certificate and prove possession of its private key. The
handshake runs in userspace in tlshd; the kernel hands tlshd the
credentials by keyring serial number over the handshake genetlink
upcall.

The only front end today is two undocumented integer mount options:

    mount -o xprtsec=mtls,cert_serial=723847,privkey_serial=723848 \
          server:/export /mnt

The administrator must load the cert and key into the keyring out of
band, discover the integer serials, and paste them onto the command
line. Serials are opaque, non-reproducible across boots, and easy to
transpose. There is also no isolation: nfs_tls_key_verify() does a
global key_lookup() on the serial, and the .nfs keyring created in
fs/nfs/inode.c is module-global and never referenced again -- any tlshd
that learns a serial can read the key.

This RFC proposes a named, per-mount client-identity interface backed
by a provisioning CLI, and fixes the keyring to isolate credentials per
network namespace. The kernel handshake ABI (integer serials over
genetlink) does not change.


The cross-subsystem ask: a per-netns .nfs keyring
=================================================

Network namespace is the correct isolation domain. tlshd is bound to a
network namespace, not a user namespace: it services sockets passed up
from the kernel over the per-netns handshake genetlink socket, and one
tlshd runs per network namespace that needs TLS-protected mounts.

  - Replace the dead module-global .nfs keyring with one keyring per
    network namespace, held in struct nfs_net (fs/nfs/netns.h) and
    allocated at nfs_net_init(). The keys subsystem otherwise
    namespaces on user_namespace, so this is a kernel-held object
    referenced from nfs_net (like today's global keyring, but one per
    netns). The DNS resolver's per-netns key scoping (net->key_domain,
    request_key_net()) is precedent that netns-scoped key handling is
    acceptable.

  - tlshd attaches at handshake time, not at launch. This matters: the
    keyring may be empty or freshly created when tlshd starts, so
    linking it by name at startup is the wrong model. Instead NFS sets
    ta_keyring to the netns .nfs keyring serial in
    xs_tls_handshake_sync(), the kernel sends it as
    HANDSHAKE_A_ACCEPT_KEYRING, and tlshd links that serial into its
    session keyring per handshake -- the path tlshd already implements.
    Linking grants tlshd possession of the keyring and, through it, of
    the possessor-scoped cert and privkey keys.

  - Credential keys are created possessor-readable only (no
    KEY_USR_READ). That is what makes isolation enforceable rather
    than advisory: a key provisioned in namespace A is absent from B's
    keyring and unreadable by B's tlshd even if its serial leaks.

Open question, and where I most want input: userspace -- the
provisioning CLI and mount.nfs -- needs to name the kernel-held netns
keyring in order to add and search keys. Candidates, modeled on
KEYCTL_GET_PERSISTENT (security/keys/persistent.c):

  (a) a new keyctl command that links the caller's netns .nfs keyring
      into a destination keyring and returns its serial;
  (b) an NFS-specific request_key key type the module instantiates to
      point at the netns keyring;
  (c) a per-netns serial exported via procfs or netlink.

The per-netns keyring decision itself I consider settled; the retrieval
primitive is the open one. There is also a user_namespace accounting
nuance: keys added by userspace are quota-charged against a key_user
keyed by user_namespace even though the keyring lives in nfs_net. I
would like the keyrings folks to confirm the quota and ownership
interaction is sane when the user_ns and net_ns boundaries do not
coincide.


Userspace front end
===================

With the keyring in place the front end is straightforward and follows
the nvme-cli / cifscreds pattern.

A new nfs-utils tool -- working name nfstlskey, fitting the nfsidmap /
nfsconf family -- manages x.509 client identities:

    nfstlskey add  <identity> --cert cert.pem --key key.pem
    nfstlskey list
    nfstlskey remove <identity>

The add subcommand reads the PEM cert and key, converts each to DER,
and creates two "user" keys on the netns .nfs keyring ("user" because
tlshd consumes raw DER via keyctl_read_alloc()), with possessor-only
read. Description convention:

    nfs:x509:<identity>:cert
    nfs:x509:<identity>:privkey

The mount command names the identity:

    mount -o xprtsec=mtls,tls_identity=<identity> server:/export /mnt

mount.nfs runs in the caller's namespace, searches the .nfs keyring for
the two descriptions, and passes the existing cert_serial= and
privkey_serial= options to the kernel. tls_identity= is purely a
userspace convenience that resolves a name to the serials the kernel
already accepts; the raw serial options remain as a documented escape
hatch. Both get documented in nfs(5), with a new nfstlskey(8) page.

tlshd changes are minimal: confirm the per-handshake link of the passed
keyring happens before the cert and privkey serials are read, and
retire the now-unnecessary .nfs entry in the keyrings= startup path.


Future work: mTLS-protected NFSROOT
===================================

NFSROOT mounts run in the kernel at boot, in the initial network
namespace, before any userspace mount.nfs, nfstlskey, or possibly tlshd
exists. mTLS for NFSROOT therefore cannot rely on userspace
provisioning: the kernel must obtain the client cert and key itself --
the leading candidate is extracting key material from a TPM -- and
place it on the initial-netns .nfs keyring before the handshake, with a
tlshd available early (initramfs). The kernel-owned per-netns keyring
chosen here is a prerequisite: a userspace-created keyring could not be
populated at NFSROOT time. Out of scope for the initial work, but the
design must not foreclose it, and the same TPM-resident-key path would
inform an eventual PKCS#11/TPM identity-naming scheme.


Alternatives considered
=======================

  - request-key upcall keyed by server name (the nfsidmap model): most
    NFS-native and needs no mount option, but makes selection automatic
    rather than per-mount explicit. Worth revisiting if per-server
    auto-selection is ever wanted.

  - File paths in the mount options (strongSwan / wpa_supplicant model):
    forces mount.nfs to read key files and load the keyring on every
    mount and offers no reusable provisioned identity.

  - Document the raw serials only: does nothing for namespacing.


Comments welcome, particularly on the keyring retrieval primitive and
whether network namespace is the binding you would expect.

--
Chuck Lever