[RFC] Keyrings: How to make them more useful

[RFC] Keyrings: How to make them more useful

[David Howells]

Hi Jarkko, Steve, Chuck, Mimi, et al.,

I think work needs to be done on the keyrings subsystem to make them more
useful for network filesystems and other kernel services such as TLS and
crypto.

There are a number of issues that I think need addressing:

 (1) One of the flaws in the initial design is that whilst keys have a type
     (which is necessary), this has to be specified as part of the lookup or
     the search, which is overly restrictive.

     It probably would have been better to search by description alone and
     then, if a key is found, have any type of key with that description
     returned and let the app/service investigate the key to find the type.

     Now, this is still possible to implement on top of the existing API: just
     allow a NULL type to be passed in - but we might need some way to
     enumerate all the keys with that description, but of different types.
     Possibly, the search function should return all the matching keys.

     Possibly, within the kernel, for each keyring, all the keys of the same
     description can be stored within a group structure, and the search
     returns the group.  This could also have the added benefit of maybe
     making it easier to handle updates.

 (2) For certain applications, keys need versioning - and we need to be able
     to get access to older versions (at least to some extent) of the keys.
     An example of this is cifs where (if I understand it correctly) the key
     version gets cranked, but not all servers may have caught up yet, so we
     need to be able to try the keys in descending order of version.

     This could also work within the group idea mentioned above.

 (3) For certain applications, such as AFS and AF_RXRPC, we may need to be
     able to keep a number of keys around that have the same description
     (e.g. cell name) and basic type (e.g. rxrpc) and version, but that have
     different crypto types (e.g. Rx security classes and Kerberos types, such
     as RxGK+aes256-cts-hmac-sha1-96, RxGK+aes128-cts-hmac-sha256-128 or
     RxKAD) as different servers in the same cell might not support all or we
     might be implementing a server that is offering multiple crypto types.

     So we might need a "subtype" as well as a version.

 (4) I think the keyring ACLs idea need to be revived.  We have a whole bunch
     of different keyrings, each with a specific 'domain' of usage for the
     keys contained therein for checking signatures on things.  Can we reduce
     this to one keyring and use ACLs to declare the specific purposes for
     which a key may be used or the specific tasks that may use it?  Use
     special subject IDs (ie. not simply UIDs/GIDs) to mark this.

 (5) Replace the upcall mechanism with a listenable service channel, so that a
     userspace service (possibly part of systemd or driven from systemd) can
     listen on it and perform key creation/maintenance services.

     From previous discussions with the systemd maintainer, it would be a lot
     easier for them to manage if the key is attached to a file descriptor -
     at least for the duration of the maintenance operation.

     Further, this needs to be containerised in some way so that requests from
     different containers can be handled separately - and can be
     distinguished.

 (6) Move away from keeping DNS records in a keyring, but rather keep them in
     some sort of shrinkable list.  They could still be looked up over a
     secure channel.

To aid with at least (1), (2) and (3) and possibly (4), I think it might be
worth adding an extended add_key() system call that takes an additional
parameter string:

	key_serial_t add_key2(const char *type,
			      const char *description,
			      const char *parameters,
			      const void payload, size_t plen,
			      key_serial_t keyring);

The parameters would get passed to the key type driver for it to extract
things like version number and subtype from without the need to try and fold
it into the payload (which may, for example, be a binary ticket obtained from
kerberos).  Though possibly that is a bad example as the kerberos ticket may
contain multiple keys.

Also, maybe add a multi-key adding syscall for when the payload may contain
multiple keys, each to be added separately:

	int add_keys(const char *type,
		     const char *description,
		     const char *parameters,
		     const void payload, size_t plen,
		     key_serial_t keyring);

When it comes to keyrings, I'm thinking that the keyring needs to change such
that the index holds CoW groups of keys of the same description, but of
different type, version and subtype, e.g.:

	struct key_group {
		struct rcu_head		rcu;
		struct key_group	*replacement;
		char			*description;
		unsigned int		seq;
		refcount_t		ref;
		int			nr_keys;
		struct {
			unsigned long	version;
			struct key __rcu *key;
		} key_list[];
	};

and that these groups should be made available to kernel services upon
searching.  I'm tempted to put the version as part of the group as a whole,
making it easier to ditch a set of the same version, but that could make RCU
CoW-ness tricky.

I could then add two new keyctls, one to unlink all the keys in a keyring that
match description and, optionally, type and parameters (e.g. of a particular
version):

	int keyctl_scrub(const char *type, /* can be NULL */
			 const char *description,
			 const char *parameters, /* can be NULL */
			 key_serial_t keyring);

and one to list all the keys matching a description and, optionally, type and
parameters:

	int list_keys(const char *type, /* can be NULL */
		      const char *description,
		      const char *parameters, /* can be NULL */
		      key_serial_t keyring,
		      key_serial_t *list,
		      size_t list_size);

Thoughts?

Thanks,
David

Re: [RFC] Keyrings: How to make them more useful

[Benjamin Coddington]

On 12 Jun 2025, at 8:36, David Howells wrote:

> Hi Jarkko, Steve, Chuck, Mimi, et al.,
>
> I think work needs to be done on the keyrings subsystem to make them more
> useful for network filesystems and other kernel services such as TLS and
> crypto.
>
> There are a number of issues that I think need addressing:
>
>  (1) One of the flaws in the initial design is that whilst keys have a type
>      (which is necessary), this has to be specified as part of the lookup or
>      the search, which is overly restrictive.
>
>      It probably would have been better to search by description alone and
>      then, if a key is found, have any type of key with that description
>      returned and let the app/service investigate the key to find the type.
>
>      Now, this is still possible to implement on top of the existing API: just
>      allow a NULL type to be passed in - but we might need some way to
>      enumerate all the keys with that description, but of different types.
>      Possibly, the search function should return all the matching keys.
>
>      Possibly, within the kernel, for each keyring, all the keys of the same
>      description can be stored within a group structure, and the search
>      returns the group.  This could also have the added benefit of maybe
>      making it easier to handle updates.
>
>  (2) For certain applications, keys need versioning - and we need to be able
>      to get access to older versions (at least to some extent) of the keys.
>      An example of this is cifs where (if I understand it correctly) the key
>      version gets cranked, but not all servers may have caught up yet, so we
>      need to be able to try the keys in descending order of version.
>
>      This could also work within the group idea mentioned above.
>
>  (3) For certain applications, such as AFS and AF_RXRPC, we may need to be
>      able to keep a number of keys around that have the same description
>      (e.g. cell name) and basic type (e.g. rxrpc) and version, but that have
>      different crypto types (e.g. Rx security classes and Kerberos types, such
>      as RxGK+aes256-cts-hmac-sha1-96, RxGK+aes128-cts-hmac-sha256-128 or
>      RxKAD) as different servers in the same cell might not support all or we
>      might be implementing a server that is offering multiple crypto types.
>
>      So we might need a "subtype" as well as a version.
>
>  (4) I think the keyring ACLs idea need to be revived.  We have a whole bunch
>      of different keyrings, each with a specific 'domain' of usage for the
>      keys contained therein for checking signatures on things.  Can we reduce
>      this to one keyring and use ACLs to declare the specific purposes for
>      which a key may be used or the specific tasks that may use it?  Use
>      special subject IDs (ie. not simply UIDs/GIDs) to mark this.
>
>  (5) Replace the upcall mechanism with a listenable service channel, so that a
>      userspace service (possibly part of systemd or driven from systemd) can
>      listen on it and perform key creation/maintenance services.

>      From previous discussions with the systemd maintainer, it would be a lot
>      easier for them to manage if the key is attached to a file descriptor -
>      at least for the duration of the maintenance operation.
>
>      Further, this needs to be containerised in some way so that requests from
>      different containers can be handled separately - and can be
>      distinguished.

Indeed one challenge on this front is configuring how to stitch together the
various callers and recievers especially when one wants an upcall from one
set of namespaces to be serviced within another.

I had previously posted some work in this area that fleshes out the idea of
a "key agent" which is a userspace process that can receive a notification
to instantiate a key.  The nice part (IMO) of this idea is that the keyagent
is represented by a key itself, so the channel is available to any process
that has the keyagent key in their keyrings.

This allows a system to have a single keyagent for all processes/namespaces
for a particular key type, or to build a more granular configuration where
agents are confined within the same (or different) namespaces as the calling
process.

I'd be happy to continue work on this front, since long-term it would allow
us to convert various NFS upcall mechanisms such that NFS access within a
container wouldn't require duplicated gssd/svcgssd/idmapper userspace
processes in every container if all the system needed was to use a single
global instance.  It would also allow the partitioning of secure
cryptographic material (like keytabs and certificate secrets) from
containers that might still want to use NFS, but not divulge those secrets
to the processes in that container.

Ben

Re: [RFC] Keyrings: How to make them more useful

[James Bottomley]

On Thu, 2025-06-12 at 13:36 +0100, David Howells wrote:
[...]
> Thoughts?

One of the problems I keep tripping over is different special casing
for user keyrings (which are real struct key structures) and system
keyrings which are special values of the pointer in struct key *.

For examples of what this special handling does, just look at things
like bpf_trace.c:bpf_lookup_{user|system}_key

Since the serial allocation code has a hard coded not less than 3
(which looks for all the world like it was designed to mean the two
system keyring id's were never used as user serial numbers) I think we
could simply allow the two system keyring ids to be passed into
lookup_user_key() (which now might be a bit misnamed) and special case
not freeing it in put_key().

Regards,

James

Re: [RFC] Keyrings: How to make them more useful

[David Howells]

James Bottomley <James.Bottomley@HansenPartnership.com> wrote:

> One of the problems I keep tripping over is different special casing
> for user keyrings (which are real struct key structures) and system
> keyrings which are special values of the pointer in struct key *.

It's meant to be like that.  The trusted system keyrings are static within
system_keyring.c and not so easily accessible by kernel modules for
direct modification, bypassing the security checks.

Obviously this is merely a bit of obscurity and enforcement isn't possible
against kernel code that is determined to modify those keyrings or otherwise
interfere in the verification process.

> For examples of what this special handling does, just look at things
> like bpf_trace.c:bpf_lookup_{user|system}_key
> 
> Since the serial allocation code has a hard coded not less than 3
> (which looks for all the world like it was designed to mean the two
> system keyring id's were never used as user serial numbers)

That's just a coincidence.  The <3 thing predates the advent of those system
keyring magic pointers.

> I think we could simply allow the two system keyring ids to be passed into
> lookup_user_key() (which now might be a bit misnamed) and special case not
> freeing it in put_key().

If you want to make lookup_user_key() provide access to specific keyrings like
this, just use the next negative numbers - it's not like we're likely to run
out soon.

But I'd rather not let lookup_user_key() return pointers to these keyrings...

David

Re: [RFC] Keyrings: How to make them more useful

[James Bottomley]

On Thu, 2025-06-12 at 21:36 +0100, David Howells wrote:
> James Bottomley <James.Bottomley@HansenPartnership.com> wrote:
> 
> > One of the problems I keep tripping over is different special
> > casing for user keyrings (which are real struct key structures) and
> > system keyrings which are special values of the pointer in struct
> > key *.
> 
> It's meant to be like that.  The trusted system keyrings are static
> within system_keyring.c and not so easily accessible by kernel
> modules for direct modification, bypassing the security checks.
> 
> Obviously this is merely a bit of obscurity and enforcement isn't
> possible against kernel code that is determined to modify those
> keyrings or otherwise interfere in the verification process.

Yes, I get that, and wasn't proposing to alter it, merely make them a
bit more usable as part of the standard workflow.

Are the permissions also deliberate?  It would be useful for ordinary
users to see the certificates in there in case they want to condition
anything on signatures with internal keys.

> > For examples of what this special handling does, just look at
> > things like bpf_trace.c:bpf_lookup_{user|system}_key
> > 
> > Since the serial allocation code has a hard coded not less than 3
> > (which looks for all the world like it was designed to mean the two
> > system keyring id's were never used as user serial numbers)
> 
> That's just a coincidence.  The <3 thing predates the advent of those
> system keyring magic pointers.

Well coincidence or not it makes the scheme workable, but I've no
objection to using negatives as suggested below, either.

> > I think we could simply allow the two system keyring ids to be
> > passed into lookup_user_key() (which now might be a bit misnamed)
> > and special case not freeing it in put_key().
> 
> If you want to make lookup_user_key() provide access to specific
> keyrings like this, just use the next negative numbers - it's not
> like we're likely to run out soon.
> 
> But I'd rather not let lookup_user_key() return pointers to these
> keyrings...

That wasn't what I was proposing.  If we were to allow it I think it
would return the standard pointer that's actually simply a number as we
do now.  That would be enough to unify all the workflows into
pkcs7_verify_..() and crash soon enough if the key pointers were used
inside the kernel for stuff the system keyrings aren't supposed to do.

Regards,

James

Re: [RFC] Keyrings: How to make them more useful

[Mimi Zohar]

On Thu, 2025-06-12 at 13:36 +0100, David Howells wrote:

[ ...]

>  (4) I think the keyring ACLs idea need to be revived.  We have a whole bunch
>      of different keyrings, each with a specific 'domain' of usage for the
>      keys contained therein for checking signatures on things.  Can we reduce
>      this to one keyring and use ACLs to declare the specific purposes for
>      which a key may be used or the specific tasks that may use it?  Use
>      special subject IDs (ie. not simply UIDs/GIDs) to mark this.

David, which keyrings are you referring to?  What do you mean by 'domain' of
usage?  At what level of granularity are you thinking of?  This needs to be
describe in more detail.

thanks,

Mimi

Re: [RFC] Keyrings: How to make them more useful

[Jarkko Sakkinen]

On Thu, Jun 12, 2025 at 01:36:50PM +0100, David Howells wrote:
> Hi Jarkko, Steve, Chuck, Mimi, et al.,
> 
> I think work needs to be done on the keyrings subsystem to make them more
> useful for network filesystems and other kernel services such as TLS and
> crypto.
> 
> There are a number of issues that I think need addressing:
> 
>  (1) One of the flaws in the initial design is that whilst keys have a type
>      (which is necessary), this has to be specified as part of the lookup or
>      the search, which is overly restrictive.
> 
>      It probably would have been better to search by description alone and
>      then, if a key is found, have any type of key with that description
>      returned and let the app/service investigate the key to find the type.
> 
>      Now, this is still possible to implement on top of the existing API: just
>      allow a NULL type to be passed in - but we might need some way to
>      enumerate all the keys with that description, but of different types.
>      Possibly, the search function should return all the matching keys.
> 
>      Possibly, within the kernel, for each keyring, all the keys of the same
>      description can be stored within a group structure, and the search
>      returns the group.  This could also have the added benefit of maybe
>      making it easier to handle updates.

All keys matching the description can be collected by iterating all the
key types (instead of passing NULL).

Having only a syscall for the process has of course much better
concurrency properties.  Just trying to understand the improvements by
actual measure, that's all.

>  (2) For certain applications, keys need versioning - and we need to be able
>      to get access to older versions (at least to some extent) of the keys.
>      An example of this is cifs where (if I understand it correctly) the key
>      version gets cranked, but not all servers may have caught up yet, so we
>      need to be able to try the keys in descending order of version.
> 
>      This could also work within the group idea mentioned above.
> 
>  (3) For certain applications, such as AFS and AF_RXRPC, we may need to be
>      able to keep a number of keys around that have the same description
>      (e.g. cell name) and basic type (e.g. rxrpc) and version, but that have
>      different crypto types (e.g. Rx security classes and Kerberos types, such
>      as RxGK+aes256-cts-hmac-sha1-96, RxGK+aes128-cts-hmac-sha256-128 or
>      RxKAD) as different servers in the same cell might not support all or we
>      might be implementing a server that is offering multiple crypto types.
> 
>      So we might need a "subtype" as well as a version.

1. How the history is capped? I presume it is a fixed-size circular
   buffer, where the oldest entry is deleted, when the key is updated.
1. How history queried by the caller?
2. How a newer version is revoked in favor of a newer version?

>  (4) I think the keyring ACLs idea need to be revived.  We have a whole bunch
>      of different keyrings, each with a specific 'domain' of usage for the
>      keys contained therein for checking signatures on things.  Can we reduce
>      this to one keyring and use ACLs to declare the specific purposes for
>      which a key may be used or the specific tasks that may use it?  Use
>      special subject IDs (ie. not simply UIDs/GIDs) to mark this.

Is subject ID some kind of token that a process can hold (and possibly
can be sent to a process)?

> 
>  (5) Replace the upcall mechanism with a listenable service channel, so that a
>      userspace service (possibly part of systemd or driven from systemd) can
>      listen on it and perform key creation/maintenance services.
> 
>      From previous discussions with the systemd maintainer, it would be a lot
>      easier for them to manage if the key is attached to a file descriptor -
>      at least for the duration of the maintenance operation.
> 
>      Further, this needs to be containerised in some way so that requests from
>      different containers can be handled separately - and can be
>      distinguished

This sounds like an universally sane idea (outside the scope of the
patch set).

> 
>  (6) Move away from keeping DNS records in a keyring, but rather keep them in
>      some sort of shrinkable list.  They could still be looked up over a
>      secure channel.

Don't expertise to understand the benefits of this change. 


> 
> To aid with at least (1), (2) and (3) and possibly (4), I think it might be
> worth adding an extended add_key() system call that takes an additional
> parameter string:
> 
> 	key_serial_t add_key2(const char *type,
> 			      const char *description,
> 			      const char *parameters,
> 			      const void payload, size_t plen,
> 			      key_serial_t keyring);
> 
> The parameters would get passed to the key type driver for it to extract
> things like version number and subtype from without the need to try and fold
> it into the payload (which may, for example, be a binary ticket obtained from
> kerberos).  Though possibly that is a bad example as the kerberos ticket may
> contain multiple keys.

Does the parameter list mimic kernel command-line style of comma
separated attributes?

BR, Jarkko