Re: Breakage in ktls-utils with nfs keyring?

[Sagi Grimberg <sagi@grimberg.me>]

On 02/05/2026 6:08, Chuck Lever wrote:
>
> On Fri, May 1, 2026, at 4:19 PM, Scott Mayhew wrote:
>> On Thu, 30 Apr 2026, Chuck Lever wrote:
>>
>>> Cc'ing the ktls-utils development list.
>>>
>>> On Thu, Apr 30, 2026, at 9:32 AM, Sagi Grimberg wrote:
>>>> Hey Chuck,
>>>>
>>>> Upstream ktls-utils fails passing client certificate and private key
>>>> using the .nfs keyring.
>>>> Bisecting leads commit facd084e43fc ("tlshd: Client-side dual
>>>> certificate support").
>>>>
>>>> I manually apply this (probably wrong) change and keyring works:
>>>> --
>>>> diff --git a/src/tlshd/client.c b/src/tlshd/client.c
>>>> index 2664ffb..a946797 100644
>>>> --- a/src/tlshd/client.c
>>>> +++ b/src/tlshd/client.c
>>>> @@ -327,7 +327,7 @@ tlshd_x509_retrieve_key_cb(gnutls_session_t session,
>>>>           } else {
>>>>                   tlshd_log_debug("%s: Selecting x509.certificate from
>>>> conf file", __func__);
>>>>                   *pcert_length = tlshd_certs_len;
>>>> -               *pcert = tlshd_certs + tlshd_pq_certs_len;
>>>> +               *pcert = tlshd_certs;
>>>>                   *privkey = tlshd_privkey;
>>>>           }
>>>>           return 0;
>>>> --
>>>>
>>>> But, I have a feeling its not the correct change...
>>>
>>> Scott, can you triage this?
>> So when I added the dual certificate support, I didn't touch any of the
>> keyring code.  Frankly, I'm not entirely sure what is the right way to
>> set it up and the docs are pretty much nonexistent.  As far as I can
>> tell:
>>
>> - you need to load nfs.ko first so that the .nfs keyring gets created
>>    via nfs_init_keyring()
>> - you need to restart tlshd so that it links the .nfs keyring into its
>>    session keyring (I tried loading nfs.ko at boot via modules-load.d,
>>    but tlshd still reported an error saying it couldn't find the .nfs
>>    keyring)
>> - you need to convert the cert and key to DER format
>> - you need to add the cert and key to the .nfs keyring, e.g.
>>
>>    keyctl padd user "nfs_cert" %:.nfs < smayhew-rawhide.crt.der
>>    keyctl padd user "nfs_key" %:.nfs < smayhew-rawhide.key.der
>>
>> - then you mount w/ '-o xprtsec=mtls,cert_serial=...,privkey_serial=...'
>>
>> Is that somewhat accurate?

It is.

>>    Is there a better way to do it?

Have a script/automation SW.

>>   It seems
>> like a lot more work than just using the config file.

Well in some cases, storing credentials on a persistent file is not a 
viable option.
For nvme there is a userspace utility that helps with this to some extent.

> It is more work because keyring support for the NFS consumers is still
> aspirational/experimental.

Can you elaborate? I think people expect to be able to pass certs/keys to
tlshd the .nfs keyring. Also I expected it to work (as it used to).

>
> I've pushed your patch to a "fixes" branch for folks to try out. I'm not
> sure yet whether we want a 1.4.1 release with this fix, since keyring
> support for NFS is "not finished".

I understand that there are features that are not supported via the
keyring interface. But I think that users expect things that used to work to
continue working. My personal opinion is that releasing this fix is 
appropriate
given that this is a regression.

Is keyring support for NFS marked as "experimental" or "not finished" 
anywhere?
[I should register to kernel-tls-handshake mailing list]