Re: [PATCH v12 5/6] tls: add hardware offload key update support

[Sabrina Dubroca <sd@queasysnail.net>]

2026-04-09, 10:46:40 -0700, Rishikesh Jethwani wrote:
> On Mon, Apr 6, 2026 at 1:59 PM Sabrina Dubroca <sd@queasysnail.net> wrote:
> >
> > 2026-04-02, 17:55:10 -0600, Rishikesh Jethwani wrote:
> > > During a TLS 1.3 KeyUpdate the NIC key cannot be replaced immediately
> > > if previously encrypted HW records are awaiting ACK. start_rekey sets
> > > up a temporary SW context with the new key and redirects sendmsg through
> > > tls_sw_sendmsg_locked. When no records are pending, complete_rekey runs
> > > inline during setsockopt. Otherwise, clean_acked sets REKEY_READY once
> > > all old-key records are ACKed, and the next sendmsg calls complete_rekey.
> > > complete_rekey flushes remaining SW records, reinstalls HW offload at
> > > the current write_seq, and frees the temporary context.
> > >
> > > If another KeyUpdate arrives while a rekey is already pending,
> > > start_rekey just re-keys the existing SW AEAD in place.
> > >
> > > If complete_rekey fails (tls_dev_add or crypto_aead_setkey),
> > > we stay in SW mode (REKEY_FAILED) until a subsequent rekey
> > > succeeds, while maintaining TLS_HW configuration.
> > >
> > > Tested on Mellanox ConnectX-6 Dx (Crypto Enabled) with multiple
> > > TLS 1.3 key update cycles.
> >
> > Something here doesn't seem to work. I have a very simple
> > client/server pair where one side just loops doing large send()s and
> > does a rekey (send keyupdate + change key) every N iterations (I've
> > set N large enough that it goes about 5 seconds between rekeys), and
> > the other receives all the data and changes its RX key when it sees a
> > keyupdate. If both sides are doing SW, it works. If I configure either
> > side to use offload, decrypt fails after the rekey unless I add a
> > small sleep() just after changing keys on the TX side.
> >
> 
> Found 2 issues with this test case:
> 
> TX HW Rekey: The start_rekey function is missing the EOR (End of
> Record) marker. Fix will be included in the next code submission.

For this, pay attention to the "[ANN] net-next is CLOSED" messages on
netdev, we're approaching the next merge window.

> RX HW Rekey: When the receiver tries to rekey, multiple new records
> are already decrypted by the NIC with old key and present in receive
> queue. To handle these records, we need to store the old key to
> encrypt with the old key and decrypt with the new key. Also, we need
> to consider back-to-back rekeys if we store the old key.
> 
> Any suggestions on how to address this?

I'm sorry, I can't spend this much time on solving this problem. I've
already allocated a significant amount of time to reviewing/testing
this submisison (including pointing out some very basic things). And
it seems you've already found a solution (store and use the old
key(s)).

-- 
Sabrina