* [PATCH v4 0/3] tls: Add TLS 1.3 hardware offload support
@ 2026-01-21 21:57 Rishikesh Jethwani
2026-01-21 21:57 ` [PATCH v4 1/3] tls: add " Rishikesh Jethwani
` (3 more replies)
0 siblings, 4 replies; 12+ messages in thread
From: Rishikesh Jethwani @ 2026-01-21 21:57 UTC (permalink / raw)
To: netdev
Cc: saeedm, tariqt, mbloch, borisp, john.fastabend, kuba, sd, davem,
pabeni, edumazet, leon, Rishikesh Jethwani
Hi all,
This patch series adds TLS 1.3 support to the kernel TLS hardware offload
infrastructure, enabling hardware acceleration for TLS 1.3 connections
including KeyUpdate (rekey) support.
Background
==========
Currently, the kernel TLS device offload only supports TLS 1.2. With
TLS 1.3 being the current standard and widely deployed, there is a
growing need to extend hardware offload support to TLS 1.3 connections.
TLS 1.3 differs from TLS 1.2 in its record format:
TLS 1.2: [Header (5)] + [Explicit IV (8)] + [Ciphertext] + [Tag (16)]
TLS 1.3: [Header (5)] + [Ciphertext + ContentType (1)] + [Tag (16)]
The key difference is that TLS 1.3 eliminates the explicit IV and
instead appends the content type byte to the plaintext before
encryption. This content type byte must be encrypted along with the
payload for proper authentication tag computation per RFC 8446.
Patch 1: TLS 1.3 hardware offload support
=========================================
Changes to tls_device.c, tls_device_fallback.c, and tls_main.c:
- Extended version validation to accept TLS_1_3_VERSION in both
tls_set_device_offload() and tls_set_device_offload_rx()
- Modified tls_device_record_close() to append the content type
byte before the authentication tag for TLS 1.3 records
- Modified tls_device_reencrypt() to use prot->prepend_size and
prot->tag_size instead of hardcoded TLS 1.2 values
- Pre-populated dummy_page with all 256 byte values for memory
allocation failure fallback path
- Updated tls_device_fallback.c to handle TLS 1.3 IV construction
(XOR with sequence number) and version-specific AAD sizes
- Rekey handling: HW offload key update (rekey) is not yet supported.
Patch 2: Hardware offload key update support
============================================
Changes to include/net/tls.h, net/tls/tls.h, tls_device.c, tls_main.c,
and tls_sw.c:
- Extended tls_set_device_offload() and tls_set_device_offload_rx()
with new_crypto_info parameter for key updates
- During rekey, the old HW context is deleted (tls_dev_del) and a new
one is added (tls_dev_add) with the updated key material
- Graceful degradation: if HW key update fails, the connection
gracefully degrades to software:
* TX: TLS_TX_DEV_CLOSED is set and sk_validate_xmit_skb switches to
tls_validate_xmit_skb_sw for software encryption
* RX: TLS_RX_DEV_DEGRADED and TLS_RX_DEV_CLOSED are set for software
decryption
* In both cases, tx_conf/rx_conf remains TLS_HW
- Record sequence management: during TX rekey, old pending records are
deleted and unacked_record_sn is reset to the new rec_seq
- Split tls_set_sw_offload() into tls_sw_ctx_init() and
tls_sw_ctx_finalize() to allow the HW offload RX path to
initialize SW context first, attempt HW setup, then
finalize (memzero new_crypto_info, call tls_finish_key_update)
- Added TLS_TX_DEV_CLOSED flag to track TX hardware context state,
to avoid double tls_dev_del call, symmetric with existing
TLS_RX_DEV_CLOSED.
Patch 3: mlx5 driver enablement
===============================
- TLS 1.3 version detection and validation with proper capability checking
- TLS 1.3 crypto context configuration using MLX5E_STATIC_PARAMS_CONTEXT_TLS_1_3
- Correct IV handling for TLS 1.3 (12-byte IV vs TLS 1.2's 4-byte salt)
- Hardware offload for both TLS 1.3 AES-GCM-128 and AES-GCM-256
Testing
=======
Tested on Mellanox ConnectX-6 Dx (Crypto Enabled).
Both TX and RX hardware offload verified working with:
- TLS 1.3 AES-GCM-128
- TLS 1.3 AES-GCM-256
- Multiple KeyUpdate cycles (rekey)
Test methodology: ktls_test : https://github.com/insanum/ktls_test/tree/master
Please review and provide feedback.
Thanks,
Rishikesh
v4:
- Split single TLS patch into two separate patches:
* Patch 1: TLS 1.3 basic HW offload support
* Patch 2: HW offload key update (rekey) support with graceful degradation
- Removed record_type check from tls_device_record_close()
- Removed Broadcom bnxt_en out-of-tree driver mention
- Link to v3: https://lore.kernel.org/netdev/20260102184708.24618-1-rjethwani@purestorage.com/
v3:
- Added note about Broadcom bnxt_en out-of-tree driver used for testing
- Link to v2: https://lore.kernel.org/netdev/20251231192322.3791912-1-rjethwani@purestorage.com/
v2:
- Fixed reverse Christmas tree ordering in variable declarations
- Combined 'err' and 'i' declarations (reviewer feedback)
- Link to v1: https://lore.kernel.org/netdev/20251230224137.3600355-1-rjethwani@purestorage.com/
Rishikesh Jethwani (3):
tls: add TLS 1.3 hardware offload support
tls: add hardware offload key update support
mlx5: TLS 1.3 hardware offload support
.../mellanox/mlx5/core/en_accel/ktls.h | 8 +-
.../mellanox/mlx5/core/en_accel/ktls_txrx.c | 14 +-
include/net/tls.h | 4 +
net/tls/tls.h | 14 +-
net/tls/tls_device.c | 319 +++++++++++++-----
net/tls/tls_device_fallback.c | 34 +-
net/tls/tls_main.c | 31 +-
net/tls/tls_sw.c | 77 +++--
8 files changed, 379 insertions(+), 122 deletions(-)
--
2.25.1
^ permalink raw reply [flat|nested] 12+ messages in thread
* [PATCH v4 1/3] tls: add TLS 1.3 hardware offload support
2026-01-21 21:57 [PATCH v4 0/3] tls: Add TLS 1.3 hardware offload support Rishikesh Jethwani
@ 2026-01-21 21:57 ` Rishikesh Jethwani
2026-01-21 21:57 ` [PATCH v4 2/3] tls: add hardware offload key update support Rishikesh Jethwani
` (2 subsequent siblings)
3 siblings, 0 replies; 12+ messages in thread
From: Rishikesh Jethwani @ 2026-01-21 21:57 UTC (permalink / raw)
To: netdev
Cc: saeedm, tariqt, mbloch, borisp, john.fastabend, kuba, sd, davem,
pabeni, edumazet, leon, Rishikesh Jethwani
Add TLS 1.3 support to the kernel TLS hardware offload infrastructure,
enabling hardware acceleration for TLS 1.3 connections on capable NICs.
TLS 1.3 differs from TLS 1.2 in several key ways that affect hardware
offload:
1. Content type handling: TLS 1.3 encrypts the content type as part of
the ciphertext (inner content type), with the outer header always
showing application_data. Added content type byte appending in
tls_device_record_close() before the authentication tag.
2. Version validation: Extended tls_set_device_offload() and
tls_set_device_offload_rx() to accept TLS_1_3_VERSION in addition
to TLS_1_2_VERSION.
3. Software fallback: Updated tls_device_fallback.c to handle TLS 1.3
IV construction (XOR with sequence number instead of explicit IV)
and version-specific AAD sizes (5 bytes for TLS 1.3 vs 13 bytes for
TLS 1.2).
4. Reencrypt path: Modified tls_device_reencrypt() to use prot->prepend_size
and prot->tag_size instead of hardcoded TLS 1.2 values.
5. Memory fallback: Pre-populate dummy_page with identity mapping for
all 256 byte values, enabling safe fallback when memory allocation
fails during TLS 1.3 content type appending.
6. Rekey handling: HW offload key update (rekey) is not yet supported.
Added upfront checks to reject rekey on HW offload connections with
-EOPNOTSUPP. For SW connections, rekey skips device offload attempts
and goes directly to software path.
Tested on Mellanox ConnectX-6 Dx (Crypto Enabled) with TLS 1.3 AES-GCM-128
and AES-GCM-256 cipher suites.
Signed-off-by: Rishikesh Jethwani <rjethwani@purestorage.com>
---
net/tls/tls_device.c | 54 +++++++++++++++++----
net/tls/tls_device_fallback.c | 34 +++++++++----
net/tls/tls_main.c | 89 +++++++++++++++++++++--------------
3 files changed, 124 insertions(+), 53 deletions(-)
diff --git a/net/tls/tls_device.c b/net/tls/tls_device.c
index 82ea407e520a..9c8dfbc668d4 100644
--- a/net/tls/tls_device.c
+++ b/net/tls/tls_device.c
@@ -319,6 +319,30 @@ static void tls_device_record_close(struct sock *sk,
struct tls_prot_info *prot = &ctx->prot_info;
struct page_frag dummy_tag_frag;
+ /* TLS 1.3: append content type byte before tag.
+ * Record structure: [Header (5)] + [Ciphertext + ContentType (1)] + [Tag (16)]
+ * The content type is encrypted with the ciphertext for authentication.
+ */
+ if (prot->version == TLS_1_3_VERSION) {
+ struct page_frag dummy_content_type_frag;
+ struct page_frag *content_type_pfrag = pfrag;
+
+ if (unlikely(pfrag->size - pfrag->offset < prot->tail_size) &&
+ !skb_page_frag_refill(prot->tail_size, pfrag, sk->sk_allocation)) {
+ /* Out of memory: use pre-populated dummy_page */
+ dummy_content_type_frag.page = dummy_page;
+ dummy_content_type_frag.offset = record_type;
+ content_type_pfrag = &dummy_content_type_frag;
+ } else {
+ /* Write content type to current pfrag */
+ unsigned char *content_type_addr;
+
+ content_type_addr = page_address(pfrag->page) + pfrag->offset;
+ *content_type_addr = record_type;
+ }
+ tls_append_frag(record, content_type_pfrag, prot->tail_size);
+ }
+
/* append tag
* device will fill in the tag, we just need to append a placeholder
* use socket memory to improve coalescing (re-using a single buffer
@@ -335,7 +359,7 @@ static void tls_device_record_close(struct sock *sk,
/* fill prepend */
tls_fill_prepend(ctx, skb_frag_address(&record->frags[0]),
- record->len - prot->overhead_size,
+ (record->len - prot->overhead_size) + prot->tail_size,
record_type);
}
@@ -883,6 +907,7 @@ static int
tls_device_reencrypt(struct sock *sk, struct tls_context *tls_ctx)
{
struct tls_sw_context_rx *sw_ctx = tls_sw_ctx_rx(tls_ctx);
+ struct tls_prot_info *prot = &tls_ctx->prot_info;
const struct tls_cipher_desc *cipher_desc;
int err, offset, copy, data_len, pos;
struct sk_buff *skb, *skb_iter;
@@ -894,7 +919,7 @@ tls_device_reencrypt(struct sock *sk, struct tls_context *tls_ctx)
DEBUG_NET_WARN_ON_ONCE(!cipher_desc || !cipher_desc->offloadable);
rxm = strp_msg(tls_strp_msg(sw_ctx));
- orig_buf = kmalloc(rxm->full_len + TLS_HEADER_SIZE + cipher_desc->iv,
+ orig_buf = kmalloc(rxm->full_len + prot->prepend_size,
sk->sk_allocation);
if (!orig_buf)
return -ENOMEM;
@@ -909,9 +934,8 @@ tls_device_reencrypt(struct sock *sk, struct tls_context *tls_ctx)
offset = rxm->offset;
sg_init_table(sg, 1);
- sg_set_buf(&sg[0], buf,
- rxm->full_len + TLS_HEADER_SIZE + cipher_desc->iv);
- err = skb_copy_bits(skb, offset, buf, TLS_HEADER_SIZE + cipher_desc->iv);
+ sg_set_buf(&sg[0], buf, rxm->full_len + prot->prepend_size);
+ err = skb_copy_bits(skb, offset, buf, prot->prepend_size);
if (err)
goto free_buf;
@@ -922,7 +946,7 @@ tls_device_reencrypt(struct sock *sk, struct tls_context *tls_ctx)
else
err = 0;
- data_len = rxm->full_len - cipher_desc->tag;
+ data_len = rxm->full_len - prot->tag_size;
if (skb_pagelen(skb) > offset) {
copy = min_t(int, skb_pagelen(skb) - offset, data_len);
@@ -1089,7 +1113,8 @@ int tls_set_device_offload(struct sock *sk)
}
crypto_info = &ctx->crypto_send.info;
- if (crypto_info->version != TLS_1_2_VERSION) {
+ if (crypto_info->version != TLS_1_2_VERSION &&
+ crypto_info->version != TLS_1_3_VERSION) {
rc = -EOPNOTSUPP;
goto release_netdev;
}
@@ -1196,7 +1221,8 @@ int tls_set_device_offload_rx(struct sock *sk, struct tls_context *ctx)
struct net_device *netdev;
int rc = 0;
- if (ctx->crypto_recv.info.version != TLS_1_2_VERSION)
+ if (ctx->crypto_recv.info.version != TLS_1_2_VERSION &&
+ ctx->crypto_recv.info.version != TLS_1_3_VERSION)
return -EOPNOTSUPP;
netdev = get_netdev_for_sock(sk);
@@ -1409,12 +1435,22 @@ static struct notifier_block tls_dev_notifier = {
int __init tls_device_init(void)
{
- int err;
+ unsigned char *page_addr;
+ int err, i;
dummy_page = alloc_page(GFP_KERNEL);
if (!dummy_page)
return -ENOMEM;
+ /* Pre-populate dummy_page with identity mapping for all byte values.
+ * This is used as fallback for TLS 1.3 content type when memory
+ * allocation fails. By populating all 256 values, we avoid needing
+ * to validate record_type at runtime.
+ */
+ page_addr = page_address(dummy_page);
+ for (i = 0; i < 256; i++)
+ page_addr[i] = (unsigned char)i;
+
destruct_wq = alloc_workqueue("ktls_device_destruct", WQ_PERCPU, 0);
if (!destruct_wq) {
err = -ENOMEM;
diff --git a/net/tls/tls_device_fallback.c b/net/tls/tls_device_fallback.c
index 03d508a45aae..d0c9b7ad8a23 100644
--- a/net/tls/tls_device_fallback.c
+++ b/net/tls/tls_device_fallback.c
@@ -55,7 +55,7 @@ static int tls_enc_record(struct aead_request *aead_req,
cipher_desc = get_cipher_desc(prot->cipher_type);
DEBUG_NET_WARN_ON_ONCE(!cipher_desc || !cipher_desc->offloadable);
- buf_size = TLS_HEADER_SIZE + cipher_desc->iv;
+ buf_size = prot->prepend_size;
len = min_t(int, *in_len, buf_size);
memcpy_from_scatterwalk(buf, in, len);
@@ -66,16 +66,23 @@ static int tls_enc_record(struct aead_request *aead_req,
return 0;
len = buf[4] | (buf[3] << 8);
- len -= cipher_desc->iv;
+ if (prot->version != TLS_1_3_VERSION)
+ len -= cipher_desc->iv;
tls_make_aad(aad, len - cipher_desc->tag, (char *)&rcd_sn, buf[0], prot);
- memcpy(iv + cipher_desc->salt, buf + TLS_HEADER_SIZE, cipher_desc->iv);
+ /* For TLS 1.2, copy explicit IV from record header.
+ * For TLS 1.3, IV was already set up and we XOR with sequence number.
+ */
+ if (prot->version == TLS_1_3_VERSION)
+ tls_xor_iv_with_seq(prot, iv, (char *)&rcd_sn);
+ else
+ memcpy(iv + cipher_desc->salt, buf + TLS_HEADER_SIZE, cipher_desc->iv);
sg_init_table(sg_in, ARRAY_SIZE(sg_in));
sg_init_table(sg_out, ARRAY_SIZE(sg_out));
- sg_set_buf(sg_in, aad, TLS_AAD_SPACE_SIZE);
- sg_set_buf(sg_out, aad, TLS_AAD_SPACE_SIZE);
+ sg_set_buf(sg_in, aad, prot->aad_size);
+ sg_set_buf(sg_out, aad, prot->aad_size);
scatterwalk_get_sglist(in, sg_in + 1);
scatterwalk_get_sglist(out, sg_out + 1);
@@ -112,7 +119,6 @@ static void tls_init_aead_request(struct aead_request *aead_req,
struct crypto_aead *aead)
{
aead_request_set_tfm(aead_req, aead);
- aead_request_set_ad(aead_req, TLS_AAD_SPACE_SIZE);
}
static struct aead_request *tls_alloc_aead_request(struct crypto_aead *aead,
@@ -303,9 +309,9 @@ static struct sk_buff *tls_enc_skb(struct tls_context *tls_ctx,
{
struct tls_offload_context_tx *ctx = tls_offload_ctx_tx(tls_ctx);
int tcp_payload_offset = skb_tcp_all_headers(skb);
+ void *buf, *iv, *aad, *dummy_buf, *salt, *iv_src;
int payload_len = skb->len - tcp_payload_offset;
const struct tls_cipher_desc *cipher_desc;
- void *buf, *iv, *aad, *dummy_buf, *salt;
struct aead_request *aead_req;
struct sk_buff *nskb = NULL;
int buf_len;
@@ -317,7 +323,10 @@ static struct sk_buff *tls_enc_skb(struct tls_context *tls_ctx,
cipher_desc = get_cipher_desc(tls_ctx->crypto_send.info.cipher_type);
DEBUG_NET_WARN_ON_ONCE(!cipher_desc || !cipher_desc->offloadable);
- buf_len = cipher_desc->salt + cipher_desc->iv + TLS_AAD_SPACE_SIZE +
+ /* Set AAD size based on TLS version */
+ aead_request_set_ad(aead_req, tls_ctx->prot_info.aad_size);
+
+ buf_len = cipher_desc->salt + cipher_desc->iv + tls_ctx->prot_info.aad_size +
sync_size + cipher_desc->tag;
buf = kmalloc(buf_len, GFP_ATOMIC);
if (!buf)
@@ -325,9 +334,16 @@ static struct sk_buff *tls_enc_skb(struct tls_context *tls_ctx,
iv = buf;
salt = crypto_info_salt(&tls_ctx->crypto_send.info, cipher_desc);
+ iv_src = crypto_info_iv(&tls_ctx->crypto_send.info, cipher_desc);
memcpy(iv, salt, cipher_desc->salt);
aad = buf + cipher_desc->salt + cipher_desc->iv;
- dummy_buf = aad + TLS_AAD_SPACE_SIZE;
+ dummy_buf = aad + tls_ctx->prot_info.aad_size;
+
+ /* For TLS 1.3, copy the full fixed IV (salt + iv portion).
+ * For TLS 1.2, iv portion will be filled from record in tls_enc_record.
+ */
+ if (tls_ctx->prot_info.version == TLS_1_3_VERSION)
+ memcpy(iv + cipher_desc->salt, iv_src, cipher_desc->iv);
nskb = alloc_skb(skb_headroom(skb) + skb->len, GFP_ATOMIC);
if (!nskb)
diff --git a/net/tls/tls_main.c b/net/tls/tls_main.c
index 56ce0bc8317b..f7c369714b85 100644
--- a/net/tls/tls_main.c
+++ b/net/tls/tls_main.c
@@ -711,49 +711,68 @@ static int do_tls_setsockopt_conf(struct sock *sk, sockptr_t optval,
}
if (tx) {
- rc = tls_set_device_offload(sk);
- conf = TLS_HW;
- if (!rc) {
- TLS_INC_STATS(sock_net(sk), LINUX_MIB_TLSTXDEVICE);
- TLS_INC_STATS(sock_net(sk), LINUX_MIB_TLSCURRTXDEVICE);
- } else {
- rc = tls_set_sw_offload(sk, 1,
- update ? crypto_info : NULL);
- if (rc)
- goto err_crypto_info;
-
- if (update) {
- TLS_INC_STATS(sock_net(sk), LINUX_MIB_TLSTXREKEYOK);
- } else {
- TLS_INC_STATS(sock_net(sk), LINUX_MIB_TLSTXSW);
- TLS_INC_STATS(sock_net(sk), LINUX_MIB_TLSCURRTXSW);
+ /* HW rekey not yet supported */
+ if (update && ctx->tx_conf == TLS_HW) {
+ rc = -EOPNOTSUPP;
+ goto err_crypto_info;
+ }
+
+ /* Only try HW offload on initial setup, not rekey */
+ if (!update) {
+ rc = tls_set_device_offload(sk);
+ conf = TLS_HW;
+ if (!rc) {
+ TLS_INC_STATS(sock_net(sk), LINUX_MIB_TLSTXDEVICE);
+ TLS_INC_STATS(sock_net(sk), LINUX_MIB_TLSCURRTXDEVICE);
+ goto out;
}
- conf = TLS_SW;
}
- } else {
- rc = tls_set_device_offload_rx(sk, ctx);
- conf = TLS_HW;
- if (!rc) {
- TLS_INC_STATS(sock_net(sk), LINUX_MIB_TLSRXDEVICE);
- TLS_INC_STATS(sock_net(sk), LINUX_MIB_TLSCURRRXDEVICE);
+
+ rc = tls_set_sw_offload(sk, 1, update ? crypto_info : NULL);
+ if (rc)
+ goto err_crypto_info;
+
+ if (update) {
+ TLS_INC_STATS(sock_net(sk), LINUX_MIB_TLSTXREKEYOK);
} else {
- rc = tls_set_sw_offload(sk, 0,
- update ? crypto_info : NULL);
- if (rc)
- goto err_crypto_info;
-
- if (update) {
- TLS_INC_STATS(sock_net(sk), LINUX_MIB_TLSRXREKEYOK);
- } else {
- TLS_INC_STATS(sock_net(sk), LINUX_MIB_TLSRXSW);
- TLS_INC_STATS(sock_net(sk), LINUX_MIB_TLSCURRRXSW);
+ TLS_INC_STATS(sock_net(sk), LINUX_MIB_TLSTXSW);
+ TLS_INC_STATS(sock_net(sk), LINUX_MIB_TLSCURRTXSW);
+ }
+ conf = TLS_SW;
+ } else {
+ /* HW rekey not yet supported */
+ if (update && ctx->rx_conf == TLS_HW) {
+ rc = -EOPNOTSUPP;
+ goto err_crypto_info;
+ }
+
+ /* Only try HW offload on initial setup, not rekey */
+ if (!update) {
+ rc = tls_set_device_offload_rx(sk, ctx);
+ conf = TLS_HW;
+ if (!rc) {
+ TLS_INC_STATS(sock_net(sk), LINUX_MIB_TLSRXDEVICE);
+ TLS_INC_STATS(sock_net(sk), LINUX_MIB_TLSCURRRXDEVICE);
+ tls_sw_strparser_arm(sk, ctx);
+ goto out;
}
- conf = TLS_SW;
}
- if (!update)
+
+ rc = tls_set_sw_offload(sk, 0, update ? crypto_info : NULL);
+ if (rc)
+ goto err_crypto_info;
+
+ if (update) {
+ TLS_INC_STATS(sock_net(sk), LINUX_MIB_TLSRXREKEYOK);
+ } else {
+ TLS_INC_STATS(sock_net(sk), LINUX_MIB_TLSRXSW);
+ TLS_INC_STATS(sock_net(sk), LINUX_MIB_TLSCURRRXSW);
tls_sw_strparser_arm(sk, ctx);
+ }
+ conf = TLS_SW;
}
+out:
if (tx)
ctx->tx_conf = conf;
else
--
2.25.1
^ permalink raw reply related [flat|nested] 12+ messages in thread
* [PATCH v4 2/3] tls: add hardware offload key update support
2026-01-21 21:57 [PATCH v4 0/3] tls: Add TLS 1.3 hardware offload support Rishikesh Jethwani
2026-01-21 21:57 ` [PATCH v4 1/3] tls: add " Rishikesh Jethwani
@ 2026-01-21 21:57 ` Rishikesh Jethwani
2026-01-22 4:09 ` kernel test robot
` (3 more replies)
2026-01-21 21:57 ` [PATCH v4 3/3] mlx5: TLS 1.3 hardware offload support Rishikesh Jethwani
2026-01-22 11:30 ` [PATCH v4 0/3] tls: Add " Tariq Toukan
3 siblings, 4 replies; 12+ messages in thread
From: Rishikesh Jethwani @ 2026-01-21 21:57 UTC (permalink / raw)
To: netdev
Cc: saeedm, tariqt, mbloch, borisp, john.fastabend, kuba, sd, davem,
pabeni, edumazet, leon, Rishikesh Jethwani
Add TLS KeyUpdate (rekey) support for hardware offload connections,
enabling key rotation on established TLS 1.3 connections without
tearing down the hardware offload.
Key changes:
1. Rekey API: Extended tls_set_device_offload() and
tls_set_device_offload_rx() with new_crypto_info parameter to
distinguish initial setup from key updates. During rekey, the old
HW context is deleted (tls_dev_del) and a new one is added
(tls_dev_add) with the updated key material.
2. Graceful degradation: If hardware key update fails, the connection
gracefully degrades to software. For TX, TLS_TX_DEV_CLOSED is set
and sk_validate_xmit_skb switches to tls_validate_xmit_skb_sw for
software encryption. For RX, TLS_RX_DEV_DEGRADED and TLS_RX_DEV_CLOSED
are set for software decryption. tx_conf/rx_conf remains TLS_HW.
3. Record sequence management: During TX rekey, old pending records
are deleted and unacked_record_sn is reset to the new rec_seq.
4. SW context refactoring: Split tls_set_sw_offload() into
tls_sw_ctx_init() and tls_sw_ctx_finalize() to allow the HW offload
RX path to initialize SW context first, attempt HW setup, then
finalize (memzero new_crypto_info, call tls_finish_key_update).
5. Added TLS_TX_DEV_CLOSED flag to track TX hardware context state,
to avoid double tls_dev_del call, symmetric with existing
TLS_RX_DEV_CLOSED.
This removes the rekey rejection checks added in the previous patch,
replacing them with full rekey support including graceful degradation.
Tested on Mellanox ConnectX-6 Dx (Crypto Enabled) with multiple
TLS 1.3 key update cycles.
Signed-off-by: Rishikesh Jethwani <rjethwani@purestorage.com>
---
include/net/tls.h | 4 +
net/tls/tls.h | 14 ++-
net/tls/tls_device.c | 273 +++++++++++++++++++++++++++++++------------
net/tls/tls_main.c | 94 +++++++--------
net/tls/tls_sw.c | 77 ++++++++----
5 files changed, 317 insertions(+), 145 deletions(-)
diff --git a/include/net/tls.h b/include/net/tls.h
index ebd2550280ae..9a203394763b 100644
--- a/include/net/tls.h
+++ b/include/net/tls.h
@@ -189,6 +189,10 @@ enum tls_context_flags {
* tls_dev_del call in tls_device_down if it happens simultaneously.
*/
TLS_RX_DEV_CLOSED = 2,
+ /* Flag for TX HW context deleted during failed rekey.
+ * Prevents double tls_dev_del in cleanup paths.
+ */
+ TLS_TX_DEV_CLOSED = 3,
};
struct cipher_context {
diff --git a/net/tls/tls.h b/net/tls/tls.h
index 2f86baeb71fc..1369ee35070a 100644
--- a/net/tls/tls.h
+++ b/net/tls/tls.h
@@ -147,6 +147,10 @@ void tls_strp_abort_strp(struct tls_strparser *strp, int err);
int init_prot_info(struct tls_prot_info *prot,
const struct tls_crypto_info *crypto_info,
const struct tls_cipher_desc *cipher_desc);
+int tls_sw_ctx_init(struct sock *sk, int tx,
+ struct tls_crypto_info *new_crypto_info);
+void tls_sw_ctx_finalize(struct sock *sk, int tx,
+ struct tls_crypto_info *new_crypto_info);
int tls_set_sw_offload(struct sock *sk, int tx,
struct tls_crypto_info *new_crypto_info);
void tls_update_rx_zc_capable(struct tls_context *tls_ctx);
@@ -229,9 +233,10 @@ static inline bool tls_strp_msg_mixed_decrypted(struct tls_sw_context_rx *ctx)
#ifdef CONFIG_TLS_DEVICE
int tls_device_init(void);
void tls_device_cleanup(void);
-int tls_set_device_offload(struct sock *sk);
+int tls_set_device_offload(struct sock *sk, struct tls_crypto_info *crypto_info);
void tls_device_free_resources_tx(struct sock *sk);
-int tls_set_device_offload_rx(struct sock *sk, struct tls_context *ctx);
+int tls_set_device_offload_rx(struct sock *sk, struct tls_context *ctx,
+ struct tls_crypto_info *crypto_info);
void tls_device_offload_cleanup_rx(struct sock *sk);
void tls_device_rx_resync_new_rec(struct sock *sk, u32 rcd_len, u32 seq);
int tls_device_decrypted(struct sock *sk, struct tls_context *tls_ctx);
@@ -240,7 +245,7 @@ static inline int tls_device_init(void) { return 0; }
static inline void tls_device_cleanup(void) {}
static inline int
-tls_set_device_offload(struct sock *sk)
+tls_set_device_offload(struct sock *sk, struct tls_crypto_info *crypto_info)
{
return -EOPNOTSUPP;
}
@@ -248,7 +253,8 @@ tls_set_device_offload(struct sock *sk)
static inline void tls_device_free_resources_tx(struct sock *sk) {}
static inline int
-tls_set_device_offload_rx(struct sock *sk, struct tls_context *ctx)
+tls_set_device_offload_rx(struct sock *sk, struct tls_context *ctx,
+ struct tls_crypto_info *crypto_info)
{
return -EOPNOTSUPP;
}
diff --git a/net/tls/tls_device.c b/net/tls/tls_device.c
index 9c8dfbc668d4..03ceaf29ff5d 100644
--- a/net/tls/tls_device.c
+++ b/net/tls/tls_device.c
@@ -79,7 +79,8 @@ static void tls_device_tx_del_task(struct work_struct *work)
netdev = rcu_dereference_protected(ctx->netdev,
!refcount_read(&ctx->refcount));
- netdev->tlsdev_ops->tls_dev_del(netdev, ctx, TLS_OFFLOAD_CTX_DIR_TX);
+ if (!test_bit(TLS_TX_DEV_CLOSED, &ctx->flags))
+ netdev->tlsdev_ops->tls_dev_del(netdev, ctx, TLS_OFFLOAD_CTX_DIR_TX);
dev_put(netdev);
ctx->netdev = NULL;
tls_device_free_ctx(ctx);
@@ -1083,12 +1084,13 @@ static struct tls_offload_context_tx *alloc_offload_ctx_tx(struct tls_context *c
return offload_ctx;
}
-int tls_set_device_offload(struct sock *sk)
+int tls_set_device_offload(struct sock *sk,
+ struct tls_crypto_info *new_crypto_info)
{
+ struct tls_crypto_info *crypto_info, *src_crypto_info;
struct tls_record_info *start_marker_record;
struct tls_offload_context_tx *offload_ctx;
const struct tls_cipher_desc *cipher_desc;
- struct tls_crypto_info *crypto_info;
struct tls_prot_info *prot;
struct net_device *netdev;
struct tls_context *ctx;
@@ -1098,8 +1100,12 @@ int tls_set_device_offload(struct sock *sk)
ctx = tls_get_ctx(sk);
prot = &ctx->prot_info;
- if (ctx->priv_ctx_tx)
- return -EEXIST;
+ /* Rekey is only supported for connections that are already
+ * using HW offload. For SW offload connections, the caller
+ * should fall back to tls_set_sw_offload() for rekey.
+ */
+ if (new_crypto_info && ctx->tx_conf != TLS_HW)
+ return -EINVAL;
netdev = get_netdev_for_sock(sk);
if (!netdev) {
@@ -1113,57 +1119,62 @@ int tls_set_device_offload(struct sock *sk)
}
crypto_info = &ctx->crypto_send.info;
- if (crypto_info->version != TLS_1_2_VERSION &&
- crypto_info->version != TLS_1_3_VERSION) {
+ src_crypto_info = new_crypto_info ?: crypto_info;
+ if (src_crypto_info->version != TLS_1_2_VERSION &&
+ src_crypto_info->version != TLS_1_3_VERSION) {
rc = -EOPNOTSUPP;
goto release_netdev;
}
- cipher_desc = get_cipher_desc(crypto_info->cipher_type);
+ cipher_desc = get_cipher_desc(src_crypto_info->cipher_type);
if (!cipher_desc || !cipher_desc->offloadable) {
rc = -EINVAL;
goto release_netdev;
}
- rc = init_prot_info(prot, crypto_info, cipher_desc);
- if (rc)
- goto release_netdev;
+ iv = crypto_info_iv(src_crypto_info, cipher_desc);
+ rec_seq = crypto_info_rec_seq(src_crypto_info, cipher_desc);
- iv = crypto_info_iv(crypto_info, cipher_desc);
- rec_seq = crypto_info_rec_seq(crypto_info, cipher_desc);
+ if (!new_crypto_info) {
+ rc = init_prot_info(prot, src_crypto_info, cipher_desc);
+ if (rc)
+ goto release_netdev;
- memcpy(ctx->tx.iv + cipher_desc->salt, iv, cipher_desc->iv);
- memcpy(ctx->tx.rec_seq, rec_seq, cipher_desc->rec_seq);
+ memcpy(ctx->tx.iv + cipher_desc->salt, iv, cipher_desc->iv);
+ memcpy(ctx->tx.rec_seq, rec_seq, cipher_desc->rec_seq);
- start_marker_record = kmalloc(sizeof(*start_marker_record), GFP_KERNEL);
- if (!start_marker_record) {
- rc = -ENOMEM;
- goto release_netdev;
- }
+ start_marker_record = kmalloc(sizeof(*start_marker_record),
+ GFP_KERNEL);
+ if (!start_marker_record) {
+ rc = -ENOMEM;
+ goto release_netdev;
+ }
- offload_ctx = alloc_offload_ctx_tx(ctx);
- if (!offload_ctx) {
- rc = -ENOMEM;
- goto free_marker_record;
- }
+ offload_ctx = alloc_offload_ctx_tx(ctx);
+ if (!offload_ctx) {
+ rc = -ENOMEM;
+ goto free_marker_record;
+ }
- rc = tls_sw_fallback_init(sk, offload_ctx, crypto_info);
- if (rc)
- goto free_offload_ctx;
+ rc = tls_sw_fallback_init(sk, offload_ctx, src_crypto_info);
+ if (rc)
+ goto free_offload_ctx;
- start_marker_record->end_seq = tcp_sk(sk)->write_seq;
- start_marker_record->len = 0;
- start_marker_record->num_frags = 0;
- list_add_tail(&start_marker_record->list, &offload_ctx->records_list);
+ start_marker_record->end_seq = tcp_sk(sk)->write_seq;
+ start_marker_record->len = 0;
+ start_marker_record->num_frags = 0;
+ list_add_tail(&start_marker_record->list,
+ &offload_ctx->records_list);
- clean_acked_data_enable(tcp_sk(sk), &tls_tcp_clean_acked);
- ctx->push_pending_record = tls_device_push_pending_record;
+ clean_acked_data_enable(tcp_sk(sk), &tls_tcp_clean_acked);
+ ctx->push_pending_record = tls_device_push_pending_record;
- /* TLS offload is greatly simplified if we don't send
- * SKBs where only part of the payload needs to be encrypted.
- * So mark the last skb in the write queue as end of record.
- */
- tcp_write_collapse_fence(sk);
+ /* TLS offload is greatly simplified if we don't send
+ * SKBs where only part of the payload needs to be encrypted.
+ * So mark the last skb in the write queue as end of record.
+ */
+ tcp_write_collapse_fence(sk);
+ }
/* Avoid offloading if the device is down
* We don't want to offload new flows after
@@ -1179,29 +1190,91 @@ int tls_set_device_offload(struct sock *sk)
goto release_lock;
}
- ctx->priv_ctx_tx = offload_ctx;
+ if (!new_crypto_info) {
+ ctx->priv_ctx_tx = offload_ctx;
+ } else {
+ char *key = crypto_info_key(src_crypto_info, cipher_desc);
+
+ offload_ctx = tls_offload_ctx_tx(ctx);
+
+ rc = crypto_aead_setkey(offload_ctx->aead_send, key,
+ cipher_desc->key);
+ if (rc)
+ goto release_lock;
+
+ /* For rekey, delete old HW context before adding new one. */
+ if (!test_bit(TLS_TX_DEV_CLOSED, &ctx->flags))
+ netdev->tlsdev_ops->tls_dev_del(netdev, ctx,
+ TLS_OFFLOAD_CTX_DIR_TX);
+ }
+
rc = netdev->tlsdev_ops->tls_dev_add(netdev, sk, TLS_OFFLOAD_CTX_DIR_TX,
- &ctx->crypto_send.info,
+ src_crypto_info,
tcp_sk(sk)->write_seq);
trace_tls_device_offload_set(sk, TLS_OFFLOAD_CTX_DIR_TX,
tcp_sk(sk)->write_seq, rec_seq, rc);
- if (rc)
- goto release_lock;
- tls_device_attach(ctx, sk, netdev);
+ if (new_crypto_info) {
+ unsigned long flags;
+ __be64 rcd_sn;
+
+ memcpy(ctx->tx.iv + cipher_desc->salt, iv, cipher_desc->iv);
+ memcpy(ctx->tx.rec_seq, rec_seq, cipher_desc->rec_seq);
+
+ spin_lock_irqsave(&offload_ctx->lock, flags);
+ /* Delete old records, can't be retransmitted with new key */
+ delete_all_records(offload_ctx);
+
+ /* Update unacked_record_sn for the new key's rec_seq.
+ * This is critical for SW fallback encryption to use
+ * the correct record sequence number after rekey.
+ */
+ memcpy(&rcd_sn, rec_seq, sizeof(rcd_sn));
+ offload_ctx->unacked_record_sn = be64_to_cpu(rcd_sn);
+ spin_unlock_irqrestore(&offload_ctx->lock, flags);
+
+ unsafe_memcpy(crypto_info, new_crypto_info,
+ cipher_desc->crypto_info,
+ /* size was checked in do_tls_setsockopt_conf */);
+ memzero_explicit(new_crypto_info, cipher_desc->crypto_info);
+ }
+
+ if (rc) {
+ if (new_crypto_info) {
+ /* HW rekey failed, gracefully degrade to SW encryption.
+ * SW fallback already has new key, IV, and rec_seq.
+ * Old HW ctx was deleted, continue with SW encryption.
+ */
+ set_bit(TLS_TX_DEV_CLOSED, &ctx->flags);
+ smp_store_release(sk->sk_validate_xmit_skb,
+ tls_validate_xmit_skb_sw);
+ } else {
+ goto release_lock;
+ }
+ } else {
+ if (new_crypto_info)
+ clear_bit(TLS_TX_DEV_CLOSED, &ctx->flags);
+
+ tls_device_attach(ctx, sk, netdev);
+
+ /* following this assignment tls_is_skb_tx_device_offloaded
+ * will return true and the context might be accessed
+ * by the netdev's xmit function.
+ */
+ smp_store_release(&sk->sk_validate_xmit_skb,
+ tls_validate_xmit_skb);
+ }
+
up_read(&device_offload_lock);
- /* following this assignment tls_is_skb_tx_device_offloaded
- * will return true and the context might be accessed
- * by the netdev's xmit function.
- */
- smp_store_release(&sk->sk_validate_xmit_skb, tls_validate_xmit_skb);
dev_put(netdev);
return 0;
release_lock:
up_read(&device_offload_lock);
+ if (new_crypto_info)
+ goto release_netdev;
clean_acked_data_disable(tcp_sk(sk));
crypto_free_aead(offload_ctx->aead_send);
free_offload_ctx:
@@ -1214,17 +1287,33 @@ int tls_set_device_offload(struct sock *sk)
return rc;
}
-int tls_set_device_offload_rx(struct sock *sk, struct tls_context *ctx)
+int tls_set_device_offload_rx(struct sock *sk, struct tls_context *ctx,
+ struct tls_crypto_info *new_crypto_info)
{
- struct tls12_crypto_info_aes_gcm_128 *info;
+ struct tls_crypto_info *crypto_info, *src_crypto_info;
+ const struct tls_cipher_desc *cipher_desc;
struct tls_offload_context_rx *context;
struct net_device *netdev;
+ char *rec_seq;
int rc = 0;
- if (ctx->crypto_recv.info.version != TLS_1_2_VERSION &&
- ctx->crypto_recv.info.version != TLS_1_3_VERSION)
+ /* Rekey is only supported for connections that are already
+ * using HW offload. For SW offload connections, the caller
+ * should fall back to tls_set_sw_offload() for rekey.
+ */
+ if (new_crypto_info && ctx->rx_conf != TLS_HW)
+ return -EINVAL;
+
+ crypto_info = &ctx->crypto_recv.info;
+ src_crypto_info = new_crypto_info ?: crypto_info;
+ if (src_crypto_info->version != TLS_1_2_VERSION &&
+ src_crypto_info->version != TLS_1_3_VERSION)
return -EOPNOTSUPP;
+ cipher_desc = get_cipher_desc(src_crypto_info->cipher_type);
+ if (!cipher_desc || !cipher_desc->offloadable)
+ return -EINVAL;
+
netdev = get_netdev_for_sock(sk);
if (!netdev) {
pr_err_ratelimited("%s: netdev not found\n", __func__);
@@ -1250,28 +1339,57 @@ int tls_set_device_offload_rx(struct sock *sk, struct tls_context *ctx)
goto release_lock;
}
- context = kzalloc(sizeof(*context), GFP_KERNEL);
- if (!context) {
- rc = -ENOMEM;
- goto release_lock;
+ if (!new_crypto_info) {
+ context = kzalloc(sizeof(*context), GFP_KERNEL);
+ if (!context) {
+ rc = -ENOMEM;
+ goto release_lock;
+ }
+ context->resync_nh_reset = 1;
+ ctx->priv_ctx_rx = context;
}
- context->resync_nh_reset = 1;
- ctx->priv_ctx_rx = context;
- rc = tls_set_sw_offload(sk, 0, NULL);
+ rc = tls_sw_ctx_init(sk, 0, new_crypto_info);
if (rc)
goto release_ctx;
+ /* For rekey, delete old HW context before adding new one. */
+ if (new_crypto_info && !test_bit(TLS_RX_DEV_CLOSED, &ctx->flags))
+ netdev->tlsdev_ops->tls_dev_del(netdev, ctx,
+ TLS_OFFLOAD_CTX_DIR_RX);
+
rc = netdev->tlsdev_ops->tls_dev_add(netdev, sk, TLS_OFFLOAD_CTX_DIR_RX,
- &ctx->crypto_recv.info,
+ src_crypto_info,
tcp_sk(sk)->copied_seq);
- info = (void *)&ctx->crypto_recv.info;
+ rec_seq = crypto_info_rec_seq(src_crypto_info, cipher_desc);
trace_tls_device_offload_set(sk, TLS_OFFLOAD_CTX_DIR_RX,
- tcp_sk(sk)->copied_seq, info->rec_seq, rc);
- if (rc)
- goto free_sw_resources;
+ tcp_sk(sk)->copied_seq, rec_seq, rc);
+ if (rc) {
+ if (new_crypto_info) {
+ /* HW rekey failed, gracefully degrade to SW decryption.
+ * SW context already set up via tls_sw_ctx_init.
+ * Old HW ctx was deleted, set degraded flag for
+ * SW fallback.
+ */
+ set_bit(TLS_RX_DEV_DEGRADED, &ctx->flags);
+ set_bit(TLS_RX_DEV_CLOSED, &ctx->flags);
+ } else {
+ goto free_sw_resources;
+ }
+ } else {
+ if (new_crypto_info) {
+ /* HW rekey succeeded, clear degraded state
+ * if previously set
+ */
+ clear_bit(TLS_RX_DEV_DEGRADED, &ctx->flags);
+ clear_bit(TLS_RX_DEV_CLOSED, &ctx->flags);
+ }
+
+ tls_device_attach(ctx, sk, netdev);
+ }
+
+ tls_sw_ctx_finalize(sk, 0, new_crypto_info);
- tls_device_attach(ctx, sk, netdev);
up_read(&device_offload_lock);
dev_put(netdev);
@@ -1280,10 +1398,15 @@ int tls_set_device_offload_rx(struct sock *sk, struct tls_context *ctx)
free_sw_resources:
up_read(&device_offload_lock);
- tls_sw_free_resources_rx(sk);
+ if (new_crypto_info)
+ goto release_netdev;
+ tls_sw_release_resources_rx(sk);
down_read(&device_offload_lock);
release_ctx:
- ctx->priv_ctx_rx = NULL;
+ if (!new_crypto_info) {
+ kfree(ctx->priv_ctx_rx);
+ ctx->priv_ctx_rx = NULL;
+ }
release_lock:
up_read(&device_offload_lock);
release_netdev:
@@ -1302,8 +1425,9 @@ void tls_device_offload_cleanup_rx(struct sock *sk)
if (!netdev)
goto out;
- netdev->tlsdev_ops->tls_dev_del(netdev, tls_ctx,
- TLS_OFFLOAD_CTX_DIR_RX);
+ if (!test_bit(TLS_RX_DEV_CLOSED, &tls_ctx->flags))
+ netdev->tlsdev_ops->tls_dev_del(netdev, tls_ctx,
+ TLS_OFFLOAD_CTX_DIR_RX);
if (tls_ctx->tx_conf != TLS_HW) {
dev_put(netdev);
@@ -1360,13 +1484,18 @@ static int tls_device_down(struct net_device *netdev)
synchronize_net();
/* Release the offload context on the driver side. */
- if (ctx->tx_conf == TLS_HW)
+ if (ctx->tx_conf == TLS_HW &&
+ !test_bit(TLS_TX_DEV_CLOSED, &ctx->flags)) {
netdev->tlsdev_ops->tls_dev_del(netdev, ctx,
TLS_OFFLOAD_CTX_DIR_TX);
+ set_bit(TLS_TX_DEV_CLOSED, &ctx->flags);
+ }
if (ctx->rx_conf == TLS_HW &&
- !test_bit(TLS_RX_DEV_CLOSED, &ctx->flags))
+ !test_bit(TLS_RX_DEV_CLOSED, &ctx->flags)) {
netdev->tlsdev_ops->tls_dev_del(netdev, ctx,
TLS_OFFLOAD_CTX_DIR_RX);
+ set_bit(TLS_RX_DEV_CLOSED, &ctx->flags);
+ }
dev_put(netdev);
diff --git a/net/tls/tls_main.c b/net/tls/tls_main.c
index f7c369714b85..f7fe6676cc4c 100644
--- a/net/tls/tls_main.c
+++ b/net/tls/tls_main.c
@@ -711,68 +711,68 @@ static int do_tls_setsockopt_conf(struct sock *sk, sockptr_t optval,
}
if (tx) {
- /* HW rekey not yet supported */
- if (update && ctx->tx_conf == TLS_HW) {
- rc = -EOPNOTSUPP;
- goto err_crypto_info;
- }
-
- /* Only try HW offload on initial setup, not rekey */
- if (!update) {
- rc = tls_set_device_offload(sk);
- conf = TLS_HW;
- if (!rc) {
+ rc = tls_set_device_offload(sk, update ? crypto_info : NULL);
+ conf = TLS_HW;
+ if (!rc) {
+ if (update) {
+ TLS_INC_STATS(sock_net(sk), LINUX_MIB_TLSTXREKEYOK);
+ } else {
TLS_INC_STATS(sock_net(sk), LINUX_MIB_TLSTXDEVICE);
TLS_INC_STATS(sock_net(sk), LINUX_MIB_TLSCURRTXDEVICE);
- goto out;
}
- }
-
- rc = tls_set_sw_offload(sk, 1, update ? crypto_info : NULL);
- if (rc)
+ } else if (update && ctx->tx_conf == TLS_HW) {
+ /* HW rekey failed - return the actual error.
+ * Cannot fall back to SW for an existing HW connection.
+ */
goto err_crypto_info;
-
- if (update) {
- TLS_INC_STATS(sock_net(sk), LINUX_MIB_TLSTXREKEYOK);
} else {
- TLS_INC_STATS(sock_net(sk), LINUX_MIB_TLSTXSW);
- TLS_INC_STATS(sock_net(sk), LINUX_MIB_TLSCURRTXSW);
+ rc = tls_set_sw_offload(sk, 1,
+ update ? crypto_info : NULL);
+ if (rc)
+ goto err_crypto_info;
+
+ if (update) {
+ TLS_INC_STATS(sock_net(sk), LINUX_MIB_TLSTXREKEYOK);
+ } else {
+ TLS_INC_STATS(sock_net(sk), LINUX_MIB_TLSTXSW);
+ TLS_INC_STATS(sock_net(sk), LINUX_MIB_TLSCURRTXSW);
+ }
+ conf = TLS_SW;
}
- conf = TLS_SW;
} else {
- /* HW rekey not yet supported */
- if (update && ctx->rx_conf == TLS_HW) {
- rc = -EOPNOTSUPP;
- goto err_crypto_info;
- }
-
- /* Only try HW offload on initial setup, not rekey */
- if (!update) {
- rc = tls_set_device_offload_rx(sk, ctx);
- conf = TLS_HW;
- if (!rc) {
+ rc = tls_set_device_offload_rx(sk, ctx,
+ update ? crypto_info : NULL);
+ conf = TLS_HW;
+ if (!rc) {
+ if (update) {
+ TLS_INC_STATS(sock_net(sk), LINUX_MIB_TLSRXREKEYOK);
+ } else {
TLS_INC_STATS(sock_net(sk), LINUX_MIB_TLSRXDEVICE);
TLS_INC_STATS(sock_net(sk), LINUX_MIB_TLSCURRRXDEVICE);
- tls_sw_strparser_arm(sk, ctx);
- goto out;
}
- }
-
- rc = tls_set_sw_offload(sk, 0, update ? crypto_info : NULL);
- if (rc)
+ } else if (update && ctx->rx_conf == TLS_HW) {
+ /* HW rekey failed - return the actual error.
+ * Cannot fall back to SW for an existing HW connection.
+ */
goto err_crypto_info;
-
- if (update) {
- TLS_INC_STATS(sock_net(sk), LINUX_MIB_TLSRXREKEYOK);
} else {
- TLS_INC_STATS(sock_net(sk), LINUX_MIB_TLSRXSW);
- TLS_INC_STATS(sock_net(sk), LINUX_MIB_TLSCURRRXSW);
- tls_sw_strparser_arm(sk, ctx);
+ rc = tls_set_sw_offload(sk, 0,
+ update ? crypto_info : NULL);
+ if (rc)
+ goto err_crypto_info;
+
+ if (update) {
+ TLS_INC_STATS(sock_net(sk), LINUX_MIB_TLSRXREKEYOK);
+ } else {
+ TLS_INC_STATS(sock_net(sk), LINUX_MIB_TLSRXSW);
+ TLS_INC_STATS(sock_net(sk), LINUX_MIB_TLSCURRRXSW);
+ }
+ conf = TLS_SW;
}
- conf = TLS_SW;
+ if (!update)
+ tls_sw_strparser_arm(sk, ctx);
}
-out:
if (tx)
ctx->tx_conf = conf;
else
diff --git a/net/tls/tls_sw.c b/net/tls/tls_sw.c
index 9937d4c810f2..2fcc0178490d 100644
--- a/net/tls/tls_sw.c
+++ b/net/tls/tls_sw.c
@@ -2775,20 +2775,19 @@ static void tls_finish_key_update(struct sock *sk, struct tls_context *tls_ctx)
ctx->saved_data_ready(sk);
}
-int tls_set_sw_offload(struct sock *sk, int tx,
- struct tls_crypto_info *new_crypto_info)
+int tls_sw_ctx_init(struct sock *sk, int tx,
+ struct tls_crypto_info *new_crypto_info)
{
struct tls_crypto_info *crypto_info, *src_crypto_info;
struct tls_sw_context_tx *sw_ctx_tx = NULL;
struct tls_sw_context_rx *sw_ctx_rx = NULL;
const struct tls_cipher_desc *cipher_desc;
- char *iv, *rec_seq, *key, *salt;
- struct cipher_context *cctx;
struct tls_prot_info *prot;
struct crypto_aead **aead;
struct tls_context *ctx;
struct crypto_tfm *tfm;
int rc = 0;
+ char *key;
ctx = tls_get_ctx(sk);
prot = &ctx->prot_info;
@@ -2809,12 +2808,10 @@ int tls_set_sw_offload(struct sock *sk, int tx,
if (tx) {
sw_ctx_tx = ctx->priv_ctx_tx;
crypto_info = &ctx->crypto_send.info;
- cctx = &ctx->tx;
aead = &sw_ctx_tx->aead_send;
} else {
sw_ctx_rx = ctx->priv_ctx_rx;
crypto_info = &ctx->crypto_recv.info;
- cctx = &ctx->rx;
aead = &sw_ctx_rx->aead_recv;
}
@@ -2830,10 +2827,7 @@ int tls_set_sw_offload(struct sock *sk, int tx,
if (rc)
goto free_priv;
- iv = crypto_info_iv(src_crypto_info, cipher_desc);
key = crypto_info_key(src_crypto_info, cipher_desc);
- salt = crypto_info_salt(src_crypto_info, cipher_desc);
- rec_seq = crypto_info_rec_seq(src_crypto_info, cipher_desc);
if (!*aead) {
*aead = crypto_alloc_aead(cipher_desc->cipher_name, 0, 0);
@@ -2877,19 +2871,6 @@ int tls_set_sw_offload(struct sock *sk, int tx,
goto free_aead;
}
- memcpy(cctx->iv, salt, cipher_desc->salt);
- memcpy(cctx->iv + cipher_desc->salt, iv, cipher_desc->iv);
- memcpy(cctx->rec_seq, rec_seq, cipher_desc->rec_seq);
-
- if (new_crypto_info) {
- unsafe_memcpy(crypto_info, new_crypto_info,
- cipher_desc->crypto_info,
- /* size was checked in do_tls_setsockopt_conf */);
- memzero_explicit(new_crypto_info, cipher_desc->crypto_info);
- if (!tx)
- tls_finish_key_update(sk, ctx);
- }
-
goto out;
free_aead:
@@ -2908,3 +2889,55 @@ int tls_set_sw_offload(struct sock *sk, int tx,
out:
return rc;
}
+
+void tls_sw_ctx_finalize(struct sock *sk, int tx,
+ struct tls_crypto_info *new_crypto_info)
+{
+ struct tls_crypto_info *crypto_info, *src_crypto_info;
+ const struct tls_cipher_desc *cipher_desc;
+ struct tls_context *ctx = tls_get_ctx(sk);
+ struct cipher_context *cctx;
+ char *iv, *salt, *rec_seq;
+
+ if (tx) {
+ crypto_info = &ctx->crypto_send.info;
+ cctx = &ctx->tx;
+ } else {
+ crypto_info = &ctx->crypto_recv.info;
+ cctx = &ctx->rx;
+ }
+
+ src_crypto_info = new_crypto_info ?: crypto_info;
+ cipher_desc = get_cipher_desc(src_crypto_info->cipher_type);
+
+ iv = crypto_info_iv(src_crypto_info, cipher_desc);
+ salt = crypto_info_salt(src_crypto_info, cipher_desc);
+ rec_seq = crypto_info_rec_seq(src_crypto_info, cipher_desc);
+
+ memcpy(cctx->iv, salt, cipher_desc->salt);
+ memcpy(cctx->iv + cipher_desc->salt, iv, cipher_desc->iv);
+ memcpy(cctx->rec_seq, rec_seq, cipher_desc->rec_seq);
+
+ if (new_crypto_info) {
+ unsafe_memcpy(crypto_info, new_crypto_info,
+ cipher_desc->crypto_info,
+ /* size was checked in do_tls_setsockopt_conf */);
+ memzero_explicit(new_crypto_info, cipher_desc->crypto_info);
+
+ if (!tx)
+ tls_finish_key_update(sk, ctx);
+ }
+}
+
+int tls_set_sw_offload(struct sock *sk, int tx,
+ struct tls_crypto_info *new_crypto_info)
+{
+ int rc;
+
+ rc = tls_sw_ctx_init(sk, tx, new_crypto_info);
+ if (rc)
+ return rc;
+
+ tls_sw_ctx_finalize(sk, tx, new_crypto_info);
+ return 0;
+}
--
2.25.1
^ permalink raw reply related [flat|nested] 12+ messages in thread
* [PATCH v4 3/3] mlx5: TLS 1.3 hardware offload support
2026-01-21 21:57 [PATCH v4 0/3] tls: Add TLS 1.3 hardware offload support Rishikesh Jethwani
2026-01-21 21:57 ` [PATCH v4 1/3] tls: add " Rishikesh Jethwani
2026-01-21 21:57 ` [PATCH v4 2/3] tls: add hardware offload key update support Rishikesh Jethwani
@ 2026-01-21 21:57 ` Rishikesh Jethwani
2026-01-22 11:30 ` [PATCH v4 0/3] tls: Add " Tariq Toukan
3 siblings, 0 replies; 12+ messages in thread
From: Rishikesh Jethwani @ 2026-01-21 21:57 UTC (permalink / raw)
To: netdev
Cc: saeedm, tariqt, mbloch, borisp, john.fastabend, kuba, sd, davem,
pabeni, edumazet, leon, Rishikesh Jethwani
Add TLS 1.3 hardware offload support to mlx5 driver, enabling both
TX and RX hardware acceleration for TLS 1.3 connections on Mellanox
ConnectX-6 Dx and newer adapters.
This patch enables:
- TLS 1.3 version detection and validation with proper capability
checking
- TLS 1.3 crypto context configuration using
MLX5E_STATIC_PARAMS_CONTEXT_TLS_1_3 (0x3)
- Correct IV handling for TLS 1.3 (12-byte IV vs TLS 1.2's 4-byte salt)
- Hardware offload for both TLS 1.3 AES-GCM-128 and AES-GCM-256 cipher
suites
Key differences from TLS 1.2:
- TLS 1.2: Only 4-byte salt copied to gcm_iv, explicit IV in each record
- TLS 1.3: Full 12-byte IV (salt + iv) copied to gcm_iv + implicit_iv
* salt (4 bytes) → gcm_iv[0:3]
* iv (8 bytes) → gcm_iv[4:7] + implicit_iv[0:3]
* Note: gcm_iv and implicit_iv are contiguous in memory
The EXTRACT_INFO_FIELDS macro is updated to also extract the 'iv' field
which is needed for TLS 1.3.
Tested on Mellanox ConnectX-6 Dx (Crypto Enabled) with TLS 1.3 AES-GCM-128
and AES-GCM-256 cipher suites.
Signed-off-by: Rishikesh Jethwani <rjethwani@purestorage.com>
---
.../ethernet/mellanox/mlx5/core/en_accel/ktls.h | 8 +++++++-
.../mellanox/mlx5/core/en_accel/ktls_txrx.c | 14 +++++++++++---
2 files changed, 18 insertions(+), 4 deletions(-)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls.h b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls.h
index 07a04a142a2e..0469ca6a0762 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls.h
@@ -30,7 +30,9 @@ static inline bool mlx5e_is_ktls_device(struct mlx5_core_dev *mdev)
return false;
return (MLX5_CAP_TLS(mdev, tls_1_2_aes_gcm_128) ||
- MLX5_CAP_TLS(mdev, tls_1_2_aes_gcm_256));
+ MLX5_CAP_TLS(mdev, tls_1_2_aes_gcm_256) ||
+ MLX5_CAP_TLS(mdev, tls_1_3_aes_gcm_128) ||
+ MLX5_CAP_TLS(mdev, tls_1_3_aes_gcm_256));
}
static inline bool mlx5e_ktls_type_check(struct mlx5_core_dev *mdev,
@@ -40,10 +42,14 @@ static inline bool mlx5e_ktls_type_check(struct mlx5_core_dev *mdev,
case TLS_CIPHER_AES_GCM_128:
if (crypto_info->version == TLS_1_2_VERSION)
return MLX5_CAP_TLS(mdev, tls_1_2_aes_gcm_128);
+ else if (crypto_info->version == TLS_1_3_VERSION)
+ return MLX5_CAP_TLS(mdev, tls_1_3_aes_gcm_128);
break;
case TLS_CIPHER_AES_GCM_256:
if (crypto_info->version == TLS_1_2_VERSION)
return MLX5_CAP_TLS(mdev, tls_1_2_aes_gcm_256);
+ else if (crypto_info->version == TLS_1_3_VERSION)
+ return MLX5_CAP_TLS(mdev, tls_1_3_aes_gcm_256);
break;
}
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls_txrx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls_txrx.c
index 570a912dd6fa..f3f90ad6c6cf 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls_txrx.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls_txrx.c
@@ -6,6 +6,7 @@
enum {
MLX5E_STATIC_PARAMS_CONTEXT_TLS_1_2 = 0x2,
+ MLX5E_STATIC_PARAMS_CONTEXT_TLS_1_3 = 0x3,
};
enum {
@@ -15,8 +16,10 @@ enum {
#define EXTRACT_INFO_FIELDS do { \
salt = info->salt; \
rec_seq = info->rec_seq; \
+ iv = info->iv; \
salt_sz = sizeof(info->salt); \
rec_seq_sz = sizeof(info->rec_seq); \
+ iv_sz = sizeof(info->iv); \
} while (0)
static void
@@ -25,8 +28,8 @@ fill_static_params(struct mlx5_wqe_tls_static_params_seg *params,
u32 key_id, u32 resync_tcp_sn)
{
char *initial_rn, *gcm_iv;
- u16 salt_sz, rec_seq_sz;
- char *salt, *rec_seq;
+ u16 salt_sz, rec_seq_sz, iv_sz;
+ char *salt, *rec_seq, *iv;
u8 tls_version;
u8 *ctx;
@@ -59,7 +62,12 @@ fill_static_params(struct mlx5_wqe_tls_static_params_seg *params,
memcpy(gcm_iv, salt, salt_sz);
memcpy(initial_rn, rec_seq, rec_seq_sz);
- tls_version = MLX5E_STATIC_PARAMS_CONTEXT_TLS_1_2;
+ if (crypto_info->crypto_info.version == TLS_1_3_VERSION) {
+ memcpy(gcm_iv + salt_sz, iv, iv_sz);
+ tls_version = MLX5E_STATIC_PARAMS_CONTEXT_TLS_1_3;
+ } else {
+ tls_version = MLX5E_STATIC_PARAMS_CONTEXT_TLS_1_2;
+ }
MLX5_SET(tls_static_params, ctx, tls_version, tls_version);
MLX5_SET(tls_static_params, ctx, const_1, 1);
--
2.25.1
^ permalink raw reply related [flat|nested] 12+ messages in thread
* Re: [PATCH v4 2/3] tls: add hardware offload key update support
2026-01-21 21:57 ` [PATCH v4 2/3] tls: add hardware offload key update support Rishikesh Jethwani
@ 2026-01-22 4:09 ` kernel test robot
2026-01-22 5:15 ` kernel test robot
` (2 subsequent siblings)
3 siblings, 0 replies; 12+ messages in thread
From: kernel test robot @ 2026-01-22 4:09 UTC (permalink / raw)
To: Rishikesh Jethwani, netdev
Cc: llvm, oe-kbuild-all, saeedm, tariqt, mbloch, borisp,
john.fastabend, kuba, sd, davem, pabeni, edumazet, leon,
Rishikesh Jethwani
Hi Rishikesh,
kernel test robot noticed the following build warnings:
[auto build test WARNING on linus/master]
[also build test WARNING on v6.19-rc6 next-20260121]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Rishikesh-Jethwani/tls-add-TLS-1-3-hardware-offload-support/20260122-060724
base: linus/master
patch link: https://lore.kernel.org/r/20260121215727.3994324-3-rjethwani%40purestorage.com
patch subject: [PATCH v4 2/3] tls: add hardware offload key update support
config: x86_64-randconfig-071-20260122 (https://download.01.org/0day-ci/archive/20260122/202601221106.NuxFLP3L-lkp@intel.com/config)
compiler: clang version 20.1.8 (https://github.com/llvm/llvm-project 87f0227cb60147a26a1eeb4fb06e3b505e9c7261)
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20260122/202601221106.NuxFLP3L-lkp@intel.com/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp@intel.com>
| Closes: https://lore.kernel.org/oe-kbuild-all/202601221106.NuxFLP3L-lkp@intel.com/
All warnings (new ones prefixed by >>):
>> net/tls/tls_device.c:1249:4: warning: 'volatile' qualifier on function type 'typeof (*sk->sk_validate_xmit_skb)' (aka 'struct sk_buff *(struct sock *, struct net_device *, struct sk_buff *)') has no effect and is a Clang extension [-Wignored-qualifiers]
1249 | smp_store_release(sk->sk_validate_xmit_skb,
| ^
include/asm-generic/barrier.h:172:55: note: expanded from macro 'smp_store_release'
172 | #define smp_store_release(p, v) do { kcsan_release(); __smp_store_release(p, v); } while (0)
| ^
arch/x86/include/asm/barrier.h:63:2: note: expanded from macro '__smp_store_release'
63 | WRITE_ONCE(*p, v); \
| ^
include/asm-generic/rwonce.h:61:2: note: expanded from macro 'WRITE_ONCE'
61 | __WRITE_ONCE(x, val); \
| ^
include/asm-generic/rwonce.h:55:4: note: expanded from macro '__WRITE_ONCE'
55 | *(volatile typeof(x) *)&(x) = (val); \
| ^
net/tls/tls_device.c:1249:4: error: non-object type 'typeof (*sk->sk_validate_xmit_skb)' (aka 'struct sk_buff *(struct sock *, struct net_device *, struct sk_buff *)') is not assignable
1249 | smp_store_release(sk->sk_validate_xmit_skb,
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1250 | tls_validate_xmit_skb_sw);
| ~~~~~~~~~~~~~~~~~~~~~~~~~
include/asm-generic/barrier.h:172:55: note: expanded from macro 'smp_store_release'
172 | #define smp_store_release(p, v) do { kcsan_release(); __smp_store_release(p, v); } while (0)
| ^~~~~~~~~~~~~~~~~~~~~~~~~
arch/x86/include/asm/barrier.h:63:2: note: expanded from macro '__smp_store_release'
63 | WRITE_ONCE(*p, v); \
| ^~~~~~~~~~~~~~~~~
include/asm-generic/rwonce.h:61:2: note: expanded from macro 'WRITE_ONCE'
61 | __WRITE_ONCE(x, val); \
| ^~~~~~~~~~~~~~~~~~~~
include/asm-generic/rwonce.h:55:30: note: expanded from macro '__WRITE_ONCE'
55 | *(volatile typeof(x) *)&(x) = (val); \
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~ ^
1 warning and 1 error generated.
vim +1249 net/tls/tls_device.c
1086
1087 int tls_set_device_offload(struct sock *sk,
1088 struct tls_crypto_info *new_crypto_info)
1089 {
1090 struct tls_crypto_info *crypto_info, *src_crypto_info;
1091 struct tls_record_info *start_marker_record;
1092 struct tls_offload_context_tx *offload_ctx;
1093 const struct tls_cipher_desc *cipher_desc;
1094 struct tls_prot_info *prot;
1095 struct net_device *netdev;
1096 struct tls_context *ctx;
1097 char *iv, *rec_seq;
1098 int rc;
1099
1100 ctx = tls_get_ctx(sk);
1101 prot = &ctx->prot_info;
1102
1103 /* Rekey is only supported for connections that are already
1104 * using HW offload. For SW offload connections, the caller
1105 * should fall back to tls_set_sw_offload() for rekey.
1106 */
1107 if (new_crypto_info && ctx->tx_conf != TLS_HW)
1108 return -EINVAL;
1109
1110 netdev = get_netdev_for_sock(sk);
1111 if (!netdev) {
1112 pr_err_ratelimited("%s: netdev not found\n", __func__);
1113 return -EINVAL;
1114 }
1115
1116 if (!(netdev->features & NETIF_F_HW_TLS_TX)) {
1117 rc = -EOPNOTSUPP;
1118 goto release_netdev;
1119 }
1120
1121 crypto_info = &ctx->crypto_send.info;
1122 src_crypto_info = new_crypto_info ?: crypto_info;
1123 if (src_crypto_info->version != TLS_1_2_VERSION &&
1124 src_crypto_info->version != TLS_1_3_VERSION) {
1125 rc = -EOPNOTSUPP;
1126 goto release_netdev;
1127 }
1128
1129 cipher_desc = get_cipher_desc(src_crypto_info->cipher_type);
1130 if (!cipher_desc || !cipher_desc->offloadable) {
1131 rc = -EINVAL;
1132 goto release_netdev;
1133 }
1134
1135 iv = crypto_info_iv(src_crypto_info, cipher_desc);
1136 rec_seq = crypto_info_rec_seq(src_crypto_info, cipher_desc);
1137
1138 if (!new_crypto_info) {
1139 rc = init_prot_info(prot, src_crypto_info, cipher_desc);
1140 if (rc)
1141 goto release_netdev;
1142
1143 memcpy(ctx->tx.iv + cipher_desc->salt, iv, cipher_desc->iv);
1144 memcpy(ctx->tx.rec_seq, rec_seq, cipher_desc->rec_seq);
1145
1146 start_marker_record = kmalloc(sizeof(*start_marker_record),
1147 GFP_KERNEL);
1148 if (!start_marker_record) {
1149 rc = -ENOMEM;
1150 goto release_netdev;
1151 }
1152
1153 offload_ctx = alloc_offload_ctx_tx(ctx);
1154 if (!offload_ctx) {
1155 rc = -ENOMEM;
1156 goto free_marker_record;
1157 }
1158
1159 rc = tls_sw_fallback_init(sk, offload_ctx, src_crypto_info);
1160 if (rc)
1161 goto free_offload_ctx;
1162
1163 start_marker_record->end_seq = tcp_sk(sk)->write_seq;
1164 start_marker_record->len = 0;
1165 start_marker_record->num_frags = 0;
1166 list_add_tail(&start_marker_record->list,
1167 &offload_ctx->records_list);
1168
1169 clean_acked_data_enable(tcp_sk(sk), &tls_tcp_clean_acked);
1170 ctx->push_pending_record = tls_device_push_pending_record;
1171
1172 /* TLS offload is greatly simplified if we don't send
1173 * SKBs where only part of the payload needs to be encrypted.
1174 * So mark the last skb in the write queue as end of record.
1175 */
1176 tcp_write_collapse_fence(sk);
1177 }
1178
1179 /* Avoid offloading if the device is down
1180 * We don't want to offload new flows after
1181 * the NETDEV_DOWN event
1182 *
1183 * device_offload_lock is taken in tls_devices's NETDEV_DOWN
1184 * handler thus protecting from the device going down before
1185 * ctx was added to tls_device_list.
1186 */
1187 down_read(&device_offload_lock);
1188 if (!(netdev->flags & IFF_UP)) {
1189 rc = -EINVAL;
1190 goto release_lock;
1191 }
1192
1193 if (!new_crypto_info) {
1194 ctx->priv_ctx_tx = offload_ctx;
1195 } else {
1196 char *key = crypto_info_key(src_crypto_info, cipher_desc);
1197
1198 offload_ctx = tls_offload_ctx_tx(ctx);
1199
1200 rc = crypto_aead_setkey(offload_ctx->aead_send, key,
1201 cipher_desc->key);
1202 if (rc)
1203 goto release_lock;
1204
1205 /* For rekey, delete old HW context before adding new one. */
1206 if (!test_bit(TLS_TX_DEV_CLOSED, &ctx->flags))
1207 netdev->tlsdev_ops->tls_dev_del(netdev, ctx,
1208 TLS_OFFLOAD_CTX_DIR_TX);
1209 }
1210
1211 rc = netdev->tlsdev_ops->tls_dev_add(netdev, sk, TLS_OFFLOAD_CTX_DIR_TX,
1212 src_crypto_info,
1213 tcp_sk(sk)->write_seq);
1214 trace_tls_device_offload_set(sk, TLS_OFFLOAD_CTX_DIR_TX,
1215 tcp_sk(sk)->write_seq, rec_seq, rc);
1216
1217 if (new_crypto_info) {
1218 unsigned long flags;
1219 __be64 rcd_sn;
1220
1221 memcpy(ctx->tx.iv + cipher_desc->salt, iv, cipher_desc->iv);
1222 memcpy(ctx->tx.rec_seq, rec_seq, cipher_desc->rec_seq);
1223
1224 spin_lock_irqsave(&offload_ctx->lock, flags);
1225 /* Delete old records, can't be retransmitted with new key */
1226 delete_all_records(offload_ctx);
1227
1228 /* Update unacked_record_sn for the new key's rec_seq.
1229 * This is critical for SW fallback encryption to use
1230 * the correct record sequence number after rekey.
1231 */
1232 memcpy(&rcd_sn, rec_seq, sizeof(rcd_sn));
1233 offload_ctx->unacked_record_sn = be64_to_cpu(rcd_sn);
1234 spin_unlock_irqrestore(&offload_ctx->lock, flags);
1235
1236 unsafe_memcpy(crypto_info, new_crypto_info,
1237 cipher_desc->crypto_info,
1238 /* size was checked in do_tls_setsockopt_conf */);
1239 memzero_explicit(new_crypto_info, cipher_desc->crypto_info);
1240 }
1241
1242 if (rc) {
1243 if (new_crypto_info) {
1244 /* HW rekey failed, gracefully degrade to SW encryption.
1245 * SW fallback already has new key, IV, and rec_seq.
1246 * Old HW ctx was deleted, continue with SW encryption.
1247 */
1248 set_bit(TLS_TX_DEV_CLOSED, &ctx->flags);
> 1249 smp_store_release(sk->sk_validate_xmit_skb,
1250 tls_validate_xmit_skb_sw);
1251 } else {
1252 goto release_lock;
1253 }
1254 } else {
1255 if (new_crypto_info)
1256 clear_bit(TLS_TX_DEV_CLOSED, &ctx->flags);
1257
1258 tls_device_attach(ctx, sk, netdev);
1259
1260 /* following this assignment tls_is_skb_tx_device_offloaded
1261 * will return true and the context might be accessed
1262 * by the netdev's xmit function.
1263 */
1264 smp_store_release(&sk->sk_validate_xmit_skb,
1265 tls_validate_xmit_skb);
1266 }
1267
1268 up_read(&device_offload_lock);
1269
1270 dev_put(netdev);
1271
1272 return 0;
1273
1274 release_lock:
1275 up_read(&device_offload_lock);
1276 if (new_crypto_info)
1277 goto release_netdev;
1278 clean_acked_data_disable(tcp_sk(sk));
1279 crypto_free_aead(offload_ctx->aead_send);
1280 free_offload_ctx:
1281 kfree(offload_ctx);
1282 ctx->priv_ctx_tx = NULL;
1283 free_marker_record:
1284 kfree(start_marker_record);
1285 release_netdev:
1286 dev_put(netdev);
1287 return rc;
1288 }
1289
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH v4 2/3] tls: add hardware offload key update support
2026-01-21 21:57 ` [PATCH v4 2/3] tls: add hardware offload key update support Rishikesh Jethwani
2026-01-22 4:09 ` kernel test robot
@ 2026-01-22 5:15 ` kernel test robot
2026-01-22 7:19 ` kernel test robot
2026-01-22 7:54 ` kernel test robot
3 siblings, 0 replies; 12+ messages in thread
From: kernel test robot @ 2026-01-22 5:15 UTC (permalink / raw)
To: Rishikesh Jethwani, netdev
Cc: oe-kbuild-all, saeedm, tariqt, mbloch, borisp, john.fastabend,
kuba, sd, davem, pabeni, edumazet, leon, Rishikesh Jethwani
Hi Rishikesh,
kernel test robot noticed the following build errors:
[auto build test ERROR on linus/master]
[also build test ERROR on v6.19-rc6 next-20260121]
[cannot apply to horms-ipvs/master]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Rishikesh-Jethwani/tls-add-TLS-1-3-hardware-offload-support/20260122-060724
base: linus/master
patch link: https://lore.kernel.org/r/20260121215727.3994324-3-rjethwani%40purestorage.com
patch subject: [PATCH v4 2/3] tls: add hardware offload key update support
config: parisc-allmodconfig (https://download.01.org/0day-ci/archive/20260122/202601221215.VSoxPBxE-lkp@intel.com/config)
compiler: hppa-linux-gcc (GCC) 15.2.0
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20260122/202601221215.VSoxPBxE-lkp@intel.com/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp@intel.com>
| Closes: https://lore.kernel.org/oe-kbuild-all/202601221215.VSoxPBxE-lkp@intel.com/
All errors (new ones prefixed by >>):
In file included from arch/parisc/include/asm/atomic.h:11,
from include/linux/atomic.h:7,
from include/crypto/aead.h:11,
from net/tls/tls_device.c:32:
net/tls/tls_device.c: In function 'tls_set_device_offload':
>> arch/parisc/include/asm/barrier.h:36:28: error: field '__val' declared as a function
36 | union { typeof(*p) __val; char __c[1]; } __u = \
| ^~~~~
include/asm-generic/barrier.h:172:55: note: in expansion of macro '__smp_store_release'
172 | #define smp_store_release(p, v) do { kcsan_release(); __smp_store_release(p, v); } while (0)
| ^~~~~~~~~~~~~~~~~~~
net/tls/tls_device.c:1249:25: note: in expansion of macro 'smp_store_release'
1249 | smp_store_release(sk->sk_validate_xmit_skb,
| ^~~~~~~~~~~~~~~~~
>> arch/parisc/include/asm/barrier.h:37:28: error: cast specifies function type
37 | { .__val = (__force typeof(*p)) (v) }; \
| ^
include/asm-generic/barrier.h:172:55: note: in expansion of macro '__smp_store_release'
172 | #define smp_store_release(p, v) do { kcsan_release(); __smp_store_release(p, v); } while (0)
| ^~~~~~~~~~~~~~~~~~~
net/tls/tls_device.c:1249:25: note: in expansion of macro 'smp_store_release'
1249 | smp_store_release(sk->sk_validate_xmit_skb,
| ^~~~~~~~~~~~~~~~~
--
In file included from arch/parisc/include/asm/atomic.h:11,
from include/linux/atomic.h:7,
from include/crypto/aead.h:11,
from tls_device.c:32:
tls_device.c: In function 'tls_set_device_offload':
>> arch/parisc/include/asm/barrier.h:36:28: error: field '__val' declared as a function
36 | union { typeof(*p) __val; char __c[1]; } __u = \
| ^~~~~
include/asm-generic/barrier.h:172:55: note: in expansion of macro '__smp_store_release'
172 | #define smp_store_release(p, v) do { kcsan_release(); __smp_store_release(p, v); } while (0)
| ^~~~~~~~~~~~~~~~~~~
tls_device.c:1249:25: note: in expansion of macro 'smp_store_release'
1249 | smp_store_release(sk->sk_validate_xmit_skb,
| ^~~~~~~~~~~~~~~~~
>> arch/parisc/include/asm/barrier.h:37:28: error: cast specifies function type
37 | { .__val = (__force typeof(*p)) (v) }; \
| ^
include/asm-generic/barrier.h:172:55: note: in expansion of macro '__smp_store_release'
172 | #define smp_store_release(p, v) do { kcsan_release(); __smp_store_release(p, v); } while (0)
| ^~~~~~~~~~~~~~~~~~~
tls_device.c:1249:25: note: in expansion of macro 'smp_store_release'
1249 | smp_store_release(sk->sk_validate_xmit_skb,
| ^~~~~~~~~~~~~~~~~
vim +/__val +36 arch/parisc/include/asm/barrier.h
fedb8da96355f5f John David Anglin 2018-08-05 32
e96ebd589debd9a John David Anglin 2020-07-30 33 #define __smp_store_release(p, v) \
e96ebd589debd9a John David Anglin 2020-07-30 34 do { \
e96ebd589debd9a John David Anglin 2020-07-30 35 typeof(p) __p = (p); \
e96ebd589debd9a John David Anglin 2020-07-30 @36 union { typeof(*p) __val; char __c[1]; } __u = \
e96ebd589debd9a John David Anglin 2020-07-30 @37 { .__val = (__force typeof(*p)) (v) }; \
e96ebd589debd9a John David Anglin 2020-07-30 38 compiletime_assert_atomic_type(*p); \
e96ebd589debd9a John David Anglin 2020-07-30 39 switch (sizeof(*p)) { \
e96ebd589debd9a John David Anglin 2020-07-30 40 case 1: \
e96ebd589debd9a John David Anglin 2020-07-30 41 asm volatile("stb,ma %0,0(%1)" \
e96ebd589debd9a John David Anglin 2020-07-30 42 : : "r"(*(__u8 *)__u.__c), "r"(__p) \
e96ebd589debd9a John David Anglin 2020-07-30 43 : "memory"); \
e96ebd589debd9a John David Anglin 2020-07-30 44 break; \
e96ebd589debd9a John David Anglin 2020-07-30 45 case 2: \
e96ebd589debd9a John David Anglin 2020-07-30 46 asm volatile("sth,ma %0,0(%1)" \
e96ebd589debd9a John David Anglin 2020-07-30 47 : : "r"(*(__u16 *)__u.__c), "r"(__p) \
e96ebd589debd9a John David Anglin 2020-07-30 48 : "memory"); \
e96ebd589debd9a John David Anglin 2020-07-30 49 break; \
e96ebd589debd9a John David Anglin 2020-07-30 50 case 4: \
e96ebd589debd9a John David Anglin 2020-07-30 51 asm volatile("stw,ma %0,0(%1)" \
e96ebd589debd9a John David Anglin 2020-07-30 52 : : "r"(*(__u32 *)__u.__c), "r"(__p) \
e96ebd589debd9a John David Anglin 2020-07-30 53 : "memory"); \
e96ebd589debd9a John David Anglin 2020-07-30 54 break; \
e96ebd589debd9a John David Anglin 2020-07-30 55 case 8: \
e96ebd589debd9a John David Anglin 2020-07-30 56 if (IS_ENABLED(CONFIG_64BIT)) \
e96ebd589debd9a John David Anglin 2020-07-30 57 asm volatile("std,ma %0,0(%1)" \
e96ebd589debd9a John David Anglin 2020-07-30 58 : : "r"(*(__u64 *)__u.__c), "r"(__p) \
e96ebd589debd9a John David Anglin 2020-07-30 59 : "memory"); \
e96ebd589debd9a John David Anglin 2020-07-30 60 break; \
e96ebd589debd9a John David Anglin 2020-07-30 61 } \
e96ebd589debd9a John David Anglin 2020-07-30 62 } while (0)
e96ebd589debd9a John David Anglin 2020-07-30 63
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH v4 2/3] tls: add hardware offload key update support
2026-01-21 21:57 ` [PATCH v4 2/3] tls: add hardware offload key update support Rishikesh Jethwani
2026-01-22 4:09 ` kernel test robot
2026-01-22 5:15 ` kernel test robot
@ 2026-01-22 7:19 ` kernel test robot
2026-01-22 7:54 ` kernel test robot
3 siblings, 0 replies; 12+ messages in thread
From: kernel test robot @ 2026-01-22 7:19 UTC (permalink / raw)
To: Rishikesh Jethwani, netdev
Cc: oe-kbuild-all, saeedm, tariqt, mbloch, borisp, john.fastabend,
kuba, sd, davem, pabeni, edumazet, leon, Rishikesh Jethwani
Hi Rishikesh,
kernel test robot noticed the following build errors:
[auto build test ERROR on linus/master]
[also build test ERROR on v6.19-rc6 next-20260121]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Rishikesh-Jethwani/tls-add-TLS-1-3-hardware-offload-support/20260122-060724
base: linus/master
patch link: https://lore.kernel.org/r/20260121215727.3994324-3-rjethwani%40purestorage.com
patch subject: [PATCH v4 2/3] tls: add hardware offload key update support
config: x86_64-rhel-9.4 (https://download.01.org/0day-ci/archive/20260122/202601220850.8AbmdIiS-lkp@intel.com/config)
compiler: gcc-14 (Debian 14.2.0-19) 14.2.0
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20260122/202601220850.8AbmdIiS-lkp@intel.com/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp@intel.com>
| Closes: https://lore.kernel.org/oe-kbuild-all/202601220850.8AbmdIiS-lkp@intel.com/
All errors (new ones prefixed by >>):
In file included from ./arch/x86/include/generated/asm/rwonce.h:1,
from include/linux/compiler.h:380,
from arch/x86/include/asm/atomic.h:5,
from include/linux/atomic.h:7,
from include/crypto/aead.h:11,
from net/tls/tls_device.c:32:
net/tls/tls_device.c: In function 'tls_set_device_offload':
>> include/asm-generic/rwonce.h:55:37: error: lvalue required as left operand of assignment
55 | *(volatile typeof(x) *)&(x) = (val); \
| ^
include/asm-generic/rwonce.h:61:9: note: in expansion of macro '__WRITE_ONCE'
61 | __WRITE_ONCE(x, val); \
| ^~~~~~~~~~~~
arch/x86/include/asm/barrier.h:63:9: note: in expansion of macro 'WRITE_ONCE'
63 | WRITE_ONCE(*p, v); \
| ^~~~~~~~~~
include/asm-generic/barrier.h:172:55: note: in expansion of macro '__smp_store_release'
172 | #define smp_store_release(p, v) do { kcsan_release(); __smp_store_release(p, v); } while (0)
| ^~~~~~~~~~~~~~~~~~~
net/tls/tls_device.c:1249:25: note: in expansion of macro 'smp_store_release'
1249 | smp_store_release(sk->sk_validate_xmit_skb,
| ^~~~~~~~~~~~~~~~~
--
In file included from arch/x86/include/generated/asm/rwonce.h:1,
from include/linux/compiler.h:380,
from arch/x86/include/asm/atomic.h:5,
from include/linux/atomic.h:7,
from include/crypto/aead.h:11,
from tls_device.c:32:
tls_device.c: In function 'tls_set_device_offload':
>> include/asm-generic/rwonce.h:55:37: error: lvalue required as left operand of assignment
55 | *(volatile typeof(x) *)&(x) = (val); \
| ^
include/asm-generic/rwonce.h:61:9: note: in expansion of macro '__WRITE_ONCE'
61 | __WRITE_ONCE(x, val); \
| ^~~~~~~~~~~~
arch/x86/include/asm/barrier.h:63:9: note: in expansion of macro 'WRITE_ONCE'
63 | WRITE_ONCE(*p, v); \
| ^~~~~~~~~~
include/asm-generic/barrier.h:172:55: note: in expansion of macro '__smp_store_release'
172 | #define smp_store_release(p, v) do { kcsan_release(); __smp_store_release(p, v); } while (0)
| ^~~~~~~~~~~~~~~~~~~
tls_device.c:1249:25: note: in expansion of macro 'smp_store_release'
1249 | smp_store_release(sk->sk_validate_xmit_skb,
| ^~~~~~~~~~~~~~~~~
vim +55 include/asm-generic/rwonce.h
e506ea451254ab Will Deacon 2019-10-15 52
e506ea451254ab Will Deacon 2019-10-15 53 #define __WRITE_ONCE(x, val) \
e506ea451254ab Will Deacon 2019-10-15 54 do { \
e506ea451254ab Will Deacon 2019-10-15 @55 *(volatile typeof(x) *)&(x) = (val); \
e506ea451254ab Will Deacon 2019-10-15 56 } while (0)
e506ea451254ab Will Deacon 2019-10-15 57
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH v4 2/3] tls: add hardware offload key update support
2026-01-21 21:57 ` [PATCH v4 2/3] tls: add hardware offload key update support Rishikesh Jethwani
` (2 preceding siblings ...)
2026-01-22 7:19 ` kernel test robot
@ 2026-01-22 7:54 ` kernel test robot
3 siblings, 0 replies; 12+ messages in thread
From: kernel test robot @ 2026-01-22 7:54 UTC (permalink / raw)
To: Rishikesh Jethwani, netdev
Cc: llvm, oe-kbuild-all, saeedm, tariqt, mbloch, borisp,
john.fastabend, kuba, sd, davem, pabeni, edumazet, leon,
Rishikesh Jethwani
Hi Rishikesh,
kernel test robot noticed the following build errors:
[auto build test ERROR on linus/master]
[also build test ERROR on v6.19-rc6 next-20260121]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Rishikesh-Jethwani/tls-add-TLS-1-3-hardware-offload-support/20260122-060724
base: linus/master
patch link: https://lore.kernel.org/r/20260121215727.3994324-3-rjethwani%40purestorage.com
patch subject: [PATCH v4 2/3] tls: add hardware offload key update support
config: x86_64-randconfig-071-20260122 (https://download.01.org/0day-ci/archive/20260122/202601221546.bR6QG2oX-lkp@intel.com/config)
compiler: clang version 20.1.8 (https://github.com/llvm/llvm-project 87f0227cb60147a26a1eeb4fb06e3b505e9c7261)
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20260122/202601221546.bR6QG2oX-lkp@intel.com/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp@intel.com>
| Closes: https://lore.kernel.org/oe-kbuild-all/202601221546.bR6QG2oX-lkp@intel.com/
All errors (new ones prefixed by >>):
net/tls/tls_device.c:1249:4: warning: 'volatile' qualifier on function type 'typeof (*sk->sk_validate_xmit_skb)' (aka 'struct sk_buff *(struct sock *, struct net_device *, struct sk_buff *)') has no effect and is a Clang extension [-Wignored-qualifiers]
1249 | smp_store_release(sk->sk_validate_xmit_skb,
| ^
include/asm-generic/barrier.h:172:55: note: expanded from macro 'smp_store_release'
172 | #define smp_store_release(p, v) do { kcsan_release(); __smp_store_release(p, v); } while (0)
| ^
arch/x86/include/asm/barrier.h:63:2: note: expanded from macro '__smp_store_release'
63 | WRITE_ONCE(*p, v); \
| ^
include/asm-generic/rwonce.h:61:2: note: expanded from macro 'WRITE_ONCE'
61 | __WRITE_ONCE(x, val); \
| ^
include/asm-generic/rwonce.h:55:4: note: expanded from macro '__WRITE_ONCE'
55 | *(volatile typeof(x) *)&(x) = (val); \
| ^
>> net/tls/tls_device.c:1249:4: error: non-object type 'typeof (*sk->sk_validate_xmit_skb)' (aka 'struct sk_buff *(struct sock *, struct net_device *, struct sk_buff *)') is not assignable
1249 | smp_store_release(sk->sk_validate_xmit_skb,
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1250 | tls_validate_xmit_skb_sw);
| ~~~~~~~~~~~~~~~~~~~~~~~~~
include/asm-generic/barrier.h:172:55: note: expanded from macro 'smp_store_release'
172 | #define smp_store_release(p, v) do { kcsan_release(); __smp_store_release(p, v); } while (0)
| ^~~~~~~~~~~~~~~~~~~~~~~~~
arch/x86/include/asm/barrier.h:63:2: note: expanded from macro '__smp_store_release'
63 | WRITE_ONCE(*p, v); \
| ^~~~~~~~~~~~~~~~~
include/asm-generic/rwonce.h:61:2: note: expanded from macro 'WRITE_ONCE'
61 | __WRITE_ONCE(x, val); \
| ^~~~~~~~~~~~~~~~~~~~
include/asm-generic/rwonce.h:55:30: note: expanded from macro '__WRITE_ONCE'
55 | *(volatile typeof(x) *)&(x) = (val); \
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~ ^
1 warning and 1 error generated.
vim +1249 net/tls/tls_device.c
1086
1087 int tls_set_device_offload(struct sock *sk,
1088 struct tls_crypto_info *new_crypto_info)
1089 {
1090 struct tls_crypto_info *crypto_info, *src_crypto_info;
1091 struct tls_record_info *start_marker_record;
1092 struct tls_offload_context_tx *offload_ctx;
1093 const struct tls_cipher_desc *cipher_desc;
1094 struct tls_prot_info *prot;
1095 struct net_device *netdev;
1096 struct tls_context *ctx;
1097 char *iv, *rec_seq;
1098 int rc;
1099
1100 ctx = tls_get_ctx(sk);
1101 prot = &ctx->prot_info;
1102
1103 /* Rekey is only supported for connections that are already
1104 * using HW offload. For SW offload connections, the caller
1105 * should fall back to tls_set_sw_offload() for rekey.
1106 */
1107 if (new_crypto_info && ctx->tx_conf != TLS_HW)
1108 return -EINVAL;
1109
1110 netdev = get_netdev_for_sock(sk);
1111 if (!netdev) {
1112 pr_err_ratelimited("%s: netdev not found\n", __func__);
1113 return -EINVAL;
1114 }
1115
1116 if (!(netdev->features & NETIF_F_HW_TLS_TX)) {
1117 rc = -EOPNOTSUPP;
1118 goto release_netdev;
1119 }
1120
1121 crypto_info = &ctx->crypto_send.info;
1122 src_crypto_info = new_crypto_info ?: crypto_info;
1123 if (src_crypto_info->version != TLS_1_2_VERSION &&
1124 src_crypto_info->version != TLS_1_3_VERSION) {
1125 rc = -EOPNOTSUPP;
1126 goto release_netdev;
1127 }
1128
1129 cipher_desc = get_cipher_desc(src_crypto_info->cipher_type);
1130 if (!cipher_desc || !cipher_desc->offloadable) {
1131 rc = -EINVAL;
1132 goto release_netdev;
1133 }
1134
1135 iv = crypto_info_iv(src_crypto_info, cipher_desc);
1136 rec_seq = crypto_info_rec_seq(src_crypto_info, cipher_desc);
1137
1138 if (!new_crypto_info) {
1139 rc = init_prot_info(prot, src_crypto_info, cipher_desc);
1140 if (rc)
1141 goto release_netdev;
1142
1143 memcpy(ctx->tx.iv + cipher_desc->salt, iv, cipher_desc->iv);
1144 memcpy(ctx->tx.rec_seq, rec_seq, cipher_desc->rec_seq);
1145
1146 start_marker_record = kmalloc(sizeof(*start_marker_record),
1147 GFP_KERNEL);
1148 if (!start_marker_record) {
1149 rc = -ENOMEM;
1150 goto release_netdev;
1151 }
1152
1153 offload_ctx = alloc_offload_ctx_tx(ctx);
1154 if (!offload_ctx) {
1155 rc = -ENOMEM;
1156 goto free_marker_record;
1157 }
1158
1159 rc = tls_sw_fallback_init(sk, offload_ctx, src_crypto_info);
1160 if (rc)
1161 goto free_offload_ctx;
1162
1163 start_marker_record->end_seq = tcp_sk(sk)->write_seq;
1164 start_marker_record->len = 0;
1165 start_marker_record->num_frags = 0;
1166 list_add_tail(&start_marker_record->list,
1167 &offload_ctx->records_list);
1168
1169 clean_acked_data_enable(tcp_sk(sk), &tls_tcp_clean_acked);
1170 ctx->push_pending_record = tls_device_push_pending_record;
1171
1172 /* TLS offload is greatly simplified if we don't send
1173 * SKBs where only part of the payload needs to be encrypted.
1174 * So mark the last skb in the write queue as end of record.
1175 */
1176 tcp_write_collapse_fence(sk);
1177 }
1178
1179 /* Avoid offloading if the device is down
1180 * We don't want to offload new flows after
1181 * the NETDEV_DOWN event
1182 *
1183 * device_offload_lock is taken in tls_devices's NETDEV_DOWN
1184 * handler thus protecting from the device going down before
1185 * ctx was added to tls_device_list.
1186 */
1187 down_read(&device_offload_lock);
1188 if (!(netdev->flags & IFF_UP)) {
1189 rc = -EINVAL;
1190 goto release_lock;
1191 }
1192
1193 if (!new_crypto_info) {
1194 ctx->priv_ctx_tx = offload_ctx;
1195 } else {
1196 char *key = crypto_info_key(src_crypto_info, cipher_desc);
1197
1198 offload_ctx = tls_offload_ctx_tx(ctx);
1199
1200 rc = crypto_aead_setkey(offload_ctx->aead_send, key,
1201 cipher_desc->key);
1202 if (rc)
1203 goto release_lock;
1204
1205 /* For rekey, delete old HW context before adding new one. */
1206 if (!test_bit(TLS_TX_DEV_CLOSED, &ctx->flags))
1207 netdev->tlsdev_ops->tls_dev_del(netdev, ctx,
1208 TLS_OFFLOAD_CTX_DIR_TX);
1209 }
1210
1211 rc = netdev->tlsdev_ops->tls_dev_add(netdev, sk, TLS_OFFLOAD_CTX_DIR_TX,
1212 src_crypto_info,
1213 tcp_sk(sk)->write_seq);
1214 trace_tls_device_offload_set(sk, TLS_OFFLOAD_CTX_DIR_TX,
1215 tcp_sk(sk)->write_seq, rec_seq, rc);
1216
1217 if (new_crypto_info) {
1218 unsigned long flags;
1219 __be64 rcd_sn;
1220
1221 memcpy(ctx->tx.iv + cipher_desc->salt, iv, cipher_desc->iv);
1222 memcpy(ctx->tx.rec_seq, rec_seq, cipher_desc->rec_seq);
1223
1224 spin_lock_irqsave(&offload_ctx->lock, flags);
1225 /* Delete old records, can't be retransmitted with new key */
1226 delete_all_records(offload_ctx);
1227
1228 /* Update unacked_record_sn for the new key's rec_seq.
1229 * This is critical for SW fallback encryption to use
1230 * the correct record sequence number after rekey.
1231 */
1232 memcpy(&rcd_sn, rec_seq, sizeof(rcd_sn));
1233 offload_ctx->unacked_record_sn = be64_to_cpu(rcd_sn);
1234 spin_unlock_irqrestore(&offload_ctx->lock, flags);
1235
1236 unsafe_memcpy(crypto_info, new_crypto_info,
1237 cipher_desc->crypto_info,
1238 /* size was checked in do_tls_setsockopt_conf */);
1239 memzero_explicit(new_crypto_info, cipher_desc->crypto_info);
1240 }
1241
1242 if (rc) {
1243 if (new_crypto_info) {
1244 /* HW rekey failed, gracefully degrade to SW encryption.
1245 * SW fallback already has new key, IV, and rec_seq.
1246 * Old HW ctx was deleted, continue with SW encryption.
1247 */
1248 set_bit(TLS_TX_DEV_CLOSED, &ctx->flags);
> 1249 smp_store_release(sk->sk_validate_xmit_skb,
1250 tls_validate_xmit_skb_sw);
1251 } else {
1252 goto release_lock;
1253 }
1254 } else {
1255 if (new_crypto_info)
1256 clear_bit(TLS_TX_DEV_CLOSED, &ctx->flags);
1257
1258 tls_device_attach(ctx, sk, netdev);
1259
1260 /* following this assignment tls_is_skb_tx_device_offloaded
1261 * will return true and the context might be accessed
1262 * by the netdev's xmit function.
1263 */
1264 smp_store_release(&sk->sk_validate_xmit_skb,
1265 tls_validate_xmit_skb);
1266 }
1267
1268 up_read(&device_offload_lock);
1269
1270 dev_put(netdev);
1271
1272 return 0;
1273
1274 release_lock:
1275 up_read(&device_offload_lock);
1276 if (new_crypto_info)
1277 goto release_netdev;
1278 clean_acked_data_disable(tcp_sk(sk));
1279 crypto_free_aead(offload_ctx->aead_send);
1280 free_offload_ctx:
1281 kfree(offload_ctx);
1282 ctx->priv_ctx_tx = NULL;
1283 free_marker_record:
1284 kfree(start_marker_record);
1285 release_netdev:
1286 dev_put(netdev);
1287 return rc;
1288 }
1289
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH v4 0/3] tls: Add TLS 1.3 hardware offload support
2026-01-21 21:57 [PATCH v4 0/3] tls: Add TLS 1.3 hardware offload support Rishikesh Jethwani
` (2 preceding siblings ...)
2026-01-21 21:57 ` [PATCH v4 3/3] mlx5: TLS 1.3 hardware offload support Rishikesh Jethwani
@ 2026-01-22 11:30 ` Tariq Toukan
2026-01-22 19:22 ` Rishikesh Jethwani
3 siblings, 1 reply; 12+ messages in thread
From: Tariq Toukan @ 2026-01-22 11:30 UTC (permalink / raw)
To: Rishikesh Jethwani, netdev
Cc: saeedm, tariqt, mbloch, borisp, john.fastabend, kuba, sd, davem,
pabeni, edumazet, leon
On 21/01/2026 23:57, Rishikesh Jethwani wrote:
> Hi all,
>
> This patch series adds TLS 1.3 support to the kernel TLS hardware offload
> infrastructure, enabling hardware acceleration for TLS 1.3 connections
> including KeyUpdate (rekey) support.
>
> Background
> ==========
> Currently, the kernel TLS device offload only supports TLS 1.2. With
> TLS 1.3 being the current standard and widely deployed, there is a
> growing need to extend hardware offload support to TLS 1.3 connections.
>
> TLS 1.3 differs from TLS 1.2 in its record format:
>
> TLS 1.2: [Header (5)] + [Explicit IV (8)] + [Ciphertext] + [Tag (16)]
> TLS 1.3: [Header (5)] + [Ciphertext + ContentType (1)] + [Tag (16)]
>
> The key difference is that TLS 1.3 eliminates the explicit IV and
> instead appends the content type byte to the plaintext before
> encryption. This content type byte must be encrypted along with the
> payload for proper authentication tag computation per RFC 8446.
>
> Patch 1: TLS 1.3 hardware offload support
> =========================================
> Changes to tls_device.c, tls_device_fallback.c, and tls_main.c:
>
> - Extended version validation to accept TLS_1_3_VERSION in both
> tls_set_device_offload() and tls_set_device_offload_rx()
> - Modified tls_device_record_close() to append the content type
> byte before the authentication tag for TLS 1.3 records
> - Modified tls_device_reencrypt() to use prot->prepend_size and
> prot->tag_size instead of hardcoded TLS 1.2 values
> - Pre-populated dummy_page with all 256 byte values for memory
> allocation failure fallback path
> - Updated tls_device_fallback.c to handle TLS 1.3 IV construction
> (XOR with sequence number) and version-specific AAD sizes
> - Rekey handling: HW offload key update (rekey) is not yet supported.
>
> Patch 2: Hardware offload key update support
> ============================================
> Changes to include/net/tls.h, net/tls/tls.h, tls_device.c, tls_main.c,
> and tls_sw.c:
>
> - Extended tls_set_device_offload() and tls_set_device_offload_rx()
> with new_crypto_info parameter for key updates
> - During rekey, the old HW context is deleted (tls_dev_del) and a new
> one is added (tls_dev_add) with the updated key material
> - Graceful degradation: if HW key update fails, the connection
> gracefully degrades to software:
> * TX: TLS_TX_DEV_CLOSED is set and sk_validate_xmit_skb switches to
> tls_validate_xmit_skb_sw for software encryption
> * RX: TLS_RX_DEV_DEGRADED and TLS_RX_DEV_CLOSED are set for software
> decryption
> * In both cases, tx_conf/rx_conf remains TLS_HW
> - Record sequence management: during TX rekey, old pending records are
> deleted and unacked_record_sn is reset to the new rec_seq
> - Split tls_set_sw_offload() into tls_sw_ctx_init() and
> tls_sw_ctx_finalize() to allow the HW offload RX path to
> initialize SW context first, attempt HW setup, then
> finalize (memzero new_crypto_info, call tls_finish_key_update)
> - Added TLS_TX_DEV_CLOSED flag to track TX hardware context state,
> to avoid double tls_dev_del call, symmetric with existing
> TLS_RX_DEV_CLOSED.
>
> Patch 3: mlx5 driver enablement
> ===============================
> - TLS 1.3 version detection and validation with proper capability checking
> - TLS 1.3 crypto context configuration using MLX5E_STATIC_PARAMS_CONTEXT_TLS_1_3
> - Correct IV handling for TLS 1.3 (12-byte IV vs TLS 1.2's 4-byte salt)
> - Hardware offload for both TLS 1.3 AES-GCM-128 and AES-GCM-256
>
> Testing
> =======
> Tested on Mellanox ConnectX-6 Dx (Crypto Enabled).
>
> Both TX and RX hardware offload verified working with:
> - TLS 1.3 AES-GCM-128
> - TLS 1.3 AES-GCM-256
> - Multiple KeyUpdate cycles (rekey)
>
> Test methodology: ktls_test : https://github.com/insanum/ktls_test/tree/master
>
> Please review and provide feedback.
>
Hi Rishikesh,
This version doesn't compile for us.
A few comments that we wanted to share on V3, but I'll share here as
this is the latest:
We tested V3 and a few issues were spotted.
We did not debug them though...
We run a server/client test with nginx and wrk.
a.
When requesting 128-bit key, ss output shows "cipher: aes-gcm-256" for
the respective connection.
For TLS 1.2 it works fine.
b.
On the wrk side, the mlx5 rx_tls_ctx did not increase (meaning no
offloaded connections were opened). It works fine on the nginx side
however...
For TLS 1.2 it works fine.
We can share more info if needed.
Regards,
Tariq
> Thanks,
> Rishikesh
>
> v4:
> - Split single TLS patch into two separate patches:
> * Patch 1: TLS 1.3 basic HW offload support
> * Patch 2: HW offload key update (rekey) support with graceful degradation
> - Removed record_type check from tls_device_record_close()
> - Removed Broadcom bnxt_en out-of-tree driver mention
> - Link to v3: https://lore.kernel.org/netdev/20260102184708.24618-1-rjethwani@purestorage.com/
>
> v3:
> - Added note about Broadcom bnxt_en out-of-tree driver used for testing
> - Link to v2: https://lore.kernel.org/netdev/20251231192322.3791912-1-rjethwani@purestorage.com/
>
> v2:
> - Fixed reverse Christmas tree ordering in variable declarations
> - Combined 'err' and 'i' declarations (reviewer feedback)
> - Link to v1: https://lore.kernel.org/netdev/20251230224137.3600355-1-rjethwani@purestorage.com/
>
> Rishikesh Jethwani (3):
> tls: add TLS 1.3 hardware offload support
> tls: add hardware offload key update support
> mlx5: TLS 1.3 hardware offload support
>
> .../mellanox/mlx5/core/en_accel/ktls.h | 8 +-
> .../mellanox/mlx5/core/en_accel/ktls_txrx.c | 14 +-
> include/net/tls.h | 4 +
> net/tls/tls.h | 14 +-
> net/tls/tls_device.c | 319 +++++++++++++-----
> net/tls/tls_device_fallback.c | 34 +-
> net/tls/tls_main.c | 31 +-
> net/tls/tls_sw.c | 77 +++--
> 8 files changed, 379 insertions(+), 122 deletions(-)
>
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH v4 0/3] tls: Add TLS 1.3 hardware offload support
2026-01-22 11:30 ` [PATCH v4 0/3] tls: Add " Tariq Toukan
@ 2026-01-22 19:22 ` Rishikesh Jethwani
2026-01-23 19:05 ` Rishikesh Jethwani
0 siblings, 1 reply; 12+ messages in thread
From: Rishikesh Jethwani @ 2026-01-22 19:22 UTC (permalink / raw)
To: Tariq Toukan
Cc: netdev, saeedm, tariqt, mbloch, borisp, john.fastabend, kuba, sd,
davem, pabeni, edumazet, leon
> Hi Rishikesh,
>
> This version doesn't compile for us.
> A few comments that we wanted to share on V3, but I'll share here as
> this is the latest:
>
> We tested V3 and a few issues were spotted.
> We did not debug them though...
>
> We run a server/client test with nginx and wrk.
>
> a.
> When requesting 128-bit key, ss output shows "cipher: aes-gcm-256" for
> the respective connection.
> For TLS 1.2 it works fine.
>
> b.
> On the wrk side, the mlx5 rx_tls_ctx did not increase (meaning no
> offloaded connections were opened). It works fine on the nginx side
> however...
> For TLS 1.2 it works fine.
>
> We can share more info if needed.
>
> Regards,
> Tariq
>
Hi Tariq,
I have fixed the typo leading to compile errors in V5.
Could you please share more info. about the setup and test, so that I
can try to reproduce the issue?
Thanks,
Rishikesh
> >
> > v4:
> > - Split single TLS patch into two separate patches:
> > * Patch 1: TLS 1.3 basic HW offload support
> > * Patch 2: HW offload key update (rekey) support with graceful degradation
> > - Removed record_type check from tls_device_record_close()
> > - Removed Broadcom bnxt_en out-of-tree driver mention
> > - Link to v3: https://lore.kernel.org/netdev/20260102184708.24618-1-rjethwani@purestorage.com/
> >
> > v3:
> > - Added note about Broadcom bnxt_en out-of-tree driver used for testing
> > - Link to v2: https://lore.kernel.org/netdev/20251231192322.3791912-1-rjethwani@purestorage.com/
> >
> > v2:
> > - Fixed reverse Christmas tree ordering in variable declarations
> > - Combined 'err' and 'i' declarations (reviewer feedback)
> > - Link to v1: https://lore.kernel.org/netdev/20251230224137.3600355-1-rjethwani@purestorage.com/
> >
> > Rishikesh Jethwani (3):
> > tls: add TLS 1.3 hardware offload support
> > tls: add hardware offload key update support
> > mlx5: TLS 1.3 hardware offload support
> >
> > .../mellanox/mlx5/core/en_accel/ktls.h | 8 +-
> > .../mellanox/mlx5/core/en_accel/ktls_txrx.c | 14 +-
> > include/net/tls.h | 4 +
> > net/tls/tls.h | 14 +-
> > net/tls/tls_device.c | 319 +++++++++++++-----
> > net/tls/tls_device_fallback.c | 34 +-
> > net/tls/tls_main.c | 31 +-
> > net/tls/tls_sw.c | 77 +++--
> > 8 files changed, 379 insertions(+), 122 deletions(-)
> >
>
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH v4 0/3] tls: Add TLS 1.3 hardware offload support
2026-01-22 19:22 ` Rishikesh Jethwani
@ 2026-01-23 19:05 ` Rishikesh Jethwani
2026-01-26 23:37 ` Rishikesh Jethwani
0 siblings, 1 reply; 12+ messages in thread
From: Rishikesh Jethwani @ 2026-01-23 19:05 UTC (permalink / raw)
To: Tariq Toukan
Cc: netdev, saeedm, tariqt, mbloch, borisp, john.fastabend, kuba, sd,
davem, pabeni, edumazet, leon
On Thu, Jan 22, 2026 at 11:22 AM Rishikesh Jethwani
<rjethwani@purestorage.com> wrote:
>
> > Hi Rishikesh,
> >
> > This version doesn't compile for us.
> > A few comments that we wanted to share on V3, but I'll share here as
> > this is the latest:
> >
> > We tested V3 and a few issues were spotted.
> > We did not debug them though...
> >
> > We run a server/client test with nginx and wrk.
> >
> > a.
> > When requesting 128-bit key, ss output shows "cipher: aes-gcm-256" for
> > the respective connection.
> > For TLS 1.2 it works fine.
> >
> > b.
> > On the wrk side, the mlx5 rx_tls_ctx did not increase (meaning no
> > offloaded connections were opened). It works fine on the nginx side
> > however...
> > For TLS 1.2 it works fine.
> >
> > We can share more info if needed.
> >
> > Regards,
> > Tariq
> >
> Hi Tariq,
>
> I have fixed the typo leading to compile errors in V5.
> Could you please share more info. about the setup and test, so that I
> can try to reproduce the issue?
>
> Thanks,
> Rishikesh
HI Tariq,
Did you run SW TLS 1.3 in your environment?
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH v4 0/3] tls: Add TLS 1.3 hardware offload support
2026-01-23 19:05 ` Rishikesh Jethwani
@ 2026-01-26 23:37 ` Rishikesh Jethwani
0 siblings, 0 replies; 12+ messages in thread
From: Rishikesh Jethwani @ 2026-01-26 23:37 UTC (permalink / raw)
To: Tariq Toukan
Cc: netdev, saeedm, tariqt, mbloch, borisp, john.fastabend, kuba, sd,
davem, pabeni, edumazet, leon
On Fri, Jan 23, 2026 at 11:05 AM Rishikesh Jethwani
<rjethwani@purestorage.com> wrote:
>
> On Thu, Jan 22, 2026 at 11:22 AM Rishikesh Jethwani
> <rjethwani@purestorage.com> wrote:
> >
> > > Hi Rishikesh,
> > >
> > > This version doesn't compile for us.
> > > A few comments that we wanted to share on V3, but I'll share here as
> > > this is the latest:
> > >
> > > We tested V3 and a few issues were spotted.
> > > We did not debug them though...
> > >
> > > We run a server/client test with nginx and wrk.
> > >
> > > a.
> > > When requesting 128-bit key, ss output shows "cipher: aes-gcm-256" for
> > > the respective connection.
> > > For TLS 1.2 it works fine.
> > >
> > > b.
> > > On the wrk side, the mlx5 rx_tls_ctx did not increase (meaning no
> > > offloaded connections were opened). It works fine on the nginx side
> > > however...
> > > For TLS 1.2 it works fine.
> > >
> > > We can share more info if needed.
> > >
> > > Regards,
> > > Tariq
> > >
> > Hi Tariq,
> >
> > I have fixed the typo leading to compile errors in V5.
> > Could you please share more info. about the setup and test, so that I
> > can try to reproduce the issue?
> >
> > Thanks,
> > Rishikesh
>
> HI Tariq,
>
> Did you run SW TLS 1.3 in your environment?
Hi Tariq,
I tested TLS 1.3 hardware offload with nginx and wrk. The ss output
shows the correct cipher, and hardware counters confirm TX/RX offload
is working as expected.
AES-256-GCM:
nginx.conf:
ssl_protocols TLSv1.3;
ssl_conf_command Ciphersuites TLS_AES_256_GCM_SHA384;
ssl_conf_command Options KTLS;
# ss -tnei 'sport = :16443 or dport = :16443' | grep -i cipher
version: 1.3 cipher: aes-gcm-256
AES-128-GCM:
nginx.conf:
ssl_protocols TLSv1.3;
ssl_conf_command Ciphersuites TLS_AES_128_GCM_SHA256;
ssl_conf_command Options KTLS;
# ss -tnei 'sport = :16443 or dport = :16443' | grep -i cipher
version: 1.3 cipher: aes-gcm-128
Both ciphers report correctly. I suspect there may be a configuration
issue on your setup. Let me know if you need help debugging.
Regards,
Rishikesh
^ permalink raw reply [flat|nested] 12+ messages in thread
end of thread, other threads:[~2026-01-26 23:37 UTC | newest]
Thread overview: 12+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-01-21 21:57 [PATCH v4 0/3] tls: Add TLS 1.3 hardware offload support Rishikesh Jethwani
2026-01-21 21:57 ` [PATCH v4 1/3] tls: add " Rishikesh Jethwani
2026-01-21 21:57 ` [PATCH v4 2/3] tls: add hardware offload key update support Rishikesh Jethwani
2026-01-22 4:09 ` kernel test robot
2026-01-22 5:15 ` kernel test robot
2026-01-22 7:19 ` kernel test robot
2026-01-22 7:54 ` kernel test robot
2026-01-21 21:57 ` [PATCH v4 3/3] mlx5: TLS 1.3 hardware offload support Rishikesh Jethwani
2026-01-22 11:30 ` [PATCH v4 0/3] tls: Add " Tariq Toukan
2026-01-22 19:22 ` Rishikesh Jethwani
2026-01-23 19:05 ` Rishikesh Jethwani
2026-01-26 23:37 ` Rishikesh Jethwani
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox