From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-yx1-f100.google.com (mail-yx1-f100.google.com [74.125.224.100]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C491926ED46 for ; Tue, 12 May 2026 21:22:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=74.125.224.100 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778620925; cv=none; b=h1vku/PQj/QxMcZZ4fe8YyIwuqAdZYHcwFbUEKzEvjNINqNoP+458fS24TpoK+q3WTF/okRUPU//Dwc008QlYJwzBOCnk/4tWsQomcH+MpGoPnBdPKm18/hR+MaBWbaIx6TgOLrXQbOdhTM5SjE7x348XnyXaOf5YShZ/UFjcaw= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778620925; c=relaxed/simple; bh=iqLtpub8kZ5IlN17DOEuchMkXiQHoPaXb44Drvi2/B4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ZAEJRl6pTFhItlhdvjoohlSmQTlwtxJDNbaFfQ+hTLKUbUq7KYCzG1id0/J1jWJKo8JM8WaN85khNaGvfFlJNHYe7R/X2MxCHSSGePpoWrA8/UWUAdVo2GPLZW51J3Jzdnl/2duaPDj/EAHV07CkNOPKdm7GyIOYODhF8HqgKRE= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=broadcom.com; spf=fail smtp.mailfrom=broadcom.com; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b=XRla2gT4; arc=none smtp.client-ip=74.125.224.100 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=broadcom.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=broadcom.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b="XRla2gT4" Received: by mail-yx1-f100.google.com with SMTP id 956f58d0204a3-6579254f996so3740469d50.1 for ; Tue, 12 May 2026 14:22:03 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778620923; x=1779225723; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=Qb9aod9fXkT5Yr30wbkkJLdf8M7XH+2hxdPY5VgmBuE=; b=DQZ9jPoQuJ5gYL+i8iaZdg49Z5/OSOeKVe4c46CIHOUkS04b8bFmmecLLTpcVhAapa fnTjyfAxOXFPeDlPB11dJTFCf3sR/HGonMy19fvuFL335P5Ei80W+QQ/ItMMojRaptOZ qBLf44z97NmoLocYgD10PWrdWbKa6oM2bJui92F0kI/IIofx7VrTOMShGFPizGpLrIBa LBeKAbFay0+RSphfHv/SkMtwTOOAIU1FN4TBs1z6WaRLvVMfo7Y6VHrB4ItK8TUwaxeP we9HRld+GY1dZQMotgHHlhNzJlDNofuMb2ZS4DbTRpDjWSJE2Xnb64RYTWgb7dl5rPtr s4Gw== X-Gm-Message-State: AOJu0YwoMwjito8LU6RylwnJVfps+WaFweGTIK4c75qFKEjBfv9rS8AG VKR2ooXgvOz1k+sxK5UljBTAkUfuc7ZIug2Mr91RNGoysGCNfs1B/T+URsA+WofoqKIhJYmKMtm R07QbLV94oxSDZqkJQUB8CnO9kSNx8UD7+Ov6Ohvkl/buES7/f9v/LQW751W+MIHmdQbrZYQFDQ 3CJaM+dZ31jA5HfOzqpqvYtQNae7B5Mm7vIXrRJDGNRJfoX7bcCBKJsiJrpj7b29nsk8PWnJEeY q3cr2colAc= X-Gm-Gg: Acq92OFMP7Fzxz38ML0X1fZuUky8VNhO9/xhg6Ts6aaWgt1Syc6Lh+BU18eDylzduG2 DnYFJ/xkVDPOSXJQGK55Jg5v8R2zOOwoKnDrnuHqHa54DFpu30dLLBV9bH7QhLYiswBjhDPW+eI CWk0npitw3tkt+m9HmkBmexlVlr69azBVtSmqFYFUBJ4YA2pWfdVeRQepKgi6Ca3v2mvT7663Ti 9GfCt7vmFgY0DC19r3UcmzTxA+aFdITLCqoiht5yOoe79Y5hCTL+8ZlFqQTssITY13x/xMV9rlh ibg9Vvs8z2KQjimTpGeLL2Z3TPNnaveHWJQfVFRCfDSGa2HP4l5WmQOaPIXJXvPIZT7L2Xyu6wo Vq/JSEgueJE0WmeVRTyO8Kfcxx1SfdeO4KZFYQs8m5vaJNd0t6h/0AUkNw+YWO2NSqSL4xsi03P dTjjws X-Received: by 2002:a05:690e:4147:b0:65d:b6ca:33fe with SMTP id 956f58d0204a3-65df82e538emr10192d50.50.1778620922761; Tue, 12 May 2026 14:22:02 -0700 (PDT) Received: from smtp-us-east1-p01-i01-si01.dlp.protect.broadcom.com ([144.49.247.127]) by smtp-relay.gmail.com with ESMTPS id 956f58d0204a3-65d96b7deebsm1149934d50.15.2026.05.12.14.22.02 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 12 May 2026 14:22:02 -0700 (PDT) X-Relaying-Domain: broadcom.com X-CFilter-Loop: Reflected Received: by mail-dl1-f69.google.com with SMTP id a92af1059eb24-12dc3d81736so5563714c88.1 for ; Tue, 12 May 2026 14:22:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; t=1778620921; x=1779225721; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Qb9aod9fXkT5Yr30wbkkJLdf8M7XH+2hxdPY5VgmBuE=; b=XRla2gT44EVAwpRYRk8QEU83yY0FnmbH5rhdNQ3nKwA2blPLhMBRcRFNVe9ZUsr+yU fyIMmqYKRkbyqkW9FSlJU+MZPWBcW0qPyIj6HJ0O48zv6axY1VqixRyEjO+DY+0XdHfA SFHZ0qN55O0B1ZFxB7pLKjOAe7VRoYc9rMf+Q= X-Received: by 2002:a05:7022:1a81:b0:12b:f616:1a4e with SMTP id a92af1059eb24-1349ab4de06mr73300c88.23.1778620921293; Tue, 12 May 2026 14:22:01 -0700 (PDT) X-Received: by 2002:a05:7022:1a81:b0:12b:f616:1a4e with SMTP id a92af1059eb24-1349ab4de06mr73276c88.23.1778620920741; Tue, 12 May 2026 14:22:00 -0700 (PDT) Received: from lvnvda3289.lvn.broadcom.net ([192.19.161.250]) by smtp.gmail.com with ESMTPSA id a92af1059eb24-132787673ffsm26603202c88.15.2026.05.12.14.21.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 May 2026 14:21:59 -0700 (PDT) From: Michael Chan To: davem@davemloft.net Cc: netdev@vger.kernel.org, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, andrew+netdev@lunn.ch, pavan.chebbi@broadcom.com, andrew.gospodarek@broadcom.com Subject: [PATCH net-next v2 15/15] bnxt_en: Add kTLS retransmission support Date: Tue, 12 May 2026 14:21:05 -0700 Message-ID: <20260512212105.3488258-16-michael.chan@broadcom.com> X-Mailer: git-send-email 2.45.4 In-Reply-To: <20260512212105.3488258-1-michael.chan@broadcom.com> References: <20260512212105.3488258-1-michael.chan@broadcom.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-DetectorID-Processed: b00c1d49-9d2e-4205-b15f-d015386d3d5e If TCP retransmits a TLS packet that requires encryption by the NIC, the TCP sequence number will go backwards and the hardware will require some assistance from the driver. The driver needs to retrieve the TLS record that covers the byte sequence of the retransmitted packet. If the retransmitted packet does not include the tag, the hardware can simply encrypt the packet using the informtaion in the TLS record. The driver provides the TLS record information for the retransmitted packet in the presync TX BD. The presync TX BD introduced in the last patch is treated very much like a TX push BD with inline data. The only exception is that no SKB will be stored for the presync TX BD. Retransmission that includes the TLS tag will be handled in future patches. Reviewed-by: Andy Gospodarek Signed-off-by: Michael Chan --- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 2 +- drivers/net/ethernet/broadcom/bnxt/bnxt.h | 2 + .../net/ethernet/broadcom/bnxt/bnxt_ethtool.c | 2 + .../net/ethernet/broadcom/bnxt/bnxt_ktls.c | 126 +++++++++++++++++- 4 files changed, 128 insertions(+), 4 deletions(-) diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c index 89c900db45ba..5697190dc541 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c @@ -499,7 +499,6 @@ static netdev_tx_t bnxt_start_xmit(struct sk_buff *skb, struct net_device *dev) txq = netdev_get_tx_queue(dev, i); txr = &bp->tx_ring[bp->tx_ring_map[i]]; - prod = txr->tx_prod; #if (MAX_SKB_FRAGS > TX_MAX_FRAGS) if (skb_shinfo(skb)->nr_frags > TX_MAX_FRAGS) { @@ -532,6 +531,7 @@ static netdev_tx_t bnxt_start_xmit(struct sk_buff *skb, struct net_device *dev) if (unlikely(!skb)) return NETDEV_TX_OK; + prod = txr->tx_prod; length = skb->len; len = skb_headlen(skb); last_frag = skb_shinfo(skb)->nr_frags; diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.h b/drivers/net/ethernet/broadcom/bnxt/bnxt.h index e0880b8c4b73..696dfe522c7b 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.h +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.h @@ -1188,6 +1188,8 @@ struct bnxt_cmn_sw_stats { enum bnxt_ktls_data_counters { BNXT_KTLS_TX_PKTS = 0, BNXT_KTLS_TX_BYTES, + BNXT_KTLS_TX_OOO_PKTS, + BNXT_KTLS_TX_DROP_NO_SYNC, BNXT_KTLS_MAX_DATA_COUNTERS, }; diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c index 66b323e94140..769058a6ec31 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c @@ -359,6 +359,8 @@ static const char *const bnxt_ring_drv_stats_arr[] = { static const char *const bnxt_ktls_data_stats[] = { [BNXT_KTLS_TX_PKTS] = "tx_tls_encrypted_packets", [BNXT_KTLS_TX_BYTES] = "tx_tls_encrypted_bytes", + [BNXT_KTLS_TX_OOO_PKTS] = "tx_tls_ooo_packets", + [BNXT_KTLS_TX_DROP_NO_SYNC] = "tx_tls_drop_no_sync", }; /* kTLS control plane counter strings indexed by enum bnxt_ktls_ctrl_counters */ diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_ktls.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_ktls.c index 263b075af621..b94418ee5436 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt_ktls.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_ktls.c @@ -289,6 +289,116 @@ int bnxt_ktls_init(struct bnxt *bp) return 0; } +static void bnxt_ktls_pre_xmit(struct bnxt *bp, struct bnxt_tx_ring_info *txr, + u32 kid, struct crypto_prefix_cmd *pre_cmd) +{ + struct bnxt_sw_tx_bd *tx_buf; + struct tx_bd_presync *psbd; + u32 bd_space, space; + u8 *pcmd; + u16 prod; + + prod = txr->tx_prod; + tx_buf = &txr->tx_buf_ring[RING_TX(bp, prod)]; + + psbd = (void *)&txr->tx_desc_ring[TX_RING(bp, prod)][TX_IDX(prod)]; + psbd->tx_bd_len_flags_type = CRYPTO_PRESYNC_BD_CMD; + psbd->tx_bd_kid = cpu_to_le32(BNXT_KID_HW(kid)); + psbd->tx_bd_opaque = + SET_TX_OPAQUE(bp, txr, prod, CRYPTO_PREFIX_CMD_BDS + 1); + + prod = NEXT_TX(prod); + pcmd = (void *)&txr->tx_desc_ring[TX_RING(bp, prod)][TX_IDX(prod)]; + bd_space = TX_DESC_CNT - TX_IDX(prod); + space = bd_space * sizeof(struct tx_bd); + if (space >= CRYPTO_PREFIX_CMD_SIZE) { + memcpy(pcmd, pre_cmd, CRYPTO_PREFIX_CMD_SIZE); + prod += CRYPTO_PREFIX_CMD_BDS; + } else { + memcpy(pcmd, pre_cmd, space); + prod += bd_space; + pcmd = (void *)&txr->tx_desc_ring[TX_RING(bp, prod)][TX_IDX(prod)]; + memcpy(pcmd, (u8 *)pre_cmd + space, + CRYPTO_PREFIX_CMD_SIZE - space); + prod += CRYPTO_PREFIX_CMD_BDS - bd_space; + } + txr->tx_prod = prod; + tx_buf->is_push = 1; + /* Minus 1 since the header psbd is a single entry short BD */ + tx_buf->inline_data_bds = CRYPTO_PREFIX_CMD_BDS - 1; +} + +static int bnxt_ktls_tx_ooo(struct bnxt *bp, struct bnxt_tx_ring_info *txr, + struct sk_buff *skb, u32 payload_len, u32 seq, + struct tls_context *tls_ctx) +{ + struct bnxt_sw_stats *sw_stats = txr->tx_cpr->sw_stats; + struct tls_offload_context_tx *tx_tls_ctx; + struct bnxt_ktls_offload_ctx_tx *kctx_tx; + u32 hdr_tcp_seq, end_seq, total_bds; + struct crypto_prefix_cmd pcmd = {}; + struct tls_record_info *record; + unsigned long flags; + bool fwd = false; + u64 rec_sn; + u8 *hdr; + int rc; + + tx_tls_ctx = tls_offload_ctx_tx(tls_ctx); + kctx_tx = __tls_driver_ctx(tls_ctx, TLS_OFFLOAD_CTX_DIR_TX); + end_seq = seq + skb->len - skb_tcp_all_headers(skb); + if (unlikely(after(seq, kctx_tx->tcp_seq_no) || + after(end_seq, kctx_tx->tcp_seq_no))) { + fwd = true; + pcmd.flags = CRYPTO_PREFIX_CMD_FLAGS_UPDATE_IN_ORDER_VAR_LE; + } + + spin_lock_irqsave(&tx_tls_ctx->lock, flags); + record = tls_get_record(tx_tls_ctx, seq, &rec_sn); + if (!record || !record->num_frags) { + rc = -EPROTO; + sw_stats->tls.counters[BNXT_KTLS_TX_DROP_NO_SYNC]++; + goto unlock_exit; + } + hdr_tcp_seq = tls_record_start_seq(record); + hdr = skb_frag_address_safe(&record->frags[0]); + + total_bds = CRYPTO_PRESYNC_BDS + skb_shinfo(skb)->nr_frags + 2; + if (bnxt_tx_avail(bp, txr) < total_bds) { + rc = -ENOSPC; + goto unlock_exit; + } + + if (before(record->end_seq - tls_ctx->prot_info.tag_size, + seq + payload_len)) { + /* retransmission includes tag bytes */ + rc = -EOPNOTSUPP; + goto unlock_exit; + } + pcmd.header_tcp_seq_num = cpu_to_le32(hdr_tcp_seq); + pcmd.start_tcp_seq_num = cpu_to_le32(seq); + pcmd.end_tcp_seq_num = cpu_to_le32(seq + payload_len - 1); + if (tls_ctx->prot_info.version == TLS_1_2_VERSION) { + u32 nonce_bytes = tls_ctx->prot_info.iv_size; + u32 retrans_off = seq - hdr_tcp_seq; + + if (retrans_off > 5 && retrans_off < 5 + nonce_bytes) + nonce_bytes = retrans_off - 5; + memcpy(pcmd.explicit_nonce, hdr + 5, nonce_bytes); + } + memcpy(&pcmd.record_seq_num[0], &rec_sn, sizeof(rec_sn)); + + rc = 0; + bnxt_ktls_pre_xmit(bp, txr, kctx_tx->kid, &pcmd); + + if (fwd) + kctx_tx->tcp_seq_no = end_seq; + +unlock_exit: + spin_unlock_irqrestore(&tx_tls_ctx->lock, flags); + return rc; +} + struct sk_buff *bnxt_ktls_xmit(struct bnxt *bp, struct bnxt_tx_ring_info *txr, struct sk_buff *skb, __le32 *lflags, u32 *kid) { @@ -297,6 +407,7 @@ struct sk_buff *bnxt_ktls_xmit(struct bnxt *bp, struct bnxt_tx_ring_info *txr, struct bnxt_ktls_offload_ctx_tx *kctx_tx; struct tls_context *tls_ctx; u32 seq, payload_len; + int rc; if (!IS_ENABLED(CONFIG_TLS_DEVICE) || !ktls || !tls_is_skb_tx_device_offloaded(skb)) @@ -316,9 +427,18 @@ struct sk_buff *bnxt_ktls_xmit(struct bnxt *bp, struct bnxt_tx_ring_info *txr, sw_stats->tls.counters[BNXT_KTLS_TX_PKTS]++; sw_stats->tls.counters[BNXT_KTLS_TX_BYTES] += payload_len; } else { - skb = tls_encrypt_skb(skb); - if (!skb) - return NULL; + sw_stats->tls.counters[BNXT_KTLS_TX_OOO_PKTS]++; + + rc = bnxt_ktls_tx_ooo(bp, txr, skb, payload_len, seq, tls_ctx); + if (rc) + return tls_encrypt_skb(skb); + + *kid = BNXT_KID_HW(kctx_tx->kid); + *lflags |= cpu_to_le32(TX_BD_FLAGS_CRYPTO_EN | + BNXT_TX_KID_LO(*kid)); + sw_stats->tls.counters[BNXT_KTLS_TX_PKTS]++; + sw_stats->tls.counters[BNXT_KTLS_TX_BYTES] += payload_len; + return skb; } return skb; } -- 2.51.0