* [PATCH 0/2] net/mana: fix transmit burst and add hardware TSO support
@ 2026-10-12 7:36 Tathagat Priyadarshi
2026-10-12 7:36 ` [PATCH 1/2] net/mana: fix transmit burst error handling Tathagat Priyadarshi
` (4 more replies)
0 siblings, 5 replies; 13+ messages in thread
From: Tathagat Priyadarshi @ 2026-10-12 7:36 UTC (permalink / raw)
To: dev; +Cc: Long Li, Wei Hu
This patch series addresses transmit burst error handling and introduces
hardware TCP Segmentation Offload (TSO) support for the Microsoft Azure
MANA PMD.
Patch 1 fixes the transmit burst loop in mana_tx_burst() to terminate
with break instead of continue upon packet errors. Under the DPDK
rte_eth_tx_burst() API contract, callers expect a contiguous prefix of
transmitted packets. Skipping a packet caused unsent packets to be treated
as unsent and freed by caller, followed by a second free upon hardware CQE
arrival.
Patch 2 adds support for hardware TCP Segmentation Offload (TSO),
formatting SGL entries with packet headers in SGE0, handling GDMA work
request flags, advertising RTE_ETH_TX_OFFLOAD_TCP_TSO, and updating
the mana driver feature table.
Tathagat Priyadarshi (2):
net/mana: fix transmit burst error handling
net/mana: add hardware TSO support
doc/guides/nics/features/mana.ini | 1 +
drivers/net/mana/gdma.c | 5 +
drivers/net/mana/mana.h | 6 +-
drivers/net/mana/tx.c | 179 +++++++++++++++++++++++++-----
4 files changed, 163 insertions(+), 28 deletions(-)
--
2.43.0
^ permalink raw reply [flat|nested] 13+ messages in thread* [PATCH 1/2] net/mana: fix transmit burst error handling 2026-10-12 7:36 [PATCH 0/2] net/mana: fix transmit burst and add hardware TSO support Tathagat Priyadarshi @ 2026-10-12 7:36 ` Tathagat Priyadarshi 2026-10-12 7:36 ` [PATCH 2/2] net/mana: add hardware TSO support Tathagat Priyadarshi ` (3 subsequent siblings) 4 siblings, 0 replies; 13+ messages in thread From: Tathagat Priyadarshi @ 2026-10-12 7:36 UTC (permalink / raw) To: dev; +Cc: Long Li, stable, Wei Hu When a packet exceeds max_send_sge or when populating SGL segments fails, mana_tx_burst() skipped the failed packet using continue. Under the DPDK rte_eth_tx_burst() API contract, callers assume that the returned count represents a contiguous prefix of transmitted packets (tx_pkts[0 .. nb_tx - 1]). Skipping a packet in the middle of a burst causes subsequent packets to be transmitted while the caller treats them as unsent and frees them. When hardware later generates CQEs for those transmitted packets, they are freed a second time, corrupting mbuf pools and free caches. Fix this by terminating the burst loop with break upon packet errors so that pkt_sent accurately reflects the contiguous prefix sent. Fixes: 7f322844f427 ("net/mana: send packets") Cc: stable@dpdk.org Signed-off-by: Tathagat Priyadarshi <tathagat.dpdk@gmail.com> --- drivers/net/mana/tx.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/net/mana/tx.c b/drivers/net/mana/tx.c index 10f2212b5d..d35030cbe5 100644 --- a/drivers/net/mana/tx.c +++ b/drivers/net/mana/tx.c @@ -281,7 +281,7 @@ mana_tx_burst(void *dpdk_txq, struct rte_mbuf **tx_pkts, uint16_t nb_pkts) if (m_pkt->nb_segs > priv->max_send_sge) { DP_LOG(ERR, "send packet segments %d exceeding max", m_pkt->nb_segs); - continue; + break; } /* Fill in the oob */ @@ -432,7 +432,7 @@ mana_tx_burst(void *dpdk_txq, struct rte_mbuf **tx_pkts, uint16_t nb_pkts) /* Skip this packet if we can't populate all segments */ if (seg_idx != m_pkt->nb_segs) - continue; + break; /* If we can at least queue post two WQEs and there are at * least two packets to send, use TX CQE suppression for the -- 2.43.0 ^ permalink raw reply related [flat|nested] 13+ messages in thread
* [PATCH 2/2] net/mana: add hardware TSO support 2026-10-12 7:36 [PATCH 0/2] net/mana: fix transmit burst and add hardware TSO support Tathagat Priyadarshi 2026-10-12 7:36 ` [PATCH 1/2] net/mana: fix transmit burst error handling Tathagat Priyadarshi @ 2026-10-12 7:36 ` Tathagat Priyadarshi 2026-10-12 21:37 ` [PATCH 0/2] net/mana: fix transmit burst and " Stephen Hemminger ` (2 subsequent siblings) 4 siblings, 0 replies; 13+ messages in thread From: Tathagat Priyadarshi @ 2026-10-12 7:36 UTC (permalink / raw) To: dev; +Cc: Long Li, Wei Hu Implement hardware TCP Segmentation Offload (TSO) in Microsoft Azure MANA PMD: - Define GDMA_WR_OOB_IN_SGL and GDMA_WR_PAD_BY_SGE0 in mana.h. - Advertise RTE_ETH_TX_OFFLOAD_TCP_TSO in MANA_DEV_TX_OFFLOAD_SUPPORT. - Handle GDMA_WR_OOB_IN_SGL and GDMA_WR_PAD_BY_SGE0 in write_dma_client_oob() in gdma.c to configure client_oob_in_sgl and last_v_bytes. - In mana_tx_burst() in tx.c: * Parse RTE_MBUF_F_TX_TCP_SEG and validate total SGEs against priv->max_send_sge and MAX_SGL_ENTRIES_FOR_TRANSMIT. * Zero out IPv4 total_length and hdr_checksum (or IPv6 payload_len) and compute pseudo-header checksum in the TCP header. * Enable tx_compute_TCP_checksum and set tx_transport_header_offset. * Format SGL so SGE0 contains packet headers (L2 + L3 + L4) and subsequent SGEs carry segment payloads, allocating MRs through mana_alloc_pmd_mr(). * Set GDMA_WR_OOB_IN_SGL | GDMA_WR_PAD_BY_SGE0 work request flags and assign client_data_unit to m_pkt->tso_segsz. - Update mana.ini to advertise TSO feature support. Signed-off-by: Tathagat Priyadarshi <tathagat.dpdk@gmail.com> --- doc/guides/nics/features/mana.ini | 1 + drivers/net/mana/gdma.c | 5 + drivers/net/mana/mana.h | 6 +- drivers/net/mana/tx.c | 177 +++++++++++++++++++++++++----- 4 files changed, 162 insertions(+), 27 deletions(-) diff --git a/doc/guides/nics/features/mana.ini b/doc/guides/nics/features/mana.ini index db555ffe27..edf9469486 100644 --- a/doc/guides/nics/features/mana.ini +++ b/doc/guides/nics/features/mana.ini @@ -11,6 +11,7 @@ Rx interrupt = Y RSS hash = Y L3 checksum offload = Y L4 checksum offload = Y +TSO = Y Basic stats = Y MTU update = Y Multiprocess aware = Y diff --git a/drivers/net/mana/gdma.c b/drivers/net/mana/gdma.c index a92aef59d2..65896b6a69 100644 --- a/drivers/net/mana/gdma.c +++ b/drivers/net/mana/gdma.c @@ -38,6 +38,11 @@ write_dma_client_oob(uint8_t *work_queue_buffer_pointer, header->inline_client_oob_size_in_dwords = client_oob_size / sizeof(uint32_t); header->client_data_unit = work_request->client_data_unit; + if (work_request->flags & GDMA_WR_OOB_IN_SGL) { + header->client_oob_in_sgl = 1; + if (work_request->flags & GDMA_WR_PAD_BY_SGE0) + header->last_v_bytes = work_request->sgl[0].size; + } DP_LOG(DEBUG, "queue buf %p sgl %u oob_h %u du %u oob_buf %p oob_b %u", work_queue_buffer_pointer, header->num_sgl_entries, diff --git a/drivers/net/mana/mana.h b/drivers/net/mana/mana.h index 79879b7626..17aa4a09ac 100644 --- a/drivers/net/mana/mana.h +++ b/drivers/net/mana/mana.h @@ -28,12 +28,16 @@ struct mana_shared_data { RTE_ETH_RX_OFFLOAD_CHECKSUM | \ RTE_ETH_RX_OFFLOAD_RSS_HASH) +#define GDMA_WR_OOB_IN_SGL (1 << 0) +#define GDMA_WR_PAD_BY_SGE0 (1 << 1) + #define MANA_DEV_TX_OFFLOAD_SUPPORT ( \ RTE_ETH_TX_OFFLOAD_VLAN_INSERT | \ RTE_ETH_TX_OFFLOAD_MULTI_SEGS | \ RTE_ETH_TX_OFFLOAD_IPV4_CKSUM | \ RTE_ETH_TX_OFFLOAD_TCP_CKSUM | \ - RTE_ETH_TX_OFFLOAD_UDP_CKSUM) + RTE_ETH_TX_OFFLOAD_UDP_CKSUM | \ + RTE_ETH_TX_OFFLOAD_TCP_TSO) #define INDIRECTION_TABLE_NUM_ELEMENTS 64 #define TOEPLITZ_HASH_KEY_SIZE_IN_BYTES 40 diff --git a/drivers/net/mana/tx.c b/drivers/net/mana/tx.c index d35030cbe5..9401928bad 100644 --- a/drivers/net/mana/tx.c +++ b/drivers/net/mana/tx.c @@ -273,14 +273,31 @@ mana_tx_burst(void *dpdk_txq, struct rte_mbuf **tx_pkts, uint16_t nb_pkts) struct transmit_oob_v2 tx_oob; struct one_sgl sgl; uint16_t seg_idx; + uint16_t num_sge; + uint16_t expected_sge; + bool is_tso = false; + uint32_t gso_hs = 0; if (txq->desc_ring_len >= txq->num_desc) break; - /* Drop the packet if it exceeds max segments */ - if (m_pkt->nb_segs > priv->max_send_sge) { + if (m_pkt->ol_flags & RTE_MBUF_F_TX_TCP_SEG) { + gso_hs = m_pkt->l2_len + m_pkt->l3_len + m_pkt->l4_len; + if (unlikely(m_pkt->data_len < gso_hs)) { + DP_LOG(ERR, "TSO header cannot be fragmented"); + break; + } + if (likely(gso_hs != 0 && m_pkt->pkt_len > gso_hs && + m_pkt->tso_segsz != 0)) + is_tso = true; + } + + expected_sge = m_pkt->nb_segs + + (is_tso && m_pkt->data_len > gso_hs ? 1 : 0); + if (expected_sge > priv->max_send_sge || + expected_sge > MAX_SGL_ENTRIES_FOR_TRANSMIT) { DP_LOG(ERR, "send packet segments %d exceeding max", - m_pkt->nb_segs); + expected_sge); break; } @@ -305,7 +322,43 @@ mana_tx_burst(void *dpdk_txq, struct rte_mbuf **tx_pkts, uint16_t nb_pkts) tx_oob.short_oob.tx_compute_IP_header_checksum = m_pkt->ol_flags & RTE_MBUF_F_TX_IP_CKSUM ? 1 : 0; - if ((m_pkt->ol_flags & RTE_MBUF_F_TX_L4_MASK) == + if (is_tso) { + struct rte_tcp_hdr *tcp_hdr = + rte_pktmbuf_mtod_offset(m_pkt, + struct rte_tcp_hdr *, + m_pkt->l2_len + m_pkt->l3_len); + + if (m_pkt->ol_flags & RTE_MBUF_F_TX_IPV4) { + struct rte_ipv4_hdr *ip_hdr = + rte_pktmbuf_mtod_offset(m_pkt, + struct rte_ipv4_hdr *, + m_pkt->l2_len); + + ip_hdr->total_length = 0; + ip_hdr->hdr_checksum = 0; + tcp_hdr->cksum = rte_ipv4_phdr_cksum(ip_hdr, + m_pkt->ol_flags); + } else if (m_pkt->ol_flags & RTE_MBUF_F_TX_IPV6) { + struct rte_ipv6_hdr *ip_hdr = + rte_pktmbuf_mtod_offset(m_pkt, + struct rte_ipv6_hdr *, + m_pkt->l2_len); + + ip_hdr->payload_len = 0; + tcp_hdr->cksum = rte_ipv6_phdr_cksum(ip_hdr, + m_pkt->ol_flags); + } else { + DP_LOG(ERR, "Invalid IP version for TSO"); + break; + } + + tx_oob.short_oob.tx_compute_IP_header_checksum = + (m_pkt->ol_flags & RTE_MBUF_F_TX_IPV4) ? 1 : 0; + tx_oob.short_oob.tx_compute_TCP_checksum = 1; + tx_oob.short_oob.tx_compute_UDP_checksum = 0; + tx_oob.short_oob.tx_transport_header_offset = + m_pkt->l2_len + m_pkt->l3_len; + } else if ((m_pkt->ol_flags & RTE_MBUF_F_TX_L4_MASK) == RTE_MBUF_F_TX_TCP_CKSUM) { struct rte_tcp_hdr *tcp_hdr; @@ -405,34 +458,100 @@ mana_tx_burst(void *dpdk_txq, struct rte_mbuf **tx_pkts, uint16_t nb_pkts) m_pkt->pkt_len); /* Create SGL for packet data buffers */ - for (seg_idx = 0; seg_idx < m_pkt->nb_segs; seg_idx++) { + if (is_tso) { struct mana_mr_cache *mr = - mana_alloc_pmd_mr(&txq->mr_btree, priv, m_seg); + mana_alloc_pmd_mr(&txq->mr_btree, priv, m_pkt); - if (!mr) { - DP_LOG(ERR, "failed to get MR, pkt_idx %u", + if (unlikely(!mr)) { + DP_LOG(ERR, "failed to get MR for TSO hdr, pkt_idx %u", pkt_idx); break; } - sgl.gdma_sgl[seg_idx].address = - rte_cpu_to_le_64(rte_pktmbuf_mtod(m_seg, - uint64_t)); - sgl.gdma_sgl[seg_idx].size = m_seg->data_len; - sgl.gdma_sgl[seg_idx].memory_key = mr->lkey; + /* SGE0: packet header only (L2 + L3 + L4) */ + sgl.gdma_sgl[0].address = + rte_cpu_to_le_64(rte_pktmbuf_mtod(m_pkt, uint64_t)); + sgl.gdma_sgl[0].size = gso_hs; + sgl.gdma_sgl[0].memory_key = mr->lkey; + num_sge = 1; + + /* If first mbuf has payload data beyond the header */ + if (m_pkt->data_len > gso_hs) { + sgl.gdma_sgl[1].address = + rte_cpu_to_le_64(rte_pktmbuf_mtod(m_pkt, + uint64_t) + + gso_hs); + sgl.gdma_sgl[1].size = m_pkt->data_len - gso_hs; + sgl.gdma_sgl[1].memory_key = mr->lkey; + num_sge = 2; + } - DP_LOG(DEBUG, - "seg idx %u addr 0x%" PRIx64 " size %x key %x", - seg_idx, sgl.gdma_sgl[seg_idx].address, - sgl.gdma_sgl[seg_idx].size, - sgl.gdma_sgl[seg_idx].memory_key); + /* Subsequent mbuf segments */ + m_seg = m_pkt->next; + while (m_seg) { + if (unlikely(num_sge >= + MAX_SGL_ENTRIES_FOR_TRANSMIT)) { + DP_LOG(ERR, + "TSO SGL entries exceed max %d", + MAX_SGL_ENTRIES_FOR_TRANSMIT); + break; + } + mr = mana_alloc_pmd_mr(&txq->mr_btree, priv, + m_seg); + if (unlikely(!mr)) { + DP_LOG(ERR, + "failed to get MR for TSO seg, pkt_idx %u", + pkt_idx); + break; + } + sgl.gdma_sgl[num_sge].address = + rte_cpu_to_le_64(rte_pktmbuf_mtod(m_seg, + uint64_t)); + sgl.gdma_sgl[num_sge].size = m_seg->data_len; + sgl.gdma_sgl[num_sge].memory_key = mr->lkey; + num_sge++; + m_seg = m_seg->next; + } - m_seg = m_seg->next; - } + if (unlikely(m_seg != NULL || num_sge < 2)) { + DP_LOG(ERR, "failed to build TSO SGL, num_sge %u", + num_sge); + break; + } + } else { + for (seg_idx = 0; seg_idx < m_pkt->nb_segs; seg_idx++) { + struct mana_mr_cache *mr = + mana_alloc_pmd_mr(&txq->mr_btree, priv, + m_seg); + + if (!mr) { + DP_LOG(ERR, + "failed to get MR, pkt_idx %u", + pkt_idx); + break; + } + + sgl.gdma_sgl[seg_idx].address = + rte_cpu_to_le_64(rte_pktmbuf_mtod(m_seg, + uint64_t)); + sgl.gdma_sgl[seg_idx].size = m_seg->data_len; + sgl.gdma_sgl[seg_idx].memory_key = mr->lkey; + + DP_LOG(DEBUG, + "seg idx %u addr 0x%" PRIx64 " size %x key %x", + seg_idx, sgl.gdma_sgl[seg_idx].address, + sgl.gdma_sgl[seg_idx].size, + sgl.gdma_sgl[seg_idx].memory_key); + + m_seg = m_seg->next; + } - /* Skip this packet if we can't populate all segments */ - if (seg_idx != m_pkt->nb_segs) - break; + /* Skip this packet if we can't populate all segments */ + if (seg_idx != m_pkt->nb_segs) + break; + + num_sge = m_pkt->nb_segs; + } /* If we can at least queue post two WQEs and there are at * least two packets to send, use TX CQE suppression for the @@ -450,7 +569,7 @@ mana_tx_burst(void *dpdk_txq, struct rte_mbuf **tx_pkts, uint16_t nb_pkts) work_req.gdma_header.struct_size = sizeof(work_req); work_req.sgl = sgl.gdma_sgl; - work_req.num_sgl_elements = m_pkt->nb_segs; + work_req.num_sgl_elements = num_sge; if (tx_oob.short_oob.packet_format == SHORT_PACKET_FORMAT) work_req.inline_oob_size_in_bytes = sizeof(struct transmit_short_oob_v2); @@ -458,8 +577,14 @@ mana_tx_burst(void *dpdk_txq, struct rte_mbuf **tx_pkts, uint16_t nb_pkts) work_req.inline_oob_size_in_bytes = sizeof(struct transmit_oob_v2); work_req.inline_oob_data = &tx_oob; - work_req.flags = 0; - work_req.client_data_unit = NOT_USING_CLIENT_DATA_UNIT; + if (is_tso) { + work_req.flags = GDMA_WR_OOB_IN_SGL | + GDMA_WR_PAD_BY_SGE0; + work_req.client_data_unit = m_pkt->tso_segsz; + } else { + work_req.flags = 0; + work_req.client_data_unit = NOT_USING_CLIENT_DATA_UNIT; + } ret = gdma_post_work_request(&txq->gdma_sq, &work_req, &wqe_size_in_bu); -- 2.43.0 ^ permalink raw reply related [flat|nested] 13+ messages in thread
* Re: [PATCH 0/2] net/mana: fix transmit burst and add hardware TSO support 2026-10-12 7:36 [PATCH 0/2] net/mana: fix transmit burst and add hardware TSO support Tathagat Priyadarshi 2026-10-12 7:36 ` [PATCH 1/2] net/mana: fix transmit burst error handling Tathagat Priyadarshi 2026-10-12 7:36 ` [PATCH 2/2] net/mana: add hardware TSO support Tathagat Priyadarshi @ 2026-10-12 21:37 ` Stephen Hemminger 2026-10-26 21:21 ` [PATCH v2 0/2] net/mana: fix tx burst error handling and add " Tathagat Priyadarshi 2026-10-27 18:21 ` [PATCH v3 " Tathagat Priyadarshi 4 siblings, 0 replies; 13+ messages in thread From: Stephen Hemminger @ 2026-10-12 21:37 UTC (permalink / raw) To: Tathagat Priyadarshi; +Cc: dev, Long Li, Wei Hu On Mon, 12 Oct 2026 07:36:24 +0000 Tathagat Priyadarshi <tathagat.dpdk@gmail.com> wrote: > This patch series addresses transmit burst error handling and introduces > hardware TCP Segmentation Offload (TSO) support for the Microsoft Azure > MANA PMD. > > Patch 1 fixes the transmit burst loop in mana_tx_burst() to terminate > with break instead of continue upon packet errors. Under the DPDK > rte_eth_tx_burst() API contract, callers expect a contiguous prefix of > transmitted packets. Skipping a packet caused unsent packets to be treated > as unsent and freed by caller, followed by a second free upon hardware CQE > arrival. > > Patch 2 adds support for hardware TCP Segmentation Offload (TSO), > formatting SGL entries with packet headers in SGE0, handling GDMA work > request flags, advertising RTE_ETH_TX_OFFLOAD_TCP_TSO, and updating > the mana driver feature table. > > Tathagat Priyadarshi (2): > net/mana: fix transmit burst error handling > net/mana: add hardware TSO support Several problems with this patch series. Review: [PATCH 1/2] net/mana: fix transmit burst error handling [PATCH 2/2] net/mana: add hardware TSO support Series summary -------------- Patch 1 correctly identifies the bug (continue breaks the prefix contract and leads to double free) but the fix trades it for a head-of-line block. Patch 2 carries a stable-worthy bounds fix that should be in patch 1, and adds more break-on-bad-packet paths. Patch 1/2 --------- Error: break on a per-packet validation failure stalls the queue. A packet that exceeds max_send_sge, or whose MR lookup fails, will fail again on every retry. Applications that loop until all packets are sent (testpmd with retry, most forwarding apps) spin forever on it. Short return from tx_burst is for ring full / backpressure only; a bad packet must be dropped, counted, and consumed so the prefix contract still holds. Keep pkt_sent for the doorbell, and return a separate count of consumed packets: uint16_t nb_done = 0; ... if (unlikely(m_pkt->nb_segs > priv->max_send_sge)) { DP_LOG(ERR, "send packet segments %d exceeding max", m_pkt->nb_segs); goto drop; } ... if (seg_idx != m_pkt->nb_segs) goto drop; ... if (!ret) { ... pkt_sent++; nb_done++; ... } else { break; /* ring full: real backpressure */ } continue; drop: txq->stats.errors++; rte_pktmbuf_free(m_pkt); nb_done++; } ... return nb_done; Note the RTE_ARCH_32 short doorbell break happens after nb_done is incremented, so the count stays correct there. Info: CQE suppression is decided before the next packet is known to be valid. If the packet after a suppressed WQE fails, no completion is generated until a later burst posts an unsuppressed WQE; with no further traffic those mbufs are held indefinitely. This existed before, but break makes it reachable from any mid-burst failure rather than only the last packet. With drop-and-continue it is back to the last-packet case only. Patch 2/2 --------- Error: same as 1/2. The new "TSO header cannot be fragmented", "Invalid IP version for TSO" and "failed to build TSO SGL" paths all break and leave the packet at the head of the burst. They are permanent per-packet failures; use the drop path. Warning: the MAX_SGL_ENTRIES_FOR_TRANSMIT check fixes an existing bug and does not belong in a feature patch. sgl.gdma_sgl[] is sized MAX_SGL_ENTRIES_FOR_TRANSMIT (30) but the old non-TSO loop writes up to nb_segs <= max_send_sge entries, and max_send_sge comes from the device with nothing clamping it to 30. Move that check to patch 1 (or its own fix with Fixes/Cc stable), and clamp what is advertised: dev_info->tx_desc_lim.nb_seg_max = RTE_MIN(priv->max_send_sge, MAX_SGL_ENTRIES_FOR_TRANSMIT); dev_info->tx_desc_lim.nb_mtu_seg_max = RTE_MIN(priv->max_send_sge, MAX_SGL_ENTRIES_FOR_TRANSMIT); Also, a TSO packet with payload in the first segment needs one extra SGE. An application that honours nb_seg_max will still get drops; either advertise nb_seg_max - 1 when TSO is enabled or document the limit in mana.rst. Warning: RTE_MBUF_F_TX_TCP_SEG implies RTE_MBUF_F_TX_TCP_CKSUM (rte_mbuf_core.h). When TCP_SEG is set but the packet falls back to the non-TSO path (tso_segsz == 0 or no payload), the TCP checksum is only offloaded if the application also set TCP_CKSUM. Make the fallback honour the implied flag: } else if ((m_pkt->ol_flags & RTE_MBUF_F_TX_L4_MASK) == RTE_MBUF_F_TX_TCP_CKSUM || (m_pkt->ol_flags & RTE_MBUF_F_TX_TCP_SEG)) { Warning: no release note. New driver features go in doc/guides/rel_notes/release_26_11.rst under New Features. Warning: commit message is a bullet list restating the diff (which macro is defined where, which field is zeroed). Say what is added and why, and how it was tested: which Azure SKU / MANA firmware, IPv4 and IPv6, single and multi-segment mbufs, e.g. testpmd "csum set tcp hw" + "tso set <mss>" with iperf through the VM. Info: the num_sge >= MAX_SGL_ENTRIES_FOR_TRANSMIT check inside the TSO segment loop cannot trigger; expected_sge was already checked against the same limit and nb_segs bounds the walk. Drop it. Info: GDMA_WR_OOB_IN_SGL and GDMA_WR_PAD_BY_SGE0 are hardware work request flags. Put them next to struct gdma_work_request (the flags member comment already refers to them) rather than between the offload capability macros. ^ permalink raw reply [flat|nested] 13+ messages in thread
* [PATCH v2 0/2] net/mana: fix tx burst error handling and add TSO support 2026-10-12 7:36 [PATCH 0/2] net/mana: fix transmit burst and add hardware TSO support Tathagat Priyadarshi ` (2 preceding siblings ...) 2026-10-12 21:37 ` [PATCH 0/2] net/mana: fix transmit burst and " Stephen Hemminger @ 2026-10-26 21:21 ` Tathagat Priyadarshi 2026-10-26 21:21 ` [PATCH v2 1/2] net/mana: fix transmit burst error handling Tathagat Priyadarshi ` (2 more replies) 2026-10-27 18:21 ` [PATCH v3 " Tathagat Priyadarshi 4 siblings, 3 replies; 13+ messages in thread From: Tathagat Priyadarshi @ 2026-10-26 21:21 UTC (permalink / raw) To: dev; +Cc: weh, longli, stephen This series fixes error handling in the MANA transmit burst and adds hardware TCP segmentation offload (TSO). Patch 1 drops a packet that cannot be sent, counting it in stats.errors, instead of skipping or stalling on it, so rte_eth_tx_burst() keeps returning a contiguous prefix and a short return means only a full ring. It also bounds the segment count by the size of the SGL array. Patch 2 adds hardware TSO for IPv4 and IPv6 TCP packets. Testing on Azure Standard_D8ds_v6 (kernel 6.8.0-1067-azure, MANA PCI 7870:00:00.0; the driver reports no firmware version), two VMs on one subnet: - VM A ran testpmd txonly, modified locally to emit TCP packets with RTE_MBUF_F_TX_TCP_SEG set (not part of this series), after "csum set ip hw 0", "csum set tcp hw 0" and "tso set 1460 0". - VM B captured the frames with tcpdump on its kernel interface, GRO off. - A 4000-byte IPv4 packet in a single mbuf, in mbufs with a header-only first segment, and in mbufs with payload in the first segment, each arrived as 1514, 1514 and 1080 byte frames with consecutive TCP sequence numbers, per-segment IP IDs and valid IP and TCP checksums. - Plain UDP with "tso set 0" was unaffected. IPv6 TSO was not tested. v2: - Patch 1: * Drop unsendable packets and return a separate consumed count instead of breaking out of the burst; break only when the ring is full. * Move the MAX_SGL_ENTRIES_FOR_TRANSMIT check here from patch 2, limit nb_seg_max and nb_mtu_seg_max to it, and add Fixes and Cc stable. - Patch 2: * Use the drop path for all per-packet TSO failures. * Honour the TCP checksum implied by RTE_MBUF_F_TX_TCP_SEG when the packet is sent unsegmented. * Bound the TSO segment walk and check the SGE count against the expected one, instead of the unreachable check in the loop. * Move GDMA_WR_OOB_IN_SGL and GDMA_WR_PAD_BY_SGE0 next to struct gdma_work_request. * Document the extra SGE in mana.rst, add a release note, and rewrite the commit message. With the drop-and-continue path, CQE suppression is back to the original behavior where only an uncompleted final packet of a burst may wait for a subsequent burst to generate a CQE. Tathagat Priyadarshi (2): net/mana: fix transmit burst error handling net/mana: add hardware TSO support doc/guides/nics/features/mana.ini | 1 + doc/guides/nics/mana.rst | 8 ++ doc/guides/rel_notes/release_26_11.rst | 4 + drivers/net/mana/gdma.c | 5 + drivers/net/mana/mana.c | 4 +- drivers/net/mana/mana.h | 6 +- drivers/net/mana/tx.c | 191 +++++++++++++++++++++---- 7 files changed, 186 insertions(+), 33 deletions(-) -- 2.43.0 ^ permalink raw reply [flat|nested] 13+ messages in thread
* [PATCH v2 1/2] net/mana: fix transmit burst error handling 2026-10-26 21:21 ` [PATCH v2 0/2] net/mana: fix tx burst error handling and add " Tathagat Priyadarshi @ 2026-10-26 21:21 ` Tathagat Priyadarshi 2026-10-26 21:21 ` [PATCH v2 2/2] net/mana: add hardware TSO support Tathagat Priyadarshi 2026-10-27 14:02 ` [PATCH v2 0/2] net/mana: fix tx burst error handling and add " Stephen Hemminger 2 siblings, 0 replies; 13+ messages in thread From: Tathagat Priyadarshi @ 2026-10-26 21:21 UTC (permalink / raw) To: dev; +Cc: weh, longli, stephen, stable mana_tx_burst() skipped, using continue, a packet that exceeded the segment limit or whose SGL could not be populated. rte_eth_tx_burst() callers treat the return value as a contiguous prefix of consumed packets, so later packets were posted while the caller considered them unsent and freed them, and their completions then freed them again. Drop such a packet instead: count it in stats.errors, free it and include it in the returned count. Return early only when the ring is full, since a packet that fails validation would fail again on every retry and stall applications that loop until all packets are sent. sgl.gdma_sgl[] holds MAX_SGL_ENTRIES_FOR_TRANSMIT entries, but max_send_sge comes from the device and was not limited, so a packet with more segments could write past the array. Reject such packets and limit the advertised nb_seg_max and nb_mtu_seg_max to the same value. Fixes: 7f322844f427 ("net/mana: send packets") Fixes: d878cb09501d ("net/mana: report device info") Cc: stable@dpdk.org Signed-off-by: Tathagat Priyadarshi <tathagat.dpdk@gmail.com> --- drivers/net/mana/mana.c | 4 ++-- drivers/net/mana/tx.c | 18 ++++++++++++++---- 2 files changed, 16 insertions(+), 6 deletions(-) diff --git a/drivers/net/mana/mana.c b/drivers/net/mana/mana.c index 30b0ce90d6..5ddbbdc3d0 100644 --- a/drivers/net/mana/mana.c +++ b/drivers/net/mana/mana.c @@ -410,9 +410,9 @@ mana_dev_info_get(struct rte_eth_dev *dev, dev_info->tx_desc_lim.nb_max = RTE_MIN(priv->max_tx_desc, UINT16_MAX); dev_info->tx_desc_lim.nb_align = MIN_BUFFERS_PER_QUEUE; dev_info->tx_desc_lim.nb_seg_max = - RTE_MIN(priv->max_send_sge, UINT16_MAX); + RTE_MIN(priv->max_send_sge, MAX_SGL_ENTRIES_FOR_TRANSMIT); dev_info->tx_desc_lim.nb_mtu_seg_max = - RTE_MIN(priv->max_send_sge, UINT16_MAX); + RTE_MIN(priv->max_send_sge, MAX_SGL_ENTRIES_FOR_TRANSMIT); /* Speed */ dev_info->speed_capa = RTE_ETH_LINK_SPEED_100G; diff --git a/drivers/net/mana/tx.c b/drivers/net/mana/tx.c index 10f2212b5d..16e63e89c9 100644 --- a/drivers/net/mana/tx.c +++ b/drivers/net/mana/tx.c @@ -192,6 +192,7 @@ mana_tx_burst(void *dpdk_txq, struct rte_mbuf **tx_pkts, uint16_t nb_pkts) int ret; void *db_page; uint16_t pkt_sent = 0; + uint16_t nb_done = 0; uint32_t num_comp, i; uint32_t expected = 0; #ifdef RTE_ARCH_32 @@ -278,10 +279,11 @@ mana_tx_burst(void *dpdk_txq, struct rte_mbuf **tx_pkts, uint16_t nb_pkts) break; /* Drop the packet if it exceeds max segments */ - if (m_pkt->nb_segs > priv->max_send_sge) { + if (unlikely(m_pkt->nb_segs > priv->max_send_sge || + m_pkt->nb_segs > MAX_SGL_ENTRIES_FOR_TRANSMIT)) { DP_LOG(ERR, "send packet segments %d exceeding max", m_pkt->nb_segs); - continue; + goto drop; } /* Fill in the oob */ @@ -432,7 +434,7 @@ mana_tx_burst(void *dpdk_txq, struct rte_mbuf **tx_pkts, uint16_t nb_pkts) /* Skip this packet if we can't populate all segments */ if (seg_idx != m_pkt->nb_segs) - continue; + goto drop; /* If we can at least queue post two WQEs and there are at * least two packets to send, use TX CQE suppression for the @@ -477,6 +479,7 @@ mana_tx_burst(void *dpdk_txq, struct rte_mbuf **tx_pkts, uint16_t nb_pkts) txq->desc_ring_len++; pkt_sent++; + nb_done++; DP_LOG(DEBUG, "nb_pkts %u pkt[%d] sent", nb_pkts, pkt_idx); @@ -499,6 +502,13 @@ mana_tx_burst(void *dpdk_txq, struct rte_mbuf **tx_pkts, uint16_t nb_pkts) pkt_idx, ret); break; } + + continue; + +drop: + txq->stats.errors++; + rte_pktmbuf_free(m_pkt); + nb_done++; } /* Ring hardware door bell */ @@ -523,5 +533,5 @@ mana_tx_burst(void *dpdk_txq, struct rte_mbuf **tx_pkts, uint16_t nb_pkts) rte_atomic_fetch_and_explicit(&txq->burst_state, ~(uint32_t)1, rte_memory_order_release); - return pkt_sent; + return nb_done; } -- 2.43.0 ^ permalink raw reply related [flat|nested] 13+ messages in thread
* [PATCH v2 2/2] net/mana: add hardware TSO support 2026-10-26 21:21 ` [PATCH v2 0/2] net/mana: fix tx burst error handling and add " Tathagat Priyadarshi 2026-10-26 21:21 ` [PATCH v2 1/2] net/mana: fix transmit burst error handling Tathagat Priyadarshi @ 2026-10-26 21:21 ` Tathagat Priyadarshi 2026-10-27 14:02 ` [PATCH v2 0/2] net/mana: fix tx burst error handling and add " Stephen Hemminger 2 siblings, 0 replies; 13+ messages in thread From: Tathagat Priyadarshi @ 2026-10-26 21:21 UTC (permalink / raw) To: dev; +Cc: weh, longli, stephen Advertise RTE_ETH_TX_OFFLOAD_TCP_TSO for IPv4 and IPv6 TCP packets. Hardware TSO offloads TCP segmentation to the adapter, reducing host CPU overhead and allowing transmit bursts of up to 64KB TCP packets at wire speed. For a packet with RTE_MBUF_F_TX_TCP_SEG and a non-zero tso_segsz, put the L2/L3/L4 headers alone in the first SGE and the payload in the following ones, pass tso_segsz as the client data unit, and request IP and TCP checksum offload with the pseudo-header checksum seeded. A first segment that also carries payload needs one extra SGE, which is documented in mana.rst. A packet that cannot be sent as TSO (headers split across segments, no IP version flag, SGL build failure) is dropped. A packet with TCP_SEG but no tso_segsz or no payload is sent unsegmented, with the TCP checksum offloaded as TCP_SEG implies. Tested on Azure Standard_D8ds_v6 (kernel 6.8.0-1067-azure, MANA PCI 7870:00:00.0; driver reports no firmware version) with DPDK testpmd: - Single segment IPv4 TCP (4000-byte packet, MSS 1460): verified segmentation into 1514, 1514, and 1080 byte frames on receiver. - Multi-segment with header alone in first segment (54, 1400, 1400, 1146): verified identical segmentation. - Multi-segment with payload in first segment (200, 1400, 1400, 1000): verified extra SGE allocation and segmentation without drops. - Non-TSO fallback (MSS 0): verified transmission as single unsegmented frames. - Verified consecutive TCP sequence numbers, incrementing IP IDs, and valid IP and TCP checksums via tcpdump on peer VM. Signed-off-by: Tathagat Priyadarshi <tathagat.dpdk@gmail.com> --- doc/guides/nics/features/mana.ini | 1 + doc/guides/nics/mana.rst | 8 ++ doc/guides/rel_notes/release_26_11.rst | 4 + drivers/net/mana/gdma.c | 5 + drivers/net/mana/mana.h | 6 +- drivers/net/mana/tx.c | 179 +++++++++++++++++++++---- 6 files changed, 173 insertions(+), 30 deletions(-) diff --git a/doc/guides/nics/features/mana.ini b/doc/guides/nics/features/mana.ini index c7c8215aab..6a4fb0a89e 100644 --- a/doc/guides/nics/features/mana.ini +++ b/doc/guides/nics/features/mana.ini @@ -10,6 +10,7 @@ Rx interrupt = Y RSS hash = Y L3 checksum offload = Y L4 checksum offload = Y +TSO = Y MTU update = Y Multiprocess aware = Y Linux = Y diff --git a/doc/guides/nics/mana.rst b/doc/guides/nics/mana.rst index 25580c1082..6752ab8bc9 100644 --- a/doc/guides/nics/mana.rst +++ b/doc/guides/nics/mana.rst @@ -109,3 +109,11 @@ To distinguish a PCI hot-remove from a service reset, the driver registers for PCI device removal events. This requires the application to call ``rte_dev_event_monitor_start()`` for removal events to be delivered (e.g. testpmd ``--hot-plug-handling`` option). + +Limitations +----------- + +* For TSO packets where the first segment contains packet payload beyond the + headers, an additional scatter-gather entry is required for the header. + In this case, the effective maximum number of segments supported is + ``nb_seg_max - 1``. diff --git a/doc/guides/rel_notes/release_26_11.rst b/doc/guides/rel_notes/release_26_11.rst index be758a0c02..7e5f463f88 100644 --- a/doc/guides/rel_notes/release_26_11.rst +++ b/doc/guides/rel_notes/release_26_11.rst @@ -164,6 +164,10 @@ New Features Added ``fdir_buffer_size`` devarg to select the Flow Director table size (``64k``, ``128k`` or ``256k``) at probe time. The default remains ``64k``. +* **Updated Microsoft MANA PMD.** + + * Added hardware TCP Segmentation Offload (TSO) support for IPv4 and IPv6. + * **Updated NVIDIA mlx5 ethernet driver.** * Increased the maximum number of secondary unicast MAC addresses from 128 to up to 4096 diff --git a/drivers/net/mana/gdma.c b/drivers/net/mana/gdma.c index a92aef59d2..65896b6a69 100644 --- a/drivers/net/mana/gdma.c +++ b/drivers/net/mana/gdma.c @@ -38,6 +38,11 @@ write_dma_client_oob(uint8_t *work_queue_buffer_pointer, header->inline_client_oob_size_in_dwords = client_oob_size / sizeof(uint32_t); header->client_data_unit = work_request->client_data_unit; + if (work_request->flags & GDMA_WR_OOB_IN_SGL) { + header->client_oob_in_sgl = 1; + if (work_request->flags & GDMA_WR_PAD_BY_SGE0) + header->last_v_bytes = work_request->sgl[0].size; + } DP_LOG(DEBUG, "queue buf %p sgl %u oob_h %u du %u oob_buf %p oob_b %u", work_queue_buffer_pointer, header->num_sgl_entries, diff --git a/drivers/net/mana/mana.h b/drivers/net/mana/mana.h index 79879b7626..5ac316f4e3 100644 --- a/drivers/net/mana/mana.h +++ b/drivers/net/mana/mana.h @@ -33,7 +33,8 @@ struct mana_shared_data { RTE_ETH_TX_OFFLOAD_MULTI_SEGS | \ RTE_ETH_TX_OFFLOAD_IPV4_CKSUM | \ RTE_ETH_TX_OFFLOAD_TCP_CKSUM | \ - RTE_ETH_TX_OFFLOAD_UDP_CKSUM) + RTE_ETH_TX_OFFLOAD_UDP_CKSUM | \ + RTE_ETH_TX_OFFLOAD_TCP_TSO) #define INDIRECTION_TABLE_NUM_ELEMENTS 64 #define TOEPLITZ_HASH_KEY_SIZE_IN_BYTES 40 @@ -208,6 +209,9 @@ struct one_sgl { struct gdma_sgl_element gdma_sgl[MAX_SGL_ENTRIES_FOR_TRANSMIT]; }; +#define GDMA_WR_OOB_IN_SGL (1 << 0) +#define GDMA_WR_PAD_BY_SGE0 (1 << 1) + struct gdma_work_request { struct gdma_header gdma_header; struct gdma_sgl_element *sgl; diff --git a/drivers/net/mana/tx.c b/drivers/net/mana/tx.c index 16e63e89c9..1421f6f7c6 100644 --- a/drivers/net/mana/tx.c +++ b/drivers/net/mana/tx.c @@ -274,15 +274,35 @@ mana_tx_burst(void *dpdk_txq, struct rte_mbuf **tx_pkts, uint16_t nb_pkts) struct transmit_oob_v2 tx_oob; struct one_sgl sgl; uint16_t seg_idx; + uint16_t num_sge; + uint16_t expected_sge; + bool is_tso = false; + uint32_t gso_hs = 0; if (txq->desc_ring_len >= txq->num_desc) break; - /* Drop the packet if it exceeds max segments */ - if (unlikely(m_pkt->nb_segs > priv->max_send_sge || - m_pkt->nb_segs > MAX_SGL_ENTRIES_FOR_TRANSMIT)) { + if (m_pkt->ol_flags & RTE_MBUF_F_TX_TCP_SEG) { + gso_hs = m_pkt->l2_len + m_pkt->l3_len + m_pkt->l4_len; + if (unlikely(m_pkt->data_len < gso_hs)) { + DP_LOG(ERR, "TSO header cannot be fragmented"); + goto drop; + } + if (likely(gso_hs != 0 && m_pkt->pkt_len > gso_hs && + m_pkt->tso_segsz != 0)) + is_tso = true; + } + + /* Drop the packet if it exceeds max segments. A TSO packet + * with payload in the first segment needs an extra SGE for + * the header. + */ + expected_sge = m_pkt->nb_segs + + (is_tso && m_pkt->data_len > gso_hs ? 1 : 0); + if (unlikely(expected_sge > priv->max_send_sge || + expected_sge > MAX_SGL_ENTRIES_FOR_TRANSMIT)) { DP_LOG(ERR, "send packet segments %d exceeding max", - m_pkt->nb_segs); + expected_sge); goto drop; } @@ -307,8 +327,45 @@ mana_tx_burst(void *dpdk_txq, struct rte_mbuf **tx_pkts, uint16_t nb_pkts) tx_oob.short_oob.tx_compute_IP_header_checksum = m_pkt->ol_flags & RTE_MBUF_F_TX_IP_CKSUM ? 1 : 0; - if ((m_pkt->ol_flags & RTE_MBUF_F_TX_L4_MASK) == - RTE_MBUF_F_TX_TCP_CKSUM) { + if (is_tso) { + struct rte_tcp_hdr *tcp_hdr = + rte_pktmbuf_mtod_offset(m_pkt, + struct rte_tcp_hdr *, + m_pkt->l2_len + m_pkt->l3_len); + + if (m_pkt->ol_flags & RTE_MBUF_F_TX_IPV4) { + struct rte_ipv4_hdr *ip_hdr = + rte_pktmbuf_mtod_offset(m_pkt, + struct rte_ipv4_hdr *, + m_pkt->l2_len); + + ip_hdr->total_length = 0; + ip_hdr->hdr_checksum = 0; + tcp_hdr->cksum = rte_ipv4_phdr_cksum(ip_hdr, + m_pkt->ol_flags); + } else if (m_pkt->ol_flags & RTE_MBUF_F_TX_IPV6) { + struct rte_ipv6_hdr *ip_hdr = + rte_pktmbuf_mtod_offset(m_pkt, + struct rte_ipv6_hdr *, + m_pkt->l2_len); + + ip_hdr->payload_len = 0; + tcp_hdr->cksum = rte_ipv6_phdr_cksum(ip_hdr, + m_pkt->ol_flags); + } else { + DP_LOG(ERR, "Invalid IP version for TSO"); + goto drop; + } + + tx_oob.short_oob.tx_compute_IP_header_checksum = + (m_pkt->ol_flags & RTE_MBUF_F_TX_IPV4) ? 1 : 0; + tx_oob.short_oob.tx_compute_TCP_checksum = 1; + tx_oob.short_oob.tx_compute_UDP_checksum = 0; + tx_oob.short_oob.tx_transport_header_offset = + m_pkt->l2_len + m_pkt->l3_len; + } else if ((m_pkt->ol_flags & RTE_MBUF_F_TX_L4_MASK) == + RTE_MBUF_F_TX_TCP_CKSUM || + (m_pkt->ol_flags & RTE_MBUF_F_TX_TCP_SEG)) { struct rte_tcp_hdr *tcp_hdr; /* HW needs partial TCP checksum */ @@ -407,34 +464,92 @@ mana_tx_burst(void *dpdk_txq, struct rte_mbuf **tx_pkts, uint16_t nb_pkts) m_pkt->pkt_len); /* Create SGL for packet data buffers */ - for (seg_idx = 0; seg_idx < m_pkt->nb_segs; seg_idx++) { + if (is_tso) { struct mana_mr_cache *mr = - mana_alloc_pmd_mr(&txq->mr_btree, priv, m_seg); + mana_alloc_pmd_mr(&txq->mr_btree, priv, m_pkt); - if (!mr) { - DP_LOG(ERR, "failed to get MR, pkt_idx %u", + if (unlikely(!mr)) { + DP_LOG(ERR, "failed to get MR for TSO hdr, pkt_idx %u", pkt_idx); - break; + goto drop; } - sgl.gdma_sgl[seg_idx].address = - rte_cpu_to_le_64(rte_pktmbuf_mtod(m_seg, - uint64_t)); - sgl.gdma_sgl[seg_idx].size = m_seg->data_len; - sgl.gdma_sgl[seg_idx].memory_key = mr->lkey; + /* SGE0: packet header only (L2 + L3 + L4) */ + sgl.gdma_sgl[0].address = + rte_cpu_to_le_64(rte_pktmbuf_mtod(m_pkt, uint64_t)); + sgl.gdma_sgl[0].size = gso_hs; + sgl.gdma_sgl[0].memory_key = mr->lkey; + num_sge = 1; + + /* If first mbuf has payload data beyond the header */ + if (m_pkt->data_len > gso_hs) { + sgl.gdma_sgl[1].address = + rte_cpu_to_le_64(rte_pktmbuf_mtod(m_pkt, + uint64_t) + + gso_hs); + sgl.gdma_sgl[1].size = m_pkt->data_len - gso_hs; + sgl.gdma_sgl[1].memory_key = mr->lkey; + num_sge = 2; + } - DP_LOG(DEBUG, - "seg idx %u addr 0x%" PRIx64 " size %x key %x", - seg_idx, sgl.gdma_sgl[seg_idx].address, - sgl.gdma_sgl[seg_idx].size, - sgl.gdma_sgl[seg_idx].memory_key); + /* Subsequent mbuf segments */ + m_seg = m_pkt->next; + while (m_seg && num_sge < expected_sge) { + mr = mana_alloc_pmd_mr(&txq->mr_btree, priv, + m_seg); + if (unlikely(!mr)) { + DP_LOG(ERR, + "failed to get MR for TSO seg, pkt_idx %u", + pkt_idx); + goto drop; + } + sgl.gdma_sgl[num_sge].address = + rte_cpu_to_le_64(rte_pktmbuf_mtod(m_seg, + uint64_t)); + sgl.gdma_sgl[num_sge].size = m_seg->data_len; + sgl.gdma_sgl[num_sge].memory_key = mr->lkey; + num_sge++; + m_seg = m_seg->next; + } - m_seg = m_seg->next; - } + if (unlikely(m_seg != NULL || num_sge != expected_sge)) { + DP_LOG(ERR, "failed to build TSO SGL, num_sge %u", + num_sge); + goto drop; + } + } else { + for (seg_idx = 0; seg_idx < m_pkt->nb_segs; seg_idx++) { + struct mana_mr_cache *mr = + mana_alloc_pmd_mr(&txq->mr_btree, priv, + m_seg); + + if (!mr) { + DP_LOG(ERR, "failed to get MR, pkt_idx %u", + pkt_idx); + break; + } + + sgl.gdma_sgl[seg_idx].address = + rte_cpu_to_le_64(rte_pktmbuf_mtod(m_seg, + uint64_t)); + sgl.gdma_sgl[seg_idx].size = m_seg->data_len; + sgl.gdma_sgl[seg_idx].memory_key = mr->lkey; + + DP_LOG(DEBUG, + "seg idx %u addr 0x%" PRIx64 " size %x key %x", + seg_idx, sgl.gdma_sgl[seg_idx].address, + sgl.gdma_sgl[seg_idx].size, + sgl.gdma_sgl[seg_idx].memory_key); + + m_seg = m_seg->next; + } - /* Skip this packet if we can't populate all segments */ - if (seg_idx != m_pkt->nb_segs) - goto drop; + /* Skip this packet if we can't populate all segments */ + if (seg_idx != m_pkt->nb_segs) + goto drop; + + num_sge = m_pkt->nb_segs; + } /* If we can at least queue post two WQEs and there are at * least two packets to send, use TX CQE suppression for the @@ -452,7 +567,7 @@ mana_tx_burst(void *dpdk_txq, struct rte_mbuf **tx_pkts, uint16_t nb_pkts) work_req.gdma_header.struct_size = sizeof(work_req); work_req.sgl = sgl.gdma_sgl; - work_req.num_sgl_elements = m_pkt->nb_segs; + work_req.num_sgl_elements = num_sge; if (tx_oob.short_oob.packet_format == SHORT_PACKET_FORMAT) work_req.inline_oob_size_in_bytes = sizeof(struct transmit_short_oob_v2); @@ -460,8 +575,14 @@ mana_tx_burst(void *dpdk_txq, struct rte_mbuf **tx_pkts, uint16_t nb_pkts) work_req.inline_oob_size_in_bytes = sizeof(struct transmit_oob_v2); work_req.inline_oob_data = &tx_oob; - work_req.flags = 0; - work_req.client_data_unit = NOT_USING_CLIENT_DATA_UNIT; + if (is_tso) { + work_req.flags = GDMA_WR_OOB_IN_SGL | + GDMA_WR_PAD_BY_SGE0; + work_req.client_data_unit = m_pkt->tso_segsz; + } else { + work_req.flags = 0; + work_req.client_data_unit = NOT_USING_CLIENT_DATA_UNIT; + } ret = gdma_post_work_request(&txq->gdma_sq, &work_req, &wqe_size_in_bu); -- 2.43.0 ^ permalink raw reply related [flat|nested] 13+ messages in thread
* Re: [PATCH v2 0/2] net/mana: fix tx burst error handling and add TSO support 2026-10-26 21:21 ` [PATCH v2 0/2] net/mana: fix tx burst error handling and add " Tathagat Priyadarshi 2026-10-26 21:21 ` [PATCH v2 1/2] net/mana: fix transmit burst error handling Tathagat Priyadarshi 2026-10-26 21:21 ` [PATCH v2 2/2] net/mana: add hardware TSO support Tathagat Priyadarshi @ 2026-10-27 14:02 ` Stephen Hemminger 2 siblings, 0 replies; 13+ messages in thread From: Stephen Hemminger @ 2026-10-27 14:02 UTC (permalink / raw) To: Tathagat Priyadarshi; +Cc: dev, weh, longli On Mon, 26 Oct 2026 21:21:23 +0000 Tathagat Priyadarshi <tathagat.dpdk@gmail.com> wrote: > This series fixes error handling in the MANA transmit burst and adds > hardware TCP segmentation offload (TSO). > > Patch 1 drops a packet that cannot be sent, counting it in stats.errors, > instead of skipping or stalling on it, so rte_eth_tx_burst() keeps > returning a contiguous prefix and a short return means only a full ring. > It also bounds the segment count by the size of the SGL array. > > Patch 2 adds hardware TSO for IPv4 and IPv6 TCP packets. > > Testing on Azure Standard_D8ds_v6 (kernel 6.8.0-1067-azure, MANA PCI > 7870:00:00.0; the driver reports no firmware version), two VMs on one > subnet: > - VM A ran testpmd txonly, modified locally to emit TCP packets with > RTE_MBUF_F_TX_TCP_SEG set (not part of this series), after > "csum set ip hw 0", "csum set tcp hw 0" and "tso set 1460 0". > - VM B captured the frames with tcpdump on its kernel interface, GRO off. > - A 4000-byte IPv4 packet in a single mbuf, in mbufs with a header-only > first segment, and in mbufs with payload in the first segment, each > arrived as 1514, 1514 and 1080 byte frames with consecutive TCP > sequence numbers, per-segment IP IDs and valid IP and TCP checksums. > - Plain UDP with "tso set 0" was unaffected. > IPv6 TSO was not tested. > > v2: > - Patch 1: > * Drop unsendable packets and return a separate consumed count instead > of breaking out of the burst; break only when the ring is full. > * Move the MAX_SGL_ENTRIES_FOR_TRANSMIT check here from patch 2, limit > nb_seg_max and nb_mtu_seg_max to it, and add Fixes and Cc stable. > - Patch 2: > * Use the drop path for all per-packet TSO failures. > * Honour the TCP checksum implied by RTE_MBUF_F_TX_TCP_SEG when the > packet is sent unsegmented. > * Bound the TSO segment walk and check the SGE count against the > expected one, instead of the unreachable check in the loop. > * Move GDMA_WR_OOB_IN_SGL and GDMA_WR_PAD_BY_SGE0 next to > struct gdma_work_request. > * Document the extra SGE in mana.rst, add a release note, and rewrite > the commit message. > > With the drop-and-continue path, CQE suppression is back to the original > behavior where only an uncompleted final packet of a burst may wait for a > subsequent burst to generate a CQE. > > Tathagat Priyadarshi (2): > net/mana: fix transmit burst error handling > net/mana: add hardware TSO support > > doc/guides/nics/features/mana.ini | 1 + > doc/guides/nics/mana.rst | 8 ++ > doc/guides/rel_notes/release_26_11.rst | 4 + > drivers/net/mana/gdma.c | 5 + > drivers/net/mana/mana.c | 4 +- > drivers/net/mana/mana.h | 6 +- > drivers/net/mana/tx.c | 191 +++++++++++++++++++++---- > 7 files changed, 186 insertions(+), 33 deletions(-) > AI review still turning up errors here. Review: [PATCH v2 0/2] net/mana: tx error handling and TSO Patch 1/2 is a good fix. Patch 2/2 has two hardware field width truncations and a checksum bug in the new non-TSO fallback. [PATCH v2 1/2] net/mana: fix transmit burst error handling Info Dropping a packet now happens mid-burst, after the previous WQE may have been posted with suppress_tx_CQE_generation set (the decision is made on pkt_idx + 1 < nb_pkts). If every remaining packet in the burst is dropped, the last posted WQE has no CQE and its mbufs (and any suppressed ones before it) are not reclaimed until a later burst posts another WQE. Same for the post-failure break and the 32-bit wqe_count break. Pre-existing, not a blocker here, but worth a follow-up: decide suppression after validation of the next packet, or never suppress the last WQE before ringing the doorbell. [PATCH v2 2/2] net/mana: add hardware TSO support Error Non-TSO fallback computes the wrong pseudo-header checksum. The fallback branch is entered for TCP_SEG packets with tso_segsz == 0 or no payload, and passes m_pkt->ol_flags (with RTE_MBUF_F_TX_TCP_SEG still set) to rte_ipv4_phdr_cksum() / rte_ipv6_phdr_cksum(). Those helpers use length 0 when TCP_SEG is set, so the seed is wrong for an unsegmented frame and the hardware emits a bad TCP checksum. Mask the flag: uint64_t ol = m_pkt->ol_flags & ~RTE_MBUF_F_TX_TCP_SEG; ... tcp_hdr->cksum = rte_ipv4_phdr_cksum(ip_hdr, ol); Also, TCP_SEG with tso_segsz == 0 or l4_len == 0 is invalid input per the mbuf API. With gso_hs == 0 the fallback writes cksum at l2_len + l3_len == 0, corrupting the Ethernet header. Drop these instead of sending them: if (m_pkt->ol_flags & RTE_MBUF_F_TX_TCP_SEG) { if (unlikely(m_pkt->tso_segsz == 0 || m_pkt->l4_len == 0 || m_pkt->data_len < gso_hs)) goto drop; is_tso = m_pkt->pkt_len > gso_hs; } Error client_data_unit is a 14-bit field in struct gdma_wqe_dma_oob, but tso_segsz is uint16_t. Any tso_segsz >= 16384 is silently truncated; 16384 becomes 0 == NOT_USING_CLIENT_DATA_UNIT and a 64KB frame is handed to the wire unsegmented. Reject it: #define MANA_TSO_MAX_SEGSZ ((1u << 14) - 1) if (unlikely(m_pkt->tso_segsz > MANA_TSO_MAX_SEGSZ)) goto drop; Error last_v_bytes is an 8-bit field, but gso_hs is l2_len + l3_len + l4_len and can exceed 255 (IPv6 extension headers, VLAN plus TCP options). gdma.c writes sgl[0].size into it and the value wraps. Drop TSO packets with gso_hs > UINT8_MAX, or state the real hardware header limit and check against that. Warning The extra SGE for a TSO header split is documented as a limitation but not reflected in dev_info. tx_desc_lim.nb_seg_max is the TSO limit (nb_mtu_seg_max is the non-TSO one), so applications and rte_gso trusting it build packets that are then dropped. Advertise the real limit and drop the mana.rst paragraph: uint16_t max_sge = RTE_MIN(priv->max_send_sge, MAX_SGL_ENTRIES_FOR_TRANSMIT); dev_info->tx_desc_lim.nb_seg_max = max_sge - 1; dev_info->tx_desc_lim.nb_mtu_seg_max = max_sge; Warning No bounds on the TSO request beyond the above. What is the maximum payload (pkt_len - gso_hs) and the minimum MSS the adapter accepts? Check both in the TSO validation and drop out-of-range packets, rather than leaving it to the device. Info The TSO and non-TSO SGL loops duplicate the MR lookup and SGE fill. One loop with a start offset on the first segment covers both: uint32_t off = 0; num_sge = 0; if (is_tso) { /* SGE0: headers only */ if (fill_sge(&sgl, num_sge++, m_pkt, 0, gso_hs, mr)) goto drop; off = gso_hs; } for (m_seg = m_pkt; m_seg; m_seg = m_seg->next, off = 0) { if (m_seg->data_len == off) continue; if (fill_sge(&sgl, num_sge++, m_seg, off, m_seg->data_len - off, mr)) goto drop; } With expected_sge already checked against the array size, the m_seg != NULL || num_sge != expected_sge re-check goes away. Info Header rewriting (total_length, hdr_checksum, cksum) in tx_burst modifies data the application may share (refcnt > 1, external buffers). The existing TCP_CKSUM path does the same, so not new, but the conventional place is tx_pkt_prepare. A mana_tx_prepare() using rte_validate_tx_offload() and rte_net_intel_cksum_prepare() would also move the segment-count checks out of the fast path. Info Commit message: drop "allowing transmit bursts of up to 64KB TCP packets at wire speed". The test log is useful, but belongs below the "---" line, not in the permanent history. ^ permalink raw reply [flat|nested] 13+ messages in thread
* [PATCH v3 0/2] net/mana: fix tx burst error handling and add TSO support 2026-10-12 7:36 [PATCH 0/2] net/mana: fix transmit burst and add hardware TSO support Tathagat Priyadarshi ` (3 preceding siblings ...) 2026-10-26 21:21 ` [PATCH v2 0/2] net/mana: fix tx burst error handling and add " Tathagat Priyadarshi @ 2026-10-27 18:21 ` Tathagat Priyadarshi 2026-10-27 18:21 ` [PATCH v3 1/2] net/mana: fix transmit burst error handling Tathagat Priyadarshi ` (2 more replies) 4 siblings, 3 replies; 13+ messages in thread From: Tathagat Priyadarshi @ 2026-10-27 18:21 UTC (permalink / raw) To: dev; +Cc: weh, longli, stephen This series fixes transmit burst error handling and adds hardware TCP segmentation offload (TSO) for the Microsoft Azure MANA PMD. Patch 1 drops packets that fail validation, counting them in stats.errors and consuming them, so rte_eth_tx_burst() maintains a contiguous prefix and short returns reflect only ring backpressure. It also clamps segment limits to MAX_SGL_ENTRIES_FOR_TRANSMIT. Patch 2 implements hardware TSO for IPv4 and IPv6 TCP traffic, formatting SGE0 for headers and subsequent SGEs for payload, handling GDMA work request flags, advertising RTE_ETH_TX_OFFLOAD_TCP_TSO, and handling non-TSO fallback checksums. Hardware TSO segmentation was verified on Azure Standard_D8ds_v6 (kernel 6.8.0-1067-azure, MANA PCI 7870:00:00.0; driver reports no firmware version) between two VMs on one subnet using an earlier revision of this series: - VM A ran testpmd txonly, modified locally to emit TCP packets with RTE_MBUF_F_TX_TCP_SEG set (not part of this series), after "csum set ip hw 0", "csum set tcp hw 0" and "tso set 1460 0". - VM B captured the frames with tcpdump on its kernel interface, GRO off. - A 4000-byte IPv4 packet in a single mbuf, in mbufs with a header-only first segment, and in mbufs with payload in the first segment, each arrived as 1514, 1514 and 1080 byte frames with consecutive TCP sequence numbers, per-segment IP IDs and valid IP and TCP checksums. - Plain UDP with "tso set 0" was unaffected. - The drop paths (packets exceeding segment limit, oversized MSS or header) and IPv6 TSO were not exercised on hardware. The v3 changes (hardware field bounds for MSS and header length, loop bound, unsegmented fallback checksum masking, and dev_info limit advertisement) were compile-tested but have not been re-run on hardware. v3: - Patch 1: no changes. - Patch 2: * Mask RTE_MBUF_F_TX_TCP_SEG in the non-TSO fallback branch so rte_ipv4_phdr_cksum / rte_ipv6_phdr_cksum calculate the full L4 length into the pseudo-header checksum. * Reject invalid TSO packets (tso_segsz > MANA_TSO_MAX_SEGSZ or gso_hs > UINT8_MAX) to prevent hardware bitfield truncations. * Reflect the extra SGE for TSO in dev_info by setting nb_seg_max to max_sge - 1 (keeping nb_mtu_seg_max as max_sge) instead of a mana.rst limitation note. * Bound the TSO segment walk by expected_sge and verify SGE count. * Streamline commit message and cover letter. v2: - Patch 1: * Drop unsendable packets and return a separate consumed count (nb_done) instead of breaking out of the burst; break only when the ring is full. * Move MAX_SGL_ENTRIES_FOR_TRANSMIT clamp from patch 2 to patch 1, and clamp dev_info nb_seg_max and nb_mtu_seg_max. - Patch 2: * Use the drop path for all per-packet TSO validation failures. * Honour implied TCP checksum when RTE_MBUF_F_TX_TCP_SEG is set but the packet is unsegmented. * Move GDMA_WR_OOB_IN_SGL and GDMA_WR_PAD_BY_SGE0 adjacent to struct gdma_work_request. * Add release notes entry and update mana.ini feature matrix. Tathagat Priyadarshi (2): net/mana: fix transmit burst error handling net/mana: add hardware TSO support doc/guides/nics/features/mana.ini | 1 + doc/guides/rel_notes/release_26_11.rst | 4 + drivers/net/mana/gdma.c | 5 + drivers/net/mana/mana.c | 9 +- drivers/net/mana/mana.h | 9 +- drivers/net/mana/tx.c | 206 +++++++++++++++++++++---- 6 files changed, 195 insertions(+), 39 deletions(-) -- 2.43.0 ^ permalink raw reply [flat|nested] 13+ messages in thread
* [PATCH v3 1/2] net/mana: fix transmit burst error handling 2026-10-27 18:21 ` [PATCH v3 " Tathagat Priyadarshi @ 2026-10-27 18:21 ` Tathagat Priyadarshi 2026-10-27 18:21 ` [PATCH v3 2/2] net/mana: add hardware TSO support Tathagat Priyadarshi 2026-10-28 14:00 ` [PATCH v3 0/2] net/mana: fix tx burst error handling and add " Stephen Hemminger 2 siblings, 0 replies; 13+ messages in thread From: Tathagat Priyadarshi @ 2026-10-27 18:21 UTC (permalink / raw) To: dev; +Cc: weh, longli, stephen, stable mana_tx_burst() skipped, using continue, a packet that exceeded the segment limit or whose SGL could not be populated. rte_eth_tx_burst() callers treat the return value as a contiguous prefix of consumed packets, so later packets were posted while the caller considered them unsent and freed them, and their completions then freed them again. Drop such a packet instead: count it in stats.errors, free it and include it in the returned count. Return early only when the ring is full, since a packet that fails validation would fail again on every retry and stall applications that loop until all packets are sent. sgl.gdma_sgl[] holds MAX_SGL_ENTRIES_FOR_TRANSMIT entries, but max_send_sge comes from the device and was not limited, so a packet with more segments could write past the array. Reject such packets and limit the advertised nb_seg_max and nb_mtu_seg_max to the same value. Fixes: 7f322844f427 ("net/mana: send packets") Fixes: d878cb09501d ("net/mana: report device info") Cc: stable@dpdk.org Signed-off-by: Tathagat Priyadarshi <tathagat.dpdk@gmail.com> --- v2: - Drop unsendable packets and return a separate consumed count (nb_done) instead of breaking out of the burst; break only when the ring is full. - Move MAX_SGL_ENTRIES_FOR_TRANSMIT clamp from patch 2 to patch 1, and clamp dev_info nb_seg_max and nb_mtu_seg_max. drivers/net/mana/mana.c | 4 ++-- drivers/net/mana/tx.c | 18 ++++++++++++++---- 2 files changed, 16 insertions(+), 6 deletions(-) diff --git a/drivers/net/mana/mana.c b/drivers/net/mana/mana.c index 30b0ce90d6..5ddbbdc3d0 100644 --- a/drivers/net/mana/mana.c +++ b/drivers/net/mana/mana.c @@ -410,9 +410,9 @@ mana_dev_info_get(struct rte_eth_dev *dev, dev_info->tx_desc_lim.nb_max = RTE_MIN(priv->max_tx_desc, UINT16_MAX); dev_info->tx_desc_lim.nb_align = MIN_BUFFERS_PER_QUEUE; dev_info->tx_desc_lim.nb_seg_max = - RTE_MIN(priv->max_send_sge, UINT16_MAX); + RTE_MIN(priv->max_send_sge, MAX_SGL_ENTRIES_FOR_TRANSMIT); dev_info->tx_desc_lim.nb_mtu_seg_max = - RTE_MIN(priv->max_send_sge, UINT16_MAX); + RTE_MIN(priv->max_send_sge, MAX_SGL_ENTRIES_FOR_TRANSMIT); /* Speed */ dev_info->speed_capa = RTE_ETH_LINK_SPEED_100G; diff --git a/drivers/net/mana/tx.c b/drivers/net/mana/tx.c index 10f2212b5d..16e63e89c9 100644 --- a/drivers/net/mana/tx.c +++ b/drivers/net/mana/tx.c @@ -192,6 +192,7 @@ mana_tx_burst(void *dpdk_txq, struct rte_mbuf **tx_pkts, uint16_t nb_pkts) int ret; void *db_page; uint16_t pkt_sent = 0; + uint16_t nb_done = 0; uint32_t num_comp, i; uint32_t expected = 0; #ifdef RTE_ARCH_32 @@ -278,10 +279,11 @@ mana_tx_burst(void *dpdk_txq, struct rte_mbuf **tx_pkts, uint16_t nb_pkts) break; /* Drop the packet if it exceeds max segments */ - if (m_pkt->nb_segs > priv->max_send_sge) { + if (unlikely(m_pkt->nb_segs > priv->max_send_sge || + m_pkt->nb_segs > MAX_SGL_ENTRIES_FOR_TRANSMIT)) { DP_LOG(ERR, "send packet segments %d exceeding max", m_pkt->nb_segs); - continue; + goto drop; } /* Fill in the oob */ @@ -432,7 +434,7 @@ mana_tx_burst(void *dpdk_txq, struct rte_mbuf **tx_pkts, uint16_t nb_pkts) /* Skip this packet if we can't populate all segments */ if (seg_idx != m_pkt->nb_segs) - continue; + goto drop; /* If we can at least queue post two WQEs and there are at * least two packets to send, use TX CQE suppression for the @@ -477,6 +479,7 @@ mana_tx_burst(void *dpdk_txq, struct rte_mbuf **tx_pkts, uint16_t nb_pkts) txq->desc_ring_len++; pkt_sent++; + nb_done++; DP_LOG(DEBUG, "nb_pkts %u pkt[%d] sent", nb_pkts, pkt_idx); @@ -499,6 +502,13 @@ mana_tx_burst(void *dpdk_txq, struct rte_mbuf **tx_pkts, uint16_t nb_pkts) pkt_idx, ret); break; } + + continue; + +drop: + txq->stats.errors++; + rte_pktmbuf_free(m_pkt); + nb_done++; } /* Ring hardware door bell */ @@ -523,5 +533,5 @@ mana_tx_burst(void *dpdk_txq, struct rte_mbuf **tx_pkts, uint16_t nb_pkts) rte_atomic_fetch_and_explicit(&txq->burst_state, ~(uint32_t)1, rte_memory_order_release); - return pkt_sent; + return nb_done; } -- 2.43.0 ^ permalink raw reply related [flat|nested] 13+ messages in thread
* [PATCH v3 2/2] net/mana: add hardware TSO support 2026-10-27 18:21 ` [PATCH v3 " Tathagat Priyadarshi 2026-10-27 18:21 ` [PATCH v3 1/2] net/mana: fix transmit burst error handling Tathagat Priyadarshi @ 2026-10-27 18:21 ` Tathagat Priyadarshi 2026-10-28 14:00 ` [PATCH v3 0/2] net/mana: fix tx burst error handling and add " Stephen Hemminger 2 siblings, 0 replies; 13+ messages in thread From: Tathagat Priyadarshi @ 2026-10-27 18:21 UTC (permalink / raw) To: dev; +Cc: weh, longli, stephen Advertise RTE_ETH_TX_OFFLOAD_TCP_TSO for IPv4 and IPv6 TCP packets. For a packet with RTE_MBUF_F_TX_TCP_SEG and a non-zero tso_segsz, put the L2/L3/L4 headers alone in the first SGE and the payload in the following ones, pass tso_segsz as the client data unit, and request IP and TCP checksum offload with the pseudo-header checksum seeded. A first segment that also carries payload needs one extra SGE, so advertise nb_seg_max as one less than the maximum supported SGE count. A packet that cannot be sent as TSO (invalid segment size, header size overflow, fragmented headers, or SGL build failure) is dropped. A packet with TCP_SEG but no payload or zero tso_segsz is sent unsegmented, with the TCP checksum offloaded as TCP_SEG implies. Signed-off-by: Tathagat Priyadarshi <tathagat.dpdk@gmail.com> --- v3: - Mask RTE_MBUF_F_TX_TCP_SEG in the non-TSO fallback branch so rte_ipv4_phdr_cksum / rte_ipv6_phdr_cksum calculate the full L4 length. - Reject invalid TSO packets (tso_segsz > MANA_TSO_MAX_SEGSZ or gso_hs > UINT8_MAX) to prevent hardware bitfield truncations. - Set dev_info nb_seg_max to max_sge - 1 to reflect the extra SGE for TSO. - Bound TSO SGL construction loop by expected_sge and verify SGE count. - Streamline commit log. v2: - Drop unsendable packets instead of break. - Honour TCP checksum offload when TCP_SEG packet is unsegmented. - Move GDMA_WR_OOB_IN_SGL and GDMA_WR_PAD_BY_SGE0 adjacent to struct gdma_work_request. - Add release notes entry and update mana.ini feature matrix. doc/guides/nics/features/mana.ini | 1 + doc/guides/rel_notes/release_26_11.rst | 4 + drivers/net/mana/gdma.c | 5 + drivers/net/mana/mana.c | 9 +- drivers/net/mana/mana.h | 9 +- drivers/net/mana/tx.c | 194 ++++++++++++++++++++----- 6 files changed, 184 insertions(+), 38 deletions(-) diff --git a/doc/guides/nics/features/mana.ini b/doc/guides/nics/features/mana.ini index c7c8215aab..6a4fb0a89e 100644 --- a/doc/guides/nics/features/mana.ini +++ b/doc/guides/nics/features/mana.ini @@ -10,6 +10,7 @@ Rx interrupt = Y RSS hash = Y L3 checksum offload = Y L4 checksum offload = Y +TSO = Y MTU update = Y Multiprocess aware = Y Linux = Y diff --git a/doc/guides/rel_notes/release_26_11.rst b/doc/guides/rel_notes/release_26_11.rst index be758a0c02..7e5f463f88 100644 --- a/doc/guides/rel_notes/release_26_11.rst +++ b/doc/guides/rel_notes/release_26_11.rst @@ -164,6 +164,10 @@ New Features Added ``fdir_buffer_size`` devarg to select the Flow Director table size (``64k``, ``128k`` or ``256k``) at probe time. The default remains ``64k``. +* **Updated Microsoft MANA PMD.** + + * Added hardware TCP Segmentation Offload (TSO) support for IPv4 and IPv6. + * **Updated NVIDIA mlx5 ethernet driver.** * Increased the maximum number of secondary unicast MAC addresses from 128 to up to 4096 diff --git a/drivers/net/mana/gdma.c b/drivers/net/mana/gdma.c index a92aef59d2..65896b6a69 100644 --- a/drivers/net/mana/gdma.c +++ b/drivers/net/mana/gdma.c @@ -38,6 +38,11 @@ write_dma_client_oob(uint8_t *work_queue_buffer_pointer, header->inline_client_oob_size_in_dwords = client_oob_size / sizeof(uint32_t); header->client_data_unit = work_request->client_data_unit; + if (work_request->flags & GDMA_WR_OOB_IN_SGL) { + header->client_oob_in_sgl = 1; + if (work_request->flags & GDMA_WR_PAD_BY_SGE0) + header->last_v_bytes = work_request->sgl[0].size; + } DP_LOG(DEBUG, "queue buf %p sgl %u oob_h %u du %u oob_buf %p oob_b %u", work_queue_buffer_pointer, header->num_sgl_entries, diff --git a/drivers/net/mana/mana.c b/drivers/net/mana/mana.c index 5ddbbdc3d0..d6a8b2dd04 100644 --- a/drivers/net/mana/mana.c +++ b/drivers/net/mana/mana.c @@ -349,6 +349,7 @@ mana_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info) { struct mana_priv *priv = dev->data->dev_private; + uint16_t max_sge; dev_info->min_mtu = RTE_ETHER_MIN_MTU; dev_info->max_mtu = MANA_MAX_MTU; @@ -406,13 +407,13 @@ mana_dev_info_get(struct rte_eth_dev *dev, dev_info->rx_desc_lim.nb_mtu_seg_max = RTE_MIN(priv->max_recv_sge, UINT16_MAX); + max_sge = RTE_MIN(priv->max_send_sge, MAX_SGL_ENTRIES_FOR_TRANSMIT); + dev_info->tx_desc_lim.nb_min = MIN_BUFFERS_PER_QUEUE; dev_info->tx_desc_lim.nb_max = RTE_MIN(priv->max_tx_desc, UINT16_MAX); dev_info->tx_desc_lim.nb_align = MIN_BUFFERS_PER_QUEUE; - dev_info->tx_desc_lim.nb_seg_max = - RTE_MIN(priv->max_send_sge, MAX_SGL_ENTRIES_FOR_TRANSMIT); - dev_info->tx_desc_lim.nb_mtu_seg_max = - RTE_MIN(priv->max_send_sge, MAX_SGL_ENTRIES_FOR_TRANSMIT); + dev_info->tx_desc_lim.nb_seg_max = max_sge > 1 ? max_sge - 1 : 1; + dev_info->tx_desc_lim.nb_mtu_seg_max = max_sge; /* Speed */ dev_info->speed_capa = RTE_ETH_LINK_SPEED_100G; diff --git a/drivers/net/mana/mana.h b/drivers/net/mana/mana.h index 79879b7626..a259a12f66 100644 --- a/drivers/net/mana/mana.h +++ b/drivers/net/mana/mana.h @@ -33,7 +33,8 @@ struct mana_shared_data { RTE_ETH_TX_OFFLOAD_MULTI_SEGS | \ RTE_ETH_TX_OFFLOAD_IPV4_CKSUM | \ RTE_ETH_TX_OFFLOAD_TCP_CKSUM | \ - RTE_ETH_TX_OFFLOAD_UDP_CKSUM) + RTE_ETH_TX_OFFLOAD_UDP_CKSUM | \ + RTE_ETH_TX_OFFLOAD_TCP_TSO) #define INDIRECTION_TABLE_NUM_ELEMENTS 64 #define TOEPLITZ_HASH_KEY_SIZE_IN_BYTES 40 @@ -208,6 +209,12 @@ struct one_sgl { struct gdma_sgl_element gdma_sgl[MAX_SGL_ENTRIES_FOR_TRANSMIT]; }; +#define GDMA_WR_OOB_IN_SGL (1 << 0) +#define GDMA_WR_PAD_BY_SGE0 (1 << 1) + +/* client_data_unit is 14 bits in gdma_wqe_dma_oob */ +#define MANA_TSO_MAX_SEGSZ ((1u << 14) - 1) + struct gdma_work_request { struct gdma_header gdma_header; struct gdma_sgl_element *sgl; diff --git a/drivers/net/mana/tx.c b/drivers/net/mana/tx.c index 16e63e89c9..19a3aaec59 100644 --- a/drivers/net/mana/tx.c +++ b/drivers/net/mana/tx.c @@ -274,15 +274,43 @@ mana_tx_burst(void *dpdk_txq, struct rte_mbuf **tx_pkts, uint16_t nb_pkts) struct transmit_oob_v2 tx_oob; struct one_sgl sgl; uint16_t seg_idx; + uint16_t num_sge; + uint16_t expected_sge; + bool is_tso = false; + uint32_t gso_hs = 0; if (txq->desc_ring_len >= txq->num_desc) break; - /* Drop the packet if it exceeds max segments */ - if (unlikely(m_pkt->nb_segs > priv->max_send_sge || - m_pkt->nb_segs > MAX_SGL_ENTRIES_FOR_TRANSMIT)) { + if (m_pkt->ol_flags & RTE_MBUF_F_TX_TCP_SEG) { + gso_hs = m_pkt->l2_len + m_pkt->l3_len + m_pkt->l4_len; + /* Header must be in first segment; last_v_bytes is 8 bits */ + if (unlikely(m_pkt->data_len < gso_hs || + gso_hs > UINT8_MAX)) { + DP_LOG(ERR, "invalid TSO packet header length %u", + gso_hs); + goto drop; + } + if (unlikely(m_pkt->tso_segsz > MANA_TSO_MAX_SEGSZ)) { + DP_LOG(ERR, "TSO segment size %u exceeds max %u", + m_pkt->tso_segsz, MANA_TSO_MAX_SEGSZ); + goto drop; + } + if (likely(gso_hs != 0 && m_pkt->pkt_len > gso_hs && + m_pkt->tso_segsz != 0)) + is_tso = true; + } + + /* Drop the packet if it exceeds max segments. A TSO packet + * with payload in the first segment needs an extra SGE for + * the header. + */ + expected_sge = m_pkt->nb_segs + + (is_tso && m_pkt->data_len > gso_hs ? 1 : 0); + if (unlikely(expected_sge > priv->max_send_sge || + expected_sge > MAX_SGL_ENTRIES_FOR_TRANSMIT)) { DP_LOG(ERR, "send packet segments %d exceeding max", - m_pkt->nb_segs); + expected_sge); goto drop; } @@ -307,9 +335,47 @@ mana_tx_burst(void *dpdk_txq, struct rte_mbuf **tx_pkts, uint16_t nb_pkts) tx_oob.short_oob.tx_compute_IP_header_checksum = m_pkt->ol_flags & RTE_MBUF_F_TX_IP_CKSUM ? 1 : 0; - if ((m_pkt->ol_flags & RTE_MBUF_F_TX_L4_MASK) == - RTE_MBUF_F_TX_TCP_CKSUM) { + if (is_tso) { + struct rte_tcp_hdr *tcp_hdr = + rte_pktmbuf_mtod_offset(m_pkt, + struct rte_tcp_hdr *, + m_pkt->l2_len + m_pkt->l3_len); + + if (m_pkt->ol_flags & RTE_MBUF_F_TX_IPV4) { + struct rte_ipv4_hdr *ip_hdr = + rte_pktmbuf_mtod_offset(m_pkt, + struct rte_ipv4_hdr *, + m_pkt->l2_len); + + ip_hdr->total_length = 0; + ip_hdr->hdr_checksum = 0; + tcp_hdr->cksum = rte_ipv4_phdr_cksum(ip_hdr, + m_pkt->ol_flags); + } else if (m_pkt->ol_flags & RTE_MBUF_F_TX_IPV6) { + struct rte_ipv6_hdr *ip_hdr = + rte_pktmbuf_mtod_offset(m_pkt, + struct rte_ipv6_hdr *, + m_pkt->l2_len); + + ip_hdr->payload_len = 0; + tcp_hdr->cksum = rte_ipv6_phdr_cksum(ip_hdr, + m_pkt->ol_flags); + } else { + DP_LOG(ERR, "Invalid IP version for TSO"); + goto drop; + } + + tx_oob.short_oob.tx_compute_IP_header_checksum = + (m_pkt->ol_flags & RTE_MBUF_F_TX_IPV4) ? 1 : 0; + tx_oob.short_oob.tx_compute_TCP_checksum = 1; + tx_oob.short_oob.tx_compute_UDP_checksum = 0; + tx_oob.short_oob.tx_transport_header_offset = + m_pkt->l2_len + m_pkt->l3_len; + } else if ((m_pkt->ol_flags & RTE_MBUF_F_TX_L4_MASK) == + RTE_MBUF_F_TX_TCP_CKSUM || + (m_pkt->ol_flags & RTE_MBUF_F_TX_TCP_SEG)) { struct rte_tcp_hdr *tcp_hdr; + uint64_t ol = m_pkt->ol_flags & ~RTE_MBUF_F_TX_TCP_SEG; /* HW needs partial TCP checksum */ @@ -323,8 +389,7 @@ mana_tx_burst(void *dpdk_txq, struct rte_mbuf **tx_pkts, uint16_t nb_pkts) ip_hdr = rte_pktmbuf_mtod_offset(m_pkt, struct rte_ipv4_hdr *, m_pkt->l2_len); - tcp_hdr->cksum = rte_ipv4_phdr_cksum(ip_hdr, - m_pkt->ol_flags); + tcp_hdr->cksum = rte_ipv4_phdr_cksum(ip_hdr, ol); } else if (m_pkt->ol_flags & RTE_MBUF_F_TX_IPV6) { struct rte_ipv6_hdr *ip_hdr; @@ -332,8 +397,7 @@ mana_tx_burst(void *dpdk_txq, struct rte_mbuf **tx_pkts, uint16_t nb_pkts) ip_hdr = rte_pktmbuf_mtod_offset(m_pkt, struct rte_ipv6_hdr *, m_pkt->l2_len); - tcp_hdr->cksum = rte_ipv6_phdr_cksum(ip_hdr, - m_pkt->ol_flags); + tcp_hdr->cksum = rte_ipv6_phdr_cksum(ip_hdr, ol); } else { DP_LOG(ERR, "Invalid input for TCP CKSUM"); } @@ -407,34 +471,92 @@ mana_tx_burst(void *dpdk_txq, struct rte_mbuf **tx_pkts, uint16_t nb_pkts) m_pkt->pkt_len); /* Create SGL for packet data buffers */ - for (seg_idx = 0; seg_idx < m_pkt->nb_segs; seg_idx++) { + if (is_tso) { struct mana_mr_cache *mr = - mana_alloc_pmd_mr(&txq->mr_btree, priv, m_seg); + mana_alloc_pmd_mr(&txq->mr_btree, priv, m_pkt); - if (!mr) { - DP_LOG(ERR, "failed to get MR, pkt_idx %u", + if (unlikely(!mr)) { + DP_LOG(ERR, "failed to get MR for TSO hdr, pkt_idx %u", pkt_idx); - break; + goto drop; } - sgl.gdma_sgl[seg_idx].address = - rte_cpu_to_le_64(rte_pktmbuf_mtod(m_seg, - uint64_t)); - sgl.gdma_sgl[seg_idx].size = m_seg->data_len; - sgl.gdma_sgl[seg_idx].memory_key = mr->lkey; + /* SGE0: packet header only (L2 + L3 + L4) */ + sgl.gdma_sgl[0].address = + rte_cpu_to_le_64(rte_pktmbuf_mtod(m_pkt, uint64_t)); + sgl.gdma_sgl[0].size = gso_hs; + sgl.gdma_sgl[0].memory_key = mr->lkey; + num_sge = 1; + + /* If first mbuf has payload data beyond the header */ + if (m_pkt->data_len > gso_hs) { + sgl.gdma_sgl[1].address = + rte_cpu_to_le_64(rte_pktmbuf_mtod(m_pkt, + uint64_t) + + gso_hs); + sgl.gdma_sgl[1].size = m_pkt->data_len - gso_hs; + sgl.gdma_sgl[1].memory_key = mr->lkey; + num_sge = 2; + } - DP_LOG(DEBUG, - "seg idx %u addr 0x%" PRIx64 " size %x key %x", - seg_idx, sgl.gdma_sgl[seg_idx].address, - sgl.gdma_sgl[seg_idx].size, - sgl.gdma_sgl[seg_idx].memory_key); + /* Subsequent mbuf segments */ + m_seg = m_pkt->next; + while (m_seg && num_sge < expected_sge) { + mr = mana_alloc_pmd_mr(&txq->mr_btree, priv, + m_seg); + if (unlikely(!mr)) { + DP_LOG(ERR, + "failed to get MR for TSO seg, pkt_idx %u", + pkt_idx); + goto drop; + } + sgl.gdma_sgl[num_sge].address = + rte_cpu_to_le_64(rte_pktmbuf_mtod(m_seg, + uint64_t)); + sgl.gdma_sgl[num_sge].size = m_seg->data_len; + sgl.gdma_sgl[num_sge].memory_key = mr->lkey; + num_sge++; + m_seg = m_seg->next; + } - m_seg = m_seg->next; - } + if (unlikely(m_seg != NULL || num_sge != expected_sge)) { + DP_LOG(ERR, "failed to build TSO SGL, num_sge %u", + num_sge); + goto drop; + } + } else { + for (seg_idx = 0; seg_idx < m_pkt->nb_segs; seg_idx++) { + struct mana_mr_cache *mr = + mana_alloc_pmd_mr(&txq->mr_btree, priv, + m_seg); + + if (!mr) { + DP_LOG(ERR, "failed to get MR, pkt_idx %u", + pkt_idx); + break; + } + + sgl.gdma_sgl[seg_idx].address = + rte_cpu_to_le_64(rte_pktmbuf_mtod(m_seg, + uint64_t)); + sgl.gdma_sgl[seg_idx].size = m_seg->data_len; + sgl.gdma_sgl[seg_idx].memory_key = mr->lkey; + + DP_LOG(DEBUG, + "seg idx %u addr 0x%" PRIx64 " size %x key %x", + seg_idx, sgl.gdma_sgl[seg_idx].address, + sgl.gdma_sgl[seg_idx].size, + sgl.gdma_sgl[seg_idx].memory_key); + + m_seg = m_seg->next; + } - /* Skip this packet if we can't populate all segments */ - if (seg_idx != m_pkt->nb_segs) - goto drop; + /* Skip this packet if we can't populate all segments */ + if (seg_idx != m_pkt->nb_segs) + goto drop; + + num_sge = m_pkt->nb_segs; + } /* If we can at least queue post two WQEs and there are at * least two packets to send, use TX CQE suppression for the @@ -452,7 +574,7 @@ mana_tx_burst(void *dpdk_txq, struct rte_mbuf **tx_pkts, uint16_t nb_pkts) work_req.gdma_header.struct_size = sizeof(work_req); work_req.sgl = sgl.gdma_sgl; - work_req.num_sgl_elements = m_pkt->nb_segs; + work_req.num_sgl_elements = num_sge; if (tx_oob.short_oob.packet_format == SHORT_PACKET_FORMAT) work_req.inline_oob_size_in_bytes = sizeof(struct transmit_short_oob_v2); @@ -460,8 +582,14 @@ mana_tx_burst(void *dpdk_txq, struct rte_mbuf **tx_pkts, uint16_t nb_pkts) work_req.inline_oob_size_in_bytes = sizeof(struct transmit_oob_v2); work_req.inline_oob_data = &tx_oob; - work_req.flags = 0; - work_req.client_data_unit = NOT_USING_CLIENT_DATA_UNIT; + if (is_tso) { + work_req.flags = GDMA_WR_OOB_IN_SGL | + GDMA_WR_PAD_BY_SGE0; + work_req.client_data_unit = m_pkt->tso_segsz; + } else { + work_req.flags = 0; + work_req.client_data_unit = NOT_USING_CLIENT_DATA_UNIT; + } ret = gdma_post_work_request(&txq->gdma_sq, &work_req, &wqe_size_in_bu); -- 2.43.0 ^ permalink raw reply related [flat|nested] 13+ messages in thread
* Re: [PATCH v3 0/2] net/mana: fix tx burst error handling and add TSO support 2026-10-27 18:21 ` [PATCH v3 " Tathagat Priyadarshi 2026-10-27 18:21 ` [PATCH v3 1/2] net/mana: fix transmit burst error handling Tathagat Priyadarshi 2026-10-27 18:21 ` [PATCH v3 2/2] net/mana: add hardware TSO support Tathagat Priyadarshi @ 2026-10-28 14:00 ` Stephen Hemminger 2026-10-28 19:37 ` Tathagat Priyadarshi 2 siblings, 1 reply; 13+ messages in thread From: Stephen Hemminger @ 2026-10-28 14:00 UTC (permalink / raw) To: Tathagat Priyadarshi; +Cc: dev, weh, longli On Tue, 27 Oct 2026 18:21:13 +0000 Tathagat Priyadarshi <tathagat.dpdk@gmail.com> wrote: > This series fixes transmit burst error handling and adds hardware TCP > segmentation offload (TSO) for the Microsoft Azure MANA PMD. > > Patch 1 drops packets that fail validation, counting them in > stats.errors and consuming them, so rte_eth_tx_burst() maintains a > contiguous prefix and short returns reflect only ring backpressure. > It also clamps segment limits to MAX_SGL_ENTRIES_FOR_TRANSMIT. > > Patch 2 implements hardware TSO for IPv4 and IPv6 TCP traffic, > formatting SGE0 for headers and subsequent SGEs for payload, handling > GDMA work request flags, advertising RTE_ETH_TX_OFFLOAD_TCP_TSO, and > handling non-TSO fallback checksums. > > Hardware TSO segmentation was verified on Azure Standard_D8ds_v6 (kernel > 6.8.0-1067-azure, MANA PCI 7870:00:00.0; driver reports no firmware > version) between two VMs on one subnet using an earlier revision of > this series: > - VM A ran testpmd txonly, modified locally to emit TCP packets with > RTE_MBUF_F_TX_TCP_SEG set (not part of this series), after > "csum set ip hw 0", "csum set tcp hw 0" and "tso set 1460 0". > - VM B captured the frames with tcpdump on its kernel interface, GRO off. > - A 4000-byte IPv4 packet in a single mbuf, in mbufs with a header-only > first segment, and in mbufs with payload in the first segment, each > arrived as 1514, 1514 and 1080 byte frames with consecutive TCP > sequence numbers, per-segment IP IDs and valid IP and TCP checksums. > - Plain UDP with "tso set 0" was unaffected. > - The drop paths (packets exceeding segment limit, oversized MSS or > header) and IPv6 TSO were not exercised on hardware. > > The v3 changes (hardware field bounds for MSS and header length, > loop bound, unsegmented fallback checksum masking, and dev_info limit > advertisement) were compile-tested but have not been re-run on hardware. > > v3: > - Patch 1: no changes. > - Patch 2: > * Mask RTE_MBUF_F_TX_TCP_SEG in the non-TSO fallback branch so > rte_ipv4_phdr_cksum / rte_ipv6_phdr_cksum calculate the full L4 > length into the pseudo-header checksum. > * Reject invalid TSO packets (tso_segsz > MANA_TSO_MAX_SEGSZ or > gso_hs > UINT8_MAX) to prevent hardware bitfield truncations. > * Reflect the extra SGE for TSO in dev_info by setting nb_seg_max to > max_sge - 1 (keeping nb_mtu_seg_max as max_sge) instead of a mana.rst > limitation note. > * Bound the TSO segment walk by expected_sge and verify SGE count. > * Streamline commit message and cover letter. > > v2: > - Patch 1: > * Drop unsendable packets and return a separate consumed count (nb_done) > instead of breaking out of the burst; break only when the ring is full. > * Move MAX_SGL_ENTRIES_FOR_TRANSMIT clamp from patch 2 to patch 1, and > clamp dev_info nb_seg_max and nb_mtu_seg_max. > - Patch 2: > * Use the drop path for all per-packet TSO validation failures. > * Honour implied TCP checksum when RTE_MBUF_F_TX_TCP_SEG is set but the > packet is unsegmented. > * Move GDMA_WR_OOB_IN_SGL and GDMA_WR_PAD_BY_SGE0 adjacent to > struct gdma_work_request. > * Add release notes entry and update mana.ini feature matrix. > > Tathagat Priyadarshi (2): > net/mana: fix transmit burst error handling > net/mana: add hardware TSO support > > doc/guides/nics/features/mana.ini | 1 + > doc/guides/rel_notes/release_26_11.rst | 4 + > drivers/net/mana/gdma.c | 5 + > drivers/net/mana/mana.c | 9 +- > drivers/net/mana/mana.h | 9 +- > drivers/net/mana/tx.c | 206 +++++++++++++++++++++---- > 6 files changed, 195 insertions(+), 39 deletions(-) > I can apply it as is. But you may want to look at the length issues reported in AI review Subject: Re: [PATCH v3 0/2] net/mana: tx burst fixes and TSO Applied to main (e3e29ed), net/mana builds clean with -Dwerror=true at each commit. Fixes: hashes verified. Patch 1 drops invalid packets and counts them in the returned value, which is the right tx_burst contract. The SGL bound fix is correct. No findings on patch 1. Patch 2 follows the Linux mana TSO path (header in SGE0, OOB_IN_SGL | PAD_BY_SGE0, tot_len/payload_len zeroed, pseudo-header seeded without length). Field widths match: last_v_bytes is 8 bits, client_data_unit is 14 bits. [PATCH v3 2/2] net/mana: add hardware TSO support Info: l4_len is not validated. tcp_hdr->cksum is written at l2_len + l3_len + 16, so with l4_len < 20 the write can land past data_len, and SGE0 would split the TCP header. Cheap to reject: if (unlikely(m_pkt->l4_len < sizeof(struct rte_tcp_hdr) || m_pkt->data_len < gso_hs || gso_hs > UINT8_MAX)) { Info: total TSO length is not bounded. pkt_len can exceed 64K with a long chain. If the hardware has a limit on TSO payload, check pkt_len - gso_hs against it and drop. Info: the header fixups (IPv4 total_length/hdr_checksum, IPv6 payload_len, TCP cksum) write into packet data from tx_burst. An application holding a reference to the mbuf (e.g. for retransmit) sees the modified headers. tx_pkt_prepare is the ethdev hook for this; the existing checksum path already does the same thing, so not a blocker for this series. Reviewed-by: Stephen Hemminger <stephen@networkplumber.org> ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH v3 0/2] net/mana: fix tx burst error handling and add TSO support 2026-10-28 14:00 ` [PATCH v3 0/2] net/mana: fix tx burst error handling and add " Stephen Hemminger @ 2026-10-28 19:37 ` Tathagat Priyadarshi 0 siblings, 0 replies; 13+ messages in thread From: Tathagat Priyadarshi @ 2026-10-28 19:37 UTC (permalink / raw) To: Stephen Hemminger; +Cc: dev, weh, longli On Wed, Oct 28, 2026 at 7:30 PM Stephen Hemminger <stephen@networkplumber.org> wrote: > > On Tue, 27 Oct 2026 18:21:13 +0000 > Tathagat Priyadarshi <tathagat.dpdk@gmail.com> wrote: > > > This series fixes transmit burst error handling and adds hardware TCP > > segmentation offload (TSO) for the Microsoft Azure MANA PMD. > > > > Patch 1 drops packets that fail validation, counting them in > > stats.errors and consuming them, so rte_eth_tx_burst() maintains a > > contiguous prefix and short returns reflect only ring backpressure. > > It also clamps segment limits to MAX_SGL_ENTRIES_FOR_TRANSMIT. > > > > Patch 2 implements hardware TSO for IPv4 and IPv6 TCP traffic, > > formatting SGE0 for headers and subsequent SGEs for payload, handling > > GDMA work request flags, advertising RTE_ETH_TX_OFFLOAD_TCP_TSO, and > > handling non-TSO fallback checksums. > > > > Hardware TSO segmentation was verified on Azure Standard_D8ds_v6 (kernel > > 6.8.0-1067-azure, MANA PCI 7870:00:00.0; driver reports no firmware > > version) between two VMs on one subnet using an earlier revision of > > this series: > > - VM A ran testpmd txonly, modified locally to emit TCP packets with > > RTE_MBUF_F_TX_TCP_SEG set (not part of this series), after > > "csum set ip hw 0", "csum set tcp hw 0" and "tso set 1460 0". > > - VM B captured the frames with tcpdump on its kernel interface, GRO off. > > - A 4000-byte IPv4 packet in a single mbuf, in mbufs with a header-only > > first segment, and in mbufs with payload in the first segment, each > > arrived as 1514, 1514 and 1080 byte frames with consecutive TCP > > sequence numbers, per-segment IP IDs and valid IP and TCP checksums. > > - Plain UDP with "tso set 0" was unaffected. > > - The drop paths (packets exceeding segment limit, oversized MSS or > > header) and IPv6 TSO were not exercised on hardware. > > > > The v3 changes (hardware field bounds for MSS and header length, > > loop bound, unsegmented fallback checksum masking, and dev_info limit > > advertisement) were compile-tested but have not been re-run on hardware. > > > > v3: > > - Patch 1: no changes. > > - Patch 2: > > * Mask RTE_MBUF_F_TX_TCP_SEG in the non-TSO fallback branch so > > rte_ipv4_phdr_cksum / rte_ipv6_phdr_cksum calculate the full L4 > > length into the pseudo-header checksum. > > * Reject invalid TSO packets (tso_segsz > MANA_TSO_MAX_SEGSZ or > > gso_hs > UINT8_MAX) to prevent hardware bitfield truncations. > > * Reflect the extra SGE for TSO in dev_info by setting nb_seg_max to > > max_sge - 1 (keeping nb_mtu_seg_max as max_sge) instead of a mana.rst > > limitation note. > > * Bound the TSO segment walk by expected_sge and verify SGE count. > > * Streamline commit message and cover letter. > > > > v2: > > - Patch 1: > > * Drop unsendable packets and return a separate consumed count (nb_done) > > instead of breaking out of the burst; break only when the ring is full. > > * Move MAX_SGL_ENTRIES_FOR_TRANSMIT clamp from patch 2 to patch 1, and > > clamp dev_info nb_seg_max and nb_mtu_seg_max. > > - Patch 2: > > * Use the drop path for all per-packet TSO validation failures. > > * Honour implied TCP checksum when RTE_MBUF_F_TX_TCP_SEG is set but the > > packet is unsegmented. > > * Move GDMA_WR_OOB_IN_SGL and GDMA_WR_PAD_BY_SGE0 adjacent to > > struct gdma_work_request. > > * Add release notes entry and update mana.ini feature matrix. > > > > Tathagat Priyadarshi (2): > > net/mana: fix transmit burst error handling > > net/mana: add hardware TSO support > > > > doc/guides/nics/features/mana.ini | 1 + > > doc/guides/rel_notes/release_26_11.rst | 4 + > > drivers/net/mana/gdma.c | 5 + > > drivers/net/mana/mana.c | 9 +- > > drivers/net/mana/mana.h | 9 +- > > drivers/net/mana/tx.c | 206 +++++++++++++++++++++---- > > 6 files changed, 195 insertions(+), 39 deletions(-) > > > > I can apply it as is. But you may want to look at the length issues > reported in AI review > > Subject: Re: [PATCH v3 0/2] net/mana: tx burst fixes and TSO > > Applied to main (e3e29ed), net/mana builds clean with -Dwerror=true > at each commit. Fixes: hashes verified. > > Patch 1 drops invalid packets and counts them in the returned value, > which is the right tx_burst contract. The SGL bound fix is correct. > No findings on patch 1. > > Patch 2 follows the Linux mana TSO path (header in SGE0, > OOB_IN_SGL | PAD_BY_SGE0, tot_len/payload_len zeroed, pseudo-header > seeded without length). Field widths match: last_v_bytes is 8 bits, > client_data_unit is 14 bits. > > [PATCH v3 2/2] net/mana: add hardware TSO support > > Info: l4_len is not validated. tcp_hdr->cksum is written at > l2_len + l3_len + 16, so with l4_len < 20 the write can land past > data_len, and SGE0 would split the TCP header. Cheap to reject: > > if (unlikely(m_pkt->l4_len < sizeof(struct rte_tcp_hdr) || > m_pkt->data_len < gso_hs || > gso_hs > UINT8_MAX)) { > > Info: total TSO length is not bounded. pkt_len can exceed 64K with > a long chain. If the hardware has a limit on TSO payload, check > pkt_len - gso_hs against it and drop. > > Info: the header fixups (IPv4 total_length/hdr_checksum, IPv6 > payload_len, TCP cksum) write into packet data from tx_burst. An > application holding a reference to the mbuf (e.g. for retransmit) > sees the modified headers. tx_pkt_prepare is the ethdev hook for > this; the existing checksum path already does the same thing, so > not a blocker for this series. > > Reviewed-by: Stephen Hemminger <stephen@networkplumber.org> Hi Stephen, Thanks for the review and build verification! Please apply v3 as is. I'll send a separate patch for the l4_len validation. The TSO payload limit isn't documented, so I left it unbounded for now and will add a check if I find the hardware limit. I will also look into adding tx_pkt_prepare for MANA. If you'd rather have the l4_len change in a v4, let me know ^ permalink raw reply [flat|nested] 13+ messages in thread
end of thread, other threads:[~2026-10-29 9:40 UTC | newest] Thread overview: 13+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2026-10-12 7:36 [PATCH 0/2] net/mana: fix transmit burst and add hardware TSO support Tathagat Priyadarshi 2026-10-12 7:36 ` [PATCH 1/2] net/mana: fix transmit burst error handling Tathagat Priyadarshi 2026-10-12 7:36 ` [PATCH 2/2] net/mana: add hardware TSO support Tathagat Priyadarshi 2026-10-12 21:37 ` [PATCH 0/2] net/mana: fix transmit burst and " Stephen Hemminger 2026-10-26 21:21 ` [PATCH v2 0/2] net/mana: fix tx burst error handling and add " Tathagat Priyadarshi 2026-10-26 21:21 ` [PATCH v2 1/2] net/mana: fix transmit burst error handling Tathagat Priyadarshi 2026-10-26 21:21 ` [PATCH v2 2/2] net/mana: add hardware TSO support Tathagat Priyadarshi 2026-10-27 14:02 ` [PATCH v2 0/2] net/mana: fix tx burst error handling and add " Stephen Hemminger 2026-10-27 18:21 ` [PATCH v3 " Tathagat Priyadarshi 2026-10-27 18:21 ` [PATCH v3 1/2] net/mana: fix transmit burst error handling Tathagat Priyadarshi 2026-10-27 18:21 ` [PATCH v3 2/2] net/mana: add hardware TSO support Tathagat Priyadarshi 2026-10-28 14:00 ` [PATCH v3 0/2] net/mana: fix tx burst error handling and add " Stephen Hemminger 2026-10-28 19:37 ` Tathagat Priyadarshi
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox