[PATCH 0/4] net/gve: fix TX queue stalls and sync DQO RDA with Linux driver

[PATCH 0/4] net/gve: fix TX queue stalls and sync DQO RDA with Linux driver

[Tathagat Priyadarshi]

This patch series aligns the Google Virtual NIC (gve) DQO RDA transmit path
with the upstream Linux kernel driver (drivers/net/ethernet/google/gve/) to
resolve transmit queue stalls and hardware validation issues:

1. net/gve: fix DQO descriptor cleaning loop bound
   Fixes the off-by-one error in gve_tx_clean_descs_dqo()
   ("while (--nb_descs)" -> "while (nb_descs--)") and avoids stalling
   the completion queue on unrecognized descriptor types.

2. net/gve: fix DQO TX queue stall on multi-seg and TSO
   Tracks descriptor intervals with last_re_idx per packet against
   GVE_TX_MIN_RE_INTERVAL (32) instead of counting packets at burst end,
   preventing descriptor ring starvation and queue deadlocks under
   multi-segment and TSO traffic.

3. net/gve: add cacheline overlap margin to DQO TX ring
   Enforces GVE_TX_MIN_DESC_PREVENT_CACHE_OVERLAP (4 descriptors) margin
   in ring space checks, avoiding CPU and device DMA cacheline collisions.

4. net/gve: add packet and TSO hardware constraints for DQO
   Augments gve_tx_validate_descs() (which enforces the 10-descriptor
   limit per packet or MTU segment) with the remaining hardware length
   constraints: [17, 9728] bytes for non-TSO packets, maximum MSS limit
   (9728 bytes) to avoid 14-bit field wrap-around, and validates header
   contiguity and payload presence on TSO packets.

Tathagat Priyadarshi (4):
  net/gve: fix DQO descriptor cleaning loop bound
  net/gve: fix DQO TX queue stall on multi-seg and TSO
  net/gve: add cacheline overlap margin to DQO TX ring
  net/gve: add packet and TSO hardware constraints for DQO

 drivers/net/gve/base/gve_desc_dqo.h |  3 ++
 drivers/net/gve/gve_ethdev.h        |  2 +-
 drivers/net/gve/gve_tx_dqo.c        | 80 +++++++++++++++++++++--------
 3 files changed, 62 insertions(+), 23 deletions(-)

-- 
2.43.0

[PATCH 1/4] net/gve: fix DQO descriptor cleaning loop bound

[Tathagat Priyadarshi]

In gve_tx_clean_descs_dqo(), the loop condition "while (--nb_descs)"
causes an off-by-one error:
- When nb_descs == 1, --nb_descs decrements to 0 on the first check,
  executing 0 iterations instead of 1.
- When nb_descs == 0, --nb_descs underflows uint16_t to 65535,
  executing 65535 iterations.

Change the condition to "while (nb_descs--)" so that exactly nb_descs
iterations are executed for any input value.

Additionally, in gve_tx_clean_dqo(), handle unrecognized completion
types by breaking from the switch rather than returning immediately,
allowing complq_tail to advance past unknown descriptors rather than
stalling the completion ring at the same entry.

Fixes: 4022f9999f56 ("net/gve: support basic Tx data path for DQO")
Cc: stable@dpdk.org

Acked-by: Joshua Washington <joshwash@google.com>
Signed-off-by: Tathagat Priyadarshi <tathagat.dpdk@gmail.com>
---
 drivers/net/gve/gve_tx_dqo.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/drivers/net/gve/gve_tx_dqo.c b/drivers/net/gve/gve_tx_dqo.c
index 5c03d3d1a2..c864d722d9 100644
--- a/drivers/net/gve/gve_tx_dqo.c
+++ b/drivers/net/gve/gve_tx_dqo.c
@@ -108,8 +108,8 @@ gve_tx_clean_dqo(struct gve_tx_queue *txq)
 		PMD_DRV_DP_LOG(DEBUG, "GVE_COMPL_TYPE_DQO_MISS ignored !!!");
 		break;
 	default:
-		PMD_DRV_DP_LOG(ERR, "unknown completion type.");
-		return;
+		PMD_DRV_DP_LOG(ERR, "unknown completion type %u", compl_desc->type);
+		break;
 	}
 
 	next++;
@@ -122,7 +122,7 @@ gve_tx_clean_dqo(struct gve_tx_queue *txq)
 
 static inline void
 gve_tx_clean_descs_dqo(struct gve_tx_queue *txq, uint16_t nb_descs) {
-	while (--nb_descs)
+	while (nb_descs--)
 		gve_tx_clean_dqo(txq);
 }
 
-- 
2.43.0

[PATCH 2/4] net/gve: fix DQO TX queue stall on multi-seg and TSO

[Tathagat Priyadarshi]

The DPDK GVE driver previously tracked transmitted packet count (re_cnt)
at the end of each burst rather than tracking descriptor intervals per
packet against GVE_TX_MIN_RE_INTERVAL (32).

In multi-segment and TSO transmissions, each packet consumes multiple
descriptors. The descriptor ring could exhaust before 32 packets were
sent, resulting in zero descriptors requesting report_event and
preventing the hardware DMA engine from returning GVE_COMPL_TYPE_DQO_DESC
completions. Without these completions, txq->nb_free was never
replenished, permanently stalling the TX queue.

Align with the Linux kernel driver (gve_tx_update_tail()) by tracking
descriptor indices with last_re_idx:
- Replace re_cnt with last_re_idx in struct gve_tx_queue.
- Reset last_re_idx = 0 in gve_reset_tx_ring_state_dqo().
- On each packet transmission, check descriptor interval
  (last_desc_idx - last_re_idx) & mask >= GVE_TX_MIN_RE_INTERVAL.
  When met, set report_event on the packet's last descriptor and
  update last_re_idx.

Fixes: 4022f9999f56 ("net/gve: support basic Tx data path for DQO")
Fixes: 403c671a46b6 ("net/gve: support TSO in DQO RDA")
Cc: stable@dpdk.org

Acked-by: Joshua Washington <joshwash@google.com>
Signed-off-by: Tathagat Priyadarshi <tathagat.dpdk@gmail.com>
---
 drivers/net/gve/gve_ethdev.h |  2 +-
 drivers/net/gve/gve_tx_dqo.c | 18 ++++++++++--------
 2 files changed, 11 insertions(+), 9 deletions(-)

diff --git a/drivers/net/gve/gve_ethdev.h b/drivers/net/gve/gve_ethdev.h
index c9a176ff17..2033f49444 100644
--- a/drivers/net/gve/gve_ethdev.h
+++ b/drivers/net/gve/gve_ethdev.h
@@ -193,8 +193,8 @@ struct gve_tx_queue {
 	uint16_t sw_size;
 	uint8_t cur_gen_bit;
 	uint32_t last_desc_cleaned;
+	uint32_t last_re_idx;
 	void **txqs;
-	uint16_t re_cnt;
 
 	/* Only valid for DQO_RDA queue format */
 	struct gve_tx_queue *complq;
diff --git a/drivers/net/gve/gve_tx_dqo.c b/drivers/net/gve/gve_tx_dqo.c
index c864d722d9..a8382ff3c4 100644
--- a/drivers/net/gve/gve_tx_dqo.c
+++ b/drivers/net/gve/gve_tx_dqo.c
@@ -264,6 +264,8 @@ gve_tx_burst_dqo(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
 	uint16_t nb_descs;
 	bool csum, tso;
 	uint16_t tx_id;
+	uint32_t last_desc_idx;
+	uint32_t last_re_interval;
 
 	pkts = txq->pkt_ring_dqo;
 	txr = txq->tx_ring;
@@ -331,20 +333,19 @@ gve_tx_burst_dqo(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
 		txd = &txr[(tx_id - 1) & mask];
 		txd->pkt.end_of_packet = 1;
 
+		last_desc_idx = (tx_id - 1) & mask;
+		last_re_interval = (last_desc_idx - txq->last_re_idx) & mask;
+		if (unlikely(last_re_interval >= GVE_TX_MIN_RE_INTERVAL)) {
+			txd->pkt.report_event = true;
+			txq->last_re_idx = last_desc_idx;
+		}
+
 		txq->nb_free -= nb_descs;
 		txq->nb_used += nb_descs;
 	}
 
 	/* update the tail pointer if any packets were processed */
 	if (nb_tx > 0) {
-		/* Request a descriptor completion on the last descriptor */
-		txq->re_cnt += nb_tx;
-		if (txq->re_cnt >= GVE_TX_MIN_RE_INTERVAL) {
-			txd = &txr[(tx_id - 1) & mask];
-			txd->pkt.report_event = true;
-			txq->re_cnt = 0;
-		}
-
 		rte_write32(tx_id, txq->qtx_tail);
 		txq->tx_tail = tx_id;
 
@@ -445,6 +446,7 @@ gve_reset_tx_ring_state_dqo(struct gve_tx_queue *txq)
 	txq->nb_used = 0;
 
 	txq->last_desc_cleaned = 0;
+	txq->last_re_idx = 0;
 	txq->nb_free = txq->nb_tx_desc - 1;
 
 	txq->complq_tail = 0;
-- 
2.43.0

Re: [PATCH 2/4] net/gve: fix DQO TX queue stall on multi-seg and TSO

[Joshua Washington]

On Mon, Oct 12, 2026 at 3:50 AM Tathagat Priyadarshi
<tathagat.dpdk@gmail.com> wrote:
>
> The DPDK GVE driver previously tracked transmitted packet count (re_cnt)
> at the end of each burst rather than tracking descriptor intervals per
> packet against GVE_TX_MIN_RE_INTERVAL (32).
>
> In multi-segment and TSO transmissions, each packet consumes multiple
> descriptors. The descriptor ring could exhaust before 32 packets were
> sent, resulting in zero descriptors requesting report_event and
> preventing the hardware DMA engine from returning GVE_COMPL_TYPE_DQO_DESC
> completions. Without these completions, txq->nb_free was never
> replenished, permanently stalling the TX queue.
>
> Align with the Linux kernel driver (gve_tx_update_tail()) by tracking
> descriptor indices with last_re_idx:
> - Replace re_cnt with last_re_idx in struct gve_tx_queue.
> - Reset last_re_idx = 0 in gve_reset_tx_ring_state_dqo().
> - On each packet transmission, check descriptor interval
>   (last_desc_idx - last_re_idx) & mask >= GVE_TX_MIN_RE_INTERVAL.
>   When met, set report_event on the packet's last descriptor and
>   update last_re_idx.
>
> Fixes: 4022f9999f56 ("net/gve: support basic Tx data path for DQO")
> Fixes: 403c671a46b6 ("net/gve: support TSO in DQO RDA")
> Cc: stable@dpdk.org
>
> Acked-by: Joshua Washington <joshwash@google.com>
> Signed-off-by: Tathagat Priyadarshi <tathagat.dpdk@gmail.com>
> ---
>  drivers/net/gve/gve_ethdev.h |  2 +-
>  drivers/net/gve/gve_tx_dqo.c | 18 ++++++++++--------
>  2 files changed, 11 insertions(+), 9 deletions(-)
>
> diff --git a/drivers/net/gve/gve_ethdev.h b/drivers/net/gve/gve_ethdev.h
> index c9a176ff17..2033f49444 100644
> --- a/drivers/net/gve/gve_ethdev.h
> +++ b/drivers/net/gve/gve_ethdev.h
> @@ -193,8 +193,8 @@ struct gve_tx_queue {
>         uint16_t sw_size;
>         uint8_t cur_gen_bit;
>         uint32_t last_desc_cleaned;
> +       uint32_t last_re_idx;
>         void **txqs;
> -       uint16_t re_cnt;
>
>         /* Only valid for DQO_RDA queue format */
>         struct gve_tx_queue *complq;
> diff --git a/drivers/net/gve/gve_tx_dqo.c b/drivers/net/gve/gve_tx_dqo.c
> index c864d722d9..a8382ff3c4 100644
> --- a/drivers/net/gve/gve_tx_dqo.c
> +++ b/drivers/net/gve/gve_tx_dqo.c
> @@ -264,6 +264,8 @@ gve_tx_burst_dqo(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
>         uint16_t nb_descs;
>         bool csum, tso;
>         uint16_t tx_id;
> +       uint32_t last_desc_idx;
> +       uint32_t last_re_interval;

Minor nit: in this driver, the common practice is to follow reverse
Christmas tree declaration ordering where practical.

>
>         pkts = txq->pkt_ring_dqo;
>         txr = txq->tx_ring;
> @@ -331,20 +333,19 @@ gve_tx_burst_dqo(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
>                 txd = &txr[(tx_id - 1) & mask];
>                 txd->pkt.end_of_packet = 1;
>
> +               last_desc_idx = (tx_id - 1) & mask;
> +               last_re_interval = (last_desc_idx - txq->last_re_idx) & mask;
> +               if (unlikely(last_re_interval >= GVE_TX_MIN_RE_INTERVAL)) {
> +                       txd->pkt.report_event = true;
> +                       txq->last_re_idx = last_desc_idx;

Should we write the tail here? Allowing the device to start processing
packets right away instead of only at the end of TX might save some
failed bursts if dealing with a bursts of many-descriptor TSO packets.
The MMIO writes would also be amortized over RE_INTERVAL descriptors.

> +               }
> +
>                 txq->nb_free -= nb_descs;
>                 txq->nb_used += nb_descs;
>         }
>
>         /* update the tail pointer if any packets were processed */
>         if (nb_tx > 0) {
> -               /* Request a descriptor completion on the last descriptor */
> -               txq->re_cnt += nb_tx;
> -               if (txq->re_cnt >= GVE_TX_MIN_RE_INTERVAL) {
> -                       txd = &txr[(tx_id - 1) & mask];
> -                       txd->pkt.report_event = true;
> -                       txq->re_cnt = 0;
> -               }
> -
>                 rte_write32(tx_id, txq->qtx_tail);
>                 txq->tx_tail = tx_id;
>
> @@ -445,6 +446,7 @@ gve_reset_tx_ring_state_dqo(struct gve_tx_queue *txq)
>         txq->nb_used = 0;
>
>         txq->last_desc_cleaned = 0;
> +       txq->last_re_idx = 0;
>         txq->nb_free = txq->nb_tx_desc - 1;
>
>         txq->complq_tail = 0;
> --
> 2.43.0
>

[PATCH 3/4] net/gve: add cacheline overlap margin to DQO TX ring

[Tathagat Priyadarshi]

The GVNIC DQO hardware DMA engine prefetches descriptors in cacheline
units (64 bytes = 4 descriptors of 16 bytes each). When producer tail
and consumer head indices share a cacheline, CPU and device DMA memory
collisions can occur.

Align with the Linux driver (gve_has_tx_slots_available()) by requiring
GVE_TX_MIN_DESC_PREVENT_CACHE_OVERLAP (4 descriptors) headroom when
checking descriptor availability before enqueueing packets.

Fixes: 4022f9999f56 ("net/gve: support basic Tx data path for DQO")
Cc: stable@dpdk.org

Acked-by: Joshua Washington <joshwash@google.com>
Signed-off-by: Tathagat Priyadarshi <tathagat.dpdk@gmail.com>
---
 drivers/net/gve/gve_tx_dqo.c | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)

diff --git a/drivers/net/gve/gve_tx_dqo.c b/drivers/net/gve/gve_tx_dqo.c
index a8382ff3c4..9210b02eb1 100644
--- a/drivers/net/gve/gve_tx_dqo.c
+++ b/drivers/net/gve/gve_tx_dqo.c
@@ -261,6 +261,7 @@ gve_tx_burst_dqo(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
 	uint16_t nb_tx = 0;
 	uint64_t bytes = 0;
 	uint64_t ol_flags;
+	uint16_t req_descs;
 	uint16_t nb_descs;
 	bool csum, tso;
 	uint16_t tx_id;
@@ -291,14 +292,17 @@ gve_tx_burst_dqo(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
 
 		nb_descs = gve_tx_pkt_nb_data_descs(tx_pkt);
 		nb_descs += tso;
+		req_descs = nb_descs + GVE_TX_MIN_DESC_PREVENT_CACHE_OVERLAP;
 
-		/* Clean if there aren't enough descriptors to send the packet. */
-		if (unlikely(!gve_can_tx(txq, nb_descs, 1))) {
+		/* Clean if there aren't enough descriptors to send the packet
+		 * while maintaining the cacheline overlap margin.
+		 */
+		if (unlikely(!gve_can_tx(txq, req_descs, 1))) {
 			int nb_to_clean = RTE_MAX(DQO_TX_MULTIPLIER * txq->rs_thresh,
-						  nb_descs);
+						  (int)req_descs);
 
 			gve_tx_clean_descs_dqo(txq, nb_to_clean);
-			if (!gve_can_tx(txq, nb_descs, 1))
+			if (!gve_can_tx(txq, req_descs, 1))
 				break;
 		}
 
-- 
2.43.0

[PATCH 4/4] net/gve: add packet and TSO hardware constraints for DQO

[Tathagat Priyadarshi]

The GVNIC DQO hardware restricts each packet (or MTU segment in TSO)
to at most 10 data descriptors (GVE_TX_MAX_DATA_DESCS), beyond which
the hardware drops traffic. In addition to this 10-descriptor limit,
the hardware imposes strict packet sizing and header constraints.

Enforce these hardware packet and TSO validation constraints in DQO
RDA mode within gve_tx_validate_descs() matching the upstream Linux
kernel driver:
- Define GVE_TX_MAX_TSO_MSS_DQO (9728 bytes) to avoid 14-bit MSS
  field overflow in struct gve_tx_tso_context_desc_dqo.
- Define GVE_TX_MIN_PKT_SIZE_DQO (17 bytes) and GVE_TX_MAX_PKT_SIZE_DQO
  (9728 bytes) to reject runts (which cause hardware DMA parse errors)
  and oversized non-TSO packets.
- In gve_tx_validate_descs():
  * Reject zero-length mbufs (nb_descs == 0 or pkt_len == 0).
  * Enforce packet size in [17, 9728] for non-TSO packets alongside
    the 10-descriptor limit (nb_descs <= GVE_TX_MAX_DATA_DESCS).
  * Reject split transport headers across mbuf segments
    (data_len < headlen).
  * Reject TSO packets missing an L4 header (l4_len == 0).
  * Reject TSO packets without payload (pkt_len <= headlen).
  * Enforce MSS limits [88, 9728] and header length limit <= 255.

Fixes: 403c671a46b6 ("net/gve: support TSO in DQO RDA")
Cc: stable@dpdk.org

Acked-by: Joshua Washington <joshwash@google.com>
Signed-off-by: Tathagat Priyadarshi <tathagat.dpdk@gmail.com>
---
 drivers/net/gve/base/gve_desc_dqo.h |  3 ++
 drivers/net/gve/gve_tx_dqo.c        | 44 ++++++++++++++++++++++++-----
 2 files changed, 40 insertions(+), 7 deletions(-)

diff --git a/drivers/net/gve/base/gve_desc_dqo.h b/drivers/net/gve/base/gve_desc_dqo.h
index c1534959c2..1740f69d73 100644
--- a/drivers/net/gve/base/gve_desc_dqo.h
+++ b/drivers/net/gve/base/gve_desc_dqo.h
@@ -12,6 +12,9 @@
 
 #define GVE_TX_MAX_HDR_SIZE_DQO 255
 #define GVE_TX_MIN_TSO_MSS_DQO 88
+#define GVE_TX_MAX_TSO_MSS_DQO 9728
+#define GVE_TX_MIN_PKT_SIZE_DQO 17
+#define GVE_TX_MAX_PKT_SIZE_DQO 9728
 
 /* Basic TX descriptor (DTYPE 0x0C) */
 struct gve_tx_pkt_desc_dqo {
diff --git a/drivers/net/gve/gve_tx_dqo.c b/drivers/net/gve/gve_tx_dqo.c
index 9210b02eb1..03ac3bd5dd 100644
--- a/drivers/net/gve/gve_tx_dqo.c
+++ b/drivers/net/gve/gve_tx_dqo.c
@@ -134,12 +134,43 @@ gve_tx_clean_descs_dqo(struct gve_tx_queue *txq, uint16_t nb_descs) {
 static inline bool
 gve_tx_validate_descs(struct rte_mbuf *tx_pkt, uint16_t nb_descs, bool is_tso)
 {
-	if (!is_tso)
+	int tso_segsz, seg_remain, mbuf_remain;
+	int num_descs, seg_offset, mbuf_len;
+	int headlen;
+
+	/* Reject zero-length packet */
+	if (unlikely(nb_descs == 0 || tx_pkt->pkt_len == 0))
+		return false;
+
+	if (!is_tso) {
+		/* Reject packets that violate hardware length constraints */
+		if (unlikely(tx_pkt->pkt_len < GVE_TX_MIN_PKT_SIZE_DQO ||
+			     tx_pkt->pkt_len > GVE_TX_MAX_PKT_SIZE_DQO))
+			return false;
 		return nb_descs <= GVE_TX_MAX_DATA_DESCS;
+	}
 
-	int tso_segsz = tx_pkt->tso_segsz;
-	int num_descs, seg_offset, mbuf_len;
-	int headlen = tx_pkt->l2_len + tx_pkt->l3_len + tx_pkt->l4_len;
+	headlen = tx_pkt->l2_len + tx_pkt->l3_len + tx_pkt->l4_len;
+
+	/* Reject split transport header */
+	if (unlikely(tx_pkt->data_len < headlen))
+		return false;
+
+	/* Reject missing TCP header on TSO */
+	if (unlikely(tx_pkt->l4_len == 0))
+		return false;
+
+	/* Reject TSO packet without payload */
+	if (unlikely(tx_pkt->pkt_len <= (uint32_t)headlen))
+		return false;
+
+	/* Enforce GVE hardware MSS and header limits */
+	if (unlikely(tx_pkt->tso_segsz < GVE_TX_MIN_TSO_MSS_DQO ||
+		     tx_pkt->tso_segsz > GVE_TX_MAX_TSO_MSS_DQO ||
+		     headlen > GVE_TX_MAX_HDR_SIZE_DQO))
+		return false;
+
+	tso_segsz = tx_pkt->tso_segsz;
 
 	/* Headers will be split into their own buffer. */
 	num_descs = 1;
@@ -150,7 +181,7 @@ gve_tx_validate_descs(struct rte_mbuf *tx_pkt, uint16_t nb_descs, bool is_tso)
 		if (!mbuf_len)
 			goto next_mbuf;
 
-		int seg_remain = tso_segsz - seg_offset;
+		seg_remain = tso_segsz - seg_offset;
 		if (num_descs == GVE_TX_MAX_DATA_DESCS && seg_remain)
 			return false;
 
@@ -162,7 +193,7 @@ gve_tx_validate_descs(struct rte_mbuf *tx_pkt, uint16_t nb_descs, bool is_tso)
 			 * is not at risk of defying this limitation.
 			 * Thus, such segments are ignored.
 			 */
-			int mbuf_remain = tx_pkt->data_len % GVE_TX_MAX_BUF_SIZE_DQO;
+			mbuf_remain = tx_pkt->data_len % GVE_TX_MAX_BUF_SIZE_DQO;
 
 			/* For each TSO segment, HW will prepend
 			 * headers. The remaining bytes of this mbuf
@@ -184,7 +215,6 @@ gve_tx_validate_descs(struct rte_mbuf *tx_pkt, uint16_t nb_descs, bool is_tso)
 			mbuf_len = tx_pkt->data_len;
 	}
 
-
 	return true;
 }
 
-- 
2.43.0

Re: [PATCH 4/4] net/gve: add packet and TSO hardware constraints for DQO

[Joshua Washington]

On Mon, Oct 12, 2026 at 3:50 AM Tathagat Priyadarshi
<tathagat.dpdk@gmail.com> wrote:
>
> The GVNIC DQO hardware restricts each packet (or MTU segment in TSO)
> to at most 10 data descriptors (GVE_TX_MAX_DATA_DESCS), beyond which
> the hardware drops traffic. In addition to this 10-descriptor limit,
> the hardware imposes strict packet sizing and header constraints.
>
> Enforce these hardware packet and TSO validation constraints in DQO
> RDA mode within gve_tx_validate_descs() matching the upstream Linux
> kernel driver:
> - Define GVE_TX_MAX_TSO_MSS_DQO (9728 bytes) to avoid 14-bit MSS
>   field overflow in struct gve_tx_tso_context_desc_dqo.
> - Define GVE_TX_MIN_PKT_SIZE_DQO (17 bytes) and GVE_TX_MAX_PKT_SIZE_DQO
>   (9728 bytes) to reject runts (which cause hardware DMA parse errors)
>   and oversized non-TSO packets.
> - In gve_tx_validate_descs():
>   * Reject zero-length mbufs (nb_descs == 0 or pkt_len == 0).
>   * Enforce packet size in [17, 9728] for non-TSO packets alongside
>     the 10-descriptor limit (nb_descs <= GVE_TX_MAX_DATA_DESCS).
>   * Reject split transport headers across mbuf segments
>     (data_len < headlen).
>   * Reject TSO packets missing an L4 header (l4_len == 0).
>   * Reject TSO packets without payload (pkt_len <= headlen).
>   * Enforce MSS limits [88, 9728] and header length limit <= 255.
>
> Fixes: 403c671a46b6 ("net/gve: support TSO in DQO RDA")
> Cc: stable@dpdk.org
>
> Acked-by: Joshua Washington <joshwash@google.com>
> Signed-off-by: Tathagat Priyadarshi <tathagat.dpdk@gmail.com>
> ---
>  drivers/net/gve/base/gve_desc_dqo.h |  3 ++
>  drivers/net/gve/gve_tx_dqo.c        | 44 ++++++++++++++++++++++++-----
>  2 files changed, 40 insertions(+), 7 deletions(-)
>
> diff --git a/drivers/net/gve/base/gve_desc_dqo.h b/drivers/net/gve/base/gve_desc_dqo.h
> index c1534959c2..1740f69d73 100644
> --- a/drivers/net/gve/base/gve_desc_dqo.h
> +++ b/drivers/net/gve/base/gve_desc_dqo.h
> @@ -12,6 +12,9 @@
>
>  #define GVE_TX_MAX_HDR_SIZE_DQO 255
>  #define GVE_TX_MIN_TSO_MSS_DQO 88
> +#define GVE_TX_MAX_TSO_MSS_DQO 9728
> +#define GVE_TX_MIN_PKT_SIZE_DQO 17
> +#define GVE_TX_MAX_PKT_SIZE_DQO 9728

GVE_TX_MAX_TSO_MSS_DQO and GVE_TX_MAX_PKT_SIZE_DQO have the same
semantic meaning: they represent the maximum wire packet allowed on
the NIC. Let's opt to use a single value here.

>
>  /* Basic TX descriptor (DTYPE 0x0C) */
>  struct gve_tx_pkt_desc_dqo {
> diff --git a/drivers/net/gve/gve_tx_dqo.c b/drivers/net/gve/gve_tx_dqo.c
> index 9210b02eb1..03ac3bd5dd 100644
> --- a/drivers/net/gve/gve_tx_dqo.c
> +++ b/drivers/net/gve/gve_tx_dqo.c
> @@ -134,12 +134,43 @@ gve_tx_clean_descs_dqo(struct gve_tx_queue *txq, uint16_t nb_descs) {
>  static inline bool
>  gve_tx_validate_descs(struct rte_mbuf *tx_pkt, uint16_t nb_descs, bool is_tso)
>  {
> -       if (!is_tso)
> +       int tso_segsz, seg_remain, mbuf_remain;
> +       int num_descs, seg_offset, mbuf_len;
> +       int headlen;
> +
> +       /* Reject zero-length packet */
> +       if (unlikely(nb_descs == 0 || tx_pkt->pkt_len == 0))
> +               return false;
> +
> +       if (!is_tso) {
> +               /* Reject packets that violate hardware length constraints */
> +               if (unlikely(tx_pkt->pkt_len < GVE_TX_MIN_PKT_SIZE_DQO ||
> +                            tx_pkt->pkt_len > GVE_TX_MAX_PKT_SIZE_DQO))
> +                       return false;
>                 return nb_descs <= GVE_TX_MAX_DATA_DESCS;
> +       }
>
> -       int tso_segsz = tx_pkt->tso_segsz;
> -       int num_descs, seg_offset, mbuf_len;
> -       int headlen = tx_pkt->l2_len + tx_pkt->l3_len + tx_pkt->l4_len;
> +       headlen = tx_pkt->l2_len + tx_pkt->l3_len + tx_pkt->l4_len;
> +
> +       /* Reject split transport header */
> +       if (unlikely(tx_pkt->data_len < headlen))
> +               return false;
> +
> +       /* Reject missing TCP header on TSO */
> +       if (unlikely(tx_pkt->l4_len == 0))
> +               return false;
> +
> +       /* Reject TSO packet without payload */
> +       if (unlikely(tx_pkt->pkt_len <= (uint32_t)headlen))
> +               return false;
> +
> +       /* Enforce GVE hardware MSS and header limits */
> +       if (unlikely(tx_pkt->tso_segsz < GVE_TX_MIN_TSO_MSS_DQO ||
> +                    tx_pkt->tso_segsz > GVE_TX_MAX_TSO_MSS_DQO ||
> +                    headlen > GVE_TX_MAX_HDR_SIZE_DQO))
> +               return false;
> +
> +       tso_segsz = tx_pkt->tso_segsz;
>
>         /* Headers will be split into their own buffer. */
>         num_descs = 1;
> @@ -150,7 +181,7 @@ gve_tx_validate_descs(struct rte_mbuf *tx_pkt, uint16_t nb_descs, bool is_tso)
>                 if (!mbuf_len)
>                         goto next_mbuf;
>
> -               int seg_remain = tso_segsz - seg_offset;
> +               seg_remain = tso_segsz - seg_offset;

Thanks for fixing the forward declarations but generally, declarations
should happen in the smallest scope possible. Since seg_remain and
mbuf_remain aren't used outside of this loop and branch, respectively,
they should be declared at those scopes. It might also be appropriate
to just declare both at the start of the loop.

>                 if (num_descs == GVE_TX_MAX_DATA_DESCS && seg_remain)
>                         return false;
>
> @@ -162,7 +193,7 @@ gve_tx_validate_descs(struct rte_mbuf *tx_pkt, uint16_t nb_descs, bool is_tso)
>                          * is not at risk of defying this limitation.
>                          * Thus, such segments are ignored.
>                          */
> -                       int mbuf_remain = tx_pkt->data_len % GVE_TX_MAX_BUF_SIZE_DQO;
> +                       mbuf_remain = tx_pkt->data_len % GVE_TX_MAX_BUF_SIZE_DQO;
>
>                         /* For each TSO segment, HW will prepend
>                          * headers. The remaining bytes of this mbuf
> @@ -184,7 +215,6 @@ gve_tx_validate_descs(struct rte_mbuf *tx_pkt, uint16_t nb_descs, bool is_tso)
>                         mbuf_len = tx_pkt->data_len;
>         }
>
> -
>         return true;
>  }
>
> --
> 2.43.0
>

Thanks for sending this!

Re: [PATCH 0/4] net/gve: fix TX queue stalls and sync DQO RDA with Linux driver

[Stephen Hemminger]

On Mon, 12 Oct 2026 10:50:19 +0000
Tathagat Priyadarshi <tathagat.dpdk@gmail.com> wrote:

> This patch series aligns the Google Virtual NIC (gve) DQO RDA transmit path
> with the upstream Linux kernel driver (drivers/net/ethernet/google/gve/) to
> resolve transmit queue stalls and hardware validation issues:
> 
> 1. net/gve: fix DQO descriptor cleaning loop bound
>    Fixes the off-by-one error in gve_tx_clean_descs_dqo()
>    ("while (--nb_descs)" -> "while (nb_descs--)") and avoids stalling
>    the completion queue on unrecognized descriptor types.
> 
> 2. net/gve: fix DQO TX queue stall on multi-seg and TSO
>    Tracks descriptor intervals with last_re_idx per packet against
>    GVE_TX_MIN_RE_INTERVAL (32) instead of counting packets at burst end,
>    preventing descriptor ring starvation and queue deadlocks under
>    multi-segment and TSO traffic.
> 
> 3. net/gve: add cacheline overlap margin to DQO TX ring
>    Enforces GVE_TX_MIN_DESC_PREVENT_CACHE_OVERLAP (4 descriptors) margin
>    in ring space checks, avoiding CPU and device DMA cacheline collisions.
> 
> 4. net/gve: add packet and TSO hardware constraints for DQO
>    Augments gve_tx_validate_descs() (which enforces the 10-descriptor
>    limit per packet or MTU segment) with the remaining hardware length
>    constraints: [17, 9728] bytes for non-TSO packets, maximum MSS limit
>    (9728 bytes) to avoid 14-bit field wrap-around, and validates header
>    contiguity and payload presence on TSO packets.
> 
> Tathagat Priyadarshi (4):
>   net/gve: fix DQO descriptor cleaning loop bound
>   net/gve: fix DQO TX queue stall on multi-seg and TSO
>   net/gve: add cacheline overlap margin to DQO TX ring
>   net/gve: add packet and TSO hardware constraints for DQO
> 
>  drivers/net/gve/base/gve_desc_dqo.h |  3 ++
>  drivers/net/gve/gve_ethdev.h        |  2 +-
>  drivers/net/gve/gve_tx_dqo.c        | 80 +++++++++++++++++++++--------
>  3 files changed, 62 insertions(+), 23 deletions(-)

The problem with the series is in the last patch.
The way tx_burst works the driver has to consume packets that cause
errors.

Subject: Re: [PATCH 0/4] net/gve: DQO Tx fixes

Applies cleanly to main; every commit builds with -Dwerror=true.
Patches 1-3 match the Linux gve driver (unknown completion types are
skipped, report_event spaced by descriptor index, 4 descriptor
cacheline margin).

Errors

[PATCH 4/4] net/gve: add packet and TSO hardware constraints for DQO

1. An invalid packet stalls the queue instead of being dropped.

       if (!gve_tx_validate_descs(tx_pkt, nb_descs, tso)) {
               txq->stats.too_many_descs++;
               break;
       }

   tx_burst returns short, so the application gets the bad mbuf back
   and retries it at the head of the next burst. It is rejected again,
   and nothing behind it is ever sent. The break was already there for
   the descriptor limit; this patch extends it to runts, oversize
   frames, bad MSS, split headers and zero length, all per-packet
   errors. A short return should mean only ring full. Linux frees the
   skb and returns NETDEV_TX_OK for the same checks.

   Validate before the ring space check, then free and continue:

       nb_descs = gve_tx_pkt_nb_data_descs(tx_pkt);
       nb_descs += tso;

       /* Drop packet if it doesn't adhere to hardware limits. */
       if (unlikely(!gve_tx_validate_descs(tx_pkt, nb_descs, tso))) {
               txq->stats.too_many_descs++;
               rte_pktmbuf_free(tx_pkt);
               nb_drop++;
               continue;
       }

       req_descs = nb_descs + GVE_TX_MIN_DESC_PREVENT_CACHE_OVERLAP;
       ...

   and use nb_tx - nb_drop for stats.packets. too_many_descs now counts
   more than descriptor overruns; renaming it, or adding a drop
   counter, would make the xstat less misleading. The rte_mbuf_check()
   failure just above has the same break and needs the same fix.

Info

[PATCH 4/4] net/gve: add packet and TSO hardware constraints for DQO

2. The non-TSO length check fixes the original DQO Tx path, not TSO
   support. Add:

   Fixes: 4022f9999f56 ("net/gve: support basic Tx data path for DQO")

[PATCH v2 0/4] net/gve: fix Tx queue stalls and sync DQO RDA with Linux driver

[Tathagat Priyadarshi]

This patch series aligns the Google Virtual NIC (gve) DQO RDA transmit path
with the upstream Linux kernel driver (drivers/net/ethernet/google/gve/) to
resolve transmit queue stalls and hardware validation issues:

1. net/gve: fix DQO descriptor cleaning loop bound
   Fixes the off-by-one error in gve_tx_clean_descs_dqo()
   ("while (--nb_descs)" -> "while (nb_descs--)") and avoids stalling
   the completion queue on unrecognized descriptor types.

2. net/gve: fix DQO Tx queue stall on multi-seg and TSO
   Tracks descriptor intervals with last_re_idx per packet against
   GVE_TX_MIN_RE_INTERVAL (32) instead of counting packets at burst end,
   preventing descriptor ring starvation and queue deadlocks under
   multi-segment and TSO traffic.

3. net/gve: add cacheline overlap margin to DQO Tx ring
   Enforces GVE_TX_MIN_DESC_PREVENT_CACHE_OVERLAP (4 descriptors) margin
   in ring space checks, avoiding CPU and device DMA cacheline collisions.

4. net/gve: add packet and TSO hardware constraints for DQO
   Augments gve_tx_validate_descs() (which enforces the 10-descriptor
   limit per packet or MTU segment) with the remaining hardware length
   constraints: [17, 9728] bytes for non-TSO packets, maximum MSS limit
   (9728 bytes), TSO total length limit (262143 bytes), and validates
   header contiguity and payload presence on TSO packets. Invalid packets
   are dropped instead of stalling the queue.

v2:
- Patch 1/4, 3/4: No changes.
- Patch 2/4:
  * Write the Tx tail when report_event is set so the device can start
    fetching descriptors before the end of the burst
    (per Joshua Washington).
  * Pop the completion tag before writing the TSO context descriptor.
  * Use reverse Christmas tree declaration ordering
    (per Joshua Washington).
- Patch 4/4:
  * Drop (free) invalid mbufs and continue the burst instead of
    returning short, which caused head-of-line blocking
    (per Stephen Hemminger).
  * Validate packets before the ring space check.
  * Exclude dropped packets from stats.packets.
  * Rename too_many_descs to dropped_pkts since it now counts all
    dropped packets (per Stephen Hemminger).
  * Use a single GVE_TX_MAX_PKT_SIZE_DQO for the max packet size and
    max MSS (per Joshua Washington).
  * Declare seg_remain and mbuf_remain in the smallest scope
    (per Joshua Washington).
  * Reject packets whose descriptor count can never fit in the Tx ring.
  * Enforce the TSO total length limit (262143 bytes).
  * Add Fixes tag for the original DQO Tx path (per Stephen Hemminger).

Tathagat Priyadarshi (4):
  net/gve: fix DQO descriptor cleaning loop bound
  net/gve: fix DQO Tx queue stall on multi-seg and TSO
  net/gve: add cacheline overlap margin to DQO Tx ring
  net/gve: add packet and TSO hardware constraints for DQO

 drivers/net/gve/base/gve_desc_dqo.h |   3 +
 drivers/net/gve/gve_ethdev.h        |   4 +-
 drivers/net/gve/gve_tx_dqo.c        | 152 ++++++++++++++++++++--------
 3 files changed, 115 insertions(+), 44 deletions(-)

-- 
2.43.0

[PATCH v2 1/4] net/gve: fix DQO descriptor cleaning loop bound

[Tathagat Priyadarshi]

In gve_tx_clean_descs_dqo(), the loop condition "while (--nb_descs)"
causes an off-by-one error:
- When nb_descs == 1, --nb_descs decrements to 0 on the first check,
  executing 0 iterations instead of 1.
- When nb_descs == 0, --nb_descs underflows uint16_t to 65535,
  executing 65535 iterations.

Change the condition to "while (nb_descs--)" so that exactly nb_descs
iterations are executed for any input value.

Additionally, in gve_tx_clean_dqo(), handle unrecognized completion
types by breaking from the switch rather than returning immediately,
allowing complq_tail to advance past unknown descriptors rather than
stalling the completion ring at the same entry.

Fixes: 4022f9999f56 ("net/gve: support basic Tx data path for DQO")
Cc: stable@dpdk.org

Signed-off-by: Tathagat Priyadarshi <tathagat.dpdk@gmail.com>
Acked-by: Joshua Washington <joshwash@google.com>
---
 drivers/net/gve/gve_tx_dqo.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/drivers/net/gve/gve_tx_dqo.c b/drivers/net/gve/gve_tx_dqo.c
index 5c03d3d1a2..c864d722d9 100644
--- a/drivers/net/gve/gve_tx_dqo.c
+++ b/drivers/net/gve/gve_tx_dqo.c
@@ -108,8 +108,8 @@ gve_tx_clean_dqo(struct gve_tx_queue *txq)
 		PMD_DRV_DP_LOG(DEBUG, "GVE_COMPL_TYPE_DQO_MISS ignored !!!");
 		break;
 	default:
-		PMD_DRV_DP_LOG(ERR, "unknown completion type.");
-		return;
+		PMD_DRV_DP_LOG(ERR, "unknown completion type %u", compl_desc->type);
+		break;
 	}
 
 	next++;
@@ -122,7 +122,7 @@ gve_tx_clean_dqo(struct gve_tx_queue *txq)
 
 static inline void
 gve_tx_clean_descs_dqo(struct gve_tx_queue *txq, uint16_t nb_descs) {
-	while (--nb_descs)
+	while (nb_descs--)
 		gve_tx_clean_dqo(txq);
 }
 
-- 
2.43.0

[PATCH v2 2/4] net/gve: fix DQO Tx queue stall on multi-seg and TSO

[Tathagat Priyadarshi]

The DPDK GVE driver previously tracked transmitted packet count (re_cnt)
at the end of each burst rather than tracking descriptor intervals per
packet against GVE_TX_MIN_RE_INTERVAL (32).

In multi-segment and TSO transmissions, each packet consumes multiple
descriptors. The descriptor ring could exhaust before 32 packets were
sent, resulting in zero descriptors requesting report_event and
preventing the hardware DMA engine from returning GVE_COMPL_TYPE_DQO_DESC
completions. Without these completions, txq->nb_free was never
replenished, permanently stalling the TX queue.

Align with the Linux kernel driver (gve_tx_update_tail()) by tracking
descriptor indices with last_re_idx:
- Replace re_cnt with last_re_idx in struct gve_tx_queue.
- Reset last_re_idx = 0 in gve_reset_tx_ring_state_dqo().
- On each packet transmission, check descriptor interval
  (last_desc_idx - last_re_idx) & mask >= GVE_TX_MIN_RE_INTERVAL.
  When met, set report_event on the packet's last descriptor and
  update last_re_idx.
- Ring the transmit doorbell immediately upon setting report_event to
  kick hardware processing early and reduce device fetch latency during
  bursts of multi-descriptor TSO packets, amortizing MMIO writes over
  RE_INTERVAL descriptors.
- Defensively pop completion tags before writing TSO context descriptors
  to avoid partial descriptor ring updates if completion tags are
  exhausted.

Fixes: 4022f9999f56 ("net/gve: support basic Tx data path for DQO")
Fixes: 403c671a46b6 ("net/gve: support TSO in DQO RDA")
Cc: stable@dpdk.org

Signed-off-by: Tathagat Priyadarshi <tathagat.dpdk@gmail.com>
---
 drivers/net/gve/gve_ethdev.h |  2 +-
 drivers/net/gve/gve_tx_dqo.c | 46 ++++++++++++++++++++----------------
 2 files changed, 27 insertions(+), 21 deletions(-)

diff --git a/drivers/net/gve/gve_ethdev.h b/drivers/net/gve/gve_ethdev.h
index c9a176ff17..2033f49444 100644
--- a/drivers/net/gve/gve_ethdev.h
+++ b/drivers/net/gve/gve_ethdev.h
@@ -193,8 +193,8 @@ struct gve_tx_queue {
 	uint16_t sw_size;
 	uint8_t cur_gen_bit;
 	uint32_t last_desc_cleaned;
+	uint32_t last_re_idx;
 	void **txqs;
-	uint16_t re_cnt;
 
 	/* Only valid for DQO_RDA queue format */
 	struct gve_tx_queue *complq;
diff --git a/drivers/net/gve/gve_tx_dqo.c b/drivers/net/gve/gve_tx_dqo.c
index c864d722d9..0839179d7f 100644
--- a/drivers/net/gve/gve_tx_dqo.c
+++ b/drivers/net/gve/gve_tx_dqo.c
@@ -250,21 +250,23 @@ gve_fill_tx_pkt_desc(struct gve_tx_queue *txq, uint16_t *tx_id,
 uint16_t
 gve_tx_burst_dqo(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
 {
+	volatile union gve_tx_desc_dqo *txr, *txd;
 	struct gve_tx_queue *txq = tx_queue;
-	volatile union gve_tx_desc_dqo *txr;
-	volatile union gve_tx_desc_dqo *txd;
-	uint16_t mask = txq->nb_tx_desc - 1;
-	struct gve_tx_pkt *pkts;
+	uint32_t last_re_interval;
 	struct rte_mbuf *tx_pkt;
-	uint16_t compl_tag;
-	const char *reason;
+	struct gve_tx_pkt *pkts;
+	uint32_t last_desc_idx;
 	uint16_t nb_tx = 0;
 	uint64_t bytes = 0;
+	const char *reason;
+	uint16_t compl_tag;
 	uint64_t ol_flags;
 	uint16_t nb_descs;
-	bool csum, tso;
 	uint16_t tx_id;
+	bool csum, tso;
+	uint16_t mask;
 
+	mask = txq->nb_tx_desc - 1;
 	pkts = txq->pkt_ring_dqo;
 	txr = txq->tx_ring;
 
@@ -306,15 +308,15 @@ gve_tx_burst_dqo(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
 			break;
 		}
 
+		if (!gve_free_compl_tags_pop(txq, &compl_tag))
+			break;
+
 		if (tso) {
 			txd = &txr[tx_id];
 			gve_tx_fill_seg_desc_dqo(txd, tx_pkt);
 			tx_id = (tx_id + 1) & mask;
 		}
 
-		if (!gve_free_compl_tags_pop(txq, &compl_tag))
-			break;
-
 		pkts[compl_tag].mbuf = tx_pkt;
 		while (tx_pkt) {
 			/* Skip writing descriptors if mbuf has no data. */
@@ -331,23 +333,26 @@ gve_tx_burst_dqo(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
 		txd = &txr[(tx_id - 1) & mask];
 		txd->pkt.end_of_packet = 1;
 
+		last_desc_idx = (tx_id - 1) & mask;
+		last_re_interval = (last_desc_idx - txq->last_re_idx) & mask;
+		if (unlikely(last_re_interval >= GVE_TX_MIN_RE_INTERVAL)) {
+			txd->pkt.report_event = true;
+			txq->last_re_idx = last_desc_idx;
+			rte_write32(tx_id, txq->qtx_tail);
+			txq->tx_tail = tx_id;
+		}
+
 		txq->nb_free -= nb_descs;
 		txq->nb_used += nb_descs;
 	}
 
-	/* update the tail pointer if any packets were processed */
-	if (nb_tx > 0) {
-		/* Request a descriptor completion on the last descriptor */
-		txq->re_cnt += nb_tx;
-		if (txq->re_cnt >= GVE_TX_MIN_RE_INTERVAL) {
-			txd = &txr[(tx_id - 1) & mask];
-			txd->pkt.report_event = true;
-			txq->re_cnt = 0;
-		}
-
+	/* update the tail pointer if any uncommitted descriptors remain */
+	if (txq->tx_tail != tx_id) {
 		rte_write32(tx_id, txq->qtx_tail);
 		txq->tx_tail = tx_id;
+	}
 
+	if (nb_tx > 0) {
 		txq->stats.packets += nb_tx;
 		txq->stats.bytes += bytes;
 		txq->stats.errors += nb_pkts - nb_tx;
@@ -445,6 +450,7 @@ gve_reset_tx_ring_state_dqo(struct gve_tx_queue *txq)
 	txq->nb_used = 0;
 
 	txq->last_desc_cleaned = 0;
+	txq->last_re_idx = 0;
 	txq->nb_free = txq->nb_tx_desc - 1;
 
 	txq->complq_tail = 0;
-- 
2.43.0

Re: [PATCH v2 2/4] net/gve: fix DQO Tx queue stall on multi-seg and TSO

[Joshua Washington]

On Tue, Oct 13, 2026 at 3:44 AM Tathagat Priyadarshi
<tathagat.dpdk@gmail.com> wrote:
>
> The DPDK GVE driver previously tracked transmitted packet count (re_cnt)
> at the end of each burst rather than tracking descriptor intervals per
> packet against GVE_TX_MIN_RE_INTERVAL (32).
>
> In multi-segment and TSO transmissions, each packet consumes multiple
> descriptors. The descriptor ring could exhaust before 32 packets were
> sent, resulting in zero descriptors requesting report_event and
> preventing the hardware DMA engine from returning GVE_COMPL_TYPE_DQO_DESC
> completions. Without these completions, txq->nb_free was never
> replenished, permanently stalling the TX queue.
>
> Align with the Linux kernel driver (gve_tx_update_tail()) by tracking
> descriptor indices with last_re_idx:
> - Replace re_cnt with last_re_idx in struct gve_tx_queue.
> - Reset last_re_idx = 0 in gve_reset_tx_ring_state_dqo().
> - On each packet transmission, check descriptor interval
>   (last_desc_idx - last_re_idx) & mask >= GVE_TX_MIN_RE_INTERVAL.
>   When met, set report_event on the packet's last descriptor and
>   update last_re_idx.
> - Ring the transmit doorbell immediately upon setting report_event to
>   kick hardware processing early and reduce device fetch latency during
>   bursts of multi-descriptor TSO packets, amortizing MMIO writes over
>   RE_INTERVAL descriptors.
> - Defensively pop completion tags before writing TSO context descriptors
>   to avoid partial descriptor ring updates if completion tags are
>   exhausted.
>
> Fixes: 4022f9999f56 ("net/gve: support basic Tx data path for DQO")
> Fixes: 403c671a46b6 ("net/gve: support TSO in DQO RDA")
> Cc: stable@dpdk.org
>
> Signed-off-by: Tathagat Priyadarshi <tathagat.dpdk@gmail.com>
> ---
>  drivers/net/gve/gve_ethdev.h |  2 +-
>  drivers/net/gve/gve_tx_dqo.c | 46 ++++++++++++++++++++----------------
>  2 files changed, 27 insertions(+), 21 deletions(-)
>
> diff --git a/drivers/net/gve/gve_ethdev.h b/drivers/net/gve/gve_ethdev.h
> index c9a176ff17..2033f49444 100644
> --- a/drivers/net/gve/gve_ethdev.h
> +++ b/drivers/net/gve/gve_ethdev.h
> @@ -193,8 +193,8 @@ struct gve_tx_queue {
>         uint16_t sw_size;
>         uint8_t cur_gen_bit;
>         uint32_t last_desc_cleaned;
> +       uint32_t last_re_idx;
>         void **txqs;
> -       uint16_t re_cnt;
>
>         /* Only valid for DQO_RDA queue format */
>         struct gve_tx_queue *complq;
> diff --git a/drivers/net/gve/gve_tx_dqo.c b/drivers/net/gve/gve_tx_dqo.c
> index c864d722d9..0839179d7f 100644
> --- a/drivers/net/gve/gve_tx_dqo.c
> +++ b/drivers/net/gve/gve_tx_dqo.c
> @@ -250,21 +250,23 @@ gve_fill_tx_pkt_desc(struct gve_tx_queue *txq, uint16_t *tx_id,
>  uint16_t
>  gve_tx_burst_dqo(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
>  {
> +       volatile union gve_tx_desc_dqo *txr, *txd;
>         struct gve_tx_queue *txq = tx_queue;
> -       volatile union gve_tx_desc_dqo *txr;
> -       volatile union gve_tx_desc_dqo *txd;
> -       uint16_t mask = txq->nb_tx_desc - 1;
> -       struct gve_tx_pkt *pkts;
> +       uint32_t last_re_interval;
>         struct rte_mbuf *tx_pkt;
> -       uint16_t compl_tag;
> -       const char *reason;
> +       struct gve_tx_pkt *pkts;
> +       uint32_t last_desc_idx;
>         uint16_t nb_tx = 0;
>         uint64_t bytes = 0;
> +       const char *reason;
> +       uint16_t compl_tag;
>         uint64_t ol_flags;
>         uint16_t nb_descs;
> -       bool csum, tso;
>         uint16_t tx_id;
> +       bool csum, tso;
> +       uint16_t mask;

It's kind of hard to tell from first glance, given how many
declarations are here and how many are changing, but are all of these
changes necessary for a fix? Things like moving txr, txd to the same
line or separating the declaration and definition of mask, even though
txq is already defined in an earlier line anyway seem unnecessary at
first glance. There might be other such changes, but I stopped there,
since those are in the first 5 lines of delta. Let's try to keep the
fix focused, only making changes that are actually necessary.

>
> +       mask = txq->nb_tx_desc - 1;
>         pkts = txq->pkt_ring_dqo;
>         txr = txq->tx_ring;
>
> @@ -306,15 +308,15 @@ gve_tx_burst_dqo(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
>                         break;
>                 }
>
> +               if (!gve_free_compl_tags_pop(txq, &compl_tag))
> +                       break;
> +
>                 if (tso) {
>                         txd = &txr[tx_id];
>                         gve_tx_fill_seg_desc_dqo(txd, tx_pkt);
>                         tx_id = (tx_id + 1) & mask;
>                 }
>
> -               if (!gve_free_compl_tags_pop(txq, &compl_tag))
> -                       break;
> -
>                 pkts[compl_tag].mbuf = tx_pkt;
>                 while (tx_pkt) {
>                         /* Skip writing descriptors if mbuf has no data. */
> @@ -331,23 +333,26 @@ gve_tx_burst_dqo(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
>                 txd = &txr[(tx_id - 1) & mask];
>                 txd->pkt.end_of_packet = 1;
>
> +               last_desc_idx = (tx_id - 1) & mask;
> +               last_re_interval = (last_desc_idx - txq->last_re_idx) & mask;
> +               if (unlikely(last_re_interval >= GVE_TX_MIN_RE_INTERVAL)) {
> +                       txd->pkt.report_event = true;
> +                       txq->last_re_idx = last_desc_idx;
> +                       rte_write32(tx_id, txq->qtx_tail);
> +                       txq->tx_tail = tx_id;
> +               }
> +
>                 txq->nb_free -= nb_descs;
>                 txq->nb_used += nb_descs;
>         }
>
> -       /* update the tail pointer if any packets were processed */
> -       if (nb_tx > 0) {
> -               /* Request a descriptor completion on the last descriptor */
> -               txq->re_cnt += nb_tx;
> -               if (txq->re_cnt >= GVE_TX_MIN_RE_INTERVAL) {
> -                       txd = &txr[(tx_id - 1) & mask];
> -                       txd->pkt.report_event = true;
> -                       txq->re_cnt = 0;
> -               }
> -
> +       /* update the tail pointer if any uncommitted descriptors remain */
> +       if (txq->tx_tail != tx_id) {
>                 rte_write32(tx_id, txq->qtx_tail);
>                 txq->tx_tail = tx_id;
> +       }
>
> +       if (nb_tx > 0) {
>                 txq->stats.packets += nb_tx;
>                 txq->stats.bytes += bytes;
>                 txq->stats.errors += nb_pkts - nb_tx;
> @@ -445,6 +450,7 @@ gve_reset_tx_ring_state_dqo(struct gve_tx_queue *txq)
>         txq->nb_used = 0;
>
>         txq->last_desc_cleaned = 0;
> +       txq->last_re_idx = 0;
>         txq->nb_free = txq->nb_tx_desc - 1;
>
>         txq->complq_tail = 0;
> --
> 2.43.0
>

[PATCH v2 3/4] net/gve: add cacheline overlap margin to DQO Tx ring

[Tathagat Priyadarshi]

The GVNIC DQO hardware DMA engine prefetches descriptors in cacheline
units (64 bytes = 4 descriptors of 16 bytes each). When producer tail
and consumer head indices share a cacheline, CPU and device DMA memory
collisions can occur.

Align with the Linux driver (gve_has_tx_slots_available()) by requiring
GVE_TX_MIN_DESC_PREVENT_CACHE_OVERLAP (4 descriptors) headroom when
checking descriptor availability before enqueueing packets.

Fixes: 4022f9999f56 ("net/gve: support basic Tx data path for DQO")
Cc: stable@dpdk.org

Signed-off-by: Tathagat Priyadarshi <tathagat.dpdk@gmail.com>
Acked-by: Joshua Washington <joshwash@google.com>
---
 drivers/net/gve/gve_tx_dqo.c | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)

diff --git a/drivers/net/gve/gve_tx_dqo.c b/drivers/net/gve/gve_tx_dqo.c
index 0839179d7f..eee1f10df1 100644
--- a/drivers/net/gve/gve_tx_dqo.c
+++ b/drivers/net/gve/gve_tx_dqo.c
@@ -260,6 +260,7 @@ gve_tx_burst_dqo(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
 	uint64_t bytes = 0;
 	const char *reason;
 	uint16_t compl_tag;
+	uint16_t req_descs;
 	uint64_t ol_flags;
 	uint16_t nb_descs;
 	uint16_t tx_id;
@@ -291,14 +292,17 @@ gve_tx_burst_dqo(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
 
 		nb_descs = gve_tx_pkt_nb_data_descs(tx_pkt);
 		nb_descs += tso;
+		req_descs = nb_descs + GVE_TX_MIN_DESC_PREVENT_CACHE_OVERLAP;
 
-		/* Clean if there aren't enough descriptors to send the packet. */
-		if (unlikely(!gve_can_tx(txq, nb_descs, 1))) {
+		/* Clean if there aren't enough descriptors to send the packet
+		 * while maintaining the cacheline overlap margin.
+		 */
+		if (unlikely(!gve_can_tx(txq, req_descs, 1))) {
 			int nb_to_clean = RTE_MAX(DQO_TX_MULTIPLIER * txq->rs_thresh,
-						  nb_descs);
+						  (int)req_descs);
 
 			gve_tx_clean_descs_dqo(txq, nb_to_clean);
-			if (!gve_can_tx(txq, nb_descs, 1))
+			if (!gve_can_tx(txq, req_descs, 1))
 				break;
 		}
 
-- 
2.43.0

[PATCH v2 4/4] net/gve: add packet and TSO hardware constraints for DQO

[Tathagat Priyadarshi]

The GVNIC DQO hardware restricts each packet (or MTU segment in TSO)
to at most 10 data descriptors (GVE_TX_MAX_DATA_DESCS), beyond which
the hardware drops traffic. In addition to this 10-descriptor limit,
the hardware imposes strict packet sizing and header constraints.

Previously, failing the descriptor limit or rte_mbuf_check() broke
from the Tx burst loop without consuming the invalid mbuf. This caused
tx_burst to return a short count, leading the application to retry the
same rejected packet indefinitely and causing Head-of-Line (HOL)
queue stalls.

Enforce hardware validation and prevent HOL blocking in DPDK:
- Validate packet and hardware constraints prior to ring space checks.
- When packet validation or rte_mbuf_check() fails, drop and free the
  invalid mbuf with rte_pktmbuf_free(), update stats, and continue the
  burst to prevent Head-of-Line (HOL) blocking.
- Exclude dropped packets from txq->stats.packets while returning the
  total consumed count to the caller.
- Rename too_many_descs in struct gve_tx_stats to dropped_pkts to reflect
  all validation and malformed mbuf drop events.
- Reject packets whose descriptor count exceeds the maximum capacity of
  the ring (accounting for reserved slots, cacheline overlap margin, and
  unreported descriptors), preventing permanent queue stalls.
- Define GVE_TX_MIN_PKT_SIZE_DQO (17 bytes) and GVE_TX_MAX_PKT_SIZE_DQO
  (9728 bytes) to enforce wire packet boundaries for non-TSO packets
  and TSO MSS.
- Define GVE_TX_MAX_TSO_LEN_DQO (262143 bytes) to enforce the hardware
  L4 payload limit in the 24-bit tso_total_len field.
- In gve_tx_validate_descs():
  * Enforce packet size in [17, 9728] for non-TSO packets alongside
    the 10-descriptor limit (nb_descs <= GVE_TX_MAX_DATA_DESCS).
  * Reject TSO packets missing an L4 header (l4_len == 0).
  * Reject packet with headers (L2-L4) not contiguous in first segment.
  * Reject TSO packets without payload (pkt_len <= headlen).
  * Enforce TSO total payload length limit (pkt_len - headlen <= 262143).
  * Enforce MSS limits [88, 9728] and header length limit <= 255.

Fixes: 4022f9999f56 ("net/gve: support basic Tx data path for DQO")
Fixes: f4a70b449bfa ("net/gve: add DQO Tx descriptor limit")
Fixes: 403c671a46b6 ("net/gve: support TSO in DQO RDA")
Cc: stable@dpdk.org

Signed-off-by: Tathagat Priyadarshi <tathagat.dpdk@gmail.com>
---
 drivers/net/gve/base/gve_desc_dqo.h |  3 +
 drivers/net/gve/gve_ethdev.h        |  2 +-
 drivers/net/gve/gve_tx_dqo.c        | 92 +++++++++++++++++++++++------
 3 files changed, 79 insertions(+), 18 deletions(-)

diff --git a/drivers/net/gve/base/gve_desc_dqo.h b/drivers/net/gve/base/gve_desc_dqo.h
index c1534959c2..53e394f01c 100644
--- a/drivers/net/gve/base/gve_desc_dqo.h
+++ b/drivers/net/gve/base/gve_desc_dqo.h
@@ -12,6 +12,9 @@
 
 #define GVE_TX_MAX_HDR_SIZE_DQO 255
 #define GVE_TX_MIN_TSO_MSS_DQO 88
+#define GVE_TX_MIN_PKT_SIZE_DQO 17
+#define GVE_TX_MAX_PKT_SIZE_DQO 9728
+#define GVE_TX_MAX_TSO_LEN_DQO ((256 * 1024) - 1)
 
 /* Basic TX descriptor (DTYPE 0x0C) */
 struct gve_tx_pkt_desc_dqo {
diff --git a/drivers/net/gve/gve_ethdev.h b/drivers/net/gve/gve_ethdev.h
index 2033f49444..d6d71c31bb 100644
--- a/drivers/net/gve/gve_ethdev.h
+++ b/drivers/net/gve/gve_ethdev.h
@@ -113,7 +113,7 @@ struct gve_tx_stats {
 	uint64_t packets;
 	uint64_t bytes;
 	uint64_t errors;
-	uint64_t too_many_descs;
+	uint64_t dropped_pkts;
 };
 
 struct gve_rx_stats {
diff --git a/drivers/net/gve/gve_tx_dqo.c b/drivers/net/gve/gve_tx_dqo.c
index eee1f10df1..94ea426961 100644
--- a/drivers/net/gve/gve_tx_dqo.c
+++ b/drivers/net/gve/gve_tx_dqo.c
@@ -126,20 +126,67 @@ gve_tx_clean_descs_dqo(struct gve_tx_queue *txq, uint16_t nb_descs) {
 		gve_tx_clean_dqo(txq);
 }
 
+/* Max descriptors a single packet may use. A larger packet can never pass
+ * gve_can_tx(): one slot is always reserved, the cacheline overlap margin
+ * must stay free, and up to GVE_TX_MIN_RE_INTERVAL - 1 descriptors posted
+ * after the last report_event are not reclaimed until more are posted.
+ */
+static inline int
+gve_tx_max_pkt_descs(const struct gve_tx_queue *txq)
+{
+	return (int)txq->nb_tx_desc - 1 - GVE_TX_MIN_DESC_PREVENT_CACHE_OVERLAP -
+		(GVE_TX_MIN_RE_INTERVAL - 1);
+}
+
 /* GVE expects at most 10 data descriptors per mtu-sized segment. Beyond this,
  * the hardware will assume the driver is malicious and stop transmitting
  * packets altogether. Validate that a packet can be sent to avoid sending
  * posting descriptors for an invalid packet.
  */
 static inline bool
-gve_tx_validate_descs(struct rte_mbuf *tx_pkt, uint16_t nb_descs, bool is_tso)
+gve_tx_validate_descs(const struct gve_tx_queue *txq, struct rte_mbuf *tx_pkt,
+		      uint16_t nb_descs, bool is_tso)
 {
-	if (!is_tso)
+	int num_descs, seg_offset, mbuf_len;
+	int tso_segsz, headlen;
+
+	/* Reject packets that can never fit in the ring */
+	if (unlikely(nb_descs > gve_tx_max_pkt_descs(txq)))
+		return false;
+
+	if (!is_tso) {
+		/* Reject packets that violate hardware length constraints */
+		if (unlikely(tx_pkt->pkt_len < GVE_TX_MIN_PKT_SIZE_DQO ||
+			     tx_pkt->pkt_len > GVE_TX_MAX_PKT_SIZE_DQO))
+			return false;
 		return nb_descs <= GVE_TX_MAX_DATA_DESCS;
+	}
 
-	int tso_segsz = tx_pkt->tso_segsz;
-	int num_descs, seg_offset, mbuf_len;
-	int headlen = tx_pkt->l2_len + tx_pkt->l3_len + tx_pkt->l4_len;
+	headlen = tx_pkt->l2_len + tx_pkt->l3_len + tx_pkt->l4_len;
+
+	/* Reject missing TCP header on TSO */
+	if (unlikely(tx_pkt->l4_len == 0))
+		return false;
+
+	/* Reject packet with headers (L2-L4) not contiguous in first segment */
+	if (unlikely(tx_pkt->data_len < headlen))
+		return false;
+
+	/* Reject TSO packet without payload */
+	if (unlikely(tx_pkt->pkt_len <= (uint32_t)headlen))
+		return false;
+
+	/* Enforce GVE hardware TSO total length limit */
+	if (unlikely(tx_pkt->pkt_len - headlen > GVE_TX_MAX_TSO_LEN_DQO))
+		return false;
+
+	/* Enforce GVE hardware MSS and header limits */
+	if (unlikely(tx_pkt->tso_segsz < GVE_TX_MIN_TSO_MSS_DQO ||
+		     tx_pkt->tso_segsz > GVE_TX_MAX_PKT_SIZE_DQO ||
+		     headlen > GVE_TX_MAX_HDR_SIZE_DQO))
+		return false;
+
+	tso_segsz = tx_pkt->tso_segsz;
 
 	/* Headers will be split into their own buffer. */
 	num_descs = 1;
@@ -147,14 +194,18 @@ gve_tx_validate_descs(struct rte_mbuf *tx_pkt, uint16_t nb_descs, bool is_tso)
 	mbuf_len = tx_pkt->data_len - headlen;
 
 	while (tx_pkt) {
+		int seg_remain;
+
 		if (!mbuf_len)
 			goto next_mbuf;
 
-		int seg_remain = tso_segsz - seg_offset;
+		seg_remain = tso_segsz - seg_offset;
 		if (num_descs == GVE_TX_MAX_DATA_DESCS && seg_remain)
 			return false;
 
 		if (seg_remain < mbuf_len) {
+			int mbuf_remain = tx_pkt->data_len % GVE_TX_MAX_BUF_SIZE_DQO;
+
 			seg_offset = mbuf_len % tso_segsz;
 			/* The MSS is bound from above by 9728B, so a
 			 * single TSO segment in the middle of an mbuf
@@ -162,7 +213,6 @@ gve_tx_validate_descs(struct rte_mbuf *tx_pkt, uint16_t nb_descs, bool is_tso)
 			 * is not at risk of defying this limitation.
 			 * Thus, such segments are ignored.
 			 */
-			int mbuf_remain = tx_pkt->data_len % GVE_TX_MAX_BUF_SIZE_DQO;
 
 			/* For each TSO segment, HW will prepend
 			 * headers. The remaining bytes of this mbuf
@@ -184,7 +234,6 @@ gve_tx_validate_descs(struct rte_mbuf *tx_pkt, uint16_t nb_descs, bool is_tso)
 			mbuf_len = tx_pkt->data_len;
 	}
 
-
 	return true;
 }
 
@@ -256,6 +305,7 @@ gve_tx_burst_dqo(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
 	struct rte_mbuf *tx_pkt;
 	struct gve_tx_pkt *pkts;
 	uint32_t last_desc_idx;
+	uint16_t nb_drop = 0;
 	uint16_t nb_tx = 0;
 	uint64_t bytes = 0;
 	const char *reason;
@@ -281,9 +331,13 @@ gve_tx_burst_dqo(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
 					       txq->rs_thresh);
 
 
-		if (rte_mbuf_check(tx_pkt, true, &reason)) {
+		if (unlikely(rte_mbuf_check(tx_pkt, true, &reason))) {
 			PMD_DRV_DP_LOG(DEBUG, "Invalid mbuf: %s", reason);
-			break;
+			txq->stats.dropped_pkts++;
+			txq->stats.errors++;
+			rte_pktmbuf_free(tx_pkt);
+			nb_drop++;
+			continue;
 		}
 
 		ol_flags = tx_pkt->ol_flags;
@@ -292,6 +346,16 @@ gve_tx_burst_dqo(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
 
 		nb_descs = gve_tx_pkt_nb_data_descs(tx_pkt);
 		nb_descs += tso;
+
+		/* Drop packet if it doesn't adhere to hardware limits. */
+		if (unlikely(!gve_tx_validate_descs(txq, tx_pkt, nb_descs, tso))) {
+			txq->stats.dropped_pkts++;
+			txq->stats.errors++;
+			rte_pktmbuf_free(tx_pkt);
+			nb_drop++;
+			continue;
+		}
+
 		req_descs = nb_descs + GVE_TX_MIN_DESC_PREVENT_CACHE_OVERLAP;
 
 		/* Clean if there aren't enough descriptors to send the packet
@@ -306,12 +370,6 @@ gve_tx_burst_dqo(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
 				break;
 		}
 
-		/* Drop packet if it doesn't adhere to hardware limits. */
-		if (!gve_tx_validate_descs(tx_pkt, nb_descs, tso)) {
-			txq->stats.too_many_descs++;
-			break;
-		}
-
 		if (!gve_free_compl_tags_pop(txq, &compl_tag))
 			break;
 
@@ -357,7 +415,7 @@ gve_tx_burst_dqo(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
 	}
 
 	if (nb_tx > 0) {
-		txq->stats.packets += nb_tx;
+		txq->stats.packets += (nb_tx - nb_drop);
 		txq->stats.bytes += bytes;
 		txq->stats.errors += nb_pkts - nb_tx;
 	}
-- 
2.43.0

Re: [PATCH v2 4/4] net/gve: add packet and TSO hardware constraints for DQO

[Joshua Washington]

On Tue, Oct 13, 2026 at 3:44 AM Tathagat Priyadarshi
<tathagat.dpdk@gmail.com> wrote:
>
> The GVNIC DQO hardware restricts each packet (or MTU segment in TSO)
> to at most 10 data descriptors (GVE_TX_MAX_DATA_DESCS), beyond which
> the hardware drops traffic. In addition to this 10-descriptor limit,
> the hardware imposes strict packet sizing and header constraints.
>
> Previously, failing the descriptor limit or rte_mbuf_check() broke
> from the Tx burst loop without consuming the invalid mbuf. This caused
> tx_burst to return a short count, leading the application to retry the
> same rejected packet indefinitely and causing Head-of-Line (HOL)
> queue stalls.
>
> Enforce hardware validation and prevent HOL blocking in DPDK:
> - Validate packet and hardware constraints prior to ring space checks.
> - When packet validation or rte_mbuf_check() fails, drop and free the
>   invalid mbuf with rte_pktmbuf_free(), update stats, and continue the
>   burst to prevent Head-of-Line (HOL) blocking.
> - Exclude dropped packets from txq->stats.packets while returning the
>   total consumed count to the caller.
> - Rename too_many_descs in struct gve_tx_stats to dropped_pkts to reflect
>   all validation and malformed mbuf drop events.
> - Reject packets whose descriptor count exceeds the maximum capacity of
>   the ring (accounting for reserved slots, cacheline overlap margin, and
>   unreported descriptors), preventing permanent queue stalls.
> - Define GVE_TX_MIN_PKT_SIZE_DQO (17 bytes) and GVE_TX_MAX_PKT_SIZE_DQO
>   (9728 bytes) to enforce wire packet boundaries for non-TSO packets
>   and TSO MSS.
> - Define GVE_TX_MAX_TSO_LEN_DQO (262143 bytes) to enforce the hardware
>   L4 payload limit in the 24-bit tso_total_len field.
> - In gve_tx_validate_descs():
>   * Enforce packet size in [17, 9728] for non-TSO packets alongside
>     the 10-descriptor limit (nb_descs <= GVE_TX_MAX_DATA_DESCS).
>   * Reject TSO packets missing an L4 header (l4_len == 0).
>   * Reject packet with headers (L2-L4) not contiguous in first segment.
>   * Reject TSO packets without payload (pkt_len <= headlen).
>   * Enforce TSO total payload length limit (pkt_len - headlen <= 262143).
>   * Enforce MSS limits [88, 9728] and header length limit <= 255.

I agree with the AI code review on this patch description.
> 4. The commit message lists the diff line by line. Cut it to the
> problem (invalid packets stalled the queue; hardware limits were
> not enforced) and the fix (drop and continue; the limits enforced).

Can we trim this message down a bit, and more succinctly explain the
error condition and how this patch fixes it? If it cannot be describe
succinctly, that might be a sign that the patch should be split up
into smaller logical pieces.

>
> Fixes: 4022f9999f56 ("net/gve: support basic Tx data path for DQO")
> Fixes: f4a70b449bfa ("net/gve: add DQO Tx descriptor limit")
> Fixes: 403c671a46b6 ("net/gve: support TSO in DQO RDA")
> Cc: stable@dpdk.org
>
> Signed-off-by: Tathagat Priyadarshi <tathagat.dpdk@gmail.com>
> ---
>  drivers/net/gve/base/gve_desc_dqo.h |  3 +
>  drivers/net/gve/gve_ethdev.h        |  2 +-
>  drivers/net/gve/gve_tx_dqo.c        | 92 +++++++++++++++++++++++------
>  3 files changed, 79 insertions(+), 18 deletions(-)
>
> diff --git a/drivers/net/gve/base/gve_desc_dqo.h b/drivers/net/gve/base/gve_desc_dqo.h
> index c1534959c2..53e394f01c 100644
> --- a/drivers/net/gve/base/gve_desc_dqo.h
> +++ b/drivers/net/gve/base/gve_desc_dqo.h
> @@ -12,6 +12,9 @@
>
>  #define GVE_TX_MAX_HDR_SIZE_DQO 255
>  #define GVE_TX_MIN_TSO_MSS_DQO 88
> +#define GVE_TX_MIN_PKT_SIZE_DQO 17
> +#define GVE_TX_MAX_PKT_SIZE_DQO 9728
> +#define GVE_TX_MAX_TSO_LEN_DQO ((256 * 1024) - 1)
>
>  /* Basic TX descriptor (DTYPE 0x0C) */
>  struct gve_tx_pkt_desc_dqo {
> diff --git a/drivers/net/gve/gve_ethdev.h b/drivers/net/gve/gve_ethdev.h
> index 2033f49444..d6d71c31bb 100644
> --- a/drivers/net/gve/gve_ethdev.h
> +++ b/drivers/net/gve/gve_ethdev.h
> @@ -113,7 +113,7 @@ struct gve_tx_stats {
>         uint64_t packets;
>         uint64_t bytes;
>         uint64_t errors;
> -       uint64_t too_many_descs;
> +       uint64_t dropped_pkts;

Is there any reason to change this to a general "dropped_pkts" instead
of adding a different stat if there is a different error condition
that needs to be captured? These are extended stats, so if we can give
more specific drop conditions, I think we should aim to. However,
there is nothing wrong with also introducing a new aggregate
dropped_pkts stat. Not sure if adding new stats would be considered
scope creep, though.

>  };
>
>  struct gve_rx_stats {
> diff --git a/drivers/net/gve/gve_tx_dqo.c b/drivers/net/gve/gve_tx_dqo.c
> index eee1f10df1..94ea426961 100644
> --- a/drivers/net/gve/gve_tx_dqo.c
> +++ b/drivers/net/gve/gve_tx_dqo.c
> @@ -126,20 +126,67 @@ gve_tx_clean_descs_dqo(struct gve_tx_queue *txq, uint16_t nb_descs) {
>                 gve_tx_clean_dqo(txq);
>  }
>
> +/* Max descriptors a single packet may use. A larger packet can never pass
> + * gve_can_tx(): one slot is always reserved, the cacheline overlap margin
> + * must stay free, and up to GVE_TX_MIN_RE_INTERVAL - 1 descriptors posted
> + * after the last report_event are not reclaimed until more are posted.
> + */
> +static inline int
> +gve_tx_max_pkt_descs(const struct gve_tx_queue *txq)
> +{
> +       return (int)txq->nb_tx_desc - 1 - GVE_TX_MIN_DESC_PREVENT_CACHE_OVERLAP -
> +               (GVE_TX_MIN_RE_INTERVAL - 1);
> +}
> +
>  /* GVE expects at most 10 data descriptors per mtu-sized segment. Beyond this,
>   * the hardware will assume the driver is malicious and stop transmitting
>   * packets altogether. Validate that a packet can be sent to avoid sending
>   * posting descriptors for an invalid packet.
>   */
>  static inline bool
> -gve_tx_validate_descs(struct rte_mbuf *tx_pkt, uint16_t nb_descs, bool is_tso)
> +gve_tx_validate_descs(const struct gve_tx_queue *txq, struct rte_mbuf *tx_pkt,
> +                     uint16_t nb_descs, bool is_tso)
>  {
> -       if (!is_tso)
> +       int num_descs, seg_offset, mbuf_len;
> +       int tso_segsz, headlen;
> +
> +       /* Reject packets that can never fit in the ring */
> +       if (unlikely(nb_descs > gve_tx_max_pkt_descs(txq)))
> +               return false;
> +
> +       if (!is_tso) {
> +               /* Reject packets that violate hardware length constraints */
> +               if (unlikely(tx_pkt->pkt_len < GVE_TX_MIN_PKT_SIZE_DQO ||
> +                            tx_pkt->pkt_len > GVE_TX_MAX_PKT_SIZE_DQO))
> +                       return false;
>                 return nb_descs <= GVE_TX_MAX_DATA_DESCS;
> +       }
>
> -       int tso_segsz = tx_pkt->tso_segsz;
> -       int num_descs, seg_offset, mbuf_len;
> -       int headlen = tx_pkt->l2_len + tx_pkt->l3_len + tx_pkt->l4_len;
> +       headlen = tx_pkt->l2_len + tx_pkt->l3_len + tx_pkt->l4_len;
> +
> +       /* Reject missing TCP header on TSO */
> +       if (unlikely(tx_pkt->l4_len == 0))
> +               return false;
> +
> +       /* Reject packet with headers (L2-L4) not contiguous in first segment */
> +       if (unlikely(tx_pkt->data_len < headlen))
> +               return false;
> +
> +       /* Reject TSO packet without payload */
> +       if (unlikely(tx_pkt->pkt_len <= (uint32_t)headlen))
> +               return false;
> +
> +       /* Enforce GVE hardware TSO total length limit */
> +       if (unlikely(tx_pkt->pkt_len - headlen > GVE_TX_MAX_TSO_LEN_DQO))
> +               return false;
> +
> +       /* Enforce GVE hardware MSS and header limits */
> +       if (unlikely(tx_pkt->tso_segsz < GVE_TX_MIN_TSO_MSS_DQO ||
> +                    tx_pkt->tso_segsz > GVE_TX_MAX_PKT_SIZE_DQO ||
> +                    headlen > GVE_TX_MAX_HDR_SIZE_DQO))
> +               return false;
> +
> +       tso_segsz = tx_pkt->tso_segsz;
>
>         /* Headers will be split into their own buffer. */
>         num_descs = 1;
> @@ -147,14 +194,18 @@ gve_tx_validate_descs(struct rte_mbuf *tx_pkt, uint16_t nb_descs, bool is_tso)
>         mbuf_len = tx_pkt->data_len - headlen;
>
>         while (tx_pkt) {
> +               int seg_remain;
> +
>                 if (!mbuf_len)
>                         goto next_mbuf;
>
> -               int seg_remain = tso_segsz - seg_offset;
> +               seg_remain = tso_segsz - seg_offset;
>                 if (num_descs == GVE_TX_MAX_DATA_DESCS && seg_remain)
>                         return false;
>
>                 if (seg_remain < mbuf_len) {
> +                       int mbuf_remain = tx_pkt->data_len % GVE_TX_MAX_BUF_SIZE_DQO;
> +
>                         seg_offset = mbuf_len % tso_segsz;
>                         /* The MSS is bound from above by 9728B, so a
>                          * single TSO segment in the middle of an mbuf
> @@ -162,7 +213,6 @@ gve_tx_validate_descs(struct rte_mbuf *tx_pkt, uint16_t nb_descs, bool is_tso)
>                          * is not at risk of defying this limitation.
>                          * Thus, such segments are ignored.
>                          */
> -                       int mbuf_remain = tx_pkt->data_len % GVE_TX_MAX_BUF_SIZE_DQO;
>
>                         /* For each TSO segment, HW will prepend
>                          * headers. The remaining bytes of this mbuf
> @@ -184,7 +234,6 @@ gve_tx_validate_descs(struct rte_mbuf *tx_pkt, uint16_t nb_descs, bool is_tso)
>                         mbuf_len = tx_pkt->data_len;
>         }
>
> -
>         return true;
>  }
>
> @@ -256,6 +305,7 @@ gve_tx_burst_dqo(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
>         struct rte_mbuf *tx_pkt;
>         struct gve_tx_pkt *pkts;
>         uint32_t last_desc_idx;
> +       uint16_t nb_drop = 0;

Minor nit: maybe we can capture the number of packet sent after we
write the descriptors instead? Then we can increment in one place
instead of two, and remove the math in getting the number of packets
sent.

>         uint16_t nb_tx = 0;
>         uint64_t bytes = 0;
>         const char *reason;
> @@ -281,9 +331,13 @@ gve_tx_burst_dqo(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
>                                                txq->rs_thresh);
>
>
> -               if (rte_mbuf_check(tx_pkt, true, &reason)) {
> +               if (unlikely(rte_mbuf_check(tx_pkt, true, &reason))) {
>                         PMD_DRV_DP_LOG(DEBUG, "Invalid mbuf: %s", reason);
> -                       break;
> +                       txq->stats.dropped_pkts++;
> +                       txq->stats.errors++;
> +                       rte_pktmbuf_free(tx_pkt);
> +                       nb_drop++;
> +                       continue;
>                 }
>
>                 ol_flags = tx_pkt->ol_flags;
> @@ -292,6 +346,16 @@ gve_tx_burst_dqo(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
>
>                 nb_descs = gve_tx_pkt_nb_data_descs(tx_pkt);
>                 nb_descs += tso;
> +
> +               /* Drop packet if it doesn't adhere to hardware limits. */
> +               if (unlikely(!gve_tx_validate_descs(txq, tx_pkt, nb_descs, tso))) {
> +                       txq->stats.dropped_pkts++;
> +                       txq->stats.errors++;
> +                       rte_pktmbuf_free(tx_pkt);
> +                       nb_drop++;
> +                       continue;
> +               }
> +
>                 req_descs = nb_descs + GVE_TX_MIN_DESC_PREVENT_CACHE_OVERLAP;
>
>                 /* Clean if there aren't enough descriptors to send the packet
> @@ -306,12 +370,6 @@ gve_tx_burst_dqo(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
>                                 break;
>                 }
>
> -               /* Drop packet if it doesn't adhere to hardware limits. */
> -               if (!gve_tx_validate_descs(tx_pkt, nb_descs, tso)) {
> -                       txq->stats.too_many_descs++;
> -                       break;
> -               }
> -
>                 if (!gve_free_compl_tags_pop(txq, &compl_tag))
>                         break;
>
> @@ -357,7 +415,7 @@ gve_tx_burst_dqo(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
>         }
>
>         if (nb_tx > 0) {
> -               txq->stats.packets += nb_tx;
> +               txq->stats.packets += (nb_tx - nb_drop);
>                 txq->stats.bytes += bytes;
>                 txq->stats.errors += nb_pkts - nb_tx;
>         }
> --
> 2.43.0
>

Re: [PATCH v2 4/4] net/gve: add packet and TSO hardware constraints for DQO

[Tathagat Priyadarshi]

Thanks Joshua,

I'll get this sorted out. In trying to incorporate all the feedback
from you, Stephen, and AI suggestions at once, I ended up losing focus
on the main scope.
I'm going to break this work down into smaller, focused commits so
it’s much easier to review and test.

On Wed, Oct 14, 2026 at 6:20 AM Joshua Washington <joshwash@google.com> wrote:
>
> On Tue, Oct 13, 2026 at 3:44 AM Tathagat Priyadarshi
> <tathagat.dpdk@gmail.com> wrote:
> >
> > The GVNIC DQO hardware restricts each packet (or MTU segment in TSO)
> > to at most 10 data descriptors (GVE_TX_MAX_DATA_DESCS), beyond which
> > the hardware drops traffic. In addition to this 10-descriptor limit,
> > the hardware imposes strict packet sizing and header constraints.
> >
> > Previously, failing the descriptor limit or rte_mbuf_check() broke
> > from the Tx burst loop without consuming the invalid mbuf. This caused
> > tx_burst to return a short count, leading the application to retry the
> > same rejected packet indefinitely and causing Head-of-Line (HOL)
> > queue stalls.
> >
> > Enforce hardware validation and prevent HOL blocking in DPDK:
> > - Validate packet and hardware constraints prior to ring space checks.
> > - When packet validation or rte_mbuf_check() fails, drop and free the
> >   invalid mbuf with rte_pktmbuf_free(), update stats, and continue the
> >   burst to prevent Head-of-Line (HOL) blocking.
> > - Exclude dropped packets from txq->stats.packets while returning the
> >   total consumed count to the caller.
> > - Rename too_many_descs in struct gve_tx_stats to dropped_pkts to reflect
> >   all validation and malformed mbuf drop events.
> > - Reject packets whose descriptor count exceeds the maximum capacity of
> >   the ring (accounting for reserved slots, cacheline overlap margin, and
> >   unreported descriptors), preventing permanent queue stalls.
> > - Define GVE_TX_MIN_PKT_SIZE_DQO (17 bytes) and GVE_TX_MAX_PKT_SIZE_DQO
> >   (9728 bytes) to enforce wire packet boundaries for non-TSO packets
> >   and TSO MSS.
> > - Define GVE_TX_MAX_TSO_LEN_DQO (262143 bytes) to enforce the hardware
> >   L4 payload limit in the 24-bit tso_total_len field.
> > - In gve_tx_validate_descs():
> >   * Enforce packet size in [17, 9728] for non-TSO packets alongside
> >     the 10-descriptor limit (nb_descs <= GVE_TX_MAX_DATA_DESCS).
> >   * Reject TSO packets missing an L4 header (l4_len == 0).
> >   * Reject packet with headers (L2-L4) not contiguous in first segment.
> >   * Reject TSO packets without payload (pkt_len <= headlen).
> >   * Enforce TSO total payload length limit (pkt_len - headlen <= 262143).
> >   * Enforce MSS limits [88, 9728] and header length limit <= 255.
>
> I agree with the AI code review on this patch description.
> > 4. The commit message lists the diff line by line. Cut it to the
> > problem (invalid packets stalled the queue; hardware limits were
> > not enforced) and the fix (drop and continue; the limits enforced).
>
> Can we trim this message down a bit, and more succinctly explain the
> error condition and how this patch fixes it? If it cannot be describe
> succinctly, that might be a sign that the patch should be split up
> into smaller logical pieces.
>
> >
> > Fixes: 4022f9999f56 ("net/gve: support basic Tx data path for DQO")
> > Fixes: f4a70b449bfa ("net/gve: add DQO Tx descriptor limit")
> > Fixes: 403c671a46b6 ("net/gve: support TSO in DQO RDA")
> > Cc: stable@dpdk.org
> >
> > Signed-off-by: Tathagat Priyadarshi <tathagat.dpdk@gmail.com>
> > ---
> >  drivers/net/gve/base/gve_desc_dqo.h |  3 +
> >  drivers/net/gve/gve_ethdev.h        |  2 +-
> >  drivers/net/gve/gve_tx_dqo.c        | 92 +++++++++++++++++++++++------
> >  3 files changed, 79 insertions(+), 18 deletions(-)
> >
> > diff --git a/drivers/net/gve/base/gve_desc_dqo.h b/drivers/net/gve/base/gve_desc_dqo.h
> > index c1534959c2..53e394f01c 100644
> > --- a/drivers/net/gve/base/gve_desc_dqo.h
> > +++ b/drivers/net/gve/base/gve_desc_dqo.h
> > @@ -12,6 +12,9 @@
> >
> >  #define GVE_TX_MAX_HDR_SIZE_DQO 255
> >  #define GVE_TX_MIN_TSO_MSS_DQO 88
> > +#define GVE_TX_MIN_PKT_SIZE_DQO 17
> > +#define GVE_TX_MAX_PKT_SIZE_DQO 9728
> > +#define GVE_TX_MAX_TSO_LEN_DQO ((256 * 1024) - 1)
> >
> >  /* Basic TX descriptor (DTYPE 0x0C) */
> >  struct gve_tx_pkt_desc_dqo {
> > diff --git a/drivers/net/gve/gve_ethdev.h b/drivers/net/gve/gve_ethdev.h
> > index 2033f49444..d6d71c31bb 100644
> > --- a/drivers/net/gve/gve_ethdev.h
> > +++ b/drivers/net/gve/gve_ethdev.h
> > @@ -113,7 +113,7 @@ struct gve_tx_stats {
> >         uint64_t packets;
> >         uint64_t bytes;
> >         uint64_t errors;
> > -       uint64_t too_many_descs;
> > +       uint64_t dropped_pkts;
>
> Is there any reason to change this to a general "dropped_pkts" instead
> of adding a different stat if there is a different error condition
> that needs to be captured? These are extended stats, so if we can give
> more specific drop conditions, I think we should aim to. However,
> there is nothing wrong with also introducing a new aggregate
> dropped_pkts stat. Not sure if adding new stats would be considered
> scope creep, though.
>
> >  };
> >
> >  struct gve_rx_stats {
> > diff --git a/drivers/net/gve/gve_tx_dqo.c b/drivers/net/gve/gve_tx_dqo.c
> > index eee1f10df1..94ea426961 100644
> > --- a/drivers/net/gve/gve_tx_dqo.c
> > +++ b/drivers/net/gve/gve_tx_dqo.c
> > @@ -126,20 +126,67 @@ gve_tx_clean_descs_dqo(struct gve_tx_queue *txq, uint16_t nb_descs) {
> >                 gve_tx_clean_dqo(txq);
> >  }
> >
> > +/* Max descriptors a single packet may use. A larger packet can never pass
> > + * gve_can_tx(): one slot is always reserved, the cacheline overlap margin
> > + * must stay free, and up to GVE_TX_MIN_RE_INTERVAL - 1 descriptors posted
> > + * after the last report_event are not reclaimed until more are posted.
> > + */
> > +static inline int
> > +gve_tx_max_pkt_descs(const struct gve_tx_queue *txq)
> > +{
> > +       return (int)txq->nb_tx_desc - 1 - GVE_TX_MIN_DESC_PREVENT_CACHE_OVERLAP -
> > +               (GVE_TX_MIN_RE_INTERVAL - 1);
> > +}
> > +
> >  /* GVE expects at most 10 data descriptors per mtu-sized segment. Beyond this,
> >   * the hardware will assume the driver is malicious and stop transmitting
> >   * packets altogether. Validate that a packet can be sent to avoid sending
> >   * posting descriptors for an invalid packet.
> >   */
> >  static inline bool
> > -gve_tx_validate_descs(struct rte_mbuf *tx_pkt, uint16_t nb_descs, bool is_tso)
> > +gve_tx_validate_descs(const struct gve_tx_queue *txq, struct rte_mbuf *tx_pkt,
> > +                     uint16_t nb_descs, bool is_tso)
> >  {
> > -       if (!is_tso)
> > +       int num_descs, seg_offset, mbuf_len;
> > +       int tso_segsz, headlen;
> > +
> > +       /* Reject packets that can never fit in the ring */
> > +       if (unlikely(nb_descs > gve_tx_max_pkt_descs(txq)))
> > +               return false;
> > +
> > +       if (!is_tso) {
> > +               /* Reject packets that violate hardware length constraints */
> > +               if (unlikely(tx_pkt->pkt_len < GVE_TX_MIN_PKT_SIZE_DQO ||
> > +                            tx_pkt->pkt_len > GVE_TX_MAX_PKT_SIZE_DQO))
> > +                       return false;
> >                 return nb_descs <= GVE_TX_MAX_DATA_DESCS;
> > +       }
> >
> > -       int tso_segsz = tx_pkt->tso_segsz;
> > -       int num_descs, seg_offset, mbuf_len;
> > -       int headlen = tx_pkt->l2_len + tx_pkt->l3_len + tx_pkt->l4_len;
> > +       headlen = tx_pkt->l2_len + tx_pkt->l3_len + tx_pkt->l4_len;
> > +
> > +       /* Reject missing TCP header on TSO */
> > +       if (unlikely(tx_pkt->l4_len == 0))
> > +               return false;
> > +
> > +       /* Reject packet with headers (L2-L4) not contiguous in first segment */
> > +       if (unlikely(tx_pkt->data_len < headlen))
> > +               return false;
> > +
> > +       /* Reject TSO packet without payload */
> > +       if (unlikely(tx_pkt->pkt_len <= (uint32_t)headlen))
> > +               return false;
> > +
> > +       /* Enforce GVE hardware TSO total length limit */
> > +       if (unlikely(tx_pkt->pkt_len - headlen > GVE_TX_MAX_TSO_LEN_DQO))
> > +               return false;
> > +
> > +       /* Enforce GVE hardware MSS and header limits */
> > +       if (unlikely(tx_pkt->tso_segsz < GVE_TX_MIN_TSO_MSS_DQO ||
> > +                    tx_pkt->tso_segsz > GVE_TX_MAX_PKT_SIZE_DQO ||
> > +                    headlen > GVE_TX_MAX_HDR_SIZE_DQO))
> > +               return false;
> > +
> > +       tso_segsz = tx_pkt->tso_segsz;
> >
> >         /* Headers will be split into their own buffer. */
> >         num_descs = 1;
> > @@ -147,14 +194,18 @@ gve_tx_validate_descs(struct rte_mbuf *tx_pkt, uint16_t nb_descs, bool is_tso)
> >         mbuf_len = tx_pkt->data_len - headlen;
> >
> >         while (tx_pkt) {
> > +               int seg_remain;
> > +
> >                 if (!mbuf_len)
> >                         goto next_mbuf;
> >
> > -               int seg_remain = tso_segsz - seg_offset;
> > +               seg_remain = tso_segsz - seg_offset;
> >                 if (num_descs == GVE_TX_MAX_DATA_DESCS && seg_remain)
> >                         return false;
> >
> >                 if (seg_remain < mbuf_len) {
> > +                       int mbuf_remain = tx_pkt->data_len % GVE_TX_MAX_BUF_SIZE_DQO;
> > +
> >                         seg_offset = mbuf_len % tso_segsz;
> >                         /* The MSS is bound from above by 9728B, so a
> >                          * single TSO segment in the middle of an mbuf
> > @@ -162,7 +213,6 @@ gve_tx_validate_descs(struct rte_mbuf *tx_pkt, uint16_t nb_descs, bool is_tso)
> >                          * is not at risk of defying this limitation.
> >                          * Thus, such segments are ignored.
> >                          */
> > -                       int mbuf_remain = tx_pkt->data_len % GVE_TX_MAX_BUF_SIZE_DQO;
> >
> >                         /* For each TSO segment, HW will prepend
> >                          * headers. The remaining bytes of this mbuf
> > @@ -184,7 +234,6 @@ gve_tx_validate_descs(struct rte_mbuf *tx_pkt, uint16_t nb_descs, bool is_tso)
> >                         mbuf_len = tx_pkt->data_len;
> >         }
> >
> > -
> >         return true;
> >  }
> >
> > @@ -256,6 +305,7 @@ gve_tx_burst_dqo(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
> >         struct rte_mbuf *tx_pkt;
> >         struct gve_tx_pkt *pkts;
> >         uint32_t last_desc_idx;
> > +       uint16_t nb_drop = 0;
>
> Minor nit: maybe we can capture the number of packet sent after we
> write the descriptors instead? Then we can increment in one place
> instead of two, and remove the math in getting the number of packets
> sent.
>
> >         uint16_t nb_tx = 0;
> >         uint64_t bytes = 0;
> >         const char *reason;
> > @@ -281,9 +331,13 @@ gve_tx_burst_dqo(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
> >                                                txq->rs_thresh);
> >
> >
> > -               if (rte_mbuf_check(tx_pkt, true, &reason)) {
> > +               if (unlikely(rte_mbuf_check(tx_pkt, true, &reason))) {
> >                         PMD_DRV_DP_LOG(DEBUG, "Invalid mbuf: %s", reason);
> > -                       break;
> > +                       txq->stats.dropped_pkts++;
> > +                       txq->stats.errors++;
> > +                       rte_pktmbuf_free(tx_pkt);
> > +                       nb_drop++;
> > +                       continue;
> >                 }
> >
> >                 ol_flags = tx_pkt->ol_flags;
> > @@ -292,6 +346,16 @@ gve_tx_burst_dqo(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
> >
> >                 nb_descs = gve_tx_pkt_nb_data_descs(tx_pkt);
> >                 nb_descs += tso;
> > +
> > +               /* Drop packet if it doesn't adhere to hardware limits. */
> > +               if (unlikely(!gve_tx_validate_descs(txq, tx_pkt, nb_descs, tso))) {
> > +                       txq->stats.dropped_pkts++;
> > +                       txq->stats.errors++;
> > +                       rte_pktmbuf_free(tx_pkt);
> > +                       nb_drop++;
> > +                       continue;
> > +               }
> > +
> >                 req_descs = nb_descs + GVE_TX_MIN_DESC_PREVENT_CACHE_OVERLAP;
> >
> >                 /* Clean if there aren't enough descriptors to send the packet
> > @@ -306,12 +370,6 @@ gve_tx_burst_dqo(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
> >                                 break;
> >                 }
> >
> > -               /* Drop packet if it doesn't adhere to hardware limits. */
> > -               if (!gve_tx_validate_descs(tx_pkt, nb_descs, tso)) {
> > -                       txq->stats.too_many_descs++;
> > -                       break;
> > -               }
> > -
> >                 if (!gve_free_compl_tags_pop(txq, &compl_tag))
> >                         break;
> >
> > @@ -357,7 +415,7 @@ gve_tx_burst_dqo(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
> >         }
> >
> >         if (nb_tx > 0) {
> > -               txq->stats.packets += nb_tx;
> > +               txq->stats.packets += (nb_tx - nb_drop);
> >                 txq->stats.bytes += bytes;
> >                 txq->stats.errors += nb_pkts - nb_tx;
> >         }
> > --
> > 2.43.0
> >

Re: [PATCH v2 0/4] net/gve: fix Tx queue stalls and sync DQO RDA with Linux driver

[Stephen Hemminger]

On Tue, 13 Oct 2026 10:43:44 +0000
Tathagat Priyadarshi <tathagat.dpdk@gmail.com> wrote:

> This patch series aligns the Google Virtual NIC (gve) DQO RDA transmit path
> with the upstream Linux kernel driver (drivers/net/ethernet/google/gve/) to
> resolve transmit queue stalls and hardware validation issues:
> 
> 1. net/gve: fix DQO descriptor cleaning loop bound
>    Fixes the off-by-one error in gve_tx_clean_descs_dqo()
>    ("while (--nb_descs)" -> "while (nb_descs--)") and avoids stalling
>    the completion queue on unrecognized descriptor types.
> 
> 2. net/gve: fix DQO Tx queue stall on multi-seg and TSO
>    Tracks descriptor intervals with last_re_idx per packet against
>    GVE_TX_MIN_RE_INTERVAL (32) instead of counting packets at burst end,
>    preventing descriptor ring starvation and queue deadlocks under
>    multi-segment and TSO traffic.
> 
> 3. net/gve: add cacheline overlap margin to DQO Tx ring
>    Enforces GVE_TX_MIN_DESC_PREVENT_CACHE_OVERLAP (4 descriptors) margin
>    in ring space checks, avoiding CPU and device DMA cacheline collisions.
> 
> 4. net/gve: add packet and TSO hardware constraints for DQO
>    Augments gve_tx_validate_descs() (which enforces the 10-descriptor
>    limit per packet or MTU segment) with the remaining hardware length
>    constraints: [17, 9728] bytes for non-TSO packets, maximum MSS limit
>    (9728 bytes), TSO total length limit (262143 bytes), and validates
>    header contiguity and payload presence on TSO packets. Invalid packets
>    are dropped instead of stalling the queue.


Looks good you may want to address these comments from AI.

Subject: Re: [PATCH v2 0/4] net/gve: DQO Tx fixes

Applies cleanly to main; every commit builds with -Dwerror=true.
v2 drops invalid packets and continues instead of stalling the burst,
pops the completion tag before writing the TSO context descriptor,
and adds the Fixes tags. All three Fixes hashes resolve.

Info

[PATCH v2 2/4] net/gve: fix DQO Tx queue stall on multi-seg and TSO

1. Ringing the doorbell each time report_event is set is a
   performance change, not part of the stall fix. Descriptors posted
   earlier in the burst are reclaimed on the next burst either way,
   and Linux rings once per xmit batch. It is harmless, but for a
   stable backport it is better as a separate patch with numbers. At
   least drop "amortizing MMIO writes": it adds MMIO writes.

[PATCH v2 4/4] net/gve: add packet and TSO hardware constraints for DQO

2. tso_total_len is 24 bits wide; 262143 is the hardware limit (see
   the comment in the Linux gve_prep_tso()), not the field width.
   Reword: "Define GVE_TX_MAX_TSO_LEN_DQO (262143), the hardware limit
   on TSO payload length."

3. dropped_pkts is not in tx_xstats_name_offset[], so nothing reports
   it; too_many_descs had the same problem. Export it:

       { "dropped_pkts", TX_QUEUE_STATS_OFFSET(dropped_pkts) },

   or drop the counter and rely on errors.

4. The commit message lists the diff line by line. Cut it to the
   problem (invalid packets stalled the queue; hardware limits were
   not enforced) and the fix (drop and continue; the limits enforced).

Reviewed-by: Stephen Hemminger <stephen@networkplumber.org>

Re: [PATCH v2 0/4] net/gve: fix Tx queue stalls and sync DQO RDA with Linux driver

[Stephen Hemminger]

On Tue, 13 Oct 2026 10:43:44 +0000
Tathagat Priyadarshi <tathagat.dpdk@gmail.com> wrote:

> This patch series aligns the Google Virtual NIC (gve) DQO RDA transmit path
> with the upstream Linux kernel driver (drivers/net/ethernet/google/gve/) to
> resolve transmit queue stalls and hardware validation issues:
> 
> 1. net/gve: fix DQO descriptor cleaning loop bound
>    Fixes the off-by-one error in gve_tx_clean_descs_dqo()
>    ("while (--nb_descs)" -> "while (nb_descs--)") and avoids stalling
>    the completion queue on unrecognized descriptor types.
> 
> 2. net/gve: fix DQO Tx queue stall on multi-seg and TSO
>    Tracks descriptor intervals with last_re_idx per packet against
>    GVE_TX_MIN_RE_INTERVAL (32) instead of counting packets at burst end,
>    preventing descriptor ring starvation and queue deadlocks under
>    multi-segment and TSO traffic.
> 
> 3. net/gve: add cacheline overlap margin to DQO Tx ring
>    Enforces GVE_TX_MIN_DESC_PREVENT_CACHE_OVERLAP (4 descriptors) margin
>    in ring space checks, avoiding CPU and device DMA cacheline collisions.
> 
> 4. net/gve: add packet and TSO hardware constraints for DQO
>    Augments gve_tx_validate_descs() (which enforces the 10-descriptor
>    limit per packet or MTU segment) with the remaining hardware length
>    constraints: [17, 9728] bytes for non-TSO packets, maximum MSS limit
>    (9728 bytes), TSO total length limit (262143 bytes), and validates
>    header contiguity and payload presence on TSO packets. Invalid packets
>    are dropped instead of stalling the queue.
> 
> v2:
> - Patch 1/4, 3/4: No changes.

Don't know if this got sent yesterday.
Here was AI review of this

Subject: Re: [PATCH v2 0/4] net/gve: DQO Tx fixes

Applies cleanly to main; every commit builds with -Dwerror=true.
v2 drops invalid packets and continues instead of stalling the burst,
pops the completion tag before writing the TSO context descriptor,
and adds the Fixes tags. All three Fixes hashes resolve.

Info

[PATCH v2 2/4] net/gve: fix DQO Tx queue stall on multi-seg and TSO

1. Ringing the doorbell each time report_event is set is a
   performance change, not part of the stall fix. Descriptors posted
   earlier in the burst are reclaimed on the next burst either way,
   and Linux rings once per xmit batch. It is harmless, but for a
   stable backport it is better as a separate patch with numbers. At
   least drop "amortizing MMIO writes": it adds MMIO writes.

[PATCH v2 4/4] net/gve: add packet and TSO hardware constraints for DQO

2. tso_total_len is 24 bits wide; 262143 is the hardware limit (see
   the comment in the Linux gve_prep_tso()), not the field width.
   Reword: "Define GVE_TX_MAX_TSO_LEN_DQO (262143), the hardware limit
   on TSO payload length."

3. dropped_pkts is not in tx_xstats_name_offset[], so nothing reports
   it; too_many_descs had the same problem. Export it:

       { "dropped_pkts", TX_QUEUE_STATS_OFFSET(dropped_pkts) },

   or drop the counter and rely on errors.

4. The commit message lists the diff line by line. Cut it to the
   problem (invalid packets stalled the queue; hardware limits were
   not enforced) and the fix (drop and continue; the limits enforced).

[PATCH v3 00/10] net/gve: fix DQO Tx queue stalls

[Tathagat Priyadarshi]

This series fixes several ways the gve DQO RDA Tx path can stall or
post invalid descriptors: completion handling bugs (1-2), a report
event interval that can starve the ring (3), a stray TSO context
descriptor when completion tags run out (4), a missing cacheline
margin (5), invalid packets blocking the burst (6-8), and missing
device length checks (9-10).

v3:
- Split the series into one fix per patch, per review feedback.
  The code is the same as v2 except where noted below.
- 1/10, 2/10: v2 1/4, split in two. Kept Joshua's Ack.
- 3/10: v2 2/4 without the declaration reordering
  (per Joshua Washington) and without ringing the doorbell on each
  report event, which is a performance change to be sent separately
  with numbers (per Stephen Hemminger).
- 4/10: the completion tag reorder from v2 2/4, as its own patch.
  Fixes tag now points at the commit that made the tag pop fallible.
- 5/10: v2 3/4, unchanged. Kept Joshua's Ack.
- 6/10-10/10: v2 4/4, split:
  * Keep too_many_descs and add per-reason invalid_mbufs and
    invalid_pkts counters instead of a generic dropped_pkts
    (per Joshua Washington), and export them as xstats
    (per Stephen Hemminger).
  * Count sent packets after writing descriptors instead of
    subtracting drops (per Joshua Washington).
  * Move the length and TSO header checks into gve_tx_validate_pkt(),
    leaving gve_tx_validate_descs() to descriptor counts, without
    the declaration moves.
  * Reword the TSO length limit (per Stephen Hemminger) and shorten
    the commit messages.
- Stephen's Reviewed-by on v2 is not carried over because of the
  rework.

Tathagat Priyadarshi (10):
  net/gve: fix DQO descriptor cleaning loop bound
  net/gve: fix DQO completion queue stall on unknown type
  net/gve: fix DQO Tx report event interval
  net/gve: fix stray TSO context descriptor on tag exhaustion
  net/gve: add cacheline overlap margin to DQO Tx ring
  net/gve: fix Tx stall on invalid packets
  net/gve: export Tx drop counters in xstats
  net/gve: reject Tx packets that cannot fit in the ring
  net/gve: enforce DQO Tx packet length limits
  net/gve: enforce DQO TSO header and length limits

 drivers/net/gve/base/gve_desc_dqo.h |   3 +
 drivers/net/gve/gve_ethdev.c        |   3 +
 drivers/net/gve/gve_ethdev.h        |   4 +-
 drivers/net/gve/gve_tx_dqo.c        | 113 +++++++++++++++++++++-------
 4 files changed, 96 insertions(+), 27 deletions(-)

-- 
2.43.0

[PATCH v3 01/10] net/gve: fix DQO descriptor cleaning loop bound

[Tathagat Priyadarshi]

gve_tx_clean_descs_dqo() loops on "while (--nb_descs)", which
processes one completion fewer than requested, and none when asked
for one.

Use a post-decrement so exactly nb_descs completions are processed.

Fixes: 4022f9999f56 ("net/gve: support basic Tx data path for DQO")
Cc: stable@dpdk.org

Signed-off-by: Tathagat Priyadarshi <tathagat.dpdk@gmail.com>
Acked-by: Joshua Washington <joshwash@google.com>
---
 drivers/net/gve/gve_tx_dqo.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/net/gve/gve_tx_dqo.c b/drivers/net/gve/gve_tx_dqo.c
index 5c03d3d1a2..e094761b38 100644
--- a/drivers/net/gve/gve_tx_dqo.c
+++ b/drivers/net/gve/gve_tx_dqo.c
@@ -122,7 +122,7 @@ gve_tx_clean_dqo(struct gve_tx_queue *txq)
 
 static inline void
 gve_tx_clean_descs_dqo(struct gve_tx_queue *txq, uint16_t nb_descs) {
-	while (--nb_descs)
+	while (nb_descs--)
 		gve_tx_clean_dqo(txq);
 }
 
-- 
2.43.0

[PATCH v3 02/10] net/gve: fix DQO completion queue stall on unknown type

[Tathagat Priyadarshi]

When gve_tx_clean_dqo() sees an unknown completion type, it returns
without advancing complq_tail. Every later call stops at the same
entry, so Tx completions are never processed again.

Log the type and skip the entry.

Fixes: 4022f9999f56 ("net/gve: support basic Tx data path for DQO")
Cc: stable@dpdk.org

Signed-off-by: Tathagat Priyadarshi <tathagat.dpdk@gmail.com>
Acked-by: Joshua Washington <joshwash@google.com>
---
 drivers/net/gve/gve_tx_dqo.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/net/gve/gve_tx_dqo.c b/drivers/net/gve/gve_tx_dqo.c
index e094761b38..c864d722d9 100644
--- a/drivers/net/gve/gve_tx_dqo.c
+++ b/drivers/net/gve/gve_tx_dqo.c
@@ -108,8 +108,8 @@ gve_tx_clean_dqo(struct gve_tx_queue *txq)
 		PMD_DRV_DP_LOG(DEBUG, "GVE_COMPL_TYPE_DQO_MISS ignored !!!");
 		break;
 	default:
-		PMD_DRV_DP_LOG(ERR, "unknown completion type.");
-		return;
+		PMD_DRV_DP_LOG(ERR, "unknown completion type %u", compl_desc->type);
+		break;
 	}
 
 	next++;
-- 
2.43.0

[PATCH v3 03/10] net/gve: fix DQO Tx report event interval

[Tathagat Priyadarshi]

A descriptor completion is requested once every 32 packets, and Tx
descriptors are only reclaimed on these completions. With
multi-segment or TSO packets the ring can fill before 32 packets are
posted, so no completion is ever requested and the Tx queue stalls
for good.

Request a completion once GVE_TX_MIN_RE_INTERVAL descriptors have
been posted since the last one, as the Linux driver does.

Fixes: 4022f9999f56 ("net/gve: support basic Tx data path for DQO")
Fixes: 403c671a46b6 ("net/gve: support TSO in DQO RDA")
Cc: stable@dpdk.org

Signed-off-by: Tathagat Priyadarshi <tathagat.dpdk@gmail.com>
---
 drivers/net/gve/gve_ethdev.h |  2 +-
 drivers/net/gve/gve_tx_dqo.c | 22 ++++++++++++++--------
 2 files changed, 15 insertions(+), 9 deletions(-)

diff --git a/drivers/net/gve/gve_ethdev.h b/drivers/net/gve/gve_ethdev.h
index c9a176ff17..8349b1c92e 100644
--- a/drivers/net/gve/gve_ethdev.h
+++ b/drivers/net/gve/gve_ethdev.h
@@ -194,7 +194,7 @@ struct gve_tx_queue {
 	uint8_t cur_gen_bit;
 	uint32_t last_desc_cleaned;
 	void **txqs;
-	uint16_t re_cnt;
+	uint16_t last_re_idx;
 
 	/* Only valid for DQO_RDA queue format */
 	struct gve_tx_queue *complq;
diff --git a/drivers/net/gve/gve_tx_dqo.c b/drivers/net/gve/gve_tx_dqo.c
index c864d722d9..a2ebadd0b8 100644
--- a/drivers/net/gve/gve_tx_dqo.c
+++ b/drivers/net/gve/gve_tx_dqo.c
@@ -254,6 +254,8 @@ gve_tx_burst_dqo(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
 	volatile union gve_tx_desc_dqo *txr;
 	volatile union gve_tx_desc_dqo *txd;
 	uint16_t mask = txq->nb_tx_desc - 1;
+	uint16_t last_re_interval;
+	uint16_t last_desc_idx;
 	struct gve_tx_pkt *pkts;
 	struct rte_mbuf *tx_pkt;
 	uint16_t compl_tag;
@@ -331,20 +333,23 @@ gve_tx_burst_dqo(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
 		txd = &txr[(tx_id - 1) & mask];
 		txd->pkt.end_of_packet = 1;
 
+		/* Request a descriptor completion once at least
+		 * GVE_TX_MIN_RE_INTERVAL descriptors have been posted
+		 * since the last request.
+		 */
+		last_desc_idx = (tx_id - 1) & mask;
+		last_re_interval = (last_desc_idx - txq->last_re_idx) & mask;
+		if (unlikely(last_re_interval >= GVE_TX_MIN_RE_INTERVAL)) {
+			txd->pkt.report_event = true;
+			txq->last_re_idx = last_desc_idx;
+		}
+
 		txq->nb_free -= nb_descs;
 		txq->nb_used += nb_descs;
 	}
 
 	/* update the tail pointer if any packets were processed */
 	if (nb_tx > 0) {
-		/* Request a descriptor completion on the last descriptor */
-		txq->re_cnt += nb_tx;
-		if (txq->re_cnt >= GVE_TX_MIN_RE_INTERVAL) {
-			txd = &txr[(tx_id - 1) & mask];
-			txd->pkt.report_event = true;
-			txq->re_cnt = 0;
-		}
-
 		rte_write32(tx_id, txq->qtx_tail);
 		txq->tx_tail = tx_id;
 
@@ -445,6 +450,7 @@ gve_reset_tx_ring_state_dqo(struct gve_tx_queue *txq)
 	txq->nb_used = 0;
 
 	txq->last_desc_cleaned = 0;
+	txq->last_re_idx = 0;
 	txq->nb_free = txq->nb_tx_desc - 1;
 
 	txq->complq_tail = 0;
-- 
2.43.0

[PATCH v3 04/10] net/gve: fix stray TSO context descriptor on tag exhaustion

[Tathagat Priyadarshi]

For TSO packets, the context descriptor is written and tx_id advanced
before a completion tag is allocated. If no tag is available, the
burst stops, but when earlier packets in the burst were sent, the
tail written at the end of the burst still covers the context
descriptor. The device then receives it with no data descriptors
behind it, and the slot is never deducted from nb_free.

Allocate the completion tag before writing any descriptor.

Fixes: 5fd289ff3a00 ("net/gve: support out-of-order completions on DQ Tx")
Cc: stable@dpdk.org

Signed-off-by: Tathagat Priyadarshi <tathagat.dpdk@gmail.com>
---
 drivers/net/gve/gve_tx_dqo.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/drivers/net/gve/gve_tx_dqo.c b/drivers/net/gve/gve_tx_dqo.c
index a2ebadd0b8..36d56b7d26 100644
--- a/drivers/net/gve/gve_tx_dqo.c
+++ b/drivers/net/gve/gve_tx_dqo.c
@@ -308,15 +308,15 @@ gve_tx_burst_dqo(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
 			break;
 		}
 
+		if (!gve_free_compl_tags_pop(txq, &compl_tag))
+			break;
+
 		if (tso) {
 			txd = &txr[tx_id];
 			gve_tx_fill_seg_desc_dqo(txd, tx_pkt);
 			tx_id = (tx_id + 1) & mask;
 		}
 
-		if (!gve_free_compl_tags_pop(txq, &compl_tag))
-			break;
-
 		pkts[compl_tag].mbuf = tx_pkt;
 		while (tx_pkt) {
 			/* Skip writing descriptors if mbuf has no data. */
-- 
2.43.0

[PATCH v3 05/10] net/gve: add cacheline overlap margin to DQO Tx ring

[Tathagat Priyadarshi]

The GVNIC DQO hardware DMA engine prefetches descriptors in cacheline
units (64 bytes = 4 descriptors of 16 bytes each). When producer tail
and consumer head indices share a cacheline, CPU and device DMA memory
collisions can occur.

Align with the Linux driver (gve_has_tx_slots_available()) by requiring
GVE_TX_MIN_DESC_PREVENT_CACHE_OVERLAP (4 descriptors) headroom when
checking descriptor availability before enqueueing packets.

Fixes: 4022f9999f56 ("net/gve: support basic Tx data path for DQO")
Cc: stable@dpdk.org

Signed-off-by: Tathagat Priyadarshi <tathagat.dpdk@gmail.com>
Acked-by: Joshua Washington <joshwash@google.com>
---
 drivers/net/gve/gve_tx_dqo.c | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)

diff --git a/drivers/net/gve/gve_tx_dqo.c b/drivers/net/gve/gve_tx_dqo.c
index 36d56b7d26..c5ae2136ad 100644
--- a/drivers/net/gve/gve_tx_dqo.c
+++ b/drivers/net/gve/gve_tx_dqo.c
@@ -259,6 +259,7 @@ gve_tx_burst_dqo(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
 	struct gve_tx_pkt *pkts;
 	struct rte_mbuf *tx_pkt;
 	uint16_t compl_tag;
+	uint16_t req_descs;
 	const char *reason;
 	uint16_t nb_tx = 0;
 	uint64_t bytes = 0;
@@ -291,14 +292,17 @@ gve_tx_burst_dqo(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
 
 		nb_descs = gve_tx_pkt_nb_data_descs(tx_pkt);
 		nb_descs += tso;
+		req_descs = nb_descs + GVE_TX_MIN_DESC_PREVENT_CACHE_OVERLAP;
 
-		/* Clean if there aren't enough descriptors to send the packet. */
-		if (unlikely(!gve_can_tx(txq, nb_descs, 1))) {
+		/* Clean if there aren't enough descriptors to send the packet
+		 * while maintaining the cacheline overlap margin.
+		 */
+		if (unlikely(!gve_can_tx(txq, req_descs, 1))) {
 			int nb_to_clean = RTE_MAX(DQO_TX_MULTIPLIER * txq->rs_thresh,
-						  nb_descs);
+						  (int)req_descs);
 
 			gve_tx_clean_descs_dqo(txq, nb_to_clean);
-			if (!gve_can_tx(txq, nb_descs, 1))
+			if (!gve_can_tx(txq, req_descs, 1))
 				break;
 		}
 
-- 
2.43.0

[PATCH v3 06/10] net/gve: fix Tx stall on invalid packets

[Tathagat Priyadarshi]

When a packet fails rte_mbuf_check() or exceeds the descriptor limit,
the Tx burst stops at that packet without consuming it. The
application retries the same packet, it fails again, and the queue
never moves.

Free such packets and continue with the burst. Validate before the
ring space check, so that an oversized packet is dropped instead of
stopping the burst there. Count rte_mbuf_check() failures in a new
invalid_mbufs counter.

Fixes: f4a70b449bfa ("net/gve: add DQO Tx descriptor limit")
Fixes: f33ce4445ee6 ("net/gve: validate Tx packet before sending")
Cc: stable@dpdk.org

Signed-off-by: Tathagat Priyadarshi <tathagat.dpdk@gmail.com>
---
 drivers/net/gve/gve_ethdev.h |  1 +
 drivers/net/gve/gve_tx_dqo.c | 24 +++++++++++++++---------
 2 files changed, 16 insertions(+), 9 deletions(-)

diff --git a/drivers/net/gve/gve_ethdev.h b/drivers/net/gve/gve_ethdev.h
index 8349b1c92e..24ef3a4552 100644
--- a/drivers/net/gve/gve_ethdev.h
+++ b/drivers/net/gve/gve_ethdev.h
@@ -114,6 +114,7 @@ struct gve_tx_stats {
 	uint64_t bytes;
 	uint64_t errors;
 	uint64_t too_many_descs;
+	uint64_t invalid_mbufs;
 };
 
 struct gve_rx_stats {
diff --git a/drivers/net/gve/gve_tx_dqo.c b/drivers/net/gve/gve_tx_dqo.c
index c5ae2136ad..44d2fa870f 100644
--- a/drivers/net/gve/gve_tx_dqo.c
+++ b/drivers/net/gve/gve_tx_dqo.c
@@ -261,6 +261,7 @@ gve_tx_burst_dqo(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
 	uint16_t compl_tag;
 	uint16_t req_descs;
 	const char *reason;
+	uint16_t nb_sent = 0;
 	uint16_t nb_tx = 0;
 	uint64_t bytes = 0;
 	uint64_t ol_flags;
@@ -283,7 +284,9 @@ gve_tx_burst_dqo(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
 
 		if (rte_mbuf_check(tx_pkt, true, &reason)) {
 			PMD_DRV_DP_LOG(DEBUG, "Invalid mbuf: %s", reason);
-			break;
+			txq->stats.invalid_mbufs++;
+			rte_pktmbuf_free(tx_pkt);
+			continue;
 		}
 
 		ol_flags = tx_pkt->ol_flags;
@@ -292,6 +295,14 @@ gve_tx_burst_dqo(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
 
 		nb_descs = gve_tx_pkt_nb_data_descs(tx_pkt);
 		nb_descs += tso;
+
+		/* Drop packet if it doesn't adhere to hardware limits. */
+		if (!gve_tx_validate_descs(tx_pkt, nb_descs, tso)) {
+			txq->stats.too_many_descs++;
+			rte_pktmbuf_free(tx_pkt);
+			continue;
+		}
+
 		req_descs = nb_descs + GVE_TX_MIN_DESC_PREVENT_CACHE_OVERLAP;
 
 		/* Clean if there aren't enough descriptors to send the packet
@@ -306,12 +317,6 @@ gve_tx_burst_dqo(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
 				break;
 		}
 
-		/* Drop packet if it doesn't adhere to hardware limits. */
-		if (!gve_tx_validate_descs(tx_pkt, nb_descs, tso)) {
-			txq->stats.too_many_descs++;
-			break;
-		}
-
 		if (!gve_free_compl_tags_pop(txq, &compl_tag))
 			break;
 
@@ -350,6 +355,7 @@ gve_tx_burst_dqo(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
 
 		txq->nb_free -= nb_descs;
 		txq->nb_used += nb_descs;
+		nb_sent++;
 	}
 
 	/* update the tail pointer if any packets were processed */
@@ -357,9 +363,9 @@ gve_tx_burst_dqo(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
 		rte_write32(tx_id, txq->qtx_tail);
 		txq->tx_tail = tx_id;
 
-		txq->stats.packets += nb_tx;
+		txq->stats.packets += nb_sent;
 		txq->stats.bytes += bytes;
-		txq->stats.errors += nb_pkts - nb_tx;
+		txq->stats.errors += nb_pkts - nb_sent;
 	}
 
 	return nb_tx;
-- 
2.43.0

[PATCH v3 07/10] net/gve: export Tx drop counters in xstats

[Tathagat Priyadarshi]

too_many_descs is counted but missing from the Tx xstats table, so it
cannot be read. Export it together with invalid_mbufs.

Fixes: f4a70b449bfa ("net/gve: add DQO Tx descriptor limit")
Cc: stable@dpdk.org

Signed-off-by: Tathagat Priyadarshi <tathagat.dpdk@gmail.com>
---
 drivers/net/gve/gve_ethdev.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/drivers/net/gve/gve_ethdev.c b/drivers/net/gve/gve_ethdev.c
index c990920a4d..cbfb07edf6 100644
--- a/drivers/net/gve/gve_ethdev.c
+++ b/drivers/net/gve/gve_ethdev.c
@@ -983,6 +983,8 @@ static const struct gve_xstats_name_offset tx_xstats_name_offset[] = {
 	{ "packets", TX_QUEUE_STATS_OFFSET(packets) },
 	{ "bytes",   TX_QUEUE_STATS_OFFSET(bytes) },
 	{ "errors",  TX_QUEUE_STATS_OFFSET(errors) },
+	{ "too_many_descs", TX_QUEUE_STATS_OFFSET(too_many_descs) },
+	{ "invalid_mbufs", TX_QUEUE_STATS_OFFSET(invalid_mbufs) },
 };
 
 static const struct gve_xstats_name_offset rx_xstats_name_offset[] = {
-- 
2.43.0

[PATCH v3 08/10] net/gve: reject Tx packets that cannot fit in the ring

[Tathagat Priyadarshi]

A TSO packet needing more descriptors than the ring can ever have
free fails the ring space check on every burst and blocks the queue.
One slot is always reserved, the cacheline overlap margin must stay
free, and up to GVE_TX_MIN_RE_INTERVAL - 1 posted descriptors are not
reclaimed until more are posted.

Drop such packets and count them as too_many_descs.

Fixes: 403c671a46b6 ("net/gve: support TSO in DQO RDA")
Cc: stable@dpdk.org

Signed-off-by: Tathagat Priyadarshi <tathagat.dpdk@gmail.com>
---
 drivers/net/gve/gve_tx_dqo.c | 20 ++++++++++++++++++--
 1 file changed, 18 insertions(+), 2 deletions(-)

diff --git a/drivers/net/gve/gve_tx_dqo.c b/drivers/net/gve/gve_tx_dqo.c
index 44d2fa870f..fd36d92fab 100644
--- a/drivers/net/gve/gve_tx_dqo.c
+++ b/drivers/net/gve/gve_tx_dqo.c
@@ -126,14 +126,30 @@ gve_tx_clean_descs_dqo(struct gve_tx_queue *txq, uint16_t nb_descs) {
 		gve_tx_clean_dqo(txq);
 }
 
+/* Max descriptors a single packet may use. A larger packet can never pass
+ * gve_can_tx(): one slot is always reserved, the cacheline overlap margin
+ * must stay free, and up to GVE_TX_MIN_RE_INTERVAL - 1 descriptors posted
+ * after the last report_event are not reclaimed until more are posted.
+ */
+static inline int
+gve_tx_max_pkt_descs(const struct gve_tx_queue *txq)
+{
+	return (int)txq->nb_tx_desc - 1 - GVE_TX_MIN_DESC_PREVENT_CACHE_OVERLAP -
+		(GVE_TX_MIN_RE_INTERVAL - 1);
+}
+
 /* GVE expects at most 10 data descriptors per mtu-sized segment. Beyond this,
  * the hardware will assume the driver is malicious and stop transmitting
  * packets altogether. Validate that a packet can be sent to avoid sending
  * posting descriptors for an invalid packet.
  */
 static inline bool
-gve_tx_validate_descs(struct rte_mbuf *tx_pkt, uint16_t nb_descs, bool is_tso)
+gve_tx_validate_descs(const struct gve_tx_queue *txq, struct rte_mbuf *tx_pkt,
+		      uint16_t nb_descs, bool is_tso)
 {
+	if (nb_descs > gve_tx_max_pkt_descs(txq))
+		return false;
+
 	if (!is_tso)
 		return nb_descs <= GVE_TX_MAX_DATA_DESCS;
 
@@ -297,7 +313,7 @@ gve_tx_burst_dqo(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
 		nb_descs += tso;
 
 		/* Drop packet if it doesn't adhere to hardware limits. */
-		if (!gve_tx_validate_descs(tx_pkt, nb_descs, tso)) {
+		if (!gve_tx_validate_descs(txq, tx_pkt, nb_descs, tso)) {
 			txq->stats.too_many_descs++;
 			rte_pktmbuf_free(tx_pkt);
 			continue;
-- 
2.43.0

[PATCH v3 09/10] net/gve: enforce DQO Tx packet length limits

[Tathagat Priyadarshi]

The device requires non-TSO packets to be 17 to 9728 bytes long, as
noted in the Linux driver, but packets of any length are posted. The
device treats an out-of-range packet as coming from a malicious
driver: it drops the packet and disables the datapath, stalling all
queues.

Drop such packets in the driver and count them in a new invalid_pkts
counter.

Fixes: 4022f9999f56 ("net/gve: support basic Tx data path for DQO")
Cc: stable@dpdk.org

Signed-off-by: Tathagat Priyadarshi <tathagat.dpdk@gmail.com>
---
 drivers/net/gve/base/gve_desc_dqo.h |  2 ++
 drivers/net/gve/gve_ethdev.c        |  1 +
 drivers/net/gve/gve_ethdev.h        |  1 +
 drivers/net/gve/gve_tx_dqo.c        | 17 +++++++++++++++++
 4 files changed, 21 insertions(+)

diff --git a/drivers/net/gve/base/gve_desc_dqo.h b/drivers/net/gve/base/gve_desc_dqo.h
index c1534959c2..d50d88765e 100644
--- a/drivers/net/gve/base/gve_desc_dqo.h
+++ b/drivers/net/gve/base/gve_desc_dqo.h
@@ -12,6 +12,8 @@
 
 #define GVE_TX_MAX_HDR_SIZE_DQO 255
 #define GVE_TX_MIN_TSO_MSS_DQO 88
+#define GVE_TX_MIN_PKT_SIZE_DQO 17
+#define GVE_TX_MAX_PKT_SIZE_DQO 9728
 
 /* Basic TX descriptor (DTYPE 0x0C) */
 struct gve_tx_pkt_desc_dqo {
diff --git a/drivers/net/gve/gve_ethdev.c b/drivers/net/gve/gve_ethdev.c
index cbfb07edf6..167dcbce7b 100644
--- a/drivers/net/gve/gve_ethdev.c
+++ b/drivers/net/gve/gve_ethdev.c
@@ -985,6 +985,7 @@ static const struct gve_xstats_name_offset tx_xstats_name_offset[] = {
 	{ "errors",  TX_QUEUE_STATS_OFFSET(errors) },
 	{ "too_many_descs", TX_QUEUE_STATS_OFFSET(too_many_descs) },
 	{ "invalid_mbufs", TX_QUEUE_STATS_OFFSET(invalid_mbufs) },
+	{ "invalid_pkts", TX_QUEUE_STATS_OFFSET(invalid_pkts) },
 };
 
 static const struct gve_xstats_name_offset rx_xstats_name_offset[] = {
diff --git a/drivers/net/gve/gve_ethdev.h b/drivers/net/gve/gve_ethdev.h
index 24ef3a4552..2464474426 100644
--- a/drivers/net/gve/gve_ethdev.h
+++ b/drivers/net/gve/gve_ethdev.h
@@ -115,6 +115,7 @@ struct gve_tx_stats {
 	uint64_t errors;
 	uint64_t too_many_descs;
 	uint64_t invalid_mbufs;
+	uint64_t invalid_pkts;
 };
 
 struct gve_rx_stats {
diff --git a/drivers/net/gve/gve_tx_dqo.c b/drivers/net/gve/gve_tx_dqo.c
index fd36d92fab..c23b222ada 100644
--- a/drivers/net/gve/gve_tx_dqo.c
+++ b/drivers/net/gve/gve_tx_dqo.c
@@ -126,6 +126,17 @@ gve_tx_clean_descs_dqo(struct gve_tx_queue *txq, uint16_t nb_descs) {
 		gve_tx_clean_dqo(txq);
 }
 
+/* Validate packet lengths against the device limits. */
+static inline bool
+gve_tx_validate_pkt(const struct rte_mbuf *tx_pkt, bool is_tso)
+{
+	if (!is_tso)
+		return tx_pkt->pkt_len >= GVE_TX_MIN_PKT_SIZE_DQO &&
+		       tx_pkt->pkt_len <= GVE_TX_MAX_PKT_SIZE_DQO;
+
+	return true;
+}
+
 /* Max descriptors a single packet may use. A larger packet can never pass
  * gve_can_tx(): one slot is always reserved, the cacheline overlap margin
  * must stay free, and up to GVE_TX_MIN_RE_INTERVAL - 1 descriptors posted
@@ -309,6 +320,12 @@ gve_tx_burst_dqo(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
 		tso = !!(ol_flags & RTE_MBUF_F_TX_TCP_SEG);
 		csum = !!(ol_flags & GVE_TX_CKSUM_OFFLOAD_MASK_DQO);
 
+		if (!gve_tx_validate_pkt(tx_pkt, tso)) {
+			txq->stats.invalid_pkts++;
+			rte_pktmbuf_free(tx_pkt);
+			continue;
+		}
+
 		nb_descs = gve_tx_pkt_nb_data_descs(tx_pkt);
 		nb_descs += tso;
 
-- 
2.43.0

[PATCH v3 10/10] net/gve: enforce DQO TSO header and length limits

[Tathagat Priyadarshi]

TSO packets are not checked against the device limits before use.
The device requires an MSS of 88 to 9728 bytes and a payload of at
most 262143 bytes, as noted in the Linux driver. A packet outside
these limits triggers malicious driver detection, which drops it and
disables the datapath, stalling all queues.

The missing checks also break the driver itself: a tso_segsz of 0
divides by zero in gve_tx_validate_descs(), headers longer than 255
bytes are truncated by the 8-bit header_len field, and headers not
contained in the first segment give gve_tx_validate_descs() a
negative first segment length.

Drop TSO packets that violate these limits and count them as
invalid_pkts.

Fixes: 403c671a46b6 ("net/gve: support TSO in DQO RDA")
Fixes: be8f0eb81f98 ("net/gve: fix DQO TSO descriptor limit")
Cc: stable@dpdk.org

Signed-off-by: Tathagat Priyadarshi <tathagat.dpdk@gmail.com>
---
 drivers/net/gve/base/gve_desc_dqo.h |  1 +
 drivers/net/gve/gve_tx_dqo.c        | 14 +++++++++++++-
 2 files changed, 14 insertions(+), 1 deletion(-)

diff --git a/drivers/net/gve/base/gve_desc_dqo.h b/drivers/net/gve/base/gve_desc_dqo.h
index d50d88765e..53e394f01c 100644
--- a/drivers/net/gve/base/gve_desc_dqo.h
+++ b/drivers/net/gve/base/gve_desc_dqo.h
@@ -14,6 +14,7 @@
 #define GVE_TX_MIN_TSO_MSS_DQO 88
 #define GVE_TX_MIN_PKT_SIZE_DQO 17
 #define GVE_TX_MAX_PKT_SIZE_DQO 9728
+#define GVE_TX_MAX_TSO_LEN_DQO ((256 * 1024) - 1)
 
 /* Basic TX descriptor (DTYPE 0x0C) */
 struct gve_tx_pkt_desc_dqo {
diff --git a/drivers/net/gve/gve_tx_dqo.c b/drivers/net/gve/gve_tx_dqo.c
index c23b222ada..0b328e4af4 100644
--- a/drivers/net/gve/gve_tx_dqo.c
+++ b/drivers/net/gve/gve_tx_dqo.c
@@ -130,11 +130,23 @@ gve_tx_clean_descs_dqo(struct gve_tx_queue *txq, uint16_t nb_descs) {
 static inline bool
 gve_tx_validate_pkt(const struct rte_mbuf *tx_pkt, bool is_tso)
 {
+	uint32_t headlen;
+
 	if (!is_tso)
 		return tx_pkt->pkt_len >= GVE_TX_MIN_PKT_SIZE_DQO &&
 		       tx_pkt->pkt_len <= GVE_TX_MAX_PKT_SIZE_DQO;
 
-	return true;
+	/* Headers must be present, fit the context descriptor, and be
+	 * contained in the first segment; the payload must not be empty.
+	 */
+	headlen = tx_pkt->l2_len + tx_pkt->l3_len + tx_pkt->l4_len;
+	if (tx_pkt->l4_len == 0 || headlen > GVE_TX_MAX_HDR_SIZE_DQO ||
+	    tx_pkt->data_len < headlen || tx_pkt->pkt_len <= headlen)
+		return false;
+
+	return tx_pkt->pkt_len - headlen <= GVE_TX_MAX_TSO_LEN_DQO &&
+	       tx_pkt->tso_segsz >= GVE_TX_MIN_TSO_MSS_DQO &&
+	       tx_pkt->tso_segsz <= GVE_TX_MAX_PKT_SIZE_DQO;
 }
 
 /* Max descriptors a single packet may use. A larger packet can never pass
-- 
2.43.0

Re: [PATCH v3 00/10] net/gve: fix DQO Tx queue stalls

[Stephen Hemminger]

On Fri, 16 Oct 2026 06:26:37 +0000
Tathagat Priyadarshi <tathagat.dpdk@gmail.com> wrote:

> This series fixes several ways the gve DQO RDA Tx path can stall or
> post invalid descriptors: completion handling bugs (1-2), a report
> event interval that can starve the ring (3), a stray TSO context
> descriptor when completion tags run out (4), a missing cacheline
> margin (5), invalid packets blocking the burst (6-8), and missing
> device length checks (9-10).


Looks good, but does not apply to main now.

Also need a ack from GVE maintainer across all of the patches