* [PATCH v2 1/4] net/gve: fix DQO descriptor cleaning loop bound
2026-10-13 10:43 ` [PATCH v2 0/4] net/gve: fix Tx " Tathagat Priyadarshi
@ 2026-10-13 10:43 ` Tathagat Priyadarshi
2026-10-13 10:43 ` [PATCH v2 2/4] net/gve: fix DQO Tx queue stall on multi-seg and TSO Tathagat Priyadarshi
` (3 subsequent siblings)
4 siblings, 0 replies; 16+ messages in thread
From: Tathagat Priyadarshi @ 2026-10-13 10:43 UTC (permalink / raw)
To: dev; +Cc: joshwash, stephen, stable
In gve_tx_clean_descs_dqo(), the loop condition "while (--nb_descs)"
causes an off-by-one error:
- When nb_descs == 1, --nb_descs decrements to 0 on the first check,
executing 0 iterations instead of 1.
- When nb_descs == 0, --nb_descs underflows uint16_t to 65535,
executing 65535 iterations.
Change the condition to "while (nb_descs--)" so that exactly nb_descs
iterations are executed for any input value.
Additionally, in gve_tx_clean_dqo(), handle unrecognized completion
types by breaking from the switch rather than returning immediately,
allowing complq_tail to advance past unknown descriptors rather than
stalling the completion ring at the same entry.
Fixes: 4022f9999f56 ("net/gve: support basic Tx data path for DQO")
Cc: stable@dpdk.org
Signed-off-by: Tathagat Priyadarshi <tathagat.dpdk@gmail.com>
Acked-by: Joshua Washington <joshwash@google.com>
---
drivers/net/gve/gve_tx_dqo.c | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/drivers/net/gve/gve_tx_dqo.c b/drivers/net/gve/gve_tx_dqo.c
index 5c03d3d1a2..c864d722d9 100644
--- a/drivers/net/gve/gve_tx_dqo.c
+++ b/drivers/net/gve/gve_tx_dqo.c
@@ -108,8 +108,8 @@ gve_tx_clean_dqo(struct gve_tx_queue *txq)
PMD_DRV_DP_LOG(DEBUG, "GVE_COMPL_TYPE_DQO_MISS ignored !!!");
break;
default:
- PMD_DRV_DP_LOG(ERR, "unknown completion type.");
- return;
+ PMD_DRV_DP_LOG(ERR, "unknown completion type %u", compl_desc->type);
+ break;
}
next++;
@@ -122,7 +122,7 @@ gve_tx_clean_dqo(struct gve_tx_queue *txq)
static inline void
gve_tx_clean_descs_dqo(struct gve_tx_queue *txq, uint16_t nb_descs) {
- while (--nb_descs)
+ while (nb_descs--)
gve_tx_clean_dqo(txq);
}
--
2.43.0
^ permalink raw reply related [flat|nested] 16+ messages in thread* [PATCH v2 2/4] net/gve: fix DQO Tx queue stall on multi-seg and TSO
2026-10-13 10:43 ` [PATCH v2 0/4] net/gve: fix Tx " Tathagat Priyadarshi
2026-10-13 10:43 ` [PATCH v2 1/4] net/gve: fix DQO descriptor cleaning loop bound Tathagat Priyadarshi
@ 2026-10-13 10:43 ` Tathagat Priyadarshi
2026-10-14 0:30 ` Joshua Washington
2026-10-13 10:43 ` [PATCH v2 3/4] net/gve: add cacheline overlap margin to DQO Tx ring Tathagat Priyadarshi
` (2 subsequent siblings)
4 siblings, 1 reply; 16+ messages in thread
From: Tathagat Priyadarshi @ 2026-10-13 10:43 UTC (permalink / raw)
To: dev; +Cc: joshwash, stephen, stable
The DPDK GVE driver previously tracked transmitted packet count (re_cnt)
at the end of each burst rather than tracking descriptor intervals per
packet against GVE_TX_MIN_RE_INTERVAL (32).
In multi-segment and TSO transmissions, each packet consumes multiple
descriptors. The descriptor ring could exhaust before 32 packets were
sent, resulting in zero descriptors requesting report_event and
preventing the hardware DMA engine from returning GVE_COMPL_TYPE_DQO_DESC
completions. Without these completions, txq->nb_free was never
replenished, permanently stalling the TX queue.
Align with the Linux kernel driver (gve_tx_update_tail()) by tracking
descriptor indices with last_re_idx:
- Replace re_cnt with last_re_idx in struct gve_tx_queue.
- Reset last_re_idx = 0 in gve_reset_tx_ring_state_dqo().
- On each packet transmission, check descriptor interval
(last_desc_idx - last_re_idx) & mask >= GVE_TX_MIN_RE_INTERVAL.
When met, set report_event on the packet's last descriptor and
update last_re_idx.
- Ring the transmit doorbell immediately upon setting report_event to
kick hardware processing early and reduce device fetch latency during
bursts of multi-descriptor TSO packets, amortizing MMIO writes over
RE_INTERVAL descriptors.
- Defensively pop completion tags before writing TSO context descriptors
to avoid partial descriptor ring updates if completion tags are
exhausted.
Fixes: 4022f9999f56 ("net/gve: support basic Tx data path for DQO")
Fixes: 403c671a46b6 ("net/gve: support TSO in DQO RDA")
Cc: stable@dpdk.org
Signed-off-by: Tathagat Priyadarshi <tathagat.dpdk@gmail.com>
---
drivers/net/gve/gve_ethdev.h | 2 +-
drivers/net/gve/gve_tx_dqo.c | 46 ++++++++++++++++++++----------------
2 files changed, 27 insertions(+), 21 deletions(-)
diff --git a/drivers/net/gve/gve_ethdev.h b/drivers/net/gve/gve_ethdev.h
index c9a176ff17..2033f49444 100644
--- a/drivers/net/gve/gve_ethdev.h
+++ b/drivers/net/gve/gve_ethdev.h
@@ -193,8 +193,8 @@ struct gve_tx_queue {
uint16_t sw_size;
uint8_t cur_gen_bit;
uint32_t last_desc_cleaned;
+ uint32_t last_re_idx;
void **txqs;
- uint16_t re_cnt;
/* Only valid for DQO_RDA queue format */
struct gve_tx_queue *complq;
diff --git a/drivers/net/gve/gve_tx_dqo.c b/drivers/net/gve/gve_tx_dqo.c
index c864d722d9..0839179d7f 100644
--- a/drivers/net/gve/gve_tx_dqo.c
+++ b/drivers/net/gve/gve_tx_dqo.c
@@ -250,21 +250,23 @@ gve_fill_tx_pkt_desc(struct gve_tx_queue *txq, uint16_t *tx_id,
uint16_t
gve_tx_burst_dqo(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
{
+ volatile union gve_tx_desc_dqo *txr, *txd;
struct gve_tx_queue *txq = tx_queue;
- volatile union gve_tx_desc_dqo *txr;
- volatile union gve_tx_desc_dqo *txd;
- uint16_t mask = txq->nb_tx_desc - 1;
- struct gve_tx_pkt *pkts;
+ uint32_t last_re_interval;
struct rte_mbuf *tx_pkt;
- uint16_t compl_tag;
- const char *reason;
+ struct gve_tx_pkt *pkts;
+ uint32_t last_desc_idx;
uint16_t nb_tx = 0;
uint64_t bytes = 0;
+ const char *reason;
+ uint16_t compl_tag;
uint64_t ol_flags;
uint16_t nb_descs;
- bool csum, tso;
uint16_t tx_id;
+ bool csum, tso;
+ uint16_t mask;
+ mask = txq->nb_tx_desc - 1;
pkts = txq->pkt_ring_dqo;
txr = txq->tx_ring;
@@ -306,15 +308,15 @@ gve_tx_burst_dqo(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
break;
}
+ if (!gve_free_compl_tags_pop(txq, &compl_tag))
+ break;
+
if (tso) {
txd = &txr[tx_id];
gve_tx_fill_seg_desc_dqo(txd, tx_pkt);
tx_id = (tx_id + 1) & mask;
}
- if (!gve_free_compl_tags_pop(txq, &compl_tag))
- break;
-
pkts[compl_tag].mbuf = tx_pkt;
while (tx_pkt) {
/* Skip writing descriptors if mbuf has no data. */
@@ -331,23 +333,26 @@ gve_tx_burst_dqo(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
txd = &txr[(tx_id - 1) & mask];
txd->pkt.end_of_packet = 1;
+ last_desc_idx = (tx_id - 1) & mask;
+ last_re_interval = (last_desc_idx - txq->last_re_idx) & mask;
+ if (unlikely(last_re_interval >= GVE_TX_MIN_RE_INTERVAL)) {
+ txd->pkt.report_event = true;
+ txq->last_re_idx = last_desc_idx;
+ rte_write32(tx_id, txq->qtx_tail);
+ txq->tx_tail = tx_id;
+ }
+
txq->nb_free -= nb_descs;
txq->nb_used += nb_descs;
}
- /* update the tail pointer if any packets were processed */
- if (nb_tx > 0) {
- /* Request a descriptor completion on the last descriptor */
- txq->re_cnt += nb_tx;
- if (txq->re_cnt >= GVE_TX_MIN_RE_INTERVAL) {
- txd = &txr[(tx_id - 1) & mask];
- txd->pkt.report_event = true;
- txq->re_cnt = 0;
- }
-
+ /* update the tail pointer if any uncommitted descriptors remain */
+ if (txq->tx_tail != tx_id) {
rte_write32(tx_id, txq->qtx_tail);
txq->tx_tail = tx_id;
+ }
+ if (nb_tx > 0) {
txq->stats.packets += nb_tx;
txq->stats.bytes += bytes;
txq->stats.errors += nb_pkts - nb_tx;
@@ -445,6 +450,7 @@ gve_reset_tx_ring_state_dqo(struct gve_tx_queue *txq)
txq->nb_used = 0;
txq->last_desc_cleaned = 0;
+ txq->last_re_idx = 0;
txq->nb_free = txq->nb_tx_desc - 1;
txq->complq_tail = 0;
--
2.43.0
^ permalink raw reply related [flat|nested] 16+ messages in thread* Re: [PATCH v2 2/4] net/gve: fix DQO Tx queue stall on multi-seg and TSO
2026-10-13 10:43 ` [PATCH v2 2/4] net/gve: fix DQO Tx queue stall on multi-seg and TSO Tathagat Priyadarshi
@ 2026-10-14 0:30 ` Joshua Washington
0 siblings, 0 replies; 16+ messages in thread
From: Joshua Washington @ 2026-10-14 0:30 UTC (permalink / raw)
To: Tathagat Priyadarshi; +Cc: dev, stephen, stable
On Tue, Oct 13, 2026 at 3:44 AM Tathagat Priyadarshi
<tathagat.dpdk@gmail.com> wrote:
>
> The DPDK GVE driver previously tracked transmitted packet count (re_cnt)
> at the end of each burst rather than tracking descriptor intervals per
> packet against GVE_TX_MIN_RE_INTERVAL (32).
>
> In multi-segment and TSO transmissions, each packet consumes multiple
> descriptors. The descriptor ring could exhaust before 32 packets were
> sent, resulting in zero descriptors requesting report_event and
> preventing the hardware DMA engine from returning GVE_COMPL_TYPE_DQO_DESC
> completions. Without these completions, txq->nb_free was never
> replenished, permanently stalling the TX queue.
>
> Align with the Linux kernel driver (gve_tx_update_tail()) by tracking
> descriptor indices with last_re_idx:
> - Replace re_cnt with last_re_idx in struct gve_tx_queue.
> - Reset last_re_idx = 0 in gve_reset_tx_ring_state_dqo().
> - On each packet transmission, check descriptor interval
> (last_desc_idx - last_re_idx) & mask >= GVE_TX_MIN_RE_INTERVAL.
> When met, set report_event on the packet's last descriptor and
> update last_re_idx.
> - Ring the transmit doorbell immediately upon setting report_event to
> kick hardware processing early and reduce device fetch latency during
> bursts of multi-descriptor TSO packets, amortizing MMIO writes over
> RE_INTERVAL descriptors.
> - Defensively pop completion tags before writing TSO context descriptors
> to avoid partial descriptor ring updates if completion tags are
> exhausted.
>
> Fixes: 4022f9999f56 ("net/gve: support basic Tx data path for DQO")
> Fixes: 403c671a46b6 ("net/gve: support TSO in DQO RDA")
> Cc: stable@dpdk.org
>
> Signed-off-by: Tathagat Priyadarshi <tathagat.dpdk@gmail.com>
> ---
> drivers/net/gve/gve_ethdev.h | 2 +-
> drivers/net/gve/gve_tx_dqo.c | 46 ++++++++++++++++++++----------------
> 2 files changed, 27 insertions(+), 21 deletions(-)
>
> diff --git a/drivers/net/gve/gve_ethdev.h b/drivers/net/gve/gve_ethdev.h
> index c9a176ff17..2033f49444 100644
> --- a/drivers/net/gve/gve_ethdev.h
> +++ b/drivers/net/gve/gve_ethdev.h
> @@ -193,8 +193,8 @@ struct gve_tx_queue {
> uint16_t sw_size;
> uint8_t cur_gen_bit;
> uint32_t last_desc_cleaned;
> + uint32_t last_re_idx;
> void **txqs;
> - uint16_t re_cnt;
>
> /* Only valid for DQO_RDA queue format */
> struct gve_tx_queue *complq;
> diff --git a/drivers/net/gve/gve_tx_dqo.c b/drivers/net/gve/gve_tx_dqo.c
> index c864d722d9..0839179d7f 100644
> --- a/drivers/net/gve/gve_tx_dqo.c
> +++ b/drivers/net/gve/gve_tx_dqo.c
> @@ -250,21 +250,23 @@ gve_fill_tx_pkt_desc(struct gve_tx_queue *txq, uint16_t *tx_id,
> uint16_t
> gve_tx_burst_dqo(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
> {
> + volatile union gve_tx_desc_dqo *txr, *txd;
> struct gve_tx_queue *txq = tx_queue;
> - volatile union gve_tx_desc_dqo *txr;
> - volatile union gve_tx_desc_dqo *txd;
> - uint16_t mask = txq->nb_tx_desc - 1;
> - struct gve_tx_pkt *pkts;
> + uint32_t last_re_interval;
> struct rte_mbuf *tx_pkt;
> - uint16_t compl_tag;
> - const char *reason;
> + struct gve_tx_pkt *pkts;
> + uint32_t last_desc_idx;
> uint16_t nb_tx = 0;
> uint64_t bytes = 0;
> + const char *reason;
> + uint16_t compl_tag;
> uint64_t ol_flags;
> uint16_t nb_descs;
> - bool csum, tso;
> uint16_t tx_id;
> + bool csum, tso;
> + uint16_t mask;
It's kind of hard to tell from first glance, given how many
declarations are here and how many are changing, but are all of these
changes necessary for a fix? Things like moving txr, txd to the same
line or separating the declaration and definition of mask, even though
txq is already defined in an earlier line anyway seem unnecessary at
first glance. There might be other such changes, but I stopped there,
since those are in the first 5 lines of delta. Let's try to keep the
fix focused, only making changes that are actually necessary.
>
> + mask = txq->nb_tx_desc - 1;
> pkts = txq->pkt_ring_dqo;
> txr = txq->tx_ring;
>
> @@ -306,15 +308,15 @@ gve_tx_burst_dqo(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
> break;
> }
>
> + if (!gve_free_compl_tags_pop(txq, &compl_tag))
> + break;
> +
> if (tso) {
> txd = &txr[tx_id];
> gve_tx_fill_seg_desc_dqo(txd, tx_pkt);
> tx_id = (tx_id + 1) & mask;
> }
>
> - if (!gve_free_compl_tags_pop(txq, &compl_tag))
> - break;
> -
> pkts[compl_tag].mbuf = tx_pkt;
> while (tx_pkt) {
> /* Skip writing descriptors if mbuf has no data. */
> @@ -331,23 +333,26 @@ gve_tx_burst_dqo(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
> txd = &txr[(tx_id - 1) & mask];
> txd->pkt.end_of_packet = 1;
>
> + last_desc_idx = (tx_id - 1) & mask;
> + last_re_interval = (last_desc_idx - txq->last_re_idx) & mask;
> + if (unlikely(last_re_interval >= GVE_TX_MIN_RE_INTERVAL)) {
> + txd->pkt.report_event = true;
> + txq->last_re_idx = last_desc_idx;
> + rte_write32(tx_id, txq->qtx_tail);
> + txq->tx_tail = tx_id;
> + }
> +
> txq->nb_free -= nb_descs;
> txq->nb_used += nb_descs;
> }
>
> - /* update the tail pointer if any packets were processed */
> - if (nb_tx > 0) {
> - /* Request a descriptor completion on the last descriptor */
> - txq->re_cnt += nb_tx;
> - if (txq->re_cnt >= GVE_TX_MIN_RE_INTERVAL) {
> - txd = &txr[(tx_id - 1) & mask];
> - txd->pkt.report_event = true;
> - txq->re_cnt = 0;
> - }
> -
> + /* update the tail pointer if any uncommitted descriptors remain */
> + if (txq->tx_tail != tx_id) {
> rte_write32(tx_id, txq->qtx_tail);
> txq->tx_tail = tx_id;
> + }
>
> + if (nb_tx > 0) {
> txq->stats.packets += nb_tx;
> txq->stats.bytes += bytes;
> txq->stats.errors += nb_pkts - nb_tx;
> @@ -445,6 +450,7 @@ gve_reset_tx_ring_state_dqo(struct gve_tx_queue *txq)
> txq->nb_used = 0;
>
> txq->last_desc_cleaned = 0;
> + txq->last_re_idx = 0;
> txq->nb_free = txq->nb_tx_desc - 1;
>
> txq->complq_tail = 0;
> --
> 2.43.0
>
^ permalink raw reply [flat|nested] 16+ messages in thread
* [PATCH v2 3/4] net/gve: add cacheline overlap margin to DQO Tx ring
2026-10-13 10:43 ` [PATCH v2 0/4] net/gve: fix Tx " Tathagat Priyadarshi
2026-10-13 10:43 ` [PATCH v2 1/4] net/gve: fix DQO descriptor cleaning loop bound Tathagat Priyadarshi
2026-10-13 10:43 ` [PATCH v2 2/4] net/gve: fix DQO Tx queue stall on multi-seg and TSO Tathagat Priyadarshi
@ 2026-10-13 10:43 ` Tathagat Priyadarshi
2026-10-13 10:43 ` [PATCH v2 4/4] net/gve: add packet and TSO hardware constraints for DQO Tathagat Priyadarshi
2026-10-13 15:22 ` [PATCH v2 0/4] net/gve: fix Tx queue stalls and sync DQO RDA with Linux driver Stephen Hemminger
4 siblings, 0 replies; 16+ messages in thread
From: Tathagat Priyadarshi @ 2026-10-13 10:43 UTC (permalink / raw)
To: dev; +Cc: joshwash, stephen, stable
The GVNIC DQO hardware DMA engine prefetches descriptors in cacheline
units (64 bytes = 4 descriptors of 16 bytes each). When producer tail
and consumer head indices share a cacheline, CPU and device DMA memory
collisions can occur.
Align with the Linux driver (gve_has_tx_slots_available()) by requiring
GVE_TX_MIN_DESC_PREVENT_CACHE_OVERLAP (4 descriptors) headroom when
checking descriptor availability before enqueueing packets.
Fixes: 4022f9999f56 ("net/gve: support basic Tx data path for DQO")
Cc: stable@dpdk.org
Signed-off-by: Tathagat Priyadarshi <tathagat.dpdk@gmail.com>
Acked-by: Joshua Washington <joshwash@google.com>
---
drivers/net/gve/gve_tx_dqo.c | 12 ++++++++----
1 file changed, 8 insertions(+), 4 deletions(-)
diff --git a/drivers/net/gve/gve_tx_dqo.c b/drivers/net/gve/gve_tx_dqo.c
index 0839179d7f..eee1f10df1 100644
--- a/drivers/net/gve/gve_tx_dqo.c
+++ b/drivers/net/gve/gve_tx_dqo.c
@@ -260,6 +260,7 @@ gve_tx_burst_dqo(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
uint64_t bytes = 0;
const char *reason;
uint16_t compl_tag;
+ uint16_t req_descs;
uint64_t ol_flags;
uint16_t nb_descs;
uint16_t tx_id;
@@ -291,14 +292,17 @@ gve_tx_burst_dqo(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
nb_descs = gve_tx_pkt_nb_data_descs(tx_pkt);
nb_descs += tso;
+ req_descs = nb_descs + GVE_TX_MIN_DESC_PREVENT_CACHE_OVERLAP;
- /* Clean if there aren't enough descriptors to send the packet. */
- if (unlikely(!gve_can_tx(txq, nb_descs, 1))) {
+ /* Clean if there aren't enough descriptors to send the packet
+ * while maintaining the cacheline overlap margin.
+ */
+ if (unlikely(!gve_can_tx(txq, req_descs, 1))) {
int nb_to_clean = RTE_MAX(DQO_TX_MULTIPLIER * txq->rs_thresh,
- nb_descs);
+ (int)req_descs);
gve_tx_clean_descs_dqo(txq, nb_to_clean);
- if (!gve_can_tx(txq, nb_descs, 1))
+ if (!gve_can_tx(txq, req_descs, 1))
break;
}
--
2.43.0
^ permalink raw reply related [flat|nested] 16+ messages in thread* [PATCH v2 4/4] net/gve: add packet and TSO hardware constraints for DQO
2026-10-13 10:43 ` [PATCH v2 0/4] net/gve: fix Tx " Tathagat Priyadarshi
` (2 preceding siblings ...)
2026-10-13 10:43 ` [PATCH v2 3/4] net/gve: add cacheline overlap margin to DQO Tx ring Tathagat Priyadarshi
@ 2026-10-13 10:43 ` Tathagat Priyadarshi
2026-10-14 0:50 ` Joshua Washington
2026-10-13 15:22 ` [PATCH v2 0/4] net/gve: fix Tx queue stalls and sync DQO RDA with Linux driver Stephen Hemminger
4 siblings, 1 reply; 16+ messages in thread
From: Tathagat Priyadarshi @ 2026-10-13 10:43 UTC (permalink / raw)
To: dev; +Cc: joshwash, stephen, stable
The GVNIC DQO hardware restricts each packet (or MTU segment in TSO)
to at most 10 data descriptors (GVE_TX_MAX_DATA_DESCS), beyond which
the hardware drops traffic. In addition to this 10-descriptor limit,
the hardware imposes strict packet sizing and header constraints.
Previously, failing the descriptor limit or rte_mbuf_check() broke
from the Tx burst loop without consuming the invalid mbuf. This caused
tx_burst to return a short count, leading the application to retry the
same rejected packet indefinitely and causing Head-of-Line (HOL)
queue stalls.
Enforce hardware validation and prevent HOL blocking in DPDK:
- Validate packet and hardware constraints prior to ring space checks.
- When packet validation or rte_mbuf_check() fails, drop and free the
invalid mbuf with rte_pktmbuf_free(), update stats, and continue the
burst to prevent Head-of-Line (HOL) blocking.
- Exclude dropped packets from txq->stats.packets while returning the
total consumed count to the caller.
- Rename too_many_descs in struct gve_tx_stats to dropped_pkts to reflect
all validation and malformed mbuf drop events.
- Reject packets whose descriptor count exceeds the maximum capacity of
the ring (accounting for reserved slots, cacheline overlap margin, and
unreported descriptors), preventing permanent queue stalls.
- Define GVE_TX_MIN_PKT_SIZE_DQO (17 bytes) and GVE_TX_MAX_PKT_SIZE_DQO
(9728 bytes) to enforce wire packet boundaries for non-TSO packets
and TSO MSS.
- Define GVE_TX_MAX_TSO_LEN_DQO (262143 bytes) to enforce the hardware
L4 payload limit in the 24-bit tso_total_len field.
- In gve_tx_validate_descs():
* Enforce packet size in [17, 9728] for non-TSO packets alongside
the 10-descriptor limit (nb_descs <= GVE_TX_MAX_DATA_DESCS).
* Reject TSO packets missing an L4 header (l4_len == 0).
* Reject packet with headers (L2-L4) not contiguous in first segment.
* Reject TSO packets without payload (pkt_len <= headlen).
* Enforce TSO total payload length limit (pkt_len - headlen <= 262143).
* Enforce MSS limits [88, 9728] and header length limit <= 255.
Fixes: 4022f9999f56 ("net/gve: support basic Tx data path for DQO")
Fixes: f4a70b449bfa ("net/gve: add DQO Tx descriptor limit")
Fixes: 403c671a46b6 ("net/gve: support TSO in DQO RDA")
Cc: stable@dpdk.org
Signed-off-by: Tathagat Priyadarshi <tathagat.dpdk@gmail.com>
---
drivers/net/gve/base/gve_desc_dqo.h | 3 +
drivers/net/gve/gve_ethdev.h | 2 +-
drivers/net/gve/gve_tx_dqo.c | 92 +++++++++++++++++++++++------
3 files changed, 79 insertions(+), 18 deletions(-)
diff --git a/drivers/net/gve/base/gve_desc_dqo.h b/drivers/net/gve/base/gve_desc_dqo.h
index c1534959c2..53e394f01c 100644
--- a/drivers/net/gve/base/gve_desc_dqo.h
+++ b/drivers/net/gve/base/gve_desc_dqo.h
@@ -12,6 +12,9 @@
#define GVE_TX_MAX_HDR_SIZE_DQO 255
#define GVE_TX_MIN_TSO_MSS_DQO 88
+#define GVE_TX_MIN_PKT_SIZE_DQO 17
+#define GVE_TX_MAX_PKT_SIZE_DQO 9728
+#define GVE_TX_MAX_TSO_LEN_DQO ((256 * 1024) - 1)
/* Basic TX descriptor (DTYPE 0x0C) */
struct gve_tx_pkt_desc_dqo {
diff --git a/drivers/net/gve/gve_ethdev.h b/drivers/net/gve/gve_ethdev.h
index 2033f49444..d6d71c31bb 100644
--- a/drivers/net/gve/gve_ethdev.h
+++ b/drivers/net/gve/gve_ethdev.h
@@ -113,7 +113,7 @@ struct gve_tx_stats {
uint64_t packets;
uint64_t bytes;
uint64_t errors;
- uint64_t too_many_descs;
+ uint64_t dropped_pkts;
};
struct gve_rx_stats {
diff --git a/drivers/net/gve/gve_tx_dqo.c b/drivers/net/gve/gve_tx_dqo.c
index eee1f10df1..94ea426961 100644
--- a/drivers/net/gve/gve_tx_dqo.c
+++ b/drivers/net/gve/gve_tx_dqo.c
@@ -126,20 +126,67 @@ gve_tx_clean_descs_dqo(struct gve_tx_queue *txq, uint16_t nb_descs) {
gve_tx_clean_dqo(txq);
}
+/* Max descriptors a single packet may use. A larger packet can never pass
+ * gve_can_tx(): one slot is always reserved, the cacheline overlap margin
+ * must stay free, and up to GVE_TX_MIN_RE_INTERVAL - 1 descriptors posted
+ * after the last report_event are not reclaimed until more are posted.
+ */
+static inline int
+gve_tx_max_pkt_descs(const struct gve_tx_queue *txq)
+{
+ return (int)txq->nb_tx_desc - 1 - GVE_TX_MIN_DESC_PREVENT_CACHE_OVERLAP -
+ (GVE_TX_MIN_RE_INTERVAL - 1);
+}
+
/* GVE expects at most 10 data descriptors per mtu-sized segment. Beyond this,
* the hardware will assume the driver is malicious and stop transmitting
* packets altogether. Validate that a packet can be sent to avoid sending
* posting descriptors for an invalid packet.
*/
static inline bool
-gve_tx_validate_descs(struct rte_mbuf *tx_pkt, uint16_t nb_descs, bool is_tso)
+gve_tx_validate_descs(const struct gve_tx_queue *txq, struct rte_mbuf *tx_pkt,
+ uint16_t nb_descs, bool is_tso)
{
- if (!is_tso)
+ int num_descs, seg_offset, mbuf_len;
+ int tso_segsz, headlen;
+
+ /* Reject packets that can never fit in the ring */
+ if (unlikely(nb_descs > gve_tx_max_pkt_descs(txq)))
+ return false;
+
+ if (!is_tso) {
+ /* Reject packets that violate hardware length constraints */
+ if (unlikely(tx_pkt->pkt_len < GVE_TX_MIN_PKT_SIZE_DQO ||
+ tx_pkt->pkt_len > GVE_TX_MAX_PKT_SIZE_DQO))
+ return false;
return nb_descs <= GVE_TX_MAX_DATA_DESCS;
+ }
- int tso_segsz = tx_pkt->tso_segsz;
- int num_descs, seg_offset, mbuf_len;
- int headlen = tx_pkt->l2_len + tx_pkt->l3_len + tx_pkt->l4_len;
+ headlen = tx_pkt->l2_len + tx_pkt->l3_len + tx_pkt->l4_len;
+
+ /* Reject missing TCP header on TSO */
+ if (unlikely(tx_pkt->l4_len == 0))
+ return false;
+
+ /* Reject packet with headers (L2-L4) not contiguous in first segment */
+ if (unlikely(tx_pkt->data_len < headlen))
+ return false;
+
+ /* Reject TSO packet without payload */
+ if (unlikely(tx_pkt->pkt_len <= (uint32_t)headlen))
+ return false;
+
+ /* Enforce GVE hardware TSO total length limit */
+ if (unlikely(tx_pkt->pkt_len - headlen > GVE_TX_MAX_TSO_LEN_DQO))
+ return false;
+
+ /* Enforce GVE hardware MSS and header limits */
+ if (unlikely(tx_pkt->tso_segsz < GVE_TX_MIN_TSO_MSS_DQO ||
+ tx_pkt->tso_segsz > GVE_TX_MAX_PKT_SIZE_DQO ||
+ headlen > GVE_TX_MAX_HDR_SIZE_DQO))
+ return false;
+
+ tso_segsz = tx_pkt->tso_segsz;
/* Headers will be split into their own buffer. */
num_descs = 1;
@@ -147,14 +194,18 @@ gve_tx_validate_descs(struct rte_mbuf *tx_pkt, uint16_t nb_descs, bool is_tso)
mbuf_len = tx_pkt->data_len - headlen;
while (tx_pkt) {
+ int seg_remain;
+
if (!mbuf_len)
goto next_mbuf;
- int seg_remain = tso_segsz - seg_offset;
+ seg_remain = tso_segsz - seg_offset;
if (num_descs == GVE_TX_MAX_DATA_DESCS && seg_remain)
return false;
if (seg_remain < mbuf_len) {
+ int mbuf_remain = tx_pkt->data_len % GVE_TX_MAX_BUF_SIZE_DQO;
+
seg_offset = mbuf_len % tso_segsz;
/* The MSS is bound from above by 9728B, so a
* single TSO segment in the middle of an mbuf
@@ -162,7 +213,6 @@ gve_tx_validate_descs(struct rte_mbuf *tx_pkt, uint16_t nb_descs, bool is_tso)
* is not at risk of defying this limitation.
* Thus, such segments are ignored.
*/
- int mbuf_remain = tx_pkt->data_len % GVE_TX_MAX_BUF_SIZE_DQO;
/* For each TSO segment, HW will prepend
* headers. The remaining bytes of this mbuf
@@ -184,7 +234,6 @@ gve_tx_validate_descs(struct rte_mbuf *tx_pkt, uint16_t nb_descs, bool is_tso)
mbuf_len = tx_pkt->data_len;
}
-
return true;
}
@@ -256,6 +305,7 @@ gve_tx_burst_dqo(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
struct rte_mbuf *tx_pkt;
struct gve_tx_pkt *pkts;
uint32_t last_desc_idx;
+ uint16_t nb_drop = 0;
uint16_t nb_tx = 0;
uint64_t bytes = 0;
const char *reason;
@@ -281,9 +331,13 @@ gve_tx_burst_dqo(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
txq->rs_thresh);
- if (rte_mbuf_check(tx_pkt, true, &reason)) {
+ if (unlikely(rte_mbuf_check(tx_pkt, true, &reason))) {
PMD_DRV_DP_LOG(DEBUG, "Invalid mbuf: %s", reason);
- break;
+ txq->stats.dropped_pkts++;
+ txq->stats.errors++;
+ rte_pktmbuf_free(tx_pkt);
+ nb_drop++;
+ continue;
}
ol_flags = tx_pkt->ol_flags;
@@ -292,6 +346,16 @@ gve_tx_burst_dqo(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
nb_descs = gve_tx_pkt_nb_data_descs(tx_pkt);
nb_descs += tso;
+
+ /* Drop packet if it doesn't adhere to hardware limits. */
+ if (unlikely(!gve_tx_validate_descs(txq, tx_pkt, nb_descs, tso))) {
+ txq->stats.dropped_pkts++;
+ txq->stats.errors++;
+ rte_pktmbuf_free(tx_pkt);
+ nb_drop++;
+ continue;
+ }
+
req_descs = nb_descs + GVE_TX_MIN_DESC_PREVENT_CACHE_OVERLAP;
/* Clean if there aren't enough descriptors to send the packet
@@ -306,12 +370,6 @@ gve_tx_burst_dqo(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
break;
}
- /* Drop packet if it doesn't adhere to hardware limits. */
- if (!gve_tx_validate_descs(tx_pkt, nb_descs, tso)) {
- txq->stats.too_many_descs++;
- break;
- }
-
if (!gve_free_compl_tags_pop(txq, &compl_tag))
break;
@@ -357,7 +415,7 @@ gve_tx_burst_dqo(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
}
if (nb_tx > 0) {
- txq->stats.packets += nb_tx;
+ txq->stats.packets += (nb_tx - nb_drop);
txq->stats.bytes += bytes;
txq->stats.errors += nb_pkts - nb_tx;
}
--
2.43.0
^ permalink raw reply related [flat|nested] 16+ messages in thread* Re: [PATCH v2 4/4] net/gve: add packet and TSO hardware constraints for DQO
2026-10-13 10:43 ` [PATCH v2 4/4] net/gve: add packet and TSO hardware constraints for DQO Tathagat Priyadarshi
@ 2026-10-14 0:50 ` Joshua Washington
0 siblings, 0 replies; 16+ messages in thread
From: Joshua Washington @ 2026-10-14 0:50 UTC (permalink / raw)
To: Tathagat Priyadarshi; +Cc: dev, stephen, stable
On Tue, Oct 13, 2026 at 3:44 AM Tathagat Priyadarshi
<tathagat.dpdk@gmail.com> wrote:
>
> The GVNIC DQO hardware restricts each packet (or MTU segment in TSO)
> to at most 10 data descriptors (GVE_TX_MAX_DATA_DESCS), beyond which
> the hardware drops traffic. In addition to this 10-descriptor limit,
> the hardware imposes strict packet sizing and header constraints.
>
> Previously, failing the descriptor limit or rte_mbuf_check() broke
> from the Tx burst loop without consuming the invalid mbuf. This caused
> tx_burst to return a short count, leading the application to retry the
> same rejected packet indefinitely and causing Head-of-Line (HOL)
> queue stalls.
>
> Enforce hardware validation and prevent HOL blocking in DPDK:
> - Validate packet and hardware constraints prior to ring space checks.
> - When packet validation or rte_mbuf_check() fails, drop and free the
> invalid mbuf with rte_pktmbuf_free(), update stats, and continue the
> burst to prevent Head-of-Line (HOL) blocking.
> - Exclude dropped packets from txq->stats.packets while returning the
> total consumed count to the caller.
> - Rename too_many_descs in struct gve_tx_stats to dropped_pkts to reflect
> all validation and malformed mbuf drop events.
> - Reject packets whose descriptor count exceeds the maximum capacity of
> the ring (accounting for reserved slots, cacheline overlap margin, and
> unreported descriptors), preventing permanent queue stalls.
> - Define GVE_TX_MIN_PKT_SIZE_DQO (17 bytes) and GVE_TX_MAX_PKT_SIZE_DQO
> (9728 bytes) to enforce wire packet boundaries for non-TSO packets
> and TSO MSS.
> - Define GVE_TX_MAX_TSO_LEN_DQO (262143 bytes) to enforce the hardware
> L4 payload limit in the 24-bit tso_total_len field.
> - In gve_tx_validate_descs():
> * Enforce packet size in [17, 9728] for non-TSO packets alongside
> the 10-descriptor limit (nb_descs <= GVE_TX_MAX_DATA_DESCS).
> * Reject TSO packets missing an L4 header (l4_len == 0).
> * Reject packet with headers (L2-L4) not contiguous in first segment.
> * Reject TSO packets without payload (pkt_len <= headlen).
> * Enforce TSO total payload length limit (pkt_len - headlen <= 262143).
> * Enforce MSS limits [88, 9728] and header length limit <= 255.
I agree with the AI code review on this patch description.
> 4. The commit message lists the diff line by line. Cut it to the
> problem (invalid packets stalled the queue; hardware limits were
> not enforced) and the fix (drop and continue; the limits enforced).
Can we trim this message down a bit, and more succinctly explain the
error condition and how this patch fixes it? If it cannot be describe
succinctly, that might be a sign that the patch should be split up
into smaller logical pieces.
>
> Fixes: 4022f9999f56 ("net/gve: support basic Tx data path for DQO")
> Fixes: f4a70b449bfa ("net/gve: add DQO Tx descriptor limit")
> Fixes: 403c671a46b6 ("net/gve: support TSO in DQO RDA")
> Cc: stable@dpdk.org
>
> Signed-off-by: Tathagat Priyadarshi <tathagat.dpdk@gmail.com>
> ---
> drivers/net/gve/base/gve_desc_dqo.h | 3 +
> drivers/net/gve/gve_ethdev.h | 2 +-
> drivers/net/gve/gve_tx_dqo.c | 92 +++++++++++++++++++++++------
> 3 files changed, 79 insertions(+), 18 deletions(-)
>
> diff --git a/drivers/net/gve/base/gve_desc_dqo.h b/drivers/net/gve/base/gve_desc_dqo.h
> index c1534959c2..53e394f01c 100644
> --- a/drivers/net/gve/base/gve_desc_dqo.h
> +++ b/drivers/net/gve/base/gve_desc_dqo.h
> @@ -12,6 +12,9 @@
>
> #define GVE_TX_MAX_HDR_SIZE_DQO 255
> #define GVE_TX_MIN_TSO_MSS_DQO 88
> +#define GVE_TX_MIN_PKT_SIZE_DQO 17
> +#define GVE_TX_MAX_PKT_SIZE_DQO 9728
> +#define GVE_TX_MAX_TSO_LEN_DQO ((256 * 1024) - 1)
>
> /* Basic TX descriptor (DTYPE 0x0C) */
> struct gve_tx_pkt_desc_dqo {
> diff --git a/drivers/net/gve/gve_ethdev.h b/drivers/net/gve/gve_ethdev.h
> index 2033f49444..d6d71c31bb 100644
> --- a/drivers/net/gve/gve_ethdev.h
> +++ b/drivers/net/gve/gve_ethdev.h
> @@ -113,7 +113,7 @@ struct gve_tx_stats {
> uint64_t packets;
> uint64_t bytes;
> uint64_t errors;
> - uint64_t too_many_descs;
> + uint64_t dropped_pkts;
Is there any reason to change this to a general "dropped_pkts" instead
of adding a different stat if there is a different error condition
that needs to be captured? These are extended stats, so if we can give
more specific drop conditions, I think we should aim to. However,
there is nothing wrong with also introducing a new aggregate
dropped_pkts stat. Not sure if adding new stats would be considered
scope creep, though.
> };
>
> struct gve_rx_stats {
> diff --git a/drivers/net/gve/gve_tx_dqo.c b/drivers/net/gve/gve_tx_dqo.c
> index eee1f10df1..94ea426961 100644
> --- a/drivers/net/gve/gve_tx_dqo.c
> +++ b/drivers/net/gve/gve_tx_dqo.c
> @@ -126,20 +126,67 @@ gve_tx_clean_descs_dqo(struct gve_tx_queue *txq, uint16_t nb_descs) {
> gve_tx_clean_dqo(txq);
> }
>
> +/* Max descriptors a single packet may use. A larger packet can never pass
> + * gve_can_tx(): one slot is always reserved, the cacheline overlap margin
> + * must stay free, and up to GVE_TX_MIN_RE_INTERVAL - 1 descriptors posted
> + * after the last report_event are not reclaimed until more are posted.
> + */
> +static inline int
> +gve_tx_max_pkt_descs(const struct gve_tx_queue *txq)
> +{
> + return (int)txq->nb_tx_desc - 1 - GVE_TX_MIN_DESC_PREVENT_CACHE_OVERLAP -
> + (GVE_TX_MIN_RE_INTERVAL - 1);
> +}
> +
> /* GVE expects at most 10 data descriptors per mtu-sized segment. Beyond this,
> * the hardware will assume the driver is malicious and stop transmitting
> * packets altogether. Validate that a packet can be sent to avoid sending
> * posting descriptors for an invalid packet.
> */
> static inline bool
> -gve_tx_validate_descs(struct rte_mbuf *tx_pkt, uint16_t nb_descs, bool is_tso)
> +gve_tx_validate_descs(const struct gve_tx_queue *txq, struct rte_mbuf *tx_pkt,
> + uint16_t nb_descs, bool is_tso)
> {
> - if (!is_tso)
> + int num_descs, seg_offset, mbuf_len;
> + int tso_segsz, headlen;
> +
> + /* Reject packets that can never fit in the ring */
> + if (unlikely(nb_descs > gve_tx_max_pkt_descs(txq)))
> + return false;
> +
> + if (!is_tso) {
> + /* Reject packets that violate hardware length constraints */
> + if (unlikely(tx_pkt->pkt_len < GVE_TX_MIN_PKT_SIZE_DQO ||
> + tx_pkt->pkt_len > GVE_TX_MAX_PKT_SIZE_DQO))
> + return false;
> return nb_descs <= GVE_TX_MAX_DATA_DESCS;
> + }
>
> - int tso_segsz = tx_pkt->tso_segsz;
> - int num_descs, seg_offset, mbuf_len;
> - int headlen = tx_pkt->l2_len + tx_pkt->l3_len + tx_pkt->l4_len;
> + headlen = tx_pkt->l2_len + tx_pkt->l3_len + tx_pkt->l4_len;
> +
> + /* Reject missing TCP header on TSO */
> + if (unlikely(tx_pkt->l4_len == 0))
> + return false;
> +
> + /* Reject packet with headers (L2-L4) not contiguous in first segment */
> + if (unlikely(tx_pkt->data_len < headlen))
> + return false;
> +
> + /* Reject TSO packet without payload */
> + if (unlikely(tx_pkt->pkt_len <= (uint32_t)headlen))
> + return false;
> +
> + /* Enforce GVE hardware TSO total length limit */
> + if (unlikely(tx_pkt->pkt_len - headlen > GVE_TX_MAX_TSO_LEN_DQO))
> + return false;
> +
> + /* Enforce GVE hardware MSS and header limits */
> + if (unlikely(tx_pkt->tso_segsz < GVE_TX_MIN_TSO_MSS_DQO ||
> + tx_pkt->tso_segsz > GVE_TX_MAX_PKT_SIZE_DQO ||
> + headlen > GVE_TX_MAX_HDR_SIZE_DQO))
> + return false;
> +
> + tso_segsz = tx_pkt->tso_segsz;
>
> /* Headers will be split into their own buffer. */
> num_descs = 1;
> @@ -147,14 +194,18 @@ gve_tx_validate_descs(struct rte_mbuf *tx_pkt, uint16_t nb_descs, bool is_tso)
> mbuf_len = tx_pkt->data_len - headlen;
>
> while (tx_pkt) {
> + int seg_remain;
> +
> if (!mbuf_len)
> goto next_mbuf;
>
> - int seg_remain = tso_segsz - seg_offset;
> + seg_remain = tso_segsz - seg_offset;
> if (num_descs == GVE_TX_MAX_DATA_DESCS && seg_remain)
> return false;
>
> if (seg_remain < mbuf_len) {
> + int mbuf_remain = tx_pkt->data_len % GVE_TX_MAX_BUF_SIZE_DQO;
> +
> seg_offset = mbuf_len % tso_segsz;
> /* The MSS is bound from above by 9728B, so a
> * single TSO segment in the middle of an mbuf
> @@ -162,7 +213,6 @@ gve_tx_validate_descs(struct rte_mbuf *tx_pkt, uint16_t nb_descs, bool is_tso)
> * is not at risk of defying this limitation.
> * Thus, such segments are ignored.
> */
> - int mbuf_remain = tx_pkt->data_len % GVE_TX_MAX_BUF_SIZE_DQO;
>
> /* For each TSO segment, HW will prepend
> * headers. The remaining bytes of this mbuf
> @@ -184,7 +234,6 @@ gve_tx_validate_descs(struct rte_mbuf *tx_pkt, uint16_t nb_descs, bool is_tso)
> mbuf_len = tx_pkt->data_len;
> }
>
> -
> return true;
> }
>
> @@ -256,6 +305,7 @@ gve_tx_burst_dqo(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
> struct rte_mbuf *tx_pkt;
> struct gve_tx_pkt *pkts;
> uint32_t last_desc_idx;
> + uint16_t nb_drop = 0;
Minor nit: maybe we can capture the number of packet sent after we
write the descriptors instead? Then we can increment in one place
instead of two, and remove the math in getting the number of packets
sent.
> uint16_t nb_tx = 0;
> uint64_t bytes = 0;
> const char *reason;
> @@ -281,9 +331,13 @@ gve_tx_burst_dqo(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
> txq->rs_thresh);
>
>
> - if (rte_mbuf_check(tx_pkt, true, &reason)) {
> + if (unlikely(rte_mbuf_check(tx_pkt, true, &reason))) {
> PMD_DRV_DP_LOG(DEBUG, "Invalid mbuf: %s", reason);
> - break;
> + txq->stats.dropped_pkts++;
> + txq->stats.errors++;
> + rte_pktmbuf_free(tx_pkt);
> + nb_drop++;
> + continue;
> }
>
> ol_flags = tx_pkt->ol_flags;
> @@ -292,6 +346,16 @@ gve_tx_burst_dqo(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
>
> nb_descs = gve_tx_pkt_nb_data_descs(tx_pkt);
> nb_descs += tso;
> +
> + /* Drop packet if it doesn't adhere to hardware limits. */
> + if (unlikely(!gve_tx_validate_descs(txq, tx_pkt, nb_descs, tso))) {
> + txq->stats.dropped_pkts++;
> + txq->stats.errors++;
> + rte_pktmbuf_free(tx_pkt);
> + nb_drop++;
> + continue;
> + }
> +
> req_descs = nb_descs + GVE_TX_MIN_DESC_PREVENT_CACHE_OVERLAP;
>
> /* Clean if there aren't enough descriptors to send the packet
> @@ -306,12 +370,6 @@ gve_tx_burst_dqo(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
> break;
> }
>
> - /* Drop packet if it doesn't adhere to hardware limits. */
> - if (!gve_tx_validate_descs(tx_pkt, nb_descs, tso)) {
> - txq->stats.too_many_descs++;
> - break;
> - }
> -
> if (!gve_free_compl_tags_pop(txq, &compl_tag))
> break;
>
> @@ -357,7 +415,7 @@ gve_tx_burst_dqo(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
> }
>
> if (nb_tx > 0) {
> - txq->stats.packets += nb_tx;
> + txq->stats.packets += (nb_tx - nb_drop);
> txq->stats.bytes += bytes;
> txq->stats.errors += nb_pkts - nb_tx;
> }
> --
> 2.43.0
>
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PATCH v2 0/4] net/gve: fix Tx queue stalls and sync DQO RDA with Linux driver
2026-10-13 10:43 ` [PATCH v2 0/4] net/gve: fix Tx " Tathagat Priyadarshi
` (3 preceding siblings ...)
2026-10-13 10:43 ` [PATCH v2 4/4] net/gve: add packet and TSO hardware constraints for DQO Tathagat Priyadarshi
@ 2026-10-13 15:22 ` Stephen Hemminger
4 siblings, 0 replies; 16+ messages in thread
From: Stephen Hemminger @ 2026-10-13 15:22 UTC (permalink / raw)
To: Tathagat Priyadarshi; +Cc: dev, joshwash
On Tue, 13 Oct 2026 10:43:44 +0000
Tathagat Priyadarshi <tathagat.dpdk@gmail.com> wrote:
> This patch series aligns the Google Virtual NIC (gve) DQO RDA transmit path
> with the upstream Linux kernel driver (drivers/net/ethernet/google/gve/) to
> resolve transmit queue stalls and hardware validation issues:
>
> 1. net/gve: fix DQO descriptor cleaning loop bound
> Fixes the off-by-one error in gve_tx_clean_descs_dqo()
> ("while (--nb_descs)" -> "while (nb_descs--)") and avoids stalling
> the completion queue on unrecognized descriptor types.
>
> 2. net/gve: fix DQO Tx queue stall on multi-seg and TSO
> Tracks descriptor intervals with last_re_idx per packet against
> GVE_TX_MIN_RE_INTERVAL (32) instead of counting packets at burst end,
> preventing descriptor ring starvation and queue deadlocks under
> multi-segment and TSO traffic.
>
> 3. net/gve: add cacheline overlap margin to DQO Tx ring
> Enforces GVE_TX_MIN_DESC_PREVENT_CACHE_OVERLAP (4 descriptors) margin
> in ring space checks, avoiding CPU and device DMA cacheline collisions.
>
> 4. net/gve: add packet and TSO hardware constraints for DQO
> Augments gve_tx_validate_descs() (which enforces the 10-descriptor
> limit per packet or MTU segment) with the remaining hardware length
> constraints: [17, 9728] bytes for non-TSO packets, maximum MSS limit
> (9728 bytes), TSO total length limit (262143 bytes), and validates
> header contiguity and payload presence on TSO packets. Invalid packets
> are dropped instead of stalling the queue.
Looks good you may want to address these comments from AI.
Subject: Re: [PATCH v2 0/4] net/gve: DQO Tx fixes
Applies cleanly to main; every commit builds with -Dwerror=true.
v2 drops invalid packets and continues instead of stalling the burst,
pops the completion tag before writing the TSO context descriptor,
and adds the Fixes tags. All three Fixes hashes resolve.
Info
[PATCH v2 2/4] net/gve: fix DQO Tx queue stall on multi-seg and TSO
1. Ringing the doorbell each time report_event is set is a
performance change, not part of the stall fix. Descriptors posted
earlier in the burst are reclaimed on the next burst either way,
and Linux rings once per xmit batch. It is harmless, but for a
stable backport it is better as a separate patch with numbers. At
least drop "amortizing MMIO writes": it adds MMIO writes.
[PATCH v2 4/4] net/gve: add packet and TSO hardware constraints for DQO
2. tso_total_len is 24 bits wide; 262143 is the hardware limit (see
the comment in the Linux gve_prep_tso()), not the field width.
Reword: "Define GVE_TX_MAX_TSO_LEN_DQO (262143), the hardware limit
on TSO payload length."
3. dropped_pkts is not in tx_xstats_name_offset[], so nothing reports
it; too_many_descs had the same problem. Export it:
{ "dropped_pkts", TX_QUEUE_STATS_OFFSET(dropped_pkts) },
or drop the counter and rely on errors.
4. The commit message lists the diff line by line. Cut it to the
problem (invalid packets stalled the queue; hardware limits were
not enforced) and the fix (drop and continue; the limits enforced).
Reviewed-by: Stephen Hemminger <stephen@networkplumber.org>
^ permalink raw reply [flat|nested] 16+ messages in thread