* FAILED: patch "[PATCH] gve: fix Rx queue stall on alloc failure" failed to apply to 6.1-stable tree
@ 2026-07-29 14:41 gregkh
2026-07-31 15:44 ` [PATCH 6.1.y] gve: fix Rx queue stall on alloc failure Eddie Phillips
2026-07-31 17:52 ` [PATCH v2 " Eddie Phillips
0 siblings, 2 replies; 5+ messages in thread
From: gregkh @ 2026-07-29 14:41 UTC (permalink / raw)
To: eddiephillips, hramamurthy, jordanrhee, kuba, przemyslaw.kitszel; +Cc: stable
The patch below does not apply to the 6.1-stable tree.
If someone wants it applied there, or to any other stable or longterm
tree, then please email the backport, including the original git commit
id to <stable@vger.kernel.org>.
To reproduce the conflict and resubmit, you may use the following commands:
git fetch https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/ linux-6.1.y
git checkout FETCH_HEAD
git cherry-pick -x b65352a1bac64442ad95e64f385b40ccb9f1b0db
# <resolve conflicts, build, test, etc.>
git commit -s
git send-email --to '<stable@vger.kernel.org>' --in-reply-to '2026072934-rule-library-9954@gregkh' --subject-prefix 'PATCH 6.1.y' 'HEAD^..'
Possible dependencies:
thanks,
greg k-h
------------------ original commit in Linus's tree ------------------
From b65352a1bac64442ad95e64f385b40ccb9f1b0db Mon Sep 17 00:00:00 2001
From: Eddie Phillips <eddiephillips@google.com>
Date: Thu, 9 Jul 2026 21:19:06 +0000
Subject: [PATCH] gve: fix Rx queue stall on alloc failure
When the system is under extreme memory pressure, page allocations can
fail during the Rx buffer refill loop. If the number of buffers posted
to hardware falls below a critical low threshold and the refill loop
exits due to allocation failures, the queue can stall:
1. The device drops incoming packets because there are no descriptors.
2. Since no packets are processed, no Rx completions are generated.
3. Because no completions occur, NAPI is never scheduled, preventing
the refill loop from running again even after memory is freed.
This results in a permanent queue stall.
Resolve this by introducing a starvation recovery timer for each Rx queue.
If the number of buffers posted to hardware falls below a critical low
threshold, start a timer to periodically reschedule NAPI. Once NAPI runs
and successfully refills the queue above the threshold, the timer is
not rescheduled.
The threshold is set to 32 because a single maximum-sized Receive Segment
Coalescing (RSC) packet can consume up to 19 descriptors in the Rx path.
Lower thresholds (such as 8 or 16) would be insufficient to process a
complete maximum-sized RSC packet, risking packet drops or unexpected
hardware behavior under memory pressure. Setting the threshold to 32
guarantees a safe margin to handle at least one full RSC packet.
Cc: stable@vger.kernel.org
Fixes: 9b8dd5e5ea48 ("gve: DQO: Add RX path")
Reviewed-by: Jordan Rhee <jordanrhee@google.com>
Signed-off-by: Eddie Phillips <eddiephillips@google.com>
Signed-off-by: Harshitha Ramamurthy <hramamurthy@google.com>
Reviewed-by: Przemek Kitszel <przemyslaw.kitszel@intel.com>
Link: https://patch.msgid.link/20260709211906.3322883-1-hramamurthy@google.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
diff --git a/drivers/net/ethernet/google/gve/gve.h b/drivers/net/ethernet/google/gve/gve.h
index 1d66d3834f7e..c280ff35ee77 100644
--- a/drivers/net/ethernet/google/gve/gve.h
+++ b/drivers/net/ethernet/google/gve/gve.h
@@ -13,6 +13,7 @@
#include <linux/netdevice.h>
#include <linux/net_tstamp.h>
#include <linux/pci.h>
+#include <linux/timer.h>
#include <linux/ptp_clock_kernel.h>
#include <linux/u64_stats_sync.h>
#include <net/page_pool/helpers.h>
@@ -41,6 +42,7 @@
/* Interval to schedule a stats report update, 20000ms. */
#define GVE_STATS_REPORT_TIMER_PERIOD 20000
+#define GVE_RX_NAPI_RESCHED_MS 20 /* msecs */
/* Numbers of NIC tx/rx stats in stats report. */
#define NIC_TX_STATS_REPORT_NUM 0
@@ -341,6 +343,7 @@ struct gve_rx_ring {
struct xdp_rxq_info xdp_rxq;
struct xsk_buff_pool *xsk_pool;
struct page_frag_cache page_cache; /* Page cache to allocate XDP frames */
+ struct timer_list starvation_timer; /* for queue starvation recovery */
};
/* A TX desc ring entry */
diff --git a/drivers/net/ethernet/google/gve/gve_rx_dqo.c b/drivers/net/ethernet/google/gve/gve_rx_dqo.c
index 02cba280d81a..8271f731a91f 100644
--- a/drivers/net/ethernet/google/gve/gve_rx_dqo.c
+++ b/drivers/net/ethernet/google/gve/gve_rx_dqo.c
@@ -18,6 +18,16 @@
#include <net/tcp.h>
#include <net/xdp_sock_drv.h>
+static void gve_rx_starvation_timer(struct timer_list *t)
+{
+ struct gve_rx_ring *rx = timer_container_of(rx, t, starvation_timer);
+ struct gve_priv *priv = rx->gve;
+ struct gve_notify_block *block;
+
+ block = &priv->ntfy_blocks[rx->ntfy_id];
+ napi_schedule(&block->napi);
+}
+
static void gve_rx_free_hdr_bufs(struct gve_priv *priv, struct gve_rx_ring *rx)
{
struct device *hdev = &priv->pdev->dev;
@@ -120,6 +130,7 @@ void gve_rx_stop_ring_dqo(struct gve_priv *priv, int idx)
if (rx->dqo.page_pool)
page_pool_disable_direct_recycling(rx->dqo.page_pool);
+ timer_shutdown_sync(&rx->starvation_timer);
gve_remove_napi(priv, ntfy_idx);
gve_rx_remove_from_block(priv, idx);
gve_rx_reset_ring_dqo(priv, idx);
@@ -208,8 +219,10 @@ static int gve_rx_alloc_hdr_bufs(struct gve_priv *priv, struct gve_rx_ring *rx,
void gve_rx_start_ring_dqo(struct gve_priv *priv, int idx)
{
int ntfy_idx = gve_rx_idx_to_ntfy(priv, idx);
+ struct gve_rx_ring *rx = &priv->rx[idx];
gve_rx_add_to_block(priv, idx);
+ timer_setup(&rx->starvation_timer, gve_rx_starvation_timer, 0);
gve_add_napi(priv, ntfy_idx, gve_napi_poll_dqo);
}
@@ -365,6 +378,7 @@ void gve_rx_post_buffers_dqo(struct gve_rx_ring *rx)
struct gve_rx_compl_queue_dqo *complq = &rx->dqo.complq;
struct gve_rx_buf_queue_dqo *bufq = &rx->dqo.bufq;
struct gve_priv *priv = rx->gve;
+ u32 num_bufs_avail_to_hw;
u32 num_avail_slots;
u32 num_full_slots;
u32 num_posted = 0;
@@ -400,6 +414,26 @@ void gve_rx_post_buffers_dqo(struct gve_rx_ring *rx)
}
rx->fill_cnt += num_posted;
+
+ /* If the queue has fewer than GVE_RX_BUF_THRESH_DQO descriptors
+ * visible to the hardware, the hardware is in danger of starving
+ * and cannot trigger interrupts.
+ *
+ * We use a threshold of 32 because a single maximum-sized RSC
+ * packet can consume up to 19 descriptors in the Rx path. Lower
+ * thresholds (e.g., 8 or 16) would be unsafe as they could cause
+ * the device to drop/stall on a maximum-sized RSC packet.
+ *
+ * Start the timer to periodically reschedule NAPI and recover.
+ */
+ num_bufs_avail_to_hw =
+ ((bufq->tail & ~(GVE_RX_BUF_THRESH_DQO - 1)) -
+ bufq->head) & bufq->mask;
+
+ if (num_bufs_avail_to_hw < GVE_RX_BUF_THRESH_DQO) {
+ mod_timer(&rx->starvation_timer,
+ jiffies + msecs_to_jiffies(GVE_RX_NAPI_RESCHED_MS));
+ }
}
static void gve_rx_skb_csum(struct sk_buff *skb,
^ permalink raw reply related [flat|nested] 5+ messages in thread
* [PATCH 6.1.y] gve: fix Rx queue stall on alloc failure
2026-07-29 14:41 FAILED: patch "[PATCH] gve: fix Rx queue stall on alloc failure" failed to apply to 6.1-stable tree gregkh
@ 2026-07-31 15:44 ` Eddie Phillips
2026-07-31 16:58 ` Eddie Phillips
2026-08-01 1:40 ` Sasha Levin
2026-07-31 17:52 ` [PATCH v2 " Eddie Phillips
1 sibling, 2 replies; 5+ messages in thread
From: Eddie Phillips @ 2026-07-31 15:44 UTC (permalink / raw)
To: stable
Cc: Eddie Phillips, Jordan Rhee, Harshitha Ramamurthy,
Przemek Kitszel, Jakub Kicinski
When the system is under extreme memory pressure, page allocations can
fail during the Rx buffer refill loop. If the number of buffers posted
to hardware falls below a critical low threshold and the refill loop
exits due to allocation failures, the queue can stall:
1. The device drops incoming packets because there are no descriptors.
2. Since no packets are processed, no Rx completions are generated.
3. Because no completions occur, NAPI is never scheduled, preventing
the refill loop from running again even after memory is freed.
This results in a permanent queue stall.
Resolve this by introducing a starvation recovery timer for each Rx queue.
If the number of buffers posted to hardware falls below a critical low
threshold, start a timer to periodically reschedule NAPI. Once NAPI runs
and successfully refills the queue above the threshold, the timer is
not rescheduled.
The threshold is set to 32 because a single maximum-sized Receive Segment
Coalescing (RSC) packet can consume up to 19 descriptors in the Rx path.
Lower thresholds (such as 8 or 16) would be insufficient to process a
complete maximum-sized RSC packet, risking packet drops or unexpected
hardware behavior under memory pressure. Setting the threshold to 32
guarantees a safe margin to handle at least one full RSC packet.
Cc: stable@vger.kernel.org
Fixes: 9b8dd5e5ea48 ("gve: DQO: Add RX path")
Reviewed-by: Jordan Rhee <jordanrhee@google.com>
Signed-off-by: Eddie Phillips <eddiephillips@google.com>
Signed-off-by: Harshitha Ramamurthy <hramamurthy@google.com>
Reviewed-by: Przemek Kitszel <przemyslaw.kitszel@intel.com>
Link: https://patch.msgid.link/20260709211906.3322883-1-hramamurthy@google.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
(cherry picked from commit b65352a1bac64442ad95e64f385b40ccb9f1b0db)
---
drivers/net/ethernet/google/gve/gve.h | 2 ++
1 file changed, 2 insertions(+)
diff --git a/drivers/net/ethernet/google/gve/gve.h b/drivers/net/ethernet/google/gve/gve.h
index c5e1312b9283..056f57166362 100644
--- a/drivers/net/ethernet/google/gve/gve.h
+++ b/drivers/net/ethernet/google/gve/gve.h
@@ -10,6 +10,7 @@
#include <linux/dma-mapping.h>
#include <linux/netdevice.h>
#include <linux/pci.h>
+#include <linux/timer.h>
#include <linux/u64_stats_sync.h>
#include "gve_desc.h"
@@ -226,6 +227,7 @@ struct gve_rx_ring {
struct u64_stats_sync statss; /* sync stats for 32bit archs */
struct gve_rx_ctx ctx; /* Info for packet currently being processed in this ring. */
+ struct timer_list starvation_timer; /* for queue starvation recovery */
};
/* A TX desc ring entry */
--
2.55.0.571.g244d577d93-goog
^ permalink raw reply related [flat|nested] 5+ messages in thread
* Re: [PATCH 6.1.y] gve: fix Rx queue stall on alloc failure
2026-07-31 15:44 ` [PATCH 6.1.y] gve: fix Rx queue stall on alloc failure Eddie Phillips
@ 2026-07-31 16:58 ` Eddie Phillips
2026-08-01 1:40 ` Sasha Levin
1 sibling, 0 replies; 5+ messages in thread
From: Eddie Phillips @ 2026-07-31 16:58 UTC (permalink / raw)
To: stable; +Cc: Jordan Rhee, Harshitha Ramamurthy, Przemek Kitszel,
Jakub Kicinski
On Fri, Jul 31, 2026 at 8:45 AM Eddie Phillips <eddiephillips@google.com> wrote:
I have made an error here and failed to include changes in
gve_rx_dqo.c and gve_main.c.
A v2 will be sent, sorry for the noise.
^ permalink raw reply [flat|nested] 5+ messages in thread
* [PATCH v2 6.1.y] gve: fix Rx queue stall on alloc failure
2026-07-29 14:41 FAILED: patch "[PATCH] gve: fix Rx queue stall on alloc failure" failed to apply to 6.1-stable tree gregkh
2026-07-31 15:44 ` [PATCH 6.1.y] gve: fix Rx queue stall on alloc failure Eddie Phillips
@ 2026-07-31 17:52 ` Eddie Phillips
1 sibling, 0 replies; 5+ messages in thread
From: Eddie Phillips @ 2026-07-31 17:52 UTC (permalink / raw)
To: stable
Cc: Eddie Phillips, Jordan Rhee, Harshitha Ramamurthy,
Przemek Kitszel, Jakub Kicinski
When the system is under extreme memory pressure, page allocations can
fail during the Rx buffer refill loop. If the number of buffers posted
to hardware falls below a critical low threshold and the refill loop
exits due to allocation failures, the queue can stall:
1. The device drops incoming packets because there are no descriptors.
2. Since no packets are processed, no Rx completions are generated.
3. Because no completions occur, NAPI is never scheduled, preventing
the refill loop from running again even after memory is freed.
This results in a permanent queue stall.
Resolve this by introducing a starvation recovery timer for each Rx queue.
If the number of buffers posted to hardware falls below a critical low
threshold, start a timer to periodically reschedule NAPI. Once NAPI runs
and successfully refills the queue above the threshold, the timer is
not rescheduled.
The threshold is set to 32 because a single maximum-sized Receive Segment
Coalescing (RSC) packet can consume up to 19 descriptors in the Rx path.
Lower thresholds (such as 8 or 16) would be insufficient to process a
complete maximum-sized RSC packet, risking packet drops or unexpected
hardware behavior under memory pressure. Setting the threshold to 32
guarantees a safe margin to handle at least one full RSC packet.
Cc: stable@vger.kernel.org
Fixes: 9b8dd5e5ea48 ("gve: DQO: Add RX path")
Reviewed-by: Jordan Rhee <jordanrhee@google.com>
Signed-off-by: Eddie Phillips <eddiephillips@google.com>
Signed-off-by: Harshitha Ramamurthy <hramamurthy@google.com>
Reviewed-by: Przemek Kitszel <przemyslaw.kitszel@intel.com>
Link: https://patch.msgid.link/20260709211906.3322883-1-hramamurthy@google.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
(cherry picked from commit b65352a1bac64442ad95e64f385b40ccb9f1b0db)
---
Changes in v2:
- v1 Link: https://lore.kernel.org/stable/20260731154449.2755799-1-eddiephillips@google.com/
- Included backports in gve_main.c and gve_rx_dqo.c. v1 only included
gve.h by mistake. Sorry for the noise.
---
drivers/net/ethernet/google/gve/gve.h | 3 ++
drivers/net/ethernet/google/gve/gve_main.c | 3 ++
drivers/net/ethernet/google/gve/gve_rx_dqo.c | 33 ++++++++++++++++++++
3 files changed, 39 insertions(+)
diff --git a/drivers/net/ethernet/google/gve/gve.h b/drivers/net/ethernet/google/gve/gve.h
index c5e1312b9283..7de492b0d361 100644
--- a/drivers/net/ethernet/google/gve/gve.h
+++ b/drivers/net/ethernet/google/gve/gve.h
@@ -10,6 +10,7 @@
#include <linux/dma-mapping.h>
#include <linux/netdevice.h>
#include <linux/pci.h>
+#include <linux/timer.h>
#include <linux/u64_stats_sync.h>
#include "gve_desc.h"
@@ -35,6 +36,7 @@
/* Interval to schedule a stats report update, 20000ms. */
#define GVE_STATS_REPORT_TIMER_PERIOD 20000
+#define GVE_RX_NAPI_RESCHED_MS 20 /* msecs */
/* Numbers of NIC tx/rx stats in stats report. */
#define NIC_TX_STATS_REPORT_NUM 0
@@ -226,6 +228,7 @@ struct gve_rx_ring {
struct u64_stats_sync statss; /* sync stats for 32bit archs */
struct gve_rx_ctx ctx; /* Info for packet currently being processed in this ring. */
+ struct timer_list starvation_timer; /* for queue starvation recovery */
};
/* A TX desc ring entry */
diff --git a/drivers/net/ethernet/google/gve/gve_main.c b/drivers/net/ethernet/google/gve/gve_main.c
index 209e9526a6fd..fc2a516a3bee 100644
--- a/drivers/net/ethernet/google/gve/gve_main.c
+++ b/drivers/net/ethernet/google/gve/gve_main.c
@@ -529,6 +529,9 @@ static void gve_remove_napi(struct gve_priv *priv, int ntfy_idx)
{
struct gve_notify_block *block = &priv->ntfy_blocks[ntfy_idx];
+ if (block->rx && !gve_is_gqi(priv))
+ timer_shutdown_sync(&block->rx->starvation_timer);
+
netif_napi_del(&block->napi);
disable_irq(block->irq);
}
diff --git a/drivers/net/ethernet/google/gve/gve_rx_dqo.c b/drivers/net/ethernet/google/gve/gve_rx_dqo.c
index 0a36b284de10..adae0f5181ea 100644
--- a/drivers/net/ethernet/google/gve/gve_rx_dqo.c
+++ b/drivers/net/ethernet/google/gve/gve_rx_dqo.c
@@ -16,6 +16,16 @@
#include <net/ipv6.h>
#include <net/tcp.h>
+static void gve_rx_starvation_timer(struct timer_list *t)
+{
+ struct gve_rx_ring *rx = from_timer(rx, t, starvation_timer);
+ struct gve_priv *priv = rx->gve;
+ struct gve_notify_block *block;
+
+ block = &priv->ntfy_blocks[rx->ntfy_id];
+ napi_schedule(&block->napi);
+}
+
static int gve_buf_ref_cnt(struct gve_rx_buf_state_dqo *bs)
{
return page_count(bs->page_info.page) - bs->page_info.pagecnt_bias;
@@ -185,6 +195,7 @@ static void gve_rx_free_ring_dqo(struct gve_priv *priv, int idx)
completion_queue_slots = rx->dqo.complq.mask + 1;
buffer_queue_slots = rx->dqo.bufq.mask + 1;
+ timer_shutdown_sync(&rx->starvation_timer);
gve_rx_remove_from_block(priv, idx);
if (rx->q_resources) {
@@ -237,6 +248,7 @@ static int gve_rx_alloc_ring_dqo(struct gve_priv *priv, int idx)
memset(rx, 0, sizeof(*rx));
rx->gve = priv;
rx->q_num = idx;
+ timer_setup(&rx->starvation_timer, gve_rx_starvation_timer, 0);
rx->dqo.bufq.mask = buffer_queue_slots - 1;
rx->dqo.complq.num_free_slots = completion_queue_slots;
rx->dqo.complq.mask = completion_queue_slots - 1;
@@ -337,6 +349,7 @@ void gve_rx_post_buffers_dqo(struct gve_rx_ring *rx)
u32 num_avail_slots;
u32 num_full_slots;
u32 num_posted = 0;
+ u32 num_bufs_avail_to_hw;
num_full_slots = (bufq->tail - bufq->head) & bufq->mask;
num_avail_slots = bufq->mask - num_full_slots;
@@ -374,6 +387,26 @@ void gve_rx_post_buffers_dqo(struct gve_rx_ring *rx)
}
rx->fill_cnt += num_posted;
+
+ /* If the queue has fewer than GVE_RX_BUF_THRESH_DQO descriptors
+ * visible to the hardware, the hardware is in danger of starving
+ * and cannot trigger interrupts.
+ *
+ * We use a threshold of 32 because a single maximum-sized RSC
+ * packet can consume up to 19 descriptors in the Rx path. Lower
+ * thresholds (e.g., 8 or 16) would be unsafe as they could cause
+ * the device to drop/stall on a maximum-sized RSC packet.
+ *
+ * Start the timer to periodically reschedule NAPI and recover.
+ */
+ num_bufs_avail_to_hw =
+ ((bufq->tail & ~(GVE_RX_BUF_THRESH_DQO - 1)) -
+ bufq->head) & bufq->mask;
+
+ if (num_bufs_avail_to_hw < GVE_RX_BUF_THRESH_DQO) {
+ mod_timer(&rx->starvation_timer,
+ jiffies + msecs_to_jiffies(GVE_RX_NAPI_RESCHED_MS));
+ }
}
static void gve_try_recycle_buf(struct gve_priv *priv, struct gve_rx_ring *rx,
--
2.55.0.571.g244d577d93-goog
^ permalink raw reply related [flat|nested] 5+ messages in thread
* Re: [PATCH 6.1.y] gve: fix Rx queue stall on alloc failure
2026-07-31 15:44 ` [PATCH 6.1.y] gve: fix Rx queue stall on alloc failure Eddie Phillips
2026-07-31 16:58 ` Eddie Phillips
@ 2026-08-01 1:40 ` Sasha Levin
1 sibling, 0 replies; 5+ messages in thread
From: Sasha Levin @ 2026-08-01 1:40 UTC (permalink / raw)
To: stable
Cc: Sasha Levin, Eddie Phillips, Jordan Rhee, Harshitha Ramamurthy,
Przemek Kitszel, Jakub Kicinski
On Fri, Jul 31, 2026 at 08:44:49AM -0700, Eddie Phillips wrote:
> (cherry picked from commit b65352a1bac64442ad95e64f385b40ccb9f1b0db)
> ---
> drivers/net/ethernet/google/gve/gve.h | 2 ++
> 1 file changed, 2 insertions(+)
This 6.1.y version only contains the gve.h hunk: the <linux/timer.h>
include and the "struct timer_list starvation_timer" member. The
starvation timer callback, the timer_setup(), the mod_timer() threshold
check in gve_rx_post_buffers_dqo() and the timer_shutdown_sync() teardown
are all missing, so it applies cleanly but is a no-op that only adds a
dead struct member - the queue stall stays unfixed. (The new member is
also indented with spaces rather than a tab.) Could you send a v2 with
the complete backport?
While you're at it, could you send a 6.6.y backport too? 6.6 failed the
same way and still doesn't carry b65352a1bac6 ("gve: fix Rx queue stall
on alloc failure") at all. 6.6 and 6.1 have the same helpers as 5.15, so
the shape of your 5.15.y patch (timer_setup() in gve_rx_alloc_ring_dqo(),
timer_shutdown_sync() in gve_rx_free_ring_dqo() plus gve_remove_napi())
should carry over with little change.
Your 5.15.y patch itself looks good, but I'm holding it until the 6.6 and
6.1 backports are ready - otherwise the older tree ships the fix while
the newer ones stay broken. Once all three are in hand I'll queue them
together, newest tree first.
--
Thanks,
Sasha
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2026-08-01 1:40 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-07-29 14:41 FAILED: patch "[PATCH] gve: fix Rx queue stall on alloc failure" failed to apply to 6.1-stable tree gregkh
2026-07-31 15:44 ` [PATCH 6.1.y] gve: fix Rx queue stall on alloc failure Eddie Phillips
2026-07-31 16:58 ` Eddie Phillips
2026-08-01 1:40 ` Sasha Levin
2026-07-31 17:52 ` [PATCH v2 " Eddie Phillips
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).