[PATCH net 0/2] gve: fix crashes on invalid TX queue indices

[PATCH net 0/2] gve: fix crashes on invalid TX queue indices

[Joshua Washington]

From: Ankit Garg <nktgrg@google.com>

This series fixes a kernel panic in the GVE driver caused by
out-of-bounds array access when the network stack provides an invalid
TX queue index.

The issue impacts both GQI and DQO queue formats. For both cases, the
driver is updated to validate the queue index and drop the packet if
the index is out of range.

Ankit Garg (2):
  gve: drop packets on invalid queue indices in GQI TX path
  gve: drop packets on invalid queue indices in DQO TX path

 drivers/net/ethernet/google/gve/gve_tx.c     | 12 +++++++++---
 drivers/net/ethernet/google/gve/gve_tx_dqo.c |  9 ++++++++-
 2 files changed, 17 insertions(+), 4 deletions(-)

-- 
2.52.0.351.gbe84eed79e-goog

[PATCH net 1/2] gve: drop packets on invalid queue indices in GQI TX path

[Joshua Washington]

From: Ankit Garg <nktgrg@google.com>

The driver currently assumes that the skb queue mapping is within the
range of configured TX queues. However, the stack may provide an index
that exceeds the number of active queues.

In GQI format, an out-of-range index triggered a warning but continues
to dereference tx array, potentially causing a crash like below:

[    6.700970] Call Trace:
[    6.703576]  ? __warn+0x94/0xe0
[    6.706863]  ? gve_tx+0xa9f/0xc30 [gve]
[    6.712223]  ? gve_tx+0xa9f/0xc30 [gve]
[    6.716197]  ? report_bug+0xb1/0xe0
[    6.721195]  ? do_error_trap+0x9e/0xd0
[    6.725084]  ? do_invalid_op+0x36/0x40
[    6.730355]  ? gve_tx+0xa9f/0xc30 [gve]
[    6.734353]  ? invalid_op+0x14/0x20
[    6.739372]  ? gve_tx+0xa9f/0xc30 [gve]
[    6.743350]  ? netif_skb_features+0xcf/0x2a0
[    6.749137]  dev_hard_start_xmit+0xd7/0x240

Change that behavior to log a warning and drop the packet.

Cc: stable@vger.kernel.org
Fixes: f5cedc84a30d ("gve: Add transmit and receive support")
Signed-off-by: Ankit Garg <nktgrg@google.com>
Reviewed-by: Harshitha Ramamurthy <hramamurthy@google.com>
Signed-off-by: Joshua Washington <joshwash@google.com>
---
 drivers/net/ethernet/google/gve/gve_tx.c | 12 +++++++++---
 1 file changed, 9 insertions(+), 3 deletions(-)

diff --git a/drivers/net/ethernet/google/gve/gve_tx.c b/drivers/net/ethernet/google/gve/gve_tx.c
index 97efc8d..30d1686 100644
--- a/drivers/net/ethernet/google/gve/gve_tx.c
+++ b/drivers/net/ethernet/google/gve/gve_tx.c
@@ -739,12 +739,18 @@ drop:
 netdev_tx_t gve_tx(struct sk_buff *skb, struct net_device *dev)
 {
 	struct gve_priv *priv = netdev_priv(dev);
+	u16 qid = skb_get_queue_mapping(skb);
 	struct gve_tx_ring *tx;
 	int nsegs;
 
-	WARN(skb_get_queue_mapping(skb) >= priv->tx_cfg.num_queues,
-	     "skb queue index out of range");
-	tx = &priv->tx[skb_get_queue_mapping(skb)];
+	if (unlikely(qid >= priv->tx_cfg.num_queues)) {
+		net_warn_ratelimited("%s: skb qid %d out of range, num tx queue %d. dropping packet",
+				     dev->name, qid, priv->tx_cfg.num_queues);
+		dev_kfree_skb_any(skb);
+		return NETDEV_TX_OK;
+	}
+
+	tx = &priv->tx[qid];
 	if (unlikely(gve_maybe_stop_tx(priv, tx, skb))) {
 		/* We need to ring the txq doorbell -- we have stopped the Tx
 		 * queue for want of resources, but prior calls to gve_tx()
-- 
2.52.0.351.gbe84eed79e-goog

[PATCH net 2/2] gve: drop packets on invalid queue indices in DQO TX path

[Joshua Washington]

From: Ankit Garg <nktgrg@google.com>

The driver currently assumes that the skb queue mapping is within the
range of configured TX queues. However, the stack may provide an index
that exceeds the number of active queues.

In DQO format, driver doesn't perform any validation and continues to
dereference tx array, potentially causing a crash like below (trace is
from GQI format, but how we handle OOB queue is same in both formats).

[    6.700970] Call Trace:
[    6.703576]  ? __warn+0x94/0xe0
[    6.706863]  ? gve_tx+0xa9f/0xc30 [gve]
[    6.712223]  ? gve_tx+0xa9f/0xc30 [gve]
[    6.716197]  ? report_bug+0xb1/0xe0
[    6.721195]  ? do_error_trap+0x9e/0xd0
[    6.725084]  ? do_invalid_op+0x36/0x40
[    6.730355]  ? gve_tx+0xa9f/0xc30 [gve]
[    6.734353]  ? invalid_op+0x14/0x20
[    6.739372]  ? gve_tx+0xa9f/0xc30 [gve]
[    6.743350]  ? netif_skb_features+0xcf/0x2a0
[    6.749137]  dev_hard_start_xmit+0xd7/0x240

Change that behavior to log a warning and drop the packet.

Cc: stable@vger.kernel.org
Fixes: a57e5de476be ("gve: DQO: Add TX path")
Signed-off-by: Ankit Garg <nktgrg@google.com>
Reviewed-by: Harshitha Ramamurthy <hramamurthy@google.com>
Signed-off-by: Joshua Washington <joshwash@google.com>
---
 drivers/net/ethernet/google/gve/gve_tx_dqo.c | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/drivers/net/ethernet/google/gve/gve_tx_dqo.c b/drivers/net/ethernet/google/gve/gve_tx_dqo.c
index 40b89b3..8ebcc84 100644
--- a/drivers/net/ethernet/google/gve/gve_tx_dqo.c
+++ b/drivers/net/ethernet/google/gve/gve_tx_dqo.c
@@ -1045,9 +1045,16 @@ static void gve_xsk_reorder_queue_pop_dqo(struct gve_tx_ring *tx)
 netdev_tx_t gve_tx_dqo(struct sk_buff *skb, struct net_device *dev)
 {
 	struct gve_priv *priv = netdev_priv(dev);
+	u16 qid = skb_get_queue_mapping(skb);
 	struct gve_tx_ring *tx;

-	tx = &priv->tx[skb_get_queue_mapping(skb)];
+	if (unlikely(qid >= priv->tx_cfg.num_queues)) {
+		net_warn_ratelimited("%s: skb qid %d out of range, num tx queue %d. dropping packet",
+				     dev->name, qid, priv->tx_cfg.num_queues);
+		dev_kfree_skb_any(skb);
+		return NETDEV_TX_OK;
+	}
+	tx = &priv->tx[qid];
 	if (unlikely(gve_try_tx_skb(priv, tx, skb) < 0)) {
 		/* We need to ring the txq doorbell -- we have stopped the Tx
 		 * queue for want of resources, but prior calls to gve_tx()
--
2.52.0.351.gbe84eed79e-goog

Re: [PATCH net 0/2] gve: fix crashes on invalid TX queue indices

[Jakub Kicinski]

On Mon,  5 Jan 2026 15:25:02 -0800 Joshua Washington wrote:
> This series fixes a kernel panic in the GVE driver caused by
> out-of-bounds array access when the network stack provides an invalid
> TX queue index.

Do you know how? I seem to recall we had such issues due to bugs
in the qdisc layer, most of which were fixed.

Fixing this at the source, if possible, would be far preferable
to sprinkling this condition to all the drivers.

Re: [PATCH net 0/2] gve: fix crashes on invalid TX queue indices

[Ankit Garg]

On Tue, Jan 6, 2026 at 6:22 PM Jakub Kicinski <kuba@kernel.org> wrote:
>
> On Mon,  5 Jan 2026 15:25:02 -0800 Joshua Washington wrote:
> > This series fixes a kernel panic in the GVE driver caused by
> > out-of-bounds array access when the network stack provides an invalid
> > TX queue index.
>
> Do you know how? I seem to recall we had such issues due to bugs
> in the qdisc layer, most of which were fixed.
>
> Fixing this at the source, if possible, would be far preferable
> to sprinkling this condition to all the drivers.
That matches our observation—we have encountered this panic on older
kernels (specifically Rocky Linux 8) but have not been able to
reproduce it on recent upstream kernels.

Could you point us to the specific qdisc fixes you recall? We'd like
to verify if the issue we are seeing on the older kernel is indeed one
of those known/fixed bugs.

If it turns out this is fully resolved in the core network stack
upstream, we can drop this patch for the mainline driver. However, if
there is ambiguity, do you think there is value in keeping this check
to prevent the driver from crashing on invalid input?

Thanks,
Ankit Garg

Re: [PATCH net 0/2] gve: fix crashes on invalid TX queue indices

[Jakub Kicinski]

On Thu, 8 Jan 2026 07:35:59 -0800 Ankit Garg wrote:
> On Tue, Jan 6, 2026 at 6:22 PM Jakub Kicinski <kuba@kernel.org> wrote:
> > On Mon,  5 Jan 2026 15:25:02 -0800 Joshua Washington wrote:  
> > > This series fixes a kernel panic in the GVE driver caused by
> > > out-of-bounds array access when the network stack provides an invalid
> > > TX queue index.  
> >
> > Do you know how? I seem to recall we had such issues due to bugs
> > in the qdisc layer, most of which were fixed.
> >
> > Fixing this at the source, if possible, would be far preferable
> > to sprinkling this condition to all the drivers.  
> 
> That matches our observation—we have encountered this panic on older
> kernels (specifically Rocky Linux 8) but have not been able to
> reproduce it on recent upstream kernels.
> 
> Could you point us to the specific qdisc fixes you recall? We'd like
> to verify if the issue we are seeing on the older kernel is indeed one
> of those known/fixed bugs.

Very old - ac5b70198adc25

> If it turns out this is fully resolved in the core network stack
> upstream, we can drop this patch for the mainline driver. However, if
> there is ambiguity, do you think there is value in keeping this check
> to prevent the driver from crashing on invalid input?

The API contract is that the stack does not send frames for queues
which don't exist (> real_num_tx_queues) down to the drivers.
There's no ambiguity, IMO, if the stack sends such frames its a bug
in the stack.

Re: [PATCH net 0/2] gve: fix crashes on invalid TX queue indices

[Eric Dumazet]

On Thu, Jan 8, 2026 at 4:36 PM Ankit Garg <nktgrg@google.com> wrote:
>
> On Tue, Jan 6, 2026 at 6:22 PM Jakub Kicinski <kuba@kernel.org> wrote:
> >
> > On Mon,  5 Jan 2026 15:25:02 -0800 Joshua Washington wrote:
> > > This series fixes a kernel panic in the GVE driver caused by
> > > out-of-bounds array access when the network stack provides an invalid
> > > TX queue index.
> >
> > Do you know how? I seem to recall we had such issues due to bugs
> > in the qdisc layer, most of which were fixed.
> >
> > Fixing this at the source, if possible, would be far preferable
> > to sprinkling this condition to all the drivers.
> That matches our observation—we have encountered this panic on older
> kernels (specifically Rocky Linux 8) but have not been able to
> reproduce it on recent upstream kernels.

What is the kernel version used in Rocky Linux 8 ?

Note that the test against real_num_tx_queues is done before reaching
the Qdisc layer.

It might help to give a stack trace of a panic.

>
> Could you point us to the specific qdisc fixes you recall? We'd like
> to verify if the issue we are seeing on the older kernel is indeed one
> of those known/fixed bugs.
>
> If it turns out this is fully resolved in the core network stack
> upstream, we can drop this patch for the mainline driver. However, if
> there is ambiguity, do you think there is value in keeping this check
> to prevent the driver from crashing on invalid input?

We already have many costly checks, and netdev_core_pick_tx() should
already prevent such panic.

>
> Thanks,
> Ankit Garg

Re: [PATCH net 0/2] gve: fix crashes on invalid TX queue indices

[Ankit Garg]

On Thu, Jan 8, 2026 at 8:37 AM Eric Dumazet <edumazet@google.com> wrote:
>
> On Thu, Jan 8, 2026 at 4:36 PM Ankit Garg <nktgrg@google.com> wrote:
> >
> > On Tue, Jan 6, 2026 at 6:22 PM Jakub Kicinski <kuba@kernel.org> wrote:
> > >
> > > On Mon,  5 Jan 2026 15:25:02 -0800 Joshua Washington wrote:
> > > > This series fixes a kernel panic in the GVE driver caused by
> > > > out-of-bounds array access when the network stack provides an invalid
> > > > TX queue index.
> > >
> > > Do you know how? I seem to recall we had such issues due to bugs
> > > in the qdisc layer, most of which were fixed.
> > >
> > > Fixing this at the source, if possible, would be far preferable
> > > to sprinkling this condition to all the drivers.
> > That matches our observation—we have encountered this panic on older
> > kernels (specifically Rocky Linux 8) but have not been able to
> > reproduce it on recent upstream kernels.
>
> What is the kernel version used in Rocky Linux 8 ?
>
The kernel version where we observed this is 4.18.0 (full version
4.18.0-553.81.1+2.1.el8_10_ciq)

> Note that the test against real_num_tx_queues is done before reaching
> the Qdisc layer.
>
> It might help to give a stack trace of a panic.
>
Crash happens in the sch_direct_xmit path per the trace.

I wonder if sch_direct_xmit is acting as an optimization to bypass the
queueing layer, and if that is somehow bypassing the queue index
checks you mentioned?

I'll try to dig a bit deeper into that specific flow, but here is the
trace in the meantime:

Call Trace:
? __warn+0x94/0xe0
? gve_tx+0xa9f/0xc30 [gve]
? gve_tx+0xa9f/0xc30 [gve]
? report_bug+0xb1/0xe0
? do_error_trap+0x9e/0xd0
? do_invalid_op+0x36/0x40
? gve_tx+0xa9f/0xc30 [gve]
? invalid_op+0x14/0x20
? gve_tx+0xa9f/0xc30 [gve]
? netif_skb_features+0xcf/0x2a0
dev_hard_start_xmit+0xd7/0x240
sch_direct_xmit+0x9f/0x370
__dev_queue_xmit+0xa04/0xc50
ip_finish_output2+0x26d/0x430
? __ip_finish_output+0xdf/0x1d0
ip_output+0x70/0xf0
__ip_queue_xmit+0x165/0x400
__tcp_transmit_skb+0xa6b/0xb90
tcp_connect+0xae3/0xd40
tcp_v4_connect+0x476/0x4f0
__inet_stream_connect+0xda/0x380
> >
> > Could you point us to the specific qdisc fixes you recall? We'd like
> > to verify if the issue we are seeing on the older kernel is indeed one
> > of those known/fixed bugs.
> >
> > If it turns out this is fully resolved in the core network stack
> > upstream, we can drop this patch for the mainline driver. However, if
> > there is ambiguity, do you think there is value in keeping this check
> > to prevent the driver from crashing on invalid input?
>
> We already have many costly checks, and netdev_core_pick_tx() should
> already prevent such panic.
>
> >
> > Thanks,
> > Ankit Garg

Re: [PATCH net 0/2] gve: fix crashes on invalid TX queue indices

[Eric Dumazet]

On Thu, Jan 8, 2026 at 9:53 PM Ankit Garg <nktgrg@google.com> wrote:
>
> On Thu, Jan 8, 2026 at 8:37 AM Eric Dumazet <edumazet@google.com> wrote:
> >
> > On Thu, Jan 8, 2026 at 4:36 PM Ankit Garg <nktgrg@google.com> wrote:
> > >
> > > On Tue, Jan 6, 2026 at 6:22 PM Jakub Kicinski <kuba@kernel.org> wrote:
> > > >
> > > > On Mon,  5 Jan 2026 15:25:02 -0800 Joshua Washington wrote:
> > > > > This series fixes a kernel panic in the GVE driver caused by
> > > > > out-of-bounds array access when the network stack provides an invalid
> > > > > TX queue index.
> > > >
> > > > Do you know how? I seem to recall we had such issues due to bugs
> > > > in the qdisc layer, most of which were fixed.
> > > >
> > > > Fixing this at the source, if possible, would be far preferable
> > > > to sprinkling this condition to all the drivers.
> > > That matches our observation—we have encountered this panic on older
> > > kernels (specifically Rocky Linux 8) but have not been able to
> > > reproduce it on recent upstream kernels.
> >
> > What is the kernel version used in Rocky Linux 8 ?
> >
> The kernel version where we observed this is 4.18.0 (full version
> 4.18.0-553.81.1+2.1.el8_10_ciq)
>
> > Note that the test against real_num_tx_queues is done before reaching
> > the Qdisc layer.
> >
> > It might help to give a stack trace of a panic.
> >
> Crash happens in the sch_direct_xmit path per the trace.
>
> I wonder if sch_direct_xmit is acting as an optimization to bypass the
> queueing layer, and if that is somehow bypassing the queue index
> checks you mentioned?
>
> I'll try to dig a bit deeper into that specific flow, but here is the
> trace in the meantime:

Jakub, the issue is that before 4.20, calling synchronize_rcu()
instead of synchronize_rcu_bh()
was probably a bug. I suspect we had more issues like that.

 __dev_queue_xmit takes a rcu_read_lock_bh(), while the code (that you
added in 2018 [1])
to update the queue netif_set_real_num_tx_queues does synchronize_net()
(aka synchronize_rcu()) and in earlier times, it would mean that this
would maybe return too soon (say on preemptible kernels)

[1] commit ac5b70198adc25c73fba28de4f78adcee8f6be0b
Author: Jakub Kicinski <jakub.kicinski@netronome.com>
Date:   Mon Feb 12 21:35:31 2018 -0800

    net: fix race on decreasing number of TX queues

    netif_set_real_num_tx_queues() can be called when netdev is up.
    That usually happens when user requests change of number of
    channels/rings with ethtool -L.  The procedure for changing
    the number of queues involves resetting the qdiscs and setting
    dev->num_tx_queues to the new value.  When the new value is
    lower than the old one, extra care has to be taken to ensure
    ordering of accesses to the number of queues vs qdisc reset.

    Currently the queues are reset before new dev->num_tx_queues
    is assigned, leaving a window of time where packets can be
    enqueued onto the queues going down, leading to a likely
    crash in the drivers, since most drivers don't check if TX
    skbs are assigned to an active queue.

    Fixes: e6484930d7c7 ("net: allocate tx queues in register_netdevice")
    Signed-off-by: Jakub Kicinski <jakub.kicinski@netronome.com>
    Signed-off-by: David S. Miller <davem@davemloft.net>

So perhaps a fix for pre 4.20 kernel would be: (I kept the
synchronize_net() to be really cautious and because I really do not
want to test)

diff --git a/net/core/dev.c b/net/core/dev.c
index 93243479085fb1d61031ed2136f5aee22d8f313d..4dd1db70561d35fe2097afc86764dd82bfd0bf27
100644
--- a/net/core/dev.c
+++ b/net/core/dev.c
@@ -2636,6 +2636,7 @@ int netif_set_real_num_tx_queues(struct
net_device *dev, unsigned int txq)

                if (disabling) {
                        synchronize_net();
+                       synchronize_rcu_bh();
                        qdisc_reset_all_tx_gt(dev, txq);
 #ifdef CONFIG_XPS
                        netif_reset_xps_queues_gt(dev, txq);

Re: [PATCH net 0/2] gve: fix crashes on invalid TX queue indices

[Jakub Kicinski]

On Fri, 30 Jan 2026 21:56:07 +0100 Eric Dumazet wrote:
> Jakub, the issue is that before 4.20, calling synchronize_rcu()
> instead of synchronize_rcu_bh()
> was probably a bug. I suspect we had more issues like that.
> 
>  __dev_queue_xmit takes a rcu_read_lock_bh(), while the code (that you
> added in 2018 [1])
> to update the queue netif_set_real_num_tx_queues does synchronize_net()
> (aka synchronize_rcu()) and in earlier times, it would mean that this
> would maybe return too soon (say on preemptible kernels)
> 
> [...]
> 
> So perhaps a fix for pre 4.20 kernel would be: (I kept the
> synchronize_net() to be really cautious and because I really do not
> want to test)

Sounds entirely plausible, FWIW. Ankit, this would mean that you have
to convince RHEL / Rocky to take Eric's patch. Oldest kernel we can
patch upstream is 5.10, AFAIK.