stable.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCH net] gve: fix checksum state and gso_segs of HW-GRO packets
@ 2026-11-13 19:55 Joshua Washington
  2026-11-13 20:37 ` Joshua Washington
                   ` (2 more replies)
  0 siblings, 3 replies; 5+ messages in thread
From: Joshua Washington @ 2026-11-13 19:55 UTC (permalink / raw)
  To: netdev
  Cc: Joshua Washington, Harshitha Ramamurthy, Andrew Lunn,
	David S. Miller, Eric Dumazet, Jakub Kicinski, Paolo Abeni,
	Jordan Rhee, Ankit Garg, Tim Hostetler, Willem de Bruijn,
	linux-kernel, stable, Praveen Kaligineedi

From: Ankit Garg <nktgrg@google.com>

With HW-GRO enabled, gve_rx_complete_rsc() turns a coalesced packet into
a GSO skb by setting gso_size, gso_segs and gso_type, but leaves it
CHECKSUM_UNNECESSARY with th->check as the device wrote it. Unlike
tcp4_gro_complete() and tcp6_gro_complete() for a SW-GRO packet, it does
not seed th->check with the TCP pseudo-header checksum or set
CHECKSUM_PARTIAL.

The GRO stack completes a HW-GRO skb itself only if it holds the skb for
merging (since commit 5eddb24901ee ("gro: add support of (hw)gro packets
to gro stack")). Otherwise the skb is delivered as is, e.g. with rx-gro
off or for a packet that is flushed immediately (e.g. with PSH set). If
such an skb is forwarded out of a TSO-capable device, netif_needs_gso()
accepts it as CHECKSUM_UNNECESSARY and the device segments it with a
bogus checksum seed, so every segment is dropped by the receiver. This
blackholes forwarded TCP traffic on VMs acting as routers or NAT
gateways and was observed on VMs in real use.

Fix it in a manner consistent with other drivers with HW-GRO: set the
network and transport headers and, for packets whose checksum the device
verified, seed th->check with the pseudo-header checksum and call
tcp_gro_complete(), passing gso_segs through NAPI_GRO_CB(skb)->count.
skb->data points at the Ethernet header on the napi_gro_frags() path and
at the IP header on the header-split and copybreak paths, so account for
that when setting the network header. Since the header layout now
matters, tell IPv4 from IPv6 by the EtherType, as qede and bnx2x do,
rather than by the packet type.

Also compute gso_segs from the TCP payload length. It was computed from
skb->data_len, which is 0 for a packet that rx_copybreak copied whole
into the linear part, leaving such packets with gso_segs 0 and a
non-zero gso_size.

Fixes: e637c244b954 ("gve: Advertise NETIF_F_GRO_HW instead of NETIF_F_LRO")
Cc: stable@vger.kernel.org
Reviewed-by: Joshua Washington <joshwash@google.com>
Reviewed-by: Jordan Rhee <jordanrhee@google.com>
Reviewed-by: Praveen Kaligineedi <pkaligineedi@google.com>
Signed-off-by: Ankit Garg <nktgrg@google.com>
---
 drivers/net/ethernet/google/gve/gve_rx_dqo.c | 69 +++++++++++++++-----
 1 file changed, 53 insertions(+), 16 deletions(-)

diff --git a/drivers/net/ethernet/google/gve/gve_rx_dqo.c b/drivers/net/ethernet/google/gve/gve_rx_dqo.c
index 82ed5e48b00a..58696d26c698 100644
--- a/drivers/net/ethernet/google/gve/gve_rx_dqo.c
+++ b/drivers/net/ethernet/google/gve/gve_rx_dqo.c
@@ -13,6 +13,7 @@
 #include <linux/ipv6.h>
 #include <linux/skbuff.h>
 #include <linux/slab.h>
+#include <net/gro.h>
 #include <net/ip6_checksum.h>
 #include <net/ipv6.h>
 #include <net/tcp.h>
@@ -1000,13 +1001,34 @@ static int gve_rx_dqo(struct napi_struct *napi, struct gve_rx_ring *rx,
 	return -ENOMEM;
 }
 
+/* Prepare a HW-GRO packet the way tcp4/6_gro_complete() prepare a SW-GRO
+ * packet, so that it can be resegmented by GSO or TSO if it is forwarded:
+ * seed th->check with the TCP pseudo-header checksum and set CHECKSUM_PARTIAL.
+ */
+static void gve_rx_fixup_gro_checksum(struct sk_buff *skb)
+{
+	int len = skb->len - skb_transport_offset(skb);
+	struct tcphdr *th = tcp_hdr(skb);
+
+	if (skb_shinfo(skb)->gso_type & SKB_GSO_TCPV4)
+		th->check = ~tcp_v4_check(len, ip_hdr(skb)->saddr,
+					  ip_hdr(skb)->daddr, 0);
+	else
+		th->check = ~tcp_v6_check(len, &ipv6_hdr(skb)->saddr,
+					  &ipv6_hdr(skb)->daddr, 0);
+
+	NAPI_GRO_CB(skb)->count = skb_shinfo(skb)->gso_segs;
+	tcp_gro_complete(skb);
+}
+
 static int gve_rx_complete_rsc(struct sk_buff *skb,
 			       const struct gve_rx_compl_desc_dqo *desc,
 			       struct gve_ptype ptype)
 {
 	struct skb_shared_info *shinfo = skb_shinfo(skb);
-	int rsc_segments, rsc_seg_len, hdr_len;
+	int rsc_seg_len, hdr_len, nhoff, thoff;
 	skb_frag_t *frag;
+	__be16 proto;
 	void *va;
 
 	/* HW-GRO only coalesces TCP. */
@@ -1017,20 +1039,11 @@ static int gve_rx_complete_rsc(struct sk_buff *skb,
 	if (!rsc_seg_len)
 		return 0;
 
-	switch (ptype.l3_type) {
-	case GVE_L3_TYPE_IPV4:
-		shinfo->gso_type = SKB_GSO_TCPV4;
-		break;
-	case GVE_L3_TYPE_IPV6:
-		shinfo->gso_type = SKB_GSO_TCPV6;
-		break;
-	default:
-		return -EINVAL;
-	}
-
 	if (skb_headlen(skb)) {
-		/* With header-split, payload is in the non-linear part */
-		rsc_segments = DIV_ROUND_UP(skb->data_len, rsc_seg_len);
+		/* With header-split or copybreak, eth_type_trans() has already
+		 * pulled the Ethernet header.
+		 */
+		nhoff = 0;
 	} else {
 		/* HW-GRO packets are guaranteed to have complete TCP/IP
 		 * headers in frag[0] when header-split is not enabled.
@@ -1039,7 +1052,6 @@ static int gve_rx_complete_rsc(struct sk_buff *skb,
 		va = skb_frag_address(frag);
 		hdr_len =
 			eth_get_headlen(skb->dev, va, skb_frag_size(frag));
-		rsc_segments = DIV_ROUND_UP(skb->len - hdr_len, rsc_seg_len);
 		skb_copy_to_linear_data(skb, va, hdr_len);
 		skb_frag_size_sub(frag, hdr_len);
 		/* Verify we didn't empty the fragment completely as that could
@@ -1049,10 +1061,35 @@ static int gve_rx_complete_rsc(struct sk_buff *skb,
 		skb_frag_off_add(frag, hdr_len);
 		skb->data_len -= hdr_len;
 		skb->tail += hdr_len;
+		/* napi_gro_frags() pulls the Ethernet header later. */
+		nhoff = ETH_HLEN;
 	}
+
+	skb_set_network_header(skb, nhoff);
+	proto = nhoff ? ((struct ethhdr *)skb->data)->h_proto : skb->protocol;
+	switch (proto) {
+	case htons(ETH_P_IP):
+		shinfo->gso_type = SKB_GSO_TCPV4;
+		thoff = nhoff + ip_hdrlen(skb);
+		break;
+	case htons(ETH_P_IPV6):
+		shinfo->gso_type = SKB_GSO_TCPV6;
+		thoff = nhoff + sizeof(struct ipv6hdr);
+		break;
+	default:
+		return -EINVAL;
+	}
+	skb_set_transport_header(skb, thoff);
+
 	shinfo->gso_size = rsc_seg_len;
-	shinfo->gso_segs = rsc_segments;
+	shinfo->gso_segs = DIV_ROUND_UP(skb->len - thoff - tcp_hdrlen(skb),
+					rsc_seg_len);
+
+	if (!IS_ENABLED(CONFIG_INET) ||
+	    unlikely(skb->ip_summed != CHECKSUM_UNNECESSARY))
+		return 0;
 
+	gve_rx_fixup_gro_checksum(skb);
 	return 0;
 }
 

base-commit: dee3429e94d8089713645706df678ec54a049e13
-- 
2.56.0.611.g1030f0b940-goog


^ permalink raw reply related	[flat|nested] 5+ messages in thread

* Re: [PATCH net] gve: fix checksum state and gso_segs of HW-GRO packets
  2026-11-13 19:55 [PATCH net] gve: fix checksum state and gso_segs of HW-GRO packets Joshua Washington
@ 2026-11-13 20:37 ` Joshua Washington
  2026-11-14 15:14 ` Eric Dumazet
  2026-11-17 20:11 ` netdev-bot+sashiko
  2 siblings, 0 replies; 5+ messages in thread
From: Joshua Washington @ 2026-11-13 20:37 UTC (permalink / raw)
  To: netdev
  Cc: Harshitha Ramamurthy, Andrew Lunn, David S. Miller, Eric Dumazet,
	Jakub Kicinski, Paolo Abeni, Jordan Rhee, Ankit Garg,
	Tim Hostetler, Willem de Bruijn, linux-kernel, stable,
	Praveen Kaligineedi

This patch is missing my signoff. I will respin, adding the singoff
and removing the reviewed-by, after the 24 hour period.

-- Josh

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [PATCH net] gve: fix checksum state and gso_segs of HW-GRO packets
  2026-11-13 19:55 [PATCH net] gve: fix checksum state and gso_segs of HW-GRO packets Joshua Washington
  2026-11-13 20:37 ` Joshua Washington
@ 2026-11-14 15:14 ` Eric Dumazet
  2026-11-16  4:32   ` Ankit Garg
  2026-11-17 20:11 ` netdev-bot+sashiko
  2 siblings, 1 reply; 5+ messages in thread
From: Eric Dumazet @ 2026-11-14 15:14 UTC (permalink / raw)
  To: Joshua Washington
  Cc: netdev, Harshitha Ramamurthy, Andrew Lunn, David S. Miller,
	Jakub Kicinski, Paolo Abeni, Jordan Rhee, Ankit Garg,
	Tim Hostetler, Willem de Bruijn, linux-kernel, stable,
	Praveen Kaligineedi

Le ven. 13 nov. 2026 à 20:56, Joshua Washington <joshwash@google.com> a écrit :
>
> From: Ankit Garg <nktgrg@google.com>
>
> With HW-GRO enabled, gve_rx_complete_rsc() turns a coalesced packet into
> a GSO skb by setting gso_size, gso_segs and gso_type, but leaves it
> CHECKSUM_UNNECESSARY with th->check as the device wrote it. Unlike
> tcp4_gro_complete() and tcp6_gro_complete() for a SW-GRO packet, it does
> not seed th->check with the TCP pseudo-header checksum or set
> CHECKSUM_PARTIAL.
>
> The GRO stack completes a HW-GRO skb itself only if it holds the skb for
> merging (since commit 5eddb24901ee ("gro: add support of (hw)gro packets
> to gro stack")). Otherwise the skb is delivered as is, e.g. with rx-gro
> off or for a packet that is flushed immediately (e.g. with PSH set). If
> such an skb is forwarded out of a TSO-capable device, netif_needs_gso()
> accepts it as CHECKSUM_UNNECESSARY and the device segments it with a
> bogus checksum seed, so every segment is dropped by the receiver. This
> blackholes forwarded TCP traffic on VMs acting as routers or NAT
> gateways and was observed on VMs in real use.
>
> Fix it in a manner consistent with other drivers with HW-GRO: set the
> network and transport headers and, for packets whose checksum the device
> verified, seed th->check with the pseudo-header checksum and call
> tcp_gro_complete(), passing gso_segs through NAPI_GRO_CB(skb)->count.
> skb->data points at the Ethernet header on the napi_gro_frags() path and
> at the IP header on the header-split and copybreak paths, so account for
> that when setting the network header. Since the header layout now
> matters, tell IPv4 from IPv6 by the EtherType, as qede and bnx2x do,
> rather than by the packet type.
>
> Also compute gso_segs from the TCP payload length. It was computed from
> skb->data_len, which is 0 for a packet that rx_copybreak copied whole
> into the linear part, leaving such packets with gso_segs 0 and a
> non-zero gso_size.
>
> Fixes: e637c244b954 ("gve: Advertise NETIF_F_GRO_HW instead of NETIF_F_LRO")
> Cc: stable@vger.kernel.org
> Reviewed-by: Joshua Washington <joshwash@google.com>
> Reviewed-by: Jordan Rhee <jordanrhee@google.com>
> Reviewed-by: Praveen Kaligineedi <pkaligineedi@google.com>
> Signed-off-by: Ankit Garg <nktgrg@google.com>
> ---
>  drivers/net/ethernet/google/gve/gve_rx_dqo.c | 69 +++++++++++++++-----
>  1 file changed, 53 insertions(+), 16 deletions(-)
>
> diff --git a/drivers/net/ethernet/google/gve/gve_rx_dqo.c b/drivers/net/ethernet/google/gve/gve_rx_dqo.c
> index 82ed5e48b00a..58696d26c698 100644
> --- a/drivers/net/ethernet/google/gve/gve_rx_dqo.c
> +++ b/drivers/net/ethernet/google/gve/gve_rx_dqo.c
> @@ -13,6 +13,7 @@
>  #include <linux/ipv6.h>
>  #include <linux/skbuff.h>
>  #include <linux/slab.h>
> +#include <net/gro.h>
>  #include <net/ip6_checksum.h>
>  #include <net/ipv6.h>
>  #include <net/tcp.h>
> @@ -1000,13 +1001,34 @@ static int gve_rx_dqo(struct napi_struct *napi, struct gve_rx_ring *rx,
>         return -ENOMEM;
>  }
>
> +/* Prepare a HW-GRO packet the way tcp4/6_gro_complete() prepare a SW-GRO
> + * packet, so that it can be resegmented by GSO or TSO if it is forwarded:
> + * seed th->check with the TCP pseudo-header checksum and set CHECKSUM_PARTIAL.
> + */
> +static void gve_rx_fixup_gro_checksum(struct sk_buff *skb)
> +{
> +       int len = skb->len - skb_transport_offset(skb);
> +       struct tcphdr *th = tcp_hdr(skb);
> +
> +       if (skb_shinfo(skb)->gso_type & SKB_GSO_TCPV4)
> +               th->check = ~tcp_v4_check(len, ip_hdr(skb)->saddr,
> +                                         ip_hdr(skb)->daddr, 0);
> +       else
> +               th->check = ~tcp_v6_check(len, &ipv6_hdr(skb)->saddr,
> +                                         &ipv6_hdr(skb)->daddr, 0);
> +
> +       NAPI_GRO_CB(skb)->count = skb_shinfo(skb)->gso_segs;
> +       tcp_gro_complete(skb);
> +}
> +
>  static int gve_rx_complete_rsc(struct sk_buff *skb,
>                                const struct gve_rx_compl_desc_dqo *desc,
>                                struct gve_ptype ptype)
>  {
>         struct skb_shared_info *shinfo = skb_shinfo(skb);
> -       int rsc_segments, rsc_seg_len, hdr_len;
> +       int rsc_seg_len, hdr_len, nhoff, thoff;
>         skb_frag_t *frag;
> +       __be16 proto;
>         void *va;
>
>         /* HW-GRO only coalesces TCP. */
> @@ -1017,20 +1039,11 @@ static int gve_rx_complete_rsc(struct sk_buff *skb,
>         if (!rsc_seg_len)
>                 return 0;
>
> -       switch (ptype.l3_type) {
> -       case GVE_L3_TYPE_IPV4:
> -               shinfo->gso_type = SKB_GSO_TCPV4;
> -               break;
> -       case GVE_L3_TYPE_IPV6:
> -               shinfo->gso_type = SKB_GSO_TCPV6;
> -               break;
> -       default:
> -               return -EINVAL;

Before this patch the L3 type came from ptype.l3_type, now it comes from
the EtherType. A VLAN tagged frame (gve does not strip tags) would have
been accepted before, and is now dropped and accounted in
rx_desc_err_dropped_pkt. nhoff = ETH_HLEN has the same assumption.

Can the device coalesce VLAN tagged frames ?

If not, please say so in the changelog. Otherwise this needs to be
handled, like idpf_rx_rsc() does with eth_type_vlan().

> -       }
> -
>         if (skb_headlen(skb)) {
> -               /* With header-split, payload is in the non-linear part */
> -               rsc_segments = DIV_ROUND_UP(skb->data_len, rsc_seg_len);
> +               /* With header-split or copybreak, eth_type_trans() has already
> +                * pulled the Ethernet header.
> +                */
> +               nhoff = 0;
>         } else {
>                 /* HW-GRO packets are guaranteed to have complete TCP/IP
>                  * headers in frag[0] when header-split is not enabled.
> @@ -1039,7 +1052,6 @@ static int gve_rx_complete_rsc(struct sk_buff *skb,
>                 va = skb_frag_address(frag);
>                 hdr_len =
>                         eth_get_headlen(skb->dev, va, skb_frag_size(frag));
> -               rsc_segments = DIV_ROUND_UP(skb->len - hdr_len, rsc_seg_len);
>                 skb_copy_to_linear_data(skb, va, hdr_len);
>                 skb_frag_size_sub(frag, hdr_len);
>                 /* Verify we didn't empty the fragment completely as that could
> @@ -1049,10 +1061,35 @@ static int gve_rx_complete_rsc(struct sk_buff *skb,
>                 skb_frag_off_add(frag, hdr_len);
>                 skb->data_len -= hdr_len;
>                 skb->tail += hdr_len;
> +               /* napi_gro_frags() pulls the Ethernet header later. */
> +               nhoff = ETH_HLEN;
>         }
> +
> +       skb_set_network_header(skb, nhoff);
> +       proto = nhoff ? ((struct ethhdr *)skb->data)->h_proto : skb->protocol;
> +       switch (proto) {
> +       case htons(ETH_P_IP):
> +               shinfo->gso_type = SKB_GSO_TCPV4;
> +               thoff = nhoff + ip_hdrlen(skb);
> +               break;
> +       case htons(ETH_P_IPV6):
> +               shinfo->gso_type = SKB_GSO_TCPV6;
> +               thoff = nhoff + sizeof(struct ipv6hdr);
> +               break;
> +       default:
> +               return -EINVAL;
> +       }
> +       skb_set_transport_header(skb, thoff);
> +
>         shinfo->gso_size = rsc_seg_len;
> -       shinfo->gso_segs = rsc_segments;
> +       shinfo->gso_segs = DIV_ROUND_UP(skb->len - thoff - tcp_hdrlen(skb),
> +                                       rsc_seg_len);
> +
> +       if (!IS_ENABLED(CONFIG_INET) ||
> +           unlikely(skb->ip_summed != CHECKSUM_UNNECESSARY))
> +               return 0;
>
> +       gve_rx_fixup_gro_checksum(skb);

thoff is derived from iph->ihl, and tcp_hdrlen() from th->doff, both read
from the packet, and nothing checks that the TCP header is actually in
the linear part before we read th->doff and write th->check.

On the copybreak path the head was allocated by napi_alloc_skb(buf_len),
so a short frame with desc->rsc set and a large ihl ends up reading and
writing past the copied data. Same thing on the napi_get_frags() path if
eth_get_headlen() returned less than the full headers.

The old code was trusting the device as well, but it was not writing
into the headers.

I think we want something like this, before using tcp_hdr(skb):

        if (unlikely(thoff + sizeof(struct tcphdr) > skb_headlen(skb)))
                return -EINVAL;

        if (unlikely(skb->len < thoff + tcp_hdrlen(skb)))
                return -EINVAL;

(skb_headlen() is hdr_len on the napi_get_frags() path at this point,
so one check covers both cases.)

This also makes sure gso_segs can not be computed from a negative length.

Thanks.

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [PATCH net] gve: fix checksum state and gso_segs of HW-GRO packets
  2026-11-14 15:14 ` Eric Dumazet
@ 2026-11-16  4:32   ` Ankit Garg
  0 siblings, 0 replies; 5+ messages in thread
From: Ankit Garg @ 2026-11-16  4:32 UTC (permalink / raw)
  To: Eric Dumazet
  Cc: Joshua Washington, netdev, Harshitha Ramamurthy, Andrew Lunn,
	David S. Miller, Jakub Kicinski, Paolo Abeni, Jordan Rhee,
	Tim Hostetler, Willem de Bruijn, linux-kernel, stable,
	Praveen Kaligineedi

On Sat, Nov 14, 2026 at 7:14 AM Eric Dumazet <edumazet@kernel.org> wrote:
>
> Le ven. 13 nov. 2026 à 20:56, Joshua Washington <joshwash@google.com> a écrit :
> >
> > From: Ankit Garg <nktgrg@google.com>
> >
> > With HW-GRO enabled, gve_rx_complete_rsc() turns a coalesced packet into
> > a GSO skb by setting gso_size, gso_segs and gso_type, but leaves it
> > CHECKSUM_UNNECESSARY with th->check as the device wrote it. Unlike
> > tcp4_gro_complete() and tcp6_gro_complete() for a SW-GRO packet, it does
> > not seed th->check with the TCP pseudo-header checksum or set
> > CHECKSUM_PARTIAL.
> >
> > The GRO stack completes a HW-GRO skb itself only if it holds the skb for
> > merging (since commit 5eddb24901ee ("gro: add support of (hw)gro packets
> > to gro stack")). Otherwise the skb is delivered as is, e.g. with rx-gro
> > off or for a packet that is flushed immediately (e.g. with PSH set). If
> > such an skb is forwarded out of a TSO-capable device, netif_needs_gso()
> > accepts it as CHECKSUM_UNNECESSARY and the device segments it with a
> > bogus checksum seed, so every segment is dropped by the receiver. This
> > blackholes forwarded TCP traffic on VMs acting as routers or NAT
> > gateways and was observed on VMs in real use.
> >
> > Fix it in a manner consistent with other drivers with HW-GRO: set the
> > network and transport headers and, for packets whose checksum the device
> > verified, seed th->check with the pseudo-header checksum and call
> > tcp_gro_complete(), passing gso_segs through NAPI_GRO_CB(skb)->count.
> > skb->data points at the Ethernet header on the napi_gro_frags() path and
> > at the IP header on the header-split and copybreak paths, so account for
> > that when setting the network header. Since the header layout now
> > matters, tell IPv4 from IPv6 by the EtherType, as qede and bnx2x do,
> > rather than by the packet type.
> >
> > Also compute gso_segs from the TCP payload length. It was computed from
> > skb->data_len, which is 0 for a packet that rx_copybreak copied whole
> > into the linear part, leaving such packets with gso_segs 0 and a
> > non-zero gso_size.
> >
> > Fixes: e637c244b954 ("gve: Advertise NETIF_F_GRO_HW instead of NETIF_F_LRO")
> > Cc: stable@vger.kernel.org
> > Reviewed-by: Joshua Washington <joshwash@google.com>
> > Reviewed-by: Jordan Rhee <jordanrhee@google.com>
> > Reviewed-by: Praveen Kaligineedi <pkaligineedi@google.com>
> > Signed-off-by: Ankit Garg <nktgrg@google.com>
> > ---
> >  drivers/net/ethernet/google/gve/gve_rx_dqo.c | 69 +++++++++++++++-----
> >  1 file changed, 53 insertions(+), 16 deletions(-)
> >
> > diff --git a/drivers/net/ethernet/google/gve/gve_rx_dqo.c b/drivers/net/ethernet/google/gve/gve_rx_dqo.c
> > index 82ed5e48b00a..58696d26c698 100644
> > --- a/drivers/net/ethernet/google/gve/gve_rx_dqo.c
> > +++ b/drivers/net/ethernet/google/gve/gve_rx_dqo.c
> > @@ -13,6 +13,7 @@
> >  #include <linux/ipv6.h>
> >  #include <linux/skbuff.h>
> >  #include <linux/slab.h>
> > +#include <net/gro.h>
> >  #include <net/ip6_checksum.h>
> >  #include <net/ipv6.h>
> >  #include <net/tcp.h>
> > @@ -1000,13 +1001,34 @@ static int gve_rx_dqo(struct napi_struct *napi, struct gve_rx_ring *rx,
> >         return -ENOMEM;
> >  }
> >
> > +/* Prepare a HW-GRO packet the way tcp4/6_gro_complete() prepare a SW-GRO
> > + * packet, so that it can be resegmented by GSO or TSO if it is forwarded:
> > + * seed th->check with the TCP pseudo-header checksum and set CHECKSUM_PARTIAL.
> > + */
> > +static void gve_rx_fixup_gro_checksum(struct sk_buff *skb)
> > +{
> > +       int len = skb->len - skb_transport_offset(skb);
> > +       struct tcphdr *th = tcp_hdr(skb);
> > +
> > +       if (skb_shinfo(skb)->gso_type & SKB_GSO_TCPV4)
> > +               th->check = ~tcp_v4_check(len, ip_hdr(skb)->saddr,
> > +                                         ip_hdr(skb)->daddr, 0);
> > +       else
> > +               th->check = ~tcp_v6_check(len, &ipv6_hdr(skb)->saddr,
> > +                                         &ipv6_hdr(skb)->daddr, 0);
> > +
> > +       NAPI_GRO_CB(skb)->count = skb_shinfo(skb)->gso_segs;
> > +       tcp_gro_complete(skb);
> > +}
> > +
> >  static int gve_rx_complete_rsc(struct sk_buff *skb,
> >                                const struct gve_rx_compl_desc_dqo *desc,
> >                                struct gve_ptype ptype)
> >  {
> >         struct skb_shared_info *shinfo = skb_shinfo(skb);
> > -       int rsc_segments, rsc_seg_len, hdr_len;
> > +       int rsc_seg_len, hdr_len, nhoff, thoff;
> >         skb_frag_t *frag;
> > +       __be16 proto;
> >         void *va;
> >
> >         /* HW-GRO only coalesces TCP. */
> > @@ -1017,20 +1039,11 @@ static int gve_rx_complete_rsc(struct sk_buff *skb,
> >         if (!rsc_seg_len)
> >                 return 0;
> >
> > -       switch (ptype.l3_type) {
> > -       case GVE_L3_TYPE_IPV4:
> > -               shinfo->gso_type = SKB_GSO_TCPV4;
> > -               break;
> > -       case GVE_L3_TYPE_IPV6:
> > -               shinfo->gso_type = SKB_GSO_TCPV6;
> > -               break;
> > -       default:
> > -               return -EINVAL;
>
> Before this patch the L3 type came from ptype.l3_type, now it comes from
> the EtherType. A VLAN tagged frame (gve does not strip tags) would have
> been accepted before, and is now dropped and accounted in
> rx_desc_err_dropped_pkt. nhoff = ETH_HLEN has the same assumption.
>
> Can the device coalesce VLAN tagged frames ?
>
> If not, please say so in the changelog. Otherwise this needs to be
> handled, like idpf_rx_rsc() does with eth_type_vlan().
>
GVE device does not coalesce VLAN tagged frames, so they are never
completed with desc->rsc set. I will explicitly mention that in v2.
> > -       }
> > -
> >         if (skb_headlen(skb)) {
> > -               /* With header-split, payload is in the non-linear part */
> > -               rsc_segments = DIV_ROUND_UP(skb->data_len, rsc_seg_len);
> > +               /* With header-split or copybreak, eth_type_trans() has already
> > +                * pulled the Ethernet header.
> > +                */
> > +               nhoff = 0;
> >         } else {
> >                 /* HW-GRO packets are guaranteed to have complete TCP/IP
> >                  * headers in frag[0] when header-split is not enabled.
> > @@ -1039,7 +1052,6 @@ static int gve_rx_complete_rsc(struct sk_buff *skb,
> >                 va = skb_frag_address(frag);
> >                 hdr_len =
> >                         eth_get_headlen(skb->dev, va, skb_frag_size(frag));
> > -               rsc_segments = DIV_ROUND_UP(skb->len - hdr_len, rsc_seg_len);
> >                 skb_copy_to_linear_data(skb, va, hdr_len);
> >                 skb_frag_size_sub(frag, hdr_len);
> >                 /* Verify we didn't empty the fragment completely as that could
> > @@ -1049,10 +1061,35 @@ static int gve_rx_complete_rsc(struct sk_buff *skb,
> >                 skb_frag_off_add(frag, hdr_len);
> >                 skb->data_len -= hdr_len;
> >                 skb->tail += hdr_len;
> > +               /* napi_gro_frags() pulls the Ethernet header later. */
> > +               nhoff = ETH_HLEN;
> >         }
> > +
> > +       skb_set_network_header(skb, nhoff);
> > +       proto = nhoff ? ((struct ethhdr *)skb->data)->h_proto : skb->protocol;
> > +       switch (proto) {
> > +       case htons(ETH_P_IP):
> > +               shinfo->gso_type = SKB_GSO_TCPV4;
> > +               thoff = nhoff + ip_hdrlen(skb);
> > +               break;
> > +       case htons(ETH_P_IPV6):
> > +               shinfo->gso_type = SKB_GSO_TCPV6;
> > +               thoff = nhoff + sizeof(struct ipv6hdr);
> > +               break;
> > +       default:
> > +               return -EINVAL;
> > +       }
> > +       skb_set_transport_header(skb, thoff);
> > +
> >         shinfo->gso_size = rsc_seg_len;
> > -       shinfo->gso_segs = rsc_segments;
> > +       shinfo->gso_segs = DIV_ROUND_UP(skb->len - thoff - tcp_hdrlen(skb),
> > +                                       rsc_seg_len);
> > +
> > +       if (!IS_ENABLED(CONFIG_INET) ||
> > +           unlikely(skb->ip_summed != CHECKSUM_UNNECESSARY))
> > +               return 0;
> >
> > +       gve_rx_fixup_gro_checksum(skb);
>
> thoff is derived from iph->ihl, and tcp_hdrlen() from th->doff, both read
> from the packet, and nothing checks that the TCP header is actually in
> the linear part before we read th->doff and write th->check.
>
> On the copybreak path the head was allocated by napi_alloc_skb(buf_len),
> so a short frame with desc->rsc set and a large ihl ends up reading and
> writing past the copied data. Same thing on the napi_get_frags() path if
> eth_get_headlen() returned less than the full headers.
>
> The old code was trusting the device as well, but it was not writing
> into the headers.
>
> I think we want something like this, before using tcp_hdr(skb):
>
>         if (unlikely(thoff + sizeof(struct tcphdr) > skb_headlen(skb)))
>                 return -EINVAL;
>
>         if (unlikely(skb->len < thoff + tcp_hdrlen(skb)))
>                 return -EINVAL;
>
> (skb_headlen() is hdr_len on the napi_get_frags() path at this point,
> so one check covers both cases.)
>
> This also makes sure gso_segs can not be computed from a negative length.
>
> Thanks.
Agreed, I'll add these in v2.

Thanks for the review!

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [PATCH net] gve: fix checksum state and gso_segs of HW-GRO packets
  2026-11-13 19:55 [PATCH net] gve: fix checksum state and gso_segs of HW-GRO packets Joshua Washington
  2026-11-13 20:37 ` Joshua Washington
  2026-11-14 15:14 ` Eric Dumazet
@ 2026-11-17 20:11 ` netdev-bot+sashiko
  2 siblings, 0 replies; 5+ messages in thread
From: netdev-bot+sashiko @ 2026-11-17 20:11 UTC (permalink / raw)
  To: joshwash
  Cc: netdev, hramamurthy, andrew+netdev, davem, edumazet, kuba, pabeni,
	jordanrhee, nktgrg, thostet, willemb, linux-kernel, stable,
	pkaligineedi

Thank you for your contribution! Sashiko AI review found 2 potential
issue(s) to consider.

Critical: 0 · High: 1 · Medium: 1 · Low: 0

If you believe none of the issues are legitimate, please reply to each
issue of Medium severity or higher and explain why the AI is wrong.
If any of them are legitimate and you plan to respin, please let patchwork
know by including "pw-bot: cr" as a separate line at the end of your reply
(one such reply per series is enough).

- [High] Regression: HW-GRO packets in VLAN-tagged TCP flows are now
  dropped.
- [Medium] For IPv6, gve_rx_complete_rsc() assumes TCP always starts
  right after the fixed 40-byte header: `thoff = nhoff + sizeof(struct…

--

gve: fix checksum state and gso_segs of HW-GRO packets

The patch makes gve_rx_complete_rsc() set the network and transport
headers for HW-GRO packets. For packets the device verified, it also seeds
th->check with the pseudo-header checksum and calls tcp_gro_complete() so
the skb becomes CHECKSUM_PARTIAL. It also tells IPv4 from IPv6 by the
EtherType instead of the device packet type, and computes gso_segs from
the TCP payload length.

> diff --git a/drivers/net/ethernet/google/gve/gve_rx_dqo.c b/drivers/net/ethernet/google/gve/gve_rx_dqo.c
> index 82ed5e48b00ab..58696d26c6986 100644
> --- a/drivers/net/ethernet/google/gve/gve_rx_dqo.c
> +++ b/drivers/net/ethernet/google/gve/gve_rx_dqo.c

[ ... ]

> @@ -1017,20 +1039,11 @@ static int gve_rx_complete_rsc(struct sk_buff *skb,
>  	if (!rsc_seg_len)
>  		return 0;
>  
> -	switch (ptype.l3_type) {
> -	case GVE_L3_TYPE_IPV4:
> -		shinfo->gso_type = SKB_GSO_TCPV4;
> -		break;
> -	case GVE_L3_TYPE_IPV6:
> -		shinfo->gso_type = SKB_GSO_TCPV6;
> -		break;
> -	default:
> -		return -EINVAL;
> -	}
> -

[ ... ]

> @@ -1049,10 +1061,35 @@ static int gve_rx_complete_rsc(struct sk_buff *skb,
>  		skb_frag_off_add(frag, hdr_len);
>  		skb->data_len -= hdr_len;
>  		skb->tail += hdr_len;
> +		/* napi_gro_frags() pulls the Ethernet header later. */
> +		nhoff = ETH_HLEN;
>  	}
> +
> +	skb_set_network_header(skb, nhoff);
> +	proto = nhoff ? ((struct ethhdr *)skb->data)->h_proto : skb->protocol;
> +	switch (proto) {
> +	case htons(ETH_P_IP):
> +		shinfo->gso_type = SKB_GSO_TCPV4;
> +		thoff = nhoff + ip_hdrlen(skb);
> +		break;
> +	case htons(ETH_P_IPV6):
> +		shinfo->gso_type = SKB_GSO_TCPV6;
> +		thoff = nhoff + sizeof(struct ipv6hdr);
> +		break;
> +	default:
> +		return -EINVAL;
> +	}

[Severity: High]
Could this cause HW-GRO packets from VLAN-tagged TCP flows to be dropped?

gve sets no NETIF_F_HW_VLAN_* features, and struct
gve_rx_compl_desc_dqo has no VLAN tag field. A tagged frame therefore
arrives here with h_proto (or the skb->protocol set by eth_type_trans())
equal to ETH_P_8021Q or ETH_P_8021AD. That value hits the default case and
returns -EINVAL. The error then ends up as a silent drop:

gve_rx_poll_dqo()
  gve_rx_complete_skb()
    gve_rx_complete_rsc()
      return -EINVAL;
  gve_rx_free_skb(napi, rx);
  rx->rx_desc_err_dropped_pkt++;

Before this patch, gso_type came from ptype.l3_type, which does not depend
on the L2 layout. These packets then went on to napi_gro_frags() or
napi_gro_receive() and were delivered.

The fixed nhoff of 0 or ETH_HLEN also assumes there is no tag. Adding the
VLAN EtherTypes to this switch alone would leave the network and transport
headers pointing at the tag instead of the IP and TCP headers.

The commit message says:

  "tell IPv4 from IPv6 by the EtherType, as qede and bnx2x do"

Both of those drivers strip VLAN tags in hardware. For example, qede sets
NETIF_F_HW_VLAN_CTAG_RX. Also, on an unknown protocol qede_gro_receive()
only logs an error and still delivers the skb.

Would it be better to keep using ptype.l3_type here? Or, for an
unrecognized EtherType, deliver the skb without the fixup instead of
returning -EINVAL? The patch is also tagged for stable, and the commit
message does not mention this behavior change.

> +	skb_set_transport_header(skb, thoff);
> +
>  	shinfo->gso_size = rsc_seg_len;
> -	shinfo->gso_segs = rsc_segments;
> +	shinfo->gso_segs = DIV_ROUND_UP(skb->len - thoff - tcp_hdrlen(skb),
> +					rsc_seg_len);
> +
> +	if (!IS_ENABLED(CONFIG_INET) ||
> +	    unlikely(skb->ip_summed != CHECKSUM_UNNECESSARY))
> +		return 0;
>  
> +	gve_rx_fixup_gro_checksum(skb);

[Severity: Medium]
For IPv6, is it safe to assume TCP always starts right after the fixed
40-byte header?

	case htons(ETH_P_IPV6):
		shinfo->gso_type = SKB_GSO_TCPV6;
		thoff = nhoff + sizeof(struct ipv6hdr);

Nothing checks that ipv6_hdr(skb)->nexthdr == IPPROTO_TCP, and
desc->ipv6_ex_add is not checked here.

gve_rx_skb_csum() withholds CHECKSUM_UNNECESSARY only when ipv6_ex_add is
set:

	} else if (ptype.l3_type == GVE_L3_TYPE_IPV6) {
		/* Checksum should be skipped if this flag is set. */
		if (unlikely(desc->ipv6_ex_add))
			return;
	}

According to gve_desc_dqo.h, that bit only covers Destination Options and
Routing headers. Suppose the device coalesces a TCP flow that carries, for
example, a Hop-by-Hop header. The transport header would then land on the
extension header, and tcp_hdrlen() would read a bogus doff, so gso_segs
would be wrong.

gve_rx_fixup_gro_checksum() would then write the pseudo-header seed at
thoff + 16. With an 8-byte Hop-by-Hop header, that offset is the upper
half of the real ack_seq:

	th->check = ~tcp_v6_check(len, &ipv6_hdr(skb)->saddr,
				  &ipv6_hdr(skb)->daddr, 0);

tcp_gro_complete() would also set csum_start and csum_offset from the same
wrong offset.

When ipv6_ex_add is set, the checksum write is skipped. The transport
header and gso_segs are still set from the assumed offset, though. Before
this patch, the RSC path did not read or write the TCP header.

The driver doesn't show whether the device's RSC engine ever coalesces
IPv6 packets with extension headers. qede and bnx2x make the same
fixed-offset assumption. If the device can do this, should the code check
nexthdr, or skip the fixup when nexthdr is not IPPROTO_TCP?

-- 
Sashiko AI review · https://netdev-ai.bots.linux.dev/sashiko/#/patchset/20261113195553.2220562-1-joshwash%40google.com

^ permalink raw reply	[flat|nested] 5+ messages in thread

end of thread, other threads:[~2026-11-17 20:11 UTC | newest]

Thread overview: 5+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-11-13 19:55 [PATCH net] gve: fix checksum state and gso_segs of HW-GRO packets Joshua Washington
2026-11-13 20:37 ` Joshua Washington
2026-11-14 15:14 ` Eric Dumazet
2026-11-16  4:32   ` Ankit Garg
2026-11-17 20:11 ` netdev-bot+sashiko

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).