[PATCH net] net/mlx5: Correctly set gso_segs when LRO is used

[PATCH net] net/mlx5: Correctly set gso_segs when LRO is used

[Christoph Paasch via B4 Relay]

From: Christoph Paasch <cpaasch@openai.com>

When gso_segs is left at 0, a number of assumptions will end up being
incorrect throughout the stack.

For example, in the GRO-path, we set NAPI_GRO_CB()->count to gso_segs.
So, if a non-LRO'ed packet followed by an LRO'ed packet is being
processed in GRO, the first one will have NAPI_GRO_CB()->count set to 1 and
the next one to 0 (in dev_gro_receive()).
Since commit 531d0d32de3e
("net/mlx5: Correctly set gso_size when LRO is used")
these packets will get merged (as their gso_size now matches).
So, we end up in gro_complete() with NAPI_GRO_CB()->count == 1 and thus
don't call inet_gro_complete(). Meaning, checksum-validation in
tcp_checksum_complete() will fail with a "hw csum failure".

Even before the above mentioned commit, incorrect gso_segs means that other
things like TCP's accounting of incoming packets (tp->segs_in,
data_segs_in, rcv_ooopack) will be incorrect. Which means that if one
does bytes_received/data_segs_in, the result will be bigger than the
MTU.

Fix this by initializing gso_segs correctly when LRO is used.

Fixes: e586b3b0baee ("net/mlx5: Ethernet Datapath files")
Reported-by: Gal Pressman <gal@nvidia.com>
Closes: https://lore.kernel.org/netdev/6583783f-f0fb-4fb1-a415-feec8155bc69@nvidia.com/
Signed-off-by: Christoph Paasch <cpaasch@openai.com>
---
 drivers/net/ethernet/mellanox/mlx5/core/en_rx.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c
index 7462514c7f3d1606339ede13a6207c1629ab65a3..da3e340c99b72ce27861cccaa5bd722c1b446a55 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c
@@ -1567,6 +1567,7 @@ static inline void mlx5e_build_rx_skb(struct mlx5_cqe64 *cqe,
 		unsigned int hdrlen = mlx5e_lro_update_hdr(skb, cqe, cqe_bcnt);
 
 		skb_shinfo(skb)->gso_size = DIV_ROUND_UP(cqe_bcnt - hdrlen, lro_num_seg);
+		skb_shinfo(skb)->gso_segs = lro_num_seg;
 		/* Subtract one since we already counted this as one
 		 * "regular" packet in mlx5e_complete_rx_cqe()
 		 */

---
base-commit: afd8c2c9e2e29c6c7705635bea2960593976dacc
change-id: 20250729-mlx5_gso_segs-8e5ea2d4b9b0

Best regards,
-- 
Christoph Paasch <cpaasch@openai.com>

Re: [PATCH net] net/mlx5: Correctly set gso_segs when LRO is used

[Gal Pressman]

On 29/07/2025 21:34, Christoph Paasch via B4 Relay wrote:
> From: Christoph Paasch <cpaasch@openai.com>
> 
> When gso_segs is left at 0, a number of assumptions will end up being
> incorrect throughout the stack.
> 
> For example, in the GRO-path, we set NAPI_GRO_CB()->count to gso_segs.
> So, if a non-LRO'ed packet followed by an LRO'ed packet is being
> processed in GRO, the first one will have NAPI_GRO_CB()->count set to 1 and
> the next one to 0 (in dev_gro_receive()).
> Since commit 531d0d32de3e
> ("net/mlx5: Correctly set gso_size when LRO is used")
> these packets will get merged (as their gso_size now matches).
> So, we end up in gro_complete() with NAPI_GRO_CB()->count == 1 and thus
> don't call inet_gro_complete(). Meaning, checksum-validation in
> tcp_checksum_complete() will fail with a "hw csum failure".
> 
> Even before the above mentioned commit, incorrect gso_segs means that other
> things like TCP's accounting of incoming packets (tp->segs_in,
> data_segs_in, rcv_ooopack) will be incorrect. Which means that if one
> does bytes_received/data_segs_in, the result will be bigger than the
> MTU.
> 
> Fix this by initializing gso_segs correctly when LRO is used.
> 
> Fixes: e586b3b0baee ("net/mlx5: Ethernet Datapath files")

Maybe we should put an additional Fixes line for the gso_size patch?
It doesn't directly fix it, but it will clearly emphasize the importance
of picking up this patch together with the other one.

> Reported-by: Gal Pressman <gal@nvidia.com>
> Closes: https://lore.kernel.org/netdev/6583783f-f0fb-4fb1-a415-feec8155bc69@nvidia.com/
> Signed-off-by: Christoph Paasch <cpaasch@openai.com>

Thanks Christoph,
Reviewed-by: Gal Pressman <gal@nvidia.com>

Re: [PATCH net] net/mlx5: Correctly set gso_segs when LRO is used

[Eric Dumazet]

On Wed, Jul 30, 2025 at 4:06 AM Gal Pressman <gal@nvidia.com> wrote:
>
> On 29/07/2025 21:34, Christoph Paasch via B4 Relay wrote:
> > From: Christoph Paasch <cpaasch@openai.com>
> >
> > When gso_segs is left at 0, a number of assumptions will end up being
> > incorrect throughout the stack.
> >
> > For example, in the GRO-path, we set NAPI_GRO_CB()->count to gso_segs.
> > So, if a non-LRO'ed packet followed by an LRO'ed packet is being
> > processed in GRO, the first one will have NAPI_GRO_CB()->count set to 1 and
> > the next one to 0 (in dev_gro_receive()).
> > Since commit 531d0d32de3e
> > ("net/mlx5: Correctly set gso_size when LRO is used")
> > these packets will get merged (as their gso_size now matches).
> > So, we end up in gro_complete() with NAPI_GRO_CB()->count == 1 and thus
> > don't call inet_gro_complete(). Meaning, checksum-validation in
> > tcp_checksum_complete() will fail with a "hw csum failure".
> >
> > Even before the above mentioned commit, incorrect gso_segs means that other
> > things like TCP's accounting of incoming packets (tp->segs_in,
> > data_segs_in, rcv_ooopack) will be incorrect. Which means that if one
> > does bytes_received/data_segs_in, the result will be bigger than the
> > MTU.
> >
> > Fix this by initializing gso_segs correctly when LRO is used.
> >
> > Fixes: e586b3b0baee ("net/mlx5: Ethernet Datapath files")
>
> Maybe we should put an additional Fixes line for the gso_size patch?
> It doesn't directly fix it, but it will clearly emphasize the importance
> of picking up this patch together with the other one.
>
> > Reported-by: Gal Pressman <gal@nvidia.com>
> > Closes: https://lore.kernel.org/netdev/6583783f-f0fb-4fb1-a415-feec8155bc69@nvidia.com/
> > Signed-off-by: Christoph Paasch <cpaasch@openai.com>
>
> Thanks Christoph,
> Reviewed-by: Gal Pressman <gal@nvidia.com>

I do not think we need many Fixes: tag.

Reviewed-by: Eric Dumazet <edumazet@google.com>

If we really want to be precise, the issue also came when GRO got
support for GRO packets ;)

commit 5eddb24901ee    gro: add support of (hw)gro packets to gro stack

This commit really implied that both gso_size and gso_segs had to be
set by drivers RX paths.

It seems drivers/net/ethernet/google/gve/gve_rx_dqo.c has a similar issue.

gve_rx_complete_rsc() sets gso_size but not gso_segs

shinfo->gso_size = le16_to_cpu(desc->rsc_seg_len);

Re: [PATCH net] net/mlx5: Correctly set gso_segs when LRO is used

[Christoph Paasch]

On Wed, Jul 30, 2025 at 5:28 AM Eric Dumazet <edumazet@google.com> wrote:
>
> On Wed, Jul 30, 2025 at 4:06 AM Gal Pressman <gal@nvidia.com> wrote:
> >
> > On 29/07/2025 21:34, Christoph Paasch via B4 Relay wrote:
> > > From: Christoph Paasch <cpaasch@openai.com>
> > >
> > > When gso_segs is left at 0, a number of assumptions will end up being
> > > incorrect throughout the stack.
> > >
> > > For example, in the GRO-path, we set NAPI_GRO_CB()->count to gso_segs.
> > > So, if a non-LRO'ed packet followed by an LRO'ed packet is being
> > > processed in GRO, the first one will have NAPI_GRO_CB()->count set to 1 and
> > > the next one to 0 (in dev_gro_receive()).
> > > Since commit 531d0d32de3e
> > > ("net/mlx5: Correctly set gso_size when LRO is used")
> > > these packets will get merged (as their gso_size now matches).
> > > So, we end up in gro_complete() with NAPI_GRO_CB()->count == 1 and thus
> > > don't call inet_gro_complete(). Meaning, checksum-validation in
> > > tcp_checksum_complete() will fail with a "hw csum failure".
> > >
> > > Even before the above mentioned commit, incorrect gso_segs means that other
> > > things like TCP's accounting of incoming packets (tp->segs_in,
> > > data_segs_in, rcv_ooopack) will be incorrect. Which means that if one
> > > does bytes_received/data_segs_in, the result will be bigger than the
> > > MTU.
> > >
> > > Fix this by initializing gso_segs correctly when LRO is used.
> > >
> > > Fixes: e586b3b0baee ("net/mlx5: Ethernet Datapath files")
> >
> > Maybe we should put an additional Fixes line for the gso_size patch?
> > It doesn't directly fix it, but it will clearly emphasize the importance
> > of picking up this patch together with the other one.
> >
> > > Reported-by: Gal Pressman <gal@nvidia.com>
> > > Closes: https://lore.kernel.org/netdev/6583783f-f0fb-4fb1-a415-feec8155bc69@nvidia.com/
> > > Signed-off-by: Christoph Paasch <cpaasch@openai.com>
> >
> > Thanks Christoph,
> > Reviewed-by: Gal Pressman <gal@nvidia.com>
>
> I do not think we need many Fixes: tag.
>
> Reviewed-by: Eric Dumazet <edumazet@google.com>
>
> If we really want to be precise, the issue also came when GRO got
> support for GRO packets ;)
>
> commit 5eddb24901ee    gro: add support of (hw)gro packets to gro stack
>
> This commit really implied that both gso_size and gso_segs had to be
> set by drivers RX paths.
>
> It seems drivers/net/ethernet/google/gve/gve_rx_dqo.c has a similar issue.
>
> gve_rx_complete_rsc() sets gso_size but not gso_segs
>
> shinfo->gso_size = le16_to_cpu(desc->rsc_seg_len);

I see! I can send a fix, but won't have the ability to actually test
it. So, maybe better if someone else takes this one.

Christoph

Re: [PATCH net] net/mlx5: Correctly set gso_segs when LRO is used

[Willem de Bruijn]

Christoph Paasch wrote:
> On Wed, Jul 30, 2025 at 5:28 AM Eric Dumazet <edumazet@google.com> wrote:
> >
> > On Wed, Jul 30, 2025 at 4:06 AM Gal Pressman <gal@nvidia.com> wrote:
> > >
> > > On 29/07/2025 21:34, Christoph Paasch via B4 Relay wrote:
> > > > From: Christoph Paasch <cpaasch@openai.com>
> > > >
> > > > When gso_segs is left at 0, a number of assumptions will end up being
> > > > incorrect throughout the stack.
> > > >
> > > > For example, in the GRO-path, we set NAPI_GRO_CB()->count to gso_segs.
> > > > So, if a non-LRO'ed packet followed by an LRO'ed packet is being
> > > > processed in GRO, the first one will have NAPI_GRO_CB()->count set to 1 and
> > > > the next one to 0 (in dev_gro_receive()).
> > > > Since commit 531d0d32de3e
> > > > ("net/mlx5: Correctly set gso_size when LRO is used")
> > > > these packets will get merged (as their gso_size now matches).
> > > > So, we end up in gro_complete() with NAPI_GRO_CB()->count == 1 and thus
> > > > don't call inet_gro_complete(). Meaning, checksum-validation in
> > > > tcp_checksum_complete() will fail with a "hw csum failure".
> > > >
> > > > Even before the above mentioned commit, incorrect gso_segs means that other
> > > > things like TCP's accounting of incoming packets (tp->segs_in,
> > > > data_segs_in, rcv_ooopack) will be incorrect. Which means that if one
> > > > does bytes_received/data_segs_in, the result will be bigger than the
> > > > MTU.
> > > >
> > > > Fix this by initializing gso_segs correctly when LRO is used.
> > > >
> > > > Fixes: e586b3b0baee ("net/mlx5: Ethernet Datapath files")
> > >
> > > Maybe we should put an additional Fixes line for the gso_size patch?
> > > It doesn't directly fix it, but it will clearly emphasize the importance
> > > of picking up this patch together with the other one.
> > >
> > > > Reported-by: Gal Pressman <gal@nvidia.com>
> > > > Closes: https://lore.kernel.org/netdev/6583783f-f0fb-4fb1-a415-feec8155bc69@nvidia.com/
> > > > Signed-off-by: Christoph Paasch <cpaasch@openai.com>
> > >
> > > Thanks Christoph,
> > > Reviewed-by: Gal Pressman <gal@nvidia.com>
> >
> > I do not think we need many Fixes: tag.
> >
> > Reviewed-by: Eric Dumazet <edumazet@google.com>
> >
> > If we really want to be precise, the issue also came when GRO got
> > support for GRO packets ;)
> >
> > commit 5eddb24901ee    gro: add support of (hw)gro packets to gro stack
> >
> > This commit really implied that both gso_size and gso_segs had to be
> > set by drivers RX paths.
> >
> > It seems drivers/net/ethernet/google/gve/gve_rx_dqo.c has a similar issue.
> >
> > gve_rx_complete_rsc() sets gso_size but not gso_segs
> >
> > shinfo->gso_size = le16_to_cpu(desc->rsc_seg_len);
> 
> I see! I can send a fix, but won't have the ability to actually test
> it. So, maybe better if someone else takes this one.

Thanks. The GVE team will send a fix.

Re: [PATCH net] net/mlx5: Correctly set gso_segs when LRO is used

[patchwork-bot+netdevbpf]

Hello:

This patch was applied to netdev/net.git (main)
by Jakub Kicinski <kuba@kernel.org>:

On Tue, 29 Jul 2025 11:34:00 -0700 you wrote:
> From: Christoph Paasch <cpaasch@openai.com>
> 
> When gso_segs is left at 0, a number of assumptions will end up being
> incorrect throughout the stack.
> 
> For example, in the GRO-path, we set NAPI_GRO_CB()->count to gso_segs.
> So, if a non-LRO'ed packet followed by an LRO'ed packet is being
> processed in GRO, the first one will have NAPI_GRO_CB()->count set to 1 and
> the next one to 0 (in dev_gro_receive()).
> Since commit 531d0d32de3e
> ("net/mlx5: Correctly set gso_size when LRO is used")
> these packets will get merged (as their gso_size now matches).
> So, we end up in gro_complete() with NAPI_GRO_CB()->count == 1 and thus
> don't call inet_gro_complete(). Meaning, checksum-validation in
> tcp_checksum_complete() will fail with a "hw csum failure".
> 
> [...]

Here is the summary with links:
  - [net] net/mlx5: Correctly set gso_segs when LRO is used
    https://git.kernel.org/netdev/net/c/77bf1c55b2ac

You are awesome, thank you!
-- 
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html