GRO issue with kernel 3.4.94 (icmp fragmentation needed)

GRO issue with kernel 3.4.94 (icmp fragmentation needed)

[jungwon park]

When using the linux router is turned on GRO, router send the 'fragmentation
needed' packets to the sender.
and the sender's packets are dropped.
The router seems to handle the larger than MTU(1500) with TCP segments by
GRO.

topology : [Host A] - [Linux Router] - [Host B] kernel : 3.4.94 GRO : on MTU
: 1500

tcpdmp :

10:36:36.592752 IP (tos 0x0, ttl 127, id 15685, offset 0, flags [DF], proto
TCP (6), length 198) x.x.x.x.49512 > x.x.x.x.445: P 908:1066(158) ack 587
win 254
10:36:36.592911 IP (tos 0x0, ttl 128, id 9712, offset 0, flags [DF], proto
TCP (6), length 117) x.x.x.x.445 > x.x.x.x.49512: P 587:664(77) ack 1066 win
252
10:36:36.594552 IP (tos 0x0, ttl 127, id 15686, offset 0, flags [DF], proto
TCP (6), length 390) x.x.x.x.49512 > x.x.x.x.445: P 1066:1416(350) ack 664
win 254
10:36:36.596544 IP (tos 0x0, ttl 128, id 9713, offset 0, flags [DF], proto
TCP (6), length 2960) x.x.x.x.445 > x.x.x.x.49512: . 664:3584(2920) ack 1416
win 251
10:36:36.596566 IP (tos 0xc0, ttl 64, id 8339, offset 0, flags [none], proto
ICMP (1), length 576) x.x.x.x > x.x.x.x: ICMP x.x.x.x unreachable - need to
frag (mtu 1500), length 556
	IP (tos 0x0, ttl 128, id 9713, offset 0, flags [DF], proto TCP (6),
length 2960) x.x.x.x.445 > x.x.x.x.49512:  tcp 2940 [bad hdr length 0 - too
short, < 20][|icmp]

When I turned off GRO, the router operate normally, and there is no problem.
and with 3.4.91 kernel, the router has no problem.

I doubt 'ipv4: ip_forward: fix inverted local_df test' patch.
(http://patchwork.ozlabs.org/patch/345509/)
When I revert this patch, the router has no problem.

(sorry my poor english.)

Re: GRO issue with kernel 3.4.94 (icmp fragmentation needed)

[Florian Westphal]

jungwon park <jwpark2@whitecode.co.kr> wrote:
> When using the linux router is turned on GRO, router send the 'fragmentation
> needed' packets to the sender.

Indeed 8-(

> When I turned off GRO, the router operate normally, and there is no problem.
> and with 3.4.91 kernel, the router has no problem.
> 
> I doubt 'ipv4: ip_forward: fix inverted local_df test' patch.
> (http://patchwork.ozlabs.org/patch/345509/)
> When I revert this patch, the router has no problem.

Can you please cherry-pick following patch on top of vanilla 3.4.92?
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=21d1196a35f5686c4323e42a62fdb4b23b0ab4a3

commit 21d1196a35f5686c4323e42a62fdb4b23b0ab4a3
Author: Eric Dumazet <edumazet@google.com>
ipv4: set transport header earlier

I think that should fix this bug, it should apply cleanly on top of
3.4.y tree.

[ patch is in 3.11, also backported to 3.10.y tree ]

The problem is that, when dealing with GRO packets, we try to determine
the size of the individual packets.  To do this, we rely on the
transport header.

Unfortunately the transport header is not set for the forward path in 3.4,
so we look at the network header instead.

RE: GRO issue with kernel 3.4.94 (icmp fragmentation needed)

[jungwon park]

Good!
When I apply the patch(ipv4: set transport header earlier), there is no
problem.
Thanks, Florian.

-----Original Message-----
From: Florian Westphal [mailto:fw@strlen.de] 
Sent: Friday, June 27, 2014 6:05 PM
To: jungwon park
Cc: netdev@vger.kernel.org
Subject: Re: GRO issue with kernel 3.4.94 (icmp fragmentation needed)

jungwon park <jwpark2@whitecode.co.kr> wrote:
> When using the linux router is turned on GRO, router send the 
> 'fragmentation needed' packets to the sender.

Indeed 8-(

> When I turned off GRO, the router operate normally, and there is no
problem.
> and with 3.4.91 kernel, the router has no problem.
> 
> I doubt 'ipv4: ip_forward: fix inverted local_df test' patch.
> (http://patchwork.ozlabs.org/patch/345509/)
> When I revert this patch, the router has no problem.

Can you please cherry-pick following patch on top of vanilla 3.4.92?
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=2
1d1196a35f5686c4323e42a62fdb4b23b0ab4a3

commit 21d1196a35f5686c4323e42a62fdb4b23b0ab4a3
Author: Eric Dumazet <edumazet@google.com>
ipv4: set transport header earlier

I think that should fix this bug, it should apply cleanly on top of 3.4.y
tree.

[ patch is in 3.11, also backported to 3.10.y tree ]

The problem is that, when dealing with GRO packets, we try to determine the
size of the individual packets.  To do this, we rely on the transport
header.

Unfortunately the transport header is not set for the forward path in 3.4,
so we look at the network header instead.

Re: GRO issue with kernel 3.4.94 (icmp fragmentation needed)

[David Miller]

From: Florian Westphal <fw@strlen.de>
Date: Wed, 30 Jul 2014 08:37:03 +0200

> However, given the numerous problems reported with the GRO changes in
> older stable kernels I think the commit
> 29a3cd46644ec8098dbe1c12f89643b5c11831a9 in 3.4.y should be reverted.
> 
> [ In 3.2.y its caa5344994778a2b4725b2d75c74430f76925e4a ]
 ...
> David, in case I remember wrong and you did not recommend the revert
> after all, do you want me to submit 3.2/3.4.y backport of Erics
> 'set transport header earlier' patch?

Yes, the above two patches above should be reverted.

Thanks Florian.