Problem with PMTU discovery on ICMP packets

Problem with PMTU discovery on ICMP packets

[Svenning Soerensen]

Hi there,

I think there is a bug in the 2.4 icmp code regarding PMTU discovery.
It seems to be inconsistent between reboots: at one boot echo replies always
have the DF bit set, while after another boot they don't, indicating that
this is caused by an uninitialized parameter.
Even 'echo 1 > /proc/sys/net/ipv4/ip_no_pmtu_disc' doesn't change this
behaviour.

I can't think of any legitimate reasons for doing PMTU discovery on ICMP
packets, and since it actually in some situations breaks ping in combination
with FreeS/WAN (I can elaborate, if anyone is interested), I would suggest
to turn it off in a consistent manner.
The patch below (against 2.4.4) should accomplish this.

Svenning

--- linux/net/ipv4/icmp.c	2001/04/28 12:30:34	1.1.1.2
+++ linux/net/ipv4/icmp.c	2001/05/05 11:06:41
@@ -1006,6 +1006,7 @@
 	icmp_socket->sk->allocation=GFP_ATOMIC;
 	icmp_socket->sk->sndbuf = SK_WMEM_MAX*2;
 	icmp_socket->sk->protinfo.af_inet.ttl = MAXTTL;
+	icmp_socket->sk->protinfo.af_inet.pmtudisc = IP_PMTUDISC_DONT;
 
 	/* Unhash it so that IP input processing does not even
 	 * see it, we do not wish this socket to see incoming

Re: Problem with PMTU discovery on ICMP packets

[David S. Miller]

Svenning Soerensen writes:
 > I think there is a bug in the 2.4 icmp code regarding PMTU discovery.
 > It seems to be inconsistent between reboots: at one boot echo replies always
 > have the DF bit set, while after another boot they don't, indicating that
 > this is caused by an uninitialized parameter.
 > Even 'echo 1 > /proc/sys/net/ipv4/ip_no_pmtu_disc' doesn't change this
 > behaviour.
 > 
 > I can't think of any legitimate reasons for doing PMTU discovery on ICMP
 > packets, and since it actually in some situations breaks ping in combination
 > with FreeS/WAN (I can elaborate, if anyone is interested), I would suggest
 > to turn it off in a consistent manner.
 > The patch below (against 2.4.4) should accomplish this.

I want to understand why on some boots it's on while on some
boots it is off, that makes no sense.

Before the piece of code you patched, we call ops->create() which is
inet_create in net/ipv4/af_inet.c, there is sets the PMTU discovery
disposition based upon the setting in ipv4_config which should always
have the same setting at the point during boot, every boot.

You need to figure out why it sometimes gets set and sometimes does
not, then we can figure out how to fix it.

Later,
David S. Miller
davem@redhat.com

RE: Problem with PMTU discovery on ICMP packets

[Svenning Soerensen]

> From: David S. Miller [mailto:davem@redhat.com]

> I want to understand why on some boots it's on while on some
> boots it is off, that makes no sense.
> 
> Before the piece of code you patched, we call ops->create() which is
> inet_create in net/ipv4/af_inet.c, there is sets the PMTU discovery
> disposition based upon the setting in ipv4_config which should always
> have the same setting at the point during boot, every boot.
> 
> You need to figure out why it sometimes gets set and sometimes does
> not, then we can figure out how to fix it.

Yes, I see your point. I guess I made an incorrect assumption about it
being changed between reboots. It could be related to routing or something
instead. I'll have to dig a bit further to find a pattern. 

However, even if I *do* find the pattern, I still think it is reasonable
to turn off PMTU discovery for ICMP explicitly, instead of based on the
setting of ipv4_config:

1) The setting for the one global ICMP socket gets set once and for all
at initialization time and doesn't track the setting of ip_no_pmtu_disc.

2) Even if it *did* track ip_no_pmtu_disc, the missing per-protocol
granularity of ip_no_pmtu_disc means that you couldn't have PMTU discovery
enabled for TCP, while having it disabled for ICMP.

3) I don't see any reason for wanting to do automatic PMTU discovery for
ICMP packets; normally they are only sent once, and you'll risk it being
dropped somewhere along its path if the DF bit is set.

Svenning

RE: Problem with PMTU discovery on ICMP packets

[David S. Miller]

Svenning Soerensen writes:
 > Yes, I see your point. I guess I made an incorrect assumption about it
 > being changed between reboots. It could be related to routing or something
 > instead. I'll have to dig a bit further to find a pattern. 
 > 
 > However, even if I *do* find the pattern, I still think it is reasonable
 > to turn off PMTU discovery for ICMP explicitly, instead of based on the
 > setting of ipv4_config:

I totally agree with you.

But I first want to know the real story behind this reboot anomaly.
Then we will fix any new bug we discover, and apply the icmp patch as
well.

Later,
David S. Miller
davem@redhat.com

RE: Problem with PMTU discovery on ICMP packets

[Svenning Soerensen]

> From: David S. Miller [mailto:davem@redhat.com]

> But I first want to know the real story behind this reboot anomaly.
> Then we will fix any new bug we discover, and apply the icmp patch as
> well.

I've done a bit more testing. The behaviour doesn't change across reboots.
Instead, it seems to be the case that:
If the packet fits within the MTU of the outgoing interface, DF is set.
If the packet doesn't fit, and thus gets fragmented, DF is clear on all
fragments.
Does this make sense?

I think the reason it looked like it changed across reboots might be caused
by the fact that it was influenced by a FreeS/WAN tunnel, which, after a
reboot, didn't always get started successfully.
If the tunnel was down, the packets (2 kB pings) would go out eth0 fragmented
and with DF clear.
If the tunnel was up, they would get routed through the ipsec0 interface
(with a MTU of 16kB) unfragmented and with DF set.

In the latter (normal) case, after arrival and decapsulation at the endpoint
of the tunnel, the packet is doomed because it needs fragmentation to travel
further, but DF is set.


Svenning

RE: Problem with PMTU discovery on ICMP packets

[David S. Miller]

Svenning Soerensen writes:
 > I've done a bit more testing. The behaviour doesn't change across reboots.
 > Instead, it seems to be the case that:
 > If the packet fits within the MTU of the outgoing interface, DF is set.
 > If the packet doesn't fit, and thus gets fragmented, DF is clear on all
 > fragments.
 > Does this make sense?

Yes.  I've put the following patch into my tree.
Thanks for doing the detective work.

--- net/ipv4/icmp.c.~1~	Sun Apr 29 21:40:40 2001
+++ net/ipv4/icmp.c	Wed May  9 18:20:58 2001
@@ -3,7 +3,7 @@
  *	
  *		Alan Cox, <alan@redhat.com>
  *
- *	Version: $Id: icmp.c,v 1.75 2001/04/30 04:40:40 davem Exp $
+ *	Version: $Id: icmp.c,v 1.76 2001/05/10 01:20:58 davem Exp $
  *
  *	This program is free software; you can redistribute it and/or
  *	modify it under the terms of the GNU General Public License
@@ -1006,6 +1006,7 @@
 	icmp_socket->sk->allocation=GFP_ATOMIC;
 	icmp_socket->sk->sndbuf = SK_WMEM_MAX*2;
 	icmp_socket->sk->protinfo.af_inet.ttl = MAXTTL;
+	icmp_socket->sk->protinfo.af_inet.pmtudisc = IP_PMTUDISC_DONT;
 
 	/* Unhash it so that IP input processing does not even
 	 * see it, we do not wish this socket to see incoming