icmp_unreachable uses wrong ip

icmp_unreachable uses wrong ip

[J. Simonetti]

[-- Attachment #1: Type: text/plain, Size: 561 bytes --]

I've recently come to notice that traceroutes through a linux router use
the wrong ip (ip of exitting interface) wich should actually be the ip
of the incomming interface.

I've found a trivial patch (attached) wich resolves this. Perhaps this
is something to include (or have a sysctl to toggle this behaviour). I
unfortunately have no knowledge of programming, so the sysctl option
would have te be done by a volunteer.

Regards,

Jeroen Simonetti
--
Netland Internet Services   http://www.netland.nl
I despise the pleasure of pleasing people whom I despise.

[-- Attachment #2: icmp_traceroute.patch --]
[-- Type: text/x-patch, Size: 380 bytes --]

--- net/ipv4/icmp.orig   2005-05-02 04:55:53.512447464 +0200
+++ net/ipv4/icmp.c      2005-05-02 04:56:21.370212440 +0200
@@ -512,7 +512,7 @@
 
	saddr = iph->daddr;
	if (!(rt->rt_flags & RTCF_LOCAL))
-		saddr = 0;
+		saddr = inet_select_addr(skb_in->dev, 0, RT_SCOPE_LINK);
 
	tos = icmp_pointers[type].error ? ((iph->tos & IPTOS_TOS_MASK) |
					   IPTOS_PREC_INTERNETCONTROL) :

Re: icmp_unreachable uses wrong ip

[Hasso Tepper]

[-- Attachment #1: Type: text/plain, Size: 1094 bytes --]

J. Simonetti wrote:
> I've recently come to notice that traceroutes through a linux router use
> the wrong ip (ip of exitting interface) wich should actually be the ip
> of the incomming interface.

There is nothing _wrong_ with this behaviour. Linux just behaves this way.

> I've found a trivial patch (attached) wich resolves this. Perhaps this
> is something to include (or have a sysctl to toggle this behaviour). I
> unfortunately have no knowledge of programming, so the sysctl option
> would have te be done by a volunteer.

Similar patches have been posted to the list repeatedly AFAIK with no any 
response from developers. Can someone enlighten us why? Is there something 
wrong with it? This is the feature people administrating routers would like 
to have. It makes debugging complicated topologies much easier and all 
routers I have seen behave this way.

I'm attaching patch which was posted to the list some time ago. I'm using it 
for some time already. I can't find the post from the archive though at the 
moment.


-- 
Hasso Tepper
Elion Enterprises Ltd.
WAN administrator

[-- Attachment #2: linux-2.4.26-icmperrors.patch --]
[-- Type: text/x-diff, Size: 2298 bytes --]

diff -ru linux-2.4.26/include/linux/sysctl.h linux-2.4.26-icmperrors/include/linux/sysctl.h
--- linux-2.4.26/include/linux/sysctl.h	2004-04-14 14:05:40.000000000 +0100
+++ linux-2.4.26-icmperrors/include/linux/sysctl.h	2004-09-23 22:52:16.000000000 +0100
@@ -314,6 +314,7 @@
 	NET_IPV4_IPFRAG_SECRET_INTERVAL=94,
 	NET_TCP_WESTWOOD=95,
 	NET_IPV4_IGMP_MAX_MSF=96,
+	NET_IPV4_ICMP_ERRORS_USE_INBOUND_IFADDR=97,
 };
 
 enum {
diff -ru linux-2.4.26/net/ipv4/icmp.c linux-2.4.26-icmperrors/net/ipv4/icmp.c
--- linux-2.4.26/net/ipv4/icmp.c	2004-04-14 14:05:41.000000000 +0100
+++ linux-2.4.26-icmperrors/net/ipv4/icmp.c	2004-09-23 22:52:38.000000000 +0100
@@ -162,6 +162,7 @@
 
 int sysctl_icmp_ratelimit = 1*HZ;
 int sysctl_icmp_ratemask = 0x1818;
+int sysctl_icmp_errors_use_inbound_ifaddr = 0;
 
 /*
  *	ICMP control array. This specifies what to do with each ICMP.
@@ -452,8 +453,12 @@
 #endif
 
 	saddr = iph->daddr;
-	if (!(rt->rt_flags & RTCF_LOCAL))
-		saddr = 0;
+	if (!(rt->rt_flags & RTCF_LOCAL)) {
+		if(sysctl_icmp_errors_use_inbound_ifaddr)
+			saddr = inet_select_addr(skb_in->dev, 0, RT_SCOPE_LINK);
+		else
+			saddr = 0;
+	}
 
 	tos = icmp_pointers[type].error ?
 		((iph->tos & IPTOS_TOS_MASK) | IPTOS_PREC_INTERNETCONTROL) :
diff -ru linux-2.4.26/net/ipv4/sysctl_net_ipv4.c linux-2.4.26-icmperrors/net/ipv4/sysctl_net_ipv4.c
--- linux-2.4.26/net/ipv4/sysctl_net_ipv4.c	2004-04-14 14:05:41.000000000 +0100
+++ linux-2.4.26-icmperrors/net/ipv4/sysctl_net_ipv4.c	2004-09-23 22:53:07.000000000 +0100
@@ -22,6 +22,7 @@
 extern int sysctl_icmp_echo_ignore_all;
 extern int sysctl_icmp_echo_ignore_broadcasts;
 extern int sysctl_icmp_ignore_bogus_error_responses;
+extern int sysctl_icmp_errors_use_inbound_ifaddr;
 
 /* From ip_fragment.c */
 extern int sysctl_ipfrag_low_thresh;
@@ -181,6 +182,9 @@
 	{NET_IPV4_ICMP_IGNORE_BOGUS_ERROR_RESPONSES, "icmp_ignore_bogus_error_responses",
 	 &sysctl_icmp_ignore_bogus_error_responses, sizeof(int), 0644, NULL,
 	 &proc_dointvec},
+	{NET_IPV4_ICMP_ERRORS_USE_INBOUND_IFADDR, "icmp_errors_use_inbound_ifaddr",
+	 &sysctl_icmp_errors_use_inbound_ifaddr, sizeof(int), 0644, NULL,
+	 &proc_dointvec},
 	{NET_IPV4_ROUTE, "route", NULL, 0, 0555, ipv4_route_table},
 #ifdef CONFIG_IP_MULTICAST
 	{NET_IPV4_IGMP_MAX_MEMBERSHIPS, "igmp_max_memberships",

Re: icmp_unreachable uses wrong ip

[Patrick McHardy]

Hasso Tepper wrote:
> Similar patches have been posted to the list repeatedly AFAIK with no any 
> response from developers. Can someone enlighten us why? Is there something 
> wrong with it? This is the feature people administrating routers would like 
> to have. It makes debugging complicated topologies much easier and all 
> routers I have seen behave this way.

Why can't you simply add the prefered source address to the route?

Regards
Patrick

Re: icmp_unreachable uses wrong ip

[Hasso Tepper]

Patrick McHardy wrote:
> Hasso Tepper wrote:
> > Similar patches have been posted to the list repeatedly AFAIK with no
> > any response from developers. Can someone enlighten us why? Is there
> > something wrong with it? This is the feature people administrating
> > routers would like to have. It makes debugging complicated topologies
> > much easier and all routers I have seen behave this way.
>
> Why can't you simply add the prefered source address to the route?

Because I don't know what it is. Router A knows what's the best path from 
router A to router B, but he can't know (at least in cases where there are 
more than 1 path between them) what's the best path from router B to router 
A. Therefore you canät say which one is incoming interface in router A for 
traffic from router B. And even if you know it in some moment, topology 
might change in next moment (dynamic routing) etc.

-- 
Hasso

Re: icmp_unreachable uses wrong ip

[Patrick McHardy]

Hasso Tepper wrote:
> Patrick McHardy wrote:
> 
>>Why can't you simply add the prefered source address to the route?
> 
> Because I don't know what it is. Router A knows what's the best path from 
> router A to router B, but he can't know (at least in cases where there are 
> more than 1 path between them) what's the best path from router B to router 
> A. Therefore you canät say which one is incoming interface in router A for 
> traffic from router B. And even if you know it in some moment, topology 
> might change in next moment (dynamic routing) etc.

Your patch can't guarantee that the address used is the same that was
used as nexthop by the previous hop in the path when multiple addresses
are configured on the incoming interface. So I don't think it achieves
much of your goal of making debugging complicated topologies easier.

Regards
Patrick

Re: icmp_unreachable uses wrong ip

[Hasso Tepper]

Patrick McHardy wrote:
> Hasso Tepper wrote:
> > Patrick McHardy wrote:
> >>Why can't you simply add the prefered source address to the route?
> >
> > Because I don't know what it is. Router A knows what's the best path
> > from router A to router B, but he can't know (at least in cases where
> > there are more than 1 path between them) what's the best path from
> > router B to router A. Therefore you canät say which one is incoming
> > interface in router A for traffic from router B. And even if you know
> > it in some moment, topology might change in next moment (dynamic
> > routing) etc.
>
> Your patch can't guarantee that the address used is the same that was
> used as nexthop by the previous hop in the path when multiple addresses
> are configured on the incoming interface. So I don't think it achieves
> much of your goal of making debugging complicated topologies easier.

At first I don't care what was used as nexthop. I want to know which 
physical link was used. Having multiple addresses in the same link is more 
corner case in core network anyway.

And can you explain what theoretical possibilities router has to obtain info 
what address was used as nexthop by neighbour?


regards,

-- 
Hasso Tepper
Elion Enterprises Ltd.
WAN administrator

Re: icmp_unreachable uses wrong ip

[Patrick McHardy]

Hasso Tepper wrote:
> Patrick McHardy wrote:
>>
>>Your patch can't guarantee that the address used is the same that was
>>used as nexthop by the previous hop in the path when multiple addresses
>>are configured on the incoming interface. So I don't think it achieves
>>much of your goal of making debugging complicated topologies easier.
> 
> At first I don't care what was used as nexthop. I want to know which 
> physical link was used. Having multiple addresses in the same link is more 
> corner case in core network anyway.

But when multiple addresses are used the result can be even more
confusing. I don't like inconsistent behaviour, and this patch works
sometimes and sometimes it doesn't.

> And can you explain what theoretical possibilities router has to obtain info 
> what address was used as nexthop by neighbour?

I can think of none.

Regards
Patrick

Re: icmp_unreachable uses wrong ip

[Hasso Tepper]

Patrick McHardy wrote:
> But when multiple addresses are used the result can be even more
> confusing. I don't like inconsistent behaviour, and this patch works
> sometimes and sometimes it doesn't.

I see no behaviour you can define as "it doesn't work". Purpose of this 
patch is to provide info about links (not addresses, you can't have this 
info) used to forward packets and it does the job. 

> > And can you explain what theoretical possibilities router has to obtain
> > info what address was used as nexthop by neighbour?
>
> I can think of none.

Exactly.

regards,

-- 
Hasso Tepper
Elion Enterprises Ltd.
WAN administrator

Re: icmp_unreachable uses wrong ip

[Patrick McHardy]

Hasso Tepper wrote:
> Patrick McHardy wrote:
> 
>>But when multiple addresses are used the result can be even more
>>confusing. I don't like inconsistent behaviour, and this patch works
>>sometimes and sometimes it doesn't.
> 
> 
> I see no behaviour you can define as "it doesn't work". Purpose of this 
> patch is to provide info about links (not addresses, you can't have this 
> info) used to forward packets and it does the job. 

Well, arguably it can be called "doesn't work" if addresses not used
at all during transmit of the packet show up in traceroute.

Regards
Patrick

Re: icmp_unreachable uses wrong ip

[Lennert Buytenhek]

On Wed, May 04, 2005 at 01:37:40AM +0200, Patrick McHardy wrote:

> >>But when multiple addresses are used the result can be even more
> >>confusing. I don't like inconsistent behaviour, and this patch works
> >>sometimes and sometimes it doesn't.
> > 
> > I see no behaviour you can define as "it doesn't work". Purpose of this 
> > patch is to provide info about links (not addresses, you can't have this 
> > info) used to forward packets and it does the job. 
> 
> Well, arguably it can be called "doesn't work" if addresses not used
> at all during transmit of the packet show up in traceroute.

That argument doesn't hold, since exactly the same situation occurs
if we use the outgoing address as we do now.

With asymmetric routing, the incoming interface, interface to
the destination, and the interface back to the source might all
be different, so we can end up with:

       path from r3 back to a
       +-------------------+
       |                   |
       V                   |
       a --- r1 --- r2 --- r3 --- r4 --- r5 --- b

The address of the 'upper' interface of r3 is likewise "not used
at all during transmit of the packet", but it is the address we
currently send the ICMP from.


--L

Re: icmp_unreachable uses wrong ip

[Meelis Roos]

PM> Your patch can't guarantee that the address used is the same that was
PM> used as nexthop by the previous hop in the path when multiple addresses
PM> are configured on the incoming interface. So I don't think it achieves
PM> much of your goal of making debugging complicated topologies easier.

Nevertheless this patch seems to be the most logical thing to do and is
_much_ better than current state IMHO. Principle of least surprise.

-- 
Meelis Roos

Re: icmp_unreachable uses wrong ip

[Hasso Tepper]

Hasso Tepper wrote:
> I'm attaching patch which was posted to the list some time ago. I'm using
> it for some time already. I can't find the post from the archive though
> at the moment.

Chris Wilson was probably author of this patch.
http://marc.theaimsgroup.com/?l=linux-net&m=109595048606145&w=2


-- 
Hasso Tepper
Elion Enterprises Ltd.
WAN administrator

Re: icmp_unreachable uses wrong ip

[Lennert Buytenhek]

On Mon, May 02, 2005 at 03:21:19PM +0200, J. Simonetti wrote:

> I've recently come to notice that traceroutes through a linux router use
> the wrong ip (ip of exitting interface) wich should actually be the ip
> of the incomming interface.
> 
> I've found a trivial patch (attached) wich resolves this. Perhaps this
> is something to include (or have a sysctl to toggle this behaviour). I
> unfortunately have no knowledge of programming, so the sysctl option
> would have te be done by a volunteer.

For what it's worth, I would love to see something like this in.
In the presence of asymmetric routing, the way linux routers show
up on traceroutes has always been slightly confusing to me.

(I know, "Every other router vendor out there does it this way."
is not a good reason for doing it the same way.)


--L

Re: icmp_unreachable uses wrong ip

[Pekka Savola]

On Tue, 3 May 2005, Lennert Buytenhek wrote:
> (I know, "Every other router vendor out there does it this way."
> is not a good reason for doing it the same way.)

There's no specification requiring or recommending either of these 
approaches, either for v6 or v4 AFAICS so you can't depend on that 
behaviour..

Do you have specific data to back up "every other router vendor..." ? 
I doubt it's quite as uniform as that, but if so, it would certainly 
be a major motivation for a change.

-- 
Pekka Savola                 "You each name yourselves king, yet the
Netcore Oy                    kingdom bleeds."
Systems. Networks. Security. -- George R.R. Martin: A Clash of Kings

Re: icmp_unreachable uses wrong ip

[Lennert Buytenhek]

On Tue, May 03, 2005 at 12:42:01PM +0300, Pekka Savola wrote:

> >(I know, "Every other router vendor out there does it this way."
> >is not a good reason for doing it the same way.)
> 
> There's no specification requiring or recommending either of these 
> approaches, either for v6 or v4 AFAICS so you can't depend on that 
> behaviour..
> 
> Do you have specific data to back up "every other router vendor..." ? 

Sorry, I wasn't claiming that every other router vendor out there
does it that way, just merely trying to state that that would not
be a good argument.

The ones I've worked with in the past do, and I just did some
simple tests (doing traceroute from a foreign IP in a different
network and looking at the ICMP time-exceededs arriving at that
foreign IP) to confirm that:
- Juniper M5 and M20 both running JUNOS 7.0R2.7
- Cisco 7206VXR and 12k(unsure which model) running unknown IOS versions
- Foundry FastIron 3 (the 15-slot chassis) running BIR06636.bin

do all behave opposite of the linux way, i.e. sending ICMPs with (one
of) the source address(es) of the interface where the original packet
(that we're sending an ICMP for) came in.

I did also find one router that behaves the linux way:
- Foundry TurboIron/8 running 07.800A

Any other vendors that we're interested in?


--L

Re: icmp_unreachable uses wrong ip

[Hasso Tepper]

Lennert Buytenhek wrote:
> The ones I've worked with in the past do, and I just did some
> simple tests (doing traceroute from a foreign IP in a different
> network and looking at the ICMP time-exceededs arriving at that
> foreign IP) to confirm that:
> - Juniper M5 and M20 both running JUNOS 7.0R2.7
> - Cisco 7206VXR and 12k(unsure which model) running unknown IOS versions

I haven't seen JUNOS or IOS versions behave differently.

> - Foundry FastIron 3 (the 15-slot chassis) running BIR06636.bin

I can add to it (probably all) Extreme Networks line. Tested with both 
software lines - Summit 24e3 and Inferno (Black Diamond 6808).

> do all behave opposite of the linux way, i.e. sending ICMPs with (one
> of) the source address(es) of the interface where the original packet
> (that we're sending an ICMP for) came in.


-- 
Hasso Tepper
Elion Enterprises Ltd.
WAN administrator

Re: icmp_unreachable uses wrong ip

[Glen Turner]

Pekka Savola wrote:
> 
> 
> There's no specification requiring or recommending either of these 
> approaches, either for v6 or v4 AFAICS so you can't depend on that 
> behaviour..

In the lack of a specification, doing what works best for
network operators would be nice :-)

And that behaviour is that addresses in a traceroute should
be ping-able.  That is, you're tracing the route and there's
loss at one point, so you want to determine how much loss.

The outgoing interface's address may not be reached down the
same path as used by the traceroute, whereis the incoming address
obviously is (or if it isn't, then that itself is interesting
diagnostic information).

Similarly, using the incoming interface gives more information
about asymetric routes.

In short, since using the incoming address gives more useful
output and the specification is silent as to which address
is allowable, using the incoming interface's address would
be the most useful choice.

Cheers,
Glen

-- 
  Glen Turner         Tel: (08) 8303 3936 or +61 8 8303 3936
  Australia's Academic & Research Network  www.aarnet.edu.au