Determining IP:port corresponding to an ICMP port unreachable

Determining IP:port corresponding to an ICMP port unreachable

[Erik De Bonte]

When an ICMP port unreachable message is received and corresponds to a UDP
socket, is there a way to determine the corresponding unreachable IP and
port?  I'm able to retrieve the IP, but not the port.  From looking through
the kernel source, it appears that the port is never extracted from the
payload section of the ICMP message.  If this is indeed a limitation of the
kernel, is there a plan to "fix" it in the future?

Here are the details on my particular situation:

I'm working on a game server which interacts with a large number of clients
via a single UDP socket.  Occasionally, one of the clients will die without
sending a disconnect message.  When this happens, I'd like to remove the
client as quickly as possible to avoid leaving a ghost in the world that
other players will see.  In the worst case scenario, the session will time
out after the server hasn't heard from the client in x seconds.  However, if
I watch for ICMP port unreachable messages, I should frequently be able to
react more quickly.

With Winsock, this is easy to do.  Recvfrom fails, an error code tells me
that an ICMP port unreachable was received, and the address parameter of
recvfrom is filled in with the dead client's IP and port.  On Linux (I'm
using 2.2.16, but 2.4.x code appears to be the same in this respect),
recvfrom fails and errno is set to ECONNREFUSED indicating an ICMP port
unreachable was received.  However, the address is not filled in.  I'm able
to retrieve the IP via recvmsg with the MSG_ERRQUEUE flag (and the
IP_RECVERR sockopt), but the port that it gives me is bogus.

I apologize if this seems too application specific for linux-kernel, but
this appears to be a limitation of the kernel, and I haven't been able to
find any info elsewhere.

Thanks,
Erik

Erik L. De Bonte
Lead Server Programmer
LithTech, Inc. - http://www.lithtech.com

Re: Determining IP:port corresponding to an ICMP port unreachable

[Nadav Har'El]

On Mon, Jul 30, 2001, Erik De Bonte wrote about "Determining IP:port corresponding to an ICMP port unreachable":
> When an ICMP port unreachable message is received and corresponds to a UDP
> socket, is there a way to determine the corresponding unreachable IP and
> port?  I'm able to retrieve the IP, but not the port.  From looking through
> the kernel source, it appears that the port is never extracted from the
> payload section of the ICMP message.  If this is indeed a limitation of the
> kernel, is there a plan to "fix" it in the future?

If you recvfrom (for example) on a UDP socket (which, obviously, has some
port number) on which you sent a message previously, recvfrom will return
(-1) (with errno=connection refused) if an ICMP port unreachable was received
by the kernel for this port. This kind of error is asynchronous, in the sense
that you will get it some time later after sending the original message (you
could have sent and received a dozen other messages in the meantime).

For connected()ed sockets, this behavior is indeed useful - you know which
port sent the message, which host and port was meant to get that message
(because the socket is connected() and only sends to one host/port).

But for non-connected()ed sockets, you can only find out the host sending the
ICMP message. Note that sometimes (e.g., with host unreachable errors) you
don't even know the host you orignally sent the message to (that is burried
in the IP heard inside the ICMP data) - only the host that sent you the
error. And you don't know any port number (again, the port number is inside
the ICMP packet, but you have no access to it - this is what you wrote too).

This is why the original BSD behavior was to pass these errors only on
connect()ed sockets. Linux decided to give those errors on unconnect()ed
sockets - while it is usually not useful, it fits more closely with RFC 1122
which says in section 4.1.2.3: "UDP MUST pass to the application layer all
ICMP error messages that it receives from the IP layer".

There's a discussion about this issue in Stevens' book ""UNIX Network
Programming", section 8.9 (Elementary UDP Sockets, Server Not Running),
page 221, and he discusses why the socket API is problematic in that respect.

I think the only recourse you have (if you really want to know which
host/port every ICMP message is about) is to listen on a raw socket, which
you open with something like
	in_icmp=socket(AF_INET, SOCK_RAW, IPPROTO_ICMP);
	shutdown(in_icmp,SHUT_WR); /* optional (we don't intend to write) */

And then you'll get full ICMP packets (all of them!) - and you'll have to
pick out the ones intended for your port(s), and then take out the destination
ip and port inside the ip header that is inside the ICMP packet (not the
ip header of the ICMP packet itself!). This is rather ugly, because it requires
you to understand how IP and UDP headers look like.
Note that you need superuser permissions to create (but not to read) a
raw socket.

for example (just unchecked pseudo-code based on stuff I found in the
traceroute(8) sourcecode, but it will work with minor modifications)

#include <netinet/ip_icmp.h>
#include <netinet/udp.h>

parse_icmp(unsigned char *packet, int cc)
{
	struct ip *ip=(struct ip *) packet;
	struct icmp *icp;
	int hlen, icmp_len;

	hlen = ip->ip_hl << 2;
	if (cc < hlen + ICMP_MINLEN)
		/* packet too short... */

	icmp_len = cc-hlen;
	icp=(struct icmp *)(packet + hlen);

	if(icp->icmp_type==ICMP_TIMXCEED||
        icp->icmp_type==ICMP_UNREACH){
	/* this is an ICMP error message and it carries part of the original
	   packet, so we can see what this error is refering to. */
		struct ip *inner_ip = &icp->icmp_ip;
		hlen = inner_ip->ip_hl << 2;
		if(inner_ip->ip_p == IPPROTO_UDP){
		    /* This is a time exceeded error on some UDP packet we sent.
		       Note that it is guaranteed we get back the UDP header
                      (8 bytes) of the original ICMP packet (see the RFC),
                      but not necessarily the whole packet.
		      inner_udp is part of the original ICMP packet:
		    */
		    struct udphdr *inner_udp =
                      (struct udphdr *)((u_char *)inner_ip + hlen);
                    /* the UDP header is 8 bytes: see udp.h. we'll use
                       sizeof(udphdr) for clarity, but that will always be 8 */
                    if(icmp_len < hlen + sizeof(struct udphdr))
                       /** error: packet too short. do something. **/

		    /* now we can check if this is an error on a packet sent
                       by one of our sockets */
		    if(!is_one_of_my_sockets(inner_ip->ip_src, inner_udp->source)
			return; /* not one of ours */
		    /* just as an example, do something on port unreachable
		       errors */
		    if(icp->icmp_type==ICMP_UNREACH &&
			icp->icmp_code==ICMP_UNREACH_PORT
				){
			/* do something about this error. The problematic
			   packet was sent from our socket at ip
                           inner_ip->ip_src and port inner_udp->source,
			   and it was sent to ip inner_ip->ip_dst, port
			   inner_udp->dest; The error was returned from host
			   ip->ip_src (not always inner_ip->ip_src */
			...
		     }
		}	
	}
}

I hope this helps.

-- 
Nadav Har'El                        |         Tuesday, Jul 31 2001, 11 Av 5761
nyh@math.technion.ac.il             |-----------------------------------------
Phone: +972-53-245868, ICQ 13349191 |The space between my ears was
http://nadav.harel.org.il           |intentionally left blank.

Re: Determining IP:port corresponding to an ICMP port unreachable

[Philippe Troin]

"Nadav Har'El" <nyh@math.technion.ac.il> writes:

> On Mon, Jul 30, 2001, Erik De Bonte wrote about "Determining IP:port
> corresponding to an ICMP port unreachable":

> > When an ICMP port unreachable message is received and corresponds
> > to a UDP socket, is there a way to determine the corresponding
> > unreachable IP and port?  I'm able to retrieve the IP, but not the
> > port.  From looking through the kernel source, it appears that the
> > port is never extracted from the payload section of the ICMP
> > message.  If this is indeed a limitation of the kernel, is there a
> > plan to "fix" it in the future?
> 
> If you recvfrom (for example) on a UDP socket (which, obviously, has
> some port number) on which you sent a message previously, recvfrom
> will return (-1) (with errno=connection refused) if an ICMP port
> unreachable was received by the kernel for this port. This kind of
> error is asynchronous, in the sense that you will get it some time
> later after sending the original message (you could have sent and
> received a dozen other messages in the meantime).
> 
> For connected()ed sockets, this behavior is indeed useful - you know
> which port sent the message, which host and port was meant to get
> that message (because the socket is connected() and only sends to
> one host/port).
> 
> But for non-connected()ed sockets, you can only find out the host
> sending the ICMP message. Note that sometimes (e.g., with host
> unreachable errors) you don't even know the host you orignally sent
> the message to (that is burried in the IP heard inside the ICMP
> data) - only the host that sent you the error. And you don't know
> any port number (again, the port number is inside the ICMP packet,
> but you have no access to it - this is what you wrote too).
> 
> This is why the original BSD behavior was to pass these errors only
> on connect()ed sockets. Linux decided to give those errors on
> unconnect()ed sockets - while it is usually not useful, it fits more
> closely with RFC 1122 which says in section 4.1.2.3: "UDP MUST pass
> to the application layer all ICMP error messages that it receives
> from the IP layer".
> 
> There's a discussion about this issue in Stevens' book ""UNIX
> Network Programming", section 8.9 (Elementary UDP Sockets, Server
> Not Running), page 221, and he discusses why the socket API is
> problematic in that respect.
> 
> I think the only recourse you have (if you really want to know which
> host/port every ICMP message is about) is to listen on a raw socket, which
> you open with something like
> 	in_icmp=socket(AF_INET, SOCK_RAW, IPPROTO_ICMP);
> 	shutdown(in_icmp,SHUT_WR); /* optional (we don't intend to write) */
> 
> And then you'll get full ICMP packets (all of them!) - and you'll
> have to pick out the ones intended for your port(s), and then take
> out the destination ip and port inside the ip header that is inside
> the ICMP packet (not the ip header of the ICMP packet itself!). This
> is rather ugly, because it requires you to understand how IP and UDP
> headers look like.  Note that you need superuser permissions to
> create (but not to read) a raw socket.

Nah, on linux, use setsockopt with IP_RECVERR.
man 7 ip

Phil.

RE: Determining IP:port corresponding to an ICMP port unreachable

[Erik De Bonte]

Philippe Troin said:
> Nah, on linux, use setsockopt with IP_RECVERR.
> man 7 ip

I've already tried that, but was only able to retrieve the IP, not the port.
>From my original message: "I'm able to retrieve the IP via recvmsg with the
MSG_ERRQUEUE flag (and the IP_RECVERR sockopt), but the port that it gives
me is bogus."

Nadav Har'El said:
> But for non-connected()ed sockets, you can only find out the host
> sending the ICMP message.

Why?  The remote port is in the ICMP message (64-bits of the undeliverable
message's header are in there), right?  Why can't the kernel net code
extract the port and give it to me?  It's obviously possible, since Winsock
does it.**

-Erik

** Btw, please don't take that as Linux bashing.  Our server runs
significantly faster on Linux, which is awesome!  However, there are a few
small issues like this one that are causing problems for me.

-----Original Message-----
From: Philippe Troin [mailto:phil@fifi.org]
Sent: Tuesday, July 31, 2001 9:42 AM
To: Nadav Har'El
Cc: Erik De Bonte; linux-kernel@vger.kernel.org
Subject: Re: Determining IP:port corresponding to an ICMP port
unreachable


"Nadav Har'El" <nyh@math.technion.ac.il> writes:

> On Mon, Jul 30, 2001, Erik De Bonte wrote about "Determining IP:port
> corresponding to an ICMP port unreachable":

> > When an ICMP port unreachable message is received and corresponds
> > to a UDP socket, is there a way to determine the corresponding
> > unreachable IP and port?  I'm able to retrieve the IP, but not the
> > port.  From looking through the kernel source, it appears that the
> > port is never extracted from the payload section of the ICMP
> > message.  If this is indeed a limitation of the kernel, is there a
> > plan to "fix" it in the future?
> 
> If you recvfrom (for example) on a UDP socket (which, obviously, has
> some port number) on which you sent a message previously, recvfrom
> will return (-1) (with errno=connection refused) if an ICMP port
> unreachable was received by the kernel for this port. This kind of
> error is asynchronous, in the sense that you will get it some time
> later after sending the original message (you could have sent and
> received a dozen other messages in the meantime).
> 
> For connected()ed sockets, this behavior is indeed useful - you know
> which port sent the message, which host and port was meant to get
> that message (because the socket is connected() and only sends to
> one host/port).
> 
> But for non-connected()ed sockets, you can only find out the host
> sending the ICMP message. Note that sometimes (e.g., with host
> unreachable errors) you don't even know the host you orignally sent
> the message to (that is burried in the IP heard inside the ICMP
> data) - only the host that sent you the error. And you don't know
> any port number (again, the port number is inside the ICMP packet,
> but you have no access to it - this is what you wrote too).
> 
> This is why the original BSD behavior was to pass these errors only
> on connect()ed sockets. Linux decided to give those errors on
> unconnect()ed sockets - while it is usually not useful, it fits more
> closely with RFC 1122 which says in section 4.1.2.3: "UDP MUST pass
> to the application layer all ICMP error messages that it receives
> from the IP layer".
> 
> There's a discussion about this issue in Stevens' book ""UNIX
> Network Programming", section 8.9 (Elementary UDP Sockets, Server
> Not Running), page 221, and he discusses why the socket API is
> problematic in that respect.
> 
> I think the only recourse you have (if you really want to know which
> host/port every ICMP message is about) is to listen on a raw socket, which
> you open with something like
> 	in_icmp=socket(AF_INET, SOCK_RAW, IPPROTO_ICMP);
> 	shutdown(in_icmp,SHUT_WR); /* optional (we don't intend to write) */
> 
> And then you'll get full ICMP packets (all of them!) - and you'll
> have to pick out the ones intended for your port(s), and then take
> out the destination ip and port inside the ip header that is inside
> the ICMP packet (not the ip header of the ICMP packet itself!). This
> is rather ugly, because it requires you to understand how IP and UDP
> headers look like.  Note that you need superuser permissions to
> create (but not to read) a raw socket.

Nah, on linux, use setsockopt with IP_RECVERR.
man 7 ip

Phil.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Determining IP:port corresponding to an ICMP port unreachable

[Nadav Har'El]

On Tue, Jul 31, 2001, Erik De Bonte wrote about "RE: Determining IP:port corresponding to an ICMP port unreachable":
> Nadav Har'El said:
> > But for non-connected()ed sockets, you can only find out the host
> > sending the ICMP message.
> 
> Why?  The remote port is in the ICMP message (64-bits of the undeliverable
> message's header are in there), right?  Why can't the kernel net code
> extract the port and give it to me?  It's obviously possible, since Winsock
> does it.**

I outlined the problem with the standard socket API (note I said API, not
theoretical possibility to look at the packet content) in my previous message,
including a pointer to Stevens' book which explains the issue far better
than I can.

Anyway, since the IP_RECVERR is a "hack" to get more information which is
not available with the standard API, it's theoretically possible to add to it
anything, including the destination IP address and port on the original
packet. Read ip(7) carefully: it would appear that either the SOCK_EE_OFFENDER
macro or the actual data (not anciliary data) received from the error queue
can help you.

Too bad that this IP_RECVERR seems to be a completely non-standard Linux-only
feature...

-- 
Nadav Har'El                        |         Tuesday, Jul 31 2001, 12 Av 5761
nyh@math.technion.ac.il             |-----------------------------------------
Phone: +972-53-245868, ICQ 13349191 |Jury: Twelve people who determine which
http://nadav.harel.org.il           |client has the better lawyer.

Re: Determining IP:port corresponding to an ICMP port unreachable

[Alexey Kuznetsov]

Hello!

> >From my original message: "I'm able to retrieve the IP via recvmsg with the
> MSG_ERRQUEUE flag (and the IP_RECVERR sockopt), but the port that it gives
> me is bogus."

Sigh, please, send sample yet. It works here by some strange reason,
apparently, my applet is buggy. :-)


> extract the port and give it to me?  It's obviously possible, since Winsock
> does it.**

Well, better ask those brave MS folks why they did this. :-) 

Actually, please, check this again: I was not aware that nt posix env
break bsd api in such wicked way. This can be useful as a strong argument
in some curcumstances.

Alexey