[LARTC] Policy routing question

[LARTC] Policy routing question

[Peter Rabbitson]

Hi,

I have a testing multihome setup, with the default gateway being one of 
the links and using policy routing to honor requests for a specific 
link. Everything works as expected when I request a specific IP to bind 
to. But if I request a specific interface things fall apart in ways that 
I can not explain:

default gw (WORKS)
----------
rabbit@Thesaurus:~$ ping -c 1 yahoo.com
PING yahoo.com (216.109.112.135) 56(84) bytes of data.
64 bytes from w2.rc.vip.dcn.yahoo.com (216.109.112.135): icmp_seq=1 
ttlH time\x142 ms


request IP on same link as default gw (WORKS)
-------------------------------------
rabbit@Thesaurus:~$ ping -I 192.168.9.102 -c 1 yahoo.com
PING yahoo.com (66.94.234.13) from 192.168.9.102 : 56(84) bytes of data.
64 bytes from w2.rc.vip.scd.yahoo.com (66.94.234.13): icmp_seq=1 ttlG 
time\x176 ms

request IP on secondary link (WORKS)
----------------------------
rabbit@Thesaurus:~$ ping -I 172.16.0.2 -c 1 yahoo.com
PING yahoo.com (216.109.112.135) from 172.16.0.2 : 56(84) bytes of data.
64 bytes from w2.rc.vip.dcn.yahoo.com (216.109.112.135): icmp_seq=1 
ttlG time\x146 ms

request interface of default gw link (WORKS)
------------------------------------
rabbit@Thesaurus:~$ ping -I eth1 -c 1 yahoo.com
PING yahoo.com (66.94.234.13) from 192.168.9.102 eth1: 56(84) bytes of data.
64 bytes from w2.rc.vip.scd.yahoo.com (66.94.234.13): icmp_seq=1 ttlG 
time\x176 ms

request secondary interface (FAILS)
---------------------------
rabbit@Thesaurus:~$ ping -I eth0 -c 1 yahoo.com
PING yahoo.com (216.109.112.135) from 192.168.9.102 eth0: 56(84) bytes 
of data.
 From 172.16.0.2 icmp_seq=1 Destination Host Unreachable


I went over the setup again and again, but I can't figure out why the 
last ping attempt fails. Any pointers are welcome!

Thanks

Peter


Here is the setup:

ip addr
----------
2: eth0: <BROADCAST,MULTICAST,UP,10000> mtu 1500 qdisc pfifo_fast qlen 1000
     link/ether 00:11:09:8d:4f:c1 brd ff:ff:ff:ff:ff:ff
     inet 172.16.0.2/24 brd 172.16.0.255 scope global eth0
3: eth1: <BROADCAST,MULTICAST,UP,10000> mtu 1500 qdisc pfifo_fast qlen 1000
     link/ether 00:04:e2:80:b4:97 brd ff:ff:ff:ff:ff:ff
     inet 192.168.9.102/24 brd 192.168.9.255 scope global eth1


ip ro show table all
-----------------------
default via 172.16.0.1 dev eth0  table 10
default via 192.168.9.1 dev eth1  table 20
default via 192.168.9.1 dev eth1  table default
172.16.0.0/24 dev eth0  proto kernel  scope link  src 172.16.0.2
192.168.9.0/24 dev eth1  proto kernel  scope link  src 192.168.9.102
broadcast 127.255.255.255 dev lo  table local  proto kernel  scope link 
  src 127.0.0.1
broadcast 192.168.9.0 dev eth1  table local  proto kernel  scope link 
src 192.168.9.102
broadcast 172.16.0.0 dev eth0  table local  proto kernel  scope link 
src 172.16.0.2
local 192.168.9.102 dev eth1  table local  proto kernel  scope host  src 
192.168.9.102
local 172.16.0.2 dev eth0  table local  proto kernel  scope host  src 
172.16.0.2
broadcast 192.168.9.255 dev eth1  table local  proto kernel  scope link 
  src 192.168.9.102
broadcast 172.16.0.255 dev eth0  table local  proto kernel  scope link 
src 172.16.0.2
broadcast 127.0.0.0 dev lo  table local  proto kernel  scope link  src 
127.0.0.1
local 127.0.0.1 dev lo  table local  proto kernel  scope host  src 
127.0.0.1
local 127.0.0.0/8 dev lo  table local  proto kernel  scope host  src 
127.0.0.1


ip ru
-----
0:      from all lookup local
5:      from all lookup main
10:     from all iif eth0 lookup 10
11:     from 172.16.0.0/24 lookup 10
20:     from all iif eth1 lookup 20
21:     from 192.168.9.0/24 lookup 20
100:    from all lookup default


no netfilter rules of any sort (all policies set at ACCEPT)

_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

Re: [LARTC] Policy routing question

[Peter Rabbitson]

No takers on this question? I investigated further and it seems that 
this is a specific problem with iputils-ping. It seems that regardless 
of the supplied interface name, the source IP is chosen to be closest t 
the default gateway. OTOH my ability to follow C code is next to 
minimal, and I would really appreciate if someone can confirm this. Here 
is the part I believe is relevant:

if (source.sin_addr.s_addr = 0) {
	socklen_t alen;
	struct sockaddr_in dst = whereto;
	int probe_fd = socket(AF_INET, SOCK_DGRAM, 0);

	if (probe_fd < 0) {
		perror("socket");
		exit(2);
	}
	if (device) {
		struct ifreq ifr;
		memset(&ifr, 0, sizeof(ifr));
		strncpy(ifr.ifr_name, device, IFNAMSIZ-1);
		if (setsockopt(probe_fd, SOL_SOCKET, SO_BINDTODEVICE, device, 
strlen(device)+1) = -1) {
			if (IN_MULTICAST(ntohl(dst.sin_addr.s_addr))) {
				struct ip_mreqn imr;
				if (ioctl(probe_fd, SIOCGIFINDEX, &ifr) < 0) {
					fprintf(stderr, "ping: unknown iface %s\n", device);
					exit(2);
				}
				memset(&imr, 0, sizeof(imr));
				imr.imr_ifindex = ifr.ifr_ifindex;
				if (setsockopt(probe_fd, SOL_IP, IP_MULTICAST_IF, &imr, sizeof(imr)) 
= -1) {
					perror("ping: IP_MULTICAST_IF");
					exit(2);
				}
			}
		}
	}

	if (settos &&
		setsockopt(probe_fd, IPPROTO_IP, IP_TOS, (char *)&settos, sizeof(int)) 
< 0)
		perror("Warning: error setting QOS sockopts");

	dst.sin_port = htons(1025);
	if (nroute)
		dst.sin_addr.s_addr = route[0];
	if (connect(probe_fd, (struct sockaddr*)&dst, sizeof(dst)) = -1) {
		if (errno = EACCES) {
			if (broadcast_pings = 0) {
				fprintf(stderr, "Do you want to ping broadcast? Then -b\n");
				exit(2);
			}
			fprintf(stderr, "WARNING: pinging broadcast address\n");
			if (setsockopt(probe_fd, SOL_SOCKET, SO_BROADCAST,
					   &broadcast_pings, sizeof(broadcast_pings)) < 0) {
				perror ("can't set broadcasting");
				exit(2);
			}
			if (connect(probe_fd, (struct sockaddr*)&dst, sizeof(dst)) = -1) {
				perror("connect");
				exit(2);
			}
		} else {
			perror("connect");
			exit(2);
		}
	}
	alen = sizeof(source);
	if (getsockname(probe_fd, (struct sockaddr*)&source, &alen) = -1) {
		perror("getsockname");
		exit(2);
	}
	source.sin_port = 0;
	close(probe_fd);
} while (0);


Peter Rabbitson wrote:
> Hi,
> 
> I have a testing multihome setup, with the default gateway being one of 
> the links and using policy routing to honor requests for a specific 
> link. Everything works as expected when I request a specific IP to bind 
> to. But if I request a specific interface things fall apart in ways that 
> I can not explain:
> 
> default gw (WORKS)
> ----------
> rabbit@Thesaurus:~$ ping -c 1 yahoo.com
> PING yahoo.com (216.109.112.135) 56(84) bytes of data.
> 64 bytes from w2.rc.vip.dcn.yahoo.com (216.109.112.135): icmp_seq=1 
> ttlH time\x142 ms
> 
> 
> request IP on same link as default gw (WORKS)
> -------------------------------------
> rabbit@Thesaurus:~$ ping -I 192.168.9.102 -c 1 yahoo.com
> PING yahoo.com (66.94.234.13) from 192.168.9.102 : 56(84) bytes of data.
> 64 bytes from w2.rc.vip.scd.yahoo.com (66.94.234.13): icmp_seq=1 ttlG 
> time\x176 ms
> 
> request IP on secondary link (WORKS)
> ----------------------------
> rabbit@Thesaurus:~$ ping -I 172.16.0.2 -c 1 yahoo.com
> PING yahoo.com (216.109.112.135) from 172.16.0.2 : 56(84) bytes of data.
> 64 bytes from w2.rc.vip.dcn.yahoo.com (216.109.112.135): icmp_seq=1 
> ttlG time\x146 ms
> 
> request interface of default gw link (WORKS)
> ------------------------------------
> rabbit@Thesaurus:~$ ping -I eth1 -c 1 yahoo.com
> PING yahoo.com (66.94.234.13) from 192.168.9.102 eth1: 56(84) bytes of 
> data.
> 64 bytes from w2.rc.vip.scd.yahoo.com (66.94.234.13): icmp_seq=1 ttlG 
> time\x176 ms
> 
> request secondary interface (FAILS)
> ---------------------------
> rabbit@Thesaurus:~$ ping -I eth0 -c 1 yahoo.com
> PING yahoo.com (216.109.112.135) from 192.168.9.102 eth0: 56(84) bytes 
> of data.
>  From 172.16.0.2 icmp_seq=1 Destination Host Unreachable
> 
> 
> I went over the setup again and again, but I can't figure out why the 
> last ping attempt fails. Any pointers are welcome!
> 
> Thanks
> 
> Peter
> 
> 
> Here is the setup:
> 
> ip addr
> ----------
> 2: eth0: <BROADCAST,MULTICAST,UP,10000> mtu 1500 qdisc pfifo_fast qlen 1000
>     link/ether 00:11:09:8d:4f:c1 brd ff:ff:ff:ff:ff:ff
>     inet 172.16.0.2/24 brd 172.16.0.255 scope global eth0
> 3: eth1: <BROADCAST,MULTICAST,UP,10000> mtu 1500 qdisc pfifo_fast qlen 1000
>     link/ether 00:04:e2:80:b4:97 brd ff:ff:ff:ff:ff:ff
>     inet 192.168.9.102/24 brd 192.168.9.255 scope global eth1
> 
> 
> ip ro show table all
> -----------------------
> default via 172.16.0.1 dev eth0  table 10
> default via 192.168.9.1 dev eth1  table 20
> default via 192.168.9.1 dev eth1  table default
> 172.16.0.0/24 dev eth0  proto kernel  scope link  src 172.16.0.2
> 192.168.9.0/24 dev eth1  proto kernel  scope link  src 192.168.9.102
> broadcast 127.255.255.255 dev lo  table local  proto kernel  scope link 
>  src 127.0.0.1
> broadcast 192.168.9.0 dev eth1  table local  proto kernel  scope link 
> src 192.168.9.102
> broadcast 172.16.0.0 dev eth0  table local  proto kernel  scope link src 
> 172.16.0.2
> local 192.168.9.102 dev eth1  table local  proto kernel  scope host  src 
> 192.168.9.102
> local 172.16.0.2 dev eth0  table local  proto kernel  scope host  src 
> 172.16.0.2
> broadcast 192.168.9.255 dev eth1  table local  proto kernel  scope link 
>  src 192.168.9.102
> broadcast 172.16.0.255 dev eth0  table local  proto kernel  scope link 
> src 172.16.0.2
> broadcast 127.0.0.0 dev lo  table local  proto kernel  scope link  src 
> 127.0.0.1
> local 127.0.0.1 dev lo  table local  proto kernel  scope host  src 
> 127.0.0.1
> local 127.0.0.0/8 dev lo  table local  proto kernel  scope host  src 
> 127.0.0.1
> 
> 
> ip ru
> -----
> 0:      from all lookup local
> 5:      from all lookup main
> 10:     from all iif eth0 lookup 10
> 11:     from 172.16.0.0/24 lookup 10
> 20:     from all iif eth1 lookup 20
> 21:     from 192.168.9.0/24 lookup 20
> 100:    from all lookup default
> 
> 
> no netfilter rules of any sort (all policies set at ACCEPT)
> 
> _______________________________________________
> LARTC mailing list
> LARTC@mailman.ds9a.nl
> http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

RE: [LARTC] Policy routing question

[Salim S I]

Older versions of ping does not support interface with I option. It
won't give error, but it simply won't work. I had such an issue and was
solved with the latest ping tool.
Have you tried using tcpdump to capture packets from interfaces? 

-----Original Message-----
From: lartc-bounces@mailman.ds9a.nl
[mailto:lartc-bounces@mailman.ds9a.nl] On Behalf Of Peter Rabbitson
Sent: Wednesday, August 22, 2007 4:09 PM
To: lartc@mailman.ds9a.nl
Subject: Re: [LARTC] Policy routing question

No takers on this question? I investigated further and it seems that 
this is a specific problem with iputils-ping. It seems that regardless 
of the supplied interface name, the source IP is chosen to be closest t 
the default gateway. OTOH my ability to follow C code is next to 
minimal, and I would really appreciate if someone can confirm this. Here

is the part I believe is relevant:

if (source.sin_addr.s_addr = 0) {
	socklen_t alen;
	struct sockaddr_in dst = whereto;
	int probe_fd = socket(AF_INET, SOCK_DGRAM, 0);

	if (probe_fd < 0) {
		perror("socket");
		exit(2);
	}
	if (device) {
		struct ifreq ifr;
		memset(&ifr, 0, sizeof(ifr));
		strncpy(ifr.ifr_name, device, IFNAMSIZ-1);
		if (setsockopt(probe_fd, SOL_SOCKET, SO_BINDTODEVICE,
device, 
strlen(device)+1) = -1) {
			if (IN_MULTICAST(ntohl(dst.sin_addr.s_addr))) {
				struct ip_mreqn imr;
				if (ioctl(probe_fd, SIOCGIFINDEX, &ifr)
< 0) {
					fprintf(stderr, "ping: unknown
iface %s\n", device);
					exit(2);
				}
				memset(&imr, 0, sizeof(imr));
				imr.imr_ifindex = ifr.ifr_ifindex;
				if (setsockopt(probe_fd, SOL_IP,
IP_MULTICAST_IF, &imr, sizeof(imr)) 
= -1) {
					perror("ping: IP_MULTICAST_IF");
					exit(2);
				}
			}
		}
	}

	if (settos &&
		setsockopt(probe_fd, IPPROTO_IP, IP_TOS, (char
*)&settos, sizeof(int)) 
< 0)
		perror("Warning: error setting QOS sockopts");

	dst.sin_port = htons(1025);
	if (nroute)
		dst.sin_addr.s_addr = route[0];
	if (connect(probe_fd, (struct sockaddr*)&dst, sizeof(dst)) =
-1) {
		if (errno = EACCES) {
			if (broadcast_pings = 0) {
				fprintf(stderr, "Do you want to ping
broadcast? Then -b\n");
				exit(2);
			}
			fprintf(stderr, "WARNING: pinging broadcast
address\n");
			if (setsockopt(probe_fd, SOL_SOCKET,
SO_BROADCAST,
					   &broadcast_pings,
sizeof(broadcast_pings)) < 0) {
				perror ("can't set broadcasting");
				exit(2);
			}
			if (connect(probe_fd, (struct sockaddr*)&dst,
sizeof(dst)) = -1) {
				perror("connect");
				exit(2);
			}
		} else {
			perror("connect");
			exit(2);
		}
	}
	alen = sizeof(source);
	if (getsockname(probe_fd, (struct sockaddr*)&source, &alen) =
-1) {
		perror("getsockname");
		exit(2);
	}
	source.sin_port = 0;
	close(probe_fd);
} while (0);


Peter Rabbitson wrote:
> Hi,
> 
> I have a testing multihome setup, with the default gateway being one
of 
> the links and using policy routing to honor requests for a specific 
> link. Everything works as expected when I request a specific IP to
bind 
> to. But if I request a specific interface things fall apart in ways
that 
> I can not explain:
> 
> default gw (WORKS)
> ----------
> rabbit@Thesaurus:~$ ping -c 1 yahoo.com
> PING yahoo.com (216.109.112.135) 56(84) bytes of data.
> 64 bytes from w2.rc.vip.dcn.yahoo.com (216.109.112.135): icmp_seq=1 
> ttlH time\x142 ms
> 
> 
> request IP on same link as default gw (WORKS)
> -------------------------------------
> rabbit@Thesaurus:~$ ping -I 192.168.9.102 -c 1 yahoo.com
> PING yahoo.com (66.94.234.13) from 192.168.9.102 : 56(84) bytes of
data.
> 64 bytes from w2.rc.vip.scd.yahoo.com (66.94.234.13): icmp_seq=1
ttlG 
> time\x176 ms
> 
> request IP on secondary link (WORKS)
> ----------------------------
> rabbit@Thesaurus:~$ ping -I 172.16.0.2 -c 1 yahoo.com
> PING yahoo.com (216.109.112.135) from 172.16.0.2 : 56(84) bytes of
data.
> 64 bytes from w2.rc.vip.dcn.yahoo.com (216.109.112.135): icmp_seq=1 
> ttlG time\x146 ms
> 
> request interface of default gw link (WORKS)
> ------------------------------------
> rabbit@Thesaurus:~$ ping -I eth1 -c 1 yahoo.com
> PING yahoo.com (66.94.234.13) from 192.168.9.102 eth1: 56(84) bytes of

> data.
> 64 bytes from w2.rc.vip.scd.yahoo.com (66.94.234.13): icmp_seq=1
ttlG 
> time\x176 ms
> 
> request secondary interface (FAILS)
> ---------------------------
> rabbit@Thesaurus:~$ ping -I eth0 -c 1 yahoo.com
> PING yahoo.com (216.109.112.135) from 192.168.9.102 eth0: 56(84) bytes

> of data.
>  From 172.16.0.2 icmp_seq=1 Destination Host Unreachable
> 
> 
> I went over the setup again and again, but I can't figure out why the 
> last ping attempt fails. Any pointers are welcome!
> 
> Thanks
> 
> Peter
> 
> 
> Here is the setup:
> 
> ip addr
> ----------
> 2: eth0: <BROADCAST,MULTICAST,UP,10000> mtu 1500 qdisc pfifo_fast qlen
1000
>     link/ether 00:11:09:8d:4f:c1 brd ff:ff:ff:ff:ff:ff
>     inet 172.16.0.2/24 brd 172.16.0.255 scope global eth0
> 3: eth1: <BROADCAST,MULTICAST,UP,10000> mtu 1500 qdisc pfifo_fast qlen
1000
>     link/ether 00:04:e2:80:b4:97 brd ff:ff:ff:ff:ff:ff
>     inet 192.168.9.102/24 brd 192.168.9.255 scope global eth1
> 
> 
> ip ro show table all
> -----------------------
> default via 172.16.0.1 dev eth0  table 10
> default via 192.168.9.1 dev eth1  table 20
> default via 192.168.9.1 dev eth1  table default
> 172.16.0.0/24 dev eth0  proto kernel  scope link  src 172.16.0.2
> 192.168.9.0/24 dev eth1  proto kernel  scope link  src 192.168.9.102
> broadcast 127.255.255.255 dev lo  table local  proto kernel  scope
link 
>  src 127.0.0.1
> broadcast 192.168.9.0 dev eth1  table local  proto kernel  scope link 
> src 192.168.9.102
> broadcast 172.16.0.0 dev eth0  table local  proto kernel  scope link
src 
> 172.16.0.2
> local 192.168.9.102 dev eth1  table local  proto kernel  scope host
src 
> 192.168.9.102
> local 172.16.0.2 dev eth0  table local  proto kernel  scope host  src 
> 172.16.0.2
> broadcast 192.168.9.255 dev eth1  table local  proto kernel  scope
link 
>  src 192.168.9.102
> broadcast 172.16.0.255 dev eth0  table local  proto kernel  scope link

> src 172.16.0.2
> broadcast 127.0.0.0 dev lo  table local  proto kernel  scope link  src

> 127.0.0.1
> local 127.0.0.1 dev lo  table local  proto kernel  scope host  src 
> 127.0.0.1
> local 127.0.0.0/8 dev lo  table local  proto kernel  scope host  src 
> 127.0.0.1
> 
> 
> ip ru
> -----
> 0:      from all lookup local
> 5:      from all lookup main
> 10:     from all iif eth0 lookup 10
> 11:     from 172.16.0.0/24 lookup 10
> 20:     from all iif eth1 lookup 20
> 21:     from 192.168.9.0/24 lookup 20
> 100:    from all lookup default
> 
> 
> no netfilter rules of any sort (all policies set at ACCEPT)
> 
> _______________________________________________
> LARTC mailing list
> LARTC@mailman.ds9a.nl
> http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc


_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc