[LARTC] Routing that doesn't route

[Greg Scott <GregScott@InfraSupportEtc.com>]

>	But you have to explain your problem with more details or
> may be to show me some URLs if it is explained somewhere.

Thanks Julian - 

Here is the issue I am fighting, in detail:

I have a VPN situation with two offices, one in St. Peter,
Minnesota, the other in Minneapolis.  The VPN servers in this
case are Windows 2000 servers.  They are doing Microsoft PPTP
tunneling, which uses the GRE protocol.  Each branch has a
Linux based firewall system.  At each office, both the firewall
and Win2000 VPN servers are connected to both the inside and 
outside LANs.

The St. Peter firewall system is based on Red Hat Linux 7.0 and
the 2.2 kernel.  It is actually 3 computers - an inside firewall,
an outside firewall, and a DMZ in the middle with an ftp server.  
The system in Minneapolis is based on Red Hat Linux 7.1 and the 
2.4 kernel.

The VPN IP address range in St. Peter is 172.16.0.0/20.
The VPN IP address range in Minneapolis is 172.16.16.0/20.

From either office, traffic bound for the other office should go 
through the Win2000 VPN server.  Any other traffic should go through 
that office's firewall system.  The default gateways for all systems 
on each side are their respective firewall systems.  The default 
gateway for systems in St. Peter is 172.16.0.254.  The default 
gateway for systems in Minneapolis is 172.16.16.1.

Each Linux firewall/router has a default route to the Internet.
Each also has a static route back to the local Windows 2000 VPN
server.  Here is the route table from the Minneapolis firewall:
eth0 is the interface connected to the Internet, eth1 is connected
to the internal LAN.  172.16.16.3 is the IP address of the Win 2000
VPN server.  

[root@csfampls-fw /etc]# /sbin/route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use
Iface
mmm.mmm.228.32  0.0.0.0         255.255.255.224 U     0      0        0 eth0
sss.sss.200.64  mmm.mmm.228.34  255.255.255.224 UG    0      0        0 eth0
172.16.16.0     0.0.0.0         255.255.240.0   U     0      0        0 eth1
172.16.0.0      172.16.16.3     255.255.240.0   UG    0      0        0 eth1
127.0.0.0       0.0.0.0         255.0.0.0       U     0      0        0 lo
0.0.0.0         206.144.228.62  0.0.0.0         UG    0      0        0 eth0
[root@csfampls-fw /etc]# 

As you can see, the Minneapolis firewall should route packets bound
for the other end of the VPN tunnel back out eth1 and over to the
Win 2000 VPN server.  It should forward anything else to the Internet.

The St. Peter firewall system has similar routes setup.

Here is the problem:  When a host inside St. Peter tries to ping
a host inside Minneapolis, the St. Peter host never sees the
reply.  If I define a static route directly on that host in 
Minneapolis via the Win 2000 VPN server, then the replies make it all 
the way back to the original sender in St. Peter.  

Other protocols work fine.  I tried some tests with DNS and ftp from
St. Peter to Minneapolis and they make the round trip just fine.  Only 
ICMP ECHO REPLY packets fail the round trip.

Thinking that maybe my firewall rules in Minneapolis were filtering 
packets that should not have been filtered, I removed all firewall 
rules and set the default iptables policy to ACCEPT.  No change in
symptoms.  

I also tried various packet traces using tcpdump on Linux and Microsoft
NETMON on the Win 2000 server.  I found that the target system in
Minneapolis received the ICMP PINGs and sent back ECHO REPLY messages.
The messages made it to the Minneapolis firewall/router, but never
routed to the Win 2000 VPN server.

How is this possible since I have a static route on the Minneapolis
firewall over to the VPN server?

Here is a tcpdump trace I took from the Minneapolis firewall
(forgive the text wrapping):


16:25:15.459714 < 172.16.16.2 > 172.16.0.252: icmp: echo reply
16:25:16.459714 < 172.16.16.2 > 172.16.0.252: icmp: echo reply
16:25:17.459714 < 172.16.16.2 > 172.16.0.252: icmp: echo reply
16:25:18.459714 < 172.16.16.2 > 172.16.0.252: icmp: echo reply
16:25:18.879714 > arp who-has 172.16.16.3 tell 172.16.16.1 (0:4:76:1a:f8:1f)
16:25:18.879714 < arp reply 172.16.16.3 is-at 0:50:8b:e7:ed:ff
(0:4:76:1a:f8:1f)
16:25:18.879714 > 172.16.16.1.32770 > 172.16.16.2.domain: 45803+ PTR?
3.16.16.172.in-addr.arpa. (42) (DF)
16:25:18.879714 < 172.16.16.2.domain > ns1.internet-connections.net.domain:
3072+ PTR? 3.16.16.172.in-addr.arpa. (42)
16:25:19.049714 > ns1.internet-connections.net.domain > 172.16.16.2.domain:
3072 NXDomain* 0/1/0 (108) (DF)
16:25:19.049714 < 172.16.16.2.domain > 172.16.16.1.32770: 45803 NXDomain*
0/1/0 (108)
16:25:19.459714 < 172.16.16.2 > 172.16.0.252: icmp: echo reply

40 packets received by filter


As you can see, this system gets a bunch of icmp echo reply packets
from an internal host.  He does an ARP to find the Win 2000 VPN server,
so far so good.  Then he does a reverse DNS lookup and routes that
stuff out to the Internet.  But - here's the key - he NEVER FORWARDS
THE ECHO REPLY PACKET OVER TO THE VPN SERVER!!!!

This problem is keeping my awake at night and really messing with
my mind.  Does anyone have any ideas???

thanks

- Greg Scott

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/2.4Routing/