[LARTC] Shortcut routes

["Dean Gibson (Network Administrator)" <noc@ultimeth.net>]

[-- Attachment #1: Type: text/plain, Size: 3745 bytes --]

I have two Linux (RH v9) routers connected to the Internet (separate DSL connections), each with two EtherNet cards.

Router #1 has static IP address "a.a.a.1" for the internal LAN, and static IP address "x.x.x.x" for the Internet connection;  here's what the "route command shows:

Kernel IP routing table
Destination  Gateway  Genmask         Flags Metric Ref    Use Iface
x.x.x.0      0.0.0.0  255.255.255.0   U     0      0        0 eth1
a.a.a.0      0.0.0.0  255.255.255.0   U     0      0        0 eth0
169.254.0.0  0.0.0.0  255.255.0.0     U     0      0        0 eth1
127.0.0.0    0.0.0.0  255.0.0.0       U     0      0        0 lo
0.0.0.0      x.x.x.1  0.0.0.0         UG    0      0        0 eth1

Router #2 has static IP address "a.a.a.2" for the internal LAN, and DHCP IP address "y.y.y.y" for the Internet connection;  here's what the "route command shows:

Kernel IP routing table
Destination  Gateway  Genmask         Flags Metric Ref    Use Iface
y.y.y.0      0.0.0.0  255.255.255.0   U     0      0        0 eth1
a.a.a.0      0.0.0.0  255.255.255.0   U     0      0        0 eth0
169.254.0.0  0.0.0.0  255.255.0.0     U     0      0        0 eth1
127.0.0.0    0.0.0.0  255.0.0.0       U     0      0        0 lo
0.0.0.0      y.y.y.1  0.0.0.0         UG    0      0        0 eth1

This works, but since router #1 has several server daemons running (HTTP, DNS, etc), and since router #2 is the default gateway for internal hosts on the a.a.a.0/24 network, any access to servers on router #1 goes out through router #2 and the Internet in order to get to router #1 (and similarly to get back);  this is a performance hit due to the (relatively) slow outbound DSL speeds (128Kbit/s) involved.

So, I decided to add a "shortcut" route on router #2:  "route add x.x.x.x eth0";  here's what the "route command now shows:

Kernel IP routing table
Destination  Gateway  Genmask         Flags Metric Ref    Use Iface
x.x.x.x      0.0.0.0  255.255.255.255 UH    0      0        0 eth0
y.y.y.0      0.0.0.0  255.255.255.0   U     0      0        0 eth1
a.a.a.0      0.0.0.0  255.255.255.0   U     0      0        0 eth0
169.254.0.0  0.0.0.0  255.255.0.0     U     0      0        0 eth1
127.0.0.0    0.0.0.0  255.0.0.0       U     0      0        0 lo
0.0.0.0      y.y.y.1  0.0.0.0         UG    0      0        0 eth1

This solves the performance problem for accessing servers on router #1, BUT now any access initiated from router #1 to router #2 fails.  I added logging entries in the "mangle" table for "iptables", and it shows the packets from router #2 to router #1 getting through the "PREROUTING" stage, but no further.  If I remove the added route, access from #2 to #1 works AND I see the packets getting beyond the "PREROUTING" stage to either the "INPUT" or "FORWARD" stages.  Note that when testing this, there is nothing in the "filter" or "nat" tables.

Now, I can solve this by a reciprocal "route add y.y.y.y eth0" on router #1 (which works).  However, y.y.y.y is a DHCP address from my ISP, so that's only a temporary fix until the IP address changes.

My big question is to really understand what is going on.  Here is the iptables/routing diagram I got from Rusty's documentation:

 --->PRE--->[ROUTE]-->FWD-------->POST---->
  Conntrack    |     Mangle  ^   Mangle
  Mangle       |     Filter  |   NAT (Src)
  NAT (Dst)    |             |   Conntrack
  (QDisc)      |          [ROUTE]
               v             |
               IN Filter    OUT Conntrack
               |  Conntrack  ^  Mangle
               |  Mangle     |  NAT (Dst)
               v             |  Filter

Why is the routing code apparently dropping the packets from router #1 to router #2 (but only for connections initiated from #1)?

-- Dean

[-- Attachment #2: Type: text/html, Size: 7256 bytes --]