Linux Advanced Routing and Traffic Control list
 help / color / mirror / Atom feed
* [LARTC] Re: need help with policy routing.
@ 2001-02-12 22:46 tc
  0 siblings, 0 replies; only message in thread
From: tc @ 2001-02-12 22:46 UTC (permalink / raw)
  To: lartc

<PRE>aha.  i figured it out:

&gt;<i> /sbin/ifconfig lo:1 64.211.224.163 netmask 255.255.255.240 broadcast
</I>&gt;<i> 64.211.224.175 up
</I>&gt;<i> /sbin/route add -host 64.211.224.163 dev lo:1
</I>&gt;<i> /sbin/ifconfig lo:2 64.211.224.166 netmask 255.255.255.240 broadcast
</I>&gt;<i> 64.211.224.175 up
</I>&gt;<i> /sbin/route add -host 64.211.224.166 dev lo:2
</I>&gt;<i> /sbin/ifconfig lo:3 64.211.224.168 netmask 255.255.255.240 broadcast
</I>&gt;<i> 64.211.224.175 up
</I>&gt;<i> /sbin/route add -host 64.211.224.168 dev lo:3
</I>
those netmasks should be 255.255.255.255.  i _think_ this has fixed all my
problems.  initial tests look much better.  i have more to investigate
yet.

-tcl.


On Mon, 12 Feb 2001, tc lewis wrote:

&gt;<i> 
</I>&gt;<i> so, i have a pretty complex (for me, that is) setup on this one machine
</I>&gt;<i> that acts as a nameserver and mail server and some other stuff and answers
</I>&gt;<i> to a handful of ips.  it's also a &quot;real server&quot; behind an lvs director.
</I>&gt;<i> the machine in question is running a modified redhat 6.2 with a 2.2.17ext3
</I>&gt;<i> kernel (stock 2.2.17 + ext3 patches + nfs patches).
</I>&gt;<i> 
</I>&gt;<i> let me try to describe this as best i can.
</I>&gt;<i> 
</I>&gt;<i> our external network is 64.211.224.160/28.  161 is the router/gateway to
</I>&gt;<i> the rest of the world.  162 is an auth nameserver.  163 is an auth
</I>&gt;<i> nameserver.  164 is the ip used for outgoing connections from behind
</I>&gt;<i> masquerading.  165 is for web traffic.  166 is for incoming mail.  and i
</I>&gt;<i> just put 169 in as a standalone machine.
</I>&gt;<i> 
</I>&gt;<i> the 164 masquerading server allows the nameserver/mailserver to send
</I>&gt;<i> requests to the outside world:
</I>&gt;<i> MASQ       all  ------  192.168.1.21         0.0.0.0/0             n/a
</I>&gt;<i> 
</I>&gt;<i> the lvs director basically handles all incoming traffic and forwards it to
</I>&gt;<i> the right place:
</I>&gt;<i> IP Virtual Server version 1.0.0-beta1 (size@96)
</I>&gt;<i> Prot LocalAddress:Port Scheduler Flags
</I>&gt;<i>   -&gt; RemoteAddress:Port          Forward Weight ActiveConn InActConn
</I>&gt;<i> TCP  64.211.224.165:443 lc persistent 360
</I>&gt;<i>   -&gt; 192.168.1.101:443           Route   1      0          0         
</I>&gt;<i>   -&gt; 192.168.1.102:443           Route   1      0          0         
</I>&gt;<i> UDP  64.211.224.162:53 lc
</I>&gt;<i>   -&gt; 192.168.1.11:53             Route   1      0          349       
</I>&gt;<i> UDP  64.211.224.163:53 lc
</I>&gt;<i>   -&gt; 192.168.1.12:53             Route   1      0          183       
</I>&gt;<i> TCP  64.211.224.163:53 lc
</I>&gt;<i>   -&gt; 192.168.1.12:53             Route   1      0          0         
</I>&gt;<i> TCP  64.211.224.162:53 lc
</I>&gt;<i>   -&gt; 192.168.1.11:53             Route   1      0          0         
</I>&gt;<i> TCP  64.211.224.166:22 lc
</I>&gt;<i>   -&gt; 192.168.1.21:22             Route   1      0          0         
</I>&gt;<i> TCP  64.211.224.168:22 lc
</I>&gt;<i>   -&gt; 192.168.1.21:22             Route   1      16         0         
</I>&gt;<i> TCP  64.211.224.166:25 lc
</I>&gt;<i>   -&gt; 192.168.1.21:25             Route   1      0          0         
</I>&gt;<i> TCP  64.211.224.165:80 lc
</I>&gt;<i>   -&gt; 192.168.1.101:80            Route   1      0          3         
</I>&gt;<i>   -&gt; 192.168.1.102:80            Route   1      0          1         
</I>&gt;<i> 
</I>&gt;<i> then there's the &quot;phl&quot; machine which handles dns and mail:
</I>&gt;<i> [<A HREF="mailto:root@phl">root@phl</A> /root]# /sbin/ifconfig 
</I>&gt;<i> eth0      Link encap:Ethernet  HWaddr 00:D0:B7:65:EC:48  
</I>&gt;<i>           inet addr:192.168.1.21  Bcast:192.168.1.255  Mask:255.255.255.0
</I>&gt;<i>           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
</I>&gt;<i>           RX packets:24535885 errors:0 dropped:0 overruns:0 frame:0
</I>&gt;<i>           TX packets:24655159 errors:0 dropped:0 overruns:0 carrier:0
</I>&gt;<i>           collisions:0 txqueuelen:100 
</I>&gt;<i>           Interrupt:11 Base address:0x2800 
</I>&gt;<i> 
</I>&gt;<i> eth0:0    Link encap:Ethernet  HWaddr 00:D0:B7:65:EC:48  
</I>&gt;<i>           inet addr:192.168.1.11  Bcast:192.168.1.255  Mask:255.255.255.0
</I>&gt;<i>           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
</I>&gt;<i>           Interrupt:11 Base address:0x2800 
</I>&gt;<i> 
</I>&gt;<i> eth0:1    Link encap:Ethernet  HWaddr 00:D0:B7:65:EC:48  
</I>&gt;<i>           inet addr:192.168.1.12  Bcast:192.168.1.255  Mask:255.255.255.0
</I>&gt;<i>           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
</I>&gt;<i>           Interrupt:11 Base address:0x2800 
</I>&gt;<i> 
</I>&gt;<i> eth0:2    Link encap:Ethernet  HWaddr 00:D0:B7:65:EC:48  
</I>&gt;<i>           inet addr:192.168.1.13  Bcast:192.168.1.255  Mask:255.255.255.0
</I>&gt;<i>           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
</I>&gt;<i>           Interrupt:11 Base address:0x2800 
</I>&gt;<i> 
</I>&gt;<i> eth0:3    Link encap:Ethernet  HWaddr 00:D0:B7:65:EC:48  
</I>&gt;<i>           inet addr:192.168.1.14  Bcast:192.168.1.255  Mask:255.255.255.0
</I>&gt;<i>           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
</I>&gt;<i>           Interrupt:11 Base address:0x2800 
</I>&gt;<i> 
</I>&gt;<i> eth0:4    Link encap:Ethernet  HWaddr 00:D0:B7:65:EC:48  
</I>&gt;<i>           inet addr:192.168.1.10  Bcast:192.168.1.255  Mask:255.255.255.0
</I>&gt;<i>           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
</I>&gt;<i>           Interrupt:11 Base address:0x2800 
</I>&gt;<i> 
</I>&gt;<i> eth1      Link encap:Ethernet  HWaddr 00:C0:95:E2:85:40  
</I>&gt;<i>           inet addr:192.168.2.21  Bcast:192.168.2.255  Mask:255.255.255.0
</I>&gt;<i>           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
</I>&gt;<i>           RX packets:20102464 errors:0 dropped:0 overruns:0 frame:0
</I>&gt;<i>           TX packets:19892838 errors:6 dropped:0 overruns:3 carrier:6
</I>&gt;<i>           collisions:0 txqueuelen:100 
</I>&gt;<i>           Interrupt:11 Base address:0x3000 
</I>&gt;<i> 
</I>&gt;<i> eth1:0    Link encap:Ethernet  HWaddr 00:C0:95:E2:85:40  
</I>&gt;<i>           inet addr:192.168.2.13  Bcast:192.168.2.255  Mask:255.255.255.0
</I>&gt;<i>           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
</I>&gt;<i>           Interrupt:11 Base address:0x3000 
</I>&gt;<i> 
</I>&gt;<i> eth1:1    Link encap:Ethernet  HWaddr 00:C0:95:E2:85:40  
</I>&gt;<i>           inet addr:192.168.2.14  Bcast:192.168.2.255  Mask:255.255.255.0
</I>&gt;<i>           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
</I>&gt;<i>           Interrupt:11 Base address:0x3000 
</I>&gt;<i> 
</I>&gt;<i> eth1:2    Link encap:Ethernet  HWaddr 00:C0:95:E2:85:40  
</I>&gt;<i>           inet addr:192.168.2.10  Bcast:192.168.2.255  Mask:255.255.255.0
</I>&gt;<i>           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
</I>&gt;<i>           Interrupt:11 Base address:0x3000 
</I>&gt;<i> 
</I>&gt;<i> eth2      Link encap:Ethernet  HWaddr 00:C0:95:E2:85:41  
</I>&gt;<i>           inet addr:192.168.3.21  Bcast:192.168.3.255  Mask:255.255.255.0
</I>&gt;<i>           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
</I>&gt;<i>           RX packets:74336 errors:0 dropped:0 overruns:0 frame:0
</I>&gt;<i>           TX packets:111705 errors:16 dropped:0 overruns:2 carrier:28
</I>&gt;<i>           collisions:0 txqueuelen:100 
</I>&gt;<i>           Interrupt:10 Base address:0x3080 
</I>&gt;<i> 
</I>&gt;<i> eth2:0    Link encap:Ethernet  HWaddr 00:C0:95:E2:85:41  
</I>&gt;<i>           inet addr:192.168.3.13  Bcast:192.168.3.255  Mask:255.255.255.0
</I>&gt;<i>           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
</I>&gt;<i>           Interrupt:10 Base address:0x3080 
</I>&gt;<i> 
</I>&gt;<i> eth2:1    Link encap:Ethernet  HWaddr 00:C0:95:E2:85:41  
</I>&gt;<i>           inet addr:192.168.3.14  Bcast:192.168.3.255  Mask:255.255.255.0
</I>&gt;<i>           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
</I>&gt;<i>           Interrupt:10 Base address:0x3080 
</I>&gt;<i> 
</I>&gt;<i> eth2:2    Link encap:Ethernet  HWaddr 00:C0:95:E2:85:41  
</I>&gt;<i>           inet addr:192.168.3.10  Bcast:192.168.3.255  Mask:255.255.255.0
</I>&gt;<i>           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
</I>&gt;<i>           Interrupt:10 Base address:0x3080 
</I>&gt;<i> 
</I>&gt;<i> lo        Link encap:Local Loopback  
</I>&gt;<i>           inet addr:127.0.0.1  Mask:255.0.0.0
</I>&gt;<i>           UP LOOPBACK RUNNING  MTU:3924  Metric:1
</I>&gt;<i>           RX packets:191349 errors:0 dropped:0 overruns:0 frame:0
</I>&gt;<i>           TX packets:191349 errors:0 dropped:0 overruns:0 carrier:0
</I>&gt;<i>           collisions:0 txqueuelen:0 
</I>&gt;<i> 
</I>&gt;<i> lo:0      Link encap:Local Loopback  
</I>&gt;<i>           inet addr:64.211.224.162  Mask:255.255.255.240
</I>&gt;<i>           UP LOOPBACK RUNNING  MTU:3924  Metric:1
</I>&gt;<i> 
</I>&gt;<i> lo:1      Link encap:Local Loopback  
</I>&gt;<i>           inet addr:64.211.224.163  Mask:255.255.255.240
</I>&gt;<i>           UP LOOPBACK RUNNING  MTU:3924  Metric:1
</I>&gt;<i> 
</I>&gt;<i> lo:2      Link encap:Local Loopback  
</I>&gt;<i>           inet addr:64.211.224.166  Mask:255.255.255.240
</I>&gt;<i>           UP LOOPBACK RUNNING  MTU:3924  Metric:1
</I>&gt;<i> 
</I>&gt;<i> lo:3      Link encap:Local Loopback  
</I>&gt;<i>           inet addr:64.211.224.168  Mask:255.255.255.240
</I>&gt;<i>           UP LOOPBACK RUNNING  MTU:3924  Metric:1
</I>&gt;<i> 
</I>&gt;<i> [<A HREF="mailto:root@phl">root@phl</A> /root]# /sbin/route -n
</I>&gt;<i> Kernel IP routing table
</I>&gt;<i> Destination     Gateway         Genmask         Flags Metric Ref    Use
</I>&gt;<i> Iface
</I>&gt;<i> 64.211.224.166  0.0.0.0         255.255.255.255 UH    0      0        0 lo
</I>&gt;<i> 192.168.2.10    0.0.0.0         255.255.255.255 UH    0      0        0
</I>&gt;<i> eth1
</I>&gt;<i> 192.168.2.13    0.0.0.0         255.255.255.255 UH    0      0        0
</I>&gt;<i> eth1
</I>&gt;<i> 192.168.1.21    0.0.0.0         255.255.255.255 UH    0      0        0
</I>&gt;<i> eth0
</I>&gt;<i> 192.168.3.21    0.0.0.0         255.255.255.255 UH    0      0        0
</I>&gt;<i> eth2
</I>&gt;<i> 64.211.224.162  0.0.0.0         255.255.255.255 UH    0      0        0 lo
</I>&gt;<i> 64.211.224.163  0.0.0.0         255.255.255.255 UH    0      0        0 lo
</I>&gt;<i> 192.168.2.14    0.0.0.0         255.255.255.255 UH    0      0        0
</I>&gt;<i> eth1
</I>&gt;<i> 192.168.1.11    0.0.0.0         255.255.255.255 UH    0      0        0
</I>&gt;<i> eth0
</I>&gt;<i> 192.168.1.10    0.0.0.0         255.255.255.255 UH    0      0        0
</I>&gt;<i> eth0
</I>&gt;<i> 192.168.3.10    0.0.0.0         255.255.255.255 UH    0      0        0
</I>&gt;<i> eth2
</I>&gt;<i> 192.168.1.13    0.0.0.0         255.255.255.255 UH    0      0        0
</I>&gt;<i> eth0
</I>&gt;<i> 192.168.3.13    0.0.0.0         255.255.255.255 UH    0      0        0
</I>&gt;<i> eth2
</I>&gt;<i> 192.168.2.21    0.0.0.0         255.255.255.255 UH    0      0        0
</I>&gt;<i> eth1
</I>&gt;<i> 192.168.1.12    0.0.0.0         255.255.255.255 UH    0      0        0
</I>&gt;<i> eth0
</I>&gt;<i> 64.211.224.168  0.0.0.0         255.255.255.255 UH    0      0        0 lo
</I>&gt;<i> 192.168.1.14    0.0.0.0         255.255.255.255 UH    0      0        0
</I>&gt;<i> eth0
</I>&gt;<i> 192.168.3.14    0.0.0.0         255.255.255.255 UH    0      0        0
</I>&gt;<i> eth2
</I>&gt;<i> 64.211.224.160  0.0.0.0         255.255.255.240 U     0      0        0
</I>&gt;<i> eth0
</I>&gt;<i> 192.168.3.0     0.0.0.0         255.255.255.0   U     0      0        0
</I>&gt;<i> eth2
</I>&gt;<i> 192.168.2.0     0.0.0.0         255.255.255.0   U     0      0        0
</I>&gt;<i> eth1
</I>&gt;<i> 192.168.1.0     0.0.0.0         255.255.255.0   U     0      0        0
</I>&gt;<i> eth0
</I>&gt;<i> 127.0.0.0       0.0.0.0         255.0.0.0       U     0      0        0 lo
</I>&gt;<i> 
</I>&gt;<i> [<A HREF="mailto:root@phl">root@phl</A> /root]# cat /etc/sysctl.conf 
</I>&gt;<i> # Disables packet forwarding
</I>&gt;<i> net.ipv4.ip_forward = 1
</I>&gt;<i> # Enables source route verification
</I>&gt;<i> net.ipv4.conf.all.rp_filter = 1
</I>&gt;<i> # Disables automatic defragmentation (needed for masquerading, LVS)
</I>&gt;<i> net.ipv4.ip_always_defrag = 0
</I>&gt;<i> # Disables the magic-sysrq key
</I>&gt;<i> kernel.sysrq = 1
</I>&gt;<i> 
</I>&gt;<i> # -tcl.
</I>&gt;<i> net.ipv4.conf.all.send_redirects = 0
</I>&gt;<i> net.ipv4.conf.eth0.send_redirects = 0
</I>&gt;<i> net.ipv4.conf.all.hidden = 1
</I>&gt;<i> net.ipv4.conf.lo.hidden = 1
</I>&gt;<i> #
</I>&gt;<i> [<A HREF="mailto:root@phl">root@phl</A> /root]# tail --lines 30 /etc/rc.d/rc.local 
</I>&gt;<i> 
</I>&gt;<i> #
</I>&gt;<i> # -tcl.
</I>&gt;<i> #
</I>&gt;<i> # the whole static-routes / network scripts / lo:# / gateway being on a
</I>&gt;<i> # different device than ips on the same network / bl ah blah lah sajdhsd.
</I>&gt;<i> # totally flaky.  let's just do it all here.
</I>&gt;<i> #
</I>&gt;<i> /sbin/sysctl -p
</I>&gt;<i> /sbin/route add -net 64.211.224.160 netmask 255.255.255.240 dev eth0
</I>&gt;<i> #/sbin/route add default gw 64.211.224.161 dev eth0
</I>&gt;<i> ##/sbin/arp -s 64.211.224.161 00:30:B6:67:00:40
</I>&gt;<i> /sbin/arp -s 64.211.224.161 00:30:B6:67:00:AA
</I>&gt;<i> #/sbin/ip rule add prio 100 from 192.168.1.0/24 table 100
</I>&gt;<i> #/sbin/ip route add table 100 0/0 via 192.168.1.1 dev eth0
</I>&gt;<i> /sbin/ifconfig lo:0 64.211.224.162 netmask 255.255.255.240 broadcast
</I>&gt;<i> 64.211.224.175 up
</I>&gt;<i> /sbin/route add -host 64.211.224.162 dev lo:0
</I>&gt;<i> /sbin/ifconfig lo:1 64.211.224.163 netmask 255.255.255.240 broadcast
</I>&gt;<i> 64.211.224.175 up
</I>&gt;<i> /sbin/route add -host 64.211.224.163 dev lo:1
</I>&gt;<i> /sbin/ifconfig lo:2 64.211.224.166 netmask 255.255.255.240 broadcast
</I>&gt;<i> 64.211.224.175 up
</I>&gt;<i> /sbin/route add -host 64.211.224.166 dev lo:2
</I>&gt;<i> /sbin/ifconfig lo:3 64.211.224.168 netmask 255.255.255.240 broadcast
</I>&gt;<i> 64.211.224.175 up
</I>&gt;<i> /sbin/route add -host 64.211.224.168 dev lo:3
</I>&gt;<i> #/sbin/ip rule add prio 33000 from 192.168.1.0/24 table 100
</I>&gt;<i> /sbin/ip route add table 100 0/0 via 192.168.1.1 dev eth0
</I>&gt;<i> #/sbin/ip rule add prio 34000 from 0/0 table 200
</I>&gt;<i> /sbin/ip route add table 200 0/0 via 64.211.224.161 dev eth0
</I>&gt;<i> /sbin/ip rule add prio 33000 from 64.211.224.160/28 table 200
</I>&gt;<i> /sbin/ip rule add prio 34000 from 0/0 table 100
</I>&gt;<i> #
</I>&gt;<i> 
</I>&gt;<i> [<A HREF="mailto:root@phl">root@phl</A> /root]# ip rule
</I>&gt;<i> 0:      from all lookup local 
</I>&gt;<i> 32766:  from all lookup main 
</I>&gt;<i> 32767:  from all lookup 253 
</I>&gt;<i> 33000:  from 64.211.224.160/28 lookup 200 
</I>&gt;<i> 34000:  from all lookup 100 
</I>&gt;<i> [<A HREF="mailto:root@phl">root@phl</A> /root]# ip route
</I>&gt;<i> 64.211.224.166 dev lo  scope link  src 64.211.224.166 
</I>&gt;<i> 192.168.2.10 dev eth1  scope link  src 192.168.2.10 
</I>&gt;<i> 192.168.2.13 dev eth1  scope link  src 192.168.2.13 
</I>&gt;<i> 192.168.1.21 dev eth0  scope link 
</I>&gt;<i> 192.168.3.21 dev eth2  scope link 
</I>&gt;<i> 64.211.224.162 dev lo  scope link  src 64.211.224.162 
</I>&gt;<i> 64.211.224.163 dev lo  scope link  src 64.211.224.163 
</I>&gt;<i> 192.168.2.14 dev eth1  scope link  src 192.168.2.14 
</I>&gt;<i> 192.168.1.11 dev eth0  scope link  src 192.168.1.11 
</I>&gt;<i> 192.168.1.10 dev eth0  scope link  src 192.168.1.10 
</I>&gt;<i> 192.168.3.10 dev eth2  scope link  src 192.168.3.10 
</I>&gt;<i> 192.168.1.13 dev eth0  scope link  src 192.168.1.13 
</I>&gt;<i> 192.168.3.13 dev eth2  scope link  src 192.168.3.13 
</I>&gt;<i> 192.168.2.21 dev eth1  scope link 
</I>&gt;<i> 192.168.1.12 dev eth0  scope link  src 192.168.1.12 
</I>&gt;<i> 64.211.224.168 dev lo  scope link  src 64.211.224.168 
</I>&gt;<i> 192.168.1.14 dev eth0  scope link  src 192.168.1.14 
</I>&gt;<i> 192.168.3.14 dev eth2  scope link  src 192.168.3.14 
</I>&gt;<i> 64.211.224.160/28 dev eth0  scope link 
</I>&gt;<i> 192.168.3.0/24 dev eth2  proto kernel  scope link  src 192.168.3.21 
</I>&gt;<i> 192.168.2.0/24 dev eth1  proto kernel  scope link  src 192.168.2.21 
</I>&gt;<i> 192.168.1.0/24 dev eth0  proto kernel  scope link  src 192.168.1.21 
</I>&gt;<i> 127.0.0.0/8 dev lo  scope link 
</I>&gt;<i> [<A HREF="mailto:root@phl">root@phl</A> /root]# ip route list table 100
</I>&gt;<i> default via 192.168.1.1 dev eth0 
</I>&gt;<i> [<A HREF="mailto:root@phl">root@phl</A> /root]# ip route list table 200
</I>&gt;<i> default via 64.211.224.161 dev eth0 
</I>&gt;<i> [<A HREF="mailto:root@phl">root@phl</A> /root]# 
</I>&gt;<i> 
</I>&gt;<i> 
</I>&gt;<i> the end result of this is that, well, for example, a nameservice query get
</I>&gt;<i> directed through the lvs director to the phl real server, which answers it
</I>&gt;<i> via direct routing.  phl can also get to the outside world to deliver mail
</I>&gt;<i> / make dns queries of its own via the masquerading.  the policy routing
</I>&gt;<i> says that traffic with a source ip of 64.211.224.160/28 gets sent via
</I>&gt;<i> 64.211.224.161 (direct routing instead of nat/masq), whereas traffic with
</I>&gt;<i> a source ip of anything else should go through 192.168.1.1 and be
</I>&gt;<i> masqueraded.  those 192.168.2 and .3 and whatever other networks on there
</I>&gt;<i> can be ignored.
</I>&gt;<i> 
</I>&gt;<i> /me breathes.
</I>&gt;<i> 
</I>&gt;<i> ok.  so all that has been working perfectly for months.  the problem is
</I>&gt;<i> that now i added a machine on 64.211.224.169 to do mail serving and stuff
</I>&gt;<i> for our employees and some other stuff.  for example, mail to
</I>&gt;<i> @mybiz-inc.com gets delivered to 64.211.224.169, while mail to @mybiz.com
</I>&gt;<i> gets directed to 64.211.224.166 (through the lvs director and to phl).
</I>&gt;<i> the problem is that phl can't send traffic to 64.211.224.169 -- phl seems 
</I>&gt;<i> to think that 64.211.224.169 is on its loopback interface.  64.211.224.169
</I>&gt;<i> tries to make nameservice queries for 169.160-175.224.211.64.in-addr.arpa
</I>&gt;<i> and *.mybiz.com to 64.211.224.162 and 64.211.224.163 (the auth nameservers
</I>&gt;<i> for that -- phl handles them), but phl never responds.  phl also tries to
</I>&gt;<i> deliver mail to 64.211.224.169, but it can't send traffic there.
</I>&gt;<i> 
</I>&gt;<i> check out:
</I>&gt;<i> [<A HREF="mailto:root@phl">root@phl</A> /root]# tcpdump -n host 64.211.224.169 and not port 53 &amp;
</I>&gt;<i> [1] 20668
</I>&gt;<i> User level filter, protocol ALL, datagram packet socket
</I>&gt;<i> tcpdump: listening on all devices
</I>&gt;<i> [<A HREF="mailto:root@phl">root@phl</A> /root]# ping -n -c 5 64.211.224.169
</I>&gt;<i> PING 64.211.224.169 (64.211.224.169) from 64.211.224.169 : 56(84) bytes of
</I>&gt;<i> data.
</I>&gt;<i> 14:04:36.653475   lo &gt; 64.211.224.169 &gt; 64.211.224.169: icmp: echo request
</I>&gt;<i> 14:04:36.653475   lo &lt; 64.211.224.169 &gt; 64.211.224.169: icmp: echo request
</I>&gt;<i> 14:04:36.653506   lo &gt; 64.211.224.169 &gt; 64.211.224.169: icmp: echo reply
</I>&gt;<i> 14:04:36.653506   lo &lt; 64.211.224.169 &gt; 64.211.224.169: icmp: echo reply
</I>&gt;<i> 64 bytes from 64.211.224.169: icmp_seq=0 ttl%5 timec usec
</I>&gt;<i> 14:04:37.649412   lo &gt; 64.211.224.169 &gt; 64.211.224.169: icmp: echo request
</I>&gt;<i> 14:04:37.649412   lo &lt; 64.211.224.169 &gt; 64.211.224.169: icmp: echo request
</I>&gt;<i> 14:04:37.649430   lo &gt; 64.211.224.169 &gt; 64.211.224.169: icmp: echo reply
</I>&gt;<i> 14:04:37.649430   lo &lt; 64.211.224.169 &gt; 64.211.224.169: icmp: echo reply
</I>&gt;<i> 64 bytes from 64.211.224.169: icmp_seq=1 ttl%5 time4 usec
</I>&gt;<i> 14:04:38.649446   lo &gt; 64.211.224.169 &gt; 64.211.224.169: icmp: echo request
</I>&gt;<i> 14:04:38.649446   lo &lt; 64.211.224.169 &gt; 64.211.224.169: icmp: echo request
</I>&gt;<i> 14:04:38.649462   lo &gt; 64.211.224.169 &gt; 64.211.224.169: icmp: echo reply
</I>&gt;<i> 14:04:38.649462   lo &lt; 64.211.224.169 &gt; 64.211.224.169: icmp: echo reply
</I>&gt;<i> 64 bytes from 64.211.224.169: icmp_seq=2 ttl%5 time( usec
</I>&gt;<i> 14:04:39.649495   lo &gt; 64.211.224.169 &gt; 64.211.224.169: icmp: echo request
</I>&gt;<i> 14:04:39.649495   lo &lt; 64.211.224.169 &gt; 64.211.224.169: icmp: echo request
</I>&gt;<i> 14:04:39.649516   lo &gt; 64.211.224.169 &gt; 64.211.224.169: icmp: echo reply
</I>&gt;<i> 14:04:39.649516   lo &lt; 64.211.224.169 &gt; 64.211.224.169: icmp: echo reply
</I>&gt;<i> 64 bytes from 64.211.224.169: icmp_seq=3 ttl%5 time7 usec
</I>&gt;<i> 14:04:40.649527   lo &gt; 64.211.224.169 &gt; 64.211.224.169: icmp: echo request
</I>&gt;<i> 14:04:40.649527   lo &lt; 64.211.224.169 &gt; 64.211.224.169: icmp: echo request
</I>&gt;<i> 14:04:40.649545   lo &gt; 64.211.224.169 &gt; 64.211.224.169: icmp: echo reply
</I>&gt;<i> 14:04:40.649545   lo &lt; 64.211.224.169 &gt; 64.211.224.169: icmp: echo reply
</I>&gt;<i> 64 bytes from 64.211.224.169: icmp_seq=4 ttl%5 time1 usec
</I>&gt;<i> 
</I>&gt;<i> --- 64.211.224.169 ping statistics ---
</I>&gt;<i> 5 packets transmitted, 5 packets received, 0% packet loss
</I>&gt;<i> round-trip min/avg/max/mdev = 0.028/0.038/0.063/0.014 ms
</I>&gt;<i> [<A HREF="mailto:root@phl">root@phl</A> /root]# fg
</I>&gt;<i> tcpdump -n host 64.211.224.169 and not port 53
</I>&gt;<i> 
</I>&gt;<i> 158 packets received by filter
</I>&gt;<i> [<A HREF="mailto:root@phl">root@phl</A> /root]# 
</I>&gt;<i> 
</I>&gt;<i> 
</I>&gt;<i> when 169 tries to telnet to 166 port 25 (which gets directed to phl):
</I>&gt;<i> [<A HREF="mailto:root@phl">root@phl</A> /root]# tcpdump -n host 64.211.224.169 and not port 53
</I>&gt;<i> User level filter, protocol ALL, datagram packet socket
</I>&gt;<i> tcpdump: listening on all devices
</I>&gt;<i> 14:05:20.460200 eth0 B arp who-has 64.211.224.169 tell 64.211.224.162
</I>&gt;<i> 14:05:50.883915 eth0 B arp who-has 64.211.224.166 tell 64.211.224.169
</I>&gt;<i> 14:05:50.884155 eth0 &lt; 64.211.224.169.1058 &gt; 64.211.224.166.smtp: S
</I>&gt;<i> 4151665104:4151665104(0) win 32120 &lt;mss 1460,sackOK,timestamp 25658644
</I>&gt;<i> 0,nop,wscale 0&gt; (DF)
</I>&gt;<i> 14:05:53.879424 eth0 &lt; 64.211.224.169.1058 &gt; 64.211.224.166.smtp: S
</I>&gt;<i> 4151665104:4151665104(0) win 32120 &lt;mss 1460,sackOK,timestamp 25658944
</I>&gt;<i> 0,nop,wscale 0&gt; (DF)
</I>&gt;<i> 
</I>&gt;<i> 725 packets received by filter
</I>&gt;<i> 
</I>&gt;<i> no response is ever sent.
</I>&gt;<i> 
</I>&gt;<i> 
</I>&gt;<i> when phl tries to send mail to mybiz-inc.com:
</I>&gt;<i> [<A HREF="mailto:root@phl">root@phl</A> /root]# dnsmx mybiz-inc.com
</I>&gt;<i> 0 mail.mybiz-inc.com
</I>&gt;<i> [<A HREF="mailto:root@phl">root@phl</A> /root]# dnsip mail.mybiz-inc.com
</I>&gt;<i> 64.211.224.169 
</I>&gt;<i> [<A HREF="mailto:root@phl">root@phl</A> /root]# telnet 64.211.224.169 25
</I>&gt;<i> Trying 64.211.224.169...
</I>&gt;<i> Connected to inc.mybiz.com (64.211.224.169).
</I>&gt;<i> Escape character is '^]'.
</I>&gt;<i> 220 phl.usa.mybiz ESMTP
</I>&gt;<i> ^]q
</I>&gt;<i> 
</I>&gt;<i> Connection closed.
</I>&gt;<i> [<A HREF="mailto:root@phl">root@phl</A> /root]# 
</I>&gt;<i> 
</I>&gt;<i> 14:07:39.001323   lo &gt; 64.211.224.169.1549 &gt; 64.211.224.169.smtp: S
</I>&gt;<i> 4291120419:4291120419(0) win 31072 &lt;mss 3884,sackOK,timestamp 441773751
</I>&gt;<i> 0,nop,wscale 0&gt; (DF)
</I>&gt;<i> 14:07:39.001323   lo &lt; 64.211.224.169.1549 &gt; 64.211.224.169.smtp: S
</I>&gt;<i> 4291120419:4291120419(0) win 31072 &lt;mss 3884,sackOK,timestamp 441773751
</I>&gt;<i> 0,nop,wscale 0&gt; (DF)
</I>&gt;<i> 14:07:39.001367   lo &gt; 64.211.224.169.smtp &gt; 64.211.224.169.1549: S
</I>&gt;<i> 200723:200723(0) ack 4291120420 win 31072 &lt;mss 3884,sackOK,timestamp
</I>&gt;<i> 441773751 441773751,nop,wscale 0&gt; (DF)
</I>&gt;<i> 14:07:39.001367   lo &lt; 64.211.224.169.smtp &gt; 64.211.224.169.1549: S
</I>&gt;<i> 200723:200723(0) ack 4291120420 win 31072 &lt;mss 3884,sackOK,timestamp
</I>&gt;<i> 441773751 441773751,nop,wscale 0&gt; (DF)
</I>&gt;<i> 14:07:39.001390   lo &gt; 64.211.224.169.1549 &gt; 64.211.224.169.smtp: . 1:1(0)
</I>&gt;<i> ack 1 win 31072 &lt;nop,nop,timestamp 441773751 441773751&gt; (DF)
</I>&gt;<i> 14:07:39.001390   lo &lt; 64.211.224.169.1549 &gt; 64.211.224.169.smtp: . 1:1(0)
</I>&gt;<i> ack 1 win 31072 &lt;nop,nop,timestamp 441773751 441773751&gt; (DF)
</I>&gt;<i> 14:07:39.007531   lo &gt; 64.211.224.169.smtp &gt; 64.211.224.169.1549: P
</I>&gt;<i> 1:26(25) ack 1 win 31072 &lt;nop,nop,timestamp 441773752 441773751&gt; (DF)
</I>&gt;<i> 14:07:39.007531   lo &lt; 64.211.224.169.smtp &gt; 64.211.224.169.1549: P
</I>&gt;<i> 1:26(25) ack 1 win 31072 &lt;nop,nop,timestamp 441773752 441773751&gt; (DF)
</I>&gt;<i> 14:07:39.007570   lo &gt; 64.211.224.169.1549 &gt; 64.211.224.169.smtp: . 1:1(0)
</I>&gt;<i> ack 26 win 31047 &lt;nop,nop,timestamp 441773752 441773752&gt; (DF)
</I>&gt;<i> 14:07:39.007570   lo &lt; 64.211.224.169.1549 &gt; 64.211.224.169.smtp: . 1:1(0)
</I>&gt;<i> ack 26 win 31047 &lt;nop,nop,timestamp 441773752 441773752&gt; (DF)
</I>&gt;<i> Connected to inc.mybiz.com (64.211.224.169).
</I>&gt;<i> 
</I>&gt;<i> 
</I>&gt;<i> it tries to send to itself.
</I>&gt;<i> 
</I>&gt;<i> does anyone have any idea why phl would think 64.211.224.169 is on its lo?
</I>&gt;<i> it seems to think that for all of 64.211.224.160/28.  if i telnet to port
</I>&gt;<i> 25 on any ip in that range, phl directs the request to itself on lo just
</I>&gt;<i> like 169.
</I>&gt;<i> 
</I>&gt;<i> anyone even understand this?  heh.  i'm seriously confused myself.
</I>&gt;<i> 
</I>&gt;<i> i'd love to hear any ideas.
</I>&gt;<i> 
</I>&gt;<i> -tcl.
</I>&gt;<i> 
</I>&gt;<i> 
</I>&gt;<i> _______________________________________________
</I>&gt;<i> LinuxVirtualServer.org mailing list - <A HREF="mailto:lvs-users@LinuxVirtualServer.org">lvs-users@LinuxVirtualServer.org</A>
</I>&gt;<i> Send requests to <A HREF="mailto:lvs-users-request@LinuxVirtualServer.org">lvs-users-request@LinuxVirtualServer.org</A>
</I>&gt;<i> or go to <A HREF="http://www.in-addr.de/mailman/listinfo/lvs-users">http://www.in-addr.de/mailman/listinfo/lvs-users</A>
</I>&gt;<i> 
</I>


</PRE>

^ permalink raw reply	[flat|nested] only message in thread

only message in thread, other threads:[~2001-02-12 22:46 UTC | newest]

Thread overview: (only message) (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2001-02-12 22:46 [LARTC] Re: need help with policy routing tc

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox