[LARTC] Need information on multi-homing

[LARTC] Need information on multi-homing

[Bruce Perens]

Is there a multi-homing FAQ? I've not been able to find it.

Here's my system's configuration:

	eth0:	SDSL at 216.15.108.186 (dnai-net)
	eth1:	ADSL at 67.114.175.138 (sbc-net)
	eth2:	DHCP network at 192.168.1.1 .
	lo:	127.0.0.1

	There is a firewall box acting as the DHCP server and router for the
	systems on the DHCP net. The Linux system doesn't run DHCP or
	masquerading.

If a connection is accepted on the IP address of one of these interfaces,
I'd like its packets to go out on the same interface. If there is a failure
on the path to eth0, connections to eth1 should continue to work, and vice
versa. But right now, eth0 has the default route and just about every packet
goes out of eth0 regardless of what interface they come in. So, I tried this:

	ip rule add from 216.15.108.186 table dnai-net 
	ip rule add from 67.114.175.138 table sbc-net
	ip route add default via 216.15.108.186 dev eth0 table dnai-net
	ip route add default via 67.114.175.138 dev eth1 table sbc-net

This doesn't work. I get a destination unreachable if I try to follow either
route. So, what am I doing wrong?

	Thanks

	Bruce Perens
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] Need information on multi-homing

[bert hubert]

On Sat, Mar 02, 2002 at 03:49:49PM -0800, Bruce Perens wrote:
> Is there a multi-homing FAQ? I've not been able to find it.

No, but it is very high on my agenda. Multihoming and loadbalancing are
among the most popular subjects if I count by my inbox.

> Here's my system's configuration:
> 
> 	eth0:	SDSL at 216.15.108.186 (dnai-net)
> 	eth1:	ADSL at 67.114.175.138 (sbc-net)
> 	eth2:	DHCP network at 192.168.1.1 .
> 	lo:	127.0.0.1
> 
> 	There is a firewall box acting as the DHCP server and router for the
> 	systems on the DHCP net. The Linux system doesn't run DHCP or
> 	masquerading.

So what IP address do packets have that come from the firewall box? 

> If a connection is accepted on the IP address of one of these interfaces,
> I'd like its packets to go out on the same interface. If there is a failure
> on the path to eth0, connections to eth1 should continue to work, and vice
> versa. But right now, eth0 has the default route and just about every packet
> goes out of eth0 regardless of what interface they come in. So, I tried this:
> 
> 	ip rule add from 216.15.108.186 table dnai-net 
> 	ip rule add from 67.114.175.138 table sbc-net
> 	ip route add default via 216.15.108.186 dev eth0 table dnai-net
> 	ip route add default via 67.114.175.138 dev eth1 table sbc-net
> 
> This doesn't work. I get a destination unreachable if I try to follow either
> route. So, what am I doing wrong?

You have no default route left for packets which came in over the firewall
box, which probably has a 192.168.1.x address? I think that is the number
one thing going wrong.

Now, as to what you want, you will need a way to separate connections to go
out over either eth0 or eth1 in a persistent way. This mostly means sending
out half your internal IP range over one interface and the other half over
the other.

This in turn implies SNAT or masquerading because you need to change the
source address - packets with the address of eth0 won't travel the net well
over eth1.

Regards,

bert hubert

-- 
http://www.PowerDNS.com          Versatile DNS Software & Services
http://www.tk                              the dot in .tk
http://lartc.org           Linux Advanced Routing & Traffic Control HOWTO
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] Need information on multi-homing

[Bruce Perens]

Bert Hubert:
> So what IP address do packets have that come from the firewall box? 

Oops, I didn't give enough information.  The Linux box is not a router -
it's a multi-homed server. The DHCP net has one of those retail router
boxes that does DHCP, NAT, and is a gateway to one of the DSL networks.

If I try to follow a policy route to either of the DSL interfaces from the
Linux system itself, not _through_ it as a router.

	ip rule add from 216.15.108.186 table dnai-net 
	ip rule add from 67.114.175.138 table sbc-net
	ip route add default via 216.15.108.186 dev eth0 table dnai-net
	ip route add default via 67.114.175.138 dev eth1 table sbc-net

	ping -I 67.114.175.138 www.gnu.org

I get an IMAP "unreachable" message back from the interface.

What I want is to be able to bind to the IP address of either of the DSL
interfaces, and have the packets routed to that interface reliably.
If there's a network partition effecting one DSL carrier and not the other,
it should still be possible to reach the system and the return packets should
be on the same network as the incoming ones.

Load-balancing is secondary, but I have some ideas there. For example, why not
have squid alternate addresses so that receive data is interleaved across the
two DSL lines? Wouldn't that balance better than simply routing half of
the internet through each interface?

	Thanks

	Bruce
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] Need information on multi-homing

[bert hubert]

On Sat, Mar 02, 2002 at 09:36:59PM -0800, Bruce Perens wrote:
> Bert Hubert:
> > So what IP address do packets have that come from the firewall box? 
> 
> Oops, I didn't give enough information.  The Linux box is not a router -
> it's a multi-homed server. The DHCP net has one of those retail router
> boxes that does DHCP, NAT, and is a gateway to one of the DSL networks.

Ah ok, so we have the following:


	eth0 SDSL
        ---  
         |     
       Linux -------- [router appliance] ---- ADSL -internet
         |    eth1  
        ---
        eth2 DHCP

> If I try to follow a policy route to either of the DSL interfaces from the
> Linux system itself, not _through_ it as a router.
> 
> 	ip rule add from 216.15.108.186 table dnai-net 
> 	ip rule add from 67.114.175.138 table sbc-net
> 	ip route add default via 216.15.108.186 dev eth0 table dnai-net
> 	ip route add default via 67.114.175.138 dev eth1 table sbc-net
> 
> 	ping -I 67.114.175.138 www.gnu.org
> 
> I get an IMAP "unreachable" message back from the interface.

I still lack information - what are the IP addresses of the Linux machine?
Right now you are telling the kernel to route packets FROM 216.15.108.186 TO
216.15.108.186, which isn't happening :-)

> What I want is to be able to bind to the IP address of either of the DSL
> interfaces, and have the packets routed to that interface reliably.
> If there's a network partition effecting one DSL carrier and not the other,
> it should still be possible to reach the system and the return packets should
> be on the same network as the incoming ones.

You should make sure the policy routes are made from the IP address of eth0
and eth1, and go to the addresses of the routers there. In that case, ping
-I should work. Right now you aren't routing anything to anything, leading
to an ICMP unreachable.

> Load-balancing is secondary, but I have some ideas there. For example, why not
> have squid alternate addresses so that receive data is interleaved across the
> two DSL lines? Wouldn't that balance better than simply routing half of
> the internet through each interface?

It isn't persistent. Many internet sites rely on the fact that users keep
coming from the same IP address during their session.

Regards,

bert

-- 
http://www.PowerDNS.com          Versatile DNS Software & Services
http://www.tk                              the dot in .tk
http://lartc.org           Linux Advanced Routing & Traffic Control HOWTO
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] Need information on multi-homing

[Bruce Perens]

On Sun, Mar 03, 2002 at 12:16:54PM +0100, bert hubert wrote:
> Right now you are telling the kernel to route packets FROM 216.15.108.186 TO
> 216.15.108.186, which isn't happening :-)

I should have known better than _that_. OK, it works now. Thanks very
much.

>> why not have squid alternate addresses so that receive data is interleaved
>> across the two DSL lines? Wouldn't that balance better than simply routing
>> half of the internet through each interface?
 
> It isn't persistent. Many internet sites rely on the fact that users keep
> coming from the same IP address during their session.

Good point. I don't know how, or if, proxies like squid use persistent
connections. I suspect that some load-balancing schemes already break them.
apt-get (Debian's package downloader) already knows how to parallelize hosts.
It would be nice to parallelize the interfaces at the same time.

FYI, this is a picture of my net.

	768K SDSL ------------------------------------ eth0


	1.5M..384K/128K ADSL ------------------------- eth1
					^
					| (10MBPS)
					|
				Retail Router Box		Linux Box
				Serves local DHCP
				hosts with DHCP, NAT,
				"firewall", gateway.
					^
					|
					|
        Local DHCP net ------------------------------- eth2

All networks are 100/10 MBit full-duplex switches. Both DSL modems and
the uplink port of the retail router run 10 MBit half-duplex, everything
else runs 100 MBit full-duplex. Not much should be going through
the retail router box, as the Linux box serves the local net with squid,
IMAP, SMTP, and telnet. I could dispense with the router box and route
with Linux if I wanted to.

Attached is what I ended up with in /etc/network/interfaces:

	Thanks

	Bruce

auto eth0
iface eth0 inet static
    address 216.15.108.186
    network 216.15.108.184
    netmask 255.255.255.248
    broadcast 216.15.108.255
    # gateway 216.15.108.185
    up ip rule add from 216.15.108.186 table dnai-net 
    up ip route add default via 216.15.108.185 dev eth0 table dnai-net
    up ip route add default via 216.15.108.185 dev eth0 metric 1 table main
    down ip rule del from 216.15.108.186 table dnai-net 
    down ip route del default via 216.15.108.185 dev eth0 table dnai-net
    down ip route del default via 216.15.108.185 dev eth0 metric 1 table main

auto eth1
iface eth1 inet static
    address 67.114.175.138
    network 67.114.175.137
    netmask 255.255.255.248
    broadcast 67.114.175.143
    up ip rule add from 67.114.175.138 table sbc-net
    up ip route add default via 67.114.175.137 dev eth1 table sbc-net
    up ip route add default via 67.114.175.137 dev eth1 metric 2 table main
    down ip rule del from 67.114.175.138 table sbc-net
    down ip route del default via 67.114.175.137 dev eth1 table sbc-net
    down ip route del default via 67.114.175.137 dev eth1 metric 2 table main

auto eth2
iface eth2 inet static
    address 192.168.1.1
    network 192.168.1.0
    netmask 255.255.255.0
    broadcast 192.168.1.255
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/