Re: [LARTC] Load balancing on multiple NICs with iproute

[Scott Russell <scottrus@raleigh.ibm.com>]

Thanks for the quick reply. Based on reading the HOWTO examples I thought
this was what I wanted to do and did some quick tests similar to what you
outlined below. My tests failed so I assumed I was missing the obvious and
posted to the list.

Well, I followed the examples below and ran the following commands in order:

  ip rule add from 9.37.145.124 table 124
  ip rule add from 9.37.145.120 table 120
  ip rule add from 9.37.145.127 table 127 
  ip route add default via 9.37.144.1 dev tr0 table 124
  ip route add default via 9.37.144.1 dev tr1 table 120
  ip route add default via 9.37.144.1 dev tr2 table 127
  ip route flush cache

A bit of back ground:
  
  tr0 = 9.37.145.124/20
  tr1 = 9.37.145.120/20
  tr2 = 9.37.145.127/20
  default GW = 9.37.144.1

So based on the above rules I would expect connections that originate on tr2
to stay on tr2. To test this I ftped to the system using the IP of tr2 and
started receiving a file. I then shut down interface tr0 with /sbin/ifdown
tr0. When I did so my file xfer stopped as well. When tr0 was brought back
up the xfer resumed. Bringing down tr2 and tr1 had no effect on the transfer
at all. (Expected since most likely the ftpd opened the ftp-data connection
on tr0 instead of tr2 as I wanted.)


Here's what my rules look like after running the above commands:

  [scottrus@linux scottrus]$ ip rule show
  0:      from all lookup local 
  32763:  from 9.37.145.127 lookup 127 
  32764:  from 9.37.145.120 lookup 120 
  32765:  from 9.37.145.124 lookup 124 
  32766:  from all lookup main 
  32767:  from all lookup 253 

Since the rules 23763-65 all appear before the main table I would expect
them to take precedence over anything in the main table. Is this correct?

So here's what my main routing table looks like after running the above rules:

  [scottrus@linux scottrus]$ ip route
  10.200.1.33 dev eth0  scope link 
  9.37.145.120 dev tr1  scope link 
  9.37.145.127 dev tr2  scope link 
  9.37.145.124 dev tr0  scope link 
  10.200.1.0/24 dev eth0  proto kernel  scope link  src 10.200.1.33 
  9.37.144.0/20 dev tr0  proto kernel  scope link  src 9.37.145.124 
  9.37.144.0/20 dev tr1  proto kernel  scope link  src 9.37.145.120 
  9.37.144.0/20 dev tr2  proto kernel  scope link  src 9.37.145.127 
  10.200.0.0/16 via 10.200.1.1 dev eth0 
  127.0.0.0/8 dev lo  scope link 
  default via 9.37.144.1 dev tr2 
  default via 9.37.144.1 dev tr1 
  default via 9.37.144.1 dev tr0 

I expect that the rules I inserted are being ignored (and hence the tables
the rules point to are ignored as well). This means that the primary routing
table is being used and seems to explain why things are going out tr0 by
default.

Am I off base on what I'm seeing going on? I think I also have ECMP (Equal
Cost MultiPath) compiled into my kernel, is that affecting things as well?

Again, thanks for the help.

-- Scott

On Sat, Mar 17, 2001 at 12:45:43PM +0100, bert hubert wrote:
> On Fri, Mar 16, 2001 at 09:06:18PM -0500, Scott Russell wrote:
> 
> > I'm using DNS to do cheap and easy round robin style connections to the box. 
> > 
> >   [scottrus@linux scottrus]$ host ftp3.linux.ibm.com
> >   ftp3.linux.ibm.com has address 9.37.145.124
> >   ftp3.linux.ibm.com has address 9.37.145.120
> >   ftp3.linux.ibm.com has address 9.37.145.127
> 
> DNS is wildly underestimated as a loadbalancing/distribution device. It
> works very well, especially if you set your TTL pretty low.
> 
> > 1) Incoming requests are responded to on the same interface they came in on.
> > For example an ftp connection coming in on tr1 has all TX / RX packets stick
> > to tr1. 
> 
> This is possible, I think.
> 
> > As you can see I've tried simply adding default routes for each token ring
> > interface and I think this is a step in the right direction but I'm missing
> > something.
> 
> Add source routing, as described in
> http://ds9a.nl/2.4Routing/HOWTO//cvs/2.4routing/output/2.4routing-4.html#ss4.1
> 
> Route packets with a source of 9.37.145.124 to trX, .120 to trY and 127 to
> trZ.
> 
> > Any help would be great. Examples would be better of course. Thanks much!
> 
> # ip rule add from 9.37.145.124 table 124
> # ip rule add from 9.37.145.120 table 120
> # ip rule add from 9.37.145.127 table 127
> 
> # ip route add default via 9.37.145.1 dev tr0 table 124
> # ip route add default via 9.37.145.1 dev tr1 table 120
> # ip route add default via 9.37.145.1 dev tr2 table 127
> # ip route flush cache
> 
> Something like that should work. I messed up the ip addresses I think, but
> hey :-)
> 
> Regards,
> 
> bert
> 
> -- 
> http://www.PowerDNS.com      Versatile DNS Services  
> Trilab                       The Technology People   
> 'SYN! .. SYN|ACK! .. ACK!' - the mating call of the internet
> 
> _______________________________________________
> LARTC mailing list / LARTC@mailman.ds9a.nl
> http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/2.4Routing/

-- 
Regards,
 Scott Russell (scottrus@raleigh.ibm.com)
 Linux Technology Center, System Admin, RHCE.
 T/L 441-9289 / External 919-543-9289
 http://bzimage.raleigh.ibm.com/webcam


_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/2.4Routing/