Iptables for load balancing internet traffic

Iptables for load balancing internet traffic

[Danny]

Hi everyone,


Is there a perfect solution to use iptables for load balancing outgoing 
internet traffic ?

I am using Debian Sarge 3.1 [ kernel 2.4.27 ] [ iptables v1.2.11 ]  I 
chose this kernel specifically for this purpose.

I have tried the below 2 techniques  : -


1 > Tried using random patch
iptables -t mangle -A OUTPUT -o ! eth0    -m random --average 50 -j MARK 
--set-mark 1
iptables -t mangle -A PREROUTING -i eth0  -m random --average 50 -j MARK 
--set-mark 1
ip ro add table 10 default via 10.90.101.1  dev eth2
ip ru add fwmark 1 table 10
ip ro fl ca


iptables -t mangle -A OUTPUT -o ! eth0   -m random --average 50 -j MARK 
--set-mark 2
iptables -t mangle -A PREROUTING -i eth0 -m random --average 50 -j MARK 
--set-mark 2
ip ro add table 20 default via 10.90.102.1 dev eth3
ip ru add fwmark 2 table 20
ip ro fl ca


iptables -t nat -A POSTROUTING -o eth2 -j SNAT --to 10.90.101.2
iptables -t nat -A POSTROUTING -o eth3 -j SNAT --to 10.90.102.2
ip ro add default nexthop via 10.90.101.1  dev eth2 weight 1 nexthop via 
10.90.102.1 dev eth3 weight 1

#-------------end -------------
Well this didnt do the job for me ! It was breaking TCP connections. It 
was load balancing UDP packets well though !


2> Tried using nth patch
iptables -t mangle -A PREROUTING -i eth0 -m state --state NEW   -m nth 
--every 2 --counter 2  --packet 0  -j CONNMARK  --set-mark 1
iptables -t mangle -A PREROUTING -i eth0 -m state --state NEW   -m nth 
--every 2 --counter 2  --packet 1  -j CONNMARK  --set-mark 2
iptables -t nat -A POSTROUTING -o eth2 -j SNAT --to 10.90.101.2
iptables -t nat -A POSTROUTING -o eth3 -j SNAT --to 10.90.102.2
ip ro add table 10 default via 10.90.101.1  dev eth2
ip ru add fwmark 1 table 10
ip ro fl ca
ip ro add table 20 default via 10.90.102.1 dev eth3
ip ru add fwmark 2 table 20
ip ro fl ca
ip ro add default nexthop via 10.90.101.1  dev eth2 weight 1 nexthop via 
10.90.102.1 dev eth3 weight 1
#----end

This kept the tcp connections intact. But it wasnt distributing the 
internet traffic evenly.
The ditribution of connections through the links are not even.

# iptables -L PREROUTING -t mangle -nv
Chain PREROUTING (policy ACCEPT 82796 packets, 20M bytes)
pkts bytes target     prot opt in     out     source               
destination
  8   480 CONNMARK   all  --  eth0   *       0.0.0.0/0            
0.0.0.0/0           state NEW every 2th counter #2 packet #0 CONNMARK 
set 0x1
  8   480 CONNMARK   all  --  eth0   *       0.0.0.0/0            
0.0.0.0/0           state NEW every 2th counter #2 packet #1 CONNMARK 
set 0x2

-- this shows that connections have been marked evenly. But 
/proc/net/ip_conntrack doesnt show even number of connections.

# sed  's/^.*\(mark=[12]\).*$/\1/g'   /proc/net/ip_conntrack  | grep -v 
"mark=0"
mark=2
mark=2
mark=2
mark=2
mark=2
mark=2
mark=2
mark=1
mark=2
mark=2


Most of the connections here are marked 2. [ sometime most of them are 
marked 1 ]



Is there any other perfect solution for load balancing internet traffic ?

I have gone through many forums, but I havent got the perfect solution yet.


Thank you,
Danny

Re: Iptables for load balancing internet traffic

[bill]

Danny a écrit :
> Is there a perfect solution to use iptables for load balancing outgoing 
> internet traffic ?
> 
> I am using Debian Sarge 3.1 [ kernel 2.4.27 ] [ iptables v1.2.11 ]  I 
> chose this kernel specifically for this purpose.
> 
> I have tried the below 2 techniques  : -
> 1 > Tried using random patch
> 2> Tried using nth patch

And did you tried the CLUSTERIP target?
    CLUSTERIP
This module allows you to configure a  simple  cluster  of  nodes  that
share a certain IP and MAC address without an explicit load balancer in
front of them.  Connections  are  statically  distributed  between  the
nodes in this cluster.

Re: Iptables for load balancing internet traffic

[Danny]

>
bill wrote:
>
> And did you tried the CLUSTERIP target?
>    CLUSTERIP
> This module allows you to configure a  simple  cluster  of  nodes  that
> share a certain IP and MAC address without an explicit load balancer in
> front of them.  Connections  are  statically  distributed  between  the
> nodes in this cluster.
>
>
Well, I havent tried this. But this looks good for an inbound traffic to 
a specific port.
In my case, the traffic is outgoing and has to be NATed.

I am sure that there is some way out, using nth patch. Someone might 
have done this.


Thank you,
- Danny

Re: Iptables for load balancing internet traffic

[Jorge Nerín]

On 11/14/06, Danny <dineshg@hostway.com> wrote:
>
> >
> bill wrote:
> >
> > And did you tried the CLUSTERIP target?
> >    CLUSTERIP
> > This module allows you to configure a  simple  cluster  of  nodes  that
> > share a certain IP and MAC address without an explicit load balancer in
> > front of them.  Connections  are  statically  distributed  between  the
> > nodes in this cluster.
> >
> >
> Well, I havent tried this. But this looks good for an inbound traffic to
> a specific port.
> In my case, the traffic is outgoing and has to be NATed.
>
> I am sure that there is some way out, using nth patch. Someone might
> have done this.
>
>
> Thank you,
> - Danny
>
>

Yes there is a way, if you search you will find this:

Routing for multiple uplinks/providers
http://lartc.org/howto/lartc.rpdb.multiple-links.html

It works.

-- 
Jorge Nerín
<jnerin@gmail.com>

Re: Iptables for load balancing internet traffic

[Danny]

Jorge Nerín wrote:
>
> Yes there is a way, if you search you will find this:
>
> Routing for multiple uplinks/providers
> http://lartc.org/howto/lartc.rpdb.multiple-links.html
>
> It works.
>
 >> Note that balancing will not be perfect, as it is route based, and 
routes are cached.

Yup ! I tested this. But it doesnt do a perfect Round Robin [ in this 
case nth patch worked better ].


- Danny

Re: Iptables for load balancing internet traffic

[Jorge Nerín]

On 11/14/06, Danny <dineshg@hostway.com> wrote:
> Jorge Nerín wrote:
> >
> > Yes there is a way, if you search you will find this:
> >
> > Routing for multiple uplinks/providers
> > http://lartc.org/howto/lartc.rpdb.multiple-links.html
> >
> > It works.
> >
>  >> Note that balancing will not be perfect, as it is route based, and
> routes are cached.
>
> Yup ! I tested this. But it doesnt do a perfect Round Robin [ in this
> case nth patch worked better ].
>
>
> - Danny
>
>

You will not get a perfect round robin with few conections because all
the packets related to a connection must use always the same interface
or else it would break as the ip address would be changing as seen by
the other side.

If you have a lot of connections this will work almost perfectly as
the deviation from the perfect round robin will get smaller when the
number of samples (connections) gets bigger and all of them balance
each other.

-- 
Jorge Nerín
<jnerin@gmail.com>