using iptables for poor-man's load balancing?

using iptables for poor-man's load balancing?

[Ian Douglas]

We currently have two machines (soon to be 4) that we'd like to balance our web
traffic to. Say for argument's sake that our public IP is 1.2.3.4 and our
internal LAN machines are:
  192.168.1.1
  192.168.1.12
  192.168.1.16
  192.168.1.17

Just curious if the following rules would work to round-robin the connections:

/sbin/iptables -t nat -A PREROUTING -p udp -d 1.2.3.4 --dport 80  -j DNAT \
   --to-destination 192.168.1.1:80 \
   --to-destination 192.168.1.12:80 \
   --to-destination 192.168.1.16:80 \
   --to-destination 192.168.1.17:80
/sbin/iptables -t nat -A PREROUTING -p udp -d 1.2.3.4 --dport 433  -j DNAT \
   --to-destination 192.168.1.1:80 \
   --to-destination 192.168.1.12:80 \
   --to-destination 192.168.1.16:80 \
   --to-destination 192.168.1.17:80

??

Thanks for any feedback or suggestions.

Ian

RE: using iptables for poor-man's load balancing?

[Ian Douglas]

> Just curious if the following rules would work to round-robin the connections:

> /sbin/iptables -t nat -A PREROUTING -p udp -d 1.2.3.4 --dport 80  -j DNAT \
                                         ^^^

Rules will be duplicated for 'tcp' as well as 'udp'. Just curious though if the
format looks correct.

-id

RE: using iptables for poor-man's load balancing?

[Ian Douglas]

> Say for argument's sake that our public IP is 1.2.3.4 and our
> internal LAN machines are:
>   192.168.1.1
>   192.168.1.12

(cut two of them out since they're not actually ready to run yet)

> Just curious if the following rules would work to round-robin the connections:
>
> /sbin/iptables -t nat -A PREROUTING -p udp -d 1.2.3.4 --dport 80  -j DNAT \
>    --to-destination 192.168.1.1:80 \
>    --to-destination 192.168.1.12:80
> /sbin/iptables -t nat -A PREROUTING -p udp -d 1.2.3.4 --dport 433  -j DNAT \
>    --to-destination 192.168.1.1:80 \
>    --to-destination 192.168.1.12:80

I tested this last night and it didn't work - every request went to 1.1

Should I be using "--to 192.168.1.1:80" instead of "--to-destination
192.168.1.1:80" ? I've seen documentation show the use of --to and a working
script for port forwarding that uses --to-destination

As a followup:

# uname -a
Linux icv.com 2.4.18-18.7.x #1 Wed Nov 13 20:29:30 EST 2002 i686 unknown

# rpm -qa | grep iptables
iptables-1.2.5-3
iptables-ipv6-1.2.5-3

# iptables -V
iptables v1.2.5

Andrej (Tink) suggested I write the list again and ask what version of iptables
introduced 'multiple targets' for port forwarding.

-id

Re: using iptables for poor-man's load balancing?

[Joel Newkirk]

On Wednesday 19 February 2003 06:17 pm, Ian Douglas wrote:
> > Say for argument's sake that our public IP is 1.2.3.4 and our
> > internal LAN machines are:
> >   192.168.1.1
> >   192.168.1.12
>
> (cut two of them out since they're not actually ready to run yet)
>
> > Just curious if the following rules would work to round-robin the
> > connections:
> >
> > /sbin/iptables -t nat -A PREROUTING -p udp -d 1.2.3.4 --dport 80  -j
> > DNAT \ --to-destination 192.168.1.1:80 \
> >    --to-destination 192.168.1.12:80

AFAIK, you can only DNAT to a contiguous range of IPs from a single rule, 
and the rule construction you have here will ignore 'excess' 
destinations.  If you have (or can arrange) your internal machines to 
have sequential IPs, then try something like this (which is 
"by-the-book"):

/sbin/iptables -t nat -A PREROUTING -p TCP -d 1.2.3.4 --dport 80  \
-j DNAT --to 192.168.1.1-192.168.1.5

The port number is only needed after the new IP if you are redirecting to 
a different port than the original destination, and "--to" is an 
acceptable substitute for both "--to-source" and "--to-destination".

j

Re: using iptables for poor-man's load balancing?

[Joel Newkirk]

On Wednesday 19 February 2003 07:12 pm, Joel Newkirk wrote:

> AFAIK, you can only DNAT to a contiguous range of IPs from a single
> rule, and the rule construction you have here will ignore 'excess'
> destinations.  

Well, I was wrong again.  :^)  Digging a bit further after posting this I 
find that multiple "-to" entries ARE valid, and should do what you want.  
The only reason I can think of (now) that all your traffic went to the 
first on the list is that there simply wasn't any load to speak of.  How 
were you testing?  Multiple simultaneous connections?  Otherwise (from 
my latest reading :^) it will simply keep sending traffic to the first 
on the list, only using the next one if there is more traffic 
'currently' (presumably based on the connection-tracking data) on the 
first destination than on the second.

> j

j

RE: using iptables for poor-man's load balancing?

[Ian Douglas]

> The only reason I can think of (now) that all your traffic went to the
> first on the list is that there simply wasn't any load to speak of.  How
> were you testing?

By blasting traffic at the system that's doing the packet forwarding. Perhaps I
can write some different code on the web servers that will hold the connection
for a while (ie: call a perl script that does a 'sleep 60' or something) and
test it that way.

> Multiple simultaneous connections?

Yes. I have a script that cycles through a perl script (I'll call it
blasticv.pl) that calls another perl script (I'll call it icv.pl) with 3 varying
parameters... each occurrence of that icv.pl makes a connection to the web
server to send and retrieve a chunk of data. "blasticv.pl" cycles through and
calls icv.pl 100 times with each of the 3 parameters, and not sleeping at all in
the loop. This should simulate 300 requests on the web servers that, given the
timing to complete a single request, would mean we'd have about 200 active
requests at the peak of activity, yet every single 'hit' on the systems landed
on 1.1, and not a single hit on 1.12.

> it will simply keep sending traffic to the first
> on the list, only using the next one if there is more traffic
> 'currently' (presumably based on the connection-tracking data) on the
> first destination than on the second.

... which is what I read, also, yet it seemed that causing a good volume of busy
traffic didn't forward anything to 1.12

-id

Re: using iptables for poor-man's load balancing?

[Joel Newkirk]

Hmmm.  A random neuron-firing leads me to another idea:

Try testing from multiple source IPs simultaneously.  Adding 2-3 alias 
interfaces on the test client (eth0=192.168.1.1,eth0:1=192.168.1.2, etc) 
and distributing your test connections across them could VERY possibly 
make the difference.  (two separate machines would guarantee a valid 
test, but I suspect multiple IP's would be sufficient)  Connection 
tracking may see that all the traffic is between the same two IP's 
(before the DNAT) and keep it coherent by always DNATting to the same 
destination.

If that's not it, (and I have a strange feeling it IS) I have two more 
suggestions:^)

1 - Try the contiguous-IP setup if possible, even if just changing the 
two servers to a different subnet for the test.  (and changing the IP of 
the iptables box to match, obviously, or adding a new IP as an alias on 
the internal interface)

2 - Modify your test approach to transfer a sizeable file on each 
connection.  Maybe a 1mb file, and try several simultaneous:

wget -q -O - http://server/onemegfile.tmp >/dev/null

Not a tremendous amount of traffic, but certainly enough to ensure 
several active connections.

j

On Wednesday 19 February 2003 07:55 pm, Ian Douglas wrote:
> > The only reason I can think of (now) that all your traffic went to
> > the first on the list is that there simply wasn't any load to speak
> > of.  How were you testing?
>
> By blasting traffic at the system that's doing the packet forwarding.
> Perhaps I can write some different code on the web servers that will
> hold the connection for a while (ie: call a perl script that does a
> 'sleep 60' or something) and test it that way.
>
> > Multiple simultaneous connections?
>
> Yes. I have a script that cycles through a perl script (I'll call it
> blasticv.pl) that calls another perl script (I'll call it icv.pl) with
> 3 varying parameters... each occurrence of that icv.pl makes a
> connection to the web server to send and retrieve a chunk of data.
> "blasticv.pl" cycles through and calls icv.pl 100 times with each of
> the 3 parameters, and not sleeping at all in the loop. This should
> simulate 300 requests on the web servers that, given the timing to
> complete a single request, would mean we'd have about 200 active
> requests at the peak of activity, yet every single 'hit' on the
> systems landed on 1.1, and not a single hit on 1.12.
>
> > it will simply keep sending traffic to the first
> > on the list, only using the next one if there is more traffic
> > 'currently' (presumably based on the connection-tracking data) on
> > the first destination than on the second.
>
> ... which is what I read, also, yet it seemed that causing a good
> volume of busy traffic didn't forward anything to 1.12
>
> -id

Re: using iptables for poor-man's load balancing?

[Fabrice MARIE]

Hello Ian,

On Wednesday 19 February 2003 05:53, Ian Douglas wrote:
> We currently have two machines (soon to be 4) that we'd like to balance our
> web traffic to. Say for argument's sake that our public IP is 1.2.3.4 and
> our internal LAN machines are:
>   192.168.1.1
>   192.168.1.12
>   192.168.1.16
>   192.168.1.17
> Just curious if the following rules would work to round-robin the
> connections:
> [...]
> Thanks for any feedback or suggestions.

I believe you can do that with the nth patch:
http://www.netfilter.org/documentation/HOWTO/netfilter-extensions-HOWTO-3.html#ss3.9
Even though I wrote the original patch, it's been greatly improved by Richard Wagner
<rwagner AT cloudnet.com> who added support for multiple counters and support for
matching on individual packets. [That's why I say "I believe" the match can do it].
It should work. Let us know of the outcome.

Have a nice day,

Fabrice.
--
Fabrice MARIE

"Silly hacker, root is for administrators"
       -Unknown

Re: using iptables for poor-man's load balancing?

[Andrej Ricnik]

>Just curious if the following rules would work to round-robin the 
>connections:
To my understanding it's not robin-go-round but distributing
new requests to the currently least used IP (the one with the
smallest number of active connections), which should pretty
much do what you want, anyway.


Cheers,
Tink

-----BEGIN GEEK CODE BLOCK-----
Version: 3.12
GCM/IT/O d-(++)@ a C+(+++)@ UL++>+++ P+>+++ L++ E+@ W+@ N+ o? K? w---@(+) 
O+++(+)@ M-@ V? PS+ PE- Y+ PGP++ t- 5- X- R-(*) tv-@ b+ DI++ D-- G++ !e* 
h--- r++ z?
------END GEEK CODE BLOCK------


_________________________________________________________________
Help STOP SPAM with the new MSN 8 and get 2 months FREE*  
http://join.msn.com/?page=features/junkmail

RE: using iptables for poor-man's load balancing?

[Ian Douglas]

> >Just curious if the following rules would work to round-robin the
> >connections:
> To my understanding it's not robin-go-round but distributing
> new requests to the currently least used IP (the one with the
> smallest number of active connections), which should pretty
> much do what you want, anyway.

Tink,

That didn't seem to work, all requests kept going to the same machine (first one
listed).

Any other ideas on how I can balance port 80 and port 443 across multiple
machines with iptables?

-id

RE: using iptables for poor-man's load balancing?

[mpboden]

i was recently reading the "Iptables Tutorial 1.1.16" by Oskar Andreasson,
and i'm getting the impression that your rules might be written incorrectly.
of course, i could be wrong, but if you check the following link,
http://iptables-tutorial.frozentux.net/chunkyhtml/targets.html, he
specifically talks about load balancing. in essence, he specifies a range of
ip addresses that the packets would randomly go to, and this is specified in
only one "--to-destination" instead of two as you have it written. the
following rule would send the packets randomly to any of the servers with
ip's from 192.168.1.1. through 192.168.1.12.

iptables -t nat -A PREROUTING -p tcp -d 1.2.3.4 --dport 80 -j
DNAT --to-destination 192.168.1.1-192.168.1.12

so perhaps a comma would work in your case if you specifically need to have
the ip's as you have them. i've never tried this, but it seems to make sense
to me.

iptables -t nat -A PREROUTING -p tcp -d 1.2.3.4 --dport 80 -j
DNAT --to-destination 192.168.1.1,192.168.1.12

furthermore, he mentions adding two more specific rules in the nat table to
allow hosts on the LAN as well as the firewall computer itself to access the
servers properly. please check those out.

mike



> Message: 7
> From: "Ian Douglas" <ian@icreditvision.com>
> To: <netfilter@lists.netfilter.org>
> Subject: RE: using iptables for poor-man's load balancing?
> Date: Wed, 19 Feb 2003 15:17:48 -0800
>
> > Say for argument's sake that our public IP is 1.2.3.4 and our
> > internal LAN machines are:
> >   192.168.1.1
> >   192.168.1.12
>
> (cut two of them out since they're not actually ready to run yet)
>
> > Just curious if the following rules would work to round-robin the
connections:
> >
> > /sbin/iptables -t nat -A PREROUTING -p udp -d 1.2.3.4 --dport 80  -j
DNAT \
> >    --to-destination 192.168.1.1:80 \
> >    --to-destination 192.168.1.12:80
> > /sbin/iptables -t nat -A PREROUTING -p udp -d 1.2.3.4 --dport 433  -j
DNAT \
> >    --to-destination 192.168.1.1:80 \
> >    --to-destination 192.168.1.12:80
>
> I tested this last night and it didn't work - every request went to 1.1
>
> Should I be using "--to 192.168.1.1:80" instead of "--to-destination
> 192.168.1.1:80" ? I've seen documentation show the use of --to and a
working
> script for port forwarding that uses --to-destination
>
> As a followup:
>
> # uname -a
> Linux icv.com 2.4.18-18.7.x #1 Wed Nov 13 20:29:30 EST 2002 i686 unknown
>
> # rpm -qa | grep iptables
> iptables-1.2.5-3
> iptables-ipv6-1.2.5-3
>
> # iptables -V
> iptables v1.2.5
>
> Andrej (Tink) suggested I write the list again and ask what version of
iptables
> introduced 'multiple targets' for port forwarding.
>

RE: using iptables for poor-man's load balancing?

[SBlaze]

I missed the begining of this thread...so ignore this and sorry if it doesn't
help

If you are using Linux this may be of some help

http://lartc.org/

It's a routing and traffic control Howto. Hope this helps

--- mpboden <mpboden@surfcity.net> wrote:
> i was recently reading the "Iptables Tutorial 1.1.16" by Oskar Andreasson,
> and i'm getting the impression that your rules might be written incorrectly.
> of course, i could be wrong, but if you check the following link,
> http://iptables-tutorial.frozentux.net/chunkyhtml/targets.html, he
> specifically talks about load balancing. in essence, he specifies a range of
> ip addresses that the packets would randomly go to, and this is specified in
> only one "--to-destination" instead of two as you have it written. the
> following rule would send the packets randomly to any of the servers with
> ip's from 192.168.1.1. through 192.168.1.12.
> 
> iptables -t nat -A PREROUTING -p tcp -d 1.2.3.4 --dport 80 -j
> DNAT --to-destination 192.168.1.1-192.168.1.12
> 
> so perhaps a comma would work in your case if you specifically need to have
> the ip's as you have them. i've never tried this, but it seems to make sense
> to me.
> 
> iptables -t nat -A PREROUTING -p tcp -d 1.2.3.4 --dport 80 -j
> DNAT --to-destination 192.168.1.1,192.168.1.12
> 
> furthermore, he mentions adding two more specific rules in the nat table to
> allow hosts on the LAN as well as the firewall computer itself to access the
> servers properly. please check those out.
> 
> mike
> 
> 
> 
> > Message: 7
> > From: "Ian Douglas" <ian@icreditvision.com>
> > To: <netfilter@lists.netfilter.org>
> > Subject: RE: using iptables for poor-man's load balancing?
> > Date: Wed, 19 Feb 2003 15:17:48 -0800
> >
> > > Say for argument's sake that our public IP is 1.2.3.4 and our
> > > internal LAN machines are:
> > >   192.168.1.1
> > >   192.168.1.12
> >
> > (cut two of them out since they're not actually ready to run yet)
> >
> > > Just curious if the following rules would work to round-robin the
> connections:
> > >
> > > /sbin/iptables -t nat -A PREROUTING -p udp -d 1.2.3.4 --dport 80  -j
> DNAT \
> > >    --to-destination 192.168.1.1:80 \
> > >    --to-destination 192.168.1.12:80
> > > /sbin/iptables -t nat -A PREROUTING -p udp -d 1.2.3.4 --dport 433  -j
> DNAT \
> > >    --to-destination 192.168.1.1:80 \
> > >    --to-destination 192.168.1.12:80
> >
> > I tested this last night and it didn't work - every request went to 1.1
> >
> > Should I be using "--to 192.168.1.1:80" instead of "--to-destination
> > 192.168.1.1:80" ? I've seen documentation show the use of --to and a
> working
> > script for port forwarding that uses --to-destination
> >
> > As a followup:
> >
> > # uname -a
> > Linux icv.com 2.4.18-18.7.x #1 Wed Nov 13 20:29:30 EST 2002 i686 unknown
> >
> > # rpm -qa | grep iptables
> > iptables-1.2.5-3
> > iptables-ipv6-1.2.5-3
> >
> > # iptables -V
> > iptables v1.2.5
> >
> > Andrej (Tink) suggested I write the list again and ask what version of
> iptables
> > introduced 'multiple targets' for port forwarding.
> >
> 
> 


=====
"No touchy NO TOUCHY! Emperor Kuzko -=Emperor's New Groove=-"

__________________________________________________
Do you Yahoo!?
Yahoo! Shopping - Send Flowers for Valentine's Day
http://shopping.yahoo.com