load-balancing with netfilter: round robin or what??

load-balancing with netfilter: round robin or what??

[Pedro Salazar]

Greetings,

I pretend to load-balancing to two servers but apparently I think only
one machine is getting the requests.

These are the rules I run in iptables:

/sbin/iptables -v -P INPUT ACCEPT
/sbin/iptables -v -P OUTPUT ACCEPT
/sbin/iptables -v -P FORWARD ACCEPT
/sbin/iptables -v -t nat -A PREROUTING --dst 192.168.89.44 -p tcp \
 --dport 9000 -j DNAT --to-destination 192.168.89.158:80 \
 --to-destination 192.168.89.159:80
/sbin/iptables -v -t nat -A POSTROUTING --dst 192.168.89.158/31 \ 
-p tcp --dport 80 -j SNAT --to-source 192.168.89.44
/sbin/iptables -v -t nat -A OUTPUT -p tcp -m tcp --dport 9000 \ 
--dst 192.168.89.44 -j DNAT --to-destination 192.168.89.158:80 \
 --to-destination 192.168.89.159:80

As you can see, I have two destinations hosts, 192.168.89.158/31. The
192.168.89.44 is the gateway itself that is redirecting requests on port
9000 to the two http servers. All of them are in the same network.

However, I think only the .158 is getting the requests (I have verified
that with tcpdump).

So, what is wrong or what is needed? I supposed the requests would be
load-balanced...

I'm using a linux redhat 7.3 box with default kernel 2.4.18-3. 

thanks,
Pedro Salazar.
-- 
PS
pedro-b-salazar@ptinovacao.pt
PGP:0E129E31D803BC61

Re: load-balancing with netfilter: round robin or what??

[Jonathan G - Mailing List]

Hi pedro / Hola pedro,

what do you want? LB for firewalls or LB for web servers? I didn't 
understand your question.

If the reality is that you need a load balancing of web servers you may 
deploy easy dns round robin. this could be easy to configure but a bad 
idea for high information volume.

If you prefer you may implement HA (high availability) with a master and 
a slave box. With this config one of your server will ask connections 
until it falls, then the second box will answer connections to the 
moment tha first comes up. This is the most used implementation to give 
continuosuly service.

I have deployed in many installations this configuration for firewalls. 
I have used VRRP (virtual redundancy routing protocol) on nokia boxes 
for CheckPoint Firewall-1 clusters and works fine. You may use the same 
idea installing the vrrpd daemon in the www boxes. Could run fine.

If you don't know how vrrp works, it implements a virtual mac and a 
virtual ip, so form iptables you only have to forward traffic to the 
virual ip. The machine that is up will take the connection and will 
server the pages.

Best regards,

jonathan



Pedro Salazar wrote:
> Greetings,
> 
> I pretend to load-balancing to two servers but apparently I think only
> one machine is getting the requests.
> 
> These are the rules I run in iptables:
> 
> /sbin/iptables -v -P INPUT ACCEPT
> /sbin/iptables -v -P OUTPUT ACCEPT
> /sbin/iptables -v -P FORWARD ACCEPT
> /sbin/iptables -v -t nat -A PREROUTING --dst 192.168.89.44 -p tcp \
>  --dport 9000 -j DNAT --to-destination 192.168.89.158:80 \
>  --to-destination 192.168.89.159:80
> /sbin/iptables -v -t nat -A POSTROUTING --dst 192.168.89.158/31 \ 
> -p tcp --dport 80 -j SNAT --to-source 192.168.89.44
> /sbin/iptables -v -t nat -A OUTPUT -p tcp -m tcp --dport 9000 \ 
> --dst 192.168.89.44 -j DNAT --to-destination 192.168.89.158:80 \
>  --to-destination 192.168.89.159:80
> 
> As you can see, I have two destinations hosts, 192.168.89.158/31. The
> 192.168.89.44 is the gateway itself that is redirecting requests on port
> 9000 to the two http servers. All of them are in the same network.
> 
> However, I think only the .158 is getting the requests (I have verified
> that with tcpdump).
> 
> So, what is wrong or what is needed? I supposed the requests would be
> load-balanced...
> 
> I'm using a linux redhat 7.3 box with default kernel 2.4.18-3. 
> 
> thanks,
> Pedro Salazar.

-- 
__________________________________________________________________
Jonathan Gonzalez - SureStorm.com Security Site - Madrid/MA/SPAIN
http://www.surestorm.com -  GnuPG Key ID = 0xAA3EAC08

/"\
\ /  ASCII RIBBON CAMPAIGN
  X   Against HTML mail & Microsoft attachments
/ \

Re: load-balancing with netfilter: round robin or what??

[Pedro Salazar]

Greetings,

On Mon, 2003-10-06 at 12:23, Jonathan G - Mailing List wrote:
> Hi pedro / Hola pedro,
> 

Hi Jonathan,

(BTW: I'm portuguese and not spanish, so it's "Olá" and not "Hola", but
you have tried ;-)

> what do you want? LB for firewalls or LB for web servers? I didn't 
> understand your question.
> 

For production purposes I'll have a black box (specific hardware) that
will do the load-balancing and fail-over redundancy (high availability)
by IP (not by service).


> If the reality is that you need a load balancing of web servers you may 
> deploy easy dns round robin. this could be easy to configure but a bad 
> idea for high information volume.
> 

No, I'm not interested in DNS round-robin.

> If you prefer you may implement HA (high availability) with a master and 
> a slave box. With this config one of your server will ask connections 
> until it falls, then the second box will answer connections to the 
> moment tha first comes up. This is the most used implementation to give 
> continuosuly service.
> 
> I have deployed in many installations this configuration for firewalls. 
> I have used VRRP (virtual redundancy routing protocol) on nokia boxes 
> for CheckPoint Firewall-1 clusters and works fine. You may use the same 
> idea installing the vrrpd daemon in the www boxes. Could run fine.
> 

Yes, I already tried VRRP for high-availability. One would be the
master, and one of the others slaves would assume that role in case of
primary failure. But I never used the VRRP in a load-balancing way but
always in high-availability mode.

Since my application servers (JBOSS/J2EE) and database servers (Oracle
RAC) are in *cluster* mode, I want to deliver requests in a
load-balancing way, to distribute the pay load to all of them. And it
would be to the cluster it self to decide what would be the node to
dispatch the request.

For now, since I haven't yet the black box to do the load balancing, I'm
simulating with NAT. However, I don't know what is the algorithm to
distribute the requests. Only when I pick 1000 requests I see
"load-balancing". Until then, single and manual requests, only one
server is handling the requests. For sure, I can say that it's not a
round robin algorithm. 

My question is, could I put a round robin specific algorithm or any
other politic though NAT??? Or should I go for another solution? There
are more advanced solutions as Linux Virtual Server to do the job, but
it would be a more complex solution.

BTW, I'm using Linux Redhat AS 2.1. I believe that could exist a
solution integrated on it...

thanks,
Pedro Salazar.
-- 
PS
pedro-b-salazar@ptinovacao.pt
PGP:0E129E31D803BC61

Re: load-balancing with netfilter: round robin or what??

[Pedro Salazar]

> BTW, I'm using Linux Redhat AS 2.1. I believe that could exist a
> solution integrated on it...


Redhat AS only for the application servers and the Oracle RAC. My linux
box with NAT is Redhat 7.3 (now with kernel 2.4.22).

-- 
PS
pedro-b-salazar@ptinovacao.pt
PGP:0E129E31D803BC61

Re: load-balancing with netfilter: round robin or what??

[Joel Newkirk]

Olá Pedro.  (g - I read ahead ;^)

On Mon, 2003-10-06 at 05:59, Pedro Salazar wrote:
> Greetings,
> 
> I pretend to load-balancing to two servers but apparently I think only
> one machine is getting the requests.
> 
> These are the rules I run in iptables:

> /sbin/iptables -v -t nat -A PREROUTING --dst 192.168.89.44 -p tcp \
>  --dport 9000 -j DNAT --to-destination 192.168.89.158:80 \
>  --to-destination 192.168.89.159:80

> /sbin/iptables -v -t nat -A OUTPUT -p tcp -m tcp --dport 9000 \ 
> --dst 192.168.89.44 -j DNAT --to-destination 192.168.89.158:80 \
>  --to-destination 192.168.89.159:80
> 
> As you can see, I have two destinations hosts, 192.168.89.158/31. The
> 192.168.89.44 is the gateway itself that is redirecting requests on port
> 9000 to the two http servers. All of them are in the same network.

But the rule performing the balancing is in OUTPUT table, and so only
affects connections from the NAT box itself...  Also, I think your
double --to construction is invalid, iptables accepts it but it appears
never to work.

Try this rule:

/sbin/iptables -t nat -A PREROUTING -d 192.168.89.44 -p tcp --dport  \
9000 -j DNAT --to 192.168.89.158-192.168.89.159:80

It will distribute new connections 'randomly' across the DNAT ip space,
in this case two IPs.  This isn't 'real' load-balancing, but it's a
quick and easy substitute that is sufficient for many purposes.

j

--- You know you have ADD when...You run Linux just because KDE lets you
have 16 simultaneous desktops.

Re: load-balancing with netfilter: round robin or what??

[Joel Newkirk]

Olá Pedro.  (g - I read ahead ;^)

On Mon, 2003-10-06 at 05:59, Pedro Salazar wrote:
> Greetings,
> 
> I pretend to load-balancing to two servers but apparently I think only
> one machine is getting the requests.
> 
> These are the rules I run in iptables:

> /sbin/iptables -v -t nat -A PREROUTING --dst 192.168.89.44 -p tcp \
>  --dport 9000 -j DNAT --to-destination 192.168.89.158:80 \
>  --to-destination 192.168.89.159:80

> /sbin/iptables -v -t nat -A OUTPUT -p tcp -m tcp --dport 9000 \ 
> --dst 192.168.89.44 -j DNAT --to-destination 192.168.89.158:80 \
>  --to-destination 192.168.89.159:80
> 
> As you can see, I have two destinations hosts, 192.168.89.158/31. The
> 192.168.89.44 is the gateway itself that is redirecting requests on port
> 9000 to the two http servers. All of them are in the same network.

But the rule performing the balancing is in OUTPUT table, and so only
affects connections from the NAT box itself...  Also, I think your
double --to construction is invalid, iptables accepts it but it appears
never to look past the first IP.

Try this rule:

/sbin/iptables -t nat -A PREROUTING -d 192.168.89.44 -p tcp --dport  \
9000 -j DNAT --to 192.168.89.158-192.168.89.159:80

It will distribute new connections 'randomly' across the DNAT ip space,
in this case two IPs.  This isn't 'real' load-balancing, but it's a
quick and easy substitute that is sufficient for many purposes.

j

RE: load-balancing with netfilter: round robin or what??

[George Vieira]

Round robin DNS isn't so bad, I mean www.microsoft.com ,www.ibm.com and other large sites do it so depending on the scale of your site and estimate number of hits your going to get, use a combination of round robin and possibly 2 firewalls with multiple webservers behind it (web farm)..

The only way for iptables to do it without using some sort of proxy or RRDNS is to use the p-o-m NTH patch and possibly CONNMARK to do it..

Here's a snip from a past post by Ramin Dousti which may give you some guidance..

<snip>
> Any more suggestions??? any other ideas????
Yes. CONNMARK the conn's and route them accordingly:

iptables -t nat -A POSTROUTING -o ppp0 -j SNAT --to-source <ppp0-ip-addr>
iptables -t nat -A POSTROUTING -o ppp1 -j SNAT --to-source <ppp1-ip-addr>
iptables -t nat -A POSTROUTING -o ppp2 -j SNAT --to-source <ppp2-ip-addr>
iptables -t nat -A POSTROUTING -o ppp3 -j SNAT --to-source <ppp3-ip-addr>

iptables -t mangle -A PREROUTING -m nth --every 4 --packet 1 -m state --state new  -j CONNMARK --set-mark 1
iptables -t mangle -A PREROUTING -m nth --every 4 --packet 2 -m state --state new  -j CONNMARK --set-mark 2
iptables -t mangle -A PREROUTING -m nth --every 4 --packet 3 -m state --state new  -j CONNMARK --set-mark 3
iptables -t mangle -A PREROUTING -m nth --every 4 --packet 0 -m state --state new  -j CONNMARK --set-mark 4

iptables -t mangle -A OUTPUT     -m nth --every 4 --packet 1 -m state --state new  -j CONNMARK --set-mark 1
iptables -t mangle -A OUTPUT     -m nth --every 4 --packet 2 -m state --state new  -j CONNMARK --set-mark 2
iptables -t mangle -A OUTPUT     -m nth --every 4 --packet 3 -m state --state new  -j CONNMARK --set-mark 3
iptables -t mangle -A OUTPUT     -m nth --every 4 --packet 0 -m state --state new  -j CONNMARK --set-mark 4

iptables -t mangle -A POSTROUTING -m connmark --mark 1 -j ROUTE --oif ppp1
iptables -t mangle -A POSTROUTING -m connmark --mark 2 -j ROUTE --oif ppp2
iptables -t mangle -A POSTROUTING -m connmark --mark 3 -j ROUTE --oif ppp3
iptables -t mangle -A POSTROUTING -m connmark --mark 4 -j ROUTE --oif ppp0

Haven't had the chance to test it, though.

Ramin
</snip>

Thanks,
____________________________________________
George Vieira
Systems Manager
georgev@citadelcomputer.com.au

Citadel Computer Systems Pty Ltd
http://www.citadelcomputer.com.au

Phone   : +61 2 9955 2644
HelpDesk: +61 2 9955 2698
 

> -----Original Message-----
> From: Pedro Salazar [mailto:pedro-b-salazar@ptinovacao.pt]
> Sent: Monday, 6 October 2003 8:00 PM
> To: netfilter@lists.netfilter.org
> Subject: load-balancing with netfilter: round robin or what??
> 
> 
> Greetings,
> 
> I pretend to load-balancing to two servers but apparently I think only
> one machine is getting the requests.
> 
> These are the rules I run in iptables:
> 
> /sbin/iptables -v -P INPUT ACCEPT
> /sbin/iptables -v -P OUTPUT ACCEPT
> /sbin/iptables -v -P FORWARD ACCEPT
> /sbin/iptables -v -t nat -A PREROUTING --dst 192.168.89.44 -p 
> tcp --dport 9000 -j DNAT --to-destination 192.168.89.158:80 
> --to-destination 192.168.89.159:80
> /sbin/iptables -v -t nat -A POSTROUTING --dst 
> 192.168.89.158/31 -p tcp --dport 80 -j SNAT --to-source 192.168.89.44
> /sbin/iptables -v -t nat -A OUTPUT -p tcp -m tcp --dport 9000 
> --dst 192.168.89.44 -j DNAT --to-destination 
> 192.168.89.158:80 --to-destination 192.168.89.159:80
> 
> As you can see, I have two destinations hosts, 192.168.89.158/31. The
> 192.168.89.44 is the gateway itself that is redirecting 
> requests on port
> 9000 to the two http servers. All of them are in the same network.
> 
> However, I think only the .158 is getting the requests (I 
> have verified
> that with tcpdump).
> 
> So, what is wrong or what is needed? I supposed the requests would be
> load-balanced...
> 
> I'm using a linux redhat 7.3 box with default kernel 2.4.18-3. 
> 
> thanks,
> Pedro Salazar.
> -- 
> PS
> pedro-b-salazar@ptinovacao.pt
> PGP:0E129E31D803BC61
> 
> 
> 

Re: load-balancing with netfilter: round robin or what??

[Pedro Salazar]

> But the rule performing the balancing is in OUTPUT table, and so only
> affects connections from the NAT box itself...  Also, I think your
> double --to construction is invalid, iptables accepts it but it appears
> never to work.
> 
> Try this rule:
> 
> /sbin/iptables -t nat -A PREROUTING -d 192.168.89.44 -p tcp --dport  \
> 9000 -j DNAT --to 192.168.89.158-192.168.89.159:80
> 
> It will distribute new connections 'randomly' across the DNAT ip space,
> in this case two IPs.  This isn't 'real' load-balancing, but it's a
> quick and easy substitute that is sufficient for many purposes.
> 

Apparently you have right. I changed to a simple "--to" as you said, and
now I noticed that the requests are quite "round-robin". However, the 2
"--to" have worked also but weren't so *load-balanced* as with a simple
"--to" (I verified with a large massive requests). But don't forget that
my IP addresses are sequencial and in another scenario I probably must
use the rule with 2 "--to", no?

Thanks! (to all of you)

Pedro Salazar. 
-- 
PS
pedro-b-salazar@ptinovacao.pt
PGP:0E129E31D803BC61

RE: load-balancing with netfilter: round robin or what??

[Khan, Arshad]

how does this compare with the Linux Virtual Server project?

Arshad

-----Original Message-----
From: Pedro Salazar [mailto:pedro-b-salazar@ptinovacao.pt]
Sent: 07 October 2003 10:36
To: newkirk@newkirk.us
Cc: netfilter@lists.netfilter.org
Subject: Re: load-balancing with netfilter: round robin or what??


> But the rule performing the balancing is in OUTPUT table, and so only
> affects connections from the NAT box itself...  Also, I think your
> double --to construction is invalid, iptables accepts it but it appears
> never to work.
> 
> Try this rule:
> 
> /sbin/iptables -t nat -A PREROUTING -d 192.168.89.44 -p tcp --dport  \
> 9000 -j DNAT --to 192.168.89.158-192.168.89.159:80
> 
> It will distribute new connections 'randomly' across the DNAT ip space,
> in this case two IPs.  This isn't 'real' load-balancing, but it's a
> quick and easy substitute that is sufficient for many purposes.
> 

Apparently you have right. I changed to a simple "--to" as you said, and
now I noticed that the requests are quite "round-robin". However, the 2
"--to" have worked also but weren't so *load-balanced* as with a simple
"--to" (I verified with a large massive requests). But don't forget that
my IP addresses are sequencial and in another scenario I probably must
use the rule with 2 "--to", no?

Thanks! (to all of you)

Pedro Salazar. 
-- 
PS
pedro-b-salazar@ptinovacao.pt
PGP:0E129E31D803BC61

RE: load-balancing with netfilter: round robin or what??

[Pedro Salazar]

On Tue, 2003-10-07 at 10:41, Khan, Arshad wrote:
> how does this compare with the Linux Virtual Server project?
> 
> Arshad
> 

I'm not an expert of Linux Virtual Server (on contrary, my experience
with LVS is too limited to readings), but I know that LVS could do the
same thing -- and in a more powerful way, what netfilter does by itself.

In another words, LVS may also uses the netfilter technology below and
more. For instance, there is a section in LVS docs about LVS-NAT
(section 13). The great thing of LVS is that I will have a solution with
high-availability on the layer of service which is higher than NAT (only
routing) or VRRP (only check IP).

Pedro Salazar.
-- 
PS
pedro-b-salazar@ptinovacao.pt
PGP:0E129E31D803BC61