Trying to achieve fail over to multiple DNATed destinations

Trying to achieve fail over to multiple DNATed destinations

[Mark Ruth]

Hello Everyone

I have tried to search alot about my situation over the Internet and
the mailing list archive of netfilter but could not exactly find an
answer. Please accept my apologies if this has been answered before,
in that case please point me to the right direction.

The issue at hand is as follows,

I am doing DNATing on my Linux box (Centos 6 32 bit) and actually
forwarding any connection at a specific local ip on my box to specific
target IP. Its a Kind of proxy behaviour we are studying.

This is an example rule,

-A PREROUTING -d 10.10.10.20 -i eth0 -p tcp -m tcp --dport 80 -j DNAT
--to-destination 201.23.11.22
-A PREROUTING -d 10.10.10.20 -i eth0 -p tcp -m tcp --dport 443 -j DNAT
--to-destination 201.23.11.22


where 10.10.10.20 is an ip alias on my linux box i.e performing nating and,
201.23.11.22 is a destination web server, lets say www.example.com
(IPs are not real off course)

So the above two rules actually NAT any connection attempt from my
network users to my Linux box's ip address 10.10.10.20 and port 80 and
443 and forward them to destination IP of www.example.com (same ports
off course)

The problem is that the target webserver (i.e. www.example.com ) is
loadbalanced between like 10 servers having 10 public IPs (which are
NOT in series), but since IPtables can not accept domain names
(correct me if i m wrong) I have to select 1 ip address (i.e.
201.23.11.22 ) from the pool of 10 available, for my iptables rule.

For some reason if the target webserver (www.example.com) shutdown 1
of its server and lets say they shutdown the exact server IP which i
was using in my rule, my users can not connect to www.example.com any
more untill i debug this and change the destination ip address in my
iptables rules.

Now i want to handle this situation with some sort of failover rule
that can forward users to next ip in list if 1st DOES NOT RESPOND for
some reason. Another thing that I want to achieve is to distribute
load between target's multiple servers rather then just 1 IP address,
something as follows,

-A PREROUTING -d 10.10.10.20 -i eth0 -p tcp -m tcp --dport 80 -j DNAT
--to-destination 201.23.11.22 --to-destination 201.23.12.21
-to-destination 201.23.11.74 .....

where,

201.23.11.22
201.23.12.21
201.23.11.74

are all pointing to www.example.com, note that the destination IPs are
not in series, therefore IP range can not be used.

I read that multiple --to-destinations were supported in previous
versions of iptables but not anymore. Hence to summarize, I want to
achieve some sort of failover with loadbalancing in above mention
scenario. Failover is more important.

I will highly appreciate if someone can help me out resolving this issue.

Regards,
Mark

Re: Trying to achieve fail over to multiple DNATed destinations

[Jan Engelhardt]

On Tuesday 2011-08-09 00:36, Mark Ruth wrote:
>[DNAT]
>
>The problem is that the target webserver (i.e. www.example.com ) is
>loadbalanced between like 10 servers having 10 public IPs (which are
>NOT in series), [...]
>my users can not connect to www.example.com any
>more untill i debug this and change the destination ip address in my
>iptables rules.
>[...]
>I read that multiple --to-destinations were supported in previous
>versions of iptables but not anymore. Hence to summarize, I want to
>achieve some sort of failover with loadbalancing in above mention
>scenario. Failover is more important.

NAT is not a substitute for a proper layer-7 proxy and/or loadbalancer.


>Now i want to handle this situation with some sort of failover rule
>that can forward users to next ip in list if 1st DOES NOT RESPOND for
>some reason.

Even without NAT, a user can't do anything if his browser resolved
a hostname to an address out of a round-robing pool and that
one server happens to be down.

Re: Trying to achieve fail over to multiple DNATed destinations

[Mark Ruth]

Thanks for your response Jan, I was just reading your PDF of
Netfilter_modules :)

Yeah I do understand that Its not a replacement but I was wondering if
its possible in someway like using any existing extension/modules to
iptables or something. Something in iptables that can track that the
request which was forwarded via X rule did not return any response and
timeout to another rule or some other strategy you may think of.

Redarding browser resolving to another IP, for this lets assume that
users on my network are accessing www.example.com via IP address i.e.
10.10.10.20 (internal one) and not via domain name. They are accessing
it from code and not browser where they need to specify IP and not
domain name etc.

Regards,
Mark

On Tue, Aug 9, 2011 at 3:46 AM, Jan Engelhardt <jengelh@medozas.de> wrote:
>
> On Tuesday 2011-08-09 00:36, Mark Ruth wrote:
>>[DNAT]
>>
>>The problem is that the target webserver (i.e. www.example.com ) is
>>loadbalanced between like 10 servers having 10 public IPs (which are
>>NOT in series), [...]
>>my users can not connect to www.example.com any
>>more untill i debug this and change the destination ip address in my
>>iptables rules.
>>[...]
>>I read that multiple --to-destinations were supported in previous
>>versions of iptables but not anymore. Hence to summarize, I want to
>>achieve some sort of failover with loadbalancing in above mention
>>scenario. Failover is more important.
>
> NAT is not a substitute for a proper layer-7 proxy and/or loadbalancer.
>
>
>>Now i want to handle this situation with some sort of failover rule
>>that can forward users to next ip in list if 1st DOES NOT RESPOND for
>>some reason.
>
> Even without NAT, a user can't do anything if his browser resolved
> a hostname to an address out of a round-robing pool and that
> one server happens to be down.
>

Re: Trying to achieve fail over to multiple DNATed destinations

[Grant Taylor]

On 08/08/11 17:36, Mark Ruth wrote:
> The problem is that the target webserver (i.e. www.example.com ) is
> loadbalanced between like 10 servers having 10 public IPs (which are
> NOT in series), but since IPtables can not accept domain names
> (correct me if i m wrong) I have to select 1 ip address (i.e.
> 201.23.11.22 ) from the pool of 10 available, for my iptables rule.

It is my (mis)understanding that the iptables command will accept host 
names and resolve them to an IP which is inserted in to the kernel.  So, 
you are close.

> Now i want to handle this situation with some sort of failover rule
> that can forward users to next ip in list if 1st DOES NOT RESPOND for
> some reason. Another thing that I want to achieve is to distribute
> load between target's multiple servers rather then just 1 IP address,
> something as follows,
...
> I will highly appreciate if someone can help me out resolving this issue.

Have you done any looking at the Linux Virtual Server project 
(http://www.linuxvirtualserver.org/)?

I think you are using it a little backwards of what is normally done 
(in-to-out vs out-to-in) but I think it would still work.

I.e. you create a LVS on your CentOS box that uses the 10 real IPs out 
on the internet as the real servers.  LVS will then distribute the load 
between them and monitor their response times and know when to take one 
out of the pool based on lack of many different criteria (response time 
or lack of response).

I also think that LVS has an added advantage for you (unless memory is 
failing me, again) is that LVS runs in kernel space (much like IPTables) 
with a few user space commands to configure it, where as an application 
layer proxy would actually run in user space.

I've not messed with LVS in a long time, but from memory, I think it 
could be made to do what you are wanting to do.  Further, LVS was 
relatively trivial to install and configure.



Grant. . . .

Re: Trying to achieve fail over to multiple DNATed destinations

[Tyler J. Wagner]

On 2011-08-09 20:57, Grant Taylor wrote:
> On 08/08/11 17:36, Mark Ruth wrote:
>> The problem is that the target webserver (i.e. www.example.com ) is
>> loadbalanced between like 10 servers having 10 public IPs (which are
>> NOT in series), but since IPtables can not accept domain names
>> (correct me if i m wrong) I have to select 1 ip address (i.e.
>> 201.23.11.22 ) from the pool of 10 available, for my iptables rule.
> 
> It is my (mis)understanding that the iptables command will accept host
> names and resolve them to an IP which is inserted in to the kernel.  So,
> you are close.

Yes, it will. However, this will make loading your firewall rules block
on DNS. If you are loading the firewall as the interface comes up, that
may be a problem for you. For critical systems, especially routers, I
use only IP addresses in the firewall script.

Regards,
Tyler

-- 
"No one can terrorize a whole nation, unless we are all his accomplices."
   -- Edward R. Murrow