[LARTC] custom routing (two gateways)

[LARTC] custom routing (two gateways)

[Christian Parpart]

[-- Attachment #1.1: Type: text/plain, Size: 1834 bytes --]

Hi all,

i'm having a somewhat stupid problem I can't get rid of.

we've a server that accepts incoming world connections 
from a load balancer (10.10.10.4) to port 80,
and we still want to serve incoming ssh/http from the firewall (10.10.10.1) 
routed to this host (10.10.10.90) and their reply packets of cause shall be 
send out through the firewall.
unfortunately, both hosts (the load balancer (LB) and the firewall(FW)) are on 
the same subnet (10.10.10.0/24) and though on the same interface (eth0), but 
I need to find a solution.

So, packets send from the LB shall get their answer through the LB as nexthop 
of couse, as well as incoming packets from the FW shall have response packets 
send out to the FW as nexthop, too.

But how to realise this?

server:           10.10.10.90 (this is the problem host)
firewall(FW):     10.10.10.1 (we receive (mostly) ssh/https conns from it)
loadbalancer(LB): 10.10.10.4 (we receive http conns from it)

FW and LB are accepting/forwarding and routing connections from the world to 
our server.

server ~ # ip route list
192.168.0.0/24 dev eth1  proto kernel  scope link  src 192.168.0.90
10.10.10.0/24 dev eth0  proto kernel  scope link  src 10.10.10.90
127.0.0.0/8 dev lo  scope link
default via 10.10.10.1 dev eth0

You see, default traffic is routed through the firewall as the default 
gateway... but now, we want to have outgoing traffic caused from incoming 
packets from the load balancer to be routed back through the loadbalancer 
itself.

I tried here several approaches, like adding custom routing tables, and 
modifying the tables (including main) either I got no answers routed to the 
FW or no traffic got routed to the LB.

Can you please give me a hint on how to find the right way?

Thanks in advance,
Christian Parpart.

[-- Attachment #1.2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

[-- Attachment #2: Type: text/plain, Size: 143 bytes --]

_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

Re: [LARTC] custom routing (two gateways)

[Grant Taylor]

On 07/09/07 13:07, Christian Parpart wrote:
> I tried here several approaches, like adding custom routing tables, 
> and modifying the tables (including main) either I got no answers 
> routed to the FW or no traffic got routed to the LB.

I think you were on the right path.

I would set up a custom routing table for traffic associated with the 
load balancer to use.  This load balancer routing table would use the 
load balancer as the default gateway.

You would then use ip rule(s) to determine which traffic would deviate 
from the normal default routing tables and use the load balancer routing 
table.  This could probably be done based on source port on the web 
server, or based on connection / packet marks in IPTables.  However you 
do it, you will probably need an additional routing table.

Keep going the direction you were, or perhaps post some of what you did 
try and let us take a look at it to see if you were close.



Grant. . . .
_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

Re: [LARTC] custom routing (two gateways)

[Christian Parpart]

[-- Attachment #1.1: Type: text/plain, Size: 3347 bytes --]

On Monday 09 July 2007 21:01:26 Grant Taylor wrote:
> On 07/09/07 13:07, Christian Parpart wrote:
> > I tried here several approaches, like adding custom routing tables,
> > and modifying the tables (including main) either I got no answers
> > routed to the FW or no traffic got routed to the LB.
>
> I think you were on the right path.
>
> I would set up a custom routing table for traffic associated with the
> load balancer to use.  This load balancer routing table would use the
> load balancer as the default gateway.
>
> You would then use ip rule(s) to determine which traffic would deviate
> from the normal default routing tables and use the load balancer routing
> table.  This could probably be done based on source port on the web
> server, or based on connection / packet marks in IPTables.  However you
> do it, you will probably need an additional routing table.
>
> Keep going the direction you were, or perhaps post some of what you did
> try and let us take a look at it to see if you were close.

I finally found a way, and your hint (select by server port number) finally 
helped me to get rid of it :)

The following is the script to be executed at bootup to setup the additional 
routing table.

 1 #! /bin/bash
 2 # sets up additional routing table for load balancer traffic on a node
 3
 4 
# --------------------------------------------------------------------------------
 5 LB_IP=10.10.10.4            # load balancer IP
 6 LB_IF=eth0                  # ethernet interface the load balancer is 
talking from/to
 7
 8 rt_table_name=loadbalancer  # LB routing table name
 9 rt_table_num=200            # LB routing table ID
10
11 fwmark=1                    # FW mark to use for LB traffic
12
13 service_port=8000           # HTTP port for lighttpd on local mashine that
14                             # serves for the load balancer
15
16 
# --------------------------------------------------------------------------------
17
18 # just ensure that we have a routing table called loadbalancer
19 if ! grep -q ${rt_table_name} /etc/iproute2/rt_tables; then
20     echo "${rt_table_num} ${rt_table_name}" >> /etc/iproute2/rt_tables
21 fi
22
23 # add a default route for communication from LB<->this_host
24 ip route flush table ${rt_table_name}
25 ip route add default via ${LB_IP} table ${rt_table_name} dev ${LB_IF}
26
27 # add a selector rule for which packets we want to use the LB routing table
28 ip rule add fwmark ${fwmark} table ${rt_table_name}
29
30 # finally lets mark all packets that shall be send out to the LB
31 iptables -t mangle -A OUTPUT -p tcp --sport ${service_port} -j 
MARK --set-mark ${fwmark}
32 # --(doesn't work? why?)-- iptables -t mangle -A INPUT -p tcp --dport 
${service_port} -j CONNMARK --set-mark ${fwmark}


However, you might see, that I first tried to fwmark all packets by connection 
matching, 
using CONNMARK. so that I only need to select all incoming traffic that came 
from the load balancer 
as the previous hop and with our service port 8000, to let mark iptables 
itself all further connection
related packets automatically. but this didn't work out, unfortunately, and I 
am using the OUTPUT table
to match the packets.
What is the better approach anyway?

Thanks for your help,
Christian Parpart.

[-- Attachment #1.2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

[-- Attachment #2: Type: text/plain, Size: 143 bytes --]

_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc