[LARTC] Load balancing using connmark

From: Francis Brosnan Blazquez <francis@aspl.es>
To: lartc@vger.kernel.org
Subject: [LARTC] Load balancing using connmark
Date: Wed, 09 May 2007 15:00:06 +0000	[thread overview]
Message-ID: <1178722806.7492.55.camel@vulcan.aspl> (raw)

Hi,

I've been implementing a load balancing solution using CONNMARK, based
on solution described by Luciano Ruete at [1]. Gracias por el post y por
apuntar en la direcciÛn correcta Luciano!

Once implemented, I've found that due to some reason packets aren't
properly marked (or improperly remarked) and sent out using the wrong
interface. 

My topo setup is:

[82.123.136.74]: eth1 : mark:0x1 --\
                                    +--[FW BOX] -- eth0: 192.168.0.53
[217.146.74.82]: eth2 : mark:0x2 --/

Using conntrack tool, shows that after a while, it starts to appear
packets marked with 0x2 or 0x1 not comming from the proper source IP.

>> conntrack -L | grep mark=2 | grep '82.123.136.74'; conntrack -L |
grep mark=1 | grep '217.146.74.82'

tcp      6 425543 ESTABLISHED src\x192.168.0.178 dstÇ.216.53.249
sport\x1552 dportD3 packetsÅ8 bytesì471 srcÇ.216.53.249
dstÇ.123.136.74 sportD3 dport\x1552 packetsá5 bytesÉ909 [ASSURED]
mark=2 use=1
tcp      6 428681 ESTABLISHED src\x192.168.0.177 dstâ.139.122.12
sport#61 dportD3 packets\x122 bytes)381 srcâ.139.122.12
dstÇ.123.136.74 sportD3 dport#61 packets\x139 bytes\x14120 [ASSURED]
mark=2 use=1

This is quite odd since solution proposed at [1] looks good. I'll cite
it here for clarity (suppose I already have all ip rule stuff
installed):

iptables -t mangle -A POSTROUTING -m mark  --mark ! 0 -j ACCEPT 
iptables -t mangle -A POSTROUTING -o eth1 -j MARK --set-mark 0x1
iptables -t mangle -A POSTROUTING -o eth2 -j MARK --set-mark 0x2
iptables -t mangle -A POSTROUTING -j CONNMARK --save-mark
iptables -t mangle -A PREROUTING -j CONNMARK --restore-mark

After giving a try during several days, I've found that another firewall
solution, shorewall [2], implements built-in load balacing for free by
using the following set of instructions:

iptables -t mangle -A PREROUTING -m connmark ! --mark 0/0xFF -j CONNMARK
--restore-mark --mask 0xFF

iptables -t mangle -A OUTPUT -m connmark ! --mark 0/0xFF -j CONNMARK
--restore-mark --mask 0xFF

iptables -t mangle -N routemark
iptables -t mangle -A PREROUTING -i eth1 -m mark --mark 0/0xFF -j
routemark

iptables -t mangle -A routemark -i eth1 -j MARK --set-mark 1
iptables -t mangle -A PREROUTING -i eth2 -m mark --mark 0/0xFF -j
routemark

iptables -t mangle -A routemark -i eth2 -j MARK --set-mark 2
iptables -t mangle -A routemark -m mark ! --mark 0/0xFF -j CONNMARK
--save-mark --mask 0xFF

After a bit of testing with the second solution, it seems to behave
better, doing all marking job at the PREROUTING and OUTPUT.

Did anybody find that some packages doesn't get properly routed
according to the mark with the first solution? What you do think about
the second solution?

Cheers!

[1] http://mailman.ds9a.nl/pipermail/lartc/2006q2/018964.html
[2] http://www.shorewall.net

-- 
Francis Brosnan Blazquez <francis@aspl.es>
Advanced Software Production Line, S.L.

_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc