[LARTC] Multiple gateways setup and timeout connections

[LARTC] Multiple gateways setup and timeout connections

[Bernardo Silveira]

Hi,

I've setup a gateway using multiple default gateways and netfilter
MASQUERADE to load balance traffic between two DSL interfaces and one
dedicated link, and when I try to download something big, or when I'm
using MSN (both in clients under this gateway), sometimes, or most
times, after a while the connection timeouts. The connection doesn't
seem to change its gateway (verified using ip route list cache)

.

I've seem a message complaining about a similar problem but it didn't
have any solution. Maybe somebody have something new now.

Here goes the script I use:

#!/bin/bash
ip1="192.x.x.1"
gw1="192.x.x.254"
ip2="192.x.y.1"
gw2="192.x.y.254"
ip3="200.w.z.141"
gw3="200.w.z.129"

# Setup source IP routing rules
# ADSL1
ip rule add from $ip1 lookup 1
ip route add 192.168.8.0/24 via 192.168.8.1 table 1
ip route add 192.168.3.0/24 via 192.168.3.1 table 1
ip route add 0/0 via $gw1 table 1

# ADSL2
ip rule add from $ip2 lookup 2
ip route add 192.168.8.0/24 via 192.168.8.1 table 2
ip route add 192.168.3.0/24 via 192.168.3.1 table 2
ip route add 0/0 via $gw2 table 2

# ADSL3
ip rule add from $ip3 lookup 3
ip route add 192.168.8.0/24 via 192.168.8.1 table 3
ip route add 192.168.3.0/24 via 192.168.3.1 table 3
ip route add 0/0 via $gw3 table 3

# Setup load balancing
ip route add default equalize scope global \
  nexthop via $gw1 dev eth1 \
  nexthop via $gw2 dev eth2 \
  nexthop via $gw3 dev eth3

# Pass Internet traffic to internal network unmodified
iptables -t nat -A POSTROUTING -o eth0 -j ACCEPT

# Masquerading outbound connections from internal network
iptables -t nat -A POSTROUTING -o eth1 -j MASQUERADE
iptables -t nat -A POSTROUTING -o eth2 -j MASQUERADE
iptables -t nat -A POSTROUTING -o eth3 -j MASQUERADE

#iptables -t nat -A POSTROUTING -s 192.168.8.0/24 -j MASQUERADE
#iptables -t nat -A POSTROUTING -s 192.168.3.0/24 -j MASQUERADE

Here goes the route garbage collector configuration:
net.ipv4.neigh.eth3.gc_stale_time = 60
net.ipv4.neigh.eth2.gc_stale_time = 60
net.ipv4.neigh.eth1.gc_stale_time = 60
net.ipv4.neigh.eth0.gc_stale_time = 60
net.ipv4.neigh.lo.gc_stale_time = 60
net.ipv4.neigh.default.gc_thresh3 = 1024
net.ipv4.neigh.default.gc_thresh2 = 512
net.ipv4.neigh.default.gc_thresh1 = 128
net.ipv4.neigh.default.gc_interval = 30
net.ipv4.neigh.default.gc_stale_time = 60
net.ipv4.inet_peer_gc_maxtime = 120
net.ipv4.inet_peer_gc_mintime = 10
net.ipv4.route.gc_elasticity = 1
net.ipv4.route.gc_interval = 1
net.ipv4.route.gc_timeout = 0
net.ipv4.route.gc_min_interval = 0
net.ipv4.route.gc_thresh = 2048

The tcpdump traffic goes like this:

From the gateway:
19:08:37.404963 praia1.praiacamboriu.com.br.ftp-data >
141.porttal.com.br.1412: . 11025485:11026945(1460) ack 1 win 58400
(DF)
19:08:37.432250 141.porttal.com.br.1412 >
praia1.praiacamboriu.com.br.ftp-data: . ack 11028405 win 8760 (DF)
19:08:37.521619 praia1.praiacamboriu.com.br.ftp-data >
141.porttal.com.br.1412: . 11029865:11031325(1460) ack 1 win 58400
(DF)
19:08:37.527411 praia1.praiacamboriu.com.br.ftp-data >
141.porttal.com.br.1412: . 11028405:11029865(1460) ack 1 win 58400
(DF)
19:08:38.533879 192.168.200.1.1432 >
praia1.praiacamboriu.com.br.ftp-data: . ack 4381 win 8760 (DF)
19:08:46.182282 192.168.200.1.1430 >
praia1.praiacamboriu.com.br.ftp-data: . ack 4615497 win 0 (DF)
19:08:51.182479 192.168.200.1.1430 >
praia1.praiacamboriu.com.br.ftp-data: . ack 4615497 win 0 (DF)
19:08:53.664901 praia1.praiacamboriu.com.br.ftp-data >
141.porttal.com.br.1412: . 11028405:11029865(1460) ack 1 win 58400
(DF)
19:08:56.185343 192.168.200.1.1430 >
praia1.praiacamboriu.com.br.ftp-data: . ack 4615497 win 0 (DF)
19:09:01.507109 192.168.200.1.1430 >
praia1.praiacamboriu.com.br.ftp-data: . ack 4615497 win 0 (DF)
19:09:16.253324 192.168.200.1.1432 >
praia1.praiacamboriu.com.br.ftp-data: . ack 24821 win 0 (DF)
19:09:23.789876 141.porttal.com.br.1412 >
praia1.praiacamboriu.com.br.ftp-data: R 19400787:19400787(0) win 0
(DF)
19:09:23.833381 192.168.200.1.1430 >
praia1.praiacamboriu.com.br.ftp-data: R 20687684:20687684(0) win 0
(DF)
19:09:23.834135 192.168.200.1.1432 >
praia1.praiacamboriu.com.br.ftp-data: R 20857536:20857536(0) win 0
(DF)
19:09:26.254170 192.168.200.1.1432 >
praia1.praiacamboriu.com.br.ftp-data: R 20857536:20857536(0) win 0

And from the ftp which I was downloading:
15:37:44.490583 praia1.praiacamboriu.com.br.ftp-data >
201-002-197-003.jvece7001.e.brasiltelecom.net.br.11254: . ack 1 win
57920 <nop,nop,timestamp 88503239 0> (DF)
15:37:44.493251 praia1.praiacamboriu.com.br.ftp-data >
201-002-197-003.jvece7001.e.brasiltelecom.net.br.11254: P 1:1018(1017)
ack 1 win 57920 <nop,nop,timestamp 88503239 0> (DF)
15:37:44.670579 201-002-197-003.jvece7001.e.brasiltelecom.net.br.11254
> praia1.praiacamboriu.com.br.ftp-data: . ack 1018 win 64518
<nop,nop,timestamp 42852 88503239> (DF)
15:37:44.670668 praia1.praiacamboriu.com.br.ftp-data >
201-002-197-003.jvece7001.e.brasiltelecom.net.br.11254: FP
1018:2389(1371) ack 1 win 57920 <nop,nop,timestamp 88503257 42852>
(DF)
15:37:44.738904 201-002-197-003.jvece7001.e.brasiltelecom.net.br.11254
> praia1.praiacamboriu.com.br.ftp-data: . ack 2390 win 65535
<nop,nop,timestamp 42852 88503257> (DF)
15:37:44.744887 201-002-197-003.jvece7001.e.brasiltelecom.net.br.11254
> praia1.praiacamboriu.com.br.ftp-data: F 1:1(0) ack 2390 win 65535
<nop,nop,timestamp 42852 88503257> (DF)
15:37:44.744918 praia1.praiacamboriu.com.br.ftp-data >
201-002-197-003.jvece7001.e.brasiltelecom.net.br.11254: . ack 2 win
57920 <nop,nop,timestamp 88503264 42852> (DF)

Thanks,
Bernardo Silveira
Via IP Soluções para Internet Ltda.
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] Multiple gateways setup and timeout connections

[Bernardo Silveira]

On Mon, 7 Feb 2005 10:58:30 +0100, Uwe Kamper <uwekamper@wichte.de> wrote:
> > Hi,
> >
> > I've setup a gateway using multiple default gateways and netfilter
> > MASQUERADE to load balance traffic between two DSL interfaces and one
> > dedicated link, and when I try to download something big, or when I'm
> > using MSN (both in clients under this gateway), sometimes, or most
> > times, after a while the connection timeouts. The connection doesn't
> > seem to change its gateway (verified using ip route list cache)
> 
> Hello,
> 
> did you apply the patch mentioned under section 4.2.2 of the LARTC.org-
> HOWTO ( http://lartc.org/howto/lartc.rpdb.multiple-links.html#AEN298 or
> http://www.ssi.bg/~ja/#routes ) to your Linux kernel sources?
> 
> 
> Uwe Kamper

Yes,
I've tested with and without them, in kernels 2.4.22 and 2.6.10.

Bernardo Silveira
Via IP Soluções para Internet Ltda
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] Multiple gateways setup and timeout connections

[Nguyen Dinh Nam]

You have to CONNMARK them so outgoing connection don't get re-routed 
each time routing cache expires. I've written about prevent the expiring 
of connections in 
http://selab.edu.ms/twiki/bin/view/Networking/MultihomedLinuxNetworking

Bernardo Silveira wrote:

>Yes,
>I've tested with and without them, in kernels 2.4.22 and 2.6.10.
>  
>
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/