From: John Lister <john.lister@kickstone.com>
To: Evan Pierce <evan@pierce.co.za>
Cc: netfilter@vger.kernel.org
Subject: Re: Load Balancing issue
Date: Mon, 11 Apr 2011 13:31:35 +0100 [thread overview]
Message-ID: <4DA2F4A7.4030902@kickstone.com> (raw)
In-Reply-To: <4DA2CB4D.2070402@pierce.co.za>
Have you saved/restored the marks in the conntrack table? Otherwise they
will be lost for all subsequent packets.. eg:
-j CONNMARK --save-mark
John
On 11/04/2011 10:35, Evan Pierce wrote:
> I have read/googled/looked at but somewhere I feel I have a missed
> understanding.
>
> I have a firewall with three interfaces.
>
> interfaces are as follows:
>
> eth0: 192.168.11.11/255.255.255.0 - internal network
> eth3: 197.213.0.42/255.255.255.248 - external 512kb line
> eth4: 192.168.1.2/255.255.255.0 - external 4mb line behind adsl nat router
>
> All I want to do is to get all port 80 and port 443 traffic to go up the
> 4mb adsl line and the rest can go up the 512kb line.
>
> I have the rules as follows:
>
> ip route add table 4 default via 192.168.1.1
> iptables -t mangle -A PREROUTING -p tcp --dport 80 -s 192.168.11.0/24 -j
> MARK --set-mark 4
> iptables -t mangle -A PREROUTING -p tcp --dport 443 -s 192.168.11.0/24
> -j MARK --set-mark 4
> iptables -t nat -A POSTROUTING -o eth4 -j SNAT --to-source 192.168.1.2
> iptables -t nat -A POSTROUTING -o eth3 -j SNAT --to-source 197.213.0.42
> ip rule add fwmark 4 table 4
> ip route flush cache
>
>
> I can see the packets get marked via
>
> Chain PREROUTING (policy ACCEPT 6559 packets, 1226K bytes)
> pkts bytes target prot opt in out source
> destination
> 147 8744 MARK tcp -- any any 192.168.11.0/24
> anywhere tcp dpt:www MARK xset 0x4/0xffffffff
> 29 2191 MARK tcp -- any any 192.168.11.0/24
> anywhere tcp dpt:https MARK xset 0x4/0xffffffff
>
>
> A tcpdump shows the traffic successfully leaving port on the 4mb line
>
> root@firewall:~# tcpdump -i eth4 host www.iol.co.za
> tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
> listening on eth4, link-type EN10MB (Ethernet), capture size 96 bytes
> 11:02:47.832883 IP 192.168.1.2.48529> 196.38.8.254.www: Flags [S], seq
> 357065168, win 5840, options [mss 1460,sackOK,TS val 2871645712 ecr
> 0,nop,wscale 7], length 0
> 11:02:47.846045 IP 196.38.8.254.www> 192.168.1.2.48529: Flags [S.], seq
> 190124381, ack 357065169, win 5792, options [mss 1452,sackOK,TS val
> 1415257200 ecr 2871645712,nop,wscale 7], length 0
> 11:02:50.833491 IP 192.168.1.2.48529> 196.38.8.254.www: Flags [S], seq
> 357065168, win 5840, options [mss 1460,sackOK,TS val 2871648712 ecr
> 0,nop,wscale 7], length 0
> 11:02:50.846079 IP 196.38.8.254.www> 192.168.1.2.48529: Flags [S.], seq
> 190124381, ack 357065169, win 5792, options [mss 1452,sackOK,TS val
> 1415260200 ecr 2871645712,nop,wscale 7], length 0
> 11:02:52.015010 IP 196.38.8.254.www> 192.168.1.2.48529: Flags [S.], seq
> 190124381, ack 357065169, win 5792, options [mss 1452,sackOK,TS val
> 1415261370 ecr 2871645712,nop,wscale 7], length 0
> 11:02:56.834029 IP 192.168.1.2.48529> 196.38.8.254.www: Flags [S], seq
> 357065168, win 5840, options [mss 1460,sackOK,TS val 2871654712 ecr
> 0,nop,wscale 7], length 0
> 11:02:56.846155 IP 196.38.8.254.www> 192.168.1.2.48529: Flags [S.], seq
> 190124381, ack 357065169, win 5792, options [mss 1452,sackOK,TS val
> 1415266201 ecr 2871645712,nop,wscale 7], length 0
> 11:02:58.015083 IP 196.38.8.254.www> 192.168.1.2.48529: Flags [S.], seq
> 190124381, ack 357065169, win 5792, options [mss 1452,sackOK,TS val
> 1415267370 ecr 2871645712,nop,wscale 7], length 0
> 11:03:08.834078 IP 192.168.1.2.48529> 196.38.8.254.www: Flags [S], seq
> 357065168, win 5840, options [mss 1460,sackOK,TS val 2871666712 ecr
> 0,nop,wscale 7], length 0
> 11:03:08.846185 IP 196.38.8.254.www> 192.168.1.2.48529: Flags [S.], seq
> 190124381, ack 357065169, win 5792, options [mss 1452,sackOK,TS val
> 1415278200 ecr 2871645712,nop,wscale 7], length 0
> 11:03:10.015725 IP 196.38.8.254.www> 192.168.1.2.48529: Flags [S.], seq
> 190124381, ack 357065169, win 5792, options [mss 1452,sackOK,TS val
> 1415279370 ecr 2871645712,nop,wscale 7], length 0
> 11:03:32.834205 IP 192.168.1.2.48529> 196.38.8.254.www: Flags [S], seq
> 357065168, win 5840, options [mss 1460,sackOK,TS val 2871690712 ecr
> 0,nop,wscale 7], length 0
>
>
> and seemingly returning however the traffic is never passed through the
> firewall back to the source machine as shown by a simultaneous tcpdump
> of the internal network:
>
> root@firewall:~# tcpdump -i eth0 host www.iol.co.za
> tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
> listening on eth0, link-type EN10MB (Ethernet), capture size 96 bytes
> 11:08:44.378983 IP 192.168.11.11.53455> 196.38.8.254.www: Flags [S],
> seq 727724538, win 5840, options [mss 1460,sackOK,TS val 2872002247 ecr
> 0,nop,wscale 7], length 0
> 11:08:44.508787 IP 192.168.11.11.45208> 196.38.8.254.www: Flags [S],
> seq 687092256, win 5840, options [mss 1460,sackOK,TS val 2872002377 ecr
> 0,nop,wscale 7], length 0
> 11:08:47.379042 IP 192.168.11.11.53455> 196.38.8.254.www: Flags [S],
> seq 727724538, win 5840, options [mss 1460,sackOK,TS val 2872005247 ecr
> 0,nop,wscale 7], length 0
> 11:08:53.379575 IP 192.168.11.11.53455> 196.38.8.254.www: Flags [S],
> seq 727724538, win 5840, options [mss 1460,sackOK,TS val 2872011247 ecr
> 0,nop,wscale 7], length 0
> 11:08:59.460167 IP 192.168.11.11.53790> 196.38.8.254.www: Flags [S],
> seq 742925249, win 5840, options [mss 1460,sackOK,TS val 2872017328 ecr
> 0,nop,wscale 7], length 0
>
> So something must be wrong in my firewall rules here is a dump of
> iptables -L -v
>
> # Generated by iptables-save v1.4.4 on Mon Apr 11 11:28:27 2011
> *mangle
> :PREROUTING ACCEPT [42735:22614277]
> :INPUT ACCEPT [9112:1223454]
> :FORWARD ACCEPT [32568:21304980]
> :OUTPUT ACCEPT [6367:1574752]
> :POSTROUTING ACCEPT [39211:22923589]
> -A PREROUTING -s 192.168.11.0/24 -p tcp -m tcp --dport 80 -j MARK
> --set-xmark 0x4/0xffffffff
> -A PREROUTING -s 192.168.11.0/24 -p tcp -m tcp --dport 443 -j MARK
> --set-xmark 0x4/0xffffffff
> COMMIT
> # Completed on Mon Apr 11 11:28:27 2011
> # Generated by iptables-save v1.4.4 on Mon Apr 11 11:28:27 2011
> *nat
> :PREROUTING ACCEPT [1419:138894]
> :POSTROUTING ACCEPT [124:10161]
> :OUTPUT ACCEPT [279:27787]
> -A PREROUTING -d 197.213.0.43/32 -p tcp -m tcp --dport 80 -j DNAT
> --to-destination 192.168.11.17:80
> -A PREROUTING -d 197.213.0.43/32 -p tcp -m tcp --dport 443 -j DNAT
> --to-destination 192.168.11.17:443
> -A PREROUTING -d 197.213.0.44/32 -p tcp -m tcp --dport 80 -j DNAT
> --to-destination 192.168.11.19:80
> -A PREROUTING -d 197.213.0.44/32 -p tcp -m tcp --dport 443 -j DNAT
> --to-destination 192.168.11.19:443
> -A PREROUTING -d 197.213.0.42/32 -p tcp -m tcp --dport 143 -j DNAT
> --to-destination 192.168.11.11:143
> -A PREROUTING -d 197.213.0.45/32 -p tcp -m tcp --dport 80 -j DNAT
> --to-destination 192.168.11.11:80
> -A PREROUTING -d 197.213.0.43/32 -p tcp -m tcp --dport 60200 -j DNAT
> --to-destination 192.168.11.14:60200
> -A PREROUTING -d 197.213.0.42/32 -p tcp -m tcp --dport 5900 -j DNAT
> --to-destination 192.168.11.61:5900
> -A PREROUTING -d 197.213.0.42/32 -p tcp -m tcp --dport 3389 -j DNAT
> --to-destination 192.168.11.19:3389
> -A PREROUTING -d 197.213.0.43/32 -p tcp -m tcp --dport 22 -j DNAT
> --to-destination 192.168.11.11:22
> -A PREROUTING -i eth3 -p tcp -m tcp --dport 904 -j DNAT --to-destination
> 192.168.11.11:904
> -A PREROUTING -i eth3 -p udp -m udp --dport 904 -j DNAT --to-destination
> 192.168.11.11:904
> -A PREROUTING -i eth3 -p tcp -m tcp --dport 5900 -j DNAT
> --to-destination 192.168.11.17:5900
> -A PREROUTING -i eth3 -p udp -m udp --dport 1194 -j DNAT
> --to-destination 192.168.11.11:1194
> -A PREROUTING -i eth3 -p tcp -m tcp --dport 80 -j DNAT --to-destination
> 192.168.11.11:80
> -A PREROUTING -i eth3 -p tcp -m tcp --dport 443 -j DNAT --to-destination
> 192.168.11.11:443
> -A POSTROUTING -o eth4 -j SNAT --to-source 192.168.1.2
> -A POSTROUTING -o eth3 -j SNAT --to-source 197.213.0.42
> COMMIT
> # Completed on Mon Apr 11 11:28:27 2011
> # Generated by iptables-save v1.4.4 on Mon Apr 11 11:28:27 2011
> *filter
> :INPUT ACCEPT [85:8370]
> :FORWARD ACCEPT [1:48]
> :OUTPUT ACCEPT [115:19331]
> -A INPUT -i eth3 -m state --state RELATED,ESTABLISHED -j ACCEPT
> -A INPUT -i eth4 -m state --state RELATED,ESTABLISHED -j ACCEPT
> -A INPUT -p tcp -m tcp --dport 22 -j ACCEPT
> -A INPUT -p tcp -m tcp --dport 222 -j ACCEPT
> -A INPUT -p udp -m udp --dport 1194 -j ACCEPT
> -A INPUT -p tcp -m tcp --dport 1194 -j ACCEPT
> -A INPUT -p tcp -m tcp --dport 80 -j ACCEPT
> -A INPUT -p tcp -m tcp --dport 443 -j ACCEPT
> -A INPUT -p tcp -m tcp --dport 21 -j ACCEPT
> -A INPUT -p tcp -m tcp --dport 1723 -j ACCEPT
> -A INPUT -i tun+ -j ACCEPT
> -A INPUT -i eth3 -j DROP
> -A INPUT -i eth4 -j DROP
> -A FORWARD -s 69.93.127.55/32 -j ACCEPT
> -A FORWARD -d 69.93.127.55/32 -j ACCEPT
> -A FORWARD -s 192.168.11.19/32 -j ACCEPT
> -A FORWARD -d 192.168.11.19/32 -j ACCEPT
> -A FORWARD -s 192.168.11.11/32 -j ACCEPT
> -A FORWARD -d 192.168.11.11/32 -j ACCEPT
> -A FORWARD -s 192.168.11.12/32 -j ACCEPT
> -A FORWARD -d 192.168.11.12/32 -j ACCEPT
> -A FORWARD -s 192.168.11.21/32 -j ACCEPT
> -A FORWARD -d 192.168.11.21/32 -j ACCEPT
> -A FORWARD -s 196.38.244.20/32 -j ACCEPT
> -A FORWARD -d 196.38.244.20/32 -j ACCEPT
> -A FORWARD -p tcp -m tcp --dport 995 -j DROP
> -A FORWARD -p tcp -m tcp --dport 465 -j DROP
> -A FORWARD -p tcp -m tcp --dport 587 -j DROP
> -A FORWARD -o eth3 -p udp -m udp --dport 137 -j DROP
> -A FORWARD -o eth3 -p udp -m udp --dport 138 -j DROP
> -A FORWARD -o eth3 -p udp -m udp --dport 139 -j DROP
> -A FORWARD -o eth3 -p udp -m udp --dport 445 -j DROP
> -A FORWARD -o eth3 -p tcp -m tcp --dport 137 -j DROP
> -A FORWARD -o eth3 -p tcp -m tcp --dport 138 -j DROP
> -A FORWARD -o eth3 -p tcp -m tcp --dport 139 -j DROP
> -A FORWARD -o eth3 -p udp -m udp --dport 445 -j DROP
> -A FORWARD -o eth4 -p udp -m udp --dport 137 -j DROP
> -A FORWARD -o eth4 -p udp -m udp --dport 138 -j DROP
> -A FORWARD -o eth4 -p udp -m udp --dport 139 -j DROP
> -A FORWARD -o eth4 -p udp -m udp --dport 445 -j DROP
> -A FORWARD -o eth4 -p tcp -m tcp --dport 137 -j DROP
> -A FORWARD -o eth4 -p tcp -m tcp --dport 138 -j DROP
> -A FORWARD -o eth4 -p tcp -m tcp --dport 139 -j DROP
> -A FORWARD -o eth4 -p udp -m udp --dport 445 -j DROP
> -A FORWARD -p tcp -m tcp --dport 110 -j ACCEPT
> -A FORWARD -i eth3 -o eth0 -m state --state RELATED,ESTABLISHED -j ACCEPT
> -A FORWARD -i eth4 -o eth0 -m state --state RELATED,ESTABLISHED -j ACCEPT
> -A FORWARD -i eth0 -o eth3 -m state --state RELATED,ESTABLISHED -j ACCEPT
> -A FORWARD -i eth0 -o eth4 -m state --state RELATED,ESTABLISHED -j ACCEPT
> -A FORWARD -i tun+ -o eth0 -m state --state RELATED,ESTABLISHED -j ACCEPT
> -A FORWARD -i tun+ -j ACCEPT
> -A FORWARD -s 10.9.0.0/16 -j ACCEPT
> -A FORWARD -s 10.8.0.0/16 -j ACCEPT
> -A FORWARD -d 10.9.0.0/16 -j ACCEPT
> -A FORWARD -d 10.8.0.0/16 -j ACCEPT
> -A FORWARD -d 192.168.11.19/32 -j ACCEPT
> -A FORWARD -s 192.168.11.19/32 -j ACCEPT
> -A FORWARD -s 196.11.134.22/32 -j ACCEPT
> -A FORWARD -d 196.11.134.22/32 -j ACCEPT
> -A FORWARD -s 192.168.11.11/32 -j ACCEPT
> -A FORWARD -d 192.168.11.11/32 -j ACCEPT
> -A FORWARD -d 109.74.204.69/32 -j ACCEPT
> -A FORWARD -s 192.168.11.19/32 -p tcp -m tcp --dport 80 -j ACCEPT
> -A FORWARD -s 192.168.11.150/32 -p tcp -m tcp --dport 80 -j ACCEPT
> -A FORWARD -s 192.168.11.61/32 -p tcp -m tcp --dport 80 -j ACCEPT
> -A FORWARD -s 192.168.11.11/32 -p tcp -m tcp --dport 80 -j ACCEPT
> -A FORWARD -p tcp -m tcp --dport 80 -j DROP
> COMMIT
> # Completed on Mon Apr 11 11:28:27 2011
>
>
> I have read the rules and reread and reread and I cannot find where-ever
> I am making this obvious mistake. OS is ubuntu 10.04
>
> Any ideas?
>
> thanks
> Evan Pierce
> --
> To unsubscribe from this list: send the line "unsubscribe netfilter" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
next prev parent reply other threads:[~2011-04-11 12:31 UTC|newest]
Thread overview: 5+ messages / expand[flat|nested] mbox.gz Atom feed top
2011-04-11 9:35 Load Balancing issue Evan Pierce
2011-04-11 12:31 ` John Lister [this message]
[not found] ` <4DA2FB80.4050306@pierce.co.za>
2011-04-11 15:37 ` John Lister
2011-04-11 16:23 ` Evan Pierce
2011-04-11 17:46 ` Andrew Beverley
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=4DA2F4A7.4030902@kickstone.com \
--to=john.lister@kickstone.com \
--cc=evan@pierce.co.za \
--cc=netfilter@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).