From: Mike <ipso@snappymail.ca>
To: netfilter@vger.kernel.org
Subject: Re: v2.6.16 to v2.6.38 breaks routing? - Simplified
Date: Sun, 11 Sep 2011 21:13:21 -0700 [thread overview]
Message-ID: <4E6D86E1.60009@snappymail.ca> (raw)
In-Reply-To: <4E6D3F01.4060807@snappymail.ca>
I hate replying to my own email, but after discussions with a
knowledgeable person in the Linux advanced routing and traffic control
IRC channel, I simplified things down to just five iptable rules and
simplified the routing tables, unfortunately the result is still exactly
the same.
Here is hopefully all the necessary information including tcpdump output
with one route being successful and another failing:
root@server:/etc# ip route show table main | sort
10.8.0.0/24 dev tun0 proto kernel scope link src 10.8.0.2 metric 10
174.4.4.0/22 dev eth0 proto kernel scope link src 174.4.4.74 metric 10
192.168.1.0/24 dev eth1 proto kernel scope link src 192.168.1.1
metric 10
50.92.224.0/19 dev eth2 proto kernel scope link src 50.92.247.211
metric 10
root@server:/etc# ip route show table ISP1 | sort
default via 174.4.4.1 dev eth0
root@server:/etc# ip route show table ISP2 | sort
default via 50.92.224.1 dev eth2
root@server:/etc# ip route show table VPN1 | sort
default via 10.8.0.2 dev tun0
root@server:/etc# iptables -L -n -v
Chain INPUT (policy ACCEPT 1327 packets, 110K bytes)
pkts bytes target prot opt in out source
destination
Chain FORWARD (policy ACCEPT 4688 packets, 1071K bytes)
pkts bytes target prot opt in out source
destination
Chain OUTPUT (policy ACCEPT 766 packets, 82125 bytes)
pkts bytes target prot opt in out source
destination
root@server:/etc# iptables -L -n -v -t nat
Chain PREROUTING (policy ACCEPT 241 packets, 20425 bytes)
pkts bytes target prot opt in out source
destination
Chain INPUT (policy ACCEPT 37 packets, 6564 bytes)
pkts bytes target prot opt in out source
destination
Chain OUTPUT (policy ACCEPT 6 packets, 456 bytes)
pkts bytes target prot opt in out source
destination
Chain POSTROUTING (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source
destination
144 9466 SNAT all -- * eth0 0.0.0.0/0
0.0.0.0/0 to:174.4.4.74
66 4851 SNAT all -- * eth2 0.0.0.0/0
0.0.0.0/0 to:50.92.247.211
0 0 SNAT all -- * tun0 0.0.0.0/0
0.0.0.0/0 to:10.8.0.2
root@server:/etc# iptables -L -n -v -t mangle
Chain PREROUTING (policy ACCEPT 6143 packets, 1200K bytes)
pkts bytes target prot opt in out source
destination
111 8218 MARK all -- eth1 * 192.168.1.236
0.0.0.0/0 MARK set 0x2
2 168 MARK all -- eth1 * 192.168.1.246
0.0.0.0/0 MARK set 0x1
Chain INPUT (policy ACCEPT 1355 packets, 113K bytes)
pkts bytes target prot opt in out source
destination
Chain FORWARD (policy ACCEPT 4770 packets, 1086K bytes)
pkts bytes target prot opt in out source
destination
Chain OUTPUT (policy ACCEPT 787 packets, 84473 bytes)
pkts bytes target prot opt in out source
destination
Chain POSTROUTING (policy ACCEPT 5557 packets, 1170K bytes)
pkts bytes target prot opt in out source
destination
root@server:/etc# iptables -S
-P INPUT ACCEPT
-P FORWARD ACCEPT
-P OUTPUT ACCEPT
root@server:/etc# iptables -S -t nat
-P PREROUTING ACCEPT
-P INPUT ACCEPT
-P OUTPUT ACCEPT
-P POSTROUTING ACCEPT
-A POSTROUTING -o eth0 -j SNAT --to-source 174.4.4.74
-A POSTROUTING -o eth2 -j SNAT --to-source 50.92.247.211
-A POSTROUTING -o tun0 -j SNAT --to-source 10.8.0.2
root@server:/etc# iptables -S -t mangle
-P PREROUTING ACCEPT
-P INPUT ACCEPT
-P FORWARD ACCEPT
-P OUTPUT ACCEPT
-P POSTROUTING ACCEPT
-A PREROUTING -s 192.168.1.236/32 -i eth1 -j MARK --set-xmark 0x2/0xffffffff
-A PREROUTING -s 192.168.1.246/32 -i eth1 -j MARK --set-xmark 0x1/0xffffffff
root@server:/etc# iptables -S -t raw
-P PREROUTING ACCEPT
-P OUTPUT ACCEPT
#SUCCEEDS: TCPDUMP of ping from 192.168.1.246 (packets marked 0x01)
root@server:~# tcpdump -n -p -i eth1 icmp
20:47:46.026827 IP 192.168.1.246 > 8.8.8.8: ICMP echo request, id 14644,
seq 0, length 64
20:47:46.057753 IP 8.8.8.8 > 192.168.1.246: ICMP echo reply, id 14644,
seq 0, length 64
root@server:~# tcpdump -n -p -i eth0 icmp
20:47:46.026854 IP 174.4.4.74 > 8.8.8.8: ICMP echo request, id 14644,
seq 0, length 64
20:47:46.057734 IP 8.8.8.8 > 174.4.4.74: ICMP echo reply, id 14644, seq
0, length 64
#FAILS: TCPDUMP of ping from 192.168.1.236 (packets marked 0x02)
root@server:~# tcpdump -n -p -i eth1 icmp
20:37:41.852604 IP 192.168.1.236 > 8.8.8.8: ICMP echo request, id 3636,
seq 0, length 64
root@server:~# tcpdump -n -p -i eth2 icmp
20:37:41.852642 IP 50.92.247.211 > 8.8.8.8: ICMP echo request, id 3636,
seq 0, length 64
20:37:41.906329 IP 8.8.8.8 > 50.92.247.211: ICMP echo reply, id 3636,
seq 0, length 64
Thanks.
On 11-09-11 04:06 PM, Mike wrote:
> I'm in the process of upgrading an older Linux router from Mandriva
> running kernel v2.6.16 to Ubuntu running v2.6.38 kernel, however my
> moderately complex firewall/routing script doesn't quite work the same
> way on the newer system. The basic idea is that I have three routes to
> three different ISPs, and one to the internal network. I then mark
> packets to go out a specific ISP depending on the type of traffic.
> This all works fine if the packets are initiated from the router
> itself or from a computer on the intenral network with packets
> destined out the default ISP, but it fails completely if the packets
> are initiated from a computer on the internal network destined out an
> non-default route.
>
> What I don't understand is I diff'd the routing tables and all
> iptables commands they are virtually identical between the two
> servers, yet the newer server doesn't work as expected.
>
> Linux server 2.6.38-10-server #44-Ubuntu SMP Thu Jun 2 21:49:30 UTC
> 2011 x86_64 x86_64 x86_64 GNU/Linux
>
> eth0 = ISP1
> eth1 = Local network
> eth2 = ISP2
> tun0 = VPN to ISP3
>
> root@server:/etc# ip route show | sort
> 10.8.0.0/24 dev tun0 proto kernel scope link src 10.8.0.2
> 174.4.4.74 dev eth0 scope link src 174.4.4.74
> 174.4.4.0/22 dev eth0 proto kernel scope link src 174.4.4.74
> metric 10
> 192.168.1.0/24 dev eth1 proto kernel scope link src 192.168.1.1
> metric 10
> 50.92.224.0/19 dev eth2 proto kernel scope link src 50.92.247.211
> metric 10
> 50.92.247.211 dev eth2 scope link src 50.92.247.211
> 63.211.239.14 via 50.92.224.1 dev eth2
> 8.3.252.23 via 50.92.224.1 dev eth2
> default via 174.4.4.1 dev eth0
>
> root@server:~# ip rule show
> 0: from all lookup local
> 32760: from all fwmark 0x3 lookup VPN1
> 32761: from all fwmark 0x2 lookup ISP2
> 32762: from all fwmark 0x1 lookup ISP1
> 32763: from 10.8.0.2 lookup VPN1
> 32764: from 50.92.247.211 lookup ISP2
> 32765: from 174.4.4.74 lookup ISP1
> 32766: from all lookup main
> 32767: from all lookup default
>
> root@server:~# iptables -S
> -P INPUT ACCEPT
> -P FORWARD ACCEPT
> -P OUTPUT ACCEPT
> -A INPUT -s 207.150.193.134/32 -p tcp -m tcp --dport 5060:5061 -j ACCEPT
> -A INPUT -s 64.34.96.201/32 -p tcp -m tcp --dport 5060:5061 -j ACCEPT
> -A INPUT -s 64.34.96.202/32 -p tcp -m tcp --dport 5060:5061 -j ACCEPT
> -A INPUT -s 8.3.252.23/32 -p tcp -m tcp --dport 5060:5061 -j ACCEPT
> -A INPUT -s 63.211.239.14/32 -p tcp -m tcp --dport 5060:5061 -j ACCEPT
> -A INPUT -s 207.150.193.134/32 -p udp -m udp --dport 5060:5061 -j ACCEPT
> -A INPUT -s 64.34.96.201/32 -p udp -m udp --dport 5060:5061 -j ACCEPT
> -A INPUT -s 64.34.96.202/32 -p udp -m udp --dport 5060:5061 -j ACCEPT
> -A INPUT -s 8.3.252.23/32 -p udp -m udp --dport 5060:5061 -j ACCEPT
> -A INPUT -s 63.211.239.14/32 -p udp -m udp --dport 5060:5061 -j ACCEPT
> -A INPUT -i eth0 -p tcp -m tcp --dport 5060:5061 -j DROP
> -A INPUT -i eth2 -p tcp -m tcp --dport 5060:5061 -j DROP
> -A INPUT -i eth0 -p udp -m udp --dport 5060:5061 -j DROP
> -A INPUT -i eth2 -p udp -m udp --dport 5060:5061 -j DROP
> -A INPUT -s 68.75.86.8/32 -j DROP
> -A INPUT -s 174.133.3.178/32 -j DROP
>
> root@server:~# iptables -S -t nat
> -P PREROUTING ACCEPT
> -P INPUT ACCEPT
> -P OUTPUT ACCEPT
> -P POSTROUTING ACCEPT
> -A PREROUTING -i eth0 -p tcp -m tcp --dport 88 -j DNAT
> --to-destination 192.168.1.19
> -A PREROUTING -i eth0 -p tcp -m tcp --dport 3074 -j DNAT
> --to-destination 192.168.1.19
> -A PREROUTING -i eth2 -p tcp -m tcp --dport 88 -j DNAT
> --to-destination 192.168.1.19
> -A PREROUTING -i eth2 -p tcp -m tcp --dport 3074 -j DNAT
> --to-destination 192.168.1.19
> -A PREROUTING -i eth0 -p tcp -m tcp --dport 8080 -j DNAT
> --to-destination 192.168.1.9:80
> -A PREROUTING -i eth0 -p tcp -m tcp --dport 443 -j DNAT
> --to-destination 192.168.1.9:443
> -A PREROUTING -i eth0 -p tcp -m tcp --dport 4343 -j DNAT
> --to-destination 192.168.1.9:443
> -A PREROUTING -i eth0 -p tcp -m tcp --dport 69 -j DNAT
> --to-destination 192.168.1.9:69
> -A PREROUTING -i eth0 -p udp -m udp --dport 69 -j DNAT
> --to-destination 192.168.1.9:69
> -A PREROUTING -i eth0 -p tcp -m tcp --dport 22 -j DNAT
> --to-destination 192.168.1.9:22
> -A PREROUTING -i eth0 -p tcp -m tcp --dport 2323 -j DNAT
> --to-destination 192.168.1.201:23
> -A PREROUTING -i eth0 -p tcp -m tcp --dport 2380 -j DNAT
> --to-destination 192.168.1.201:80
> -A PREROUTING -i eth0 -p tcp -m tcp --dport 5501 -j DNAT
> --to-destination 192.168.1.98:5501
> -A PREROUTING -i eth0 -p tcp -m tcp --dport 5800 -j DNAT
> --to-destination 192.168.1.98:5800
> -A PREROUTING -i eth0 -p tcp -m tcp --dport 5900 -j DNAT
> --to-destination 192.168.1.98:5900
> -A PREROUTING -i eth0 -p tcp -m tcp --dport 5901 -j DNAT
> --to-destination 192.168.1.98:5901
> -A PREROUTING -i eth0 -p tcp -m tcp --dport 5902 -j DNAT
> --to-destination 192.168.1.98:5902
> -A PREROUTING -i eth0 -p tcp -m tcp --dport 5903 -j DNAT
> --to-destination 192.168.1.98:5903
> -A PREROUTING -i eth0 -p tcp -m tcp --dport 5904 -j DNAT
> --to-destination 192.168.1.98:5904
> -A PREROUTING -i eth0 -p tcp -m tcp --dport 5910 -j DNAT
> --to-destination 192.168.1.9:5900
> -A PREROUTING -i eth0 -p tcp -m tcp --dport 40696 -j DNAT
> --to-destination 192.168.1.99:40696
> -A PREROUTING -i eth0 -p tcp -m tcp --dport 50263 -j DNAT
> --to-destination 192.168.1.9:50263
> -A PREROUTING -i eth0 -p udp -m udp --dport 4444 -j DNAT
> --to-destination 192.168.1.9:4444
> -A PREROUTING -i eth0 -p udp -m udp --dport 6881 -j DNAT
> --to-destination 192.168.1.9:6881
> -A PREROUTING -i eth0 -p tcp -m tcp --dport 6881 -j DNAT
> --to-destination 192.168.1.9:6881
> -A PREROUTING -i eth0 -p udp -m udp --dport 1200 -j DNAT
> --to-destination 192.168.1.98:1200
> -A PREROUTING -i eth0 -p udp -m udp --dport 27000:27015 -j DNAT
> --to-destination 192.168.1.98
> -A PREROUTING -i eth0 -p tcp -m tcp --dport 27030:27039 -j DNAT
> --to-destination 192.168.1.98
> -A POSTROUTING -o tun0 -j SNAT --to-source 10.8.0.2
> -A POSTROUTING -o eth2 -j SNAT --to-source 50.92.247.211
> -A POSTROUTING -o eth0 -j SNAT --to-source 174.4.4.74
>
> root@server:~# iptables -S -t mangle
> -P PREROUTING ACCEPT
> -P INPUT ACCEPT
> -P FORWARD ACCEPT
> -P OUTPUT ACCEPT
> -P POSTROUTING ACCEPT
> -A PREROUTING -i eth1 -p udp -m udp --dport 4569 -j MARK --set-xmark
> 0x2/0xffffffff
> -A PREROUTING -p udp -m udp --dport 5060:5061 -j MARK --set-xmark
> 0x2/0xffffffff
> -A PREROUTING -p udp -m udp --dport 10000:20000 -j MARK --set-xmark
> 0x2/0xffffffff
> -A PREROUTING -s 192.168.1.19/32 -i eth1 -j MARK --set-xmark
> 0x3/0xffffffff
> -A PREROUTING -d 69.53.236.17/32 -i eth1 -p tcp -m tcp --dport 80 -j
> MARK --set-xmark 0x3/0xffffffff
> -A PREROUTING -d 69.53.236.17/32 -i eth1 -p tcp -m tcp --dport 443 -j
> MARK --set-xmark 0x3/0xffffffff
> -A PREROUTING -d 24.244.52.99/32 -i eth1 -p tcp -m tcp --dport 80 -j
> MARK --set-xmark 0x3/0xffffffff
> -A PREROUTING -d 24.244.52.81/32 -i eth1 -p tcp -m tcp --dport 80 -j
> MARK --set-xmark 0x3/0xffffffff
> -A PREROUTING -d 24.244.52.104/32 -i eth1 -p tcp -m tcp --dport 80 -j
> MARK --set-xmark 0x3/0xffffffff
> -A PREROUTING -d 24.244.52.83/32 -i eth1 -p tcp -m tcp --dport 80 -j
> MARK --set-xmark 0x3/0xffffffff
> -A PREROUTING -d 24.244.52.104/32 -i eth1 -p tcp -m tcp --dport 443 -j
> MARK --set-xmark 0x3/0xffffffff
> -A PREROUTING -d 24.244.52.83/32 -i eth1 -p tcp -m tcp --dport 443 -j
> MARK --set-xmark 0x3/0xffffffff
> -A PREROUTING -d 64.59.168.13/32 -i eth1 -j MARK --set-xmark
> 0x1/0xffffffff
> -A PREROUTING -d 64.59.168.15/32 -i eth1 -j MARK --set-xmark
> 0x1/0xffffffff
> -A PREROUTING -d 154.11.128.187/32 -i eth1 -j MARK --set-xmark
> 0x2/0xffffffff
> -A PREROUTING -d 154.11.128.59/32 -i eth1 -j MARK --set-xmark
> 0x2/0xffffffff
> -A PREROUTING -p tcp -m tcp --sport 80 -j TOS --set-tos 0x10/0x3f
> -A PREROUTING -p tcp -m tcp --sport 22 -j TOS --set-tos 0x10/0x3f
> -A PREROUTING -p tcp -m tcp --sport 4569 -j TOS --set-tos 0x10/0x3f
> -A PREROUTING -p tcp -m tcp --sport 21 -j TOS --set-tos 0x10/0x3f
> -A PREROUTING -p tcp -m tcp --sport 20 -j TOS --set-tos 0x08/0x3f
> -A PREROUTING -p icmp -m icmp --icmp-type 8 -j TOS --set-tos 0x10/0x3f
> -A PREROUTING -p udp -j TOS --set-tos 0x10/0x3f
> -A OUTPUT -p tcp -m tcp --dport 80 -j TOS --set-tos 0x10/0x3f
> -A OUTPUT -p tcp -m tcp --dport 22 -j TOS --set-tos 0x10/0x3f
> -A OUTPUT -p tcp -m tcp --dport 4569 -j TOS --set-tos 0x10/0x3f
> -A OUTPUT -p tcp -m tcp --dport 21 -j TOS --set-tos 0x10/0x3f
> -A OUTPUT -p tcp -m tcp --dport 20 -j TOS --set-tos 0x08/0x3f
> -A OUTPUT -p icmp -m icmp --icmp-type 8 -j TOS --set-tos 0x10/0x3f
> -A OUTPUT -p udp -j TOS --set-tos 0x10/0x3f
>
> root@server:~# iptables -S -t raw
> -P PREROUTING ACCEPT
> -P OUTPUT ACCEPT
>
> I enabled logging on each table/chain to try and diagnose where the
> issue is, below is the output of several single packet ping tests. The
> lines starting with "a" are logging at the first rule in each
> table/chain and lines starting with "z" are the last rule.
>
> SUCCESS: Ping from router (with marking enabled)
> Sep 10 15:36:47 server kernel: [11147.602519] aMANGLE:OUTPUT: IN=
> OUT=tun0 SRC=10.8.0.2 DST=8.8.8.8 LEN=84 TOS=0x00 PREC=0x00 TTL=64
> ID=0 DF PROTO=ICMP TYPE=8 CODE=0 ID=22025 SEQ=1 UID=0 GID=0 MARK=0x3
> Sep 10 15:36:47 server kernel: [11147.602530] zMANGLE:OUTPUT: IN=
> OUT=tun0 SRC=10.8.0.2 DST=8.8.8.8 LEN=84 TOS=0x10 PREC=0x00 TTL=64
> ID=0 DF PROTO=ICMP TYPE=8 CODE=0 ID=22025 SEQ=1 UID=0 GID=0 MARK=0x3
> Sep 10 15:36:47 server kernel: [11147.602543] aNAT:OUTPUT: IN=
> OUT=tun0 SRC=10.8.0.2 DST=8.8.8.8 LEN=84 TOS=0x10 PREC=0x00 TTL=64
> ID=0 DF PROTO=ICMP TYPE=8 CODE=0 ID=22025 SEQ=1 UID=0 GID=0 MARK=0x3
> Sep 10 15:36:47 server kernel: [11147.602550] zNAT:OUTPUT: IN=
> OUT=tun0 SRC=10.8.0.2 DST=8.8.8.8 LEN=84 TOS=0x10 PREC=0x00 TTL=64
> ID=0 DF PROTO=ICMP TYPE=8 CODE=0 ID=22025 SEQ=1 UID=0 GID=0 MARK=0x3
> Sep 10 15:36:47 server kernel: [11147.602560] aFILTER:OUTPUT: IN=
> OUT=tun0 SRC=10.8.0.2 DST=8.8.8.8 LEN=84 TOS=0x10 PREC=0x00 TTL=64
> ID=0 DF PROTO=ICMP TYPE=8 CODE=0 ID=22025 SEQ=1 UID=0 GID=0 MARK=0x3
> Sep 10 15:36:47 server kernel: [11147.602567] zFILTER:OUTPUT: IN=
> OUT=tun0 SRC=10.8.0.2 DST=8.8.8.8 LEN=84 TOS=0x10 PREC=0x00 TTL=64
> ID=0 DF PROTO=ICMP TYPE=8 CODE=0 ID=22025 SEQ=1 UID=0 GID=0 MARK=0x3
> Sep 10 15:36:47 server kernel: [11147.602575] aMANGLE:POSTROUTING: IN=
> OUT=tun0 SRC=10.8.0.2 DST=8.8.8.8 LEN=84 TOS=0x10 PREC=0x00 TTL=64
> ID=0 DF PROTO=ICMP TYPE=8 CODE=0 ID=22025 SEQ=1 UID=0 GID=0 MARK=0x3
> Sep 10 15:36:47 server kernel: [11147.602583] zMANGLE:POSTROUTING: IN=
> OUT=tun0 SRC=10.8.0.2 DST=8.8.8.8 LEN=84 TOS=0x10 PREC=0x00 TTL=64
> ID=0 DF PROTO=ICMP TYPE=8 CODE=0 ID=22025 SEQ=1 UID=0 GID=0 MARK=0x3
> Sep 10 15:36:47 server kernel: [11147.734585] aMANGLE:PREROUTING:
> IN=tun0 OUT= MAC= SRC=8.8.8.8 DST=10.8.0.2 LEN=84 TOS=0x00 PREC=0x00
> TTL=52 ID=64821 PROTO=ICMP TYPE=0 CODE=0 ID=22025 SEQ=1
> Sep 10 15:36:47 server kernel: [11147.734594] zMANGLE:PREROUTING:
> IN=tun0 OUT= MAC= SRC=8.8.8.8 DST=10.8.0.2 LEN=84 TOS=0x00 PREC=0x00
> TTL=52 ID=64821 PROTO=ICMP TYPE=0 CODE=0 ID=22025 SEQ=1
> Sep 10 15:36:47 server kernel: [11147.734602] aMANGLE:INPUT: IN=tun0
> OUT= MAC= SRC=8.8.8.8 DST=10.8.0.2 LEN=84 TOS=0x00 PREC=0x00 TTL=52
> ID=64821 PROTO=ICMP TYPE=0 CODE=0 ID=22025 SEQ=1
> Sep 10 15:36:47 server kernel: [11147.734608] zMANGLE:INPUT: IN=tun0
> OUT= MAC= SRC=8.8.8.8 DST=10.8.0.2 LEN=84 TOS=0x00 PREC=0x00 TTL=52
> ID=64821 PROTO=ICMP TYPE=0 CODE=0 ID=22025 SEQ=1
> Sep 10 15:36:47 server kernel: [11147.734614] aFILTER:INPUT: IN=tun0
> OUT= MAC= SRC=8.8.8.8 DST=10.8.0.2 LEN=84 TOS=0x00 PREC=0x00 TTL=52
> ID=64821 PROTO=ICMP TYPE=0 CODE=0 ID=22025 SEQ=1
> Sep 10 15:36:47 server kernel: [11147.734621] zFILTER:INPUT: IN=tun0
> OUT= MAC= SRC=8.8.8.8 DST=10.8.0.2 LEN=84 TOS=0x00 PREC=0x00 TTL=52
> ID=64821 PROTO=ICMP TYPE=0 CODE=0 ID=22025 SEQ=1
>
> FAILS: Ping from Client (with marking enabled 0x3)
> Sep 10 15:37:21 server kernel: [11181.508668] aMANGLE:PREROUTING:
> IN=eth1 OUT= MAC=f4:6d:04:9a:15:2d:00:1f:f3:d5:2d:78:08:00
> SRC=192.168.1.236 DST=8.8.8.8 LEN=84 TOS=0x00 PREC=0x00 TTL=64
> ID=56590 PROTO=ICMP TYPE=8 CODE=0 ID=62237 SEQ=0
> Sep 10 15:37:21 server kernel: [11181.508682] zMANGLE:PREROUTING:
> IN=eth1 OUT= MAC=f4:6d:04:9a:15:2d:00:1f:f3:d5:2d:78:08:00
> SRC=192.168.1.236 DST=8.8.8.8 LEN=84 TOS=0x10 PREC=0x00 TTL=64
> ID=56590 PROTO=ICMP TYPE=8 CODE=0 ID=62237 SEQ=0 MARK=0x3
> Sep 10 15:37:21 server kernel: [11181.508694] aNAT:PREROUTING: IN=eth1
> OUT= MAC=f4:6d:04:9a:15:2d:00:1f:f3:d5:2d:78:08:00 SRC=192.168.1.236
> DST=8.8.8.8 LEN=84 TOS=0x10 PREC=0x00 TTL=64 ID=56590 PROTO=ICMP
> TYPE=8 CODE=0 ID=62237 SEQ=0 MARK=0x3
> Sep 10 15:37:21 server kernel: [11181.508704] zNAT:PREROUTING: IN=eth1
> OUT= MAC=f4:6d:04:9a:15:2d:00:1f:f3:d5:2d:78:08:00 SRC=192.168.1.236
> DST=8.8.8.8 LEN=84 TOS=0x10 PREC=0x00 TTL=64 ID=56590 PROTO=ICMP
> TYPE=8 CODE=0 ID=62237 SEQ=0 MARK=0x3
> Sep 10 15:37:21 server kernel: [11181.508758] aMANGLE:FORWARD: IN=eth1
> OUT=tun0 SRC=192.168.1.236 DST=8.8.8.8 LEN=84 TOS=0x10 PREC=0x00
> TTL=63 ID=56590 PROTO=ICMP TYPE=8 CODE=0 ID=62237 SEQ=0 MARK=0x3
> Sep 10 15:37:21 server kernel: [11181.508765] zMANGLE:FORWARD: IN=eth1
> OUT=tun0 SRC=192.168.1.236 DST=8.8.8.8 LEN=84 TOS=0x10 PREC=0x00
> TTL=63 ID=56590 PROTO=ICMP TYPE=8 CODE=0 ID=62237 SEQ=0 MARK=0x3
> Sep 10 15:37:21 server kernel: [11181.508773] aFILTER:FORWARD: IN=eth1
> OUT=tun0 SRC=192.168.1.236 DST=8.8.8.8 LEN=84 TOS=0x10 PREC=0x00
> TTL=63 ID=56590 PROTO=ICMP TYPE=8 CODE=0 ID=62237 SEQ=0 MARK=0x3
> Sep 10 15:37:21 server kernel: [11181.508779] zFILTER:FORWARD: IN=eth1
> OUT=tun0 SRC=192.168.1.236 DST=8.8.8.8 LEN=84 TOS=0x10 PREC=0x00
> TTL=63 ID=56590 PROTO=ICMP TYPE=8 CODE=0 ID=62237 SEQ=0 MARK=0x3
> Sep 10 15:37:21 server kernel: [11181.508787] aMANGLE:POSTROUTING: IN=
> OUT=tun0 SRC=192.168.1.236 DST=8.8.8.8 LEN=84 TOS=0x10 PREC=0x00
> TTL=63 ID=56590 PROTO=ICMP TYPE=8 CODE=0 ID=62237 SEQ=0 MARK=0x3
> Sep 10 15:37:21 server kernel: [11181.508793] zMANGLE:POSTROUTING: IN=
> OUT=tun0 SRC=192.168.1.236 DST=8.8.8.8 LEN=84 TOS=0x10 PREC=0x00
> TTL=63 ID=56590 PROTO=ICMP TYPE=8 CODE=0 ID=62237 SEQ=0 MARK=0x3
> Sep 10 15:37:21 server kernel: [11181.642875] aMANGLE:PREROUTING:
> IN=tun0 OUT= MAC= SRC=8.8.8.8 DST=10.8.0.2 LEN=84 TOS=0x00 PREC=0x00
> TTL=52 ID=64822 PROTO=ICMP TYPE=0 CODE=0 ID=62237 SEQ=0
> Sep 10 15:37:21 server kernel: [11181.642885] zMANGLE:PREROUTING:
> IN=tun0 OUT= MAC= SRC=8.8.8.8 DST=10.8.0.2 LEN=84 TOS=0x00 PREC=0x00
> TTL=52 ID=64822 PROTO=ICMP TYPE=0 CODE=0 ID=62237 SEQ=0
>
> FAILS: Ping from Client (with marking enabled: 0x2)
> Sep 11 10:09:19 server kernel: [77836.447776] aMANGLE:PREROUTING:
> IN=eth1 OUT= MAC=f4:6d:04:9a:15:2d:00:1f:f3:d5:2d:78:08:00
> SRC=192.168.1.236 DST=8.8.8.8 LEN=84 TOS=0x00 PREC=0x00 TTL=64
> ID=13133 PROTO=ICMP TYPE=8 CODE=0 ID=812 SEQ=0
> Sep 11 10:09:19 server kernel: [77836.447789] zMANGLE:PREROUTING:
> IN=eth1 OUT= MAC=f4:6d:04:9a:15:2d:00:1f:f3:d5:2d:78:08:00
> SRC=192.168.1.236 DST=8.8.8.8 LEN=84 TOS=0x00 PREC=0x00 TTL=64
> ID=13133 PROTO=ICMP TYPE=8 CODE=0 ID=812 SEQ=0 MARK=0x2
> Sep 11 10:09:19 server kernel: [77836.447801] aNAT:PREROUTING: IN=eth1
> OUT= MAC=f4:6d:04:9a:15:2d:00:1f:f3:d5:2d:78:08:00 SRC=192.168.1.236
> DST=8.8.8.8 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=13133 PROTO=ICMP
> TYPE=8 CODE=0 ID=812 SEQ=0 MARK=0x2
> Sep 11 10:09:19 server kernel: [77836.447811] zNAT:PREROUTING: IN=eth1
> OUT= MAC=f4:6d:04:9a:15:2d:00:1f:f3:d5:2d:78:08:00 SRC=192.168.1.236
> DST=8.8.8.8 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=13133 PROTO=ICMP
> TYPE=8 CODE=0 ID=812 SEQ=0 MARK=0x2
> Sep 11 10:09:19 server kernel: [77836.447843] aMANGLE:FORWARD: IN=eth1
> OUT=eth2 SRC=192.168.1.236 DST=8.8.8.8 LEN=84 TOS=0x00 PREC=0x00
> TTL=63 ID=13133 PROTO=ICMP TYPE=8 CODE=0 ID=812 SEQ=0 MARK=0x2
> Sep 11 10:09:19 server kernel: [77836.447850] zMANGLE:FORWARD: IN=eth1
> OUT=eth2 SRC=192.168.1.236 DST=8.8.8.8 LEN=84 TOS=0x00 PREC=0x00
> TTL=63 ID=13133 PROTO=ICMP TYPE=8 CODE=0 ID=812 SEQ=0 MARK=0x2
> Sep 11 10:09:19 server kernel: [77836.447857] aFILTER:FORWARD: IN=eth1
> OUT=eth2 SRC=192.168.1.236 DST=8.8.8.8 LEN=84 TOS=0x00 PREC=0x00
> TTL=63 ID=13133 PROTO=ICMP TYPE=8 CODE=0 ID=812 SEQ=0 MARK=0x2
> Sep 11 10:09:19 server kernel: [77836.447863] zFILTER:FORWARD: IN=eth1
> OUT=eth2 SRC=192.168.1.236 DST=8.8.8.8 LEN=84 TOS=0x00 PREC=0x00
> TTL=63 ID=13133 PROTO=ICMP TYPE=8 CODE=0 ID=812 SEQ=0 MARK=0x2
> Sep 11 10:09:19 server kernel: [77836.447870] aMANGLE:POSTROUTING: IN=
> OUT=eth2 SRC=192.168.1.236 DST=8.8.8.8 LEN=84 TOS=0x00 PREC=0x00
> TTL=63 ID=13133 PROTO=ICMP TYPE=8 CODE=0 ID=812 SEQ=0 MARK=0x2
> Sep 11 10:09:19 server kernel: [77836.447877] zMANGLE:POSTROUTING: IN=
> OUT=eth2 SRC=192.168.1.236 DST=8.8.8.8 LEN=84 TOS=0x00 PREC=0x00
> TTL=63 ID=13133 PROTO=ICMP TYPE=8 CODE=0 ID=812 SEQ=0 MARK=0x2
> Sep 11 10:09:19 server kernel: [77836.501409] aMANGLE:PREROUTING:
> IN=eth2 OUT= MAC=00:1b:21:8c:07:34:00:90:1a:a0:7c:04:08:00 SRC=8.8.8.8
> DST=50.92.247.211 LEN=84 TOS=0x00 PREC=0x00 TTL=54 ID=5396 PROTO=ICMP
> TYPE=0 CODE=0 ID=812 SEQ=0
> Sep 11 10:09:19 server kernel: [77836.501421] zMANGLE:PREROUTING:
> IN=eth2 OUT= MAC=00:1b:21:8c:07:34:00:90:1a:a0:7c:04:08:00 SRC=8.8.8.8
> DST=50.92.247.211 LEN=84 TOS=0x00 PREC=0x00 TTL=54 ID=5396 PROTO=ICMP
> TYPE=0 CODE=0 ID=812 SEQ=0
>
> SUCCESS: Ping from client (with marking enabled: 0x1 [default ISP1])
> Sep 11 15:50:24 server kernel: [19600.171454] aMANGLE:PREROUTING:
> IN=eth1 OUT= MAC=f4:6d:04:9a:15:2d:00:1f:f3:d5:2d:78:08:00
> SRC=192.168.1.236 DST=8.8.8.8 LEN=84 TOS=0x00 PREC=0x00 TTL=64
> ID=17642 PROTO=ICMP TYPE=8 CODE=0 ID=16944 SEQ=0
> Sep 11 15:50:24 server kernel: [19600.171467] zMANGLE:PREROUTING:
> IN=eth1 OUT= MAC=f4:6d:04:9a:15:2d:00:1f:f3:d5:2d:78:08:00
> SRC=192.168.1.236 DST=8.8.8.8 LEN=84 TOS=0x00 PREC=0x00 TTL=64
> ID=17642 PROTO=ICMP TYPE=8 CODE=0 ID=16944 SEQ=0 MARK=0x1
> Sep 11 15:50:24 server kernel: [19600.171479] aNAT:PREROUTING: IN=eth1
> OUT= MAC=f4:6d:04:9a:15:2d:00:1f:f3:d5:2d:78:08:00 SRC=192.168.1.236
> DST=8.8.8.8 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=17642 PROTO=ICMP
> TYPE=8 CODE=0 ID=16944 SEQ=0 MARK=0x1
> Sep 11 15:50:24 server kernel: [19600.171489] zNAT:PREROUTING: IN=eth1
> OUT= MAC=f4:6d:04:9a:15:2d:00:1f:f3:d5:2d:78:08:00 SRC=192.168.1.236
> DST=8.8.8.8 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=17642 PROTO=ICMP
> TYPE=8 CODE=0 ID=16944 SEQ=0 MARK=0x1
> Sep 11 15:50:24 server kernel: [19600.171500] aMANGLE:FORWARD: IN=eth1
> OUT=eth0 SRC=192.168.1.236 DST=8.8.8.8 LEN=84 TOS=0x00 PREC=0x00
> TTL=63 ID=17642 PROTO=ICMP TYPE=8 CODE=0 ID=16944 SEQ=0 MARK=0x1
> Sep 11 15:50:24 server kernel: [19600.171506] zMANGLE:FORWARD: IN=eth1
> OUT=eth0 SRC=192.168.1.236 DST=8.8.8.8 LEN=84 TOS=0x00 PREC=0x00
> TTL=63 ID=17642 PROTO=ICMP TYPE=8 CODE=0 ID=16944 SEQ=0 MARK=0x1
> Sep 11 15:50:24 server kernel: [19600.171513] aFILTER:FORWARD: IN=eth1
> OUT=eth0 SRC=192.168.1.236 DST=8.8.8.8 LEN=84 TOS=0x00 PREC=0x00
> TTL=63 ID=17642 PROTO=ICMP TYPE=8 CODE=0 ID=16944 SEQ=0 MARK=0x1
> Sep 11 15:50:24 server kernel: [19600.171520] zFILTER:FORWARD: IN=eth1
> OUT=eth0 SRC=192.168.1.236 DST=8.8.8.8 LEN=84 TOS=0x00 PREC=0x00
> TTL=63 ID=17642 PROTO=ICMP TYPE=8 CODE=0 ID=16944 SEQ=0 MARK=0x1
> Sep 11 15:50:24 server kernel: [19600.171527] aMANGLE:POSTROUTING: IN=
> OUT=eth0 SRC=192.168.1.236 DST=8.8.8.8 LEN=84 TOS=0x00 PREC=0x00
> TTL=63 ID=17642 PROTO=ICMP TYPE=8 CODE=0 ID=16944 SEQ=0 MARK=0x1
> Sep 11 15:50:24 server kernel: [19600.171534] zMANGLE:POSTROUTING: IN=
> OUT=eth0 SRC=192.168.1.236 DST=8.8.8.8 LEN=84 TOS=0x00 PREC=0x00
> TTL=63 ID=17642 PROTO=ICMP TYPE=8 CODE=0 ID=16944 SEQ=0 MARK=0x1
> Sep 11 15:50:24 server kernel: [19600.214666] aMANGLE:PREROUTING:
> IN=eth0 OUT= MAC=f4:6d:04:9a:07:bd:00:30:b8:c9:5c:90:08:00 SRC=8.8.8.8
> DST=174.4.4.74 LEN=84 TOS=0x00 PREC=0x00 TTL=54 ID=59275 PROTO=ICMP
> TYPE=0 CODE=0 ID=16944 SEQ=0
> Sep 11 15:50:24 server kernel: [19600.214678] zMANGLE:PREROUTING:
> IN=eth0 OUT= MAC=f4:6d:04:9a:07:bd:00:30:b8:c9:5c:90:08:00 SRC=8.8.8.8
> DST=174.4.4.74 LEN=84 TOS=0x00 PREC=0x00 TTL=54 ID=59275 PROTO=ICMP
> TYPE=0 CODE=0 ID=16944 SEQ=0
> Sep 11 15:50:24 server kernel: [19600.214690] aMANGLE:FORWARD: IN=eth0
> OUT=eth1 SRC=8.8.8.8 DST=192.168.1.236 LEN=84 TOS=0x00 PREC=0x00
> TTL=53 ID=59275 PROTO=ICMP TYPE=0 CODE=0 ID=16944 SEQ=0
> Sep 11 15:50:24 server kernel: [19600.214696] zMANGLE:FORWARD: IN=eth0
> OUT=eth1 SRC=8.8.8.8 DST=192.168.1.236 LEN=84 TOS=0x00 PREC=0x00
> TTL=53 ID=59275 PROTO=ICMP TYPE=0 CODE=0 ID=16944 SEQ=0
> Sep 11 15:50:24 server kernel: [19600.214702] aFILTER:FORWARD: IN=eth0
> OUT=eth1 SRC=8.8.8.8 DST=192.168.1.236 LEN=84 TOS=0x00 PREC=0x00
> TTL=53 ID=59275 PROTO=ICMP TYPE=0 CODE=0 ID=16944 SEQ=0
> Sep 11 15:50:24 server kernel: [19600.214709] zFILTER:FORWARD: IN=eth0
> OUT=eth1 SRC=8.8.8.8 DST=192.168.1.236 LEN=84 TOS=0x00 PREC=0x00
> TTL=53 ID=59275 PROTO=ICMP TYPE=0 CODE=0 ID=16944 SEQ=0
> Sep 11 15:50:24 server kernel: [19600.214715] aMANGLE:POSTROUTING: IN=
> OUT=eth1 SRC=8.8.8.8 DST=192.168.1.236 LEN=84 TOS=0x00 PREC=0x00
> TTL=53 ID=59275 PROTO=ICMP TYPE=0 CODE=0 ID=16944 SEQ=0
> Sep 11 15:50:24 server kernel: [19600.214721] zMANGLE:POSTROUTING: IN=
> OUT=eth1 SRC=8.8.8.8 DST=192.168.1.236 LEN=84 TOS=0x00 PREC=0x00
> TTL=53 ID=59275 PROTO=ICMP TYPE=0 CODE=0 ID=16944 SEQ=0
>
> SUCCESS: Ping from client (WITHOUT marking)
> Sep 10 15:44:21 server kernel: [11601.127159] aMANGLE:PREROUTING:
> IN=eth1 OUT= MAC=f4:6d:04:9a:15:2d:00:1f:f3:d5:2d:78:08:00
> SRC=192.168.1.236 DST=8.8.8.8 LEN=84 TOS=0x00 PREC=0x00 TTL=64
> ID=31275 PROTO=ICMP TYPE=8 CODE=0 ID=3358 SEQ=0
> Sep 10 15:44:21 server kernel: [11601.127173] zMANGLE:PREROUTING:
> IN=eth1 OUT= MAC=f4:6d:04:9a:15:2d:00:1f:f3:d5:2d:78:08:00
> SRC=192.168.1.236 DST=8.8.8.8 LEN=84 TOS=0x10 PREC=0x00 TTL=64
> ID=31275 PROTO=ICMP TYPE=8 CODE=0 ID=3358 SEQ=0
> Sep 10 15:44:21 server kernel: [11601.127185] aNAT:PREROUTING: IN=eth1
> OUT= MAC=f4:6d:04:9a:15:2d:00:1f:f3:d5:2d:78:08:00 SRC=192.168.1.236
> DST=8.8.8.8 LEN=84 TOS=0x10 PREC=0x00 TTL=64 ID=31275 PROTO=ICMP
> TYPE=8 CODE=0 ID=3358 SEQ=0
> Sep 10 15:44:21 server kernel: [11601.127194] zNAT:PREROUTING: IN=eth1
> OUT= MAC=f4:6d:04:9a:15:2d:00:1f:f3:d5:2d:78:08:00 SRC=192.168.1.236
> DST=8.8.8.8 LEN=84 TOS=0x10 PREC=0x00 TTL=64 ID=31275 PROTO=ICMP
> TYPE=8 CODE=0 ID=3358 SEQ=0
> Sep 10 15:44:21 server kernel: [11601.127207] aMANGLE:FORWARD: IN=eth1
> OUT=eth0 SRC=192.168.1.236 DST=8.8.8.8 LEN=84 TOS=0x10 PREC=0x00
> TTL=63 ID=31275 PROTO=ICMP TYPE=8 CODE=0 ID=3358 SEQ=0
> Sep 10 15:44:21 server kernel: [11601.127213] zMANGLE:FORWARD: IN=eth1
> OUT=eth0 SRC=192.168.1.236 DST=8.8.8.8 LEN=84 TOS=0x10 PREC=0x00
> TTL=63 ID=31275 PROTO=ICMP TYPE=8 CODE=0 ID=3358 SEQ=0
> Sep 10 15:44:21 server kernel: [11601.127220] aFILTER:FORWARD: IN=eth1
> OUT=eth0 SRC=192.168.1.236 DST=8.8.8.8 LEN=84 TOS=0x10 PREC=0x00
> TTL=63 ID=31275 PROTO=ICMP TYPE=8 CODE=0 ID=3358 SEQ=0
> Sep 10 15:44:21 server kernel: [11601.127226] zFILTER:FORWARD: IN=eth1
> OUT=eth0 SRC=192.168.1.236 DST=8.8.8.8 LEN=84 TOS=0x10 PREC=0x00
> TTL=63 ID=31275 PROTO=ICMP TYPE=8 CODE=0 ID=3358 SEQ=0
> Sep 10 15:44:21 server kernel: [11601.169794] aMANGLE:FORWARD: IN=eth0
> OUT=eth1 SRC=8.8.8.8 DST=192.168.1.236 LEN=84 TOS=0x00 PREC=0x00
> TTL=53 ID=37282 PROTO=ICMP TYPE=0 CODE=0 ID=3358 SEQ=0
> Sep 10 15:44:21 server kernel: [11601.169804] zMANGLE:FORWARD: IN=eth0
> OUT=eth1 SRC=8.8.8.8 DST=192.168.1.236 LEN=84 TOS=0x00 PREC=0x00
> TTL=53 ID=37282 PROTO=ICMP TYPE=0 CODE=0 ID=3358 SEQ=0
> Sep 10 15:44:21 server kernel: [11601.169811] aFILTER:FORWARD: IN=eth0
> OUT=eth1 SRC=8.8.8.8 DST=192.168.1.236 LEN=84 TOS=0x00 PREC=0x00
> TTL=53 ID=37282 PROTO=ICMP TYPE=0 CODE=0 ID=3358 SEQ=0
> Sep 10 15:44:21 server kernel: [11601.169818] zFILTER:FORWARD: IN=eth0
> OUT=eth1 SRC=8.8.8.8 DST=192.168.1.236 LEN=84 TOS=0x00 PREC=0x00
> TTL=53 ID=37282 PROTO=ICMP TYPE=0 CODE=0 ID=3358 SEQ=0
>
> Whenever marking is enabled for anything other than the default ISP
> and the packet originates from a internal computer, the packets get
> out and upon return they seem to get dropped at the router after the
> mangle prerouting chain.
>
> I know it has to be something simple, but I'm all out of ideas at this
> point, especially since the comparison between the two servers is a
> match as far as I can tell. You can see the same output from the old
> server and new server for a comparison here:
>
> http://pastebin.com/EvmzfCe1
> http://pastebin.com/xNSt60D9
>
> Any help would be greatly appreciated.
>
--
Mike
next prev parent reply other threads:[~2011-09-12 4:13 UTC|newest]
Thread overview: 4+ messages / expand[flat|nested] mbox.gz Atom feed top
2011-09-11 23:06 v2.6.16 to v2.6.38 breaks routing? Mike
2011-09-12 4:13 ` Mike [this message]
2011-09-12 9:24 ` John Haxby
2011-09-12 14:59 ` Mike
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=4E6D86E1.60009@snappymail.ca \
--to=ipso@snappymail.ca \
--cc=netfilter@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox