From: sillysausage <sillysausage@privatedemail.net>
To: netfilter@vger.kernel.org
Subject: Re: Routing 192.168.1.0/24 to ISP and 192.168.2.0/24 to VPN using fwmark+mangle+iproute
Date: Tue, 11 Aug 2015 16:53:26 +0930 [thread overview]
Message-ID: <55C9A2EE.7010508@privatedemail.net> (raw)
In-Reply-To: <55C7658D.3030404@privatedemail.net>
So.
I've been testing this on PPP0 and it seems to work, from a
192.168.1.0/24 address I downloaded a 10MB file.
# iptables -L --line-numbers -n -v -t mangle
Chain PREROUTING (policy ACCEPT 21134 packets, 13M bytes)
num pkts bytes target prot opt in out source destination
1 42801 17M CONNMARK all -- * * 0.0.0.0/0 0.0.0.0/0 CONNMARK restore
2 16 7976 LOG all -- * * 0.0.0.0/0 0.0.0.0/0 mark match 0x2 LOG flags 0 level 7 prefix "fwmark 2: "
3 0 0 ACCEPT all -- * * 192.168.2.0/24 0.0.0.0/0 mark match 0x2
4 19 9113 MARK all -- tun0 * 0.0.0.0/0 0.0.0.0/0 MARK set 0x2
5 41491 17M LOG all -- * * 0.0.0.0/0 0.0.0.0/0 mark match 0x1 LOG flags 0 level 7 prefix "fwmark 1: "
6 21667 3778K ACCEPT all -- * * 192.168.1.0/24 0.0.0.0/0 mark match 0x1
7 19961 13M MARK all -- ppp0 * 0.0.0.0/0 0.0.0.0/0 MARK set 0x1
8 21134 13M CONNMARK all -- * * 0.0.0.0/0 0.0.0.0/0 CONNMARK save
You can see that chain 7 is hitting.
Now I tried this with uploading too, and could see that hit chain 6.
Chain PREROUTING (policy ACCEPT 5642 packets, 351K bytes)
num pkts bytes target prot opt in out source destination
1 12595 10M CONNMARK all -- * * 0.0.0.0/0 0.0.0.0/0 CONNMARK restore
2 7 3822 LOG all -- * * 0.0.0.0/0 0.0.0.0/0 mark match 0x2 LOG flags 0 level 7 prefix "fwmark 2: "
3 0 0 ACCEPT all -- * * 192.168.2.0/24 0.0.0.0/0 mark match 0x2
4 7 3822 MARK all -- tun0 * 0.0.0.0/0 0.0.0.0/0 MARK set 0x2
5 11302 10M LOG all -- * * 0.0.0.0/0 0.0.0.0/0 mark match 0x1 LOG flags 0 level 7 prefix "fwmark 1: "
6 6953 10M ACCEPT all -- * * 192.168.1.0/24 0.0.0.0/0 mark match 0x1
7 4350 267K MARK all -- ppp0 * 0.0.0.0/0 0.0.0.0/0 MARK set 0x1
8 5642 351K CONNMARK all -- * * 0.0.0.0/0 0.0.0.0/0 CONNMARK save
So now I want to look at the VPN. Pinging 8.8.8.8 I see nothing
hitting for chain 2, 3, 4.
# iptables -L --line-numbers -n -v -t mangle
Chain PREROUTING (policy ACCEPT 619 packets, 37603 bytes)
num pkts bytes target prot opt in out source destination
1 632 38676 CONNMARK all -- * * 0.0.0.0/0 0.0.0.0/0 CONNMARK restore
2 0 0 LOG all -- * * 0.0.0.0/0 0.0.0.0/0 mark match 0x2 LOG flags 0 level 7 prefix "fwmark 2: "
3 0 0 ACCEPT all -- * * 192.168.2.0/24 0.0.0.0/0 mark match 0x2
4 0 0 MARK all -- tun0 * 0.0.0.0/0 0.0.0.0/0 MARK set 0x2
5 13 1073 ACCEPT all -- * * 192.168.1.0/24 0.0.0.0/0 mark match 0x1
6 18 2042 MARK all -- ppp0 * 0.0.0.0/0 0.0.0.0/0 MARK set 0x1
7 619 37603 CONNMARK all -- * * 0.0.0.0/0 0.0.0.0/0 CONNMARK save
Nothing in the logs either....
If I ping something ON the VPN such as their DNS server 172.16.32.1
# iptables -L --line-numbers -n -v -t mangle
Chain PREROUTING (policy ACCEPT 850 packets, 55577 bytes)
num pkts bytes target prot opt in out source destination
1 882 58116 CONNMARK all -- * * 0.0.0.0/0 0.0.0.0/0 CONNMARK restore
2 9 756 LOG all -- * * 0.0.0.0/0 0.0.0.0/0 mark match 0x2 LOG flags 0 level 7 prefix "fwmark 2: "
3 5 420 ACCEPT all -- * * 192.168.2.0/24 0.0.0.0/0 mark match 0x2
4 5 420 MARK all -- tun0 * 0.0.0.0/0 0.0.0.0/0 MARK set 0x2
5 27 2119 ACCEPT all -- * * 192.168.1.0/24 0.0.0.0/0 mark match 0x1
6 49 5895 MARK all -- ppp0 * 0.0.0.0/0 0.0.0.0/0 MARK set 0x1
7 850 55577 CONNMARK all -- * * 0.0.0.0/0 0.0.0.0/0 CONNMARK save
I see that it does hit the rules. So this makes me think the mangle
rules are correct?
I can see it in the logs too
Aug 11 06:21:16 gateway kern.debug kernel:
[ 5503.361100] fwmark 2: IN=tun0 OUT= MAC= SRC=172.16.32.1 DST=route_vpn_gateway
LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=18989 PROTO=ICMP TYPE=0 CODE=0 ID=24842
SEQ=230 MARK=0x2
Aug 11 06:21:17 gateway kern.debug kernel:
[ 5503.941894] fwmark 2: IN=eth0 OUT=
SRC=192.168.2.20 DST=172.16.32.1 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=51575
DF PROTO=ICMP TYPE=8 CODE=0 ID=2
It would also indicate the forwarding is working too. Ie forwarding
from 192.168.2.20 > 192.168.2.1 > route_vpn_gateway > 172.16.32.1
# ip rule
0: from all lookup local
1: from IPLOCAL_EXTERNAL_IP lookup ISP
1: from all fwmark 0x1 lookup ISP
2: from route_vpn_gateway lookup VPN
2: from all fwmark 0x2 lookup VPN
32766: from all lookup main
32767: from all lookup defaul
# ip route sh table main
default dev ppp0 scope link metric 300
172.16.32.0/20 dev tun0 proto kernel scope link src route_vpn_gateway
192.168.0.0/30 dev eth1 proto kernel scope link src 192.168.0.2
192.168.1.0/24 dev eth0 proto kernel scope link src 192.168.1.1
192.168.2.0/24 dev eth0 proto kernel scope link src 192.168.2.1
203.16.215.199 dev ppp0 proto kernel scope link src IPLOCAL_EXTERNAL_IP
# ip route sh table ISP
default via IPLOCAL_EXTERNAL_IP dev ppp0 metric 1
192.168.1.0/24 dev eth0 scope link metric 1
# ip route sh table VPN
default via route_vpn_gateway dev tun0 metric 2
192.168.2.0/24 dev eth0 scope link metric 2
#########################################################################
# Advanced routing rule set
# Uses 192.168.1.0 via ISP
# 192.168.2.0 via VPN
#
# Packets to/from 192.168.1.0/24 are marked with 0x1 and routed to ISP
# Packets to/from 192.168.2.0/24 are marked with 0x2 and routed to VPN
#
# http://nerdboys.com/2006/05/05/conning-the-mark-multiwan-connections-using-iptables-mark-connmark-and-iproute2/
# http://nerdboys.com/2006/05/08/multiwan-connections-addendum/
#########################################################################
# Set up the mangle table
*mangle
:PREROUTING ACCEPT [0:0]
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
# Restore CONNMARK to the MARK (If one doesn't exist then no mark is set
-A PREROUTING -j CONNMARK --restore-mark --nfmask 0xffffffff --ctmask 0xffffffff
# If packet MARK is 2, then it means there is already a connection mark and the original packet came in on VPN
-A PREROUTING -m mark --mark 0x2 -j LOG --log-prefix "fwmark 2: " --log-level 7
-A PREROUTING -s 192.168.2.0/24 -m mark --mark 0x2 -j ACCEPT
# Else MARK packet as 2
-A PREROUTING -i tun0 -j MARK --set-xmark 0x2/0xffffffff
# Optimized rule mentioned in http://nerdboys.com/2006/05/08/multiwan-connections-addendum/
# didn't seem to work with that
#-A PREROUTING -i tun0 -m conntrack --ctstate NEW -m mark --mark 0x0 -j MARK --set-xmark 0x2/0xffffffff
# If packet MARK is 1, then it means there is already a connection mark and the original packet came in on ISP
-A PREROUTING -s 192.168.1.0/24 -m mark --mark 0x1 -j ACCEPT
# Else MARK packet as 1
-A PREROUTING -i ppp0 -j MARK --set-xmark 0x1/0xffffffff
# Optimized rule mentioned in http://nerdboys.com/2006/05/08/multiwan-connections-addendum/
# didn't seem to work with that
#-A PREROUTING -i ppp0 -m conntrack --ctstate NEW -m mark --mark 0x0 -j MARK --set-xmark 0x1/0xffffffff
# Save MARK to CONNMARK
-A PREROUTING -j CONNMARK --save-mark --nfmask 0xffffffff --ctmask 0xffffffff
COMMIT
# Set up the filter table
*filter
:INPUT DROP [0:0]
:FORWARD DROP [0:0]
:OUTPUT ACCEPT [0:0]
# Create rule chain per input interface for forwarding packets
:FWD_ETH0 - [0:0]
:FWD_ETH1 - [0:0]
:FWD_PPP0 - [0:0]
:FWD_TUN0 - [0:0]
# Create rule chain per input interface for input packets (for host itself)
:IN_ETH0 - [0:0]
:IN_ETH1 - [0:0]
:IN_PPP0 - [0:0]
:IN_TUN0 - [0:0]
# Pass input packet to corresponded rule chain
-A INPUT -i lo -j ACCEPT
-A INPUT -i eth0 -j IN_ETH0
-A INPUT -i eth1 -j IN_ETH1
-A INPUT -i ppp0 -j IN_PPP0
-A INPUT -i tun0 -j IN_TUN0
# TCP flag checks - block invalid flags
-A INPUT -m conntrack --ctstate INVALID -j DROP
# Log packets that are dropped in INPUT chain (useful for debugging)
-A INPUT -j LOG --log-prefix "iptables/filter/INPUT end"
# Pass forwarded packet to corresponded rule chain
-A FORWARD -i eth0 -j FWD_ETH0
-A FORWARD -i eth1 -j FWD_ETH1
-A FORWARD -i ppp0 -j FWD_PPP0
-A FORWARD -i tun0 -j FWD_TUN0
# Log packets that are dropped in FORWARD chain (useful for debugging)
-A FORWARD -j LOG --log-prefix "iptables/filter/FORWARD end"
# Forward traffic to LAN
-A FWD_ETH0 -s 192.168.1.0/24 -m conntrack --ctstate NEW,ESTABLISHED -j ACCEPT
# Forward traffic to VPN
-A FWD_ETH0 -s 192.168.2.0/24 -m conntrack --ctstate NEW,ESTABLISHED -j ACCEPT
# Forward traffic to ppp0 WAN port
-A FWD_PPP0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
# Forward ICMP from VPN, (breaks traceroute through VPN if you don't have this)
-A FWD_TUN0 -d 192.168.2.0/24 -p icmp -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
# Forward traffic to tun0 VPN port
-A FWD_TUN0 -d 192.168.2.0/24 -m conntrack --ctstate NEW,ESTABLISHED -j ACCEPT
# SSH to Router
-A IN_ETH0 -s 192.168.1.0/24 -p tcp -m tcp --dport 22 -m conntrack --ctstate NEW,ESTABLISHED -j ACCEPT
-A IN_ETH0 -s 192.168.2.0/24 -p tcp -m tcp --dport 22 -m conntrack --ctstate NEW,ESTABLISHED -j ACCEPT
# DNS to Router
-A IN_ETH0 -s 192.168.1.0/24 -p udp -m udp --dport 53 -m conntrack --ctstate NEW -j ACCEPT
-A IN_ETH0 -s 192.168.2.0/24 -p udp -m udp --dport 53 -m conntrack --ctstate NEW -j ACCEPT
# Accept traffic to router on both subnets
-A IN_ETH0 -s 192.168.1.0/24 -m conntrack --ctstate NEW,ESTABLISHED -j ACCEPT
-A IN_ETH0 -s 192.168.2.0/24 -m conntrack --ctstate NEW,ESTABLISHED -j ACCEPT
# Prevent leakages from 192.168.2.0/24 hosts when VPN goes down for some reason
-A IN_ETH0 -s 192.168.2.0/24 -o ppp0 -j REJECT --reject-with icmp-port-unreachable
# Accept incoming tracked PPP0 connections
-A IN_PPP0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
# Incoming ICMP from VPN, (breaks traceroute through VPN if you don't have this)
-A IN_TUN0 -d 192.168.2.0/24 -p icmp -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
# Accept incoming tracked connections from 192.168.2.0/24 to VPN
-A IN_TUN0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
COMMIT
*nat
:PREROUTING ACCEPT [0:0]
:INPUT ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
# Allows for network hosts to access the internet via VPN tunnel
-A POSTROUTING -s 192.168.2.0/24 -o tun0 -j MASQUERADE
# Allows for network hosts to access the internet via WAN port
-A POSTROUTING -s 192.168.1.0/24 -o ppp0 -j MASQUERADE
COMMIT
So my question is.
If 192.168.2.20 > 192.168.2.1 > route_vpn_gateway > 172.16.32.1
works why doesn't:
192.168.2.20 > 192.168.2.1 > route_vpn_gateway > 8.8.8.8
next prev parent reply other threads:[~2015-08-11 7:23 UTC|newest]
Thread overview: 5+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-08-06 17:44 Routing 192.168.1.0/24 to ISP and 192.168.2.0/24 to VPN using fwmark+mangle+iproute sillysausage
2015-08-07 13:03 ` sillysausage
2015-08-09 14:37 ` sillysausage
2015-08-11 7:23 ` sillysausage [this message]
2015-08-12 3:12 ` sillysausage
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=55C9A2EE.7010508@privatedemail.net \
--to=sillysausage@privatedemail.net \
--cc=netfilter@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox