* Wrong routing when combining ip rule with SNAT
@ 2013-09-13 5:10 Nikolaus Rath
2013-09-13 6:26 ` Vigneswaran R
2013-09-16 23:38 ` Eliezer Croitoru
0 siblings, 2 replies; 22+ messages in thread
From: Nikolaus Rath @ 2013-09-13 5:10 UTC (permalink / raw)
To: netfilter
Hello,
Thanks for working on this great networking stack!
I'm trying to set up a configuration with SNAT and routing rules, but
I'm having weird problems that I do not understand:
I've enabled packet forwarding and SNAT on the "ebox" computer as
follows:
root@ebox:~# ip route
default via 23.92.25.1 dev eth0
23.92.25.0/24 dev eth0 proto kernel scope link src 23.92.25.96
192.168.12.0/24 dev rath proto kernel scope link src 192.168.12.1
root@ebox:~# iptables -L -n -v
Chain INPUT (policy ACCEPT 1314 packets, 1736K bytes)
pkts bytes target prot opt in out source destination
Chain FORWARD (policy DROP 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
150K 62M ACCEPT all -- rath eth0 0.0.0.0/0 0.0.0.0/0
86746 200M ACCEPT all -- eth0 rath 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED
319 22076 LOG all -- * * 0.0.0.0/0 0.0.0.0/0 limit: avg 1/min burst 30 LOG flags 0 level 4 prefix "Rejected forwarding: "
393 26172 REJECT all -- * * 0.0.0.0/0 0.0.0.0/0 reject-with icmp-net-prohibited
Chain OUTPUT (policy ACCEPT 1142 packets, 2412K bytes)
pkts bytes target prot opt in out source destination
root@ebox:~# iptables -t nat -L -n -v
Chain PREROUTING (policy ACCEPT 36378 packets, 2383K bytes)
Chain INPUT (policy ACCEPT 19982 packets, 1334K bytes)
pkts bytes target prot opt in out source destination
Chain OUTPUT (policy ACCEPT 61430 packets, 4601K bytes)
pkts bytes target prot opt in out source destination
Chain POSTROUTING (policy ACCEPT 8333 packets, 564K bytes)
pkts bytes target prot opt in out source destination
69488 5081K SNAT all -- * eth0 0.0.0.0/0 0.0.0.0/0 to:23.92.25.96
From a second computer "vostro", I can now use ebox as a gateway:
root@vostro:~# ip route add 190.93.249.164 via 192.168.12.1
This works fine, now connections to whatismyip.com (190.93.249.164) go
through ebox.
However, when I try to be a bit more selective on vostro and use a
special routing table, things don't work anymore:
root@vostro:~# iptables -t mangle -L -n
Chain PREROUTING (policy ACCEPT)
target prot opt source destination
Chain INPUT (policy ACCEPT)
target prot opt source destination
Chain FORWARD (policy ACCEPT)
target prot opt source destination
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
MARK tcp -- 0.0.0.0/0 190.93.249.164 tcp dpt:80 MARK set 0x1
LOG tcp -- 0.0.0.0/0 190.93.249.164 tcp dpt:80 LOG flags 0 level 4 prefix "marked: "
Chain POSTROUTING (policy ACCEPT)
target prot opt source destination
root@vostro:~# ip route del 190.93.249.164 via 192.168.12.1
root@vostro:~# ip route add default via 192.168.12.1 table tovpn
root@vostro:~# ip rule add fwmark 0x1 table tovpn
Now connections from vostro to 190.93.249.164 still make it to ebox, and
from ebox to 190.93.249.164, but the answers get stuck on ebox:
Sep 13 04:47:53 ebox kernel: Rejected forwarding: IN=eth0 OUT=eth0 MAC=f2:3c:91:69:db:07:84:78:ac:0d:79:c1:08:00 SRC=190.93.249.164 DST=192.168.17.47 LEN=60 TOS=0x00 PREC=0x00 TTL=58 ID=0 DF PROTO=TCP SPT=80 DPT=39024 WINDOW=14480 RES=0x00 ACK SYN URGP=0
It seems that ebox tries to send the packet destined to go trough the
rath to eth0 instead, and consequency rejects them because forwarding is
only enabled from eth0 to rath.
However, this only happens when vostro has the gateway route set in a
special routing table rather than the default table -- but how does ebox
even know about that?
Can someone explain to me what is happening here and why?
Best,
-Nikolaus
--
»Time flies like an arrow, fruit flies like a Banana.«
PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6 02CF A9AD B7F8 AE4E 425C
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: Wrong routing when combining ip rule with SNAT
2013-09-13 5:10 Wrong routing when combining ip rule with SNAT Nikolaus Rath
@ 2013-09-13 6:26 ` Vigneswaran R
2013-09-13 16:09 ` Nikolaus Rath
2013-09-14 13:41 ` Pascal Hambourg
2013-09-16 23:38 ` Eliezer Croitoru
1 sibling, 2 replies; 22+ messages in thread
From: Vigneswaran R @ 2013-09-13 6:26 UTC (permalink / raw)
To: netfilter
Hello Nikolaus,
I have a doubt. It seems, rath of ebox is assigned with IP address in
the range 192.168.12.0/24. However, IP address of vostro seems to be
192.168.17.47 (assuming /24). Ebox doesn't have any route to this range.
So it try to use default route via eth0.
What I assume is, 'vostro' has IP addresses in (atleast) two ranges
(192.168.12.0/24, 192.168.17.0/24). In the default routing table, the
src IP is set to 192.168.12.x (for the packets originating from vostro).
However, the 'tovpn' table didn't specify the src IP. So, when the
'tovpn' table is being used, the packets may have got the src IP as
192.168.17.x.
I think, you can avoid this by explicitly specifying the src IP when
adding the route to 'tovpn' table,
ip route add default via 192.168.12.1 src 192.168.12.x table tovpn
Regards,
Vignesh
On 09/13/2013 10:40 AM, Nikolaus Rath wrote:
> Hello,
>
> Thanks for working on this great networking stack!
>
> I'm trying to set up a configuration with SNAT and routing rules, but
> I'm having weird problems that I do not understand:
>
> I've enabled packet forwarding and SNAT on the "ebox" computer as
> follows:
>
> root@ebox:~# ip route
> default via 23.92.25.1 dev eth0
> 23.92.25.0/24 dev eth0 proto kernel scope link src 23.92.25.96
> 192.168.12.0/24 dev rath proto kernel scope link src 192.168.12.1
>
> root@ebox:~# iptables -L -n -v
> Chain INPUT (policy ACCEPT 1314 packets, 1736K bytes)
> pkts bytes target prot opt in out source destination
>
> Chain FORWARD (policy DROP 0 packets, 0 bytes)
> pkts bytes target prot opt in out source destination
> 150K 62M ACCEPT all -- rath eth0 0.0.0.0/0 0.0.0.0/0
> 86746 200M ACCEPT all -- eth0 rath 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED
> 319 22076 LOG all -- * * 0.0.0.0/0 0.0.0.0/0 limit: avg 1/min burst 30 LOG flags 0 level 4 prefix "Rejected forwarding: "
> 393 26172 REJECT all -- * * 0.0.0.0/0 0.0.0.0/0 reject-with icmp-net-prohibited
>
> Chain OUTPUT (policy ACCEPT 1142 packets, 2412K bytes)
> pkts bytes target prot opt in out source destination
>
> root@ebox:~# iptables -t nat -L -n -v
> Chain PREROUTING (policy ACCEPT 36378 packets, 2383K bytes)
>
> Chain INPUT (policy ACCEPT 19982 packets, 1334K bytes)
> pkts bytes target prot opt in out source destination
>
> Chain OUTPUT (policy ACCEPT 61430 packets, 4601K bytes)
> pkts bytes target prot opt in out source destination
>
> Chain POSTROUTING (policy ACCEPT 8333 packets, 564K bytes)
> pkts bytes target prot opt in out source destination
> 69488 5081K SNAT all -- * eth0 0.0.0.0/0 0.0.0.0/0 to:23.92.25.96
>
>
> >From a second computer "vostro", I can now use ebox as a gateway:
>
> root@vostro:~# ip route add 190.93.249.164 via 192.168.12.1
>
> This works fine, now connections to whatismyip.com (190.93.249.164) go
> through ebox.
>
> However, when I try to be a bit more selective on vostro and use a
> special routing table, things don't work anymore:
>
> root@vostro:~# iptables -t mangle -L -n
> Chain PREROUTING (policy ACCEPT)
> target prot opt source destination
>
> Chain INPUT (policy ACCEPT)
> target prot opt source destination
>
> Chain FORWARD (policy ACCEPT)
> target prot opt source destination
>
> Chain OUTPUT (policy ACCEPT)
> target prot opt source destination
> MARK tcp -- 0.0.0.0/0 190.93.249.164 tcp dpt:80 MARK set 0x1
> LOG tcp -- 0.0.0.0/0 190.93.249.164 tcp dpt:80 LOG flags 0 level 4 prefix "marked: "
>
> Chain POSTROUTING (policy ACCEPT)
> target prot opt source destination
>
> root@vostro:~# ip route del 190.93.249.164 via 192.168.12.1
> root@vostro:~# ip route add default via 192.168.12.1 table tovpn
> root@vostro:~# ip rule add fwmark 0x1 table tovpn
>
> Now connections from vostro to 190.93.249.164 still make it to ebox, and
> from ebox to 190.93.249.164, but the answers get stuck on ebox:
>
> Sep 13 04:47:53 ebox kernel: Rejected forwarding: IN=eth0 OUT=eth0 MAC=f2:3c:91:69:db:07:84:78:ac:0d:79:c1:08:00 SRC=190.93.249.164 DST=192.168.17.47 LEN=60 TOS=0x00 PREC=0x00 TTL=58 ID=0 DF PROTO=TCP SPT=80 DPT=39024 WINDOW=14480 RES=0x00 ACK SYN URGP=0
>
> It seems that ebox tries to send the packet destined to go trough the
> rath to eth0 instead, and consequency rejects them because forwarding is
> only enabled from eth0 to rath.
>
> However, this only happens when vostro has the gateway route set in a
> special routing table rather than the default table -- but how does ebox
> even know about that?
>
> Can someone explain to me what is happening here and why?
>
>
>
> Best,
>
> -Nikolaus
>
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: Wrong routing when combining ip rule with SNAT
2013-09-13 6:26 ` Vigneswaran R
@ 2013-09-13 16:09 ` Nikolaus Rath
2013-09-13 22:03 ` Nikolaus Rath
2013-09-14 13:41 ` Pascal Hambourg
1 sibling, 1 reply; 22+ messages in thread
From: Nikolaus Rath @ 2013-09-13 16:09 UTC (permalink / raw)
To: Vigneswaran R; +Cc: netfilter
Vigneswaran R <vignesh@atc.tcs.com> writes:
> On 09/13/2013 10:40 AM, Nikolaus Rath wrote:
>> I've enabled packet forwarding and SNAT on the "ebox" computer as
>> follows:
>>
>> root@ebox:~# ip route
>> default via 23.92.25.1 dev eth0
>> 23.92.25.0/24 dev eth0 proto kernel scope link src 23.92.25.96
>> 192.168.12.0/24 dev rath proto kernel scope link src 192.168.12.1
>>
>> root@ebox:~# iptables -L -n -v
>> Chain INPUT (policy ACCEPT 1314 packets, 1736K bytes)
>> pkts bytes target prot opt in out source destination
>>
>> Chain FORWARD (policy DROP 0 packets, 0 bytes)
>> pkts bytes target prot opt in out source destination
>> 150K 62M ACCEPT all -- rath eth0 0.0.0.0/0 0.0.0.0/0
>> 86746 200M ACCEPT all -- eth0 rath 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED
>> 319 22076 LOG all -- * * 0.0.0.0/0 0.0.0.0/0 limit: avg 1/min burst 30 LOG flags 0 level 4 prefix "Rejected forwarding: "
>> 393 26172 REJECT all -- * * 0.0.0.0/0 0.0.0.0/0 reject-with icmp-net-prohibited
>>
>> Chain OUTPUT (policy ACCEPT 1142 packets, 2412K bytes)
>> pkts bytes target prot opt in out source destination
>> root@ebox:~# iptables -t nat -L -n -v
>> Chain PREROUTING (policy ACCEPT 36378 packets, 2383K bytes)
>>
>> Chain INPUT (policy ACCEPT 19982 packets, 1334K bytes)
>> pkts bytes target prot opt in out source destination
>>
>> Chain OUTPUT (policy ACCEPT 61430 packets, 4601K bytes)
>> pkts bytes target prot opt in out source destination
>>
>> Chain POSTROUTING (policy ACCEPT 8333 packets, 564K bytes)
>> pkts bytes target prot opt in out source destination
>> 69488 5081K SNAT all -- * eth0 0.0.0.0/0 0.0.0.0/0 to:23.92.25.96
>>
>> >From a second computer "vostro", I can now use ebox as a
>> gateway:
>>
>> root@vostro:~# ip route add 190.93.249.164 via 192.168.12.1
>>
>> This works fine, now connections to whatismyip.com (190.93.249.164) go
>> through ebox.
>>
>> However, when I try to be a bit more selective on vostro and use a
>> special routing table, things don't work anymore:
>>
>> root@vostro:~# iptables -t mangle -L -n
>> Chain PREROUTING (policy ACCEPT)
>> target prot opt source destination
>>
>> Chain INPUT (policy ACCEPT)
>> target prot opt source destination
>>
>> Chain FORWARD (policy ACCEPT)
>> target prot opt source destination
>>
>> Chain OUTPUT (policy ACCEPT)
>> target prot opt source destination
>> MARK tcp -- 0.0.0.0/0 190.93.249.164 tcp dpt:80 MARK set 0x1
>> LOG tcp -- 0.0.0.0/0 190.93.249.164 tcp dpt:80 LOG flags 0 level 4 prefix "marked: "
>>
>> Chain POSTROUTING (policy ACCEPT)
>> target prot opt source destination
>>
>> root@vostro:~# ip route del 190.93.249.164 via 192.168.12.1
>> root@vostro:~# ip route add default via 192.168.12.1 table tovpn
>> root@vostro:~# ip rule add fwmark 0x1 table tovpn
>>
>> Now connections from vostro to 190.93.249.164 still make it to ebox, and
>> from ebox to 190.93.249.164, but the answers get stuck on ebox:
>>
>> Sep 13 04:47:53 ebox kernel: Rejected forwarding: IN=eth0 OUT=eth0 MAC=f2:3c:91:69:db:07:84:78:ac:0d:79:c1:08:00 SRC=190.93.249.164 DST=192.168.17.47 LEN=60 TOS=0x00 PREC=0x00 TTL=58 ID=0 DF PROTO=TCP SPT=80 DPT=39024 WINDOW=14480 RES=0x00 ACK SYN URGP=0
>>
>> It seems that ebox tries to send the packet destined to go trough the
>> rath to eth0 instead, and consequency rejects them because forwarding is
>> only enabled from eth0 to rath.
>>
>> However, this only happens when vostro has the gateway route set in a
>> special routing table rather than the default table -- but how does ebox
>> even know about that?
>>
>> Can someone explain to me what is happening here and why?
>
> I have a doubt. It seems, rath of ebox is assigned with IP address in
> the range 192.168.12.0/24.
> However, IP address of vostro seems to be
> 192.168.17.47 (assuming /24). Ebox doesn't have any route to this
> range. So it try to use default route via eth0.
>
> What I assume is, 'vostro' has IP addresses in (atleast) two ranges
> (192.168.12.0/24, 192.168.17.0/24).
That's correct.
nikratio@vostro:~$ ip addr
5: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP
link/ether c8:60:00:bf:a2:7f brd ff:ff:ff:ff:ff:ff
inet 192.168.17.47/24 brd 192.168.17.255 scope global br0
valid_lft forever preferred_lft forever
inet6 fe80::ca60:ff:febf:a27f/64 scope link
valid_lft forever preferred_lft forever
6: rath: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN qlen 500
link/none
inet 192.168.12.4/24 scope global rath
valid_lft forever preferred_lft forever
> In the default routing table, the src IP is set to 192.168.12.x (for
> the packets originating from vostro). However, the 'tovpn' table
> didn't specify the src IP. So, when the 'tovpn' table is being used,
> the packets may have got the src IP as 192.168.17.x.
Hmm. This would make sense, but looking at the default table, the source
address for the route via 192.168.12.x is actually also not set:
nikratio@vostro:~$ ip route
default via 192.168.17.1 dev br0
190.93.249.164 via 192.168.12.1 dev rath
192.168.12.0/24 dev rath proto kernel scope link src 192.168.12.4
192.168.17.0/24 dev br0 proto kernel scope link src 192.168.17.47
This works just fine, despite the entry having no source address. So why
is it working in the default table, but not in the tovpn table?
> I think, you can avoid this by explicitly specifying the src IP when
> adding the route to 'tovpn' table,
>
> ip route add default via 192.168.12.1 src 192.168.12.x table tovpn
I'll of course try this nevertheless, thanks!
Best,
Nikolaus
--
Encrypted emails preferred.
PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6 02CF A9AD B7F8 AE4E 425C
»Time flies like an arrow, fruit flies like a Banana.«
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: Wrong routing when combining ip rule with SNAT
2013-09-13 16:09 ` Nikolaus Rath
@ 2013-09-13 22:03 ` Nikolaus Rath
0 siblings, 0 replies; 22+ messages in thread
From: Nikolaus Rath @ 2013-09-13 22:03 UTC (permalink / raw)
To: netfilter
Nikolaus Rath <Nikolaus@rath.org> writes:
> Vigneswaran R <vignesh@atc.tcs.com> writes:
>> On 09/13/2013 10:40 AM, Nikolaus Rath wrote:
>>> I've enabled packet forwarding and SNAT on the "ebox" computer as
>>> follows:
>>>
>>> root@ebox:~# ip route
>>> default via 23.92.25.1 dev eth0
>>> 23.92.25.0/24 dev eth0 proto kernel scope link src 23.92.25.96
>>> 192.168.12.0/24 dev rath proto kernel scope link src 192.168.12.1
>>>
>>> root@ebox:~# iptables -L -n -v
>>> Chain INPUT (policy ACCEPT 1314 packets, 1736K bytes)
>>> pkts bytes target prot opt in out source destination
>>>
>>> Chain FORWARD (policy DROP 0 packets, 0 bytes)
>>> pkts bytes target prot opt in out source destination
>>> 150K 62M ACCEPT all -- rath eth0 0.0.0.0/0 0.0.0.0/0
>>> 86746 200M ACCEPT all -- eth0 rath 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED
>>> 319 22076 LOG all -- * * 0.0.0.0/0 0.0.0.0/0 limit: avg 1/min burst 30 LOG flags 0 level 4 prefix "Rejected forwarding: "
>>> 393 26172 REJECT all -- * * 0.0.0.0/0 0.0.0.0/0 reject-with icmp-net-prohibited
>>>
>>> Chain OUTPUT (policy ACCEPT 1142 packets, 2412K bytes)
>>> pkts bytes target prot opt in out source destination
>>> root@ebox:~# iptables -t nat -L -n -v
>>> Chain PREROUTING (policy ACCEPT 36378 packets, 2383K bytes)
>>>
>>> Chain INPUT (policy ACCEPT 19982 packets, 1334K bytes)
>>> pkts bytes target prot opt in out source destination
>>>
>>> Chain OUTPUT (policy ACCEPT 61430 packets, 4601K bytes)
>>> pkts bytes target prot opt in out source destination
>>>
>>> Chain POSTROUTING (policy ACCEPT 8333 packets, 564K bytes)
>>> pkts bytes target prot opt in out source destination
>>> 69488 5081K SNAT all -- * eth0 0.0.0.0/0 0.0.0.0/0 to:23.92.25.96
>>>
>>> >From a second computer "vostro", I can now use ebox as a
>>> gateway:
>>>
>>> root@vostro:~# ip route add 190.93.249.164 via 192.168.12.1
>>>
>>> This works fine, now connections to whatismyip.com (190.93.249.164) go
>>> through ebox.
>>>
>>> However, when I try to be a bit more selective on vostro and use a
>>> special routing table, things don't work anymore:
>>>
>>> root@vostro:~# iptables -t mangle -L -n
>>> Chain PREROUTING (policy ACCEPT)
>>> target prot opt source destination
>>>
>>> Chain INPUT (policy ACCEPT)
>>> target prot opt source destination
>>>
>>> Chain FORWARD (policy ACCEPT)
>>> target prot opt source destination
>>>
>>> Chain OUTPUT (policy ACCEPT)
>>> target prot opt source destination
>>> MARK tcp -- 0.0.0.0/0 190.93.249.164 tcp dpt:80 MARK set 0x1
>>> LOG tcp -- 0.0.0.0/0 190.93.249.164 tcp dpt:80 LOG flags 0 level 4 prefix "marked: "
>>>
>>> Chain POSTROUTING (policy ACCEPT)
>>> target prot opt source destination
>>>
>>> root@vostro:~# ip route del 190.93.249.164 via 192.168.12.1
>>> root@vostro:~# ip route add default via 192.168.12.1 table tovpn
>>> root@vostro:~# ip rule add fwmark 0x1 table tovpn
>>>
>>> Now connections from vostro to 190.93.249.164 still make it to ebox, and
>>> from ebox to 190.93.249.164, but the answers get stuck on ebox:
>>>
>>> Sep 13 04:47:53 ebox kernel: Rejected forwarding: IN=eth0 OUT=eth0 MAC=f2:3c:91:69:db:07:84:78:ac:0d:79:c1:08:00 SRC=190.93.249.164 DST=192.168.17.47 LEN=60 TOS=0x00 PREC=0x00 TTL=58 ID=0 DF PROTO=TCP SPT=80 DPT=39024 WINDOW=14480 RES=0x00 ACK SYN URGP=0
>>>
>>> It seems that ebox tries to send the packet destined to go trough the
>>> rath to eth0 instead, and consequency rejects them because forwarding is
>>> only enabled from eth0 to rath.
>>>
>>> However, this only happens when vostro has the gateway route set in a
>>> special routing table rather than the default table -- but how does ebox
>>> even know about that?
>>>
>>> Can someone explain to me what is happening here and why?
>>
>> I have a doubt. It seems, rath of ebox is assigned with IP address in
>> the range 192.168.12.0/24.
>> However, IP address of vostro seems to be
>> 192.168.17.47 (assuming /24). Ebox doesn't have any route to this
>> range. So it try to use default route via eth0.
>>
>> What I assume is, 'vostro' has IP addresses in (atleast) two ranges
>> (192.168.12.0/24, 192.168.17.0/24).
>
> That's correct.
>
> nikratio@vostro:~$ ip addr
> 5: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP
> link/ether c8:60:00:bf:a2:7f brd ff:ff:ff:ff:ff:ff
> inet 192.168.17.47/24 brd 192.168.17.255 scope global br0
> valid_lft forever preferred_lft forever
> inet6 fe80::ca60:ff:febf:a27f/64 scope link
> valid_lft forever preferred_lft forever
> 6: rath: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN qlen 500
> link/none
> inet 192.168.12.4/24 scope global rath
> valid_lft forever preferred_lft forever
>
>> In the default routing table, the src IP is set to 192.168.12.x (for
>> the packets originating from vostro). However, the 'tovpn' table
>> didn't specify the src IP. So, when the 'tovpn' table is being used,
>> the packets may have got the src IP as 192.168.17.x.
>
> Hmm. This would make sense, but looking at the default table, the source
> address for the route via 192.168.12.x is actually also not set:
>
> nikratio@vostro:~$ ip route
> default via 192.168.17.1 dev br0
> 190.93.249.164 via 192.168.12.1 dev rath
> 192.168.12.0/24 dev rath proto kernel scope link src 192.168.12.4
> 192.168.17.0/24 dev br0 proto kernel scope link src 192.168.17.47
>
> This works just fine, despite the entry having no source address. So why
> is it working in the default table, but not in the tovpn table?
>
>
>> I think, you can avoid this by explicitly specifying the src IP when
>> adding the route to 'tovpn' table,
>>
>> ip route add default via 192.168.12.1 src 192.168.12.x table tovpn
>
>
> I'll of course try this nevertheless, thanks!
Nope, this doesn't help. It's still going out with the wrong src:
root@vostro:~# ip route list table tovpn
190.93.249.164 via 192.168.12.1 dev rath src 192.168.12.2
root@ebox:~# tail /var/log/kern.log
Sep 13 21:28:57 ebox kernel: Rejected forwarding: IN=eth0 OUT=eth0 MAC=f2:3c:91:69:db:07:84:78:ac:0d:79:c1:08:00 SRC=91.189.89.199 DST=192.168.17.47 LEN=76 TOS=0x00 PREC=0x00 TTL=47 ID=0 DF PROTO=UDP SPT=123 DPT=123 LEN=56
Any ideas?
Best,
Nikolaus
--
Encrypted emails preferred.
PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6 02CF A9AD B7F8 AE4E 425C
»Time flies like an arrow, fruit flies like a Banana.«
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: Wrong routing when combining ip rule with SNAT
2013-09-13 6:26 ` Vigneswaran R
2013-09-13 16:09 ` Nikolaus Rath
@ 2013-09-14 13:41 ` Pascal Hambourg
2013-09-14 15:40 ` Nikolaus Rath
1 sibling, 1 reply; 22+ messages in thread
From: Pascal Hambourg @ 2013-09-14 13:41 UTC (permalink / raw)
To: netfilter
Vigneswaran R a écrit :
> Hello Nikolaus,
>
> I have a doubt. It seems, rath of ebox is assigned with IP address in
> the range 192.168.12.0/24. However, IP address of vostro seems to be
> 192.168.17.47 (assuming /24). Ebox doesn't have any route to this range.
> So it try to use default route via eth0.
Correct.
> What I assume is, 'vostro' has IP addresses in (atleast) two ranges
> (192.168.12.0/24, 192.168.17.0/24). In the default routing table, the
> src IP is set to 192.168.12.x (for the packets originating from vostro).
> However, the 'tovpn' table didn't specify the src IP. So, when the
> 'tovpn' table is being used, the packets may have got the src IP as
> 192.168.17.x.
>
> I think, you can avoid this by explicitly specifying the src IP when
> adding the route to 'tovpn' table,
>
> ip route add default via 192.168.12.1 src 192.168.12.x table tovpn
This won't work. It's too late. The source address has already been
selected by the TCP layer when the packet was created and won't be
changed when the packet is re-routed due to the mark.
Possible workarounds :
- Add a route on ebox to let it know that 192.168.17.47 is reachable
through rath. My favourite choice.
- Use SNAT to the address of the output interface on vostro.
- Use connection mark (connmark) by iptables on ebox so that replies to
original packets received on a given interface are forwarded to the same
interface.
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: Wrong routing when combining ip rule with SNAT
2013-09-14 13:41 ` Pascal Hambourg
@ 2013-09-14 15:40 ` Nikolaus Rath
2013-09-14 17:17 ` Pascal Hambourg
2013-09-16 7:14 ` Vigneswaran R
0 siblings, 2 replies; 22+ messages in thread
From: Nikolaus Rath @ 2013-09-14 15:40 UTC (permalink / raw)
To: Pascal Hambourg; +Cc: netfilter
On 09/14/2013 06:41 AM, Pascal Hambourg wrote:
> Vigneswaran R a écrit :
>> Hello Nikolaus,
>>
>> I have a doubt. It seems, rath of ebox is assigned with IP address in
>> the range 192.168.12.0/24. However, IP address of vostro seems to be
>> 192.168.17.47 (assuming /24). Ebox doesn't have any route to this range.
>> So it try to use default route via eth0.
>
> Correct.
>
>> What I assume is, 'vostro' has IP addresses in (atleast) two ranges
>> (192.168.12.0/24, 192.168.17.0/24). In the default routing table, the
>> src IP is set to 192.168.12.x (for the packets originating from vostro).
>> However, the 'tovpn' table didn't specify the src IP. So, when the
>> 'tovpn' table is being used, the packets may have got the src IP as
>> 192.168.17.x.
>>
>> I think, you can avoid this by explicitly specifying the src IP when
>> adding the route to 'tovpn' table,
>>
>> ip route add default via 192.168.12.1 src 192.168.12.x table tovpn
>
> This won't work. It's too late. The source address has already been
> selected by the TCP layer when the packet was created and won't be
> changed when the packet is re-routed due to the mark.
I see. Out of curiosity: how is the source address selected when the
packet is created, and in which situation would the source entry in the
routing table become effective?
> Possible workarounds :
> - Add a route on ebox to let it know that 192.168.17.47 is reachable
> through rath. My favourite choice.
Yes, that would be the simplest solution. But the problem is that this
address varies depending how and where vostro got its connectivity. Or
did you mean something other than a static extra route?
> - Use SNAT to the address of the output interface on vostro.
Sounds ugly...
> - Use connection mark (connmark) by iptables on ebox so that replies to
> original packets received on a given interface are forwarded to the same
> interface.
I guess I'll try this. Could you give some more details? I'm not sure
how to create a rule that *changes* the outgoing interface.
Best,
-Nikolaus
--
»Time flies like an arrow, fruit flies like a Banana.«
PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6 02CF A9AD B7F8 AE4E 425C
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: Wrong routing when combining ip rule with SNAT
2013-09-14 15:40 ` Nikolaus Rath
@ 2013-09-14 17:17 ` Pascal Hambourg
2013-09-16 7:14 ` Vigneswaran R
1 sibling, 0 replies; 22+ messages in thread
From: Pascal Hambourg @ 2013-09-14 17:17 UTC (permalink / raw)
To: netfilter
Nikolaus Rath a écrit :
>
> I see. Out of curiosity: how is the source address selected when the
> packet is created, and in which situation would the source entry in the
> routing table become effective?
The source address selection process is quite complicated and I'm not
sure I know all the details. As far as I know, the selected source
address is, in decreasing order of priority :
- the source address specified by the sending process/socket
- the source address specified in the routing table entry used for the
destination
- the address of the output interface matching the destination prefix of
the routing table entry
- the address of the output interface
- any other suitable address of any other interface (i.e. not a loopback
address if the destination is not local)
>> Possible workarounds :
>> - Add a route on ebox to let it know that 192.168.17.47 is reachable
>> through rath. My favourite choice.
>
> Yes, that would be the simplest solution. But the problem is that this
> address varies depending how and where vostro got its connectivity. Or
> did you mean something other than a static extra route?
I mean any means to create that route. May be dynamic, using a routing
protocol. However if ath is a tunnel using the other address for
encapsulation, this workaround may not be applicable easily
(encapsulated packets must still be routed normally).
>> - Use SNAT to the address of the output interface on vostro.
>
> Sounds ugly...
Yes. But simple to setup and effective.
>> - Use connection mark (connmark) by iptables on ebox so that replies to
>> original packets received on a given interface are forwarded to the same
>> interface.
>
> I guess I'll try this. Could you give some more details? I'm not sure
> how to create a rule that *changes* the outgoing interface.
It's not very different from marking packets for advanced routing.
I haven't done that recently, but here is a basic overview :
# mark the connections of packets received on rath
iptables -t mangle -A PREROUTING -i rath -j CONNMARK --set-mark 1
# copy the connection mark to the packets received on eth0
iptables -t mangle -A PREROUTING -i eth0 -j CONNMARK --restore-mark
# use the mark to route packets received on eth0
ip rule add fwmark 1 dev eth0 table tovpn
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: Wrong routing when combining ip rule with SNAT
2013-09-14 15:40 ` Nikolaus Rath
2013-09-14 17:17 ` Pascal Hambourg
@ 2013-09-16 7:14 ` Vigneswaran R
1 sibling, 0 replies; 22+ messages in thread
From: Vigneswaran R @ 2013-09-16 7:14 UTC (permalink / raw)
To: Nikolaus Rath; +Cc: netfilter
On 09/14/2013 09:10 PM, Nikolaus Rath wrote:
> On 09/14/2013 06:41 AM, Pascal Hambourg wrote:
>> Vigneswaran R a écrit :
>>> Hello Nikolaus,
>>>
>>> I have a doubt. It seems, rath of ebox is assigned with IP address in
>>> the range 192.168.12.0/24. However, IP address of vostro seems to be
>>> 192.168.17.47 (assuming /24). Ebox doesn't have any route to this range.
>>> So it try to use default route via eth0.
>> Correct.
>>
>>> What I assume is, 'vostro' has IP addresses in (atleast) two ranges
>>> (192.168.12.0/24, 192.168.17.0/24). In the default routing table, the
>>> src IP is set to 192.168.12.x (for the packets originating from vostro).
>>> However, the 'tovpn' table didn't specify the src IP. So, when the
>>> 'tovpn' table is being used, the packets may have got the src IP as
>>> 192.168.17.x.
>>>
>>> I think, you can avoid this by explicitly specifying the src IP when
>>> adding the route to 'tovpn' table,
>>>
>>> ip route add default via 192.168.12.1 src 192.168.12.x table tovpn
>> This won't work. It's too late. The source address has already been
>> selected by the TCP layer when the packet was created and won't be
>> changed when the packet is re-routed due to the mark.
> I see. Out of curiosity: how is the source address selected when the
> packet is created, and in which situation would the source entry in the
> routing table become effective?
Following is my understanding.
For a packet created by local process, it seems, there are two places in
which the IP Rules have been evaluated and various routing tables have
been consulted.
1. Just before getting into OUTPUT chain
-- Here, the outgoing Interface, SRC IP have been decided for the packet
2. After processing OUTPUT chain and just before getting into
POSTROUTING chain
-- Here, the outgoing Interface is getting modified, if necessary.
However, the SRC IP didn't.
So, in our case,
1. Just before getting into OUTPUT chain
-- At this point, the IP Rule (based on FWMARK) doesn't evaluate to
true as the MARK is not yet set.
-- Due to that 'tovpn' table is not consulted
-- This means, there is no explicit route to 190.93.249.164. So, it
uses default route 'via 192.168.17.1 dev br0' (based on main table entry).
-- This results in SRC IP being 192.168.17.47 and the outgoing
Interface being 'br0'.
2. After processing OUTPUT chain and just before getting into
POSTROUTING chain
-- At this point, the IP Rule (based on FWMARK) evaluates to true as
the MARK is added by OUTPUT chain.
-- Now, the 'tovpn' table is also consulted which has route to
190.93.249.164 'via 192.168.12.1 dev rath'.
-- In my observation, only the outgoing Interface is updated at this
point.
-- This results in SRC IP being 192.168.17.47 and the outgoing
Interface being 'rath'.
The following workaround solves our purpose. However ugly and not
scalable, I think.
Using some condition for IP Rule which evaluates to true at the 1st step
itself. eg., (matching destination IP).
ip rule add to 190.93.249.164 table tovpn
Regards,
Vignesh
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: Wrong routing when combining ip rule with SNAT
2013-09-13 5:10 Wrong routing when combining ip rule with SNAT Nikolaus Rath
2013-09-13 6:26 ` Vigneswaran R
@ 2013-09-16 23:38 ` Eliezer Croitoru
2013-09-17 0:58 ` Nikolaus Rath
1 sibling, 1 reply; 22+ messages in thread
From: Eliezer Croitoru @ 2013-09-16 23:38 UTC (permalink / raw)
To: netfilter
Hey there,
What are you trying to achieve exactly?
I tried to understand the network topology and the network issues but
since you did not marked a target to what you want to actually get.
There is an option to actually understand the situation you are in by
just describing the need and the situation and then continue from there.
Hope for the best
Eliezer
On 09/13/2013 08:10 AM, Nikolaus Rath wrote:
> Hello,
>
> Thanks for working on this great networking stack!
>
> I'm trying to set up a configuration with SNAT and routing rules, but
> I'm having weird problems that I do not understand:
>
> I've enabled packet forwarding and SNAT on the "ebox" computer as
> follows:
>
> root@ebox:~# ip route
> default via 23.92.25.1 dev eth0
> 23.92.25.0/24 dev eth0 proto kernel scope link src 23.92.25.96
> 192.168.12.0/24 dev rath proto kernel scope link src 192.168.12.1
>
> root@ebox:~# iptables -L -n -v
> Chain INPUT (policy ACCEPT 1314 packets, 1736K bytes)
> pkts bytes target prot opt in out source destination
>
> Chain FORWARD (policy DROP 0 packets, 0 bytes)
> pkts bytes target prot opt in out source destination
> 150K 62M ACCEPT all -- rath eth0 0.0.0.0/0 0.0.0.0/0
> 86746 200M ACCEPT all -- eth0 rath 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED
> 319 22076 LOG all -- * * 0.0.0.0/0 0.0.0.0/0 limit: avg 1/min burst 30 LOG flags 0 level 4 prefix "Rejected forwarding: "
> 393 26172 REJECT all -- * * 0.0.0.0/0 0.0.0.0/0 reject-with icmp-net-prohibited
>
> Chain OUTPUT (policy ACCEPT 1142 packets, 2412K bytes)
> pkts bytes target prot opt in out source destination
>
> root@ebox:~# iptables -t nat -L -n -v
> Chain PREROUTING (policy ACCEPT 36378 packets, 2383K bytes)
>
> Chain INPUT (policy ACCEPT 19982 packets, 1334K bytes)
> pkts bytes target prot opt in out source destination
>
> Chain OUTPUT (policy ACCEPT 61430 packets, 4601K bytes)
> pkts bytes target prot opt in out source destination
>
> Chain POSTROUTING (policy ACCEPT 8333 packets, 564K bytes)
> pkts bytes target prot opt in out source destination
> 69488 5081K SNAT all -- * eth0 0.0.0.0/0 0.0.0.0/0 to:23.92.25.96
>
>
> From a second computer "vostro", I can now use ebox as a gateway:
>
> root@vostro:~# ip route add 190.93.249.164 via 192.168.12.1
>
> This works fine, now connections to whatismyip.com (190.93.249.164) go
> through ebox.
>
> However, when I try to be a bit more selective on vostro and use a
> special routing table, things don't work anymore:
>
> root@vostro:~# iptables -t mangle -L -n
> Chain PREROUTING (policy ACCEPT)
> target prot opt source destination
>
> Chain INPUT (policy ACCEPT)
> target prot opt source destination
>
> Chain FORWARD (policy ACCEPT)
> target prot opt source destination
>
> Chain OUTPUT (policy ACCEPT)
> target prot opt source destination
> MARK tcp -- 0.0.0.0/0 190.93.249.164 tcp dpt:80 MARK set 0x1
> LOG tcp -- 0.0.0.0/0 190.93.249.164 tcp dpt:80 LOG flags 0 level 4 prefix "marked: "
>
> Chain POSTROUTING (policy ACCEPT)
> target prot opt source destination
>
> root@vostro:~# ip route del 190.93.249.164 via 192.168.12.1
> root@vostro:~# ip route add default via 192.168.12.1 table tovpn
> root@vostro:~# ip rule add fwmark 0x1 table tovpn
>
> Now connections from vostro to 190.93.249.164 still make it to ebox, and
> from ebox to 190.93.249.164, but the answers get stuck on ebox:
>
> Sep 13 04:47:53 ebox kernel: Rejected forwarding: IN=eth0 OUT=eth0 MAC=f2:3c:91:69:db:07:84:78:ac:0d:79:c1:08:00 SRC=190.93.249.164 DST=192.168.17.47 LEN=60 TOS=0x00 PREC=0x00 TTL=58 ID=0 DF PROTO=TCP SPT=80 DPT=39024 WINDOW=14480 RES=0x00 ACK SYN URGP=0
>
> It seems that ebox tries to send the packet destined to go trough the
> rath to eth0 instead, and consequency rejects them because forwarding is
> only enabled from eth0 to rath.
>
> However, this only happens when vostro has the gateway route set in a
> special routing table rather than the default table -- but how does ebox
> even know about that?
>
> Can someone explain to me what is happening here and why?
>
>
>
> Best,
>
> -Nikolaus
>
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: Wrong routing when combining ip rule with SNAT
2013-09-16 23:38 ` Eliezer Croitoru
@ 2013-09-17 0:58 ` Nikolaus Rath
2013-09-17 12:35 ` Alex Bligh
` (2 more replies)
0 siblings, 3 replies; 22+ messages in thread
From: Nikolaus Rath @ 2013-09-17 0:58 UTC (permalink / raw)
To: netfilter
Hi Eliezer,
I have a VPN connection, and I want to tunnel everything through the VPN
node -- except, of course, the VPN connection itself.
The hard part is to also tunnel non-VPN connections to the VPN node
itself. In other words how do I make sure that every connection to the
external ip of the VPN node is tunneled through its internal ip --
except for the packets that form the tunnel itself?
My idea was install a default route to the internal ip of the VPN node,
use iptables to mark the VPN connections and then set up a special
routing table for those. But maybe there's an easier way?
Best,
Nikolaus
Eliezer Croitoru <eliezer@ngtech.co.il> writes:
> Hey there,
>
> What are you trying to achieve exactly?
> I tried to understand the network topology and the network issues but
> since you did not marked a target to what you want to actually get.
> There is an option to actually understand the situation you are in by
> just describing the need and the situation and then continue from there.
>
> Hope for the best
> Eliezer
>
> On 09/13/2013 08:10 AM, Nikolaus Rath wrote:
>> Hello,
>>
>> Thanks for working on this great networking stack!
>>
>> I'm trying to set up a configuration with SNAT and routing rules, but
>> I'm having weird problems that I do not understand:
>>
>> I've enabled packet forwarding and SNAT on the "ebox" computer as
>> follows:
>>
>> root@ebox:~# ip route
>> default via 23.92.25.1 dev eth0
>> 23.92.25.0/24 dev eth0 proto kernel scope link src 23.92.25.96
>> 192.168.12.0/24 dev rath proto kernel scope link src 192.168.12.1
>>
>> root@ebox:~# iptables -L -n -v
>> Chain INPUT (policy ACCEPT 1314 packets, 1736K bytes)
>> pkts bytes target prot opt in out source destination
>>
>> Chain FORWARD (policy DROP 0 packets, 0 bytes)
>> pkts bytes target prot opt in out source destination
>> 150K 62M ACCEPT all -- rath eth0 0.0.0.0/0 0.0.0.0/0
>> 86746 200M ACCEPT all -- eth0 rath 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED
>> 319 22076 LOG all -- * * 0.0.0.0/0 0.0.0.0/0 limit: avg 1/min burst 30 LOG flags 0 level 4 prefix "Rejected forwarding: "
>> 393 26172 REJECT all -- * * 0.0.0.0/0 0.0.0.0/0 reject-with icmp-net-prohibited
>>
>> Chain OUTPUT (policy ACCEPT 1142 packets, 2412K bytes)
>> pkts bytes target prot opt in out source destination
>>
>> root@ebox:~# iptables -t nat -L -n -v
>> Chain PREROUTING (policy ACCEPT 36378 packets, 2383K bytes)
>>
>> Chain INPUT (policy ACCEPT 19982 packets, 1334K bytes)
>> pkts bytes target prot opt in out source destination
>>
>> Chain OUTPUT (policy ACCEPT 61430 packets, 4601K bytes)
>> pkts bytes target prot opt in out source destination
>>
>> Chain POSTROUTING (policy ACCEPT 8333 packets, 564K bytes)
>> pkts bytes target prot opt in out source destination
>> 69488 5081K SNAT all -- * eth0 0.0.0.0/0 0.0.0.0/0 to:23.92.25.96
>>
>>
>> From a second computer "vostro", I can now use ebox as a gateway:
>>
>> root@vostro:~# ip route add 190.93.249.164 via 192.168.12.1
>>
>> This works fine, now connections to whatismyip.com (190.93.249.164) go
>> through ebox.
>>
>> However, when I try to be a bit more selective on vostro and use a
>> special routing table, things don't work anymore:
>>
>> root@vostro:~# iptables -t mangle -L -n
>> Chain PREROUTING (policy ACCEPT)
>> target prot opt source destination
>>
>> Chain INPUT (policy ACCEPT)
>> target prot opt source destination
>>
>> Chain FORWARD (policy ACCEPT)
>> target prot opt source destination
>>
>> Chain OUTPUT (policy ACCEPT)
>> target prot opt source destination
>> MARK tcp -- 0.0.0.0/0 190.93.249.164 tcp dpt:80 MARK set 0x1
>> LOG tcp -- 0.0.0.0/0 190.93.249.164 tcp dpt:80 LOG flags 0 level 4 prefix "marked: "
>>
>> Chain POSTROUTING (policy ACCEPT)
>> target prot opt source destination
>>
>> root@vostro:~# ip route del 190.93.249.164 via 192.168.12.1
>> root@vostro:~# ip route add default via 192.168.12.1 table tovpn
>> root@vostro:~# ip rule add fwmark 0x1 table tovpn
>>
>> Now connections from vostro to 190.93.249.164 still make it to ebox, and
>> from ebox to 190.93.249.164, but the answers get stuck on ebox:
>>
>> Sep 13 04:47:53 ebox kernel: Rejected forwarding: IN=eth0 OUT=eth0 MAC=f2:3c:91:69:db:07:84:78:ac:0d:79:c1:08:00 SRC=190.93.249.164 DST=192.168.17.47 LEN=60 TOS=0x00 PREC=0x00 TTL=58 ID=0 DF PROTO=TCP SPT=80 DPT=39024 WINDOW=14480 RES=0x00 ACK SYN URGP=0
>>
>> It seems that ebox tries to send the packet destined to go trough the
>> rath to eth0 instead, and consequency rejects them because forwarding is
>> only enabled from eth0 to rath.
>>
>> However, this only happens when vostro has the gateway route set in a
>> special routing table rather than the default table -- but how does ebox
>> even know about that?
>>
>> Can someone explain to me what is happening here and why?
>>
>>
>>
>> Best,
>>
>> -Nikolaus
>>
>
> --
> To unsubscribe from this list: send the line "unsubscribe netfilter" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
-Nikolaus
--
»Time flies like an arrow, fruit flies like a Banana.«
PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6 02CF A9AD B7F8 AE4E 425C
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: Wrong routing when combining ip rule with SNAT
2013-09-17 0:58 ` Nikolaus Rath
@ 2013-09-17 12:35 ` Alex Bligh
2013-09-17 23:23 ` Pascal Hambourg
2013-09-17 21:58 ` Eliezer Croitoru
2013-09-18 5:54 ` Vigneswaran R
2 siblings, 1 reply; 22+ messages in thread
From: Alex Bligh @ 2013-09-17 12:35 UTC (permalink / raw)
To: Nikolaus Rath; +Cc: Alex Bligh, netfilter
Nikolaus,
I don't think you need iptables. The way I've always done it is:
* Default route to the VPN device
* /32 route for the VPN endpoint out the physical interface to
the previous default route
The OS will take care of your source IP addresses as it will by
default set the source interface of outbound connections to the
IP address of the egress interface.
Alex
On 17 Sep 2013, at 01:58, Nikolaus Rath wrote:
> Hi Eliezer,
>
> I have a VPN connection, and I want to tunnel everything through the VPN
> node -- except, of course, the VPN connection itself.
>
> The hard part is to also tunnel non-VPN connections to the VPN node
> itself. In other words how do I make sure that every connection to the
> external ip of the VPN node is tunneled through its internal ip --
> except for the packets that form the tunnel itself?
>
> My idea was install a default route to the internal ip of the VPN node,
> use iptables to mark the VPN connections and then set up a special
> routing table for those. But maybe there's an easier way?
>
> Best,
> Nikolaus
>
> Eliezer Croitoru <eliezer@ngtech.co.il> writes:
>> Hey there,
>>
>> What are you trying to achieve exactly?
>> I tried to understand the network topology and the network issues but
>> since you did not marked a target to what you want to actually get.
>> There is an option to actually understand the situation you are in by
>> just describing the need and the situation and then continue from there.
>>
>> Hope for the best
>> Eliezer
>>
>> On 09/13/2013 08:10 AM, Nikolaus Rath wrote:
>>> Hello,
>>>
>>> Thanks for working on this great networking stack!
>>>
>>> I'm trying to set up a configuration with SNAT and routing rules, but
>>> I'm having weird problems that I do not understand:
>>>
>>> I've enabled packet forwarding and SNAT on the "ebox" computer as
>>> follows:
>>>
>>> root@ebox:~# ip route
>>> default via 23.92.25.1 dev eth0
>>> 23.92.25.0/24 dev eth0 proto kernel scope link src 23.92.25.96
>>> 192.168.12.0/24 dev rath proto kernel scope link src 192.168.12.1
>>>
>>> root@ebox:~# iptables -L -n -v
>>> Chain INPUT (policy ACCEPT 1314 packets, 1736K bytes)
>>> pkts bytes target prot opt in out source destination
>>>
>>> Chain FORWARD (policy DROP 0 packets, 0 bytes)
>>> pkts bytes target prot opt in out source destination
>>> 150K 62M ACCEPT all -- rath eth0 0.0.0.0/0 0.0.0.0/0
>>> 86746 200M ACCEPT all -- eth0 rath 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED
>>> 319 22076 LOG all -- * * 0.0.0.0/0 0.0.0.0/0 limit: avg 1/min burst 30 LOG flags 0 level 4 prefix "Rejected forwarding: "
>>> 393 26172 REJECT all -- * * 0.0.0.0/0 0.0.0.0/0 reject-with icmp-net-prohibited
>>>
>>> Chain OUTPUT (policy ACCEPT 1142 packets, 2412K bytes)
>>> pkts bytes target prot opt in out source destination
>>>
>>> root@ebox:~# iptables -t nat -L -n -v
>>> Chain PREROUTING (policy ACCEPT 36378 packets, 2383K bytes)
>>>
>>> Chain INPUT (policy ACCEPT 19982 packets, 1334K bytes)
>>> pkts bytes target prot opt in out source destination
>>>
>>> Chain OUTPUT (policy ACCEPT 61430 packets, 4601K bytes)
>>> pkts bytes target prot opt in out source destination
>>>
>>> Chain POSTROUTING (policy ACCEPT 8333 packets, 564K bytes)
>>> pkts bytes target prot opt in out source destination
>>> 69488 5081K SNAT all -- * eth0 0.0.0.0/0 0.0.0.0/0 to:23.92.25.96
>>>
>>>
>>> From a second computer "vostro", I can now use ebox as a gateway:
>>>
>>> root@vostro:~# ip route add 190.93.249.164 via 192.168.12.1
>>>
>>> This works fine, now connections to whatismyip.com (190.93.249.164) go
>>> through ebox.
>>>
>>> However, when I try to be a bit more selective on vostro and use a
>>> special routing table, things don't work anymore:
>>>
>>> root@vostro:~# iptables -t mangle -L -n
>>> Chain PREROUTING (policy ACCEPT)
>>> target prot opt source destination
>>>
>>> Chain INPUT (policy ACCEPT)
>>> target prot opt source destination
>>>
>>> Chain FORWARD (policy ACCEPT)
>>> target prot opt source destination
>>>
>>> Chain OUTPUT (policy ACCEPT)
>>> target prot opt source destination
>>> MARK tcp -- 0.0.0.0/0 190.93.249.164 tcp dpt:80 MARK set 0x1
>>> LOG tcp -- 0.0.0.0/0 190.93.249.164 tcp dpt:80 LOG flags 0 level 4 prefix "marked: "
>>>
>>> Chain POSTROUTING (policy ACCEPT)
>>> target prot opt source destination
>>>
>>> root@vostro:~# ip route del 190.93.249.164 via 192.168.12.1
>>> root@vostro:~# ip route add default via 192.168.12.1 table tovpn
>>> root@vostro:~# ip rule add fwmark 0x1 table tovpn
>>>
>>> Now connections from vostro to 190.93.249.164 still make it to ebox, and
>>> from ebox to 190.93.249.164, but the answers get stuck on ebox:
>>>
>>> Sep 13 04:47:53 ebox kernel: Rejected forwarding: IN=eth0 OUT=eth0 MAC=f2:3c:91:69:db:07:84:78:ac:0d:79:c1:08:00 SRC=190.93.249.164 DST=192.168.17.47 LEN=60 TOS=0x00 PREC=0x00 TTL=58 ID=0 DF PROTO=TCP SPT=80 DPT=39024 WINDOW=14480 RES=0x00 ACK SYN URGP=0
>>>
>>> It seems that ebox tries to send the packet destined to go trough the
>>> rath to eth0 instead, and consequency rejects them because forwarding is
>>> only enabled from eth0 to rath.
>>>
>>> However, this only happens when vostro has the gateway route set in a
>>> special routing table rather than the default table -- but how does ebox
>>> even know about that?
>>>
>>> Can someone explain to me what is happening here and why?
>>>
>>>
>>>
>>> Best,
>>>
>>> -Nikolaus
>>>
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe netfilter" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
>
> -Nikolaus
>
> --
> »Time flies like an arrow, fruit flies like a Banana.«
>
> PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6 02CF A9AD B7F8 AE4E 425C
>
> --
> To unsubscribe from this list: send the line "unsubscribe netfilter" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
>
--
Alex Bligh
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: Wrong routing when combining ip rule with SNAT
2013-09-17 0:58 ` Nikolaus Rath
2013-09-17 12:35 ` Alex Bligh
@ 2013-09-17 21:58 ` Eliezer Croitoru
2013-09-18 0:58 ` Nikolaus Rath
2013-09-18 5:54 ` Vigneswaran R
2 siblings, 1 reply; 22+ messages in thread
From: Eliezer Croitoru @ 2013-09-17 21:58 UTC (permalink / raw)
To: netfilter
Hey Nokolaus,
it really depends on the vpn tunnel software.
Some adds another interface per each session and others know that the
vpn is for a specific subnet.
From the kernel point of view since there is an endpoint to the tunnel
there should be a route either detected automatically by the kernel or
added when creating the tunnel device like in openvpn case.
the basic issue is not tunneling non-VPN connections to the VPN node
since most of the nodes should be routed by a single GW host that can
have a keep-alived or any other method to just verify that this host is
indeed up and running to prevent network outage.
A basic network setup should have on GW and only one.
If you do have a loaded network in hands make sure you study a bit first
on network infrastructure maintenance and not just "by the book" but
also hands-on experience that will give you much more power in hands
when handling a small blinking light.
In pptpd it works in another way which in not stricktly routing but also
masquerading in many cases unless you are using a default route and also
a range of dedicated addreses.
the basic setup should be in many cases:
gw <--<>--> network.
the GW has access to the network and also to the internet or any
external network.
when the network only GW is also the VPN server and NAT(MASQUERADE) all
other issues are smaller to handle.
Install a GW once or twice and you will see how that magic of the kernel
works fine.
the only main rule in this case that should be on is the MAQUERADE on
the external interface and the allow rules for the VPN.
Else then that just be creative on how much access you need to the network.
monowall was a nice firewall but pfsense is a much more sensible
solution for most networks.
If you want to use strictly Linux and not any BSD OpenSUSE, CentOS,
UBUNTU are great options but depends on your clients you will need to
choose the Distro.
(if you need more help just ask)
Eliezer
On 09/17/2013 03:58 AM, Nikolaus Rath wrote:
> Hi Eliezer,
>
> I have a VPN connection, and I want to tunnel everything through the VPN
> node -- except, of course, the VPN connection itself.
>
> The hard part is to also tunnel non-VPN connections to the VPN node
> itself. In other words how do I make sure that every connection to the
> external ip of the VPN node is tunneled through its internal ip --
> except for the packets that form the tunnel itself?
>
> My idea was install a default route to the internal ip of the VPN node,
> use iptables to mark the VPN connections and then set up a special
> routing table for those. But maybe there's an easier way?
>
> Best,
> Nikolaus
>
> Eliezer Croitoru <eliezer@ngtech.co.il> writes:
>> Hey there,
>>
>> What are you trying to achieve exactly?
>> I tried to understand the network topology and the network issues but
>> since you did not marked a target to what you want to actually get.
>> There is an option to actually understand the situation you are in by
>> just describing the need and the situation and then continue from there.
>>
>> Hope for the best
>> Eliezer
>>
>> On 09/13/2013 08:10 AM, Nikolaus Rath wrote:
>>> Hello,
>>>
>>> Thanks for working on this great networking stack!
>>>
>>> I'm trying to set up a configuration with SNAT and routing rules, but
>>> I'm having weird problems that I do not understand:
>>>
>>> I've enabled packet forwarding and SNAT on the "ebox" computer as
>>> follows:
>>>
>>> root@ebox:~# ip route
>>> default via 23.92.25.1 dev eth0
>>> 23.92.25.0/24 dev eth0 proto kernel scope link src 23.92.25.96
>>> 192.168.12.0/24 dev rath proto kernel scope link src 192.168.12.1
>>>
>>> root@ebox:~# iptables -L -n -v
>>> Chain INPUT (policy ACCEPT 1314 packets, 1736K bytes)
>>> pkts bytes target prot opt in out source destination
>>>
>>> Chain FORWARD (policy DROP 0 packets, 0 bytes)
>>> pkts bytes target prot opt in out source destination
>>> 150K 62M ACCEPT all -- rath eth0 0.0.0.0/0 0.0.0.0/0
>>> 86746 200M ACCEPT all -- eth0 rath 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED
>>> 319 22076 LOG all -- * * 0.0.0.0/0 0.0.0.0/0 limit: avg 1/min burst 30 LOG flags 0 level 4 prefix "Rejected forwarding: "
>>> 393 26172 REJECT all -- * * 0.0.0.0/0 0.0.0.0/0 reject-with icmp-net-prohibited
>>>
>>> Chain OUTPUT (policy ACCEPT 1142 packets, 2412K bytes)
>>> pkts bytes target prot opt in out source destination
>>>
>>> root@ebox:~# iptables -t nat -L -n -v
>>> Chain PREROUTING (policy ACCEPT 36378 packets, 2383K bytes)
>>>
>>> Chain INPUT (policy ACCEPT 19982 packets, 1334K bytes)
>>> pkts bytes target prot opt in out source destination
>>>
>>> Chain OUTPUT (policy ACCEPT 61430 packets, 4601K bytes)
>>> pkts bytes target prot opt in out source destination
>>>
>>> Chain POSTROUTING (policy ACCEPT 8333 packets, 564K bytes)
>>> pkts bytes target prot opt in out source destination
>>> 69488 5081K SNAT all -- * eth0 0.0.0.0/0 0.0.0.0/0 to:23.92.25.96
>>>
>>>
>>> From a second computer "vostro", I can now use ebox as a gateway:
>>>
>>> root@vostro:~# ip route add 190.93.249.164 via 192.168.12.1
>>>
>>> This works fine, now connections to whatismyip.com (190.93.249.164) go
>>> through ebox.
>>>
>>> However, when I try to be a bit more selective on vostro and use a
>>> special routing table, things don't work anymore:
>>>
>>> root@vostro:~# iptables -t mangle -L -n
>>> Chain PREROUTING (policy ACCEPT)
>>> target prot opt source destination
>>>
>>> Chain INPUT (policy ACCEPT)
>>> target prot opt source destination
>>>
>>> Chain FORWARD (policy ACCEPT)
>>> target prot opt source destination
>>>
>>> Chain OUTPUT (policy ACCEPT)
>>> target prot opt source destination
>>> MARK tcp -- 0.0.0.0/0 190.93.249.164 tcp dpt:80 MARK set 0x1
>>> LOG tcp -- 0.0.0.0/0 190.93.249.164 tcp dpt:80 LOG flags 0 level 4 prefix "marked: "
>>>
>>> Chain POSTROUTING (policy ACCEPT)
>>> target prot opt source destination
>>>
>>> root@vostro:~# ip route del 190.93.249.164 via 192.168.12.1
>>> root@vostro:~# ip route add default via 192.168.12.1 table tovpn
>>> root@vostro:~# ip rule add fwmark 0x1 table tovpn
>>>
>>> Now connections from vostro to 190.93.249.164 still make it to ebox, and
>>> from ebox to 190.93.249.164, but the answers get stuck on ebox:
>>>
>>> Sep 13 04:47:53 ebox kernel: Rejected forwarding: IN=eth0 OUT=eth0 MAC=f2:3c:91:69:db:07:84:78:ac:0d:79:c1:08:00 SRC=190.93.249.164 DST=192.168.17.47 LEN=60 TOS=0x00 PREC=0x00 TTL=58 ID=0 DF PROTO=TCP SPT=80 DPT=39024 WINDOW=14480 RES=0x00 ACK SYN URGP=0
>>>
>>> It seems that ebox tries to send the packet destined to go trough the
>>> rath to eth0 instead, and consequency rejects them because forwarding is
>>> only enabled from eth0 to rath.
>>>
>>> However, this only happens when vostro has the gateway route set in a
>>> special routing table rather than the default table -- but how does ebox
>>> even know about that?
>>>
>>> Can someone explain to me what is happening here and why?
>>>
>>>
>>>
>>> Best,
>>>
>>> -Nikolaus
>>>
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe netfilter" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
>
> -Nikolaus
>
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: Wrong routing when combining ip rule with SNAT
2013-09-17 12:35 ` Alex Bligh
@ 2013-09-17 23:23 ` Pascal Hambourg
2013-09-18 0:55 ` Nikolaus Rath
0 siblings, 1 reply; 22+ messages in thread
From: Pascal Hambourg @ 2013-09-17 23:23 UTC (permalink / raw)
To: netfilter
Alex Bligh a écrit :
>
> I don't think you need iptables. The way I've always done it is:
> * Default route to the VPN device
> * /32 route for the VPN endpoint out the physical interface to
> the previous default route
This does not meet the following OP's requirement :
>> The hard part is to also tunnel non-VPN connections to the VPN node
>> itself. In other words how do I make sure that every connection to the
>> external ip of the VPN node is tunneled through its internal ip --
>> except for the packets that form the tunnel itself?
However I am not sure this is a sensible requirement.
>> My idea was install a default route to the internal ip of the VPN node,
>> use iptables to mark the VPN connections and then set up a special
>> routing table for those.
Sounds good. Make sure that packets related to the VPN connection (e.g.
ICMP error messages) are routed outside the tunnel too. I guess that can
be done with connmark (connection mark).
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: Wrong routing when combining ip rule with SNAT
2013-09-17 23:23 ` Pascal Hambourg
@ 2013-09-18 0:55 ` Nikolaus Rath
2013-09-18 7:58 ` Alex Bligh
0 siblings, 1 reply; 22+ messages in thread
From: Nikolaus Rath @ 2013-09-18 0:55 UTC (permalink / raw)
To: netfilter
Pascal Hambourg <pascal@plouf.fr.eu.org> writes:
> Alex Bligh a écrit :
>>
>> I don't think you need iptables. The way I've always done it is:
>> * Default route to the VPN device
>> * /32 route for the VPN endpoint out the physical interface to
>> the previous default route
>
> This does not meet the following OP's requirement :
>
>>> The hard part is to also tunnel non-VPN connections to the VPN node
>>> itself. In other words how do I make sure that every connection to the
>>> external ip of the VPN node is tunneled through its internal ip --
>>> except for the packets that form the tunnel itself?
>
> However I am not sure this is a sensible requirement.
Why not? For example, the VPN node also acts as my mailserver. So
whenever I encounter firewalls that e.g. block everything but port 443
and 80, I have to establish a tunnel to be able to connect to port 25,
and then change the mail server name in my MUA to the internal name on
the VPN. Then, if I'm at a different location where I do not need the
VPN, I have to change it back to the public hostname.
That is rather annoying, and I could avoid it if I somehow get
the smtp connections to use the VPN gateway as well.
Admittedly, I could just run the VPN all the time, but I don't think
this makes the whole idea unsensible.
>>> My idea was install a default route to the internal ip of the VPN node,
>>> use iptables to mark the VPN connections and then set up a special
>>> routing table for those.
>
> Sounds good. Make sure that packets related to the VPN connection (e.g.
> ICMP error messages) are routed outside the tunnel too. I guess that can
> be done with connmark (connection mark).
Well, that was the idea that led to my original question. It doesn't
work, because the packets go out with the wrong source ip
address. Apparently, at the time that the source ip address is
determined, the connection hasn't been marked yet, so it takes the
source ip from the wrong routing table.
Best,
-Nikolaus
--
»Time flies like an arrow, fruit flies like a Banana.«
PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6 02CF A9AD B7F8 AE4E 425C
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: Wrong routing when combining ip rule with SNAT
2013-09-17 21:58 ` Eliezer Croitoru
@ 2013-09-18 0:58 ` Nikolaus Rath
0 siblings, 0 replies; 22+ messages in thread
From: Nikolaus Rath @ 2013-09-18 0:58 UTC (permalink / raw)
To: netfilter
Hi Eliezer,
Thanks for all the explanations, but my VPN is all set up and running,
and routing through it works just fine. The question is just how to
gateway non-VPN packets to the external ip of the VPN through the VPN
connection as well.
Best,
Nikolaus
Eliezer Croitoru <eliezer@ngtech.co.il> writes:
> Hey Nokolaus,
>
> it really depends on the vpn tunnel software.
> Some adds another interface per each session and others know that the
> vpn is for a specific subnet.
> From the kernel point of view since there is an endpoint to the tunnel
> there should be a route either detected automatically by the kernel or
> added when creating the tunnel device like in openvpn case.
>
> the basic issue is not tunneling non-VPN connections to the VPN node
> since most of the nodes should be routed by a single GW host that can
> have a keep-alived or any other method to just verify that this host is
> indeed up and running to prevent network outage.
>
> A basic network setup should have on GW and only one.
>
> If you do have a loaded network in hands make sure you study a bit first
> on network infrastructure maintenance and not just "by the book" but
> also hands-on experience that will give you much more power in hands
> when handling a small blinking light.
>
> In pptpd it works in another way which in not stricktly routing but also
> masquerading in many cases unless you are using a default route and also
> a range of dedicated addreses.
> the basic setup should be in many cases:
> gw <--<>--> network.
>
> the GW has access to the network and also to the internet or any
> external network.
> when the network only GW is also the VPN server and NAT(MASQUERADE) all
> other issues are smaller to handle.
> Install a GW once or twice and you will see how that magic of the kernel
> works fine.
> the only main rule in this case that should be on is the MAQUERADE on
> the external interface and the allow rules for the VPN.
> Else then that just be creative on how much access you need to the network.
> monowall was a nice firewall but pfsense is a much more sensible
> solution for most networks.
>
> If you want to use strictly Linux and not any BSD OpenSUSE, CentOS,
> UBUNTU are great options but depends on your clients you will need to
> choose the Distro.
>
> (if you need more help just ask)
>
> Eliezer
>
> On 09/17/2013 03:58 AM, Nikolaus Rath wrote:
>> Hi Eliezer,
>>
>> I have a VPN connection, and I want to tunnel everything through the VPN
>> node -- except, of course, the VPN connection itself.
>>
>> The hard part is to also tunnel non-VPN connections to the VPN node
>> itself. In other words how do I make sure that every connection to the
>> external ip of the VPN node is tunneled through its internal ip --
>> except for the packets that form the tunnel itself?
>>
>> My idea was install a default route to the internal ip of the VPN node,
>> use iptables to mark the VPN connections and then set up a special
>> routing table for those. But maybe there's an easier way?
>>
>> Best,
>> Nikolaus
>>
>> Eliezer Croitoru <eliezer@ngtech.co.il> writes:
>>> Hey there,
>>>
>>> What are you trying to achieve exactly?
>>> I tried to understand the network topology and the network issues but
>>> since you did not marked a target to what you want to actually get.
>>> There is an option to actually understand the situation you are in by
>>> just describing the need and the situation and then continue from there.
>>>
>>> Hope for the best
>>> Eliezer
>>>
>>> On 09/13/2013 08:10 AM, Nikolaus Rath wrote:
>>>> Hello,
>>>>
>>>> Thanks for working on this great networking stack!
>>>>
>>>> I'm trying to set up a configuration with SNAT and routing rules, but
>>>> I'm having weird problems that I do not understand:
>>>>
>>>> I've enabled packet forwarding and SNAT on the "ebox" computer as
>>>> follows:
>>>>
>>>> root@ebox:~# ip route
>>>> default via 23.92.25.1 dev eth0
>>>> 23.92.25.0/24 dev eth0 proto kernel scope link src 23.92.25.96
>>>> 192.168.12.0/24 dev rath proto kernel scope link src 192.168.12.1
>>>>
>>>> root@ebox:~# iptables -L -n -v
>>>> Chain INPUT (policy ACCEPT 1314 packets, 1736K bytes)
>>>> pkts bytes target prot opt in out source destination
>>>>
>>>> Chain FORWARD (policy DROP 0 packets, 0 bytes)
>>>> pkts bytes target prot opt in out source destination
>>>> 150K 62M ACCEPT all -- rath eth0 0.0.0.0/0 0.0.0.0/0
>>>> 86746 200M ACCEPT all -- eth0 rath 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED
>>>> 319 22076 LOG all -- * * 0.0.0.0/0 0.0.0.0/0 limit: avg 1/min burst 30 LOG flags 0 level 4 prefix "Rejected forwarding: "
>>>> 393 26172 REJECT all -- * * 0.0.0.0/0 0.0.0.0/0 reject-with icmp-net-prohibited
>>>>
>>>> Chain OUTPUT (policy ACCEPT 1142 packets, 2412K bytes)
>>>> pkts bytes target prot opt in out source destination
>>>>
>>>> root@ebox:~# iptables -t nat -L -n -v
>>>> Chain PREROUTING (policy ACCEPT 36378 packets, 2383K bytes)
>>>>
>>>> Chain INPUT (policy ACCEPT 19982 packets, 1334K bytes)
>>>> pkts bytes target prot opt in out source destination
>>>>
>>>> Chain OUTPUT (policy ACCEPT 61430 packets, 4601K bytes)
>>>> pkts bytes target prot opt in out source destination
>>>>
>>>> Chain POSTROUTING (policy ACCEPT 8333 packets, 564K bytes)
>>>> pkts bytes target prot opt in out source destination
>>>> 69488 5081K SNAT all -- * eth0 0.0.0.0/0 0.0.0.0/0 to:23.92.25.96
>>>>
>>>>
>>>> From a second computer "vostro", I can now use ebox as a gateway:
>>>>
>>>> root@vostro:~# ip route add 190.93.249.164 via 192.168.12.1
>>>>
>>>> This works fine, now connections to whatismyip.com (190.93.249.164) go
>>>> through ebox.
>>>>
>>>> However, when I try to be a bit more selective on vostro and use a
>>>> special routing table, things don't work anymore:
>>>>
>>>> root@vostro:~# iptables -t mangle -L -n
>>>> Chain PREROUTING (policy ACCEPT)
>>>> target prot opt source destination
>>>>
>>>> Chain INPUT (policy ACCEPT)
>>>> target prot opt source destination
>>>>
>>>> Chain FORWARD (policy ACCEPT)
>>>> target prot opt source destination
>>>>
>>>> Chain OUTPUT (policy ACCEPT)
>>>> target prot opt source destination
>>>> MARK tcp -- 0.0.0.0/0 190.93.249.164 tcp dpt:80 MARK set 0x1
>>>> LOG tcp -- 0.0.0.0/0 190.93.249.164 tcp dpt:80 LOG flags 0 level 4 prefix "marked: "
>>>>
>>>> Chain POSTROUTING (policy ACCEPT)
>>>> target prot opt source destination
>>>>
>>>> root@vostro:~# ip route del 190.93.249.164 via 192.168.12.1
>>>> root@vostro:~# ip route add default via 192.168.12.1 table tovpn
>>>> root@vostro:~# ip rule add fwmark 0x1 table tovpn
>>>>
>>>> Now connections from vostro to 190.93.249.164 still make it to ebox, and
>>>> from ebox to 190.93.249.164, but the answers get stuck on ebox:
>>>>
>>>> Sep 13 04:47:53 ebox kernel: Rejected forwarding: IN=eth0 OUT=eth0 MAC=f2:3c:91:69:db:07:84:78:ac:0d:79:c1:08:00 SRC=190.93.249.164 DST=192.168.17.47 LEN=60 TOS=0x00 PREC=0x00 TTL=58 ID=0 DF PROTO=TCP SPT=80 DPT=39024 WINDOW=14480 RES=0x00 ACK SYN URGP=0
>>>>
>>>> It seems that ebox tries to send the packet destined to go trough the
>>>> rath to eth0 instead, and consequency rejects them because forwarding is
>>>> only enabled from eth0 to rath.
>>>>
>>>> However, this only happens when vostro has the gateway route set in a
>>>> special routing table rather than the default table -- but how does ebox
>>>> even know about that?
>>>>
>>>> Can someone explain to me what is happening here and why?
>>>>
>>>>
>>>>
>>>> Best,
>>>>
>>>> -Nikolaus
>>>>
>>>
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe netfilter" in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>>
>>
>> -Nikolaus
>>
>
> --
> To unsubscribe from this list: send the line "unsubscribe netfilter" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
-Nikolaus
--
»Time flies like an arrow, fruit flies like a Banana.«
PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6 02CF A9AD B7F8 AE4E 425C
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: Wrong routing when combining ip rule with SNAT
2013-09-17 0:58 ` Nikolaus Rath
2013-09-17 12:35 ` Alex Bligh
2013-09-17 21:58 ` Eliezer Croitoru
@ 2013-09-18 5:54 ` Vigneswaran R
2013-09-18 17:51 ` Nikolaus Rath
2 siblings, 1 reply; 22+ messages in thread
From: Vigneswaran R @ 2013-09-18 5:54 UTC (permalink / raw)
To: netfilter, Nikolaus Rath
On 09/17/2013 06:28 AM, Nikolaus Rath wrote:
> Hi Eliezer,
>
> I have a VPN connection, and I want to tunnel everything through the VPN
> node -- except, of course, the VPN connection itself.
>
> The hard part is to also tunnel non-VPN connections to the VPN node
> itself. In other words how do I make sure that every connection to the
> external ip of the VPN node is tunneled through its internal ip --
> except for the packets that form the tunnel itself?
>
> My idea was install a default route to the internal ip of the VPN node,
> use iptables to mark the VPN connections and then set up a special
> routing table for those. But maybe there's an easier way?
* What is the internal IP of the VPN node? Is it 192.168.12.1 (and ebox
is the VPN node)?
* How the nodes are connected? Is it something like,
vostro <---> ebox <---> [Internet]
If so, where is 192.168.17.1 located.
Regards,
Vignesh
>
> Best,
> Nikolaus
>
> Eliezer Croitoru <eliezer@ngtech.co.il> writes:
>> Hey there,
>>
>> What are you trying to achieve exactly?
>> I tried to understand the network topology and the network issues but
>> since you did not marked a target to what you want to actually get.
>> There is an option to actually understand the situation you are in by
>> just describing the need and the situation and then continue from there.
>>
>> Hope for the best
>> Eliezer
>>
>> On 09/13/2013 08:10 AM, Nikolaus Rath wrote:
>>> Hello,
>>>
>>> Thanks for working on this great networking stack!
>>>
>>> I'm trying to set up a configuration with SNAT and routing rules, but
>>> I'm having weird problems that I do not understand:
>>>
>>> I've enabled packet forwarding and SNAT on the "ebox" computer as
>>> follows:
>>>
>>> root@ebox:~# ip route
>>> default via 23.92.25.1 dev eth0
>>> 23.92.25.0/24 dev eth0 proto kernel scope link src 23.92.25.96
>>> 192.168.12.0/24 dev rath proto kernel scope link src 192.168.12.1
>>>
>>> root@ebox:~# iptables -L -n -v
>>> Chain INPUT (policy ACCEPT 1314 packets, 1736K bytes)
>>> pkts bytes target prot opt in out source destination
>>>
>>> Chain FORWARD (policy DROP 0 packets, 0 bytes)
>>> pkts bytes target prot opt in out source destination
>>> 150K 62M ACCEPT all -- rath eth0 0.0.0.0/0 0.0.0.0/0
>>> 86746 200M ACCEPT all -- eth0 rath 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED
>>> 319 22076 LOG all -- * * 0.0.0.0/0 0.0.0.0/0 limit: avg 1/min burst 30 LOG flags 0 level 4 prefix "Rejected forwarding: "
>>> 393 26172 REJECT all -- * * 0.0.0.0/0 0.0.0.0/0 reject-with icmp-net-prohibited
>>>
>>> Chain OUTPUT (policy ACCEPT 1142 packets, 2412K bytes)
>>> pkts bytes target prot opt in out source destination
>>>
>>> root@ebox:~# iptables -t nat -L -n -v
>>> Chain PREROUTING (policy ACCEPT 36378 packets, 2383K bytes)
>>>
>>> Chain INPUT (policy ACCEPT 19982 packets, 1334K bytes)
>>> pkts bytes target prot opt in out source destination
>>>
>>> Chain OUTPUT (policy ACCEPT 61430 packets, 4601K bytes)
>>> pkts bytes target prot opt in out source destination
>>>
>>> Chain POSTROUTING (policy ACCEPT 8333 packets, 564K bytes)
>>> pkts bytes target prot opt in out source destination
>>> 69488 5081K SNAT all -- * eth0 0.0.0.0/0 0.0.0.0/0 to:23.92.25.96
>>>
>>>
>>> From a second computer "vostro", I can now use ebox as a gateway:
>>>
>>> root@vostro:~# ip route add 190.93.249.164 via 192.168.12.1
>>>
>>> This works fine, now connections to whatismyip.com (190.93.249.164) go
>>> through ebox.
>>>
>>> However, when I try to be a bit more selective on vostro and use a
>>> special routing table, things don't work anymore:
>>>
>>> root@vostro:~# iptables -t mangle -L -n
>>> Chain PREROUTING (policy ACCEPT)
>>> target prot opt source destination
>>>
>>> Chain INPUT (policy ACCEPT)
>>> target prot opt source destination
>>>
>>> Chain FORWARD (policy ACCEPT)
>>> target prot opt source destination
>>>
>>> Chain OUTPUT (policy ACCEPT)
>>> target prot opt source destination
>>> MARK tcp -- 0.0.0.0/0 190.93.249.164 tcp dpt:80 MARK set 0x1
>>> LOG tcp -- 0.0.0.0/0 190.93.249.164 tcp dpt:80 LOG flags 0 level 4 prefix "marked: "
>>>
>>> Chain POSTROUTING (policy ACCEPT)
>>> target prot opt source destination
>>>
>>> root@vostro:~# ip route del 190.93.249.164 via 192.168.12.1
>>> root@vostro:~# ip route add default via 192.168.12.1 table tovpn
>>> root@vostro:~# ip rule add fwmark 0x1 table tovpn
>>>
>>> Now connections from vostro to 190.93.249.164 still make it to ebox, and
>>> from ebox to 190.93.249.164, but the answers get stuck on ebox:
>>>
>>> Sep 13 04:47:53 ebox kernel: Rejected forwarding: IN=eth0 OUT=eth0 MAC=f2:3c:91:69:db:07:84:78:ac:0d:79:c1:08:00 SRC=190.93.249.164 DST=192.168.17.47 LEN=60 TOS=0x00 PREC=0x00 TTL=58 ID=0 DF PROTO=TCP SPT=80 DPT=39024 WINDOW=14480 RES=0x00 ACK SYN URGP=0
>>>
>>> It seems that ebox tries to send the packet destined to go trough the
>>> rath to eth0 instead, and consequency rejects them because forwarding is
>>> only enabled from eth0 to rath.
>>>
>>> However, this only happens when vostro has the gateway route set in a
>>> special routing table rather than the default table -- but how does ebox
>>> even know about that?
>>>
>>> Can someone explain to me what is happening here and why?
>>>
>>>
>>>
>>> Best,
>>>
>>> -Nikolaus
>>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe netfilter" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
> -Nikolaus
>
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: Wrong routing when combining ip rule with SNAT
2013-09-18 0:55 ` Nikolaus Rath
@ 2013-09-18 7:58 ` Alex Bligh
2013-09-18 17:38 ` Nikolaus Rath
0 siblings, 1 reply; 22+ messages in thread
From: Alex Bligh @ 2013-09-18 7:58 UTC (permalink / raw)
To: Nikolaus Rath; +Cc: Alex Bligh, netfilter
On 18 Sep 2013, at 01:55, Nikolaus Rath wrote:
> Why not? For example, the VPN node also acts as my mailserver. So
> whenever I encounter firewalls that e.g. block everything but port 443
> and 80, I have to establish a tunnel to be able to connect to port 25,
> and then change the mail server name in my MUA to the internal name on
> the VPN. Then, if I'm at a different location where I do not need the
> VPN, I have to change it back to the public hostname.
>
> That is rather annoying, and I could avoid it if I somehow get
> the smtp connections to use the VPN gateway as well.
One possibility would be to add another interface, so you are using
separate destination IP addresses for the end of the VPN tunnel
and 'everything else'. Remember the 'everything else' IP address
does not need to be public, as you'll only be reaching it by
the VPN tunnel.
Another is to use policy routing and only direct the VPN traffic
down the /32 route. This is pretty much what you were suggesting
re the marking etc. However, I would caution that this will mean
(e.g.) ICMP goes the 'wrong' way for at least one session. This
will make debugging hard, may affect pMTU discovery etc. etc.,
all of which will be bad news for reliable connections.
--
Alex Bligh
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: Wrong routing when combining ip rule with SNAT
2013-09-18 7:58 ` Alex Bligh
@ 2013-09-18 17:38 ` Nikolaus Rath
2013-09-18 20:11 ` Alex Bligh
0 siblings, 1 reply; 22+ messages in thread
From: Nikolaus Rath @ 2013-09-18 17:38 UTC (permalink / raw)
To: netfilter
Alex Bligh <alex@alex.org.uk> writes:
> On 18 Sep 2013, at 01:55, Nikolaus Rath wrote:
>
>> Why not? For example, the VPN node also acts as my mailserver. So
>> whenever I encounter firewalls that e.g. block everything but port 443
>> and 80, I have to establish a tunnel to be able to connect to port 25,
>> and then change the mail server name in my MUA to the internal name on
>> the VPN. Then, if I'm at a different location where I do not need the
>> VPN, I have to change it back to the public hostname.
>>
>> That is rather annoying, and I could avoid it if I somehow get
>> the smtp connections to use the VPN gateway as well.
>
> One possibility would be to add another interface, so you are using
> separate destination IP addresses for the end of the VPN tunnel
> and 'everything else'. Remember the 'everything else' IP address
> does not need to be public, as you'll only be reaching it by
> the VPN tunnel.
Hmm. I don't get it. Could you explain in more detail?
> Another is to use policy routing and only direct the VPN traffic
> down the /32 route. This is pretty much what you were suggesting
> re the marking etc. However, I would caution that this will mean
> (e.g.) ICMP goes the 'wrong' way for at least one session. This
> will make debugging hard, may affect pMTU discovery etc. etc.,
> all of which will be bad news for reliable connections.
I think I could live with the debugging problems, but at the moment it
is not working at all because of the source ip issues (see my very first
mail that started this thread).
Best,
Nikolaus
--
Encrypted emails preferred.
PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6 02CF A9AD B7F8 AE4E 425C
»Time flies like an arrow, fruit flies like a Banana.«
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: Wrong routing when combining ip rule with SNAT
2013-09-18 5:54 ` Vigneswaran R
@ 2013-09-18 17:51 ` Nikolaus Rath
2013-09-19 9:25 ` Vigneswaran R
0 siblings, 1 reply; 22+ messages in thread
From: Nikolaus Rath @ 2013-09-18 17:51 UTC (permalink / raw)
To: netfilter
Vigneswaran R <vignesh@atc.tcs.com> writes:
> On 09/17/2013 06:28 AM, Nikolaus Rath wrote:
>> Hi Eliezer,
>>
>> I have a VPN connection, and I want to tunnel everything through the VPN
>> node -- except, of course, the VPN connection itself.
>>
>> The hard part is to also tunnel non-VPN connections to the VPN node
>> itself. In other words how do I make sure that every connection to the
>> external ip of the VPN node is tunneled through its internal ip --
>> except for the packets that form the tunnel itself?
>>
>> My idea was install a default route to the internal ip of the VPN node,
>> use iptables to mark the VPN connections and then set up a special
>> routing table for those. But maybe there's an easier way?
>
> * What is the internal IP of the VPN node? Is it 192.168.12.1 (and
> ebox is the VPN node)?
>
> * How the nodes are connected? Is it something like,
>
> vostro <---> ebox <---> [Internet]
>
> If so, where is 192.168.17.1 located.
Ok, you asked for it. Here's my attempt at ASCII art :-).
vostro <--[192.168.17.0/24]--> evil-nat-dpi-firewall-box
^ ^
| |
| [Internet]
| |
| |
`---[192.168.12.0/24]---> ebox <---------'
So vostro is connected to some local network (e.g. 192.168.17.0/24, but
that changes depending on where I am) and to the VPN
(192.168.12.0/24). The local network is connected to the internet using
some (typically overly restricive) NAT box. ebox is connected to the
internet and to the VPN. I'm establishing the VPN by setting up an HTTP
tunnel through the internet to ebox.
Once the tunnel is established, I want that almost packet goes from
vostro to the internet to go through the VPN -- including packets
destined for ebox's public ip, but excluding packets that are destined
to port 80 of ebox's public ip (because they form the tunnel itself).
Best,
Nikolaus
--
Encrypted emails preferred.
PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6 02CF A9AD B7F8 AE4E 425C
»Time flies like an arrow, fruit flies like a Banana.«
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: Wrong routing when combining ip rule with SNAT
2013-09-18 17:38 ` Nikolaus Rath
@ 2013-09-18 20:11 ` Alex Bligh
2013-09-19 2:29 ` Nikolaus Rath
0 siblings, 1 reply; 22+ messages in thread
From: Alex Bligh @ 2013-09-18 20:11 UTC (permalink / raw)
To: Nikolaus Rath; +Cc: Alex Bligh, netfilter
On 18 Sep 2013, at 18:38, Nikolaus Rath wrote:
>>> Why not? For example, the VPN node also acts as my mailserver. So
>>> whenever I encounter firewalls that e.g. block everything but port 443
>>> and 80, I have to establish a tunnel to be able to connect to port 25,
>>> and then change the mail server name in my MUA to the internal name on
>>> the VPN. Then, if I'm at a different location where I do not need the
>>> VPN, I have to change it back to the public hostname.
>>>
>>> That is rather annoying, and I could avoid it if I somehow get
>>> the smtp connections to use the VPN gateway as well.
>>
>> One possibility would be to add another interface, so you are using
>> separate destination IP addresses for the end of the VPN tunnel
>> and 'everything else'. Remember the 'everything else' IP address
>> does not need to be public, as you'll only be reaching it by
>> the VPN tunnel.
>
> Hmm. I don't get it. Could you explain in more detail?
Assume your tunnel endpoint is 192.200.0.1 - i.e. a public IP.
Your challenge is that you want the packets constituting the
tunnel itself to go out one interface (direct to 192.200.0.1
over the internet). But you want your packets to 192.200.0.1:25
to go over the VPN. Correct?
So, assume you are not using 10.100.100.0/24. Add an interface
(I believe a loopback interface will do) with IP
address 10.100.100.1/24 on your VPN endpoint box. Ensure SMTP
etc. is listening on that interface too.
On your vpn client, do not use 192.200.0.1:25 (which by your own
admission will only work if you connect over the VPN). Rather
use 10.100.100.1:25. Use a default route over the VPN, but
route 192.200.0.1/32 to the nexthop the previous (native) default,
i.e. to the upstream router.
As 10.100.100.1 has no specific route in the client, it will be
routed through the VPN using the default route and will have the
source IP address of the VPN interface on the client. As this
will be directly connected, return traffic will also go via the
VPN.
Obviously for anything not connecting via the VPN, it can still
connect to 192.200.0.1/25. You've just added an extra IP
(which requires no routing and can be in RFC1918 space - here
10.100.100.1) to make life easier for clients that do connect
using the VPN.
Job done, no iptables required.
--
Alex Bligh
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: Wrong routing when combining ip rule with SNAT
2013-09-18 20:11 ` Alex Bligh
@ 2013-09-19 2:29 ` Nikolaus Rath
0 siblings, 0 replies; 22+ messages in thread
From: Nikolaus Rath @ 2013-09-19 2:29 UTC (permalink / raw)
To: netfilter
Alex Bligh <alex@alex.org.uk> writes:
> On 18 Sep 2013, at 18:38, Nikolaus Rath wrote:
>
>>>> Why not? For example, the VPN node also acts as my mailserver. So
>>>> whenever I encounter firewalls that e.g. block everything but port 443
>>>> and 80, I have to establish a tunnel to be able to connect to port 25,
>>>> and then change the mail server name in my MUA to the internal name on
>>>> the VPN. Then, if I'm at a different location where I do not need the
>>>> VPN, I have to change it back to the public hostname.
>>>>
>>>> That is rather annoying, and I could avoid it if I somehow get
>>>> the smtp connections to use the VPN gateway as well.
>>>
>>> One possibility would be to add another interface, so you are using
>>> separate destination IP addresses for the end of the VPN tunnel
>>> and 'everything else'. Remember the 'everything else' IP address
>>> does not need to be public, as you'll only be reaching it by
>>> the VPN tunnel.
>>
>> Hmm. I don't get it. Could you explain in more detail?
>
> Assume your tunnel endpoint is 192.200.0.1 - i.e. a public IP.
> Your challenge is that you want the packets constituting the
> tunnel itself to go out one interface (direct to 192.200.0.1
> over the internet). But you want your packets to 192.200.0.1:25
> to go over the VPN. Correct?
Correct.
> So, assume you are not using 10.100.100.0/24. Add an interface
> (I believe a loopback interface will do) with IP
> address 10.100.100.1/24 on your VPN endpoint box. Ensure SMTP
> etc. is listening on that interface too.
>
> On your vpn client, do not use 192.200.0.1:25 (which by your own
> admission will only work if you connect over the VPN). Rather
> use 10.100.100.1:25. Use a default route over the VPN, but
> route 192.200.0.1/32 to the nexthop the previous (native) default,
> i.e. to the upstream router.
Well, but the 10.100.100.1 address is also only going to work when the
VPN is up. So it wouldn't be enough to change the routing table, I'd
have to either activate the VPN permanently or change the mailserver ip
every time I (de-)activate the VPN.
Yes, it can be done. But I'm curious if there really isn't any solution
that would avoid that.
Best,
-Nikolaus
--
»Time flies like an arrow, fruit flies like a Banana.«
PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6 02CF A9AD B7F8 AE4E 425C
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: Wrong routing when combining ip rule with SNAT
2013-09-18 17:51 ` Nikolaus Rath
@ 2013-09-19 9:25 ` Vigneswaran R
0 siblings, 0 replies; 22+ messages in thread
From: Vigneswaran R @ 2013-09-19 9:25 UTC (permalink / raw)
To: netfilter, Nikolaus Rath
On 09/18/2013 11:21 PM, Nikolaus Rath wrote:
> Vigneswaran R <vignesh@atc.tcs.com> writes:
>> On 09/17/2013 06:28 AM, Nikolaus Rath wrote:
>>> Hi Eliezer,
>>>
>>> I have a VPN connection, and I want to tunnel everything through the VPN
>>> node -- except, of course, the VPN connection itself.
>>>
>>> The hard part is to also tunnel non-VPN connections to the VPN node
>>> itself. In other words how do I make sure that every connection to the
>>> external ip of the VPN node is tunneled through its internal ip --
>>> except for the packets that form the tunnel itself?
>>>
>>> My idea was install a default route to the internal ip of the VPN node,
>>> use iptables to mark the VPN connections and then set up a special
>>> routing table for those. But maybe there's an easier way?
>> * What is the internal IP of the VPN node? Is it 192.168.12.1 (and
>> ebox is the VPN node)?
>>
>> * How the nodes are connected? Is it something like,
>>
>> vostro <---> ebox <---> [Internet]
>>
>> If so, where is 192.168.17.1 located.
> Ok, you asked for it. Here's my attempt at ASCII art :-).
>
>
> vostro <--[192.168.17.0/24]--> evil-nat-dpi-firewall-box
> ^ ^
> | |
> | [Internet]
> | |
> | |
> `---[192.168.12.0/24]---> ebox <---------'
This is better :-)
> So vostro is connected to some local network (e.g. 192.168.17.0/24, but
> that changes depending on where I am) and to the VPN
> (192.168.12.0/24). The local network is connected to the internet using
> some (typically overly restricive) NAT box. ebox is connected to the
> internet and to the VPN. I'm establishing the VPN by setting up an HTTP
> tunnel through the internet to ebox.
>
> Once the tunnel is established, I want that almost packet goes from
> vostro to the internet to go through the VPN -- including packets
> destined for ebox's public ip, but excluding packets that are destined
> to port 80 of ebox's public ip (because they form the tunnel itself).
Ok. It seems using SNAT as suggested by Pascal on another mai, is
better. So, overall,
* Mark the packets destined to VPN_EXT_IP:80 to differentiate it from
the rest of the traffic
* Reroute the rest of the traffic via 'rath' Interface, using policy routing
* SNAT/MASQUERADE the packets going out via 'rath' Interface to have the
correct source IP
Regards,
Vignesh
>
>
> Best,
> Nikolaus
>
^ permalink raw reply [flat|nested] 22+ messages in thread
end of thread, other threads:[~2013-09-19 9:25 UTC | newest]
Thread overview: 22+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2013-09-13 5:10 Wrong routing when combining ip rule with SNAT Nikolaus Rath
2013-09-13 6:26 ` Vigneswaran R
2013-09-13 16:09 ` Nikolaus Rath
2013-09-13 22:03 ` Nikolaus Rath
2013-09-14 13:41 ` Pascal Hambourg
2013-09-14 15:40 ` Nikolaus Rath
2013-09-14 17:17 ` Pascal Hambourg
2013-09-16 7:14 ` Vigneswaran R
2013-09-16 23:38 ` Eliezer Croitoru
2013-09-17 0:58 ` Nikolaus Rath
2013-09-17 12:35 ` Alex Bligh
2013-09-17 23:23 ` Pascal Hambourg
2013-09-18 0:55 ` Nikolaus Rath
2013-09-18 7:58 ` Alex Bligh
2013-09-18 17:38 ` Nikolaus Rath
2013-09-18 20:11 ` Alex Bligh
2013-09-19 2:29 ` Nikolaus Rath
2013-09-17 21:58 ` Eliezer Croitoru
2013-09-18 0:58 ` Nikolaus Rath
2013-09-18 5:54 ` Vigneswaran R
2013-09-18 17:51 ` Nikolaus Rath
2013-09-19 9:25 ` Vigneswaran R
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.