Questions re iproute2, netfilter, and locally sourced packets

Questions re iproute2, netfilter, and locally sourced packets

[Ian Batterbee]

Hi,

I have the following setup:

[ internet ]---[ ADSL NAT modem ]---[ linux box ]----[Host A, Host B, Host C]

I have an ppp over ssh tunnel established from my linux box to a server at work, via the internet.  Packets entering the tunnel are masqueraded to the ppp interface's address. Default route is via the ADSL modem, which also performs NAT.

Now, here's where it gets tricky. Host A is allowed to route via ssh tunnel to work, but packets from Host B to the same destination must be sent out the default route.

So to achieve this, I set up another routing table, added a name for it into /etc/iproute2/rt_tables, added an "ip rule from [HostA] lookup vpn", and in the script that brings the tunnel up, routes are added for my work's netblocks via the ppp link to the vpn route table. This much all works.  

In the same script, I enable masquerading on the ppp interface. This appears not to work, because if I sniff from a machine on the other side of the tunnel (ie, at work), packets emerging from it still have their original IP addresses from my local subnet on them, and this obviously causes issues for the reply to get back. "iptables -v -t nat -L POSTROUTING" shows that packets are hitting the MASQUERADE rule and the out interface is ppp0, as expected, but nevertheless, source addresses are not being changed as demonstrated by the packet sniffing mentioned above.

To make things more complicated, not only is Host A allowed to route via the tunnel, but packets sourced from the linux router itself should also be allowed to go that way (or to put it another way, everything but host B and Host C should be allowed to use the tunnel).

The trouble is, while I can allow packets from the linux router by adding: "ip rule add prior 50 lookup vpn" (which obviously allows everything to use the tunnel), I can't figure out how to specifically allow locally generated packets without allowing everything unconditionally.

I considered that perhaps this would be better solved by going back to a single table, and using iptables to allow or deny packets headed to my work's netblock based on their source address. The drawback with that is that I can't say 'I want you to go this way, and you to go that way' with this approach - I can only say "you go this way, and I'll drop your packet entirely"

And finally, I have another "ip rule" which matches packets entering eth1 (the internet side) of the linux box from my linux server at work. The only way they can enter from that side is if they were destined to my modem's external address, so I have to send them back the same way, and that ip rule entry deals with that.

So, two problems need solving here - 

1. why isn't masquerading working (I read a posting that suggested that iptables and iproute2 don't play fair with each other and that may be the problem)

2. How can I identify to "ip rule" packets that are locally sourced ? (Possibly with a fwmark, but I'm not sure how to identify packets to add the mark to)

Re: Questions re iproute2, netfilter, and locally sourced packets

[Pascal Hambourg]

Hello,

Ian Batterbee a écrit :
[...]
> So to achieve this, I set up another routing table, added a name for it 
> into /etc/iproute2/rt_tables, added an "ip rule from [HostA] lookup 
> vpn", and in the script that brings the tunnel up, routes are added for 
> my work's netblocks via the ppp link to the vpn route table. This much 
> all works. 
> In the same script, I enable masquerading on the ppp interface. This 
> appears not to work, because if I sniff from a machine on the other side 
> of the tunnel (ie, at work), packets emerging from it still have their 
> original IP addresses from my local subnet on them

This is a rather known issue. Iptables's MASQUERADE may not work (may it 
work ?) when combined with source address based routing policy. If I 
understood correctly the reason is the MASQUERADE code uses a kernel 
output routing function to determine the apparent source address, but 
this function does not use the original source address (why would it, as 
its purpose is to find out the source address to use ?), so the ip rule 
does not trigger and the main routing table is used. A workaround is to 
set the apparent source address explicitly with SNAT instead of 
MASQUERADE. However, unlike MASQUERADE, SNAT assumes the output 
interface is static and won't clean up the conntrack/NAT table when the 
PPP interface goes down, but this is not a problem if the interface 
always gets the same address.

> To make things more complicated, not only is Host A allowed to route via 
> the tunnel, but packets sourced from the linux router itself should also 
> be allowed to go that way (or to put it another way, everything but host 
> B and Host C should be allowed to use the tunnel).
> 
> The trouble is, while I can allow packets from the linux router by 
> adding: "ip rule add prior 50 lookup vpn" (which obviously allows 
> everything to use the tunnel), I can't figure out how to specifically 
> allow locally generated packets without allowing everything 
> unconditionally.

What about using MARK in the mangle OUTPUT chain and fwmark in ip rule ?

Re: Questions re iproute2, netfilter, and locally sourced packets

[Menno Smits]

Ian Batterbee wrote:
> 
> 2. How can I identify to "ip rule" packets that are locally sourced ? 

I've used something like this in the past:

ip rule add iif lo lookup <some table>

That rule will match locally generated packets regardless of where they 
are going.

Hope that helps,
Menno

Re: Questions re iproute2, netfilter, and locally sourced packets

[Ian Batterbee]

>
>
>A workaround is to 
>set the apparent source address explicitly with SNAT instead of 
>MASQUERADE. However, unlike MASQUERADE, SNAT assumes the output 
>interface is static and won't clean up the conntrack/NAT table when the 
>PPP interface goes down, but this is not a problem if the interface 
>always gets the same address.
>
I suspected that may be the problem, and I'm lucky because the tunnel 
does have the same IP every time, so I'll see what I can do with SNAT.

>I can't figure out how to specifically 
>> allow locally generated packets without allowing everything 
>> unconditionally.
>  
>
>
>What about using MARK in the mangle OUTPUT chain and fwmark in ip rule ?
>

I tried that, but the fwmark filter for 'ip rule' doesn't appear to work 
(or I'm doing something wrong).

If I do this:

iptables -t mangle -I OUTPUT -j MARK --set-mark 0x0001
ip rule add prio 1100 fwmark 0x0001 lookup vpn
ip route flush cache

...then the router can ping things through the tunnel, which is good, 
but ... so can every other machine on the network, which  is bad.

if I then display the rules, it shows (other rules omitted)
1100:   from all lookup vpn

ie, the fwmark condition doesn't show in the display output. I thought 
that may just be a display problem when dumping the rules, but given the 
fact that every host can ping through the tunnel, it looks like it is 
ignoring the fwmark bit, and adding it unconditionally.

I'm running iptables 1.3.5, on kernel 2.6.16.5.

Re: Questions re iproute2, netfilter, and locally sourced packets

[Pascal Hambourg]

Ian Batterbee a écrit :
[...]
> ip rule add prio 1100 fwmark 0x0001 lookup vpn
[...]
> if I then display the rules, it shows (other rules omitted)
> 1100:   from all lookup vpn
> 
> ie, the fwmark condition doesn't show in the display output. I thought 
> that may just be a display problem when dumping the rules

The fwmark condition should show. Is the option CONFIG_IP_ROUTE_FWMARK 
enabled in the kernel ? Or maybe your 'ip' version is too old (or too 
recent) for your kernel version.

Re: Questions re iproute2, netfilter, and locally sourced packets

[Ian Batterbee]

>The fwmark condition should show. Is the option CONFIG_IP_ROUTE_FWMARK 
>enabled in the kernel ? Or maybe your 'ip' version is too old (or too 
>recent) for your kernel version.
>  
>

Ahhhhhhhhh, thank you.. that was the little piece of information I was 
missing.

<recompiles kernel with CONFIG_IP_ROUTE_FWMARK defined and wonders how 
it got disabled to begin with>