TUN interfaces loopback

TUN interfaces loopback

[Radu Rendec]

Disclaimer: I know this is a *development* list, but I feel the answer
lies deep down in the ipv4 routing code, so it's more likely that I
find help here.

That being said, I have two TUN interfaces that are "cross-connected"
in user space (i.e. whatever is read on the socket corresponding to
either interface is written to the socket of the other interface).

The problem: if I assign IPv4 addresses to both TUN interfaces, local
traffic to either address flows through the loopback interface. Getting
packets to be routed through the TUN interfaces is easy. The real
challenge is to get packets to be accepted/processed as input packets
when they pop out of the opposite TUN interface.

I tracked down the problem to ip_route_input_slow() in net/ipv4/route.c

What I tried so far:

1. Remove the implicit routes from the local table. This apparently
causes packets to be silently dropped by ip_route_input_slow(), since
it looks for a corresponding route and only delivers packets locally if
a local route is found.

2. Keep implicit routes in the local table, change the priority of the
"lookup local" ip rule, mark *output* packets with iptables and add a
higher priority rule to lookup the main table for marked packets.

By doing that, the main table is looked up on the egress path and
packets are routed through the TUN interface. When packets pop out of
the opposite TUN interface, they are no longer marked (because they are
actually different packets), so ingress routing correctly looks up the
local table.

The next problem is that packets are seen as "martians" and dropped,
most probably because of __fib_validate_source() failing due to the
fact that the input interface is not the expected one.

This is where I stopped. Any idea or help would be highly appreciated.
Please CC my private address (I am not subscribed to the list). Thanks
in advance!

Re: TUN interfaces loopback

[Lorenzo Colitti]

On Thu, Nov 26, 2015 at 1:58 AM, Radu Rendec <radu.rendec@mindbit.ro> wrote:
> I tracked down the problem to ip_route_input_slow() in net/ipv4/route.c
> [...]
> The next problem is that packets are seen as "martians" and dropped,
> most probably because of __fib_validate_source() failing due to the
> fact that the input interface is not the expected one.

Have you tried setting this?

accept_local - BOOLEAN
        Accept packets with local source addresses. In combination with
        suitable routing, this can be used to direct packets between two
        local interfaces over the wire and have them accepted properly.
        default FALSE

Re: TUN interfaces loopback

[Radu Rendec]

On Sun, 2015-11-29 at 17:40 +0900, Lorenzo Colitti wrote:
> On Thu, Nov 26, 2015 at 1:58 AM, Radu Rendec <radu.rendec@mindbit.ro> wrote:
> > I tracked down the problem to ip_route_input_slow() in net/ipv4/route.c
> > [...]
> > The next problem is that packets are seen as "martians" and dropped,
> > most probably because of __fib_validate_source() failing due to the
> > fact that the input interface is not the expected one.
> 
> Have you tried setting this?
> 
> accept_local - BOOLEAN
>         Accept packets with local source addresses. In combination with
>         suitable routing, this can be used to direct packets between two
>         local interfaces over the wire and have them accepted properly.
>         default FALSE

I have tried it, but with no luck. I also tried with rp_filter=0, which
seems to be even more permissive than accept_local.

However, after reading the code more carefully, I finally figured it
out. It turns out that it's enough to have different (and correct)
routes for the outgoing and incoming routing decisions.

So in the following configuration:
tun0 [192.168.1.1 point-to-point 192.168.1.2]
^
|
v
tun1 [192.168.1.2 point-to-point 192.168.1.1]

it's enough to do the following:

 * Remove the automatically added routes from the "local" table.
 * Add two rules (one for each tun interface) *matching the input
   interface* and directing to a custom routing table (i.e. "ip rule
   add iif tunX lookup Y").
 * Add *local* routes for each tun interface to the custom routing
   table.
The trick here is that iif is set to 0 during the outgoing route
lookup, so the outgoing routing decision will not "see" the local
routes in the custom table. It will see the implicitly added routes in
the "main" table (these will route packets through the tun interfaces).

When the packet pops out of the opposite tun interface, the incoming
routing decision has iif set properly and will match the *local* routes
in the custom table. It is essential to match a local route at this
point, or else the packet will not be processed locally (as a packet
addressed to "this host").

With routing configured this way, not only that the martians problem
went away, but it works even with rp_filter=1 and accept_local=0.

Re: TUN interfaces loopback

[Hannes Frederic Sowa]

Hello,

On Wed, Nov 25, 2015, at 17:58, Radu Rendec wrote:
> Disclaimer: I know this is a *development* list, but I feel the answer
> lies deep down in the ipv4 routing code, so it's more likely that I
> find help here.
> 
> That being said, I have two TUN interfaces that are "cross-connected"
> in user space (i.e. whatever is read on the socket corresponding to
> either interface is written to the socket of the other interface).
> 
> The problem: if I assign IPv4 addresses to both TUN interfaces, local
> traffic to either address flows through the loopback interface. Getting
> packets to be routed through the TUN interfaces is easy. The real
> challenge is to get packets to be accepted/processed as input packets
> when they pop out of the opposite TUN interface.

Why do you require them to be marked as "input" packets?

> I tracked down the problem to ip_route_input_slow() in net/ipv4/route.c
> 
> What I tried so far:
> 
> 1. Remove the implicit routes from the local table. This apparently
> causes packets to be silently dropped by ip_route_input_slow(), since
> it looks for a corresponding route and only delivers packets locally if
> a local route is found.
> 
> 2. Keep implicit routes in the local table, change the priority of the
> "lookup local" ip rule, mark *output* packets with iptables and add a
> higher priority rule to lookup the main table for marked packets.
> 
> By doing that, the main table is looked up on the egress path and
> packets are routed through the TUN interface. When packets pop out of
> the opposite TUN interface, they are no longer marked (because they are
> actually different packets), so ingress routing correctly looks up the
> local table.
> 
> The next problem is that packets are seen as "martians" and dropped,
> most probably because of __fib_validate_source() failing due to the
> fact that the input interface is not the expected one.
> 
> This is where I stopped. Any idea or help would be highly appreciated.
> Please CC my private address (I am not subscribed to the list). Thanks
> in advance!

Have you tried ip route add local-ip-addr/32 dev tun0 and handle arp on
the outside interfaces with arp proxying or other means (routing
protocols)?

Thanks,
Hannes