Policy routing + route "via" gives a strange behavior

Policy routing + route "via" gives a strange behavior

[Guido Trotter]

Hi,

I'm seeing what I think might be a strange kernel behavior when setting up a
route "via" a gateway, with policy routing. When adding a route with a gateway,
the kernel accepts it only if the gateway is reachable via that device. For
example:

ip route add default dev eth1 via 192.168.5.254

is only accepted if there is a route like:
192.168.5.0/24 dev eth1 scope link

in the main routing table. which, of course, is ok, otherwise the kernel
wouldn't be able to reach 192.168.5.254 in the first place.

Now, when adding policy routing to the mix, if I do:
ip route add table 100 default dev eth1 via 192.168.5.254

This is also refused unless a route like the one before appears in the default
table, even though it does appear in table 100. Is this the right behavior, and
if yes, why? It seems to me that it should be acceptable to have the network
route as well just in the separate routing table, since the "via" will only be
used by traffic hitting that table anyway.

Thanks a lot,

Guido

Re: Policy routing + route "via" gives a strange behavior

[Atis Elsts]

On Tuesday 20 October 2009 16:28:20 you wrote:
> This is also refused unless a route like the one before appears in the
> default table, even though it does appear in table 100. Is this the right
> behavior, and if yes, why? 

I guess what you describe is too infrequent use case for anyone to really 
care. Connected and link scoped routes are usually not added to policy 
routing tables :) Can you explain more for what kind of setup this is needed?

This "issue" could be solved by using routing table in the FIB lookup done in 
fib_check_nh(). However, doing that would break a lot more setups than it 
would "fix".
For example, if you had these rules
  from all to 1.2.3.4 fwmark 0x64 lookup 100
  from all fwmark 0x64 unreachable
then adding policy route to table 100 would fail unless nexthop 1.2.3.4 was 
used...

Anyway, you can achieve what you wish by using the "onlink" option, e.g.:
  ip route add table 100 default dev eth1 via 192.168.5.254 onlink

Atis

Re: Policy routing + route "via" gives a strange behavior

[Guido Trotter]

On Tue, Oct 20, 2009 at 07:48:39PM +0300, Atis Elsts wrote:

Hi,

Thanks for your explanation/help!

> > This is also refused unless a route like the one before appears in the
> > default table, even though it does appear in table 100. Is this the right
> > behavior, and if yes, why? 
> 
> I guess what you describe is too infrequent use case for anyone to really 
> care. Connected and link scoped routes are usually not added to policy 
> routing tables :) Can you explain more for what kind of setup this is needed?
> 

What I'm using it to is to force virtual machine traffic from a host to be
routed to a different interface (a gre interface, in my case). I set up a rule
that traffic from the guests' network (or from their interfaces) looks up a
different routing table, and in that table I set a default gateway so that any
traffic the instance would do is sent via that gateway.

> This "issue" could be solved by using routing table in the FIB lookup done in 
> fib_check_nh(). However, doing that would break a lot more setups than it 
> would "fix".
> For example, if you had these rules
>   from all to 1.2.3.4 fwmark 0x64 lookup 100
>   from all fwmark 0x64 unreachable
> then adding policy route to table 100 would fail unless nexthop 1.2.3.4 was 
> used...
> 
Not sure I follow you here, sorry! :/ How would it behave differently depending
on the rules used? If in table 100 I had something like:

ip route add 1.2.3.4 dev eth1 via 2.3.4.5

Then the traffic looking up 100 (which according to the rules is to 1.2.3.4, but
could be to something else as well) must be routed via 2.3.4.5.
Of course then in table 100 I need a route to 2.3.4.5, or I need one in the
default table (which is the only one that gets checked now).

> Anyway, you can achieve what you wish by using the "onlink" option, e.g.:
>   ip route add table 100 default dev eth1 via 192.168.5.254 onlink

I will try this, thanks! What I was doing now was something like:
ip route add 192.168.5.254/32 dev eth1 # no table 100
ip route add table 100 default dev eth1 via 192.168.5.254
ip route del 192.168.5.254/32 dev eth1

But it was kind of a nasty workaround. Although it was working :)

Thanks,

Guido

Re: Policy routing + route "via" gives a strange behavior

[Mallika Gautam]

>
>> Anyway, you can achieve what you wish by using the "onlink" option, e.g.:
>>   ip route add table 100 default dev eth1 via 192.168.5.254 onlink
>

Hi,
I am facing exactly the same issue and have to flush the main table
once the routes in the table get added. This is really nasty as I need
to this from a kernel module every time an interface is shutdown and
again brought up.

"onlink" doesn't seem to work as it "sees" the destination IP address
"onlink" instead of resolving it via the gateway. E.g. if I want to
ping 139.85.111.12 via eth1, it tries to send ARP request for
139.85.111.12  and not 192.168.5.254. So it doesn' work.

I would really link to see this behaviour fixed as the workarounds are
equally painful.

Regards
Mallika