[LARTC] iproute2: gatewayed routes in ancillary tables

[LARTC] iproute2: gatewayed routes in ancillary tables

[Martin A. Brown]

Hello all,

Tomas Bonnedahl and I recently corresponded about a rather odd behaviour
with routes in ancillary routing tables.

We both receive a "Network is unreachable" error when we try to add
gatewayed routes to ancillary tables and the main routing table does not
contain an entry for the gateway IP.

It seems that unless a route to the gateway IP exists in the main routing
table, I cannot add a route through the gateway IP, even if there is a
route to the destination in the ancillary routing table.

Why is this?

Tomas has used the workaround as described in the annotated example below,
and there is no problem with this solution, but it strikes both of us as a
bit kludgy.

See the annotated example script below.

-Martin

# -- the below set of commands are annotated to describe what
#    is happening
#
# -- empty the ancillary routing table; I chose 4 as an arbitary routing
#    table--it could be any other non-main routing table
#
ip route flush table 4
#
# -- add a route to a locally reachable destination into table 4
#    this command succeeds
#
ip route add 192.168.237.0/24 dev eth0 table 4
#
# -- adding a gatewayed route fails, returning this error message:
#
#   RTNETLINK answers: Network is unreachable
#
ip route add default via 192.168.237.1 table 4
#
# -- even a route to a non-default route....same error:
#
ip route add 192.168.236.0/24 via 192.168.237.1 table 4
#
# -- now, if we put a route for our locally connected network into
#    the main routing table....
#
ip route add 192.168.237.0/24 dev eth0 table main
#
# -- and try to add a gatewayed route to table 4, it works!
#    same with a default route (just another gatewayed route)
#
ip route add 192.168.236.0/24 via 192.168.237.1 table 4
ip route add default via 192.168.237.1 table 4
#
# -- now, if we remove the route from table main,
#    things still operate as we had initially desired
#
ip route del 192.168.237.0/24 dev eth0 table main
#
# -- when we display the routing table it looks exactly like we want
#
ip route show table 4
#
# -- should look like this:
#
#   192.168.236.0/24 via 192.168.237.1 dev eth0
#   192.168.237.0/24 dev eth0  scope link
#   default via 192.168.237.1 dev eth0

-- 
Martin A. Brown --- SecurePipe, Inc. --- mabrown@securepipe.com

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] iproute2: gatewayed routes in ancillary tables

[Julian Anastasov]

	Hello,

On Mon, 5 May 2003, Martin A. Brown wrote:

> It seems that unless a route to the gateway IP exists in the main routing
> table, I cannot add a route through the gateway IP, even if there is a
> route to the destination in the ancillary routing table.
>
> Why is this?

	Because when we create a route we need to know the
right scope to assign to the new nexthop. Once it is known we don't
need the route to the gateway (and you can delete it).
You can see that there is such comment in the code:

It is not necessary, but requires a bit of thinking

	IMO, this check is necessary because without seeing the
route to the gateway we can not know whether this gateway is:

- local IP (and the resulting route should be device-only, GW IP
is ignored)

- unicast IP (and the resulting route should be via device and GW IP)

	If you do not like this policy you can use static routes
to solve the problem.

Regards

--
Julian Anastasov <ja@ssi.bg>

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] iproute2: gatewayed routes in ancillary tables

[Martin A. Brown]

Hello Julian,

 : > It seems that unless a route to the gateway IP exists in the main routing
 : > table, I cannot add a route through the gateway IP, even if there is a
 : > route to the destination in the ancillary routing table.
 :
 : 	Because when we create a route we need to know the right scope to
 : assign to the new nexthop. Once it is known we don't need the route to
 : the gateway (and you can delete it). You can see that there is such
 : comment in the code:

Where in the code?

 : It is not necessary, but requires a bit of thinking
 :
 : 	IMO, this check is necessary because without seeing the
 : route to the gateway we can not know whether this gateway is:

I still don't understand....the nexthop for the new route already exists
in the table I have specified.

Why couldn't we see the route to the gateway in the specified table?

 : - local IP (and the resulting route should be device-only, GW IP
 : is ignored)
 :
 : - unicast IP (and the resulting route should be via device and GW IP)

I am using the parameter "via" in the problematic route statement.
Doesn't that implicitly mean that the route is a unicast IP route?

My understanding is that the destination is distinctly not a local IP,
because of the presence of the parameter "via" in the nexthop definition.

 : 	If you do not like this policy you can use static routes
 : to solve the problem.

And now I'm really feeling like a dunce!  I understand "proto static" (I
think), but I don't see how that would solve this problem.

Thanks for any insight, Julian,

-Martin

-- 
Martin A. Brown --- SecurePipe, Inc. --- mabrown@securepipe.com

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] iproute2: gatewayed routes in ancillary tables

[Julian Anastasov]

	Hello,

On Mon, 5 May 2003, Martin A. Brown wrote:

> Where in the code?
>
>  : It is not necessary, but requires a bit of thinking

net/ipv4/fib_semantics.c, fib_check_nh()

>  : 	IMO, this check is necessary because without seeing the
>  : route to the gateway we can not know whether this gateway is:
>
> I still don't understand....the nexthop for the new route already exists
> in the table I have specified.

	No, we are creating new route which has nexthop(s). For
each nexthop we create, we need to determine its scope. This
scope tells us in route lookups whether we are via GW or just
via device. If the nexthop is via unicast GW IP then we send
the traffic to this IP. If the nexthop is not via GW then
we optionally resolve the destination (dest becomes nexthop)
and then we send the traffic via the specified device.

> Why couldn't we see the route to the gateway in the specified table?

	The route to the gateway must be reachable in
the normal lookup path, adding it to table is not enough, you need
'ip rule ... table GW_TABLE' to make it reachable.

>  : - local IP (and the resulting route should be device-only, GW IP
>  : is ignored)
>  :
>  : - unicast IP (and the resulting route should be via device and GW IP)
>
> I am using the parameter "via" in the problematic route statement.
> Doesn't that implicitly mean that the route is a unicast IP route?

	No, for example, 192.168.0.1 is not unicast, it is local:

ifconfig eth0 192.168.0.1
ip route add 10.0.0.0/8 via 192.168.0.1

	In such case, we can not use this IP as GW IP.

	The same is achieved in this way:

ifconfig eth0 192.168.0.1
ip route add 10.0.0.0/8 dev eth0

	But you are right, if there is no local IP we can assume
the GW IP is unicast.

> My understanding is that the destination is distinctly not a local IP,
> because of the presence of the parameter "via" in the nexthop definition.

	After "via" you can use local IP in addition to unicast
one. You know it but the kernel is not sure.

	But, considering the fact that GW IPs are always
onlink (the kernel ignores the many variants of scopes and
lookups only for onlink GWs), I think such check could be removed
and we can assume it is "unicast GW IP" only if (we have to add
many checks in addition):

- output device is UP
- IP protocol is enabled for this device (in_dev_get(dev) succeeds),
currently, that means there are IP addresses on this device

	such checks guarantee we will not break any expectations
in other places in the kernel by creating zombie routes. May
be this was the only remaining reason to keep this check, I'm
not sure.

>  : 	If you do not like this policy you can use static routes
>  : to solve the problem.
>
> And now I'm really feeling like a dunce!  I understand "proto static" (I
> think), but I don't see how that would solve this problem.

	Sorry, after checking the source ... may be not for your case,
it is done only when the device is down, in short, the nh scope
determination is delayed until device UP event.

Regards

--
Julian Anastasov <ja@ssi.bg>

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] iproute2: gatewayed routes in ancillary tables

[Martin A. Brown]

Julian,

 : > Where in the code?
 : net/ipv4/fib_semantics.c, fib_check_nh()
 :
 : 	No, we are creating new route which has nexthop(s). For
 : each nexthop we create, we need to determine its scope. This
 : scope tells us in route lookups whether we are via GW or just
 : via device. If the nexthop is via unicast GW IP then we send
 : the traffic to this IP. If the nexthop is not via GW then
 : we optionally resolve the destination (dest becomes nexthop)
 : and then we send the traffic via the specified device.
 :
 : > Why couldn't we see the route to the gateway in the specified table?
 :
 : 	The route to the gateway must be reachable in
 : the normal lookup path, adding it to table is not enough, you need
 : 'ip rule ... table GW_TABLE' to make it reachable.

OK--this makes sense.

 : 	No, for example, 192.168.0.1 is not unicast, it is local:
 :
 : ifconfig eth0 192.168.0.1
 : ip route add 10.0.0.0/8 via 192.168.0.1
 :
 : 	In such case, we can not use this IP as GW IP.

I just tried this example, and I see exactly what you mean.  This use of
the parameter "via" strikes me as confusing and counterintuitive.

 : 	But you are right, if there is no local IP we can assume
 : the GW IP is unicast.
 :
 : > My understanding is that the destination is distinctly not a local IP,
 : > because of the presence of the parameter "via" in the nexthop definition.
 :
 : 	After "via" you can use local IP in addition to unicast
 : one. You know it but the kernel is not sure.

For me, this is new--I struggle to imagine a case where I would use this
sort of syntax.  So, if I use "via <local-IP>", the kernel will
essentially treat the route as though I had specified "dev <interface>",
right?

 : 	But, considering the fact that GW IPs are always
 : onlink (the kernel ignores the many variants of scopes and
 : lookups only for onlink GWs),

Ah-ha!  And, as well as an explanation, you have given me a solution.  In
this case, I can use the following:

ip route add 192.168.237.0/24 dev eth0 table 4
ip route add 192.168.236.0/24 via 192.168.237.1 dev eth0  onlink  table 4

The onlink tells the kernel what I know; that my use of "via" in this case
is for a nexthop as a unicast destination.

 : I think such check could be removed and we can assume it is "unicast GW
 : IP" only if (we have to add many checks in addition):
 :
 : - output device is UP
 : - IP protocol is enabled for this device (in_dev_get(dev) succeeds),
 : currently, that means there are IP addresses on this device
 :
 : 	such checks guarantee we will not break any expectations
 : in other places in the kernel by creating zombie routes. May
 : be this was the only remaining reason to keep this check, I'm
 : not sure.

Thanks for taking the time to explain this, Julian.

-Martin

-- 
Martin A. Brown --- SecurePipe, Inc. --- mabrown@securepipe.com

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] iproute2: gatewayed routes in ancillary tables

[Julian Anastasov]

	Hello,

On Tue, 6 May 2003, Martin A. Brown wrote:

>  : 	After "via" you can use local IP in addition to unicast
>  : one. You know it but the kernel is not sure.
>
> For me, this is new--I struggle to imagine a case where I would use this
> sort of syntax.  So, if I use "via <local-IP>", the kernel will
> essentially treat the route as though I had specified "dev <interface>",
> right?

	Yes, this is sort of "find dev for this local IP and use
it as outdev".

>  : 	But, considering the fact that GW IPs are always
>  : onlink (the kernel ignores the many variants of scopes and
>  : lookups only for onlink GWs),
>
> Ah-ha!  And, as well as an explanation, you have given me a solution.  In
> this case, I can use the following:
>
> ip route add 192.168.237.0/24 dev eth0 table 4
> ip route add 192.168.236.0/24 via 192.168.237.1 dev eth0  onlink  table 4

	I'm glad you saw it because I forgot about it :)

> The onlink tells the kernel what I know; that my use of "via" in this case
> is for a nexthop as a unicast destination.

	The only problem is that "onlink" can succeed even if
IP is not enabled for this device (may be there is a reason for
this) but I hope this is not a big problem.

Regards

--
Julian Anastasov <ja@ssi.bg>

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/