default route handling of udhcpc

default route handling of udhcpc

[Steffen Sledz]

Recently we hit a problem with the default route handling of udhcpc.

The provided script busybox/files/simple.script (aka /etc/udhcpc.d/50default) removes all existing default routes before setting new routes. I think that's wrong.

If multiple interfaces offer default routes (via DHCP, static config, or any other way) there is no reason to prefer one over another (without having any additional information). So the script should only add the new default route. A short search in the internet brings up a lot of useful scenarios for multiple default routes.

So i tried to remove the part from the script which removes the existing routes. This led to a confusing situation.

If the script uses "route add default gw ..." (which is the case if /sbin/ip is not installed) everything seems to work fine.

But if /sbin/ip (from the busybox package) is installed the "ip route add default via ..." results in an "ip: RTNETLINK answers: File exists" error. In my opinion this is an error in the busybox ip implementation.

Opinions?

Regards,
Steffen

-- 
DResearch Fahrzeugelektronik GmbH
Otto-Schmirgal-Str. 3, 10319 Berlin, Germany
Tel: +49 30 515932-237 mailto:sledz@dresearch-fe.de
Fax: +49 30 515932-299
Geschäftsführer: Dr. Michael Weber, Werner Mögle;
Amtsgericht Berlin Charlottenburg; HRB 130120 B;
Ust.-IDNr. DE273952058

Re: default route handling of udhcpc

[Phil Blundell]

On Mon, 2011-11-07 at 09:13 +0100, Steffen Sledz wrote:
> Recently we hit a problem with the default route handling of udhcpc.
> 
> The provided script busybox/files/simple.script (aka /etc/udhcpc.d/50default) removes all existing default routes before setting new routes. I think that's wrong.
> 
> If multiple interfaces offer default routes (via DHCP, static config, or any other way) there is no reason to prefer one over another (without having any additional information). So the script should only add the new default route. A short search in the internet brings up a lot of useful scenarios for multiple default routes.
> 
> So i tried to remove the part from the script which removes the existing routes. This led to a confusing situation.
> 
> If the script uses "route add default gw ..." (which is the case if /sbin/ip is not installed) everything seems to work fine.
> 
> But if /sbin/ip (from the busybox package) is installed the "ip route add default via ..." results in an "ip: RTNETLINK answers: File exists" error. In my opinion this is an error in the busybox ip implementation.

It's not a defect in busybox ip; this is the intended behaviour of "ip
route add" and iproute2 behaves the same.  If you want multiple routes
for the same destination then you need to use "ip route append" instead.
Arguably it is a defect in "route" that you are allowed to set multiple
equivalent routes without having to specify any ordering but, given that
"route" is old and obsolescent, I am not too enthusiastic about trying
to fix it.

As to whether it makes sense for the default script to tear down old
routes, my view is that the current behaviour probably is the correct
default.  Wanting to use multiple default routes is a fairly unusual
case and anybody who does want to do that is probably capable of writing
a custom script for whatever their installation requires.  Whereas, if
dhcpcd were to leave old routes in place by default, you would probably
end up with naive users getting stale routes left around when switching
between networks and ending up with hard-to-debug connectivity problems.

p.

Re: default route handling of udhcpc

[Steffen Sledz]

On 07.11.2011 11:11, Phil Blundell wrote:
> On Mon, 2011-11-07 at 09:13 +0100, Steffen Sledz wrote:
>> Recently we hit a problem with the default route handling of udhcpc.
>>
>> The provided script busybox/files/simple.script (aka /etc/udhcpc.d/50default) removes all existing default routes before setting new routes. I think that's wrong.
>>
>> If multiple interfaces offer default routes (via DHCP, static config, or any other way) there is no reason to prefer one over another (without having any additional information). So the script should only add the new default route. A short search in the internet brings up a lot of useful scenarios for multiple default routes.
>>
>> So i tried to remove the part from the script which removes the existing routes. This led to a confusing situation.
>>
>> If the script uses "route add default gw ..." (which is the case if /sbin/ip is not installed) everything seems to work fine.
>>
>> But if /sbin/ip (from the busybox package) is installed the "ip route add default via ..." results in an "ip: RTNETLINK answers: File exists" error. In my opinion this is an error in the busybox ip implementation.
> 
> It's not a defect in busybox ip; this is the intended behaviour of "ip
> route add" and iproute2 behaves the same.  If you want multiple routes
> for the same destination then you need to use "ip route append" instead.
> Arguably it is a defect in "route" that you are allowed to set multiple
> equivalent routes without having to specify any ordering but, given that
> "route" is old and obsolescent, I am not too enthusiastic about trying
> to fix it.

OK, thanx to clarify this.

> As to whether it makes sense for the default script to tear down old
> routes, my view is that the current behaviour probably is the correct
> default.  Wanting to use multiple default routes is a fairly unusual
> case and anybody who does want to do that is probably capable of writing
> a custom script for whatever their installation requires.  Whereas, if
> dhcpcd were to leave old routes in place by default, you would probably
> end up with naive users getting stale routes left around when switching
> between networks and ending up with hard-to-debug connectivity problems.

What do you mean with stale routes? If an interfaces supports a default route it is availailable as long as the interface is up. If the interface goes down all routes using this interface are removed.

On the contrary. As i mentioned above we really hit a problem with current setting.

* Interface ethA (static config with default route) comes up and sets this default route.
* Interface ethB (in our case a USB-NIC for diagnostic purposes, DHCP with default route) comes up and *replaces* the former default route.

scenario a)
* Interface ethB goes down and removes its default route.
* There does not exist a default route any longer. :(

scenario b)
* Interface ethA tries to go down.
* This fails because removing the not-existing default route of config ethA fails. :(

Regards,
Steffen

-- 
DResearch Fahrzeugelektronik GmbH
Otto-Schmirgal-Str. 3, 10319 Berlin, Germany
Tel: +49 30 515932-237 mailto:sledz@dresearch-fe.de
Fax: +49 30 515932-299
Geschäftsführer: Dr. Michael Weber, Werner Mögle;
Amtsgericht Berlin Charlottenburg; HRB 130120 B;
Ust.-IDNr. DE273952058

Re: default route handling of udhcpc

[Phil Blundell]

On Mon, 2011-11-07 at 11:31 +0100, Steffen Sledz wrote:
> What do you mean with stale routes? If an interfaces supports a
> default route it is availailable as long as the interface is up. If
> the interface goes down all routes using this interface are removed.

I was thinking about the case of an interface changing address without
actually going down.  For example, try this:

# ip addr add 192.168.11.2/24 dev dummy0
# ip route append default via 192.168.11.1
# ip addr add 192.168.12.2/24 dev dummy0
# ip addr del 192.168.11.2/24 dev dummy0
# ip route append default via 192.168.12.1

which, I think, will leave you with a "stale" default route pointing to
192.168.11.1 even though you no longer have any addresses on that
subnet.

If it's the case that interfaces always do go down and then up again
when changing configuration then you're right, it would be a non-issue.
I guess maybe the best thing to do is to make sure that this is true and
then we could avoid the unconditional route squashing as you suggest.

> On the contrary. As i mentioned above we really hit a problem with current setting.
> 
> * Interface ethA (static config with default route) comes up and sets this default route.
> * Interface ethB (in our case a USB-NIC for diagnostic purposes, DHCP with default route) comes up and *replaces* the former default route.
> 
> scenario a)
> * Interface ethB goes down and removes its default route.
> * There does not exist a default route any longer. :(
> 
> scenario b)
> * Interface ethA tries to go down.
> * This fails because removing the not-existing default route of config ethA fails. :(

That latter scenario sounds like a separate scripting bug: failing to
remove a route shouldn't prevent the interface going down.  But I agree,
scenario A does sound like one that we should try to support and I guess
it oughtn't to be too hard to make the default scripts work with that.

There is still an unavoidable ambiguity about which default route should
actually be used, though: at some point something, somewhere is going to
have to decide whether the "new" or "old" default route is the preferred
one.  I suppose the most sensible default policy would be to have the
newer one win.

p.

Re: default route handling of udhcpc

[Steffen Sledz]

On 07.11.2011 11:57, Phil Blundell wrote:
> On Mon, 2011-11-07 at 11:31 +0100, Steffen Sledz wrote:
>> What do you mean with stale routes? If an interfaces supports a
>> default route it is availailable as long as the interface is up. If
>> the interface goes down all routes using this interface are removed.
> 
> I was thinking about the case of an interface changing address without
> actually going down.  For example, try this:
> 
> # ip addr add 192.168.11.2/24 dev dummy0
> # ip route append default via 192.168.11.1
> # ip addr add 192.168.12.2/24 dev dummy0
> # ip addr del 192.168.11.2/24 dev dummy0
> # ip route append default via 192.168.12.1
> 
> which, I think, will leave you with a "stale" default route pointing to
> 192.168.11.1 even though you no longer have any addresses on that
> subnet.

OK, i see.

> If it's the case that interfaces always do go down and then up again
> when changing configuration then you're right, it would be a non-issue.
> I guess maybe the best thing to do is to make sure that this is true and
> then we could avoid the unconditional route squashing as you suggest.

I can't imagine that one of the OE related distros reconfigures network interfaces in described way. I believe they all use something like ifup/ifdown.

>> On the contrary. As i mentioned above we really hit a problem with current setting.
>>
>> * Interface ethA (static config with default route) comes up and sets this default route.
>> * Interface ethB (in our case a USB-NIC for diagnostic purposes, DHCP with default route) comes up and *replaces* the former default route.
>>
>> scenario a)
>> * Interface ethB goes down and removes its default route.
>> * There does not exist a default route any longer. :(
>>
>> scenario b)
>> * Interface ethA tries to go down.
>> * This fails because removing the not-existing default route of config ethA fails. :(
> 
> That latter scenario sounds like a separate scripting bug: failing to
> remove a route shouldn't prevent the interface going down.  But I agree,
> scenario A does sound like one that we should try to support and I guess
> it oughtn't to be too hard to make the default scripts work with that.
> 
> There is still an unavoidable ambiguity about which default route should
> actually be used, though: at some point something, somewhere is going to
> have to decide whether the "new" or "old" default route is the preferred
> one.  I suppose the most sensible default policy would be to have the
> newer one win.

Why do we have to make this decision? Let them both exist and the kernel make the routing decisions depending on its built in strategy and config.

Steffen

-- 
DResearch Fahrzeugelektronik GmbH
Otto-Schmirgal-Str. 3, 10319 Berlin, Germany
Tel: +49 30 515932-237 mailto:sledz@dresearch-fe.de
Fax: +49 30 515932-299
Geschäftsführer: Dr. Michael Weber, Werner Mögle;
Amtsgericht Berlin Charlottenburg; HRB 130120 B;
Ust.-IDNr. DE273952058

Re: default route handling of udhcpc

[Steffen Sledz]

On 07.11.2011 11:57, Phil Blundell wrote:
> If it's the case that interfaces always do go down and then up again
> when changing configuration then you're right, it would be a non-issue.
> I guess maybe the best thing to do is to make sure that this is true and
> then we could avoid the unconditional route squashing as you suggest.
>
>> On the contrary. As i mentioned above we really hit a problem with current setting.
>>
>> * Interface ethA (static config with default route) comes up and sets this default route.
>> * Interface ethB (in our case a USB-NIC for diagnostic purposes, DHCP with default route) comes up and *replaces* the former default route.
>>
>> scenario a)
>> * Interface ethB goes down and removes its default route.
>> * There does not exist a default route any longer. :(
>>
>> scenario b)
>> * Interface ethA tries to go down.
>> * This fails because removing the not-existing default route of config ethA fails. :(
>
> That latter scenario sounds like a separate scripting bug: failing to
> remove a route shouldn't prevent the interface going down.

It seems that we can't find a general solution for the default route handling for the moment. So i like to fix this bug at first. Can someone give a hint which scripts are involved here?

Regards,
Steffen

-- 
DResearch Fahrzeugelektronik GmbH
Otto-Schmirgal-Str. 3, 10319 Berlin, Germany
Tel: +49 30 515932-237 mailto:sledz@dresearch-fe.de
Fax: +49 30 515932-299
Geschäftsführer: Dr. Michael Weber, Werner Mögle;
Amtsgericht Berlin Charlottenburg; HRB 130120 B;
Ust.-IDNr. DE273952058