[Questions] Some issues about IPv4/IPv6 nexthop route (was Re: [PATCH net-next] ipv4/fib: send RTM_DELROUTE notify when flush fib)

[Hangbin Liu <liuhangbin@gmail.com>]

Hi Stephen, Ido, David,
On Mon, Jul 24, 2023 at 08:48:20AM -0700, Stephen Hemminger wrote:
> On Mon, 24 Jul 2023 16:56:37 +0800
> Hangbin Liu <liuhangbin@gmail.com> wrote:
> 
> > The NetworkManager keeps a cache of the routes. Missing/Wrong events mean that
> > the cache becomes inconsistent. The IPv4 will not send src route delete info
> > if it's bond to other device. While IPv6 only modify the src route instead of
> > delete it, and also no notify. So NetworkManager developers complained and
> > hope to have a consistent and clear notification about route modify/delete.
> 
> Read FRR they get it right. The routing daemons have to track kernel,
> and the semantics have been worked out for years.

Since we are talking about whether we should fix the issues or doc them. I
have some other route issues reported by NetworkManager developers. And want
discuss with you.

For IPv4, we add new route instead append the nexthop to same dest(or do I
miss something?). Since the route are not merged, the nexthop weight is not
shown, which make them look like the same for users. For IPv4, the scope is
also not shown, which look like the same for users.

While IPv6 will append another nexthop to the route if dest is same. But there
are 2 issues here:
1. the *type* and *protocol* field are actally ignored
2. when do `ip monitor route`, the info dumpped in fib6_add_rt2node()
   use the config info from user space. When means `ip monitor` show the
   incorrect type and protocol

So my questions are, should we show weight/scope for IPv4? How to deal the
type/proto info missing for IPv6? How to deal with the difference of merging
policy for IPv4/IPv6?

Here is the reproducer:

+ ip link add dummy0 up type dummy
+ ip link add dummy1 up type dummy
+ ip link add dummy2 up type dummy
+ ip addr add 172.16.104.1/24 dev dummy1
+ ip addr add 172.16.104.2/24 dev dummy2
+ ip route add 172.16.105.0/24 table 100 via 172.16.104.100 dev dummy1
+ ip route append 172.16.105.0/24 table 100 via 172.16.104.100 dev dummy2
+ ip route add 172.16.106.0/24 table 100 nexthop via 172.16.104.100 dev dummy1 weight 1
+ ip route append 172.16.106.0/24 table 100 nexthop via 172.16.104.100 dev dummy1 weight 2
+ ip route show table 100
172.16.105.0/24 via 172.16.104.100 dev dummy1
172.16.105.0/24 via 172.16.104.100 dev dummy2
172.16.106.0/24 via 172.16.104.100 dev dummy1
172.16.106.0/24 via 172.16.104.100 dev dummy1

+ ip route add local default dev dummy1 table 200
+ ip route add 172.16.107.0/24 table 200 nexthop via 172.16.104.100 dev dummy1
+ ip route prepend default dev dummy1 table 200
+ ip route append 172.16.107.0/24 table 200 nexthop via 172.16.104.100 dev dummy1
+ ip route show table 200
default dev dummy1 scope link
local default dev dummy1 scope host
172.16.107.0/24 via 172.16.104.100 dev dummy1
172.16.107.0/24 via 172.16.104.100 dev dummy1

+ ip addr add 2001:db8:101::1/64 dev dummy1
+ ip addr add 2001:db8:101::2/64 dev dummy2
+ ip route add 2001:db8:102::/64 via 2001:db8:101::10 dev dummy1 table 100
+ ip route prepend 2001:db8:102::/64 via 2001:db8:101::10 dev dummy2 table 100
+ ip route add local 2001:db8:103::/64 via 2001:db8:101::10 dev dummy1 table 100
+ ip route prepend unicast 2001:db8:103::/64 via 2001:db8:101::10 dev dummy2 table 100
+ ip monitor route &
+ sleep 1
+ ip route add 2001:db8:104::/64 via 2001:db8:101::10 dev dummy1 proto kernel table 100
2001:db8:104::/64 via 2001:db8:101::10 dev dummy1 table 100 proto kernel metric 1024 pref medium
+ ip route prepend 2001:db8:104::/64 via 2001:db8:101::10 dev dummy2 proto bgp table 100
2001:db8:104::/64 table 100 proto bgp metric 1024 pref medium
        nexthop via 2001:db8:101::10 dev dummy2 weight 1
        nexthop via 2001:db8:101::10 dev dummy1 weight 1
+ ip -6 route show table 100
2001:db8:102::/64 metric 1024 pref medium
        nexthop via 2001:db8:101::10 dev dummy1 weight 1
        nexthop via 2001:db8:101::10 dev dummy2 weight 1
local 2001:db8:103::/64 metric 1024 pref medium
        nexthop via 2001:db8:101::10 dev dummy1 weight 1
        nexthop via 2001:db8:101::10 dev dummy2 weight 1
2001:db8:104::/64 proto kernel metric 1024 pref medium
        nexthop via 2001:db8:101::10 dev dummy1 weight 1
        nexthop via 2001:db8:101::10 dev dummy2 weight 1
+ kill $!

Thanks
Hangbin