Re: [Questions] Some issues about IPv4/IPv6 nexthop route (was Re: [PATCH net-next] ipv4/fib: send RTM_DELROUTE notify when flush fib)

[Ido Schimmel <idosch@idosch.org>]

On Wed, Jul 26, 2023 at 06:17:05PM +0800, Hangbin Liu wrote:
> Hi Stephen, Ido, David,
> On Mon, Jul 24, 2023 at 08:48:20AM -0700, Stephen Hemminger wrote:
> > On Mon, 24 Jul 2023 16:56:37 +0800
> > Hangbin Liu <liuhangbin@gmail.com> wrote:
> > 
> > > The NetworkManager keeps a cache of the routes. Missing/Wrong events mean that
> > > the cache becomes inconsistent. The IPv4 will not send src route delete info
> > > if it's bond to other device. While IPv6 only modify the src route instead of
> > > delete it, and also no notify. So NetworkManager developers complained and
> > > hope to have a consistent and clear notification about route modify/delete.
> > 
> > Read FRR they get it right. The routing daemons have to track kernel,
> > and the semantics have been worked out for years.
> 
> Since we are talking about whether we should fix the issues or doc them. I
> have some other route issues reported by NetworkManager developers. And want
> discuss with you.
> 
> For IPv4, we add new route instead append the nexthop to same dest(or do I
> miss something?).

The append / prepend trick to create a multipath route is an IPv6 hack.
The correct way to install a multipath route is to add it in one go like
in the IPv4 implementation (which predates the IPv6 implementation) or
use the nexthop API.

> Since the route are not merged, the nexthop weight is not shown, which
> make them look like the same for users. For IPv4, the scope is also
> not shown, which look like the same for users.

The routes are the same, but separate. They do not form a multipath
route. Weight is meaningless for a non-multipath route.

> 
> While IPv6 will append another nexthop to the route if dest is same.

Yes, that's a hack.

> But there are 2 issues here:
> 1. the *type* and *protocol* field are actally ignored
> 2. when do `ip monitor route`, the info dumpped in fib6_add_rt2node()
>    use the config info from user space. When means `ip monitor` show the
>    incorrect type and protocol
> 
> So my questions are, should we show weight/scope for IPv4? How to deal the
> type/proto info missing for IPv6? How to deal with the difference of merging
> policy for IPv4/IPv6?

In my opinion, if you want consistent behavior between IPv4 and IPv6 for
multipath routes, then I suggest using the nexthop API. It was merged in
5.3 (IIRC) and FRR started using it by default a few years ago. Other
than a few bugs that were fixed, I don't remember many complaints. Also,
any nexthop-related features will only be implemented in the nexthop
API, not in the legacy API. Resilient nexthop groups is one example.