Re: VXLAN FDB nexthop groups: per-nexthop UDP dst port / VNI feasibility

[David Ahern <dsahern@kernel.org>]

On 7/9/26 11:30 PM, Jack Ma wrote:
> Hi David, Ido, Roopa,
> 
> I am using VXLAN FDB nexthop groups ( ip nexthop add ... fdb ,
> followed by  bridge fdb add ... nhid ) to implement L2 ECMP across
> a set of HA endpoints.
> 
> In my design, each leg is a distinct (VTEP IP, UDP destination port)
> pair. Several endpoints live behind the same node IP and are
> disambiguated by the VXLAN UDP destination port.
> 
> Reading the current code (v6.12), it looks like an FDB nexthop can
> carry only a gateway IP, so a per-leg port cannot be expressed:
> 
> •  vxlan_fdb_nh_path_select()  copies only  nhc_gw  into the synthetic
>  rdst ;  remote_port  and  remote_vni  remain zero
> ( include/net/vxlan.h ).
> •  vxlan_xmit_one()  then falls back to the device-wide port:
>  dst_port = rdst->remote_port ? rdst->remote_port : vxlan->cfg.dst_port 
> ( drivers/net/vxlan/vxlan_core.c ).
> •  rtm_to_nh_config()  rejects  NHA_ENCAP  and  NHA_OIF  on FDB
> nexthops:
>  Fdb attribute can not be used with encap, oif or blackhole 
> ( net/ipv4/nexthop.c ).
> •  vxlan_fdb_parse()  rejects  NDA_PORT ,  NDA_VNI , and  NDA_IFINDEX 
> when  NDA_NH_ID  is present:
>  DST, VNI, ifindex and port are mutually exclusive with NH_ID 
> 
> The original commits, 38428d68719c ("nexthop: support for fdb ecmp
> nexthops") and 1274e1cc4226 ("vxlan: ecmp support for mac fdb
> entries"), say "these nexthops only have ip." This matches the EVPN
> multihoming use case, where every VTEP in the segment shares one UDP
> destination port.

The first commit was 6 years ago; those brain cells have been recycled.
:-) I suspect there was no reason to consider different UDP port per
leg. Roopa: does that sound right?


> 
> My questions:
> 
> 1. Was IP-only a deliberate scoping decision for the EVPN-MH use
> case, or is there a deeper reason why a per-nexthop UDP destination
> port and VNI could not be carried on an FDB nexthop, for example
> through  NHA_ENCAP  or lwtunnel, which route nexthops already
> accept?
> 2. If legs genuinely need distinct UDP destination ports, is the
> per-remote  rdst  path ( bridge fdb ... dst ... port ... ) the
> intended mechanism, with the understanding that it provides
> head-end replication semantics rather than nexthop-object ECMP?
> 3. Would you be open in principle to a patch that lets an FDB
> nexthop carry a per-nexthop destination port and VNI? If so, I
> would be happy to prototype it as an RFC.
> 

It seems reasonable to me.