[REGRESSION] vxlan: duplicate ICMP replies with nexthop FDB entries after 3fa64d86fc64

[Sascha Frey <sf@techfak.net>]

Hi,

I am seeing a VXLAN forwarding regression with Linux 7.2.7 and later,
apparently related to commit 3fa64d86fc64 ("vxlan: reject dynamic fdb
entries that reference a nexthop id").


Environment

Linux kernel: 7.2.7, tested from the upstream source tree
Also affected: Debian backports 7.2.9 and 7.2.11
Known good: Linux 7.1.8 and 7.2.6

The affected systems are Debian-based KVM virtualization hosts. Each
host uses Linux bridges with VxLAN devices attached as bridge ports,
acting as VTEPs.

The overall network uses EVPN. Some VTEPs run on the Debian 
virtualization hosts, while other VTEPs are implemented on Mellanox
switches running Cumulus Linux 5.18 with EVPN multihoming (EVPN-MH).

FRR/Zebra installs nexthop groups and VxLAN FDB entries on the Linux
hosts.


Symptoms

ICMP replies are duplicated. A packet capture on the source host's
physical interface carrying the outer VXLAN UDP traffic shows the same
ICMP reply encapsulated towards all remote VTEP addresses.

On a remote VTEP, a capture on the vxlan interface shows only one
request and one reply per sequence number.

Disabling GRO did not help, and the problem is independent of the ICMP
packet size.


Relevant FDB state

With the unmodified 7.2.7 kernel, the following entry was missing:

90:e2:ba:33:3b:04 nhid 536870949 self extern_learn

The corresponding nexthop group existed:

id 536870949 group 268435494/268435495 scope global proto zebra fdb
id 268435494 via 10.224.90.1 scope link proto zebra fdb
id 268435495 via 10.224.90.2 scope link proto zebra fdb


Revert test

I reverted commit 3fa64d86fc64 on top of Linux 7.2.7, without other
source changes.

With this kernel:

The missing VXLAN FDB entry reappears.
The duplicate ICMP replies disappear.

The revert removes the checks in vxlan_fdb_add() and
vxlan_fdb_update_existing() that reject dynamic FDB entries referencing
a nexthop ID.

This strongly suggests that the commit is involved, although I have not
yet established the exact mechanism by which the FDB entry goes missing.

Could someone help investigate whether the new checks can interfere with
EVPN/FRR-installed nexthop FDB entries, or whether this exposes another
issue in the VXLAN FDB handling?


Best regards,
Sascha