[PATCH net v2] vxlan: reset the remote's dst cache when snooping moves a MAC

[PATCH net v2] vxlan: reset the remote's dst cache when snooping moves a MAC

[CMGS]

vxlan_snoop() handles a known MAC that now arrives from another remote
by rewriting the remote's IP address in place. It leaves the remote's
dst_cache alone, so the next packet to that MAC is routed with the dst
cached for the previous remote. When the two remotes have different
next hops, the packet leaves the wrong way. For example, if the
previous remote was on-link and the new one is behind a router, the
device resolves the new remote's address on the local link, the
neighbour stays incomplete, and traffic to the MAC is blackholed until
the cached route is invalidated, such as by "ip route flush cache".

The same branch also leaves remote_ifindex unchanged, although the
early return compares it and vxlan_xmit_one() uses it as the output
interface. When a MAC moves between an IPv6 link-local and another
address, the remote keeps the previous interface, and every later
packet from that remote enters the migration branch again.

vxlan_fdb_replace(), which makes the same change through netlink,
already resets the cache, updates remote_ifindex and clears offloaded.
Do the same when snooping.

Reproduced with network namespaces: a learning VXLAN device has one
peer VTEP on-link and another behind a router, and an inner MAC moves
from the first peer to the second. Without this patch, pings to the MAC
fail after the move until the route cache is flushed: in 10 of 10 runs
with one CPU, and in 5 of 6 runs with two CPUs, where it depends on
whether the sending CPU holds the stale entry. With the patch, they
succeed in all 16 runs. In a second setup over IPv6, the MAC moves from
a peer with a global address to a peer with a link-local one. Without
this patch, the move and every later packet from the new peer each send
an RTM_NEWNEIGH notification, 33 in each run. With the patch, only the
move sends one.

This was found by our AI-driven automated live migration tests. The fix
and this changelog were written with the help of an AI coding assistant.

Fixes: 0c1d70af924b ("net: use dst_cache for vxlan device")
Fixes: 87613de95041 ("vxlan: fix snooping for link-local IPv6 addresses")
Signed-off-by: CMGS <ilskdw@gmail.com>
---
v2:
- also update remote_ifindex and clear offloaded, as vxlan_fdb_replace() does
- add the IPv6 link-local case to the changelog and a second Fixes tag

v1: https://lore.kernel.org/netdev/20261106144406.66035-1-ilskdw@gmail.com/

 drivers/net/vxlan/vxlan_core.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/drivers/net/vxlan/vxlan_core.c b/drivers/net/vxlan/vxlan_core.c
index d11028b37..6bc54642e 100644
--- a/drivers/net/vxlan/vxlan_core.c
+++ b/drivers/net/vxlan/vxlan_core.c
@@ -1482,7 +1482,10 @@ static enum skb_drop_reason vxlan_snoop(struct net_device *dev,
 				    "%pM migrated from %pIS to %pIS\n",
 				    src_mac, &rdst->remote_ip.sa, &src_ip->sa);
 
+		dst_cache_reset(&rdst->dst_cache);
 		rdst->remote_ip = *src_ip;
+		rdst->remote_ifindex = ifindex;
+		rdst->offloaded = false;
 		vxlan_fdb_notify(vxlan, f, rdst, RTM_NEWNEIGH, true, NULL);
 	} else {
 		/* learned new entry */
-- 
2.47.3

Re: [PATCH net v2] vxlan: reset the remote's dst cache when snooping moves a MAC

[Nicolas Dichtel]

Le 10/11/2026 à 16:30, CMGS a écrit :
> vxlan_snoop() handles a known MAC that now arrives from another remote
> by rewriting the remote's IP address in place. It leaves the remote's
> dst_cache alone, so the next packet to that MAC is routed with the dst
> cached for the previous remote. When the two remotes have different
> next hops, the packet leaves the wrong way. For example, if the
> previous remote was on-link and the new one is behind a router, the
> device resolves the new remote's address on the local link, the
> neighbour stays incomplete, and traffic to the MAC is blackholed until
> the cached route is invalidated, such as by "ip route flush cache".
> 
> The same branch also leaves remote_ifindex unchanged, although the
> early return compares it and vxlan_xmit_one() uses it as the output
> interface. When a MAC moves between an IPv6 link-local and another
> address, the remote keeps the previous interface, and every later
> packet from that remote enters the migration branch again.
> 
> vxlan_fdb_replace(), which makes the same change through netlink,
> already resets the cache, updates remote_ifindex and clears offloaded.
> Do the same when snooping.
> 
> Reproduced with network namespaces: a learning VXLAN device has one
> peer VTEP on-link and another behind a router, and an inner MAC moves
> from the first peer to the second. Without this patch, pings to the MAC
> fail after the move until the route cache is flushed: in 10 of 10 runs
> with one CPU, and in 5 of 6 runs with two CPUs, where it depends on
> whether the sending CPU holds the stale entry. With the patch, they
> succeed in all 16 runs. In a second setup over IPv6, the MAC moves from
> a peer with a global address to a peer with a link-local one. Without
> this patch, the move and every later packet from the new peer each send
> an RTM_NEWNEIGH notification, 33 in each run. With the patch, only the
> move sends one.
> 
> This was found by our AI-driven automated live migration tests. The fix
> and this changelog were written with the help of an AI coding assistant.
> 
> Fixes: 0c1d70af924b ("net: use dst_cache for vxlan device")
> Fixes: 87613de95041 ("vxlan: fix snooping for link-local IPv6 addresses")
> Signed-off-by: CMGS <ilskdw@gmail.com>
You should put your real name.

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/process/submitting-patches.rst#n463

Re: [PATCH net v2] vxlan: reset the remote's dst cache when snooping moves a MAC

[Zhefu Peng]

On 10/11/2026 16:44, Nicolas Dichtel wrote:
> You should put your real name.

Thanks, I'll resend with my real name in From: and Signed-off-by:
as v3.

pw-bot: cr

Re: [PATCH net v2] vxlan: reset the remote's dst cache when snooping moves a MAC

[netdev-bot+sashiko]

Thank you for your contribution! Sashiko AI review found 3 potential
issue(s) to consider.

Critical: 0 · High: 0 · Medium: 2 · Low: 1

If you believe none of the issues are legitimate, please reply to each
issue of Medium severity or higher and explain why the AI is wrong.
If any of them are legitimate and you plan to respin, please let patchwork
know by including "pw-bot: cr" as a separate line at the end of your reply
(one such reply per series is enough).

- [Medium] In vxlan_snoop() (drivers/net/vxlan/vxlan_core.c), the
  migration branch now does `rdst->remote_ifindex = ifindex`.
- [Medium] The stale-dst_cache problem this patch claims to fix can still
  happen through a race.
- [Low] The patch adds lockless plain writes of rdst->remote_ifindex and
  of the rdst->offloaded bitfield in the vxlan_snoop() migration branch…

--

vxlan: reset the remote's dst cache when snooping moves a MAC

When a learned MAC moves to a new remote, the vxlan_snoop() migration
branch now resets the remote's dst_cache, updates remote_ifindex and
clears offloaded, the same way vxlan_fdb_replace() does. This fixes stale
cached routes after a MAC move. It also stops the repeated RTM_NEWNEIGH
notifications when a MAC moves to or from an IPv6 link-local remote.

> diff --git a/drivers/net/vxlan/vxlan_core.c b/drivers/net/vxlan/vxlan_core.c
> index d11028b374721..6bc54642e7f45 100644
> --- a/drivers/net/vxlan/vxlan_core.c
> +++ b/drivers/net/vxlan/vxlan_core.c
> @@ -1482,7 +1482,10 @@ static enum skb_drop_reason vxlan_snoop(struct net_device *dev,
>  				    "%pM migrated from %pIS to %pIS\n",
>  				    src_mac, &rdst->remote_ip.sa, &src_ip->sa);
>  
> +		dst_cache_reset(&rdst->dst_cache);
>  		rdst->remote_ip = *src_ip;

[Severity: Medium]
Can a concurrent transmit still put the old remote's route back into the
cache after this reset?

dst_cache_reset() only stores jiffies in reset_ts. The remote_ip and
remote_ifindex stores that follow are plain stores, with no lock against
the lockless xmit path. vxlan_xmit_one() reads the key from rdst before it
looks up the cache:

drivers/net/vxlan/vxlan_core.c:vxlan_xmit_one() {
    ...
	ifindex = rdst->remote_ifindex;

	if (addr_family == AF_INET) {
		key.u.ipv4.src = cfg->saddr.sin.sin_addr.s_addr;
		key.u.ipv4.dst = rdst->remote_ip.sin.sin_addr.s_addr;
    ...
}

On a miss, dst_cache_per_cpu_get() stamps the per-cpu entry with the
current jiffies:

net/core/dst_cache.c:dst_cache_per_cpu_get() {
    ...
	if (unlikely(!time_after(idst->refresh_ts,
				 READ_ONCE(dst_cache->reset_ts)) ||
    ...
fail:
	idst->refresh_ts = jiffies;
	return NULL;
}

dst_cache_set_ip4() does not touch refresh_ts. Consider this interleaving:

CPU1 (xmit to the MAC)            CPU2 (vxlan_snoop)
reads old rdst->remote_ip
                                  dst_cache_reset() (reset_ts = J)
                                  rdst->remote_ip = *src_ip
dst_cache_get() in jiffy J + 1
  miss, refresh_ts = J + 1
route lookup with old key
dst_cache_set_ip4() old route

The cached entry now looks newer than reset_ts, so later transmits to the
MAC on CPU1 would use the old remote's next hop. Packets from the new
remote take the early return, so the cache is not reset again.

Would this bring back the blackhole described in the changelog, until
something like "ip route flush cache" runs? The window is narrow, but in
the live migration case there is usually traffic to the MAC while it
moves.

Moving the reset after the field stores does not appear to close the
window. The reader still builds its key before it checks reset_ts, and
freshness is only tracked in jiffies.

vxlan_fdb_replace() uses the same order. The hash_lock it holds does not
exclude xmit readers either.

> +		rdst->remote_ifindex = ifindex;

[Severity: Medium]
Does this overwrite a user-configured egress interface with 0?

In vxlan_snoop(), ifindex is only set from src_ifindex for IPv6 link-local
sources. For IPv4 and global IPv6 sources it stays 0.

A user can create a dynamic entry with NDA_IFINDEX, for example:

  bridge fdb add <mac> dev vxlan0 dst 10.0.0.2 via eth1 dynamic

vxlan_fdb_parse() stores that in remote_ifindex. On a learning device,
the first packet from 10.0.0.2 with that inner source MAC does not take
the early return, because remote_ifindex != 0:

	if (likely(vxlan_addr_equal(&rdst->remote_ip, src_ip) &&
		   rdst->remote_ifindex == ifindex))
		return SKB_NOT_DROPPED_YET;

The static guard only skips NUD_PERMANENT and NUD_NOARP entries, so the
packet enters the migration branch. It logs "migrated from A to A" even
though the IP has not changed. With this patch, the branch also sets
remote_ifindex to 0, resets the dst cache and clears offloaded.

After that, vxlan_xmit_one() passes ifindex 0 to udp_tunnel_dst_lookup()
and falls back to sk_bound_dev_if, so the configured underlay interface
is no longer used.

vxlan_fdb_info() also stops reporting NDA_IFINDEX for the entry:

	if (rdst->remote_ifindex &&
	    nla_put_u32(skb, NDA_IFINDEX, rdst->remote_ifindex))

Before this patch the configured ifindex was kept, although every packet
produced a notification (the behaviour since 87613de95041).

Is it intended that the snoop path treats ifindex 0 as an authoritative
new value here? 0 only means the ifindex is unknown, so this path cannot
tell a user-configured ifindex from a stale learned one.

> +		rdst->offloaded = false;
>  		vxlan_fdb_notify(vxlan, f, rdst, RTM_NEWNEIGH, true, NULL);

[Severity: Low]
Should this branch take vxlan->hash_lock, as the else branch in
vxlan_snoop() already does?

The new remote_ifindex and offloaded writes are lockless and run in
softirq under RCU only. The other writers of these fields hold hash_lock:
vxlan_fdb_offloaded_set(), vxlan_fdb_replace() and
vxlan_fdb_clear_offload().

For example, vxlan_fdb_offloaded_set() is reached from mlxsw's
SWITCHDEV_VXLAN_FDB_OFFLOADED work, and mlxsw supports VXLAN_F_LEARN. It
could race like this:

CPU0 (vxlan_fdb_offloaded_set)       CPU1 (vxlan_snoop)
spin_lock_bh(&vxlan->hash_lock)
vxlan_fdb_find_rdst() matches on
  old remote_ip/remote_ifindex
                                     rdst->remote_ip = *src_ip
                                     rdst->remote_ifindex = ifindex
                                     rdst->offloaded = false
rdst->offloaded = fdb_info->offloaded

The new remote would then be reported with NTF_OFFLOADED even though the
hardware has not been programmed for it.

Two RX CPUs snooping the same MAC from different VTEPs could also race in
this branch. The result could combine remote_ip from one packet with
remote_ifindex from the other. vxlan_fdb_switchdev_notifier_info() could
then snapshot that torn tuple.

The remote_ip store predates this patch. The new remote_ifindex and
offloaded writes also have no WRITE_ONCE()/READ_ONCE() pairing, so KCSAN
is likely to report them.

Taking hash_lock here and re-checking the entry once it is held would
serialize these writes the same way vxlan_fdb_replace() is serialized.

-- 
Sashiko AI review · https://netdev-ai.bots.linux.dev/sashiko/#/patchset/20261110153020.9635-1-ilskdw%40gmail.com

Re: [PATCH net v2] vxlan: reset the remote's dst cache when snooping moves a MAC

[Zhefu Peng]

> [Severity: Medium]
> Can a concurrent transmit still put the old remote's route back into the
> cache after this reset?

Yes, the window exists: a transmit that read the old remote_ip before
the reset and reaches dst_cache_get() in a later jiffy refills the
cache with the old route. vxlan_fdb_replace() and the other tunnel
users that reset the cache after changing parameters have the same
window. Closing it needs a change to how dst_cache tracks freshness,
which is out of scope for this fix. Without the patch every move uses
the stale route; with it only this race does.

> [Severity: Medium]
> Does this overwrite a user-configured egress interface with 0?

Yes, this is a regression, and v4 has the same code. v5 compares
remote_ifindex in the early return only for link-local sources, so
traffic from the remote of an entry with a "via" device takes the early
return and the device stays set. The selftest gets a case for it.

pw-bot: cr