Vmap lookup transiently fails during atomic ruleset reload under load, leading to rule bypass

Vmap lookup transiently fails during atomic ruleset reload under load, leading to rule bypass

[Pavel Peňáz]

Dear all,
I ran into a reproducible issue with nftables, and would like to get
your view, if it is a bug, or possibly a design limitation with atomic
ruleset reload and maps. Thanks in advance for any feedback.

When performing an atomic ruleset reload (nft -f <file>) under heavy
network traffic conditions, nftables vmap lookups appear to fail
transiently. This causes packets that should match the vmap rule to
incorrectly bypass it and hit subsequent rules in the chain. This
issue has been observed even when the atomic reload uses an identical
ruleset file to the currently loaded one, suggesting a race condition
or transient inconsistency during the reload process itself rather
than a logic difference between rule versions.

Environment:
Kernel: 6.12.20+rpt-rpi-2712 (uname -r)
nftables: v1.1.2 (Commodore Bullmoose #3) (nft --version)
Hardware: Raspberry Pi 5

Minimal reproducible ruleset:

#!/usr/sbin/nft -f flush ruleset
table inet main {
chain forward {
# Forward chain with accept policy
type filter hook forward priority filter; policy accept;
# Rule 1: Accept all packets with specific saddr via anonymous vmap
ip saddr vmap { 78.128.211.42 : accept }
# Rule 2: Log packets from the same source
# This rule should normally NOT be hit due to Rule 1
ip saddr 78.128.211.42 log prefix "VMAP_BYPASS: " level warning
}}

Steps to Reproduce:

Save the above ruleset to a file (e.g., rules.nft).
Load the ruleset: sudo nft -f rules.nft
Generate significant, sustained network traffic that passes through
the forward chain and originates from the IP address specified in the
vmap (e.g., 78.128.211.42).

(Context: This was successfully reproduced by initiating a
high-bandwidth download from the server 78.128.211.42
(speedtest.cesnet.cz) to a client machine (LAN_IP_ADDR in logs)
located behind the nftables host, forcing traffic through the forward
chain).

While the heavy traffic is flowing, execute an atomic ruleset reload
using the same rules file: sudo nft -f rules.nft
Timing/Load Sensitivity: This issue appears sensitive to timing and
load. It may be necessary to run the sudo nft -f rules.nft command
multiple times during the traffic flow to trigger the race condition.
Monitor kernel logs: dmesg

Observed Behavior:

Occasionally, coinciding with the execution of nft -f rules.nft during
heavy traffic, log messages matching Rule 2 appear in the kernel log,
indicating that Rule 1 was bypassed:

Apr 26 22:37:23 hostname kernel: VMAP_BYPASS: IN=wan_if OUT=lan_if
SRC=78.128.211.42 DST=LAN_IP_ADDR LEN=1492 TOS=0x00 PREC=0x00 TTL=58
ID=2432 DF PROTO=TCP SPT=443 DPT=58956 WINDOW=294 RES=0x00 ACK URGP=0

(Note: IN/OUT interface names and DST IP have been generalized to
wan_if, lan_if, and LAN_IP_ADDR. Hostname generalized to hostname.)

Thanks!
Pavel