* [PATCH net v3] net: reduce XMIT_RECURSION_LIMIT under KASAN
@ 2026-07-27 20:04 Tristan Madani
2026-07-27 23:24 ` Jakub Kicinski
2026-08-12 8:55 ` kernel test robot
0 siblings, 2 replies; 3+ messages in thread
From: Tristan Madani @ 2026-07-27 20:04 UTC (permalink / raw)
To: netdev
Cc: davem, edumazet, kuba, pabeni, andrew+netdev, horms, maheshb,
stable, linux-kernel, Tristan Madani
From: Tristan Madani <tristan@talencesecurity.com>
Virtual network devices (ipvlan, macvlan, bonding) can enter legitimate
transmit recursion when combined with packet forwarding configurations
such as IPVS NAT. The existing XMIT_RECURSION_LIMIT (8) in
__dev_queue_xmit() detects and breaks these loops, but the allowed
depth is too high for KASAN-instrumented kernels: each recursion level
consumes significantly more stack due to KASAN inline instrumentation,
and the cumulative usage overflows the kernel stack before the limit
fires.
On x86_64, CONFIG_KASAN doubles THREAD_SIZE from 16KB to 32KB
(KASAN_STACK_ORDER=1), but KASAN per-access checks inflate individual
function frames by roughly 2-3x. For an ipvlan L3 + IPVS NAT routing
loop, objdump measurements on a non-KASAN kernel show ~1.4KB of stack
consumed per recursion level (across 17 functions from __dev_queue_xmit
through the full IP output path and back). At KASAN ~2.3x inflation
factor that becomes ~3.3KB per level. Nine levels -- reached before the
current limit fires -- total ~30KB plus the initial call chain, which
exceeds the 32KB KASAN stack. The overflow hits the VMAP_STACK guard
page and causes a non-recoverable kernel panic (BUG: stack guard page
was hit).
On non-KASAN kernels the same loop is safely caught by the existing
limit: the "Dead loop on virtual device" message fires and the packet
is dropped without any stack overflow.
Reduce XMIT_RECURSION_LIMIT to 3 when CONFIG_KASAN is enabled. This
keeps the recursion counter well within the 32KB KASAN stack budget
while preserving the established limit of 8 for production kernels.
The recursion path triggering this is:
__dev_queue_xmit -> dev_hard_start_xmit -> ipvlan_start_xmit
-> ipvlan_queue_xmit -> ipvlan_process_outbound -> ip_local_out
-> nf_hook (IPVS) -> ip_vs_in_hook -> ip_vs_nat_xmit -> ip_output
-> ip_finish_output2 -> neigh_resolve_output -> __dev_queue_xmit
Tested:
- KASAN kernel (6.8.12 x86_64): panic before fix, "Dead loop"
drop after fix.
- Non-KASAN kernel (6.8.12 x86_64): "Dead loop" drop both before
and after fix (no behavior change for production kernels).
Fixes: 2ad7bf363841 ("ipvlan: Initial check-in of the IPVLAN driver.")
Cc: stable@vger.kernel.org
Signed-off-by: Tristan Madani <tristan@talencesecurity.com>
---
v3:
- no changes, repost per maintainer request
v2: https://lore.kernel.org/20260711204700.1760374-1-tristmd@gmail.com
v1: https://lore.kernel.org/20260711134732.1385563-1-tristmd@gmail.com
include/linux/netdevice.h | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h
index 9981d637f8b54..bdcb61d352afb 100644
--- a/include/linux/netdevice.h
+++ b/include/linux/netdevice.h
@@ -3640,7 +3640,11 @@ struct page_pool_bh {
};
DECLARE_PER_CPU(struct page_pool_bh, system_page_pool);
+#ifdef CONFIG_KASAN
+#define XMIT_RECURSION_LIMIT 3
+#else
#define XMIT_RECURSION_LIMIT 8
+#endif
#ifndef CONFIG_PREEMPT_RT
static inline int dev_recursion_level(void)
--
2.47.3
^ permalink raw reply related [flat|nested] 3+ messages in thread
* Re: [PATCH net v3] net: reduce XMIT_RECURSION_LIMIT under KASAN
2026-07-27 20:04 [PATCH net v3] net: reduce XMIT_RECURSION_LIMIT under KASAN Tristan Madani
@ 2026-07-27 23:24 ` Jakub Kicinski
2026-08-12 8:55 ` kernel test robot
1 sibling, 0 replies; 3+ messages in thread
From: Jakub Kicinski @ 2026-07-27 23:24 UTC (permalink / raw)
To: Tristan Madani
Cc: netdev, davem, edumazet, pabeni, andrew+netdev, horms, maheshb,
stable, linux-kernel, Tristan Madani
On Mon, 27 Jul 2026 20:04:54 +0000 Tristan Madani wrote:
> +#ifdef CONFIG_KASAN
> +#define XMIT_RECURSION_LIMIT 3
> +#else
> #define XMIT_RECURSION_LIMIT 8
> +#endif
At least these tests (maybe more) fail with a limit this low:
- selftests/net/forwarding/vxlan_symmetric.sh
- selftests/net/forwarding/vxlan_symmetric_ipv6.sh
- selftests/net/forwarding/vxlan_asymmetric.sh
- selftests/net/forwarding/vxlan_asymmetric_ipv6.sh
- selftests/net/test_bridge_backup_port.sh
- selftests/net/test_vxlan_mdb.sh
Can in investigate what limit they need and whether their max is good
enough to avoid issues with KASAN?
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [PATCH net v3] net: reduce XMIT_RECURSION_LIMIT under KASAN
2026-07-27 20:04 [PATCH net v3] net: reduce XMIT_RECURSION_LIMIT under KASAN Tristan Madani
2026-07-27 23:24 ` Jakub Kicinski
@ 2026-08-12 8:55 ` kernel test robot
1 sibling, 0 replies; 3+ messages in thread
From: kernel test robot @ 2026-08-12 8:55 UTC (permalink / raw)
To: Tristan Madani
Cc: oe-lkp, lkp, netdev, davem, edumazet, kuba, pabeni, andrew+netdev,
horms, maheshb, stable, linux-kernel, Tristan Madani, yi1.lai
Hello,
kernel test robot noticed "kselftests-bpf.net/forwarding fail" on:
commit: ef83ebe178d46ab0d26a906be5fcd654a488c730 ("[PATCH net v3] net: reduce XMIT_RECURSION_LIMIT under KASAN")
url: https://github.com/intel-lab-lkp/linux/commits/Tristan-Madani/net-reduce-XMIT_RECURSION_LIMIT-under-KASAN/20260728-072802
base: https://git.kernel.org/cgit/linux/kernel/git/davem/net.git bd0e9289e2642f6a5c54faad304ce0f41e926d22
patch link: https://lore.kernel.org/all/20260727200454.4048141-1-tristmd@gmail.com/
patch subject: [PATCH net v3] net: reduce XMIT_RECURSION_LIMIT under KASAN
in testcase: kselftests-bpf
version:
with following parameters:
group: net/forwarding
failed cases:
forwarding.custom_multipath_hash.sh
forwarding.vxlan_asymmetric.sh
forwarding.vxlan_asymmetric_ipv6.sh
forwarding.vxlan_symmetric.sh
forwarding.vxlan_symmetric_ipv6.sh
config: x86_64-rhel-9.4-bpf
compiler: gcc-14
test machine: 16 threads Intel(R) Core(TM) i7-13620H (Raptor Lake) with 32G memory
(please refer to attached dmesg/kmsg for entire log/backtrace)
If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <yi1.lai@intel.com>
| Closes: https://lore.kernel.org/oe-lkp/202608121039.ca91f25a-lkp@intel.com
KERNEL SELFTESTS: linux_headers_dir is /usr/src/linux-headers-x86_64-rhel-9.4-bpf-ef83ebe178d46ab0d26a906be5fcd654a488c730
2026-08-04 22:51:54 sed -i s/default_timeout=45/default_timeout=300/ kselftest/runner.sh
2026-08-04 22:51:54 make -j16 TARGETS=net/forwarding run_tests
# timeout set to 0
# selftests: net/forwarding: custom_multipath_hash.sh
# TEST: ping [ OK ]
# TEST: ping6 [ OK ]
# INFO: Running IPv4 custom multipath hash tests
# TEST: Multipath hash field: Source IP (balanced) [ OK ]
# INFO: Packets sent on path1 / path2: 6600 / 6001
# TEST: Multipath hash field: Source IP (unbalanced) [ OK ]
# INFO: Packets sent on path1 / path2: 0 / 12600
# TEST: Multipath hash field: Destination IP (balanced) [ OK ]
# INFO: Packets sent on path1 / path2: 5950 / 6650
# TEST: Multipath hash field: Destination IP (unbalanced) [ OK ]
# INFO: Packets sent on path1 / path2: 0 / 12600
# TEST: Multipath hash field: Source port (balanced) [ OK ]
# INFO: Packets sent on path1 / path2: 16505 / 16265
# TEST: Multipath hash field: Source port (unbalanced) [ OK ]
# INFO: Packets sent on path1 / path2: 0 / 32771
# TEST: Multipath hash field: Destination port (balanced) [ OK ]
# INFO: Packets sent on path1 / path2: 16504 / 16266
# TEST: Multipath hash field: Destination port (unbalanced) [ OK ]
# INFO: Packets sent on path1 / path2: 32769 / 0
# INFO: Running IPv6 custom multipath hash tests
# TEST: Multipath hash field: Source IP (balanced) [ OK ]
# INFO: Packets sent on path1 / path2: 6102 / 6500
# TEST: Multipath hash field: Source IP (unbalanced) [ OK ]
# INFO: Packets sent on path1 / path2: 12600 / 0
# TEST: Multipath hash field: Destination IP (balanced) [ OK ]
# INFO: Packets sent on path1 / path2: 6100 / 6500
# TEST: Multipath hash field: Destination IP (unbalanced) [ OK ]
# INFO: Packets sent on path1 / path2: 12600 / 0
# TEST: Multipath hash field: Flowlabel (balanced) [FAIL]
# Expected traffic to be balanced, but it is not
# INFO: Packets sent on path1 / path2: 1 / 0
# TEST: Multipath hash field: Flowlabel (unbalanced) [ OK ]
# INFO: Packets sent on path1 / path2: 0 / 12600
# TEST: Multipath hash field: Source port (balanced) [ OK ]
# INFO: Packets sent on path1 / path2: 16422 / 16347
# TEST: Multipath hash field: Source port (unbalanced) [ OK ]
# INFO: Packets sent on path1 / path2: 32771 / 0
# TEST: Multipath hash field: Destination port (balanced) [ OK ]
# INFO: Packets sent on path1 / path2: 16422 / 16347
# TEST: Multipath hash field: Destination port (unbalanced) [ OK ]
# INFO: Packets sent on path1 / path2: 32769 / 0
not ok 17 selftests: net/forwarding: custom_multipath_hash.sh # exit=1
# selftests: net/forwarding: vxlan_asymmetric.sh
# RTNETLINK answers: File exists
# RTNETLINK answers: File exists
# TEST: ping: local->local vid 10->vid 20 [ OK ]
# TEST: ping: local->remote vid 10->vid 10 [ OK ]
# TEST: ping: local->remote vid 20->vid 20 [ OK ]
# TEST: ping: local->remote vid 10->vid 20 [FAIL]
# TEST: ping: local->remote vid 20->vid 10 [FAIL]
# INFO: deleting neighbours from vlan interfaces
# TEST: ping: local->local vid 10->vid 20 [ OK ]
# TEST: ping: local->remote vid 10->vid 10 [ OK ]
# TEST: ping: local->remote vid 20->vid 20 [ OK ]
# TEST: ping: local->remote vid 10->vid 20 [FAIL]
# TEST: ping: local->remote vid 20->vid 10 [FAIL]
# selftests: net/forwarding: vxlan_asymmetric_ipv6.sh
# TEST: ping6: local->local vid 10->vid 20 [ OK ]
# TEST: ping6: local->remote vid 10->vid 10 [ OK ]
# TEST: ping6: local->remote vid 20->vid 20 [ OK ]
# TEST: ping6: local->remote vid 10->vid 20 [FAIL]
# TEST: ping6: local->remote vid 20->vid 10 [FAIL]
# INFO: deleting neighbours from vlan interfaces
# TEST: ping6: local->local vid 10->vid 20 [ OK ]
# TEST: ping6: local->remote vid 10->vid 10 [ OK ]
# TEST: ping6: local->remote vid 20->vid 20 [ OK ]
# TEST: ping6: local->remote vid 10->vid 20 [FAIL]
# TEST: ping6: local->remote vid 20->vid 10 [FAIL]
not ok 100 selftests: net/forwarding: vxlan_asymmetric_ipv6.sh # exit=1
# selftests: net/forwarding: vxlan_symmetric.sh
# RTNETLINK answers: File exists
# RTNETLINK answers: File exists
# TEST: ping: local->local vid 10->vid 20 [ OK ]
# TEST: ping: local->remote vid 10->vid 10 [ OK ]
# TEST: ping: local->remote vid 20->vid 20 [ OK ]
# TEST: ping: local->remote vid 10->vid 20 [FAIL]
# TEST: ping: local->remote vid 20->vid 10 [FAIL]
not ok 111 selftests: net/forwarding: vxlan_symmetric.sh # exit=1
# timeout set to 0
# selftests: net/forwarding: vxlan_symmetric_ipv6.sh
# TEST: ping6: local->local vid 10->vid 20 [ OK ]
# TEST: ping6: local->remote vid 10->vid 10 [ OK ]
# TEST: ping6: local->remote vid 20->vid 20 [ OK ]
# TEST: ping6: local->remote vid 10->vid 20 [FAIL]
# TEST: ping6: local->remote vid 20->vid 10 [FAIL]
not ok 112 selftests: net/forwarding: vxlan_symmetric_ipv6.sh # exit=1
The kernel config and materials to reproduce are available at:
https://download.01.org/0day-ci/archive/20260812/202608121039.ca91f25a-lkp@intel.com
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2026-08-12 8:55 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-07-27 20:04 [PATCH net v3] net: reduce XMIT_RECURSION_LIMIT under KASAN Tristan Madani
2026-07-27 23:24 ` Jakub Kicinski
2026-08-12 8:55 ` kernel test robot
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox