[PATCH net] ipvlan: add xmit recursion protection

[PATCH net] ipvlan: add xmit recursion protection

[Tristan Madani]

From: Tristan Madani <tristan@talencesecurity.com>

ipvlan devices can enter infinite transmit recursion when combined with
packet forwarding configurations (such as IPVS) that route traffic back
through the same ipvlan interface.

The recursion path is:

  ipvlan_start_xmit -> ipvlan_queue_xmit -> ipvlan_xmit_mode_l3
  -> ipvlan_process_outbound -> ip_local_out -> netfilter hooks
  -> dev_queue_xmit -> ipvlan_start_xmit (recurse)

The existing per-CPU xmit recursion counter in __dev_queue_xmit()
(XMIT_RECURSION_LIMIT = 8) does detect the loop, but fires too late:
each recursion level consumes roughly 2KB of stack space through
ip_local_out and netfilter, and at 8 levels the cumulative usage
exceeds the 16KB kernel stack on x86_64. The resulting stack overflow
hits the VMAP_STACK guard page and causes a kernel panic.

Add a per-CPU counter that prevents any re-entry into
ipvlan_queue_xmit() while it is already executing on the same CPU.
This mirrors the approach used by tunnel devices (see
IP_TUNNEL_RECURSION_LIMIT in ip_tunnels.h) but with a stricter limit
appropriate for ipvlan.

Fixes: 2ad7bf363841 ("ipvlan: Initial check-in of the IPVLAN driver.")
Cc: stable@vger.kernel.org
Signed-off-by: Tristan Madani <tristan@talencesecurity.com>
---
 drivers/net/ipvlan/ipvlan_core.c | 25 +++++++++++++++++++++----
 1 file changed, 21 insertions(+), 4 deletions(-)

diff --git a/drivers/net/ipvlan/ipvlan_core.c b/drivers/net/ipvlan/ipvlan_core.c
index 835e04835..ab99eb624 100644
--- a/drivers/net/ipvlan/ipvlan_core.c
+++ b/drivers/net/ipvlan/ipvlan_core.c
@@ -9,6 +9,8 @@
 
 static u32 ipvlan_jhash_secret __read_mostly;
 
+static DEFINE_PER_CPU(int, ipvlan_xmit_depth);
+
 void ipvlan_init_secret(void)
 {
 	net_get_random_once(&ipvlan_jhash_secret, sizeof(ipvlan_jhash_secret));
@@ -676,6 +678,7 @@ int ipvlan_queue_xmit(struct sk_buff *skb, struct net_device *dev)
 {
 	struct ipvl_dev *ipvlan = netdev_priv(dev);
 	struct ipvl_port *port = ipvlan_port_get_rcu_bh(ipvlan->phy_dev);
+	int ret = NET_XMIT_DROP;
 
 	if (!port)
 		goto out;
@@ -683,18 +686,32 @@ int ipvlan_queue_xmit(struct sk_buff *skb, struct net_device *dev)
 	if (unlikely(!pskb_may_pull(skb, sizeof(struct ethhdr))))
 		goto out;
 
+	if (this_cpu_read(ipvlan_xmit_depth)) {
+		net_crit_ratelimited("ipvlan: xmit recursion detected on dev %s\n",
+				     dev->name);
+		goto out;
+	}
+
+	this_cpu_inc(ipvlan_xmit_depth);
 	switch(port->mode) {
 	case IPVLAN_MODE_L2:
-		return ipvlan_xmit_mode_l2(skb, dev);
+		ret = ipvlan_xmit_mode_l2(skb, dev);
+		break;
 	case IPVLAN_MODE_L3:
 #ifdef CONFIG_IPVLAN_L3S
 	case IPVLAN_MODE_L3S:
 #endif
-		return ipvlan_xmit_mode_l3(skb, dev);
+		ret = ipvlan_xmit_mode_l3(skb, dev);
+		break;
+	default:
+		WARN_ONCE(true, "%s called for mode = [%x]\n",
+			  __func__, port->mode);
+		kfree_skb(skb);
+		break;
 	}
+	this_cpu_dec(ipvlan_xmit_depth);
+	return ret;
 
-	/* Should not reach here */
-	WARN_ONCE(true, "%s called for mode = [%x]\n", __func__, port->mode);
 out:
 	kfree_skb(skb);
 	return NET_XMIT_DROP;
-- 
2.47.3

Re: [PATCH net] ipvlan: add xmit recursion protection

[Eric Dumazet]

On Sat, Jul 11, 2026 at 3:47 PM Tristan Madani <tristmd@gmail.com> wrote:
>
> From: Tristan Madani <tristan@talencesecurity.com>
>
> ipvlan devices can enter infinite transmit recursion when combined with
> packet forwarding configurations (such as IPVS) that route traffic back
> through the same ipvlan interface.
>
> The recursion path is:
>
>   ipvlan_start_xmit -> ipvlan_queue_xmit -> ipvlan_xmit_mode_l3
>   -> ipvlan_process_outbound -> ip_local_out -> netfilter hooks
>   -> dev_queue_xmit -> ipvlan_start_xmit (recurse)
>
> The existing per-CPU xmit recursion counter in __dev_queue_xmit()
> (XMIT_RECURSION_LIMIT = 8) does detect the loop, but fires too late:
> each recursion level consumes roughly 2KB of stack space through
> ip_local_out and netfilter, and at 8 levels the cumulative usage
> exceeds the 16KB kernel stack on x86_64. The resulting stack overflow
> hits the VMAP_STACK guard page and causes a kernel panic.
>

Please fix the existing mechanism. 2KB of stack space seems excessive.

If this is caused by some DEBUG option, it will be just fine to limit
XMIT_RECURSION_LIMIT to 3 for such debug kernels.

Adding a workaround in every virtual driver is not an option.

[PATCH net v2] net: reduce XMIT_RECURSION_LIMIT under KASAN

[Tristan Madani]

From: Tristan Madani <tristan@talencesecurity.com>

Virtual network devices (ipvlan, macvlan, bonding) can enter legitimate
transmit recursion when combined with packet forwarding configurations
such as IPVS NAT.  The existing XMIT_RECURSION_LIMIT (8) in
__dev_queue_xmit() detects and breaks these loops, but the allowed
depth is too high for KASAN-instrumented kernels: each recursion level
consumes significantly more stack due to KASAN inline instrumentation,
and the cumulative usage overflows the kernel stack before the limit
fires.

On x86_64, CONFIG_KASAN doubles THREAD_SIZE from 16KB to 32KB
(KASAN_STACK_ORDER=1), but KASAN per-access checks inflate individual
function frames by roughly 2-3x.  For an ipvlan L3 + IPVS NAT routing
loop, objdump measurements on a non-KASAN kernel show ~1.4KB of stack
consumed per recursion level (across 17 functions from __dev_queue_xmit
through the full IP output path and back).  At KASAN ~2.3x inflation
factor that becomes ~3.3KB per level.  Nine levels -- reached before the
current limit fires -- total ~30KB plus the initial call chain, which
exceeds the 32KB KASAN stack.  The overflow hits the VMAP_STACK guard
page and causes a non-recoverable kernel panic (BUG: stack guard page
was hit).

On non-KASAN kernels the same loop is safely caught by the existing
limit: the "Dead loop on virtual device" message fires and the packet
is dropped without any stack overflow.

Reduce XMIT_RECURSION_LIMIT to 3 when CONFIG_KASAN is enabled.  This
keeps the recursion counter well within the 32KB KASAN stack budget
while preserving the established limit of 8 for production kernels.

The recursion path triggering this is:

  __dev_queue_xmit -> dev_hard_start_xmit -> ipvlan_start_xmit
  -> ipvlan_queue_xmit -> ipvlan_process_outbound -> ip_local_out
  -> nf_hook (IPVS) -> ip_vs_in_hook -> ip_vs_nat_xmit -> ip_output
  -> ip_finish_output2 -> neigh_resolve_output -> __dev_queue_xmit

Tested:
  - KASAN kernel (6.8.12 x86_64): panic before fix, "Dead loop"
    drop after fix.
  - Non-KASAN kernel (6.8.12 x86_64): "Dead loop" drop both before
    and after fix (no behavior change for production kernels).

Fixes: 2ad7bf363841 ("ipvlan: Initial check-in of the IPVLAN driver.")
Cc: stable@vger.kernel.org
Signed-off-by: Tristan Madani <tristan@talencesecurity.com>
---
v2: Switch from per-driver recursion guard in ipvlan_core.c to reducing
    the global XMIT_RECURSION_LIMIT under CONFIG_KASAN, as suggested
    by Eric Dumazet.

 include/linux/netdevice.h | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h
index 9981d637f8b54..bdcb61d352afb 100644
--- a/include/linux/netdevice.h
+++ b/include/linux/netdevice.h
@@ -3640,7 +3640,11 @@ struct page_pool_bh {
 };
 DECLARE_PER_CPU(struct page_pool_bh, system_page_pool);
 
+#ifdef CONFIG_KASAN
+#define XMIT_RECURSION_LIMIT	3
+#else
 #define XMIT_RECURSION_LIMIT	8
+#endif
 
 #ifndef CONFIG_PREEMPT_RT
 static inline int dev_recursion_level(void)
-- 
2.47.3