* [PATCH net] bonding: fix skb_under_panic in bond_ns_send() over stacked VLANs
@ 2026-07-19 23:21 Xiang Mei (Microsoft)
2026-07-20 17:58 ` Jay Vosburgh
0 siblings, 1 reply; 4+ messages in thread
From: Xiang Mei (Microsoft) @ 2026-07-19 23:21 UTC (permalink / raw)
To: Jay Vosburgh, Andrew Lunn, David S . Miller, Eric Dumazet,
Jakub Kicinski, Paolo Abeni
Cc: Hangbin Liu, netdev, linux-kernel, AutonomousCodeSecurity,
tgopinath, kys, Xiang Mei (Microsoft)
bond_ns_send() builds an IPv6 Neighbor Solicitation with
ndisc_ns_create(), which reserves exactly LL_RESERVED_SPACE(dev) +
sizeof(struct ipv6hdr) of headroom for the later ip6_nd_hdr() push.
bond_handle_vlan() then inserts the collected VLAN tags into the skb;
each inner tag consumes VLAN_HLEN of that headroom via skb_push(). With
enough stacked VLAN devices between the bond and the ns_ip6_target, the
reserved IPv6 headroom is exhausted, so the subsequent
ndisc_send_skb() -> ip6_nd_hdr() -> skb_push(sizeof(struct ipv6hdr))
underflows past skb->head and hits skb_under_panic().
Restore the required headroom with skb_cow_head() after VLAN insertion
and before handing the skb to ndisc_send_skb(); drop the probe on
allocation failure. For paths that did not exhaust the headroom this is
a no-op, so previously working configurations are unaffected.
skbuff: skb_under_panic: len:84 put:40 head:... data:... tail:0x50 end:0x180 dev:veth0
kernel BUG at net/core/skbuff.c:214!
Oops: invalid opcode: 0000 [#1] SMP KASAN NOPTI
Workqueue: bond0 bond_arp_monitor
RIP: 0010:skb_panic+0x142/0x230
Call Trace:
skb_push (net/core/skbuff.c:224)
ndisc_send_skb (net/ipv6/ndisc.c:454 net/ipv6/ndisc.c:506)
bond_ns_send (drivers/net/bonding/bond_main.c:3255)
bond_ns_send_all (drivers/net/bonding/bond_main.c:3313)
bond_arp_monitor (drivers/net/bonding/bond_main.c:3458)
process_one_work (kernel/workqueue.c:3322)
worker_thread (kernel/workqueue.c:3405)
kthread (kernel/kthread.c:436)
Kernel panic - not syncing: Fatal exception
Fixes: 4e24be018eb9 ("bonding: add new parameter ns_targets")
Reported-by: AutonomousCodeSecurity@microsoft.com
Signed-off-by: Xiang Mei (Microsoft) <xmei5@asu.edu>
---
drivers/net/bonding/bond_main.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/drivers/net/bonding/bond_main.c b/drivers/net/bonding/bond_main.c
index e044fc733b8c..3ac3418c9498 100644
--- a/drivers/net/bonding/bond_main.c
+++ b/drivers/net/bonding/bond_main.c
@@ -3251,6 +3251,10 @@ static void bond_ns_send(struct slave *slave, const struct in6_addr *daddr,
addrconf_addr_solict_mult(daddr, &mcaddr);
if (bond_handle_vlan(slave, tags, skb)) {
+ if (skb_cow_head(skb, sizeof(struct ipv6hdr))) {
+ kfree_skb(skb);
+ return;
+ }
slave_update_last_tx(slave);
ndisc_send_skb(skb, &mcaddr, saddr);
}
--
2.43.0
^ permalink raw reply related [flat|nested] 4+ messages in thread
* Re: [PATCH net] bonding: fix skb_under_panic in bond_ns_send() over stacked VLANs
2026-07-19 23:21 [PATCH net] bonding: fix skb_under_panic in bond_ns_send() over stacked VLANs Xiang Mei (Microsoft)
@ 2026-07-20 17:58 ` Jay Vosburgh
2026-07-29 23:30 ` Xiang Mei
0 siblings, 1 reply; 4+ messages in thread
From: Jay Vosburgh @ 2026-07-20 17:58 UTC (permalink / raw)
To: Xiang Mei (Microsoft)
Cc: Andrew Lunn, David S . Miller, Eric Dumazet, Jakub Kicinski,
Paolo Abeni, Hangbin Liu, netdev, linux-kernel,
AutonomousCodeSecurity, tgopinath, kys
Xiang Mei (Microsoft) <xmei5@asu.edu> wrote:
>bond_ns_send() builds an IPv6 Neighbor Solicitation with
>ndisc_ns_create(), which reserves exactly LL_RESERVED_SPACE(dev) +
>sizeof(struct ipv6hdr) of headroom for the later ip6_nd_hdr() push.
>bond_handle_vlan() then inserts the collected VLAN tags into the skb;
>each inner tag consumes VLAN_HLEN of that headroom via skb_push(). With
>enough stacked VLAN devices between the bond and the ns_ip6_target, the
>reserved IPv6 headroom is exhausted, so the subsequent
>ndisc_send_skb() -> ip6_nd_hdr() -> skb_push(sizeof(struct ipv6hdr))
>underflows past skb->head and hits skb_under_panic().
How many stacked VLANs is "enough"? I'm guessing that it's
somewhat device dependent, as LL_RESERVED_SPACE includes
dev->needed_headroom, but as a ballpark here, was it more in the realm
of 3, or 30?
Also, why doesn't the skb_push called via bond_handle_vlan ->
vlan_insert_tag_set_proto -> vlan_insert_tag -> vlan_insert_inner_tag ->
__vlan_inser_inner_tag trigger the skb_under_panic?
Does adding one or two more nested VLANs move the panic into the
above call path?
>Restore the required headroom with skb_cow_head() after VLAN insertion
>and before handing the skb to ndisc_send_skb(); drop the probe on
>allocation failure. For paths that did not exhaust the headroom this is
>a no-op, so previously working configurations are unaffected.
>
> skbuff: skb_under_panic: len:84 put:40 head:... data:... tail:0x50 end:0x180 dev:veth0
> kernel BUG at net/core/skbuff.c:214!
The above text doesn't seem to match with current net-next:
pr_emerg("%s: text:%px len:%d put:%d head:%px data:%px tail:%#lx end:%#l
x dev:%s\n",
I presume you replaced the "head:" and "data:" values with
"..."; did you also edit out the "text:" that should precede "len:"?
I'm wondering because without the actual values, it's not clear
how far beyond head the data pointer went, and thus whether adding more
VLANs would move the failure into the VLAN tag code.
-J
> Oops: invalid opcode: 0000 [#1] SMP KASAN NOPTI
> Workqueue: bond0 bond_arp_monitor
> RIP: 0010:skb_panic+0x142/0x230
> Call Trace:
> skb_push (net/core/skbuff.c:224)
> ndisc_send_skb (net/ipv6/ndisc.c:454 net/ipv6/ndisc.c:506)
> bond_ns_send (drivers/net/bonding/bond_main.c:3255)
> bond_ns_send_all (drivers/net/bonding/bond_main.c:3313)
> bond_arp_monitor (drivers/net/bonding/bond_main.c:3458)
> process_one_work (kernel/workqueue.c:3322)
> worker_thread (kernel/workqueue.c:3405)
> kthread (kernel/kthread.c:436)
> Kernel panic - not syncing: Fatal exception
>
>Fixes: 4e24be018eb9 ("bonding: add new parameter ns_targets")
>Reported-by: AutonomousCodeSecurity@microsoft.com
>Signed-off-by: Xiang Mei (Microsoft) <xmei5@asu.edu>
>---
> drivers/net/bonding/bond_main.c | 4 ++++
> 1 file changed, 4 insertions(+)
>
>diff --git a/drivers/net/bonding/bond_main.c b/drivers/net/bonding/bond_main.c
>index e044fc733b8c..3ac3418c9498 100644
>--- a/drivers/net/bonding/bond_main.c
>+++ b/drivers/net/bonding/bond_main.c
>@@ -3251,6 +3251,10 @@ static void bond_ns_send(struct slave *slave, const struct in6_addr *daddr,
>
> addrconf_addr_solict_mult(daddr, &mcaddr);
> if (bond_handle_vlan(slave, tags, skb)) {
>+ if (skb_cow_head(skb, sizeof(struct ipv6hdr))) {
>+ kfree_skb(skb);
>+ return;
>+ }
> slave_update_last_tx(slave);
> ndisc_send_skb(skb, &mcaddr, saddr);
> }
>--
>2.43.0
>
---
-Jay Vosburgh, jv@jvosburgh.net
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH net] bonding: fix skb_under_panic in bond_ns_send() over stacked VLANs
2026-07-20 17:58 ` Jay Vosburgh
@ 2026-07-29 23:30 ` Xiang Mei
2026-07-30 3:50 ` Hangbin Liu
0 siblings, 1 reply; 4+ messages in thread
From: Xiang Mei @ 2026-07-29 23:30 UTC (permalink / raw)
To: Jay Vosburgh
Cc: Andrew Lunn, David S . Miller, Eric Dumazet, Jakub Kicinski,
Paolo Abeni, Hangbin Liu, netdev, linux-kernel,
AutonomousCodeSecurity, tgopinath, kys
Thanks for the review, and sorry for the slow reply.
On Mon, Jul 20, 2026 at 10:58 AM Jay Vosburgh <jv@jvosburgh.net> wrote:
>
> Xiang Mei (Microsoft) <xmei5@asu.edu> wrote:
>
> >bond_ns_send() builds an IPv6 Neighbor Solicitation with
> >ndisc_ns_create(), which reserves exactly LL_RESERVED_SPACE(dev) +
> >sizeof(struct ipv6hdr) of headroom for the later ip6_nd_hdr() push.
> >bond_handle_vlan() then inserts the collected VLAN tags into the skb;
> >each inner tag consumes VLAN_HLEN of that headroom via skb_push(). With
> >enough stacked VLAN devices between the bond and the ns_ip6_target, the
> >reserved IPv6 headroom is exhausted, so the subsequent
> >ndisc_send_skb() -> ip6_nd_hdr() -> skb_push(sizeof(struct ipv6hdr))
> >underflows past skb->head and hits skb_under_panic().
>
> How many stacked VLANs is "enough"? I'm guessing that it's
> somewhat device dependent, as LL_RESERVED_SPACE includes
> dev->needed_headroom, but as a ballpark here, was it more in the realm
> of 3, or 30?
Exactly 6, and it can only ever be 6, because that is also the deepest
stack the netdev nesting limit allows.
For the veth slave in my reproducer LL_RESERVED_SPACE(veth0) = 16, so
ndisc_alloc_skb() reserves 16 + sizeof(struct ipv6hdr) = 56 bytes. With
N VLAN devices above bond0, bond_handle_vlan() puts tags[0] in
skb->vlan_tci (costs no headroom) and pushes the other N-1 inline at
VLAN_HLEN each:
headroom before ip6_nd_hdr() = 56 - 4 * (N - 1) (it needs 40)
N = 5: 40 -> skb->data lands exactly on skb->head, passes by 0
N = 6: 36 -> 4 bytes short -> skb_under_panic()
N >= 7 is unreachable: veth0 -> bond0 -> v0..v5 already exhausts
MAX_NEST_DEV (8), so creating v6 fails with -EMLINK in
__netdev_upper_dev_link(). Sweeping N = 1..7 on the unpatched kernel
panics only at N = 6, and it panics on every boot; with the patch
applied the same sweep is clean throughout.
You are right that it is device dependent, but only in that first term.
The panic needs hlen < 20, i.e. hlen == 16, i.e. hard_header_len +
needed_headroom <= 15. For an Ethernet slave that means
needed_headroom <= 1, which covers veth and plenty of plain NICs. A
slave with needed_headroom >= 2 gets hlen == 32 and cannot hit this at
any legal stacking depth.
>
> Also, why doesn't the skb_push called via bond_handle_vlan ->
> vlan_insert_tag_set_proto -> vlan_insert_tag -> vlan_insert_inner_tag ->
> __vlan_inser_inner_tag trigger the skb_under_panic?
>
Because __vlan_insert_inner_tag() grows the head before it pushes:
if (skb_cow_head(skb, meta_len + VLAN_HLEN) < 0)
return -ENOMEM;
skb_push(skb, VLAN_HLEN);
so short headroom means a pskb_expand_head() realloc, not an underflow.
ip6_nd_hdr()'s skb_push(sizeof(struct ipv6hdr)) is the only unguarded
push in the chain, which is why the fault always lands there.
> Does adding one or two more nested VLANs move the panic into the
> above call path?
>
No, for both of the reasons above: there is no room to add more
(-EMLINK at N = 7), and skb_cow_head() would absorb it if there were.
> >Restore the required headroom with skb_cow_head() after VLAN insertion
> >and before handing the skb to ndisc_send_skb(); drop the probe on
> >allocation failure. For paths that did not exhaust the headroom this is
> >a no-op, so previously working configurations are unaffected.
> >
> > skbuff: skb_under_panic: len:84 put:40 head:... data:... tail:0x50 end:0x180 dev:veth0
> > kernel BUG at net/core/skbuff.c:214!
>
> The above text doesn't seem to match with current net-next:
>
> pr_emerg("%s: text:%px len:%d put:%d head:%px data:%px tail:%#lx end:%#l
> x dev:%s\n",
>
> I presume you replaced the "head:" and "data:" values with
> "..."; did you also edit out the "text:" that should precede "len:"?
>
Yes, sorry, I over-trimmed it. Orignal:
skbuff: skb_under_panic: text:ffffffff85eef44b len:84 put:40
head:ffff88801400b740 data:ffff88801400b73c tail:0x50 end:0x180
dev:veth0
data is 4 bytes below head, i.e. the N = 6 case above.
Now the part that matters more than the panic, and that changes what the
right fix is.
Following your VLAN-insertion question, I noticed vlan_insert_tag() passes
mac_len = ETH_HLEN, i.e. it assumes skb->data is at an Ethernet header.
That holds for bond_arp_send(), because arp_create() calls
dev_hard_header() first. It does not hold for bond_ns_send():
ndisc_ns_create() leaves both the IPv6 header and L2 to
ndisc_send_skb(), so at bond_handle_vlan() time skb->data points at the
nd_msg. __vlan_insert_inner_tag() then memmoves the first 12 bytes of
the ICMPv6 message down by 4 and writes the tag at payload offset 12,
and transport_header is left stale, so the checksum is stored 4 bytes
per tag too late.
Sniffing the slave's peer confirms it. With two VLANs (one inline tag,
VID 11):
60 00 00 00 00 1c 3a ff payload_len 28 (+4)
87 00 00 00 00 00 52 db cksum 0, 0x52db written 4 bytes late
20 01 0d b8 81 00 00 0b the 8100/000b tag sits in the target
00 01 00 00 00 00 00 00 00 00 00 02
So bond NS monitoring over more than one level of VLAN has never worked:
no peer can answer these probes. The panic at N = 6 is just where the
same bogus insertion also consumes the last of the ipv6hdr reservation.
That makes the headroom-only patch the wrong fix: it stops the crash,
but bond_handle_vlan() has already mis-tagged the packet by then, so
every 2+ VLAN probe stays corrupt.
Both symptoms have the same cause, tags inserted before any header
exists, so the fix is to build the probe in the order the stack uses:
- push the IPv6 header first, so the reservation is consumed up front
and the later tag inserts grow the head via skb_vlan_push() instead
of eating it;
- run the probe through NF_INET_LOCAL_OUT as a bare IPv6 packet,
exactly what ndisc_send_skb() feeds the hooks today. This needs a
small export (ndisc_attach_dst()) because ip6_route_me_harder()
dereferences skb_dst() unconditionally;
- only then push the link-layer header and add the VLAN tags, so they
land at a real Ethernet header and the target address is untouched.
With that applied the N = 1..6 wire capture is clean throughout. I kept
LOCAL_OUT deliberately so NS monitoring keeps traversing netfilter the
way it does today; dropping it would be simpler, but a silent behaviour
change.
I am not familiar with the bonding and ndisc internals, so I posted it as
an RFC rather than presenting it as settled:
https://lore.kernel.org/netdev/20260729230119.2717507-1-xmei5@asu.edu/T/#t
Thanks,
Xiang
> I'm wondering because without the actual values, it's not clear
> how far beyond head the data pointer went, and thus whether adding more
> VLANs would move the failure into the VLAN tag code.
>
> -J
>
> > Oops: invalid opcode: 0000 [#1] SMP KASAN NOPTI
> > Workqueue: bond0 bond_arp_monitor
> > RIP: 0010:skb_panic+0x142/0x230
> > Call Trace:
> > skb_push (net/core/skbuff.c:224)
> > ndisc_send_skb (net/ipv6/ndisc.c:454 net/ipv6/ndisc.c:506)
> > bond_ns_send (drivers/net/bonding/bond_main.c:3255)
> > bond_ns_send_all (drivers/net/bonding/bond_main.c:3313)
> > bond_arp_monitor (drivers/net/bonding/bond_main.c:3458)
> > process_one_work (kernel/workqueue.c:3322)
> > worker_thread (kernel/workqueue.c:3405)
> > kthread (kernel/kthread.c:436)
> > Kernel panic - not syncing: Fatal exception
> >
> >Fixes: 4e24be018eb9 ("bonding: add new parameter ns_targets")
> >Reported-by: AutonomousCodeSecurity@microsoft.com
> >Signed-off-by: Xiang Mei (Microsoft) <xmei5@asu.edu>
> >---
> > drivers/net/bonding/bond_main.c | 4 ++++
> > 1 file changed, 4 insertions(+)
> >
> >diff --git a/drivers/net/bonding/bond_main.c b/drivers/net/bonding/bond_main.c
> >index e044fc733b8c..3ac3418c9498 100644
> >--- a/drivers/net/bonding/bond_main.c
> >+++ b/drivers/net/bonding/bond_main.c
> >@@ -3251,6 +3251,10 @@ static void bond_ns_send(struct slave *slave, const struct in6_addr *daddr,
> >
> > addrconf_addr_solict_mult(daddr, &mcaddr);
> > if (bond_handle_vlan(slave, tags, skb)) {
> >+ if (skb_cow_head(skb, sizeof(struct ipv6hdr))) {
> >+ kfree_skb(skb);
> >+ return;
> >+ }
> > slave_update_last_tx(slave);
> > ndisc_send_skb(skb, &mcaddr, saddr);
> > }
> >--
> >2.43.0
> >
>
> ---
> -Jay Vosburgh, jv@jvosburgh.net
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH net] bonding: fix skb_under_panic in bond_ns_send() over stacked VLANs
2026-07-29 23:30 ` Xiang Mei
@ 2026-07-30 3:50 ` Hangbin Liu
0 siblings, 0 replies; 4+ messages in thread
From: Hangbin Liu @ 2026-07-30 3:50 UTC (permalink / raw)
To: Xiang Mei
Cc: Jay Vosburgh, Andrew Lunn, David S . Miller, Eric Dumazet,
Jakub Kicinski, Paolo Abeni, netdev, linux-kernel,
AutonomousCodeSecurity, tgopinath, kys
Hi Xiang,
On 29.07.2026 16:30, Xiang Mei wrote:
>> Does adding one or two more nested VLANs move the panic into the
>> above call path?
>>
>
>No, for both of the reasons above: there is no room to add more
>(-EMLINK at N = 7), and skb_cow_head() would absorb it if there were.
What about the scenario that involves nested VLANs? For example:
vlan.1 - vlan.1.2 ... vlan.1..6 - bond - bond.1 - bond.1.2 ... bond.1..6?
>Both symptoms have the same cause, tags inserted before any header
>exists, so the fix is to build the probe in the order the stack uses:
>
> - push the IPv6 header first, so the reservation is consumed up front
> and the later tag inserts grow the head via skb_vlan_push() instead
> of eating it;
>
> - run the probe through NF_INET_LOCAL_OUT as a bare IPv6 packet,
> exactly what ndisc_send_skb() feeds the hooks today. This needs a
> small export (ndisc_attach_dst()) because ip6_route_me_harder()
> dereferences skb_dst() unconditionally;
>
> - only then push the link-layer header and add the VLAN tags, so they
> land at a real Ethernet header and the target address is untouched.
>
>With that applied the N = 1..6 wire capture is clean throughout. I kept
>LOCAL_OUT deliberately so NS monitoring keeps traversing netfilter the
>way it does today; dropping it would be simpler, but a silent behaviour
>change.
>
>I am not familiar with the bonding and ndisc internals, so I posted it as
>an RFC rather than presenting it as settled:
>
> https://lore.kernel.org/netdev/20260729230119.2717507-1-xmei5@asu.edu/T/#t
I'm not sure if calling NF_INET_LOCAL_OUT in bond code is a good idea. Waiting
for other's opinion.
Thanks
Hangbin
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2026-07-30 3:50 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-07-19 23:21 [PATCH net] bonding: fix skb_under_panic in bond_ns_send() over stacked VLANs Xiang Mei (Microsoft)
2026-07-20 17:58 ` Jay Vosburgh
2026-07-29 23:30 ` Xiang Mei
2026-07-30 3:50 ` Hangbin Liu
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.