Re: Fw: [Bug 220774] New: netem is broken in 6.18

[Cong Wang <xiyou.wangcong@gmail.com>]

Hi Will, Jamal and Jakub,

I already warned you many times before you applied it. Now we have users
complaining, please let me know if you still respect users.

Also, Jamal, if I remember correctly, you said you will work on a long
term solution, now after 4 months, please let us know what your plan is.

Regards,
Cong


On Mon, Nov 10, 2025 at 12:38:07PM -0800, Stephen Hemminger wrote:
> Regression caused by:
> 
> commit ec8e0e3d7adef940cdf9475e2352c0680189d14e
> Author: William Liu <will@willsroot.io>
> Date:   Tue Jul 8 16:43:26 2025 +0000
> 
>     net/sched: Restrict conditions for adding duplicating netems to qdisc tree
>     
>     netem_enqueue's duplication prevention logic breaks when a netem
>     resides in a qdisc tree with other netems - this can lead to a
>     soft lockup and OOM loop in netem_dequeue, as seen in [1].
>     Ensure that a duplicating netem cannot exist in a tree with other
>     netems.
>     
>     Previous approaches suggested in discussions in chronological order:
>     
>     1) Track duplication status or ttl in the sk_buff struct. Considered
>     too specific a use case to extend such a struct, though this would
>     be a resilient fix and address other previous and potential future
>     DOS bugs like the one described in loopy fun [2].
>     
>     2) Restrict netem_enqueue recursion depth like in act_mirred with a
>     per cpu variable. However, netem_dequeue can call enqueue on its
>     child, and the depth restriction could be bypassed if the child is a
>     netem.
>     
>     3) Use the same approach as in 2, but add metadata in netem_skb_cb
>     to handle the netem_dequeue case and track a packet's involvement
>     in duplication. This is an overly complex approach, and Jamal
>     notes that the skb cb can be overwritten to circumvent this
>     safeguard.
>     
>     4) Prevent the addition of a netem to a qdisc tree if its ancestral
>     path contains a netem. However, filters and actions can cause a
>     packet to change paths when re-enqueued to the root from netem
>     duplication, leading us to the current solution: prevent a
>     duplicating netem from inhabiting the same tree as other netems.
>     
>     [1] https://lore.kernel.org/netdev/8DuRWwfqjoRDLDmBMlIfbrsZg9Gx50DHJc1ilxsEBNe2D6NMoigR_eIRIG0LOjMc3r10nUUZtArXx4oZBIdUfZQrwjcQhdinnMis_0G7VEk=@willsroot.io/
>     [2] https://lwn.net/Articles/719297/
>     
>     Fixes: 0afb51e72855 ("[PKT_SCHED]: netem: reinsert for duplication")
>     Reported-by: William Liu <will@willsroot.io>
>     Reported-by: Savino Dicanosa <savy@syst3mfailure.io>
>     Signed-off-by: William Liu <will@willsroot.io>
>     Signed-off-by: Savino Dicanosa <savy@syst3mfailure.io>
>     Acked-by: Jamal Hadi Salim <jhs@mojatatu.com>
>     Link: https://patch.msgid.link/20250708164141.875402-1-will@willsroot.io
>     Signed-off-by: Jakub Kicinski <kuba@kernel.org>
> 
> 
> Begin forwarded message:
> 
> Date: Mon, 10 Nov 2025 19:13:57 +0000
> From: bugzilla-daemon@kernel.org
> To: stephen@networkplumber.org
> Subject: [Bug 220774] New: netem is broken in 6.18
> 
> 
> https://bugzilla.kernel.org/show_bug.cgi?id=220774
> 
>             Bug ID: 220774
>            Summary: netem is broken in 6.18
>            Product: Networking
>            Version: 2.5
>           Hardware: All
>                 OS: Linux
>             Status: NEW
>           Severity: high
>           Priority: P3
>          Component: Other
>           Assignee: stephen@networkplumber.org
>           Reporter: jschung2@proton.me
>         Regression: No
> 
> [jschung@localhost ~]$ cat test.sh 
> #!/bin/bash
> 
> DEV="eth0"
> NUM_QUEUES=32
> DUPLICATE_PERCENT="5%"
> 
> tc qdisc del dev $DEV root > /dev/null 2>&1
> tc qdisc add dev $DEV root handle 1: mq
> 
> for i in $(seq 1 $NUM_QUEUES); do
>     HANDLE_ID=$((i * 10))
>     PARENT_ID="1:$i"
>     tc qdisc add dev $DEV parent $PARENT_ID handle ${HANDLE_ID}: netem
> duplicate $DUPLICATE_PERCENT
> done
> 
> [jschung@localhost ~]$ sudo ./test.sh 
> [  2976.073299] netem: change failed
> Error: netem: cannot mix duplicating netems with other netems in tree.
> 
> [jschung@localhost ~]$ uname -r
> 6.18.0-rc4
> 
> -- 
> You may reply to this email to add a comment.
> 
> You are receiving this mail because:
> You are the assignee for the bug.