[Patch net v8 0/9] netem: Fix skb duplication logic and prevent infinite loops

[Patch net v8 0/9] netem: Fix skb duplication logic and prevent infinite loops

[Cong Wang]

This patchset fixes the infinite loops due to duplication in netem, the
real root cause of this problem is enqueuing to the root qdisc, which is
now changed to enqueuing to the same qdisc. This is more reasonable,
more intuitive from users' perspective, less error-prone and more elegant
from kernel developers' perspective.

Please see more details in patch 4/9 which contains two pages of detailed
explanation including why it is safe and better.

This reverts the offending commits from William which clearly broke
mq+netem use cases, as reported by two users.

All the TC test cases pass with this patchset.

---
v8: Fixed test 7c3b

v7: Fixed a typo in subject
    Fixed a missing qdisc_tree_reduce_backlog()
    Added a new selftest for backlog validation

v6: Dropped the init_user_ns check patch
    Reordered the qfq patch
    Rebased to the latest -net branch

v5: Reverted the offending commits
    Added a init_user_ns check (4/9)
    Rebased to the latest -net branch

v4: Added a fix for qfq qdisc (2/6)
    Updated 1/6 patch description
    Added a patch to update the enqueue reentrant behaviour tests

v3: Fixed the root cause of enqueuing to root
    Switched back to netem_skb_cb safely
    Added two more test cases

v2: Fixed a typo
    Improved tdc selftest to check sent bytes

Cong Wang (9):
  net_sched: Check the return value of qfq_choose_next_agg()
  Revert "net/sched: Restrict conditions for adding duplicating netems
    to qdisc tree"
  Revert "selftests/tc-testing: Add tests for restrictions on netem
    duplication"
  net_sched: Implement the right netem duplication behavior
  selftests/tc-testing: Add a nested netem duplicate test
  selftests/tc-testing: Add a test case for prio with netem duplicate
  selftests/tc-testing: Add a test case for mq with netem duplicate
  selftests/tc-testing: Update test cases with netem duplicate
  selftests/tc-testing: Add a test case for HTB with netem

 net/sched/sch_netem.c                         |  67 +++-----
 net/sched/sch_qfq.c                           |   2 +
 .../tc-testing/tc-tests/infra/qdiscs.json     | 144 ++++++++++++++----
 .../tc-testing/tc-tests/qdiscs/netem.json     |  90 +++--------
 4 files changed, 153 insertions(+), 150 deletions(-)

-- 
2.34.1

[Patch net v8 1/9] net_sched: Check the return value of qfq_choose_next_agg()

[Cong Wang]

qfq_choose_next_agg() could return NULL so its return value should be
properly checked unless NULL is acceptable.

There are two cases we need to deal with:

1) q->in_serv_agg, which is okay with NULL since it is either checked or
   just compared with other pointer without dereferencing. In fact, it
   is even intentionally set to NULL in one of the cases.

2) in_serv_agg, which is a temporary local variable, which is not okay
   with NULL, since it is dereferenced immediately, hence must be checked.

This fix corrects one of the 2nd cases, and leaving the 1st case as they are.

Although this bug is triggered with the netem duplicate change, the root
cause is still within qfq qdisc.

Fixes: 462dbc9101ac ("pkt_sched: QFQ Plus: fair-queueing service at DRR cost")
Reviewed-by: Xiang Mei <xmei5@asu.edu>
Acked-by: Stephen Hemminger <stephen@networkplumber.org>
Signed-off-by: Cong Wang <xiyou.wangcong@gmail.com>
---
 net/sched/sch_qfq.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/net/sched/sch_qfq.c b/net/sched/sch_qfq.c
index 9d59090bbe93..4b963b6b041e 100644
--- a/net/sched/sch_qfq.c
+++ b/net/sched/sch_qfq.c
@@ -1147,6 +1147,8 @@ static struct sk_buff *qfq_dequeue(struct Qdisc *sch)
 		 * choose the new aggregate to serve.
 		 */
 		in_serv_agg = q->in_serv_agg = qfq_choose_next_agg(q);
+		if (!in_serv_agg)
+			return NULL;
 		skb = qfq_peek_skb(in_serv_agg, &cl, &len);
 	}
 	if (!skb)
-- 
2.34.1

[Patch net v8 2/9] Revert "net/sched: Restrict conditions for adding duplicating netems to qdisc tree"

[Cong Wang]

From: Cong Wang <cwang@multikernel.io>

This reverts commit ec8e0e3d7adef940cdf9475e2352c0680189d14e since it
breaks mq+netem.

Reported-by: Ji-Soo Chung <jschung2@proton.me>
Reported-by: Gerlinde <lrGerlinde@mailfence.com>
Closes: https://bugzilla.kernel.org/show_bug.cgi?id=220774
Signed-off-by: Cong Wang <cwang@multikernel.io>
---
 net/sched/sch_netem.c | 40 ----------------------------------------
 1 file changed, 40 deletions(-)

diff --git a/net/sched/sch_netem.c b/net/sched/sch_netem.c
index 32a5f3304046..a9ea40c13527 100644
--- a/net/sched/sch_netem.c
+++ b/net/sched/sch_netem.c
@@ -974,41 +974,6 @@ static int parse_attr(struct nlattr *tb[], int maxtype, struct nlattr *nla,
 	return 0;
 }
 
-static const struct Qdisc_class_ops netem_class_ops;
-
-static int check_netem_in_tree(struct Qdisc *sch, bool duplicates,
-			       struct netlink_ext_ack *extack)
-{
-	struct Qdisc *root, *q;
-	unsigned int i;
-
-	root = qdisc_root_sleeping(sch);
-
-	if (sch != root && root->ops->cl_ops == &netem_class_ops) {
-		if (duplicates ||
-		    ((struct netem_sched_data *)qdisc_priv(root))->duplicate)
-			goto err;
-	}
-
-	if (!qdisc_dev(root))
-		return 0;
-
-	hash_for_each(qdisc_dev(root)->qdisc_hash, i, q, hash) {
-		if (sch != q && q->ops->cl_ops == &netem_class_ops) {
-			if (duplicates ||
-			    ((struct netem_sched_data *)qdisc_priv(q))->duplicate)
-				goto err;
-		}
-	}
-
-	return 0;
-
-err:
-	NL_SET_ERR_MSG(extack,
-		       "netem: cannot mix duplicating netems with other netems in tree");
-	return -EINVAL;
-}
-
 /* Parse netlink message to set options */
 static int netem_change(struct Qdisc *sch, struct nlattr *opt,
 			struct netlink_ext_ack *extack)
@@ -1067,11 +1032,6 @@ static int netem_change(struct Qdisc *sch, struct nlattr *opt,
 	q->gap = qopt->gap;
 	q->counter = 0;
 	q->loss = qopt->loss;
-
-	ret = check_netem_in_tree(sch, qopt->duplicate, extack);
-	if (ret)
-		goto unlock;
-
 	q->duplicate = qopt->duplicate;
 
 	/* for compatibility with earlier versions.
-- 
2.34.1

[Patch net v8 3/9] Revert "selftests/tc-testing: Add tests for restrictions on netem duplication"

[Cong Wang]

From: Cong Wang <cwang@multikernel.io>

This reverts commit ecdec65ec78d67d3ebd17edc88b88312054abe0d.

Signed-off-by: Cong Wang <cwang@multikernel.io>
---
 .../tc-testing/tc-tests/infra/qdiscs.json     |  5 +-
 .../tc-testing/tc-tests/qdiscs/netem.json     | 81 -------------------
 2 files changed, 3 insertions(+), 83 deletions(-)

diff --git a/tools/testing/selftests/tc-testing/tc-tests/infra/qdiscs.json b/tools/testing/selftests/tc-testing/tc-tests/infra/qdiscs.json
index 6a39640aa2a8..ceb993ed04b2 100644
--- a/tools/testing/selftests/tc-testing/tc-tests/infra/qdiscs.json
+++ b/tools/testing/selftests/tc-testing/tc-tests/infra/qdiscs.json
@@ -702,6 +702,7 @@
             "$TC qdisc add dev $DUMMY parent 1:1 handle 2:0 netem duplicate 100%",
             "$TC filter add dev $DUMMY parent 1:0 protocol ip prio 1 u32 match ip dst 10.10.10.1/32 flowid 1:1",
             "$TC class add dev $DUMMY parent 1:0 classid 1:2 hfsc ls m2 10Mbit",
+            "$TC qdisc add dev $DUMMY parent 1:2 handle 3:0 netem duplicate 100%",
             "$TC filter add dev $DUMMY parent 1:0 protocol ip prio 2 u32 match ip dst 10.10.10.2/32 flowid 1:2",
             "ping -c 1 10.10.10.1 -I$DUMMY > /dev/null || true",
             "$TC filter del dev $DUMMY parent 1:0 protocol ip prio 1",
@@ -714,8 +715,8 @@
             {
                 "kind": "hfsc",
                 "handle": "1:",
-                "bytes": 294,
-                "packets": 3
+                "bytes": 392,
+                "packets": 4
             }
         ],
         "matchCount": "1",
diff --git a/tools/testing/selftests/tc-testing/tc-tests/qdiscs/netem.json b/tools/testing/selftests/tc-testing/tc-tests/qdiscs/netem.json
index 718d2df2aafa..3c4444961488 100644
--- a/tools/testing/selftests/tc-testing/tc-tests/qdiscs/netem.json
+++ b/tools/testing/selftests/tc-testing/tc-tests/qdiscs/netem.json
@@ -336,86 +336,5 @@
         "teardown": [
             "$TC qdisc del dev $DUMMY handle 1: root"
         ]
-    },
-    {
-        "id": "d34d",
-        "name": "NETEM test qdisc duplication restriction in qdisc tree in netem_change root",
-        "category": ["qdisc", "netem"],
-        "plugins": {
-            "requires": "nsPlugin"
-        },
-        "setup": [
-            "$TC qdisc add dev $DUMMY root handle 1: netem limit 1",
-            "$TC qdisc add dev $DUMMY parent 1: handle 2: netem limit 1"
-        ],
-        "cmdUnderTest": "$TC qdisc change dev $DUMMY handle 1: netem duplicate 50%",
-        "expExitCode": "2",
-        "verifyCmd": "$TC -s qdisc show dev $DUMMY",
-        "matchPattern": "qdisc netem",
-        "matchCount": "2",
-        "teardown": [
-            "$TC qdisc del dev $DUMMY handle 1:0 root"
-        ]
-    },
-    {
-        "id": "b33f",
-        "name": "NETEM test qdisc duplication restriction in qdisc tree in netem_change non-root",
-        "category": ["qdisc", "netem"],
-        "plugins": {
-            "requires": "nsPlugin"
-        },
-        "setup": [
-            "$TC qdisc add dev $DUMMY root handle 1: netem limit 1",
-            "$TC qdisc add dev $DUMMY parent 1: handle 2: netem limit 1"
-        ],
-        "cmdUnderTest": "$TC qdisc change dev $DUMMY handle 2: netem duplicate 50%",
-        "expExitCode": "2",
-        "verifyCmd": "$TC -s qdisc show dev $DUMMY",
-        "matchPattern": "qdisc netem",
-        "matchCount": "2",
-        "teardown": [
-            "$TC qdisc del dev $DUMMY handle 1:0 root"
-        ]
-    },
-    {
-        "id": "cafe",
-        "name": "NETEM test qdisc duplication restriction in qdisc tree",
-        "category": ["qdisc", "netem"],
-        "plugins": {
-            "requires": "nsPlugin"
-        },
-        "setup": [
-            "$TC qdisc add dev $DUMMY root handle 1: netem limit 1 duplicate 100%"
-        ],
-        "cmdUnderTest": "$TC qdisc add dev $DUMMY parent 1: handle 2: netem duplicate 100%",
-        "expExitCode": "2",
-        "verifyCmd": "$TC -s qdisc show dev $DUMMY",
-        "matchPattern": "qdisc netem",
-        "matchCount": "1",
-        "teardown": [
-            "$TC qdisc del dev $DUMMY handle 1:0 root"
-        ]
-    },
-    {
-        "id": "1337",
-        "name": "NETEM test qdisc duplication restriction in qdisc tree across branches",
-        "category": ["qdisc", "netem"],
-        "plugins": {
-            "requires": "nsPlugin"
-        },
-        "setup": [
-            "$TC qdisc add dev $DUMMY parent root handle 1:0 hfsc",
-            "$TC class add dev $DUMMY parent 1:0 classid 1:1 hfsc rt m2 10Mbit",
-            "$TC qdisc add dev $DUMMY parent 1:1 handle 2:0 netem",
-            "$TC class add dev $DUMMY parent 1:0 classid 1:2 hfsc rt m2 10Mbit"
-        ],
-        "cmdUnderTest": "$TC qdisc add dev $DUMMY parent 1:2 handle 3:0 netem duplicate 100%",
-        "expExitCode": "2",
-        "verifyCmd": "$TC -s qdisc show dev $DUMMY",
-        "matchPattern": "qdisc netem",
-        "matchCount": "1",
-        "teardown": [
-            "$TC qdisc del dev $DUMMY handle 1:0 root"
-        ]
     }
 ]
-- 
2.34.1

[Patch net v8 4/9] net_sched: Implement the right netem duplication behavior

[Cong Wang]

In the old behavior, duplicated packets were sent back to the root qdisc,
which could create dangerous infinite loops in hierarchical setups -
imagine a scenario where each level of a multi-stage netem hierarchy kept
feeding duplicates back to the top, potentially causing system instability
or resource exhaustion.

The new behavior elegantly solves this by enqueueing duplicates to the same
qdisc that created them, ensuring that packet duplication occurs exactly
once per netem stage in a controlled, predictable manner. This change
enables users to safely construct complex network emulation scenarios using
netem hierarchies (like the 4x multiplication demonstrated in testing)
without worrying about runaway packet generation, while still preserving
the intended duplication effects.

Another advantage of this approach is that it eliminates the enqueue reentrant
behaviour which triggered many vulnerabilities. See the last patch in this
patchset which updates the test cases for such vulnerabilities.

Now users can confidently chain multiple netem qdiscs together to achieve
sophisticated network impairment combinations, knowing that each stage will
apply its effects exactly once to the packet flow, making network testing
scenarios more reliable and results more deterministic.

I tested netem packet duplication in two configurations:
1. Nest netem-to-netem hierarchy using parent/child attachment
2. Single netem using prio qdisc with netem leaf

Setup commands and results:

Single netem hierarchy (prio + netem):
  tc qdisc add dev lo root handle 1: prio bands 3 priomap 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
  tc filter add dev lo parent 1:0 protocol ip matchall classid 1:1
  tc qdisc add dev lo parent 1:1 handle 10: netem limit 4 duplicate 100%

Result: 2x packet multiplication (1→2 packets)
  2 echo requests + 4 echo replies = 6 total packets

Expected behavior: Only one netem stage exists in this hierarchy, so
1 ping becomes 2 packets (100% duplication). The 2 echo requests generate
2 echo replies, which also get duplicated to 4 replies, yielding the
predictable total of 6 packets (2 requests + 4 replies).

Nest netem hierarchy (netem + netem):
  tc qdisc add dev lo root handle 1: netem limit 1000 duplicate 100%
  tc qdisc add dev lo parent 1: handle 2: netem limit 1000 duplicate 100%

Result: 4x packet multiplication (1→2→4 packets)
  4 echo requests + 16 echo replies = 20 total packets

Expected behavior: Root netem duplicates 1 ping to 2 packets, child netem
receives 2 packets and duplicates each to create 4 total packets. Since
ping operates bidirectionally, 4 echo requests generate 4 echo replies,
which also get duplicated through the same hierarchy (4→8→16), resulting
in the predictable total of 20 packets (4 requests + 16 replies).

The new netem duplication behavior does not break the documented
semantics of "creates a copy of the packet before queuing." The man page
description remains true since duplication occurs before the queuing
process, creating both original and duplicate packets that are then
enqueued. The documentation does not specify which qdisc should receive
the duplicates, only that copying happens before queuing. The implementation
choice to enqueue duplicates to the same qdisc (rather than root) is an
internal detail that maintains the documented behavior while preventing
infinite loops in hierarchical configurations.

Fixes: 0afb51e72855 ("[PKT_SCHED]: netem: reinsert for duplication")
Reported-by: William Liu <will@willsroot.io>
Reported-by: Savino Dicanosa <savy@syst3mfailure.io>
Acked-by: Stephen Hemminger <stephen@networkplumber.org>
Signed-off-by: Cong Wang <xiyou.wangcong@gmail.com>
---
 net/sched/sch_netem.c | 27 +++++++++++++++++----------
 1 file changed, 17 insertions(+), 10 deletions(-)

diff --git a/net/sched/sch_netem.c b/net/sched/sch_netem.c
index a9ea40c13527..b9a6b12951b4 100644
--- a/net/sched/sch_netem.c
+++ b/net/sched/sch_netem.c
@@ -165,6 +165,7 @@ struct netem_sched_data {
  */
 struct netem_skb_cb {
 	u64	        time_to_send;
+	u8		duplicate : 1;
 };
 
 static inline struct netem_skb_cb *netem_skb_cb(struct sk_buff *skb)
@@ -461,8 +462,16 @@ static int netem_enqueue(struct sk_buff *skb, struct Qdisc *sch,
 	skb->prev = NULL;
 
 	/* Random duplication */
-	if (q->duplicate && q->duplicate >= get_crandom(&q->dup_cor, &q->prng))
-		++count;
+	if (q->duplicate) {
+		bool dup = true;
+
+		if (netem_skb_cb(skb)->duplicate) {
+			netem_skb_cb(skb)->duplicate = 0;
+			dup = false;
+		}
+		if (dup && q->duplicate >= get_crandom(&q->dup_cor, &q->prng))
+			++count;
+	}
 
 	/* Drop packet? */
 	if (loss_event(q)) {
@@ -533,17 +542,15 @@ static int netem_enqueue(struct sk_buff *skb, struct Qdisc *sch,
 	}
 
 	/*
-	 * If doing duplication then re-insert at top of the
-	 * qdisc tree, since parent queuer expects that only one
-	 * skb will be queued.
+	 * If doing duplication then re-insert at the same qdisc,
+	 * as going back to the root would induce loops.
 	 */
 	if (skb2) {
-		struct Qdisc *rootq = qdisc_root_bh(sch);
-		u32 dupsave = q->duplicate; /* prevent duplicating a dup... */
+		unsigned int dup_len = qdisc_pkt_len(skb2);
 
-		q->duplicate = 0;
-		rootq->enqueue(skb2, rootq, to_free);
-		q->duplicate = dupsave;
+		netem_skb_cb(skb2)->duplicate = 1;
+		if (qdisc_enqueue(skb2, sch, to_free) == NET_XMIT_SUCCESS)
+			qdisc_tree_reduce_backlog(sch, -1, -dup_len);
 		skb2 = NULL;
 	}
 
-- 
2.34.1

[Patch net v8 5/9] selftests/tc-testing: Add a nested netem duplicate test

[Cong Wang]

Integrate the reproducer from William into tc-testing and use scapy
to generate packets for testing:

 # ./tdc.py -e 1676
  -- ns/SubPlugin.__init__
  -- scapy/SubPlugin.__init__
 Test 1676: NETEM nested duplicate 100%
 [ 1154.071135] v0p0id1676: entered promiscuous mode
 [ 1154.101066] virtio_net virtio0 enp1s0: entered promiscuous mode
 [ 1154.146301] virtio_net virtio0 enp1s0: left promiscuous mode
 .
 Sent 1 packets.
 [ 1154.173695] v0p0id1676: left promiscuous mode
 [ 1154.216159] v0p0id1676: entered promiscuous mode
 .
 Sent 1 packets.
 [ 1154.238398] v0p0id1676: left promiscuous mode
 [ 1154.260909] v0p0id1676: entered promiscuous mode
 .
 Sent 1 packets.
 [ 1154.282708] v0p0id1676: left promiscuous mode
 [ 1154.309399] v0p0id1676: entered promiscuous mode
 .
 Sent 1 packets.
 [ 1154.337949] v0p0id1676: left promiscuous mode
 [ 1154.360924] v0p0id1676: entered promiscuous mode
 .
 Sent 1 packets.
 [ 1154.383522] v0p0id1676: left promiscuous mode

 All test results:

 1..1
 ok 1 1676 - NETEM nested duplicate 100%

Reported-by: William Liu <will@willsroot.io>
Signed-off-by: Cong Wang <xiyou.wangcong@gmail.com>
---
 .../tc-testing/tc-tests/qdiscs/netem.json     | 25 +++++++++++++++++++
 1 file changed, 25 insertions(+)

diff --git a/tools/testing/selftests/tc-testing/tc-tests/qdiscs/netem.json b/tools/testing/selftests/tc-testing/tc-tests/qdiscs/netem.json
index 3c4444961488..03c4ceb22990 100644
--- a/tools/testing/selftests/tc-testing/tc-tests/qdiscs/netem.json
+++ b/tools/testing/selftests/tc-testing/tc-tests/qdiscs/netem.json
@@ -336,5 +336,30 @@
         "teardown": [
             "$TC qdisc del dev $DUMMY handle 1: root"
         ]
+    },
+    {
+        "id": "1676",
+        "name": "NETEM nested duplicate 100%",
+        "category": ["qdisc", "netem"],
+        "setup": [
+            "$TC qdisc add dev $DEV1 root handle 1: netem limit 1000 duplicate 100%",
+            "$TC qdisc add dev $DEV1 parent 1: handle 2: netem limit 1000 duplicate 100%"
+        ],
+        "teardown": [
+            "$TC qdisc del dev $DEV1 root"
+        ],
+        "plugins": {
+            "requires": ["nsPlugin", "scapyPlugin"]
+        },
+        "scapy": {
+            "iface": "$DEV0",
+            "count": 5,
+            "packet": "Ether()/IP(dst='10.10.10.1', src='10.10.10.10')/TCP(sport=12345, dport=80)"
+        },
+        "cmdUnderTest": "$TC -s qdisc show dev $DEV1",
+        "expExitCode": "0",
+        "verifyCmd": "$TC -s qdisc show dev $DEV1",
+        "matchPattern": "Sent [0-9]+ bytes [0-9]+ pkt",
+        "matchCount": "2"
     }
 ]
-- 
2.34.1

[Patch net v8 6/9] selftests/tc-testing: Add a test case for prio with netem duplicate

[Cong Wang]

Integrate the test case from Jamal into tc-testing:

Test 94a7: Test PRIO with NETEM duplication

All test results:

1..1
ok 1 94a7 - Test PRIO with NETEM duplication

Signed-off-by: Cong Wang <xiyou.wangcong@gmail.com>
---
 .../tc-testing/tc-tests/infra/qdiscs.json     | 29 +++++++++++++++++++
 1 file changed, 29 insertions(+)

diff --git a/tools/testing/selftests/tc-testing/tc-tests/infra/qdiscs.json b/tools/testing/selftests/tc-testing/tc-tests/infra/qdiscs.json
index ceb993ed04b2..b65fe669e00a 100644
--- a/tools/testing/selftests/tc-testing/tc-tests/infra/qdiscs.json
+++ b/tools/testing/selftests/tc-testing/tc-tests/infra/qdiscs.json
@@ -961,6 +961,35 @@
         "matchJSON": [],
         "teardown": [
             "$TC qdisc del dev $DUMMY root"
+	]
+    },
+    {
+        "id": "94a7",
+        "name": "Test PRIO with NETEM duplication",
+        "category": [
+            "qdisc",
+            "prio",
+            "netem"
+        ],
+        "plugins": {
+            "requires": [
+                "nsPlugin"
+            ]
+        },
+        "setup": [
+            "$IP link set dev $DUMMY up || true",
+            "$IP addr add 10.10.11.10/24 dev $DUMMY || true",
+            "$TC qdisc add dev $DUMMY root handle 1: prio bands 3 priomap 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0",
+            "$TC filter add dev $DUMMY parent 1:0 protocol ip matchall classid 1:1",
+            "$TC qdisc add dev $DUMMY parent 1:1 handle 10: netem limit 4 duplicate 100%"
+        ],
+        "cmdUnderTest": "ping -c1 -W0.01 -I $DUMMY 10.10.11.11",
+        "expExitCode": "1",
+        "verifyCmd": "$TC -s qdisc show dev $DUMMY | grep -A 5 'qdisc netem' | grep -E 'Sent [0-9]+ bytes [0-9]+ pkt'",
+        "matchPattern": "Sent \\d+ bytes (\\d+) pkt",
+        "matchCount": "1",
+        "teardown": [
+            "$TC qdisc del dev $DUMMY root handle 1: prio"
         ]
     },
     {
-- 
2.34.1

[Patch net v8 7/9] selftests/tc-testing: Add a test case for mq with netem duplicate

[Cong Wang]

Given that multi-queue NICs are prevalent and the global spinlock issue with
single netem instances is a known performance limitation, the setup using
mq as a parent for netem is an excellent and highly reasonable pattern for
applying netem effects like 100% duplication efficiently on modern Linux
systems.

Signed-off-by: Cong Wang <xiyou.wangcong@gmail.com>
---
 .../tc-testing/tc-tests/infra/qdiscs.json     | 31 +++++++++++++++++++
 1 file changed, 31 insertions(+)

diff --git a/tools/testing/selftests/tc-testing/tc-tests/infra/qdiscs.json b/tools/testing/selftests/tc-testing/tc-tests/infra/qdiscs.json
index b65fe669e00a..57e6b5f35070 100644
--- a/tools/testing/selftests/tc-testing/tc-tests/infra/qdiscs.json
+++ b/tools/testing/selftests/tc-testing/tc-tests/infra/qdiscs.json
@@ -1141,5 +1141,36 @@
         "teardown": [
             "$TC qdisc del dev $DUMMY root handle 1:"
         ]
+    },
+    {
+        "id": "94a8",
+        "name": "Test MQ with NETEM duplication",
+        "category": [
+            "qdisc",
+            "mq",
+            "netem"
+        ],
+        "plugins": {
+            "requires": ["nsPlugin", "scapyPlugin"]
+        },
+        "setup": [
+            "$IP link set dev $DEV1 up",
+            "$TC qdisc add dev $DEV1 root handle 1: mq",
+            "$TC qdisc add dev $DEV1 parent 1:1 handle 10: netem duplicate 100%",
+            "$TC qdisc add dev $DEV1 parent 1:2 handle 20: netem duplicate 100%"
+        ],
+        "scapy": {
+            "iface": "$DEV0",
+            "count": 5,
+            "packet": "Ether()/IP(dst='10.10.10.1', src='10.10.10.10')/ICMP()"
+        },
+        "cmdUnderTest": "$TC -s qdisc show dev $DEV1",
+        "expExitCode": "0",
+        "verifyCmd": "$TC -s qdisc show dev $DEV1 | grep -A 5 'qdisc netem' | grep -E 'Sent [0-9]+ bytes [0-9]+ pkt'",
+        "matchPattern": "Sent \\d+ bytes (\\d+) pkt",
+        "matchCount": "2",
+        "teardown": [
+            "$TC qdisc del dev $DEV1 root handle 1: mq"
+        ]
     }
 ]
-- 
2.34.1

[Patch net v8 8/9] selftests/tc-testing: Update test cases with netem duplicate

[Cong Wang]

Now netem does no longer trigger reentrant behaviour of its upper
qdiscs, the whole architecture becomes more solid and less error prone.

Keep these test cases since one of them still sucessfully caught a bug
in QFQ qdisc, but update them to the new netem enqueue behavior.

Signed-off-by: Cong Wang <xiyou.wangcong@gmail.com>
---
 .../tc-testing/tc-tests/infra/qdiscs.json     | 50 +++++++++----------
 1 file changed, 23 insertions(+), 27 deletions(-)

diff --git a/tools/testing/selftests/tc-testing/tc-tests/infra/qdiscs.json b/tools/testing/selftests/tc-testing/tc-tests/infra/qdiscs.json
index 57e6b5f35070..6cecea3f25ee 100644
--- a/tools/testing/selftests/tc-testing/tc-tests/infra/qdiscs.json
+++ b/tools/testing/selftests/tc-testing/tc-tests/infra/qdiscs.json
@@ -579,7 +579,7 @@
     },
     {
         "id": "90ec",
-        "name": "Test DRR's enqueue reentrant behaviour with netem",
+        "name": "Test DRR with NETEM duplication",
         "category": [
             "qdisc",
             "drr"
@@ -597,11 +597,11 @@
         ],
         "cmdUnderTest": "ping -c 1 -I $DUMMY 10.10.10.1 > /dev/null || true",
         "expExitCode": "0",
-        "verifyCmd": "$TC -j -s qdisc ls dev $DUMMY handle 1:0",
+        "verifyCmd": "$TC -j -s qdisc ls dev $DUMMY handle 2:0",
         "matchJSON": [
             {
-                "kind": "drr",
-                "handle": "1:",
+                "kind": "netem",
+                "handle": "2:",
                 "bytes": 196,
                 "packets": 2
             }
@@ -614,7 +614,7 @@
     },
     {
         "id": "1f1f",
-        "name": "Test ETS's enqueue reentrant behaviour with netem",
+        "name": "Test ETS with NETEM duplication",
         "category": [
             "qdisc",
             "ets"
@@ -632,15 +632,13 @@
         ],
         "cmdUnderTest": "ping -c 1 -I $DUMMY 10.10.10.1 > /dev/null || true",
         "expExitCode": "0",
-        "verifyCmd": "$TC -j -s class show dev $DUMMY",
+        "verifyCmd": "$TC -j -s qdisc show dev $DUMMY handle 2:0",
         "matchJSON": [
             {
-                "class": "ets",
-                "handle": "1:1",
-                "stats": {
-                    "bytes": 196,
-                    "packets": 2
-                }
+                "kind": "netem",
+                "handle": "2:",
+                "bytes": 196,
+                "packets": 2
             }
         ],
         "matchCount": "1",
@@ -651,7 +649,7 @@
     },
     {
         "id": "5e6d",
-        "name": "Test QFQ's enqueue reentrant behaviour with netem",
+        "name": "Test QFQ with NETEM duplication",
         "category": [
             "qdisc",
             "qfq"
@@ -669,11 +667,11 @@
         ],
         "cmdUnderTest": "ping -c 1 -I $DUMMY 10.10.10.1 > /dev/null || true",
         "expExitCode": "0",
-        "verifyCmd": "$TC -j -s qdisc ls dev $DUMMY handle 1:0",
+        "verifyCmd": "$TC -j -s qdisc ls dev $DUMMY handle 2:0",
         "matchJSON": [
             {
-                "kind": "qfq",
-                "handle": "1:",
+                "kind": "netem",
+                "handle": "2:",
                 "bytes": 196,
                 "packets": 2
             }
@@ -686,7 +684,7 @@
     },
     {
         "id": "bf1d",
-        "name": "Test HFSC's enqueue reentrant behaviour with netem",
+        "name": "Test HFSC with NETEM duplication",
         "category": [
             "qdisc",
             "hfsc"
@@ -710,13 +708,11 @@
         ],
         "cmdUnderTest": "ping -c 1 10.10.10.2 -I$DUMMY > /dev/null || true",
         "expExitCode": "0",
-        "verifyCmd": "$TC -j -s qdisc ls dev $DUMMY handle 1:0",
+        "verifyCmd": "$TC -j -s qdisc ls dev $DUMMY handle 3:0",
         "matchJSON": [
             {
-                "kind": "hfsc",
-                "handle": "1:",
-                "bytes": 392,
-                "packets": 4
+                "kind": "netem",
+                "handle": "3:"
             }
         ],
         "matchCount": "1",
@@ -727,7 +723,7 @@
     },
     {
         "id": "7c3b",
-        "name": "Test nested DRR's enqueue reentrant behaviour with netem",
+        "name": "Test nested DRR with NETEM duplication",
         "category": [
             "qdisc",
             "drr"
@@ -748,11 +744,11 @@
         ],
         "cmdUnderTest": "ping -c 1 -I $DUMMY 10.10.10.1 > /dev/null || true",
         "expExitCode": "0",
-        "verifyCmd": "$TC -j -s qdisc ls dev $DUMMY handle 1:0",
+        "verifyCmd": "$TC -j -s qdisc ls dev $DUMMY handle 3:0",
         "matchJSON": [
             {
-                "kind": "drr",
-                "handle": "1:",
+                "kind": "netem",
+                "handle": "3:",
                 "bytes": 196,
                 "packets": 2
             }
@@ -827,7 +823,7 @@
     },
     {
         "id": "309e",
-        "name": "Test HFSC eltree double add with reentrant enqueue behaviour on netem",
+        "name": "Test complex HFSC with NETEM duplication",
         "category": [
             "qdisc",
             "hfsc"
-- 
2.34.1

[Patch net v8 9/9] selftests/tc-testing: Add a test case for HTB with netem

[Cong Wang]

This test is requested by Stephen and Jakub during review,
it is for validating backlog.

Cc: Jakub Kicinski <kuba@kernel.org>
Cc: Stephen Hemminger <stephen@networkplumber.org>
Signed-off-by: Cong Wang <xiyou.wangcong@gmail.com>
---
 .../tc-testing/tc-tests/infra/qdiscs.json     | 33 +++++++++++++++++++
 1 file changed, 33 insertions(+)

diff --git a/tools/testing/selftests/tc-testing/tc-tests/infra/qdiscs.json b/tools/testing/selftests/tc-testing/tc-tests/infra/qdiscs.json
index 6cecea3f25ee..2c2ffb146a64 100644
--- a/tools/testing/selftests/tc-testing/tc-tests/infra/qdiscs.json
+++ b/tools/testing/selftests/tc-testing/tc-tests/infra/qdiscs.json
@@ -988,6 +988,39 @@
             "$TC qdisc del dev $DUMMY root handle 1: prio"
         ]
     },
+    {
+        "id": "3c5d",
+        "name": "Test HTB with NETEM duplication Backlog Accounting",
+        "category": [
+            "qdisc",
+            "htb",
+            "netem"
+        ],
+        "plugins": {
+            "requires": [
+                "nsPlugin"
+            ]
+        },
+        "setup": [
+            "$IP link set dev $DUMMY up || true",
+            "$IP addr add 10.10.11.10/24 dev $DUMMY || true",
+            "$TC qdisc add dev $DUMMY root handle 1: htb default 1",
+            "$TC class add dev $DUMMY parent 1: classid 1:1 htb rate 100mbit",
+            "$TC qdisc add dev $DUMMY parent 1:1 handle 10: netem limit 4 duplicate 100%",
+            [
+                "ping -c1 -W0.01 -I $DUMMY 10.10.11.11",
+                1
+            ]
+        ],
+        "cmdUnderTest": "$TC qdisc del dev $DUMMY handle 10: parent 1:1",
+        "expExitCode": "0",
+        "verifyCmd": "$TC -s qdisc show dev $DUMMY",
+        "matchPattern": "backlog 0b 0p",
+        "matchCount": "1",
+        "teardown": [
+            "$TC qdisc del dev $DUMMY root handle 1: htb"
+        ]
+    },
     {
         "id": "4989",
         "name": "Try to add an fq child to an ingress qdisc",
-- 
2.34.1

Re: [Patch net v8 0/9] netem: Fix skb duplication logic and prevent infinite loops

[Jamal Hadi Salim]

On Sun, Jan 18, 2026 at 1:16 AM Cong Wang <xiyou.wangcong@gmail.com> wrote:
>
> This patchset fixes the infinite loops due to duplication in netem, the
> real root cause of this problem is enqueuing to the root qdisc, which is
> now changed to enqueuing to the same qdisc. This is more reasonable,
> more intuitive from users' perspective, less error-prone and more elegant
> from kernel developers' perspective.
>
> Please see more details in patch 4/9 which contains two pages of detailed
> explanation including why it is safe and better.
>
> This reverts the offending commits from William which clearly broke
> mq+netem use cases, as reported by two users.
>
> All the TC test cases pass with this patchset.
>

These patches should not be considered for any review because they are
not following the rules that are set for the community. The rules,
which are well documented, state that you must cc all stakeholders.
When someone does this _on purpose_ such as Cong, some accountability
needs to be imposed. I would say totally ignoring these patches is one
option. Otherwise anyone can just throw a tantrum and decide those
rules dont apply to them. Either that or we modify the rules to state
it is ok to do this..

cheers,
jamal

> ---
> v8: Fixed test 7c3b
>
> v7: Fixed a typo in subject
>     Fixed a missing qdisc_tree_reduce_backlog()
>     Added a new selftest for backlog validation
>
> v6: Dropped the init_user_ns check patch
>     Reordered the qfq patch
>     Rebased to the latest -net branch
>
> v5: Reverted the offending commits
>     Added a init_user_ns check (4/9)
>     Rebased to the latest -net branch
>
> v4: Added a fix for qfq qdisc (2/6)
>     Updated 1/6 patch description
>     Added a patch to update the enqueue reentrant behaviour tests
>
> v3: Fixed the root cause of enqueuing to root
>     Switched back to netem_skb_cb safely
>     Added two more test cases
>
> v2: Fixed a typo
>     Improved tdc selftest to check sent bytes
>
> Cong Wang (9):
>   net_sched: Check the return value of qfq_choose_next_agg()
>   Revert "net/sched: Restrict conditions for adding duplicating netems
>     to qdisc tree"
>   Revert "selftests/tc-testing: Add tests for restrictions on netem
>     duplication"
>   net_sched: Implement the right netem duplication behavior
>   selftests/tc-testing: Add a nested netem duplicate test
>   selftests/tc-testing: Add a test case for prio with netem duplicate
>   selftests/tc-testing: Add a test case for mq with netem duplicate
>   selftests/tc-testing: Update test cases with netem duplicate
>   selftests/tc-testing: Add a test case for HTB with netem
>
>  net/sched/sch_netem.c                         |  67 +++-----
>  net/sched/sch_qfq.c                           |   2 +
>  .../tc-testing/tc-tests/infra/qdiscs.json     | 144 ++++++++++++++----
>  .../tc-testing/tc-tests/qdiscs/netem.json     |  90 +++--------
>  4 files changed, 153 insertions(+), 150 deletions(-)
>
> --
> 2.34.1
>
>

Re: [Patch net v8 0/9] netem: Fix skb duplication logic and prevent infinite loops

[Simon Horman]

On Sun, Jan 18, 2026 at 10:07:37AM -0500, Jamal Hadi Salim wrote:
> On Sun, Jan 18, 2026 at 1:16 AM Cong Wang <xiyou.wangcong@gmail.com> wrote:
> >
> > This patchset fixes the infinite loops due to duplication in netem, the
> > real root cause of this problem is enqueuing to the root qdisc, which is
> > now changed to enqueuing to the same qdisc. This is more reasonable,
> > more intuitive from users' perspective, less error-prone and more elegant
> > from kernel developers' perspective.
> >
> > Please see more details in patch 4/9 which contains two pages of detailed
> > explanation including why it is safe and better.
> >
> > This reverts the offending commits from William which clearly broke
> > mq+netem use cases, as reported by two users.
> >
> > All the TC test cases pass with this patchset.
> >
> 
> These patches should not be considered for any review because they are
> not following the rules that are set for the community. The rules,
> which are well documented, state that you must cc all stakeholders.
> When someone does this _on purpose_ such as Cong, some accountability
> needs to be imposed. I would say totally ignoring these patches is one
> option. Otherwise anyone can just throw a tantrum and decide those
> rules dont apply to them. Either that or we modify the rules to state
> it is ok to do this..

I'd prefer if we applied Hanlon's razor here and attribute this to
carelessness rather than mailce.

Please let's find a way to move this forward.

Re: [Patch net v8 0/9] netem: Fix skb duplication logic and prevent infinite loops

[Jamal Hadi Salim]

On Wed, Jan 21, 2026 at 12:08 PM Simon Horman <horms@kernel.org> wrote:
>
> On Sun, Jan 18, 2026 at 10:07:37AM -0500, Jamal Hadi Salim wrote:
> > On Sun, Jan 18, 2026 at 1:16 AM Cong Wang <xiyou.wangcong@gmail.com> wrote:
> > >
> > > This patchset fixes the infinite loops due to duplication in netem, the
> > > real root cause of this problem is enqueuing to the root qdisc, which is
> > > now changed to enqueuing to the same qdisc. This is more reasonable,
> > > more intuitive from users' perspective, less error-prone and more elegant
> > > from kernel developers' perspective.
> > >
> > > Please see more details in patch 4/9 which contains two pages of detailed
> > > explanation including why it is safe and better.
> > >
> > > This reverts the offending commits from William which clearly broke
> > > mq+netem use cases, as reported by two users.
> > >
> > > All the TC test cases pass with this patchset.
> > >
> >
> > These patches should not be considered for any review because they are
> > not following the rules that are set for the community. The rules,
> > which are well documented, state that you must cc all stakeholders.
> > When someone does this _on purpose_ such as Cong, some accountability
> > needs to be imposed. I would say totally ignoring these patches is one
> > option. Otherwise anyone can just throw a tantrum and decide those
> > rules dont apply to them. Either that or we modify the rules to state
> > it is ok to do this..
>
> I'd prefer if we applied Hanlon's razor here and attribute this to
> carelessness rather than mailce.
>

Making a mistake is one thing. I made a mistake by not Ccing Stephen
the other day. When I realized i hadnt, I immediately followed up,
sent him an apology, and pointed him to the thread.
This isn't the first time Cong has done this. So i dont believe this
was carelessness at all. See this thread:
https://lore.kernel.org/netdev/CAM_iQpUGvHLB2cZmdd=0a4KAW2+RALNH=_jZruE1sju2gBGTeA@mail.gmail.com/#t
He posted the patchset a few hours after i responded to him.

> Please let's find a way to move this forward

I am not going to review this patch as it stands right now. If Cong
wants to explain himself, I may reconsider (which does not imply i
will accept this patch).
If you apply this patch as is - then it is only fair we fix the rules
to show this as an example of where such rules can be ignored.

cheers,
jamal

Re: [Patch net v8 0/9] netem: Fix skb duplication logic and prevent infinite loops

[Jakub Kicinski]

On Sun, 18 Jan 2026 10:07:37 -0500 Jamal Hadi Salim wrote:
> On Sun, Jan 18, 2026 at 1:16 AM Cong Wang <xiyou.wangcong@gmail.com> wrote:
> > This patchset fixes the infinite loops due to duplication in netem, the
> > real root cause of this problem is enqueuing to the root qdisc, which is
> > now changed to enqueuing to the same qdisc. This is more reasonable,
> > more intuitive from users' perspective, less error-prone and more elegant
> > from kernel developers' perspective.
> >
> > Please see more details in patch 4/9 which contains two pages of detailed
> > explanation including why it is safe and better.
> >
> > This reverts the offending commits from William which clearly broke
> > mq+netem use cases, as reported by two users.
> >
> > All the TC test cases pass with this patchset.
> >  
> 
> These patches should not be considered for any review because they are
> not following the rules that are set for the community. The rules,
> which are well documented, state that you must cc all stakeholders.
> When someone does this _on purpose_ such as Cong, some accountability
> needs to be imposed. I would say totally ignoring these patches is one
> option. Otherwise anyone can just throw a tantrum and decide those
> rules dont apply to them. Either that or we modify the rules to state
> it is ok to do this..

We (the netdev maintainers) had a chat off list and concluded that it'd
be great if we (the two of you and one maintainer) met over VC. Either
technical solution is fine, but we want to make sure there's no ongoing
animosity. No agenda for the meeting, just want to make sure you can
have a friendly chat. We can talk about the weather, AI code reviews or
everyone's favorite animals. At the end of the meeting we can either
decide or flip a coin.

Re: [Patch net v8 0/9] netem: Fix skb duplication logic and prevent infinite loops

[Cong Wang]

On Tue, Jan 27, 2026 at 09:15:56AM -0800, Jakub Kicinski wrote:
> On Sun, 18 Jan 2026 10:07:37 -0500 Jamal Hadi Salim wrote:
> > On Sun, Jan 18, 2026 at 1:16 AM Cong Wang <xiyou.wangcong@gmail.com> wrote:
> > > This patchset fixes the infinite loops due to duplication in netem, the
> > > real root cause of this problem is enqueuing to the root qdisc, which is
> > > now changed to enqueuing to the same qdisc. This is more reasonable,
> > > more intuitive from users' perspective, less error-prone and more elegant
> > > from kernel developers' perspective.
> > >
> > > Please see more details in patch 4/9 which contains two pages of detailed
> > > explanation including why it is safe and better.
> > >
> > > This reverts the offending commits from William which clearly broke
> > > mq+netem use cases, as reported by two users.
> > >
> > > All the TC test cases pass with this patchset.
> > >  
> > 
> > These patches should not be considered for any review because they are
> > not following the rules that are set for the community. The rules,
> > which are well documented, state that you must cc all stakeholders.
> > When someone does this _on purpose_ such as Cong, some accountability
> > needs to be imposed. I would say totally ignoring these patches is one
> > option. Otherwise anyone can just throw a tantrum and decide those
> > rules dont apply to them. Either that or we modify the rules to state
> > it is ok to do this..
> 
> We (the netdev maintainers) had a chat off list and concluded that it'd
> be great if we (the two of you and one maintainer) met over VC. Either
> technical solution is fine, but we want to make sure there's no ongoing
> animosity. No agenda for the meeting, just want to make sure you can
> have a friendly chat. We can talk about the weather, AI code reviews or
> everyone's favorite animals. At the end of the meeting we can either
> decide or flip a coin.

I offered people involved here for a video chat, *check note*, in

26 Nov 2025:
https://lore.kernel.org/netdev/aSe%2FIfNSZBTTAfTA@pop-os.localdomain/
8 Jul 2025:
https://lore.kernel.org/netdev/aG2OUoDD2m5MqdSz@pop-os.localdomain/

No one responded, obviously.

Jakub, please let me know how to talk to people who kept rejecting to
talk. It is just logically impossible to me, and I have tried my best.

I wish you best luck for doing so.

Thanks!

Re: [Patch net v8 0/9] netem: Fix skb duplication logic and prevent infinite loops

[Jakub Kicinski]

On Fri, 30 Jan 2026 12:53:42 -0800 Cong Wang wrote:
> On Tue, Jan 27, 2026 at 09:15:56AM -0800, Jakub Kicinski wrote:
> > On Sun, 18 Jan 2026 10:07:37 -0500 Jamal Hadi Salim wrote:  
> > > On Sun, Jan 18, 2026 at 1:16 AM Cong Wang <xiyou.wangcong@gmail.com> wrote:  
> > We (the netdev maintainers) had a chat off list and concluded that it'd
> > be great if we (the two of you and one maintainer) met over VC. Either
> > technical solution is fine, but we want to make sure there's no ongoing
> > animosity. No agenda for the meeting, just want to make sure you can
> > have a friendly chat. We can talk about the weather, AI code reviews or
> > everyone's favorite animals. At the end of the meeting we can either
> > decide or flip a coin.  
> 
> I offered people involved here for a video chat, *check note*, in
> 
> 26 Nov 2025:
> https://lore.kernel.org/netdev/aSe%2FIfNSZBTTAfTA@pop-os.localdomain/
> 8 Jul 2025:
> https://lore.kernel.org/netdev/aG2OUoDD2m5MqdSz@pop-os.localdomain/
> 
> No one responded, obviously.

So you should be happy that the call will finally happen, right?

> Jakub, please let me know how to talk to people who kept rejecting to
> talk. It is just logically impossible to me, and I have tried my best.
> 
> I wish you best luck for doing so.

I sent you a link for scheduling the call off list.

Which part of this process of scheduling a call would you like 
me to explain further?

Re: [Patch net v8 0/9] netem: Fix skb duplication logic and prevent infinite loops

[Cong Wang]

On Fri, Jan 30, 2026 at 01:18:12PM -0800, Jakub Kicinski wrote:
> On Fri, 30 Jan 2026 12:53:42 -0800 Cong Wang wrote:
> > On Tue, Jan 27, 2026 at 09:15:56AM -0800, Jakub Kicinski wrote:
> > > On Sun, 18 Jan 2026 10:07:37 -0500 Jamal Hadi Salim wrote:  
> > > > On Sun, Jan 18, 2026 at 1:16 AM Cong Wang <xiyou.wangcong@gmail.com> wrote:  
> > > We (the netdev maintainers) had a chat off list and concluded that it'd
> > > be great if we (the two of you and one maintainer) met over VC. Either
> > > technical solution is fine, but we want to make sure there's no ongoing
> > > animosity. No agenda for the meeting, just want to make sure you can
> > > have a friendly chat. We can talk about the weather, AI code reviews or
> > > everyone's favorite animals. At the end of the meeting we can either
> > > decide or flip a coin.  
> > 
> > I offered people involved here for a video chat, *check note*, in
> > 
> > 26 Nov 2025:
> > https://lore.kernel.org/netdev/aSe%2FIfNSZBTTAfTA@pop-os.localdomain/
> > 8 Jul 2025:
> > https://lore.kernel.org/netdev/aG2OUoDD2m5MqdSz@pop-os.localdomain/
> > 
> > No one responded, obviously.
> 
> So you should be happy that the call will finally happen, right?
> 
> > Jakub, please let me know how to talk to people who kept rejecting to
> > talk. It is just logically impossible to me, and I have tried my best.
> > 
> > I wish you best luck for doing so.
> 
> I sent you a link for scheduling the call off list.
> 
> Which part of this process of scheduling a call would you like 
> me to explain further?

I have a better solution which wastes no time of anyone:
https://lore.kernel.org/netdev/20260130212021.46610-1-xiyou.wangcong@gmail.com/T/#u

Please apply that patch, all the problems would be gone. (Nothing
emotional here, it was after a very careful consideration.)

Thanks your effort, Jakub!

Regards,
Cong