From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9FA133A9870; Sat, 17 Oct 2026 16:35:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1792254907; cv=none; b=dl/VufxoC/4lmNvaZMn6r1XRMQfilvucfeSwyjCB5hkO3Kmzc/PRhHusJ7rO9s1LkI6++RE5rmBmfshDCHNZTGEEOgbVskihE2hIFOa1uZ3RRq3f67SsWJ/rSkY9IZHUJNtfsB6euuLUgSa2lzD+0kk/Kd0CKqSm0+6Oetr8jwQ= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1792254907; c=relaxed/simple; bh=Vv9A/CQNnV9hA1eWrQBi7uY8VQULBbBA1vCm2+rTJH0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=VuapjYCSVlprYsU/RFLHPzGJq5pjurLDgrJjMtgeb/8BTxdivCzoQRi7bXu5DrGgcK+4mOZnwnMy6WL3Cz1QLTRXOXmHxDrfLOhne/3Ide6TJf1Fug3MRx1AKDSyQ7A450MWzcXWReRPNOBXb1rL+1bTMKT9fNUGYh1O71sMF8o= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linuxfoundation.org header.i=@linuxfoundation.org header.b=CiNTIWZo; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linuxfoundation.org header.i=@linuxfoundation.org header.b="CiNTIWZo" Received: by smtp.kernel.org (Postfix) with ESMTPSA id E7CC11F000C4; Sat, 17 Oct 2026 16:35:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linuxfoundation.org; s=korg; t=1792254906; bh=0snXyyyf3zTz5EW68BruLef1loaafmLUbYwG1yHt8fI=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=CiNTIWZo6xexQPye0QsJ14KHVoR3B/wfEc/JXWul55xKcnaeIoL5SW3uaNiwzXuZl 6tZiD8ZpfrtAxXnAuTJc5D1vAHtik2mMVsZg32xxr3OyUpA7ioZnGb5zEztlMJKQRR qd6B3q4ImRUG+Shum/j+7q2JN86Y6CygHrbaSENA= From: Greg Kroah-Hartman To: stable@vger.kernel.org Cc: Greg Kroah-Hartman , patches@lists.linux.dev, Eric Dumazet , Jamal Hadi Salim , Jakub Kicinski , Sasha Levin Subject: [PATCH 6.12 599/869] net/sched: add qstats_cpu_drop_inc() helper Date: Sat, 17 Oct 2026 17:57:20 +0200 Message-ID: <20261017160052.400092685@linuxfoundation.org> X-Mailer: git-send-email 2.56.0 In-Reply-To: <20261017160036.979869304@linuxfoundation.org> References: <20261017160036.979869304@linuxfoundation.org> User-Agent: quilt/0.69 X-stable: review X-Patchwork-Hint: ignore Precedence: bulk X-Mailing-List: patches@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit 6.12-stable review patch. If anyone has any objections, please let me know. ------------------ From: Eric Dumazet [ Upstream commit 3f3aa77ff1c8b45ec8c9e40212f1a24a93e00df3 ] 1) Using this_cpu_inc() is better than going through this_cpu_ptr(): - Single instruction on x86. - Store tearing prevention. 2) Change tcf_action_update_stats() to use this_cpu_add(). 3) Add WRITE_ONCE() to __qdisc_qstats_drop() and qstats_drop_inc() in preparation for lockless "tc qdisc show". $ scripts/bloat-o-meter -t vmlinux.old vmlinux.new add/remove: 0/0 grow/shrink: 3/17 up/down: 72/-216 (-144) Function old new delta dualpi2_enqueue_skb 462 511 +49 tcf_ife_act 1061 1077 +16 taprio_enqueue 613 620 +7 codel_qdisc_enqueue 149 143 -6 tcf_vlan_act 684 676 -8 tcf_skbedit_act 626 618 -8 tcf_police_act 725 717 -8 tcf_mpls_act 1297 1289 -8 tcf_gate_act 310 302 -8 tcf_gact_act 222 214 -8 tcf_csum_act 2438 2430 -8 tcf_bpf_act 709 701 -8 tcf_action_update_stats 124 115 -9 pie_qdisc_enqueue 865 856 -9 pfifo_enqueue 116 107 -9 choke_enqueue 2069 2059 -10 plug_enqueue 139 128 -11 bfifo_enqueue 121 110 -11 tcf_nat_act 1501 1489 -12 gred_enqueue 1743 1668 -75 Total: Before=24388609, After=24388465, chg -0.00% Signed-off-by: Eric Dumazet Reviewed-by: Jamal Hadi Salim Link: https://patch.msgid.link/20260501135916.2566766-1-edumazet@google.com Signed-off-by: Jakub Kicinski Stable-dep-of: 5b483f7791b0 ("net/sched: act_ife: Only operate on Ethernet frames") Signed-off-by: Sasha Levin --- include/net/act_api.h | 2 +- include/net/sch_generic.h | 9 +++++++-- net/sched/act_api.c | 2 +- net/sched/act_bpf.c | 2 +- net/sched/act_ife.c | 8 ++++---- net/sched/act_mpls.c | 2 +- net/sched/act_police.c | 2 +- net/sched/act_skbedit.c | 2 +- net/sched/sch_cake.c | 2 +- net/sched/sch_fq_codel.c | 2 +- net/sched/sch_gred.c | 2 +- 11 files changed, 20 insertions(+), 15 deletions(-) diff --git a/include/net/act_api.h b/include/net/act_api.h index 76c31ce80fe52..29b72635abe71 100644 --- a/include/net/act_api.h +++ b/include/net/act_api.h @@ -238,7 +238,7 @@ static inline void tcf_action_update_bstats(struct tc_action *a, static inline void tcf_action_inc_drop_qstats(struct tc_action *a) { if (likely(a->cpu_qstats)) { - qstats_drop_inc(this_cpu_ptr(a->cpu_qstats)); + qstats_cpu_drop_inc(a->cpu_qstats); return; } spin_lock(&a->tcfa_lock); diff --git a/include/net/sch_generic.h b/include/net/sch_generic.h index 016d9cf827069..5f83b4970f792 100644 --- a/include/net/sch_generic.h +++ b/include/net/sch_generic.h @@ -976,12 +976,17 @@ static inline void qdisc_qstats_cpu_requeues_inc(struct Qdisc *sch) static inline void __qdisc_qstats_drop(struct Qdisc *sch, int count) { - sch->qstats.drops += count; + WRITE_ONCE(sch->qstats.drops, sch->qstats.drops + count); } static inline void qstats_drop_inc(struct gnet_stats_queue *qstats) { - qstats->drops++; + WRITE_ONCE(qstats->drops, qstats->drops + 1); +} + +static inline void qstats_cpu_drop_inc(struct gnet_stats_queue __percpu *qstats) +{ + this_cpu_inc(qstats->drops); } static inline void qstats_overlimit_inc(struct gnet_stats_queue *qstats) diff --git a/net/sched/act_api.c b/net/sched/act_api.c index ca851146f6958..94feb3961dcd2 100644 --- a/net/sched/act_api.c +++ b/net/sched/act_api.c @@ -1570,7 +1570,7 @@ void tcf_action_update_stats(struct tc_action *a, u64 bytes, u64 packets, if (a->cpu_bstats) { _bstats_update(this_cpu_ptr(a->cpu_bstats), bytes, packets); - this_cpu_ptr(a->cpu_qstats)->drops += drops; + this_cpu_add(a->cpu_qstats->drops, drops); if (hw) _bstats_update(this_cpu_ptr(a->cpu_bstats_hw), diff --git a/net/sched/act_bpf.c b/net/sched/act_bpf.c index e99b9269ea1a9..06d8f78b73683 100644 --- a/net/sched/act_bpf.c +++ b/net/sched/act_bpf.c @@ -76,7 +76,7 @@ TC_INDIRECT_SCOPE int tcf_bpf_act(struct sk_buff *skb, break; case TC_ACT_SHOT: action = filter_res; - qstats_drop_inc(this_cpu_ptr(prog->common.cpu_qstats)); + qstats_cpu_drop_inc(prog->common.cpu_qstats); break; case TC_ACT_UNSPEC: action = prog->tcf_action; diff --git a/net/sched/act_ife.c b/net/sched/act_ife.c index a90d7959fc432..3333714b68650 100644 --- a/net/sched/act_ife.c +++ b/net/sched/act_ife.c @@ -727,7 +727,7 @@ static int tcf_ife_decode(struct sk_buff *skb, const struct tc_action *a, tlv_data = ife_decode(skb, &metalen); if (unlikely(!tlv_data)) { - qstats_drop_inc(this_cpu_ptr(ife->common.cpu_qstats)); + qstats_cpu_drop_inc(ife->common.cpu_qstats); return TC_ACT_SHOT; } @@ -740,7 +740,7 @@ static int tcf_ife_decode(struct sk_buff *skb, const struct tc_action *a, curr_data = ife_tlv_meta_decode(tlv_data, ifehdr_end, &mtype, &dlen, NULL); if (!curr_data) { - qstats_drop_inc(this_cpu_ptr(ife->common.cpu_qstats)); + qstats_cpu_drop_inc(ife->common.cpu_qstats); return TC_ACT_SHOT; } @@ -755,7 +755,7 @@ static int tcf_ife_decode(struct sk_buff *skb, const struct tc_action *a, } if (WARN_ON(tlv_data != ifehdr_end)) { - qstats_drop_inc(this_cpu_ptr(ife->common.cpu_qstats)); + qstats_cpu_drop_inc(ife->common.cpu_qstats); return TC_ACT_SHOT; } @@ -821,7 +821,7 @@ static int tcf_ife_encode(struct sk_buff *skb, const struct tc_action *a, * so lets be conservative.. */ if ((action == TC_ACT_SHOT) || exceed_mtu) { drop: - qstats_drop_inc(this_cpu_ptr(ife->common.cpu_qstats)); + qstats_cpu_drop_inc(ife->common.cpu_qstats); return TC_ACT_SHOT; } diff --git a/net/sched/act_mpls.c b/net/sched/act_mpls.c index 44a37a71ae923..24d9cb7c1c4db 100644 --- a/net/sched/act_mpls.c +++ b/net/sched/act_mpls.c @@ -125,7 +125,7 @@ TC_INDIRECT_SCOPE int tcf_mpls_act(struct sk_buff *skb, return ret; drop: - qstats_drop_inc(this_cpu_ptr(m->common.cpu_qstats)); + qstats_cpu_drop_inc(m->common.cpu_qstats); return TC_ACT_SHOT; } diff --git a/net/sched/act_police.c b/net/sched/act_police.c index 1ad8a79486902..098eddcd36b6c 100644 --- a/net/sched/act_police.c +++ b/net/sched/act_police.c @@ -317,7 +317,7 @@ TC_INDIRECT_SCOPE int tcf_police_act(struct sk_buff *skb, qstats_overlimit_inc(this_cpu_ptr(police->common.cpu_qstats)); inc_drops: if (ret == TC_ACT_SHOT) - qstats_drop_inc(this_cpu_ptr(police->common.cpu_qstats)); + qstats_cpu_drop_inc(police->common.cpu_qstats); end: return ret; } diff --git a/net/sched/act_skbedit.c b/net/sched/act_skbedit.c index 4cddf2674373f..c52f25e0c0a09 100644 --- a/net/sched/act_skbedit.c +++ b/net/sched/act_skbedit.c @@ -85,7 +85,7 @@ TC_INDIRECT_SCOPE int tcf_skbedit_act(struct sk_buff *skb, return params->action; err: - qstats_drop_inc(this_cpu_ptr(d->common.cpu_qstats)); + qstats_cpu_drop_inc(d->common.cpu_qstats); return TC_ACT_SHOT; } diff --git a/net/sched/sch_cake.c b/net/sched/sch_cake.c index 7eb3e7485539f..a60aae42398a3 100644 --- a/net/sched/sch_cake.c +++ b/net/sched/sch_cake.c @@ -1824,7 +1824,7 @@ static s32 cake_enqueue(struct sk_buff *skb, struct Qdisc *sch, if (ack) { b->ack_drops++; - sch->qstats.drops++; + qdisc_qstats_drop(sch); ack_pkt_len = qdisc_pkt_len(ack); b->bytes += ack_pkt_len; q->buffer_used += skb->truesize - ack->truesize; diff --git a/net/sched/sch_fq_codel.c b/net/sched/sch_fq_codel.c index 4da7525c3e7cf..d41a55863d165 100644 --- a/net/sched/sch_fq_codel.c +++ b/net/sched/sch_fq_codel.c @@ -175,7 +175,7 @@ static unsigned int fq_codel_drop(struct Qdisc *sch, unsigned int max_packets, flow->cvars.count += i; q->backlogs[idx] -= len; q->memory_usage -= mem; - sch->qstats.drops += i; + __qdisc_qstats_drop(sch, i); sch->qstats.backlog -= len; sch->q.qlen -= i; return idx; diff --git a/net/sched/sch_gred.c b/net/sched/sch_gred.c index 53b348b83c913..26103178a4ce4 100644 --- a/net/sched/sch_gred.c +++ b/net/sched/sch_gred.c @@ -389,7 +389,7 @@ static int gred_offload_dump_stats(struct Qdisc *sch) packets += u64_stats_read(&hw_stats->stats.bstats[i].packets); sch->qstats.qlen += hw_stats->stats.qstats[i].qlen; sch->qstats.backlog += hw_stats->stats.qstats[i].backlog; - sch->qstats.drops += hw_stats->stats.qstats[i].drops; + __qdisc_qstats_drop(sch, hw_stats->stats.qstats[i].drops); sch->qstats.requeues += hw_stats->stats.qstats[i].requeues; sch->qstats.overlimits += hw_stats->stats.qstats[i].overlimits; } -- 2.53.0