From: Usama Arif <usama.arif@linux.dev>
To: bsegall@google.com, dietmar.eggemann@arm.com, hannes@cmpxchg.org,
juri.lelli@redhat.com, kprateek.nayak@amd.com,
linux-kernel@vger.kernel.org, mgorman@suse.de, mingo@redhat.com,
peterz@infradead.org, rostedt@goodmis.org, surenb@google.com,
vincent.guittot@linaro.org, vschneid@redhat.com,
shakeel.butt@linux.dev, riel@surriel.com, kernel-team@meta.com
Cc: Usama Arif <usama.arif@linux.dev>
Subject: [PATCH] sched/psi: use for_each_set_bit() in psi_group_change() task-count walk
Date: Tue, 14 Jul 2026 07:20:57 -0700 [thread overview]
Message-ID: <20260714142057.181135-1-usama.arif@linux.dev> (raw)
psi_group_change() walks the @clear and @set bitmasks to
decrement/increment groupc->tasks[t]. Both masks are at most
NR_PSI_TASK_COUNTS (=4) wide, dense at [0, 4), and typically
sparse. Today's form visits every position up to the highest set
bit:
for (t = 0, m = clear; m; m &= ~(1 << t), t++) {
if (!(m & (1 << t)))
continue;
...
}
so a mask with only bit 3 set still spins four times; the same
open-coded shape repeats for @set. The code is also unnecessarily
hard to read.
Switch both walks to for_each_set_bit() which is easier to read
and also more efficient. As NR_PSI_TASK_COUNTS is a compile-time
constant <= BITS_PER_LONG, find_next_bit() folds into its
small_const_nbits() fast path (single load + GENMASK + __ffs), lowering
to a bit-scan where one exists (x86 TZCNT/BSF, arm64 RBIT+CLZ).
psi_group_change() runs from psi_task_switch() and psi_task_change()
once per ancestor psi_group per event, so the saved iterations
multiply out on any hot scheduler workload.
No functional change intended.
Signed-off-by: Usama Arif <usama.arif@linux.dev>
---
kernel/sched/psi.c | 14 +++++++-------
1 file changed, 7 insertions(+), 7 deletions(-)
diff --git a/kernel/sched/psi.c b/kernel/sched/psi.c
index d9c9d9480a45..f5ae1ceb21a7 100644
--- a/kernel/sched/psi.c
+++ b/kernel/sched/psi.c
@@ -798,7 +798,8 @@ static void psi_group_change(struct psi_group *group, int cpu,
u64 now, bool wake_clock)
{
struct psi_group_cpu *groupc;
- unsigned int t, m;
+ unsigned long clear_bits, set_bits;
+ unsigned int t;
u32 state_mask;
lockdep_assert_rq_held(cpu_rq(cpu));
@@ -824,9 +825,8 @@ static void psi_group_change(struct psi_group *group, int cpu,
* The rest of the state mask is calculated based on the task
* counts. Update those first, then construct the mask.
*/
- for (t = 0, m = clear; m; m &= ~(1 << t), t++) {
- if (!(m & (1 << t)))
- continue;
+ clear_bits = clear;
+ for_each_set_bit(t, &clear_bits, NR_PSI_TASK_COUNTS) {
if (groupc->tasks[t]) {
groupc->tasks[t]--;
} else if (!psi_bug) {
@@ -838,9 +838,9 @@ static void psi_group_change(struct psi_group *group, int cpu,
}
}
- for (t = 0; set; set &= ~(1 << t), t++)
- if (set & (1 << t))
- groupc->tasks[t]++;
+ set_bits = set;
+ for_each_set_bit(t, &set_bits, NR_PSI_TASK_COUNTS)
+ groupc->tasks[t]++;
if (!group->enabled) {
/*
--
2.53.0-Meta
next reply other threads:[~2026-07-14 14:21 UTC|newest]
Thread overview: 5+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-07-14 14:20 Usama Arif [this message]
2026-07-15 3:49 ` [PATCH] sched/psi: use for_each_set_bit() in psi_group_change() task-count walk K Prateek Nayak
2026-07-15 10:47 ` Usama Arif
2026-07-15 13:54 ` Johannes Weiner
2026-07-15 17:03 ` Usama Arif
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20260714142057.181135-1-usama.arif@linux.dev \
--to=usama.arif@linux.dev \
--cc=bsegall@google.com \
--cc=dietmar.eggemann@arm.com \
--cc=hannes@cmpxchg.org \
--cc=juri.lelli@redhat.com \
--cc=kernel-team@meta.com \
--cc=kprateek.nayak@amd.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mgorman@suse.de \
--cc=mingo@redhat.com \
--cc=peterz@infradead.org \
--cc=riel@surriel.com \
--cc=rostedt@goodmis.org \
--cc=shakeel.butt@linux.dev \
--cc=surenb@google.com \
--cc=vincent.guittot@linaro.org \
--cc=vschneid@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox