From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from out-180.mta0.migadu.com (out-180.mta0.migadu.com [91.218.175.180]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9926947B42C for ; Tue, 14 Jul 2026 14:21:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=91.218.175.180 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784038870; cv=none; b=BtdBIMf52CKtcHzBji6IlmxV073W4vI69zWtpxKmlleT13KwP/vPJm1uq1eb8KLE8KiKsdzWrCCqTiE54W65MxPk8W5DT9GuGdafXCgHSNEz0pIx/wZsZmnH4JAiKtyM/P2d2LPyQ+cIKnIhsjQaNNi5LG+xkufy8VWod/IkxXk= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784038870; c=relaxed/simple; bh=c06LSu8sD77hVlRHIN28ESNPvRdrhwaNbKAlnCjPqaY=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version; b=Yj6xeSc/1gVwKd6t6Ujwl5czA67XyqwuW2VUUmq30H0YIYdLjj5vdAjHhL75JehL4HEiEGbB3ozq2CTT+IEKA9Ns5ZpXPj83liD+HVOj+B8ZowWzDoZLW6S78ONbtO0UpwDm9HPjNaj1cINqQVLYfdHq8SdmKSHkU1XgboZ0yEY= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=q3TjjIjk; arc=none smtp.client-ip=91.218.175.180 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="q3TjjIjk" X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1784038865; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding; bh=xuOtwFzrYcc0RlbJuVePSX2/UoYrZ3O0CjZBYi5MRFw=; b=q3TjjIjkWR8b5v2T6NUZgyU6pUsfag5YkgefzhrdlJpF0T39Ouu96U11d1MNkO+3ez4Y1X eGQk3or6Bq57cWjXRJKFrVJ8Wt6AQcLoQy5okTKbSxWfmju1CmAPclXyA1BrVM7vWiNsKP uL2l/b5wozZthikejj7Fy5DQhdrsWmQ= From: Usama Arif To: bsegall@google.com, dietmar.eggemann@arm.com, hannes@cmpxchg.org, juri.lelli@redhat.com, kprateek.nayak@amd.com, linux-kernel@vger.kernel.org, mgorman@suse.de, mingo@redhat.com, peterz@infradead.org, rostedt@goodmis.org, surenb@google.com, vincent.guittot@linaro.org, vschneid@redhat.com, shakeel.butt@linux.dev, riel@surriel.com, kernel-team@meta.com Cc: Usama Arif Subject: [PATCH] sched/psi: use for_each_set_bit() in psi_group_change() task-count walk Date: Tue, 14 Jul 2026 07:20:57 -0700 Message-ID: <20260714142057.181135-1-usama.arif@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Migadu-Flow: FLOW_OUT psi_group_change() walks the @clear and @set bitmasks to decrement/increment groupc->tasks[t]. Both masks are at most NR_PSI_TASK_COUNTS (=4) wide, dense at [0, 4), and typically sparse. Today's form visits every position up to the highest set bit: for (t = 0, m = clear; m; m &= ~(1 << t), t++) { if (!(m & (1 << t))) continue; ... } so a mask with only bit 3 set still spins four times; the same open-coded shape repeats for @set. The code is also unnecessarily hard to read. Switch both walks to for_each_set_bit() which is easier to read and also more efficient. As NR_PSI_TASK_COUNTS is a compile-time constant <= BITS_PER_LONG, find_next_bit() folds into its small_const_nbits() fast path (single load + GENMASK + __ffs), lowering to a bit-scan where one exists (x86 TZCNT/BSF, arm64 RBIT+CLZ). psi_group_change() runs from psi_task_switch() and psi_task_change() once per ancestor psi_group per event, so the saved iterations multiply out on any hot scheduler workload. No functional change intended. Signed-off-by: Usama Arif --- kernel/sched/psi.c | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/kernel/sched/psi.c b/kernel/sched/psi.c index d9c9d9480a45..f5ae1ceb21a7 100644 --- a/kernel/sched/psi.c +++ b/kernel/sched/psi.c @@ -798,7 +798,8 @@ static void psi_group_change(struct psi_group *group, int cpu, u64 now, bool wake_clock) { struct psi_group_cpu *groupc; - unsigned int t, m; + unsigned long clear_bits, set_bits; + unsigned int t; u32 state_mask; lockdep_assert_rq_held(cpu_rq(cpu)); @@ -824,9 +825,8 @@ static void psi_group_change(struct psi_group *group, int cpu, * The rest of the state mask is calculated based on the task * counts. Update those first, then construct the mask. */ - for (t = 0, m = clear; m; m &= ~(1 << t), t++) { - if (!(m & (1 << t))) - continue; + clear_bits = clear; + for_each_set_bit(t, &clear_bits, NR_PSI_TASK_COUNTS) { if (groupc->tasks[t]) { groupc->tasks[t]--; } else if (!psi_bug) { @@ -838,9 +838,9 @@ static void psi_group_change(struct psi_group *group, int cpu, } } - for (t = 0; set; set &= ~(1 << t), t++) - if (set & (1 << t)) - groupc->tasks[t]++; + set_bits = set; + for_each_set_bit(t, &set_bits, NR_PSI_TASK_COUNTS) + groupc->tasks[t]++; if (!group->enabled) { /* -- 2.53.0-Meta