From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-yw1-f172.google.com (mail-yw1-f172.google.com [209.85.128.172]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3EF9D30C16E for ; Thu, 18 Jun 2026 03:33:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.172 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1781753582; cv=none; b=n+RqEGuNHM32zYMN/WwDJrza03qnlS2aYhnwC0ZwMaoKUlgEy0exQJNApxSF5xf5jT6tAW7HCaWUzxWEORDejcHelUZe+oMpPz1ewqH1bFtnylOvg+G9hAKJZspkiSLR8bt7BepTwpE7bkVUIdMV/ApPQIBDsmdz/mqREDH0fl8= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1781753582; c=relaxed/simple; bh=Xs7JJOCHvd2Fu7FkBdnAd+86J5heqBcjE9PGE/nXcWg=; h=From:Date:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=M9OomAm2FzuAhd02d5Dot8D+ae6BdU+v2+UXtqzi4nszrfQZiClZnouEnVTjHprW5Uh+aU4Tsn1VwOGnItOeE/jyKumPzyL6F4ZSv1dmwvnX6o1j2M7HZtnr9fTTSFyW0ybP0olOt7UkdERXM8+pTMZ/ajsWfplUwdMlap+C+M0= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=Q/1xS9+H; arc=none smtp.client-ip=209.85.128.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Q/1xS9+H" Received: by mail-yw1-f172.google.com with SMTP id 00721157ae682-7fdb04d774aso5868407b3.2 for ; Wed, 17 Jun 2026 20:33:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1781753580; x=1782358380; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:date:from:from:to:cc:subject:date:message-id:reply-to; bh=8FgvjT8fxZVjd+L2fFPBvwISO/QVqhIVGaRx9qhm03g=; b=Q/1xS9+H3MXVbljVAtL+pmb46/Mtl9PXqzhfAgNNm1aTJyf6HWYzunivg7NSzNdAYZ a6sVAqSa6eETfqaqCmYlaFKECrDgcjbTrLA0URAEWEm5nVJfUxKnaqea3xpRKyaK1RM5 YmMhbyX2r1r3E/XKPGhBpjDBWK/wTYpNhKQFxsnAzpxgm2sR3fTmSj++ekHWemks7k3Q PZddivtFmE+mmjVSj0O0RFIOmxJpHslkCQFErNs+4jt9oHtDd2J1zysygrC5yMwuLUBG IxTca0pTv+4j9or0/CFRBUsnva8r4PFz3mCEDJwkQzrkk7p70LgmFmokN+sjDGzgRJ78 110A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1781753580; x=1782358380; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:date:from:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=8FgvjT8fxZVjd+L2fFPBvwISO/QVqhIVGaRx9qhm03g=; b=fYPGXJXInyVzqtpbo1qK+HR6TXTTB4FHv18Fjtamd8stTOA4Mt7H9gzVVIzWVJ1kwN iJJeO7xftZBkYfbD/5k4jyrYK5tUEjYHf7JqwdRtgoLfwS9FbqOW/hGcZbIOh2EVnE+V RMzFPOi4LBQw8KAEKnBbtATuGhTScJfvLw5ki9B0WS/XvTmHxz+OrTA7k9LI4g82ilEo JLrMnJwyCQlLXhwlvBdlN50jJ/BUfExn898YNfmnY7V/OcCEQJ8orRd0wvOTVbNz6CQB 6UP6gKq7DUlrO2YOPpqkicG/X5qtlKMeb/fj02CzHZlC8nlDMffen9PwSD4qh+mNVpHK itqA== X-Gm-Message-State: AOJu0Ywa1rwVpHH9OzDPqvQl5M7uj+kgfSYooKNhJv8ugS8NptABGHBG o1oj63h3xn23VYfcRIayGHVF4JjhNbxMLGQvmIG8Yx9d/56yByVVXLzJ X-Gm-Gg: AfdE7ckxEvo3FEigA21ZUMh7wDdLZOXpvXnH+Igmzzg4EO7w9csK2QQp8yTlWMTCM3z CjlmvEaFCJK2R9/fnVSrWXDGSJT6MP6hsiGT8egcJ4YvK93nh0kYov04JA2958OC9exTh5W+Mix mm+w+/VUh2IJdEY2ceOazJib8r6tcfUPfYs5OO69f5y7iV3y1DZ8jvPE1ApueiXfw8ttk9m4Vmp dIOLI3EyfeYmXFaNK0fUmVb00G/TMmIP/A5kP11dhsSisTnw4wxZsh0VZIsG1+rcgc2vyuVUpYP VCUK0pXpfReAlBAslfAWn145QhHt7ibdftuB8kb1dqW2Tv+mCPheViikviVwRHJWIYJYddASGbW QFoC/oLfQS+w3Bv2tpiwS47gKGwMqB3wJrPKkQIWkQt3Mo52Jfl06RloX71SYWWUJhzup3ch6e5 0tXFUltt41UDF/LdQ7AXMXLbnUnc2M2fQK59H2QmsXPNcWHA== X-Received: by 2002:a05:690c:6e11:b0:7d1:d41a:5721 with SMTP id 00721157ae682-800256df98fmr6104357b3.14.1781753580137; Wed, 17 Jun 2026 20:33:00 -0700 (PDT) Received: from localhost (syn-035-130-123-074.biz.spectrum.com. [35.130.123.74]) by smtp.gmail.com with ESMTPSA id 00721157ae682-7fd48ff006fsm53569357b3.40.2026.06.17.20.32.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 Jun 2026 20:32:59 -0700 (PDT) From: Yury Norov X-Google-Original-From: Yury Norov Date: Wed, 17 Jun 2026 23:32:59 -0400 To: Shrikanth Hegde Cc: linux-kernel@vger.kernel.org, mingo@kernel.org, peterz@infradead.org, juri.lelli@redhat.com, vincent.guittot@linaro.org, yury.norov@gmail.com, kprateek.nayak@amd.com, iii@linux.ibm.com, tglx@kernel.org, gregkh@linuxfoundation.org, pbonzini@redhat.com, seanjc@google.com, vschneid@redhat.com, huschle@linux.ibm.com, rostedt@goodmis.org, dietmar.eggemann@arm.com, mgorman@suse.de, bsegall@google.com, maddy@linux.ibm.com, srikar@linux.ibm.com, hdanton@sina.com, chleroy@kernel.org, vineeth@bitbyteword.org, frederic@kernel.org, arighi@nvidia.com, pauld@redhat.com, christian.loehle@arm.com, tj@kernel.org, tommaso.cucinotta@gmail.com, maz@kernel.org, rafael@kernel.org Subject: Re: [PATCH v4 06/20] sched/core: allow only preferred CPUs in is_cpu_allowed Message-ID: References: <20260617174139.155540-1-sshegde@linux.ibm.com> <20260617174139.155540-7-sshegde@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20260617174139.155540-7-sshegde@linux.ibm.com> On Wed, Jun 17, 2026 at 11:11:25PM +0530, Shrikanth Hegde wrote: > When possible, choose a preferred CPUs to pick. > > Push task mechanism uses stopper thread which going to call > select_fallback_rq and use this mechanism to pick only a preferred CPU. > > When task is affined only to non-preferred CPUs it should continue to > run there. Detect that by checking if cpus_ptr and cpu_preferred_mask > intersect or not. > > Since is_cpu_allowed can be called directly or repeatedly in > select_fallback_rq, encode the info in task_struct->has_preferred_cpu_state > if the path is via select_fallback_rq or not. > This helps to avoid N**2 complexity for the rare cases. > > Signed-off-by: Shrikanth Hegde > --- > v3->v4: > - Missing case of PF_KTHREAD is avoided. > - Add a new field in task_struct which encodes intersection of > tasks affinity and preferred CPUs and path its coming from. > > include/linux/sched.h | 1 + > kernel/sched/core.c | 34 ++++++++++++++++++++++++++++++++-- > kernel/sched/sched.h | 18 ++++++++++++++++++ > 3 files changed, 51 insertions(+), 2 deletions(-) > > diff --git a/include/linux/sched.h b/include/linux/sched.h > index fc6ecb3869dd..2d0b1a6d50ac 100644 > --- a/include/linux/sched.h > +++ b/include/linux/sched.h > @@ -1657,6 +1657,7 @@ struct task_struct { > #ifdef CONFIG_UNWIND_USER > struct unwind_task_info unwind_info; > #endif > + int has_preferred_cpu_state; Shouldn't this be protected with the config? > > /* CPU-specific state of this task: */ > struct thread_struct thread; > diff --git a/kernel/sched/core.c b/kernel/sched/core.c > index 9e16946c9d62..714816cfa975 100644 > --- a/kernel/sched/core.c > +++ b/kernel/sched/core.c > @@ -2500,6 +2500,8 @@ static inline bool rq_has_pinned_tasks(struct rq *rq) > */ > static inline bool is_cpu_allowed(struct task_struct *p, int cpu) > { > + bool task_check_preferred_cpu = false; Initialization is not needed. > + > /* When not in the task's cpumask, no point in looking further. */ > if (!task_allowed_on_cpu(p, cpu)) > return false; > @@ -2508,9 +2510,22 @@ static inline bool is_cpu_allowed(struct task_struct *p, int cpu) > if (is_migration_disabled(p)) > return cpu_online(cpu); > > + /* > + * This is essential to maintain user affinities when preferred > + * CPUs change. A task pinned on non-preferred CPU should continue > + * to run there, since this is non-user triggered. > + * > + * If CPU is non-preferred and task can run on other CPUs which are > + * currently preferred, then choose those other CPUs instead > + */ > + task_check_preferred_cpu = !cpu_preferred(cpu) && task_has_preferred_cpus(p); > + > /* Non kernel threads are not allowed during either online or offline. */ > - if (!(p->flags & PF_KTHREAD)) > + if (!(p->flags & PF_KTHREAD)) { > + if (task_check_preferred_cpu) > + return false; > return cpu_active(cpu); > + } > > /* KTHREAD_IS_PER_CPU is always allowed. */ > if (kthread_is_per_cpu(p)) > @@ -2520,6 +2535,10 @@ static inline bool is_cpu_allowed(struct task_struct *p, int cpu) > if (cpu_dying(cpu)) > return false; > > + /* Try on preferred CPU first if possible*/ > + if (task_check_preferred_cpu) > + return false; > + > /* But are allowed during online. */ > return cpu_online(cpu); > } > @@ -3549,6 +3568,14 @@ static int select_fallback_rq(int cpu, struct task_struct *p) > enum { cpuset, possible, fail } state = cpuset; > int dest_cpu; > > + /* > + * Cache value whether task's affinity spans preferred CPUs. Because it's cached, it should go inside is_cpu_allowed(), I think. > + * This helps to avoid repeating the same for each CPU > + * later in the loop. Encode call to is_cpu_allowed coming > + * via select_fallback_rq. > + */ > + p->has_preferred_cpu_state = task_has_preferred_cpus(p) << 8 | 0x1; This looks weird. Your intention is to store three states: not cached, has preferred CPUs and has not preferred CPUs, Why don't you create an enum for it? Or a couple of flags? > + > /* > * If the node that the CPU is on has been offlined, cpu_to_node() > * will return -1. There is no CPU on the node, and we should > @@ -3560,7 +3587,7 @@ static int select_fallback_rq(int cpu, struct task_struct *p) > /* Look for allowed, online CPU in same node. */ > for_each_cpu(dest_cpu, nodemask) { > if (is_cpu_allowed(p, dest_cpu)) > - return dest_cpu; > + goto clear_and_return; > } > } > > @@ -3604,6 +3631,8 @@ static int select_fallback_rq(int cpu, struct task_struct *p) > } > } > > +clear_and_return: > + p->has_preferred_cpu_state = 0; What for resetting it here? I think it should be zeroed only on update of preferred cpumask. In other words, to properly implement caching, you need to have a global counter incremented on each cpu_preferred_mask update, and in task_has_preferred_cpus() you do: { if (p->preferred_cpu_updates == atomic_read(preferred_cpumask_updates)) return p->has_preferred_cpus; p->preferred_cpu_updates = atomic_read(preferred_cpumask_updates); p->has_preferred_cpus = cpumask_intersects(...); } Do you have any numbers that justify this caching? The best practice is to put performance optimizations at the end of the series and provide some sort of benchmark supporting it. > return dest_cpu; > } > > @@ -4612,6 +4641,7 @@ static void __sched_fork(u64 clone_flags, struct task_struct *p) > init_numa_balancing(clone_flags, p); > p->wake_entry.u_flags = CSD_TYPE_TTWU; > p->migration_pending = NULL; > + p->has_preferred_cpu_state = 0; > init_sched_mm(p); > } > > diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h > index c7c2dea65edd..38fd84b0b8f8 100644 > --- a/kernel/sched/sched.h > +++ b/kernel/sched/sched.h > @@ -4213,4 +4213,22 @@ DEFINE_CLASS_IS_UNCONDITIONAL(sched_change) > > #include "ext.h" > > +/* > + * has_preferred_cpu_state is encoding two bits of information. > + * First Byte is to encode where the call to is_cpu_allowed coming from. > + * Second Byte is to encode the intersection of task affinity > + * and cpu_preferred_mask. > + * > + * If 1st Byte is set, call to is_cpu_allowed coming from select_fallback_rq. > + * That helps to avoid repeated calculation keeping time complexity same. > + */ > +static inline bool task_has_preferred_cpus(struct task_struct *p) This function should be void because you change the task state. > +{ > + int cached_value = p->has_preferred_cpu_state; > + > + if (cached_value & 0x1) > + return p->has_preferred_cpu_state >> 8; > + else > + return cpumask_intersects(p->cpus_ptr, cpu_preferred_mask); > +} > #endif /* _KERNEL_SCHED_SCHED_H */ > -- > 2.47.3