From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from MW6PR02CU001.outbound.protection.outlook.com (mail-westus2azon11012058.outbound.protection.outlook.com [52.101.48.58]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CE25C35F5E6 for ; Sat, 9 May 2026 18:03:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.48.58 ARC-Seal:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778349811; cv=fail; b=Xe2tVyCWejz5Zy8VifXIlczHomq5z9lD6JHHfCfJ6KWdRbUvtkdXGOrczgREjn2Y2JqOYxqqn5u6kCJaCxsd890Azqu9j4ZzOlNsCsWIc0oNdcGG/NAIw0vTxHolYRbZdTcc3pk2MzzK3/ayQstwb8Nirymb6E/rD6u5KqJtOgw= ARC-Message-Signature:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778349811; c=relaxed/simple; bh=LS/PB4zQ/zpIQJPEfCMrCvuBDAhdGc/oxAoQ3fqCXTQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=NzjR9OMoW3d5QSZAJ9M6/0uJ1TGWbaiGm7uBx4MYKjKo4ZpidY791F6qNEngWt0fR/ApIjPAnvf5vqmfFVGT6ON0zAsLPd5gmxBRHh+TOn+ui/6CBFgMnHpIKEde6BRRHEm5pghE4Mqfp0ucIQWgDVufUwojB+eoebPj/agdhoc= ARC-Authentication-Results:i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=XeOQoRfc; arc=fail smtp.client-ip=52.101.48.58 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="XeOQoRfc" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=T7nwvSHGqYoeOS9D5T8WtD3T2B7pa5gDAMifQ/sP8ssRhL6Q2o3TES+nET277yewG21MElVJXrsynCWyFc01WtHbLoEovFCxMeU4vGqh0QFJobGK5Yy0pc4GG+iF/xoKh/kTC0e8Eq8mfygdcNBYiQMFyK/FzUinDzWNzRSr55rSuhRhhDVI+O08Dy2w6Hfrlcq6OyyMzce4GWEPIzGcgTAIuhC3cWuLzByQa4S+ewk+cpx35dzFDsUjKnTp5txfNxwGMDaoZS1WQdGqtt/dk0+0e5ObSPDxMkxYyVNrMVksx7rBIrRanfDJ1OT8GWKWKl+Sz1TrxiSmbvjk/dvjEA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Pb8c7z7aU7m/hyREX91UityDyZLAXRi/UWatbMmity8=; b=nEb5QOuDyVPURD2WAV3IzZeKJjiLhG5DuSpoeD2J5zQBuhr2wioA3qoAIbuPwn+/r9SuwzXeH5YDeY0vHeydIXbSrodf4z9OccPFvrytQCDGSx8elb70aRdVAu+rnAencW1o4mCVwnDI3hLedUmIXE7f1Ola5e8e2ppLm3c+TAJezFNQDrURg0Gf3qQGrakRFsPC7DUGpzATvrZMOsczhfrm7ckfzVX00eYFFhpikqjTgj3TFfib4UYFoXWfiP2Otexk65iYWE1CXbd+pd+xoPF9wsqzFPvHxqidFNmGWZbCOEdPiVUGSTA/zq5K6MXyEFuEjzhkU5TyToz5eZcMjw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Pb8c7z7aU7m/hyREX91UityDyZLAXRi/UWatbMmity8=; b=XeOQoRfckqp+4ZxrUkU/QOluz0bEbnnP/GZ1Q6jUs8tQK9Ruii/SAnBtq6jKFDLnyN2EZrMtgK99TAsQdNZwjlT2dbm9UQdTcwv1OT3ZqBB6FVeos10yjJIq4U0bPcJP4YfMGASyaMoG+4kiKD9+H/vRWFEtca6VgH56LPNvifaJoY4TjiCTAT7P6R3wYRpkNVfhtuWPO2fnih6elBiAg+jrJr2IlC/8XRU5wHQ4BIFE3za1aV2QWDZjNm+sEvUMkQ4GoktDbIyKvRuJNrJAZ1/Ex179LIw/Bc3/K8/Bgex6PeZ1hhTOy1cCKbfPFKQihuhPbXASyLvHLl/4OUBSmA== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from LV8PR12MB9620.namprd12.prod.outlook.com (2603:10b6:408:2a1::19) by DM4PR12MB6039.namprd12.prod.outlook.com (2603:10b6:8:aa::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9891.21; Sat, 9 May 2026 18:03:24 +0000 Received: from LV8PR12MB9620.namprd12.prod.outlook.com ([fe80::299d:f5e0:3550:1528]) by LV8PR12MB9620.namprd12.prod.outlook.com ([fe80::299d:f5e0:3550:1528%5]) with mapi id 15.20.9891.019; Sat, 9 May 2026 18:03:24 +0000 From: Andrea Righi To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot Cc: Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider , K Prateek Nayak , Christian Loehle , Phil Auld , Koba Ko , Felix Abecassis , Balbir Singh , Joel Fernandes , Shrikanth Hegde , linux-kernel@vger.kernel.org Subject: [PATCH 3/5] sched/fair: Prefer fully-idle SMT cores in asym-capacity idle selection Date: Sat, 9 May 2026 20:01:23 +0200 Message-ID: <20260509180302.1839122-4-arighi@nvidia.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260509180302.1839122-1-arighi@nvidia.com> References: <20260509180302.1839122-1-arighi@nvidia.com> Content-Transfer-Encoding: 8bit Content-Type: text/plain X-ClientProxiedBy: MI2PEPF00000B7D.ITAP293.PROD.OUTLOOK.COM (2603:10a6:298:1::410) To LV8PR12MB9620.namprd12.prod.outlook.com (2603:10b6:408:2a1::19) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: LV8PR12MB9620:EE_|DM4PR12MB6039:EE_ X-MS-Office365-Filtering-Correlation-Id: 1e9d1cb6-534b-4ba1-0b77-08deadf5436f X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|1800799024|7416014|376014|22082099003|56012099003|18002099003; X-Microsoft-Antispam-Message-Info: 3gPIK3ZyQUx3cyGdhIE9jbVBfFdu/W6ZJJSC/toa+0WpqaffUgPvCxn6hIB2atVzWnP70dgnulsJUHnpTvbx4/+raoADJNXiyfxQsgQ8jfXyu1XC05GOmk7B+sNtBg7pr7/+Hvn/pse0n2mJdiQa6VJCf+j4KYcA0M+XWsS8gkfKjLRcaw93cu9SlaD0rJmGWmOCtIgI4hHZ1kLxVw3V1M2soi8xsSfZHfYATGAnVXqEbCIbr3QyefXMXkDdAhPuTE39h9hwAq3E1RzKPAASaIX62swJBzQLZWTpLbvSxGAcmo4wlmR0Vq3WaOHxg8xix8YFpwAFMsb98QdSk6HrON7WPPltmoiXv/TkaxevP4t70rL+eLMvBojNW4MTSyzKeeoyZK4W44LP90pmcltEA5FppO8dvSxEtWxAyid3O6gIWRolnaBcbHPFH8UqTjxJIqW4cF3rRMsm2bVisnNuLlEo6Nd5Rxvt+WHO/1ViuSZgc8NUTA9cQQNf/pzvWlNmb5oPXWwmOs5VqxYAyrhR10x4l+fJBElwxWR1dj90mMcI+2TWCde4wOvSkTOpaqJQp9BBX1a3KxCa3bt5s8dlPvcWLBQ3u/udoUWQXfaBGdzWQeKb1xCITHILjKLqUo4l7AUDCS1nVqEsObpM45JLwG7/qSic8ydutZmnQ2Rsk1u/1+Gp3CtHB6Ap9E2/3bwI X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:LV8PR12MB9620.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(1800799024)(7416014)(376014)(22082099003)(56012099003)(18002099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?jGrfQjyuzbo/v8LB4DHFpBkN4wVI5ydQ46gPoNIjx/GICnkdvQUsQXixgQTq?= =?us-ascii?Q?D5dpCmdhTWzIsyGDsiLsAyUmlpy33g41P6FTIaX0Ro3XgpSBLcHXrybGaF8D?= =?us-ascii?Q?i9MmwjobYJTk2YYYwQB+62x1+7KbsldVNOR6cSD042POS/oUs7EBuNIE5QDu?= =?us-ascii?Q?wcFRfGZbovW/520eYth+Qf7B9/EQsxnHAZjz7O2PC/70WlVT1eWdZQvUGj9e?= =?us-ascii?Q?3TKKXKJjqItH30jQrE3Su8A0KlDb8LTtBgV7CkYA4UEK5FSfli4+M55Qs7sG?= =?us-ascii?Q?xsHFPnMXFgS3B8vK02/OkAx6Qa407+2jCAEtP8i6IXQI31Z3KGRPSg9YQCMU?= =?us-ascii?Q?ytljutRwGGN7NMvsaCasDYf3b7aPOqJo4kuU4zYam6wkVe0f0Mus6Jaq9dZ0?= =?us-ascii?Q?FnqRt/t7HZFL8tf3AK8S58PQ953qlcdzSkutMjx0EsWZqeYgLVwpiLBch8l2?= =?us-ascii?Q?QJZ2TIYZGaatDCRO3APzGd8n1EfUI6qeow7hQKoKWY/GHllbEBOj/ulG+fHg?= =?us-ascii?Q?NajNY/xars2U+CnQ9hMjoABEbcvFTfeifEGnwyJ0tKNm0TPOZl8MnJrNSzD5?= =?us-ascii?Q?jKQG5J73gQdIUwfOq6u3mupFJhpJHqFADms38XfDxy/fdLQIbevXtwkLadca?= =?us-ascii?Q?TdwMEZ0GlKvOfNmQ1kMCr8Us0Ue5ahZFUbQg3uJ90yV+ZrXW4dTIYOqtYgrI?= =?us-ascii?Q?4sZoMcZbrhI3iK93tYia33kiUbdNyPTrhCMfxBhtXg0spp1hJyBjENsELQ9O?= =?us-ascii?Q?t2TRpTtALgFzLHfRWX1bVgy9gJtdD7YFCAry/4e0A1UvJ5MTTtSpyFoyWruK?= =?us-ascii?Q?kw0LUKR01sNLkVMjqKkfFXIayJL/+1XSjcuC84RN4AVoDAcwvUlzIUBv7bB9?= =?us-ascii?Q?uH51o0TIOvswVZtCKkOj5Jwlflq01y/481TWqzxiOSB0t3nYIJZQlvR6CF8G?= =?us-ascii?Q?zDJLvZlX3qTm0il2d2GPw18teEK/0WhVCJgd6rmIKmwE0wrWfnUHIxgTm8Oo?= =?us-ascii?Q?jC6julnhBfT8mKYSeQLNyNQiLzMvUWvfL9BDaaZUVhuAPy17A8iRbX45zeZS?= =?us-ascii?Q?Olo0DSo8vql5xsOdJxndGb/+ZLlTtYrT3AUCg27DbQspgna3vaChC28PnYkh?= =?us-ascii?Q?Bw9py1IYrB0xWR3kaDfWqJsq1ZjP1Nd68oMJ1AL4LFaKIyr5GgyIQv75f1fK?= =?us-ascii?Q?vRHSk9cSIdYwgmdPl+kJlyQT5UKP6enNeMrbc3YBHInepZOlsSTbdwb35DvF?= =?us-ascii?Q?qBmyKzfNwv2/NTfDMqg/GQZJRR+f+LancBe+n4Zd+atf5a5N2KvqM5TGQZQz?= =?us-ascii?Q?JixxOeKJX5iTnbI5gXOvF2e32GBAe8XHDGr1RvR5uJKQxUeOyXEW0hLo7bL5?= =?us-ascii?Q?ed8ZgwgmHgDBPULsjYjT870zFJ2AACtqorL8oJe/S6OjRUT5z029w5RDOPkG?= =?us-ascii?Q?gwygdnzeNOQqYDrUdNal7cq7fb2cR7waRQ/lvHFUkIVoRbWuzlKWEV2WRWoo?= =?us-ascii?Q?+rVCWQZ6+RkHbe/xhYtv/jleH4+9qiVcpOxYHOajP8PDogbiUBcT0IiRZUQ5?= =?us-ascii?Q?sWX7oKcfxslFGUZ8UZshOiT/jERSth4+ryb2v/uZwFh7Y6/1TTtioTrJTGJq?= =?us-ascii?Q?O31C3SSCrBw6/KJaP9LDqO51P+uV4jrRzAfOz8l4CqC9tyj8AgpWaeSkIe/g?= =?us-ascii?Q?+z66dD5L/SdOfCTN95T/Exw7HpHp9rBgsLIo0+qrwS8beTas?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 1e9d1cb6-534b-4ba1-0b77-08deadf5436f X-MS-Exchange-CrossTenant-AuthSource: LV8PR12MB9620.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 May 2026 18:03:24.5897 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: puyDbs1HWCOItSgHRrp5Vm5I3uWP05CdAQruxWYpEx2tqTcfN/HTYJNLLvnbDpNBqy8BPVZdYagJR3+Nfr5TFw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB6039 On systems with asymmetric CPU capacity (e.g., ACPI/CPPC reporting different per-core frequencies), the wakeup path uses select_idle_capacity() and prioritizes idle CPUs with higher capacity for better task placement. However, when those CPUs belong to SMT cores, their effective capacity can be much lower than the nominal capacity when the sibling thread is busy: SMT siblings compete for shared resources, so a "high capacity" CPU that is idle but whose sibling is busy does not deliver its full capacity. This effective capacity reduction cannot be modeled by the static capacity value alone. Introduce SMT awareness in the asym-capacity idle selection policy: when SMT is active, always prefer fully-idle SMT cores over partially-idle ones. Prioritizing fully-idle SMT cores yields better task placement because the effective capacity of partially-idle SMT cores is reduced; always preferring them when available leads to more accurate capacity usage on task wakeup. On an SMT system with asymmetric CPU capacities (NVIDIA Vera Rubin), SMT-aware idle selection has been shown to improve throughput by around 15-18% over NO_ASYM mainline and by around 60% over ASYM mainline, for CPU-bound workloads (NVBLAS) running an amount of tasks equal to the amount of SMT cores. Cc: Vincent Guittot Cc: Dietmar Eggemann Cc: Christian Loehle Cc: Koba Ko Reviewed-by: K Prateek Nayak Reported-by: Felix Abecassis Signed-off-by: Andrea Righi --- kernel/sched/fair.c | 119 +++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 113 insertions(+), 6 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 960a1a9696b98..6f0835c15ee11 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -8018,6 +8018,54 @@ static int select_idle_cpu(struct task_struct *p, struct sched_domain *sd, bool return idle_cpu; } +/* + * Idle-capacity scan converts util_fits_cpu() outcomes into preference ranks, + * where lower values indicate a better fit - see select_idle_capacity(). + * + * A CPU that both fits the task and sits on a fully-idle SMT core is returned + * immediately and is never assigned one of these ranks. On !SMT every CPU is + * its own "core", so the early return covers all fits-and-idle cases and the + * core-tier ranks below become unreachable. + * + * Rank Val Tier Meaning + * ------------------------------ --- ------ --------------------------- + * ASYM_IDLE_CORE_UCLAMP_MISFIT -4 core Idle core; capacity fits + * util but uclamp_min misses. + * ASYM_IDLE_CORE_COMPLETE_MISFIT -3 core Idle core; capacity does + * not fit. Still beats every + * thread-tier rank: a busy + * sibling cuts effective + * capacity more than a + * misfit hurts a quiet core. + * ASYM_IDLE_THREAD_FITS -2 thread Busy SMT sibling; capacity + * fits util + uclamp. + * ASYM_IDLE_THREAD_UCLAMP_MISFIT -1 thread Busy SMT sibling; capacity + * fits but uclamp_min misses + * (native util_fits_cpu() + * return value). + * ASYM_IDLE_COMPLETE_MISFIT 0 thread Busy SMT sibling; capacity + * does not fit. + * + * ASYM_IDLE_CORE_BIAS (-3) is an offset, not a state. On an idle core, + * fits += ASYM_IDLE_CORE_BIAS rebases thread-tier ranks into the core tier: + * + * ASYM_IDLE_THREAD_UCLAMP_MISFIT (-1) + BIAS -> CORE_UCLAMP_MISFIT (-4) + * ASYM_IDLE_COMPLETE_MISFIT (0) + BIAS -> CORE_COMPLETE_MISFIT (-3) + * + * ASYM_IDLE_THREAD_FITS (-2) is never rebased because a fully-fitting idle-core + * candidate early-returns from select_idle_capacity(). + */ +enum asym_fits_state { + ASYM_IDLE_CORE_UCLAMP_MISFIT = -4, + ASYM_IDLE_CORE_COMPLETE_MISFIT, + ASYM_IDLE_THREAD_FITS, + ASYM_IDLE_THREAD_UCLAMP_MISFIT, + ASYM_IDLE_COMPLETE_MISFIT, + + /* util_fits_cpu() bias for idle core */ + ASYM_IDLE_CORE_BIAS = -3, +}; + /* * Scan the asym_capacity domain for idle CPUs; pick the first idle one on which * the task fits. If no CPU is big enough, but there are idle ones, try to @@ -8026,8 +8074,14 @@ static int select_idle_cpu(struct task_struct *p, struct sched_domain *sd, bool static int select_idle_capacity(struct task_struct *p, struct sched_domain *sd, int target) { + /* + * On !SMT systems, has_idle_core is always false and preferred_core + * is always true (CPU == core), so the SMT preference logic below + * collapses to the plain capacity scan. + */ + bool has_idle_core = sched_smt_active() && test_idle_cores(target); unsigned long task_util, util_min, util_max, best_cap = 0; - int fits, best_fits = 0; + int fits, best_fits = ASYM_IDLE_COMPLETE_MISFIT; int cpu, best_cpu = -1; struct cpumask *cpus; @@ -8039,6 +8093,7 @@ select_idle_capacity(struct task_struct *p, struct sched_domain *sd, int target) util_max = uclamp_eff_value(p, UCLAMP_MAX); for_each_cpu_wrap(cpu, cpus, target) { + bool preferred_core = !has_idle_core || is_core_idle(cpu); unsigned long cpu_cap = capacity_of(cpu); if (!choose_idle_cpu(cpu, p)) @@ -8046,8 +8101,13 @@ select_idle_capacity(struct task_struct *p, struct sched_domain *sd, int target) fits = util_fits_cpu(task_util, util_min, util_max, cpu); - /* This CPU fits with all requirements */ - if (fits > 0) + /* + * Perfect fit: capacity satisfies util + uclamp and the CPU + * sits on a fully-idle SMT core (or this is a !SMT system). + * Short-circuit the rank-based selection and return + * immediately. + */ + if (fits > 0 && preferred_core) return cpu; /* * Only the min performance hint (i.e. uclamp_min) doesn't fit. @@ -8055,9 +8115,33 @@ select_idle_capacity(struct task_struct *p, struct sched_domain *sd, int target) */ else if (fits < 0) cpu_cap = get_actual_cpu_capacity(cpu); + /* + * fits > 0 implies we are not on a preferred core, but the util + * fits CPU capacity. Set fits to ASYM_IDLE_THREAD_FITS + * so the effective range becomes + * [ASYM_IDLE_THREAD_FITS, ASYM_IDLE_COMPLETE_MISFIT], where: + * ASYM_IDLE_COMPLETE_MISFIT - does not fit + * ASYM_IDLE_THREAD_UCLAMP_MISFIT - fits with the exception of UCLAMP_MIN + * ASYM_IDLE_THREAD_FITS - fits with the exception of preferred_core + */ + else if (fits > 0) + fits = ASYM_IDLE_THREAD_FITS; /* - * First, select CPU which fits better (-1 being better than 0). + * If we are on a preferred core, translate the range of fits + * of [ASYM_IDLE_THREAD_UCLAMP_MISFIT, ASYM_IDLE_COMPLETE_MISFIT] to + * [ASYM_IDLE_CORE_UCLAMP_MISFIT, ASYM_IDLE_CORE_COMPLETE_MISFIT]. + * This ensures that an idle core is always given priority over + * (partially) busy core. + * + * A fully fitting idle core would have returned early and hence + * fits > 0 for preferred_core need not be dealt with. + */ + if (preferred_core) + fits += ASYM_IDLE_CORE_BIAS; + + /* + * First, select CPU which fits better (lower is more preferred). * Then, select the one with best capacity at same level. */ if ((fits < best_fits) || @@ -8068,6 +8152,19 @@ select_idle_capacity(struct task_struct *p, struct sched_domain *sd, int target) } } + /* + * A value in the [ASYM_IDLE_CORE_UCLAMP_MISFIT, ASYM_IDLE_CORE_COMPLETE_MISFIT] + * range means the chosen CPU is in a fully idle SMT core. Values above + * ASYM_IDLE_CORE_COMPLETE_MISFIT mean we never ranked such a CPU best. + * + * The asym-capacity wakeup path returns from select_idle_sibling() + * after this function and never runs select_idle_cpu(), so the usual + * select_idle_cpu() tail that clears idle cores must live here when the + * idle-core preference did not win. + */ + if (has_idle_core && best_fits > ASYM_IDLE_CORE_COMPLETE_MISFIT) + set_idle_cores(target, false); + return best_cpu; } @@ -8076,12 +8173,22 @@ static inline bool asym_fits_cpu(unsigned long util, unsigned long util_max, int cpu) { - if (sched_asym_cpucap_active()) + if (sched_asym_cpucap_active()) { /* * Return true only if the cpu fully fits the task requirements * which include the utilization and the performance hints. + * + * When SMT is active, also require that the core has no busy + * siblings. + * + * Note: gating on is_core_idle() also makes the early-bailout + * candidates in select_idle_sibling() (target, prev, + * recent_used_cpu) idle-core-aware on ASYM+SMT, which the + * NO_ASYM path does not do. */ - return (util_fits_cpu(util, util_min, util_max, cpu) > 0); + return (!sched_smt_active() || is_core_idle(cpu)) && + (util_fits_cpu(util, util_min, util_max, cpu) > 0); + } return true; } -- 2.54.0