From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from CH1PR05CU001.outbound.protection.outlook.com (mail-northcentralusazon11010014.outbound.protection.outlook.com [52.101.193.14]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D5DCD405C59 for ; Sat, 16 May 2026 09:04:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.193.14 ARC-Seal:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778922285; cv=fail; b=slTCZxEhZ96B8bQu7PkW9FtokG869IoI3WaMV4Kg2fWRYphfxxWwR/tVYSm4XFCuXoyM8PpITW00kswLiW2/W26LAHAlPbg3obcG47IurfkwrQB0jkhjMEidR10GDdJoAE/x13KuMLBQ/dcdkfPWHyPOtsZt0RdVNDmot02n7ms= ARC-Message-Signature:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778922285; c=relaxed/simple; bh=X8YiMXLqQhw6sAEFEEAV+hY5apeu1FVSldLdC9D15J4=; h=Date:From:To:Cc:Subject:Message-ID:References:Content-Type: Content-Disposition:In-Reply-To:MIME-Version; b=oOkM//ZJjgJ1h6MBlRsr8ipYiqUOSRIF/0ZJ10VLYn6mR4+PsN2nNRHrf+6c94CeOgEB/onvHOBn23nwO5TYZfhZj9keK50q7grhDbni9jkIKdyeisqFof+FD8SVVhWP2Li/zR7yUmh3Miwj6G+hS3eR+sAGx1zKDqko3J0+JrY= ARC-Authentication-Results:i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=aXsYXBYL; arc=fail smtp.client-ip=52.101.193.14 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="aXsYXBYL" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=WM8c4JCRd47WYS5BRFwxkUbvI1q8FeuHwhmKl2IPGEHPRVyh6sAbMk0RuQHv5BadxSpAkTnKlADQcnApjtVkDmzUtNPCpMxpmEf3EI6PzdB8sqlTwW/VCHVqbRggokuGDCYWEgf8h1Fo69jSDBrtTrmG1RnelJWbAUnu0HPZShAOGGqkVex98k5nHf3cHSE/3coOdoqrW6H5ar8AYS2iq/5KEazE52iZrBMrV0s7ekSRgi5+rKfzseDxO9CvIXuobEdujm09ElzJGhUtHyOs4LQckzw5J6Kt8AyfZS+4lCU2GPMnyc9t3r3VM8qtdhQVXR7rodQg/PkR8zFuleKatg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=+EqJGngybBfe6ZfnmOYE5Ol9kP21LHR8Fq0mMoWWYiA=; b=ZYjK1NcvUZLNrL8srEff6QmFBETEdxNPNEULR5e2YFPE30xeYw0KIuPi08l0H/pT+rA0q3wo7SYrTBk4Qv2csI4qLtWfLbk6S7woL1+OcDGVWi777dCYC0yMlXyFActN6/+5ZEEzcZFZN9jKuNGKESfJfJgD38qSK1DiwCvIctHB2AFUKqj6HCx6Q8L71+5sUKdEgpDaXNcsAcnuV1r8pIzMxramIVwIQkotlgemLlaP3Ahi0PkQFNjnL39FtshIVgMPvvi87OscQ9zTiJCpppnxH3AXe88XcP3knqMOMTt8QHxYkcTAd8DSq6hP0GYywkJXRZFUf1b3DcDJ5pBfuQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=+EqJGngybBfe6ZfnmOYE5Ol9kP21LHR8Fq0mMoWWYiA=; b=aXsYXBYLx5HLVMAIAfHptQZFjIouwu7uyCYF2SKOWd2bfKD/+I3SYDeTYZ1BVuSsz3fH9n+UIVWxT7vCLf3rKktt9eX2GRLgomDpQBPcH7jgDCH3M9JGl+LMeWvfviaPcCQngoNKigD+XD64GzFGrvJ+dRhdOu9tPromf5GqaKGnJovToaFJIfqlNb2BkX+l0b9ptqHQOXWpdmDbmSqK6/6O7m42pdCnUH4zGIQ9jjgW+S6JV6XsbPxon5RfnoNMKurREyxNSCcrkQt8SgiaA/rsxNAERf7s6OKZuqy+81yHKiVUmyg6QAyn7TQmQ/1weCQ48qiAyMJf4bYLwCtPmQ== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from DM6PR12MB4827.namprd12.prod.outlook.com (2603:10b6:5:1d6::14) by IA0PR12MB7532.namprd12.prod.outlook.com (2603:10b6:208:43e::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.25.21; Sat, 16 May 2026 09:04:36 +0000 Received: from DM6PR12MB4827.namprd12.prod.outlook.com ([fe80::6261:3040:864b:159c]) by DM6PR12MB4827.namprd12.prod.outlook.com ([fe80::6261:3040:864b:159c%4]) with mapi id 15.20.9913.009; Sat, 16 May 2026 09:04:36 +0000 Date: Sat, 16 May 2026 11:04:25 +0200 From: Andrea Righi To: Shrikanth Hegde Cc: Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider , K Prateek Nayak , Christian Loehle , Phil Auld , Koba Ko , Felix Abecassis , Balbir Singh , Joel Fernandes , linux-kernel@vger.kernel.org, Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot Subject: Re: [PATCH 4/5] sched/fair: Reject misfit pulls onto busy SMT siblings on asym-capacity Message-ID: References: <20260509180955.1840064-1-arighi@nvidia.com> <20260509180955.1840064-5-arighi@nvidia.com> Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-ClientProxiedBy: MI0P293CA0003.ITAP293.PROD.OUTLOOK.COM (2603:10a6:290:44::9) To DM6PR12MB4827.namprd12.prod.outlook.com (2603:10b6:5:1d6::14) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6PR12MB4827:EE_|IA0PR12MB7532:EE_ X-MS-Office365-Filtering-Correlation-Id: 2dd76e80-d231-44e0-8a8e-08deb32a26fe X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|1800799024|376014|7416014|56012099003|22082099003|18002099003|4143699003|11063799003; X-Microsoft-Antispam-Message-Info: xvkFz3zF8dyP6MZdsQ+GQajfDvERi7hbaO+OryqgXD3kB2hUdlx2qCv76lg+O16qhX2B7gNPivEeZWIpdAF6/nplBvn/5AlS2O39UcUAw85MUUJdiy8TRTdpDsHDzu5EkC/7n8NXKh+0RHZj2v/mrga5RZWvMRvkbGPueGN/DqHs4fByL/7ZaYWTh/mZvQedp+g+MTTwg5pI/7EM5COaB+9qp6gcfDaYkiBBX6i/bpVMvPEoL7byCOO3ROBmVp30zZchC1uJRbjbdJo/LciIU4CCcyio/0PU5/m9znfvVI+/w9MT/ZS9DmqEQ6Mx70SH4V9RyZRkbgzHDXZGTWh4hIfSV3NpQlEpuNCD44rn7H3syJiqZ8HxfkkP7wIRpzIARk8eM9o0m5V7/Aybug0JcUGK02E5uQju8t5dwj9vA5JtIhAumCeOZMH91qlYyEy1ei1eIQAc31VKdlYPqz107ppJ1zvNu7SaF9yMU2yOqKzL7bowEjqSm3QW+JzVYdOQNMMxys0AifbsTkOigoYzTXBR22WZ0EvMPfCAVsB7VQcb05iYFYkjeWqpb8CVimPewotdPVRCitkUQRS6rlp8S6QPCFXB0RF0iAbtJ2rEZYvhHkzqUE1T+W6L9kJKIgsRrUBbULkhphY40cA3RRe2QkM0zPavAUvg9orar6AMpzkAsbAouPVl+D8iyQBI8rhP X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM6PR12MB4827.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(1800799024)(376014)(7416014)(56012099003)(22082099003)(18002099003)(4143699003)(11063799003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?1tMcfKNHR0xb7F0m3xzvH5KEHjCPNf3+MqGiPpyc+Al6siBWTIRvl/NPC3tr?= =?us-ascii?Q?0CSIxcqoXY0DT8jKlHPKi3HkYku+HGVGQ45wwmeigFE7R2txXNmcsUjxDHPS?= =?us-ascii?Q?sxQMcNoSI+JosZlC2QgF63cXwDudejzhiB+T4sTb8p6FOFpxLo8bXQBmjoay?= =?us-ascii?Q?6QwzWGd4tRxB/hgmrUaUjiMXY6yGB2/iLn70/RcdWmiPp+alQg99b8xK0E9i?= =?us-ascii?Q?9ju0RvBWLm7peQZ0VPaQnqC84RZt9wtEWe4Pm5bYeewZr9r3HBNfj5/CWu3I?= =?us-ascii?Q?D9qVXInbYUmKnut7x8s3uhI5HQW1jPqxsxQ1cKlJJM0PqEoTT3Nd/Yo3CoL+?= =?us-ascii?Q?sSNBdsQGhvcULxWw5FVc/l24x+CeT4jO83mJHfXlIbT5LdhZI2G6D2hYlXTO?= =?us-ascii?Q?168GaAfudHUJWOEPm7mO16kz/0DXWaDy2cy5+Mvachn17+dMp7+a08IM/Ink?= =?us-ascii?Q?5d2TXD74U6GGtLabPmSt+4+pt27mBlM7M70Uby2s0bp6m71Qdv+0QuTGmiGU?= =?us-ascii?Q?piJ84/FyVcVQJspQlj40IPP4hQvnsJTqr7TDmvZqXgj0UnUCA01Z1rO7zRev?= =?us-ascii?Q?pBKS+CntEOivehD9jDOuOXQjkg1uE6bh0WjQ2SlrweoATNYBnWfpsV/w7Ump?= =?us-ascii?Q?DRwpRUIUMTcZb+XTHUaOhrl0A9D7MRjxMS1miX1s4oRc08fj1CsG/HOTkWk2?= =?us-ascii?Q?fQeheWcxH+JASbRs1ey1b4RbiGtIri8LiyIgo2mUsVBaZPeRVYkhYSWU458z?= =?us-ascii?Q?U+cLAiNdg6Rm0/AbjYyznrHH75ncCakl43TRmKxGuKbRQ3aSVTEzaNmtjw1w?= =?us-ascii?Q?IYCfaGlTHcrigaz98Du7euGPjlazfy7+Zt/Jn42dEKqMe+la+SlS6g0QXtO4?= =?us-ascii?Q?+tKLgsmNLyzkyvtp+6RiCwAwgRy14f//LVgloQImlDWxcO103ItEeIEPP0qD?= =?us-ascii?Q?aLYjZBJTT+1LOm9VOwc05Rds4esP9ZEPdGtyJm2vCPSE38kF0hmv9gQvU8bo?= =?us-ascii?Q?CQRMXyLlTBlqdbGJVhV9YNfylCelxt35AYynMzRSHKLzk7M+RmHwBKAZqPmk?= =?us-ascii?Q?x7GxDyEBuf6pRDc44LGx+hsWDWz7IIJslm16kI5JTSjL232uSFEEs/+yoxPV?= =?us-ascii?Q?bgn4J2qf9QrgjFn+U5/Gj76L0OusZKwL9Q9XkMJEWvzd+0umN9ao6GRnvECy?= =?us-ascii?Q?BKXv+IjpJAMbFozw+saiKqcm/zGCbJ0aphFVt+5ICLawxqTxS/S1xs2Jvkz2?= =?us-ascii?Q?AFV0GSzBGwXmGpEDMmm3Y5bZnVwu+uHw10AMPqDkeu2xjgqfw/dmxWg+/RAv?= =?us-ascii?Q?kQcQXMZ3BFgeceJo+GaPbIjDVceEi0OoHE69V7ZeonXUYdfEwcHniL/Q80UR?= =?us-ascii?Q?Fk40h+A1xrr0OU+Jgw1DawvK9Gk+b/bwoTAM6GXK6NeAmvnQ2GZ4UKvRXWy1?= =?us-ascii?Q?c6E4j4UBTYgg76Sv0tHfSRJWEmDdKeAcCIjlwMCiyx6YvLIE2R+hwJVdhsJr?= =?us-ascii?Q?v/19JAg028bcle+ek9IHp8GUIotZ2U8g5sga4tnDyK7Lv4JN+Ar/tv5iIrOP?= =?us-ascii?Q?70pHrxAQgSzTuJSP1Lcj1tv8xDLZ1adxdAi//15UCEVM+ZmtZVKRechKOQDP?= =?us-ascii?Q?NFCTXKFAnzsEU7e0HJnQpPUB31peLbstIGFYQdXNXRxfjSrNaQFRqW/1LkzR?= =?us-ascii?Q?SjSDGmSWTGywscpeG2qIXfxkcpGqhnU6tdnsLCKVIZRZ9W/p?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 2dd76e80-d231-44e0-8a8e-08deb32a26fe X-MS-Exchange-CrossTenant-AuthSource: DM6PR12MB4827.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 May 2026 09:04:36.1665 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 5madr6zmReqDivbCOb7Pd33FoOJAplRzrmKapjUH0cxUIgguYVrLtyyBq+MAMl0jrs7SAF4XpMgy2MEBqpCgkA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA0PR12MB7532 Hi Shrikanth, On Fri, May 15, 2026 at 03:39:55PM +0530, Shrikanth Hegde wrote: > On 5/9/26 11:37 PM, Andrea Righi wrote: > > When SD_ASYM_CPUCAPACITY load balancing considers pulling a misfit task, > > capacity_of(dst_cpu) can overstate available compute if the SMT sibling is > > busy: the core does not deliver its full nominal capacity. > > > > If SMT is active and dst_cpu is not on a fully idle core, skip this > > destination so we do not migrate a misfit expecting a capacity upgrade we > > cannot actually provide. > > > > Cc: Vincent Guittot > > Cc: Dietmar Eggemann > > Cc: Christian Loehle > > Cc: Koba Ko > > Cc: K Prateek Nayak > > Reported-by: Felix Abecassis > > Signed-off-by: Andrea Righi > > --- > > kernel/sched/fair.c | 11 ++++++++++- > > 1 file changed, 10 insertions(+), 1 deletion(-) > > > > diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c > > index 6f0835c15ee11..2ddba8bd27e59 100644 > > --- a/kernel/sched/fair.c > > +++ b/kernel/sched/fair.c > > @@ -9693,6 +9693,7 @@ struct lb_env { > > int dst_cpu; > > struct rq *dst_rq; > > + bool dst_core_idle; > > struct cpumask *dst_grpmask; > > int new_dst_cpu; > > @@ -10918,10 +10919,16 @@ static bool update_sd_pick_busiest(struct lb_env *env, > > * We can use max_capacity here as reduction in capacity on some > > * CPUs in the group should either be possible to resolve > > * internally or be covered by avg_load imbalance (eventually). > > + * > > + * When SMT is active, only pull a misfit to dst_cpu if it is on a > > + * fully idle core; otherwise the effective capacity of the core is > > + * reduced and we may not actually provide more capacity than the > > + * source. > > */ > > if ((env->sd->flags & SD_ASYM_CPUCAPACITY) && > > (sgs->group_type == group_misfit_task) && > > - (!capacity_greater(capacity_of(env->dst_cpu), sg->sgc->max_capacity) || > > + (!env->dst_core_idle || > > + !capacity_greater(capacity_of(env->dst_cpu), sg->sgc->max_capacity) || > > sds->local_stat.group_type != group_has_spare)) > > return false; > > @@ -11485,6 +11492,8 @@ static inline void update_sd_lb_stats(struct lb_env *env, struct sd_lb_stats *sd > > unsigned long sum_util = 0; > > bool sg_overloaded = 0, sg_overutilized = 0; > > + env->dst_core_idle = !sched_smt_active() || is_core_idle(env->dst_cpu); > > + > > do { > > struct sg_lb_stats *sgs = &tmp_sgs; > > int local_group; > > > This is kind of similar to what ASYM_PACKING would have done at MC domain with > equal CPU capacities. i.e pull the load if the core is idle. I think that's right, semantically "only pull a misfit to dst_cpu if its core is idle" is essentially the same heuristics that SD_ASYM_PACKING ends up doing at MC: prefer destinations on cores that can actually deliver their nominal capacity. With equal per-CPU priorities the asym_packing path collapses to "prefer the idle core", which is essentially what this patch enforces for the misfit case. > > In your table in the cover-letter, if you do "NO ASYM + SIS_UTIL + ASYM_PACKING (at MC)" > does it achieve close to "ASYM + SMT + SIS_UTIL"? Christian already explored the "NO ASYM_CPUCAPACITY + SD_ASYM_PACKING" idea (https://lore.kernel.org/all/20260325181314.3875909-1-christian.loehle@arm.com). I gave it a spin on Vera at the time. Summarizing the numbers I reported on that thread (all vs. baseline = default SD_ASYM_CPUCAPACITY, no SMT awareness, on my CPU-bound workload): - SD_ASYM_PACKING at MC (Christian's RFC): ~1.5x speedup - equalize capacities within +/-5% (NO_ASYM): ~1.6x speedup - SMT-aware SD_ASYM_CPUCAPACITY (PATCH 3/5): ~1.7x speedup So SD_ASYM_PACKING seems to help, but not as much as NO_ASYM baseline (even if it's pretty close) or this series. I think the structural reason is that ASYM_PACKING at MC only fixes destination selection in load balance, it doesn't change select_idle_capacity() / asym_fits_cpu() on the wakeup path, where I think most of the placement decisions actually happen in this case. Thanks, -Andrea