From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from PH0PR06CU001.outbound.protection.outlook.com (mail-westus3azon11011023.outbound.protection.outlook.com [40.107.208.23]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0A2313D75C2 for ; Fri, 27 Mar 2026 09:44:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.208.23 ARC-Seal:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774604698; cv=fail; b=sz9xhygCDm52irEIIlzg2eVf9heTpWpCO87BXPn5y8XYsUVNfs1rEDJBju5CuVi6Vp+smdl4Kb0SmJf3/ReIVgRrme6noiHHuC+YJE3PLbNl8iscVHyUILQMWSSeiFKxPSXsWFuFtbTOSoA798fMHlgZhLFNq98028iB47MKaTQ= ARC-Message-Signature:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774604698; c=relaxed/simple; bh=DB5bvclxP+jYa+/3OIaV75mVpK/joqbNPp7q1/w08ck=; h=Date:From:To:Cc:Subject:Message-ID:References:Content-Type: Content-Disposition:In-Reply-To:MIME-Version; b=hBNGRbAPWG9k8Qio/ijqTxWS0yKe8whb5jprWfsdbqKhomAH2hIXXN6MZT4yHy7vsZ45akA8X2NfSvDDENEBb5NHLvsDkWmOLSdOeMFmco/pcpePK4OysBWevDGEAP2d4kqUZTThZHTz/erz1bd99lf/olWkgzDd+lyIBYjG5iM= ARC-Authentication-Results:i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=leK5irhg; arc=fail smtp.client-ip=40.107.208.23 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="leK5irhg" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=eP6jqlWkeeQKb6SQcKYO52lcWR32qI3zmtHGaqG9AM91g1JGA+3sG82ZB//NHF9enW2HceVtomfq6mtqGzSU5u8aejne0+1+wXHf5nhcaop14dYeXAiiVQW9MZWFFUJFifDqtW4KMgaQXiCLj7h7oFB2oUZy7wGiFV2pmDaVh5ZGAfIrVziFe9DcPah34va7LKCeF3O2C1QK0rPuTLoNLhaPG+z4uwdx6uCt4juHVHxH3h0YV8vzcvZK73XGPAvHR5CGu2LrnPlpzoAHs3GxPj37lLvu9BqNrTA6kWfuBtYtXoK76cx3w5UUHhCC7vpzB4QHwTlKuy1scRAX/dcabA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=BBBt3AqaBXgSHudDoytgZMTj/YsTfcYz3vV2zNUS3hg=; b=MCIYvOXQ0LhXhUukt1SqrDFtxc6bop/Xt2d//CL/4jKFBa1QZwRVwaQFgzRpTY+2z9sJhfTYqlwHQgQgjlTSvrNWuup9A2245J8pEZfrovHatuMuz+Gcub6dVUFdLhL0Hw9n1vFzT1EjQX395KQS/xQBKZLpk9LXtGkBO0KtFXlUXDQFgVXxymErcjwdK9RGhPrPKmO6Rv3mi8dLy3HAyY+GKo35/qGMajBJ3KzkCSyKGLjBhv/0ikyIiIaCkI8HnXQuBltqWdrQfN2wxomC9WdNeS8D1KSCuoi3ZBZdKPRZCaAgyCYv0qai9eIH82O7fThOkLbTro3iWFigI1MZIg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=BBBt3AqaBXgSHudDoytgZMTj/YsTfcYz3vV2zNUS3hg=; b=leK5irhggydy6xebT3sLN6x0TWfEHs1cGS3Hv+f6HUn0KFle6sQscliz3WZP1pqA0xt2pFjQjoIx2uvLj4r8qAmjEzCp1CLAxNYkV3vwzzOnSXJYxmLG7lgn5pYcKAnwqG+OWD2feOHgNqcnk4hK5stqZodWzFKwDJf3ge2CwmObGNne15M4USVYBmfhJqmOhouzyc3SbOs9GFXAJqy3vmwmtzqidtHkGdyHLSLIo04I9A7Mt0MRqdY5t0/LxcPqj3NI1UWlyUgYqaYDAqkyKb60R0Szuh8t56lAs4dnXEupdyqErpidtsTpU5S2GeG4gG+4N65j2UNXDTtx4b16Ig== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from LV8PR12MB9620.namprd12.prod.outlook.com (2603:10b6:408:2a1::19) by CH3PR12MB9731.namprd12.prod.outlook.com (2603:10b6:610:253::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9769.8; Fri, 27 Mar 2026 09:44:53 +0000 Received: from LV8PR12MB9620.namprd12.prod.outlook.com ([fe80::299d:f5e0:3550:1528]) by LV8PR12MB9620.namprd12.prod.outlook.com ([fe80::299d:f5e0:3550:1528%5]) with mapi id 15.20.9769.006; Fri, 27 Mar 2026 09:44:52 +0000 Date: Fri, 27 Mar 2026 10:44:42 +0100 From: Andrea Righi To: Vincent Guittot Cc: Ingo Molnar , Peter Zijlstra , Juri Lelli , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider , Christian Loehle , Koba Ko , Felix Abecassis , Balbir Singh , linux-kernel@vger.kernel.org Subject: Re: [PATCH 4/4] sched/fair: Prefer fully-idle SMT core for NOHZ idle load balancer Message-ID: References: <20260326151211.1862600-1-arighi@nvidia.com> <20260326151211.1862600-5-arighi@nvidia.com> Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-ClientProxiedBy: ZR2P278CA0025.CHEP278.PROD.OUTLOOK.COM (2603:10a6:910:46::15) To LV8PR12MB9620.namprd12.prod.outlook.com (2603:10b6:408:2a1::19) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: LV8PR12MB9620:EE_|CH3PR12MB9731:EE_ X-MS-Office365-Filtering-Correlation-Id: 9ff94599-709c-44e5-1e97-08de8be57ea7 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|376014|7416014|1800799024|56012099003|18002099003|22082099003; X-Microsoft-Antispam-Message-Info: rlJ4qtKUgEVFEdjadU2aRdyjiC2HWJQqQ3En8JIl0SDGWxIYkg19L5ibyF03ecvKJvx0li+edHlSd9DEF5ZrAsTM1n87IjUCz8gDwaxNdZxvsNr8eZFh+S2fYGdZ5NY5zJpk31grS3BcTAtBDnqHyOkuiUiNKXAZg0+74g/DszdzRgpA9MOTbuDNOBiF/655SV7UpOB2yz0byQ53YDI8hk3okDZZAfUHpCcLxYWBRkPfv4Xgwa5yhSahQ8japK6JqKYF9eP6ypv2XtdnuJgmO64YAWKdkRPJulMpwZoS0Nem19Va3QMaiiUqdjPicKcNDy8c3PoDC+3eCvfHhYENk5oXDjMX68yqIDZXGRvr3xOyJ3eyDFWu+Ot1DInS7gK6MrgSH2pi75Z360ELmx0Tb82NcpddJfhiClPFmExiQl6hFqT3YMGDeJZb5gjgU6GfrwSxLKz+7l0PEivCl7fdIxCLMIC7zKnb/g+aNZxpPKh7bgdbScPVgC1TFPvH07hDKQS6jaZaAlJonjDn1kL3CTkLCRJd34njKMuQeMFDCg/keJnL0975U68TcWWLRBLsRx6qElmmJiUffNEq9Of1nXU5zz0AZpPI+t2zelLG1lhTB/oIK/WcSuwRQtcFCYUp50hpYGwoZnW1Lz4U+k0wRvH7iSl6VSdtu4u9qLjy0DnDXi+JpdhvangI6Ct4PYQi6p/umN5T/OEU2PFE2PUKRfhQdQb8GySJmjcTNHRwzeY= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:LV8PR12MB9620.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(376014)(7416014)(1800799024)(56012099003)(18002099003)(22082099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?/J4wZ/eFpNZfHJetGz4lljP4EApz/x6OctCWSPHM4b4mU0VgdN+9Ii9tI5yF?= =?us-ascii?Q?gHIdfYygedVak88+kmUvyL69Fva0rrmytUfJf7gLlcKHfiVD0wwCG+qtTWUO?= =?us-ascii?Q?S+Q5bD4AWtgqSKCb1phuza3eSoPmliFH84FqqRvJTl7E8HKjVaNPYTB7hzUU?= =?us-ascii?Q?g/dzqQV6vE1WU+ca5hUvKrxp6KX0DQI3UZPZTPkXNiMUZBUXXNmtNAwPZv8U?= =?us-ascii?Q?nHxz2M7/yclh7FPxTEcz5YLQeu4vuEG8Db5eiZ67aJO5TGzBwHUcw74Qu99i?= =?us-ascii?Q?/kOMt0q1dMmsOOMkZJsWcgmVaDHqZhM7G05jaiK5MGr4Qtp2Hi+93JXvlhG4?= =?us-ascii?Q?3Bp/OQdmXKtWmj2RlTSc8ZxVkWISjO6pG2rIdwseFYL9vdjWHfNwJeC1t3QS?= =?us-ascii?Q?oCqpIU9gqCQDvDK5XtwyN/9/9vndEuGXO2kLDqG92E8G0h8I4TeJiwBIX/fy?= =?us-ascii?Q?T/MQtQ1yaxYepuq7Oni86RuiXvwwGPzIUCA/MVM2DeHPDhwoz69KKSFXKBq8?= =?us-ascii?Q?c5cXkBR0vjQqGkThJl2UAqzhdq45DOPkwrGnn66RzFhrpqBmhk0UdwQnPZCq?= =?us-ascii?Q?wV3EKgz0TFKBO1F3PEcotY9Gstspqli8BAGijXDeOQaUdWf07uEPizv1ZNFl?= =?us-ascii?Q?04aNpWO6xcKBEcIELZdlTHNc2E81DK23RwQxlJFLadre7YDARZz9VLRPtI/q?= =?us-ascii?Q?1vG73jeDDk1+A5EzfHg9QIhvTh95va8b9CqtK6sm45J//A3BCD4LUshgxqIn?= =?us-ascii?Q?Arpk0ZjI7sD1LSyyi3ZU2zwd7OCtFYgL4aEorAUKWukRnCTeOD1SlWh9Tpy9?= =?us-ascii?Q?9OFTdj7d2NvSIiByYIUmD+lLSLEH5y7jGYg9bBJHIKoCS6V+C0vV3w7gLoj7?= =?us-ascii?Q?Yc0N4QUz/yIzNiI/gZKBh+vhAUNAxRlIocCkZQH7UhzMWE16PhSrnkhCd0vL?= =?us-ascii?Q?SHwlGPzoyGqYp1VQKLhFWFIYBW4a60ybA3xrvRxbMYewFj7C25bcbbKNG2Le?= =?us-ascii?Q?OFXbSgPZsjde3ZS+Nh0pvufKJ7r2G7GwcnVElCP1Vy8770XcEmR+DLMonb01?= =?us-ascii?Q?igkPEg+57PiqXYuNiSQA0L16Hm+LLBZgU5C1NrPCl+N4fhrJvWRO7eYX7B1B?= =?us-ascii?Q?P7rcuIWT9eL9g8fGYUtWHqd4s1ajYteUt6dDx9m3RdtOVhd0QC14ymJypS1B?= =?us-ascii?Q?9d+s0vvrYbz3itDerr8zLtwrdM+bRkj5iSwgPqV3dm8GHc+FU7DIee776IEB?= =?us-ascii?Q?ynNMIdicE5AjXaNgD5U2pGL9bQhm3VpeHNHyTALHBp+uP7nmdWEIex+nJnj/?= =?us-ascii?Q?A+H8phlGZwuDyjAHhU4Kg+7uy0bAIAyxrWZVAK6txA2Yf34W/+AzhS8jrljq?= =?us-ascii?Q?pd01OR+lXH2IusnlnLiLuNwbZAy1PKlfZVNjQkGa47oXt6Fuq4aKZs4m+bVY?= =?us-ascii?Q?UYKX0ui8si2Q+x0a1t2drDKUJCVl/aHiOseeFSUSyIPUY2ZZYNZgojdGDwhf?= =?us-ascii?Q?d1dXwrh9BdNvEYMPxxbBD7NQnWqjAc1KcWxmFZSQs5eSjbhLCjst7DMxY/My?= =?us-ascii?Q?hOgmIR9SrB0kTjQjPkoLPAvKeMXnLl2Ol8AQYs8Bsb+bWMNDbJKDnvjQPkuH?= =?us-ascii?Q?123TbnRzEzRMimGvuXQoWy2erL1z8/DSgm5LFyltm3hh5g4zkctgjJFTsbHc?= =?us-ascii?Q?NbGVd7kYvAs6gR6YcPL9OhWMWeiAO+9H2xr8Qkjs0wndcr5KGjU9RzKiO4Px?= =?us-ascii?Q?WFbk0qrT3Q=3D=3D?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 9ff94599-709c-44e5-1e97-08de8be57ea7 X-MS-Exchange-CrossTenant-AuthSource: LV8PR12MB9620.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Mar 2026 09:44:52.8833 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: x7iJl9dsu7Bi4uNbUZSv1fpnVAAS2CLzXvuc6lTTTtcbNMuOQNoVl6+Wuz4p4EiCDheP6Z9TbLVGsbGeEmD0Cg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH3PR12MB9731 Hi Vincent, On Fri, Mar 27, 2026 at 09:45:56AM +0100, Vincent Guittot wrote: > On Thu, 26 Mar 2026 at 16:12, Andrea Righi wrote: > > > > When choosing which idle housekeeping CPU runs the idle load balancer, > > prefer one on a fully idle core if SMT is active, so balance can migrate > > work onto a CPU that still offers full effective capacity. Fall back to > > any idle candidate if none qualify. > > This one isn't straightforward for me. The ilb cpu will check all > other idle CPUs 1st and finish with itself so unless the next CPU in > the idle_cpus_mask is a sibling, this should not make a difference > > Did you see any perf diff ? I actually see a benefit, in particular, with the first patch applied I see a ~1.76x speedup, if I add this on top I get ~1.9x speedup vs baseline, which seems pretty consistent across runs (definitely not in error range). The intention with this change was to minimize SMT noise running the ILB code on a fully-idle core when possible, but I also didn't expect to see such big difference. I'll investigate more to better understand what's happening. > > > > > > Cc: Vincent Guittot > > Cc: Dietmar Eggemann > > Cc: Christian Loehle > > Cc: Koba Ko > > Reported-by: Felix Abecassis > > Signed-off-by: Andrea Righi > > --- > > kernel/sched/fair.c | 19 +++++++++++++++++-- > > 1 file changed, 17 insertions(+), 2 deletions(-) > > > > diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c > > index 593a89f688679..a1ee21f7b32f6 100644 > > --- a/kernel/sched/fair.c > > +++ b/kernel/sched/fair.c > > @@ -12733,11 +12733,15 @@ static inline int on_null_domain(struct rq *rq) > > * - When one of the busy CPUs notices that there may be an idle rebalancing > > * needed, they will kick the idle load balancer, which then does idle > > * load balancing for all the idle CPUs. > > + * > > + * - When SMT is active, prefer a CPU on a fully idle core as the ILB > > + * target, so that when it runs balance it becomes the destination CPU > > + * and can accept migrated tasks with full effective capacity. > > */ > > static inline int find_new_ilb(void) > > { > > const struct cpumask *hk_mask; > > - int ilb_cpu; > > + int ilb_cpu, fallback = -1; > > > > hk_mask = housekeeping_cpumask(HK_TYPE_KERNEL_NOISE); > > > > @@ -12746,11 +12750,22 @@ static inline int find_new_ilb(void) > > if (ilb_cpu == smp_processor_id()) > > continue; > > > > +#ifdef CONFIG_SCHED_SMT > > you can probably get rid of the CONFIG and put this special case below > sched_smt_active() Ah good point, will change this. > > > > + if (!idle_cpu(ilb_cpu)) > > + continue; > > + > > + if (fallback < 0) > > + fallback = ilb_cpu; > > + > > + if (!sched_smt_active() || is_core_idle(ilb_cpu)) > > + return ilb_cpu; > > +#else > > if (idle_cpu(ilb_cpu)) > > return ilb_cpu; > > +#endif > > } > > > > - return -1; > > + return fallback; > > } > > > > /* > > -- > > 2.53.0 > > Thanks, -Andrea