From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from CY7PR03CU001.outbound.protection.outlook.com (mail-westcentralusazon11010000.outbound.protection.outlook.com [40.93.198.0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id ADAAC332614 for ; Tue, 4 Aug 2026 12:17:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.93.198.0 ARC-Seal:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785845837; cv=fail; b=plYVST5dENgG5BrDOGgjz9xy51O2BQs8M3TMw1ySES64MKAsR1HAKFLNtUay/XXq2KIQefeMmnuah1kpkubh277k7n2ARKhsGIIJEohXYcg1iSRNXqOnherpATNhtas786WsZD4BnjOVbW9edyDfrqGytw/s9nvtdCvPZw7ZFr0= ARC-Message-Signature:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785845837; c=relaxed/simple; bh=av8ynKjgI6ZdYYWZRVAAu/A2Hpfy3BjOEOTjBESD7hs=; h=Date:From:To:Cc:Subject:Message-ID:References:Content-Type: Content-Disposition:In-Reply-To:MIME-Version; b=kxxz8vl9hFAOalIW05BdUBuMCtPS6Va1YV7LrggLl7OxQUBSmAZ5JAqaQK3pVbYdoX0JFSlbroeCpkFnsAAXqu7Tnfkb+Nqngk4NV6Ek7IqrlNTNnjX32Rjm1djR5wux9ehb2VjP5zHVPjkStrmPhaJAWcPk+6pXsjhVaZrRClQ= ARC-Authentication-Results:i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=VYtzmSgd; arc=fail smtp.client-ip=40.93.198.0 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="VYtzmSgd" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=f04I0GpRGsrgCTOV+6PN74L0Zw30HLUp4f880VZGU0+NO657ZlwgKnK74AWqVuxNPfRX9YgM6hwjsf7kCNUCo8cjNcZUhPlfRDObOGc2sZ2jTz8UMdBOGl0h5A/xWqNsSnpTgQIpAwB2IV+Iq1VFdqdDrYuMrrYIP/IbaQ8rRdmb0+RZvPFqthjVCr/QoTZ6pEP6LuMKw0BycYIPNGlf5vqNW7KGbTPqLdDhAwc8UeUa+2SSzFtLuwmioE0yFvPo7meKAHFCyZuWLeQx4rdtnEhASYH0KWmMGE2t0SRt7pOKCz7CgW07SLbLYRE90EjxijInyyasUasqw3cMKm7AVw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=jiJu0qNZkKQUh66r34b/lpwHMZFF/8lm8anYcXTgv/c=; b=BzeLINdUXGcIO+zKriiX4aw5nhs6dJYVYsBADwnZaESF+Fro6iyBrbVy+AsM85egR4X65oOJwmJyOFK/q88j7a9PMrmqtVF3bKyElrpoKPWdlIDS+XS3PD7/yZrjOk3yvM8xD5na1PYrMdPI5PgZnJbd35Vg0v4UIVekBp9/CmgiHDBKoJN1xSM7OrB+5KYbHwXj6qIZjm3Emv7i/3ERx92HWVmE6lW4OrmLi50HNOqOxPjV5mtWj22haQxpnek6GNYLM90ToW+/z93BS7rGmvMARHJv/jkWSM3/cO/KH5Vp2vPdFPtLaxiZTs8LyEK0+Dku4GyXUvoLQ37VRd6wzw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=jiJu0qNZkKQUh66r34b/lpwHMZFF/8lm8anYcXTgv/c=; b=VYtzmSgdr0qE3ZnWVOcwutGiyf2aH5uLdv+R/nulRciRofV4wUi1Gf9WRf9O41HoblsSCqF+/NuSoxyfKDQDvVLySIOEeYWZXAtZsBm/eWztn4JRH7OkzpUipXuFpR1apj8it3yhoEqu6J4mrEzrMlgnvhFzfYhs7yv25AskQdef0Gg722TP9Y5cgfeP0Y+UM8PcGFRY7OreOCwBrycEz4raVnFRaSwDkMXXBC7s6r849mMr30cBRP5gZwns0RYOf4en9ZpO9GRMSzNxiim/F5udR90dqnjofAZt9F1sKF/MC4dpuYahuAN0EVRPlq6WzTukSe+IOIM/AfvsJK0QKw== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from DM6PR12MB4827.namprd12.prod.outlook.com (2603:10b6:5:1d6::14) by DS0PR12MB8564.namprd12.prod.outlook.com (2603:10b6:8:167::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.292.16; Tue, 4 Aug 2026 12:17:08 +0000 Received: from DM6PR12MB4827.namprd12.prod.outlook.com ([fe80::6261:3040:864b:159c]) by DM6PR12MB4827.namprd12.prod.outlook.com ([fe80::6261:3040:864b:159c%5]) with mapi id 15.21.0270.016; Tue, 4 Aug 2026 12:17:08 +0000 Date: Tue, 4 Aug 2026 14:16:51 +0200 From: Andrea Righi To: Vincent Guittot Cc: K Prateek Nayak , Ingo Molnar , Peter Zijlstra , Juri Lelli , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider , Christian Loehle , Shrikanth Hegde , Phil Auld , Mete Durlu , linux-kernel@vger.kernel.org Subject: Re: [PATCH v3] sched/fair: Prefer fully idle cores for NOHZ balancing Message-ID: References: <20260731191957.3199642-1-arighi@nvidia.com> Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-ClientProxiedBy: MI3PEPF00004E9E.ITAP293.PROD.OUTLOOK.COM (2603:10a6:298:1::455) To DM6PR12MB4827.namprd12.prod.outlook.com (2603:10b6:5:1d6::14) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6PR12MB4827:EE_|DS0PR12MB8564:EE_ X-MS-Office365-Filtering-Correlation-Id: cb495f66-9ba5-403d-7d6b-08def2224c9c X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|23010399003|376014|7416014|366016|1800799024|10067099003|11063799006|56012099006|18002099003|22082099003|4143699003; X-Microsoft-Antispam-Message-Info: ncQ0wJpOzXdEZlVpQjNmMpavOkuP12jRlifamfNknLgXknBSFQXzJxIsnSrChFkKZ7nisFCOlpgUi6ODmVWflg2elsvgWDVTYpe7tt/lIqVWnGCALy42RdJgt1VMKSI3C9pMwDy/ab3ybpzSX9MGwE7husY15aOr/fEWLVPsMXlF2DrDZVjtqhSVHMh8k9+OgdD1frTM8ci6O4m60pyqtMjvDIigVz22wmPi2CknTFGNCh/p1sDgChyMbZyDRFXd17fHuW+WMxnnhw11+EODI1FoOMa4bCT0Jaua1JmZ6AKmvmzgi49ULh0ccgUfgOu/gQhdQNkZmEn0aFTmdlmwxmZF3zc/GwW2W5+NZ/PbyT9SfhXIKU/VVQDbqofOF0Qno8I7tufnAX8CBFgIeLFKkm5hM6J59Lw5r+UFZEH6qb+FgBng/rElrpem+adG3mefHlg5Ldj8yfWjRMhNiy/dmUj6qFQwTEf3U0XLjwCIQL2SFelNgLc4iBPIKWa0+6D/omtI+4BDXPqaqDLnU8QuMLXb7EVYy2CTa0Uf7D9154DWd/4pV+FKQWSQlDiJS5sHtYfbwqJLpRX0by367R903HT/Myjopx4442NUi0lJyDZZvYa861M8FV+ir7aZGxnNq6YCxDvbCVSMiaa6TbxSOcQm6qAp9erDdYz4BEH74JU= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM6PR12MB4827.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(23010399003)(376014)(7416014)(366016)(1800799024)(10067099003)(11063799006)(56012099006)(18002099003)(22082099003)(4143699003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?+/CsFTgUY0DTIsKnhr827DXiCSfMJKRmwhf3P1aNeiI4TW7tRdFkde5PZ9BB?= =?us-ascii?Q?vbTrBhiU33FeGOsKHzPMFtVU/789vbRvftmh8DP5i5yUsVsyBaZ7VAx7XfFF?= =?us-ascii?Q?xPAl7Nu7qH7vd0v+rIOi7199h9qtBOd3zqrTmHXqSPDg4CxvPzE3CWtv4zaX?= =?us-ascii?Q?LTwPG2xoh51yxjXuAxj1hjBGnF8YHSD8WMHp99JblfaGZB+ZKigz2qWU6V6W?= =?us-ascii?Q?VvpOd9CIEc+vkbjMvMkkCYYeQgewsV9J55hBFPuabyOvvCc5nvqFcS/UYUNk?= =?us-ascii?Q?B57fuNTl/p3r93v8YsFDy7qI8MP/L0Swcind8kX57jUWGtrw6/3xG9TPDFhM?= =?us-ascii?Q?NmTylpn6hqxuzk2Ss4yNxjFOvKgKH8sTGW9LDE/GQ2ywPclqGdR+53vHZ/lc?= =?us-ascii?Q?eXelhMr9J59Gvitb1ckZ9gfbeLpwdIIbLn9LfSaiMgDvx6S+1xNDZTO/K+vI?= =?us-ascii?Q?ZBoQ7w3GtZ05EWMjnd5TM8K3sNdL+5zNZz+dgNg4LqTux9D0DI6YeuQST27w?= =?us-ascii?Q?olmin5QKHett8NuGZ22gQgxA3ZkQNi7ZiLSeJMJUWb29dD7A73an2R4xNEtU?= =?us-ascii?Q?DsWt8PtPuIf2caAdLk6TxW3ZoL3Esc25Dyc3NyKTqDhSNOa19Wwdjm6ZiWUd?= =?us-ascii?Q?CPzTcZZ7c/FTtuaiPItfzPNb+eJ6fR1dVCcXeS8Px7ZjR3v/LabYOOLFPrUw?= =?us-ascii?Q?j81Zf+LF+X+aQ/UlYyeZxAN+k25yQXdvPumtUnqexBeRvQPksa+yGbBJXwVb?= =?us-ascii?Q?MLgTJXNU5pwE57V/cY/0mzEG5VQh8bUe5PNTxlG32B/Gt/0RDMHueStLjL+h?= =?us-ascii?Q?WZ6dOJYI+/orryOnmvx9XrttPUcG13CR9yiBw9Mt5bwf3ClEi2XXjOfxPJXw?= =?us-ascii?Q?mNFuAdjGvg6ikb+VgJKriVrnT6I98LmRh+NSM8Kv8F2wQF5Ib/kPt9NjegFa?= =?us-ascii?Q?o2n6mV7Iy7bqKkOIlzFJYhEF1lmbCNIGiSaEYDfO9BCgqCAmpiOtroPnd3ez?= =?us-ascii?Q?sra5zlExBmjS4QAkYYFBgUMZMutGqitSrNSfC47Xz/uKD6OqhWvEJQf9V22v?= =?us-ascii?Q?X6NktCRBhld2AGy0sEP8h8NAlchUjDWVv1mRDnlapbzBLCfq3vQjEzsxItia?= =?us-ascii?Q?ygOsMkaesQBZG0T5vmkAjfml5dy/ktlPsUclg5m+JblBSaM9IAyzL4c1Pn05?= =?us-ascii?Q?fna7fNWIFERyrXXSPbrX74spvaJgvpKu1BWfvYzPJNGOxRUPJ2D9I1t4+Ow8?= =?us-ascii?Q?vGmtSo6QhbINNtAM2/05Cv4B/zgY1j1qt3fjTWU9FoeLXnKj8sIDc5LrS+qh?= =?us-ascii?Q?fFIQGhZYUzlSl1vHHSpueGditfNUcK7w9EMd8RLl4MbtjHWGoSAeiS9AgI6V?= =?us-ascii?Q?YQdftOdgChctGjrcStNMa2cRqqU26NmTNiHGmJDLnhVpr0SmST/2hgutnDDl?= =?us-ascii?Q?QOleejppC8dfCAtFoT95GF6CXNMTGGfTCSARIvUFPvQ+lTiyShejZxGWOjUu?= =?us-ascii?Q?LA7LLrRzamA70YDOaynUNrENHSXFt19dQ5jkxN0+PqJiJQzgtBlEORd0HQOB?= =?us-ascii?Q?afVsdns3WaVf/EwwZSMgpokOWLCkc/EAajVohz78i9Pxj4KdJ+y3Udt1t2uP?= =?us-ascii?Q?0wM/JjIKTvMAgfz9wOsKGdV8MBdPaQC9pMrMhBc5Iu8W/tQ8JIkdCEnZN920?= =?us-ascii?Q?wYdjB/NPH3hOqQGp4QCeszwOV+eN1gDatoc0a6iGPaOcYJlE8s19E8i0edyN?= =?us-ascii?Q?yPvhUpLwiA=3D=3D?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: cb495f66-9ba5-403d-7d6b-08def2224c9c X-MS-Exchange-CrossTenant-AuthSource: DM6PR12MB4827.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 04 Aug 2026 12:17:08.2354 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: rdtEX5bb52UCID4uOoS5nWv5QLFH4zo0AO5zvkaDRr7pGsYnWGo79quw/wJlLfA7D8OcvHgZMJFqpEAOVkPzXQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR12MB8564 Hi Vincent and Prateek, On Tue, Aug 04, 2026 at 12:30:22PM +0200, Vincent Guittot wrote: > On Tue, 4 Aug 2026 at 11:49, K Prateek Nayak wrote: > > > > Hello Vincent, > > > > On 8/4/2026 2:12 PM, Vincent Guittot wrote: > > >> diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c > > >> index 37001c63452e5..574b6b3ee922a 100644 > > >> --- a/kernel/sched/fair.c > > >> +++ b/kernel/sched/fair.c > > >> @@ -13965,28 +13965,66 @@ static inline int on_null_domain(struct rq *rq) > > >> static inline int find_new_ilb(void) > > >> { > > >> int this_cpu = smp_processor_id(); > > >> - const struct cpumask *hk_mask; > > >> - int ilb_cpu; > > >> + struct cpumask *ilb_cpus; > > >> + int ilb_cpu, fallback = -1; > > >> + > > >> + lockdep_assert_irqs_disabled(); > > >> > > >> - hk_mask = housekeeping_cpumask(HK_TYPE_KERNEL_NOISE); > > >> + /* > > >> + * Reuse the per-CPU select_rq_mask, which is protected from concurrent > > >> + * use on this CPU by having interrupts disabled. > > >> + */ > > >> + ilb_cpus = this_cpu_cpumask_var_ptr(select_rq_mask); > > >> + cpumask_and(ilb_cpus, nohz.idle_cpus_mask, > > >> + housekeeping_cpumask(HK_TYPE_KERNEL_NOISE)); > > >> > > >> - for_each_cpu_and(ilb_cpu, nohz.idle_cpus_mask, hk_mask) { > > >> + for_each_cpu(ilb_cpu, ilb_cpus) { > > >> if (ilb_cpu == this_cpu) > > > > > > this_cpu is not idle so you can apply the same as below > > > > Dumb question: nohz_balancer_kick() already does a > > nohz_balance_exit_idle(rq) before trying to find the ilb_cpu via > > find_new_ilb() so is it even possible for this_cpu to be set on the > > nohz.idle_cpus_mask here? > > Yes that's a good point, the test is useless > > > > > Even on weakly ordered systems, reads and writes to the same location > > (nohz.idle_cpus_mask) cannot get reordered right? > > Yes Right, checking this_cpu is redundant, I'll remove it and send a v4. Thanks! -Andrea > > > > > > > > > Other than that looks good to me > > > > > >> continue; > > >> > > >> - if (idle_cpu(ilb_cpu)) > > >> - return ilb_cpu; > > >> + if (!idle_cpu(ilb_cpu)) { > > > > > > + if (ilb_cpu == this_cpu || !idle_cpu(ilb_cpu)) { > > > > Otherwise, this makes sense. > > > > -- > > Thanks and Regards, > > Prateek > >