From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from CY3PR05CU001.outbound.protection.outlook.com (mail-westcentralusazon11013070.outbound.protection.outlook.com [40.93.201.70]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0745B3FB7E5 for ; Wed, 29 Apr 2026 16:07:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.93.201.70 ARC-Seal:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777478886; cv=fail; b=Gzf6YYdiqxIfCobw443uKF0CekGSS45fDS9v4MRx1twwnqU07PVJb97AHlQxSWUPlK7RML/EZQGsRaNKME9ufC1ERr+prCywSZjFimdE0KMEpC9UxEqiDmByRFV51fM1sZkhd+EpMb/QLWewMARKxukJ2USqlzGkNt/xX+xEOuE= ARC-Message-Signature:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777478886; c=relaxed/simple; bh=V1bs59K9HUJXkDFE1yvgztFnY6iSUWMvGVN+MK+bAxw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=KsyYG750jfWtnb6Hi3h6uFGSJsHPG+D46AfVJjja6/jrBuXPpCjoAqdzEedrc+JRvjaCIfPulq3FwGIMaeGFwsyB8Pa/XkRZ5I5nfkbKxlNfrVPpmfN4Kl6sCIub2siVY06z1nBXwyxNPt6RJw+1GUXPXfQJER13s9GkQYGm5TU= ARC-Authentication-Results:i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=IcgRsMcu; arc=fail smtp.client-ip=40.93.201.70 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="IcgRsMcu" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=nsPDAl+Ucg86xB9FQyoyDRvKA9V77k1pbb2kbh7KxXDWC3DSNvNC0h4TBYvht+EUVvnjtQz5nVPiIhDA8e2Kl/Bdi4Nsyl/m9CH48UWp/pk2mmubEjNwQdQ2uIyfJt3JSrFGsByQTS30Xw7jg+WCmYZ7wxIsjFKUabMvIzOWkxcWcnkhvXPVqycy14zGjdDnCEIvZYdbQfDE8jdQelWThQhrua6pzN4QOfjks6komLLRCWayree/Gj+4T8Jr+sSSwREy0TK2dVPWZSQNu6fIEd06QDA2Sk/VlAvUBNvoW+0IoqOMPL7Lu9kQbbKgA+H2lpz3xS+xl0nQTMPym8SSag== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=hyGd6YQGebjUpX8ryq0fJ7Ki9SxwgCdiUN5z/LPzU6I=; b=g7bnNLlpgpr42iWQhzvo4Z84pJjai+hWptLW8BC83HMIHVgEOS2U5s+NjqhnwF32C9naPHBf+E4W1VCz9ZVWle8bbfJ2PsOe7AqBhtBLCkxxpV/Gnn/kkT2Mhbqo/8UqCgSJS3mbCwz0uvEiucj8lOqPNZe8ogQcQIOd2ZUJ5/fVPHa+dlWVuQZ5aFUpZcSvnH4t3m+dkGpljIdBrz0gslL07h2u/J/ptzaQrwCRr1S0RHNWi1nwnmC99qWSTQD8CnXTpvBN1DTlmk0mtQDDnSY6a7qHVYz+VfOKQUV3HQEDz9TGFiV6aFC/Gg+0GL93UddCQ+4CkMU046fQKDZ9FA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=hyGd6YQGebjUpX8ryq0fJ7Ki9SxwgCdiUN5z/LPzU6I=; b=IcgRsMcu3ecLv1Mr7cFym/tQeXQiwiPibWFHsFJ4JETX0M9OeR22UMckdfckoS+sc8+wu+ddMF9+NfjgauODVCFmqDNNJ4sb4S091479qzx9pMdxYCkE969JlUIGjh1Epgsm5OCd5QCzfUT+Y7yqB0d1jPrGMdYWjVrbdjqcKizQDSv+sM/gKlalCaCdFWBnXkq6NsP0RakCGbGoq+2k5epd43yK9buTnEWwxtDKW5kuurAl83ERMyBc3v6suFX+vSon+CWe+sfusetb7231X5n0Arbg8j0WP30sJbYmba61gSoMz23WRY+qdAzmkD+fbXVx9KlwN5S74ISYRsYtfQ== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from LV8PR12MB9620.namprd12.prod.outlook.com (2603:10b6:408:2a1::19) by CY5PR12MB6323.namprd12.prod.outlook.com (2603:10b6:930:20::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9870.18; Wed, 29 Apr 2026 16:07:50 +0000 Received: from LV8PR12MB9620.namprd12.prod.outlook.com ([fe80::299d:f5e0:3550:1528]) by LV8PR12MB9620.namprd12.prod.outlook.com ([fe80::299d:f5e0:3550:1528%5]) with mapi id 15.20.9870.013; Wed, 29 Apr 2026 16:07:50 +0000 From: Andrea Righi To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot Cc: Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider , K Prateek Nayak , Christian Loehle , Koba Ko , Felix Abecassis , Balbir Singh , Joel Fernandes , Shrikanth Hegde , linux-kernel@vger.kernel.org Subject: [PATCH v2 1/5] sched/fair: Drop redundant RCU read lock in NOHZ kick path Date: Wed, 29 Apr 2026 18:07:41 +0200 Message-ID: <20260429160741.3825316-1-arighi@nvidia.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <9212f983-913f-4a90-867a-90649e7612b7@amd.com> References: <9212f983-913f-4a90-867a-90649e7612b7@amd.com> Content-Transfer-Encoding: 8bit Content-Type: text/plain X-ClientProxiedBy: ZR0P278CA0050.CHEP278.PROD.OUTLOOK.COM (2603:10a6:910:1d::19) To LV8PR12MB9620.namprd12.prod.outlook.com (2603:10b6:408:2a1::19) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: LV8PR12MB9620:EE_|CY5PR12MB6323:EE_ X-MS-Office365-Filtering-Correlation-Id: 479d8834-2297-4047-9c9d-08dea60975f6 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|7416014|376014|366016|18002099003|56012099003|22082099003; X-Microsoft-Antispam-Message-Info: Yk9Yp1Xu+9mt08nmr5ArM5epLorLBjJKSbJsWQUC0u9e28lD8jzeVZJe+qx3k4TzttHzHFikMQf4RDaZovtBSfojxeSLOfE0Nndb0fpW0jxa1nzguVAEWPBktZgVviW6n/nnIwWdQO6QqkOopI/l7+9F3eGR5Ueni+evv5Uj+BA8aM9NNYiPlcM6iTQ4Nndxwra8YVlUij9ogHX52m6mvgo1kbDE8hGjnz/W/tR9zIbSlOGvKPUPgNqyJjM2+33P2f1aG2nw8Jn4w5nSwS3rVt8eJo93DYy1oIpLeJ0brqFNPGXuGDwMNxn71/3H/ZYVrtQ9Rj8pbNf8e5E462phs/WDftK2/CJ8i89emEkcmq8BvE0f8QCqH/M0FCXOkaglihgB1h1tIcJv6M4zLrKktEk42+mgi683Tm084zX4vjexVB9iq6UmfdRCi2TnUl2zNrvlY42riN1HaFbvxpPNliJm9HMOhagtcPS47MyOlIYC/L4jgw75lYFfKyENeSXTQwV3rtRoe/Sc2knl8UOPqHd+BsJ70hrqqo64PsO9jpEAw+RCNIyROtfRRIh3tDko1pCPNszjMA8HE44hCPD+wjPIhUbhJbTZwTO/b78QeYto4BeYXYhLZxxSvy/jCUHAPYFNw/oapy+Y1ec4825B97wtw7R7TcLmNxhJ0isCR5rz3rB0StJpfBrtnWMHm9CvDMpdbWL7p8FCCMLLFZKqsjBo2iu71AbS1K9rSuoCBFc= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:LV8PR12MB9620.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(7416014)(376014)(366016)(18002099003)(56012099003)(22082099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?5ccMFCc3Bi7LlEftXAYcBQnuLx01ePUj/tIyb2L4Ayx5kdhVtumDagTpJ/e2?= =?us-ascii?Q?bQMURoC2aoHvViozGxo10LNX4F3j0vJZmI2y4uMqpS2kA8/r2dTUM+ymo6yS?= =?us-ascii?Q?NIYtUZdHxaXDIzWeXgexlASOnw/NYqHd5nrAavgAZeYmWktm+NnTu9JfKKRk?= =?us-ascii?Q?G6vaRvfW+wC9nxaYD+xrchiNmsPvKR7Eq/XirIjxTWiScVbdq58U8j9qIFPY?= =?us-ascii?Q?KdkygCdZt16qY9EvWAsnAITn4NKP1EY+B8cUDmSFZv42RE9rSDreX/VWv0/0?= =?us-ascii?Q?fGTMRdnDFJbVC+uiUcBxksSRcb42eoJDDHyBhEFVN2YvCi41Q6oaI4PSCJM1?= =?us-ascii?Q?5YcW6xLb77UQjffU42Au+a3CCnXCpTYTQXME7Pizm5m04aIGhpjueu+UEmNd?= =?us-ascii?Q?K6SfwAOP+32rylaf+lse/wREjQ5f5MPkwIC7pe25Zz4hOqrHrH5qo+SZUUiK?= =?us-ascii?Q?edyfj+KSZMAra2TUF8gIZ64B+5ajKTFatZStTOhhJlQqFer+gNbnN2TS4XEX?= =?us-ascii?Q?M9tRlb5o9Z06mBJfyDUQixDthCvH25bDIxLIEyAbDVpqiEBNgvchBap5uOoS?= =?us-ascii?Q?6ehG+OZsZEbhYcSl0ehQkmcIspg472YDix6f+dP0u4+D/JlfNKkSJX1l2WW5?= =?us-ascii?Q?NeW+nOd9ISXJzPZdAUxkhRajISD1SSesigXzJtoPcZNe8Azx71MXXbD/CUZ/?= =?us-ascii?Q?4OUpB2udwmECgJPccq9+ILE2IScXc0KZDducROcvn8EEx3e7ik9NUkvdytsm?= =?us-ascii?Q?WrL/hZ+Wvb3HzUMm4R4tPDM1kWqlT4YxLIUnpI5wTyCzDObrygf8uOvciJ4u?= =?us-ascii?Q?PQU/U5bdv3V20qroeKlTls+GQ0o7n1l6OSG2QmA6rAIrxGurx8NmNdyw+G1i?= =?us-ascii?Q?zhKZRfB8Q9HNcD6CAuAkGZugKPI/Erp1p+4vUuGHp8pBBpAux+1DvXghDzHs?= =?us-ascii?Q?nlV5X1D60+aDXWDJOem85lzcUbklfP6MXRXw2qNEWUNGX52MFO7+ID4OFWg9?= =?us-ascii?Q?sc12iyC6T7TjiGWMxgXvAR3qJarCDTevlJFejuV8TvCl3Q4O3p9WTFQmwWoS?= =?us-ascii?Q?0xxz55zZduYWv8SP0vMbv4WyeDq3UQq0hWhLXJdUkrv/Af3VTdOrXtQyvyAz?= =?us-ascii?Q?o8ervStsuCqTbtjs3INsw70VrVkDauFG4ZN7OZX15WPc+GDRU08fLlBL+7G7?= =?us-ascii?Q?JsCKyBC/IQBh/1j8OHH2sAL/M7lXUVXr9cK+LpwuiBWQqrd/Hqaigth3vMLR?= =?us-ascii?Q?yQVSf4M9QgaaN9WBwqJQCf4M2Fkzyx3JxClSifbEzej6RPUpCDYJDnNjalKB?= =?us-ascii?Q?Y+XnFHvG+vDefxJE2VrR511Nx5dVh0J2FeONtyGnorxYKGSwW6mD+bgc6KFm?= =?us-ascii?Q?vQFW5v8Z+njf5/XwnMbdxJnH7HTp8nP9Aai8bxrGrnifyvIi5Yw/Jg8b7XRm?= =?us-ascii?Q?c7ONSqIpgpB+vebn0o9CEU7pkya9/XkuQNhaYzzj8IHyOLegJWmb/27Q/oKg?= =?us-ascii?Q?lYual6HvAZoqJcmxA8t9o+KD1GdaKkE+52/uWrByrma4R9SARHZFpfbxDd33?= =?us-ascii?Q?37scwmPP+zFdfHmZLYWasrgLSTYsAYvVF97mujMCvnpFYR393MsbPYylLtDV?= =?us-ascii?Q?p+G2Rv5tTHrDPD9vrX95uKUCmf5/g3DWAww/SJmta3NpXk1NQm1Xbp9Qr/l5?= =?us-ascii?Q?EPEsbAULvoKAtVa8o5GOCg36f5iGvY4K1kJB3+W6xrdJ+FHKWfZRW3BXQZzp?= =?us-ascii?Q?fH3GaW6xeA=3D=3D?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 479d8834-2297-4047-9c9d-08dea60975f6 X-MS-Exchange-CrossTenant-AuthSource: LV8PR12MB9620.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Apr 2026 16:07:50.0487 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: lDViYz6B77mr+7HILBTmYv95YZH388DKofA7ac5WnOSX9nj53oPapbobk9JyZznwjMYEbQPXpl196zNOF2DWXg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY5PR12MB6323 nohz_balancer_kick() is reached from sched_balance_trigger(), which is called from sched_tick(). sched_tick() runs with IRQs disabled, so the additional rcu_read_lock/unlock() used around sched_domain accesses in this path is redundant. Rely on the existing IRQ-disabled context (and the rcu_dereference_all() checking) instead. Note that the same applies to set_cpu_sd_state_idle(), which is called from the idle entry path after the IRQs have been disabled, making the rcu_dereference_all() check sufficient. No functional change intended. Suggested-by: K Prateek Nayak Reviewed-by: K Prateek Nayak Signed-off-by: Andrea Righi --- kernel/sched/fair.c | 40 ++++++++++++---------------------------- 1 file changed, 12 insertions(+), 28 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 69361c63353ad..e0f75dedc8456 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -12749,8 +12749,6 @@ static void nohz_balancer_kick(struct rq *rq) goto out; } - rcu_read_lock(); - sd = rcu_dereference_all(rq->sd); if (sd) { /* @@ -12758,8 +12756,8 @@ static void nohz_balancer_kick(struct rq *rq) * capacity, kick the ILB to see if there's a better CPU to run on: */ if (rq->cfs.h_nr_runnable >= 1 && check_cpu_capacity(rq, sd)) { - flags = NOHZ_STATS_KICK | NOHZ_BALANCE_KICK; - goto unlock; + flags |= NOHZ_STATS_KICK | NOHZ_BALANCE_KICK; + goto out; } } @@ -12775,8 +12773,8 @@ static void nohz_balancer_kick(struct rq *rq) */ for_each_cpu_and(i, sched_domain_span(sd), nohz.idle_cpus_mask) { if (sched_asym(sd, i, cpu)) { - flags = NOHZ_STATS_KICK | NOHZ_BALANCE_KICK; - goto unlock; + flags |= NOHZ_STATS_KICK | NOHZ_BALANCE_KICK; + goto out; } } } @@ -12787,10 +12785,8 @@ static void nohz_balancer_kick(struct rq *rq) * When ASYM_CPUCAPACITY; see if there's a higher capacity CPU * to run the misfit task on. */ - if (check_misfit_status(rq)) { - flags = NOHZ_STATS_KICK | NOHZ_BALANCE_KICK; - goto unlock; - } + if (check_misfit_status(rq)) + flags |= NOHZ_STATS_KICK | NOHZ_BALANCE_KICK; /* * For asymmetric systems, we do not want to nicely balance @@ -12799,10 +12795,10 @@ static void nohz_balancer_kick(struct rq *rq) * * Skip the LLC logic because it's not relevant in that case. */ - goto unlock; + goto out; } - sds = rcu_dereference_all(per_cpu(sd_llc_shared, cpu)); + sds = rcu_dereference_all(per_cpu(sd_balance_shared, cpu)); if (sds) { /* * If there is an imbalance between LLC domains (IOW we could @@ -12814,13 +12810,9 @@ static void nohz_balancer_kick(struct rq *rq) * like this LLC domain has tasks we could move. */ nr_busy = atomic_read(&sds->nr_busy_cpus); - if (nr_busy > 1) { - flags = NOHZ_STATS_KICK | NOHZ_BALANCE_KICK; - goto unlock; - } + if (nr_busy > 1) + flags |= NOHZ_STATS_KICK | NOHZ_BALANCE_KICK; } -unlock: - rcu_read_unlock(); out: if (READ_ONCE(nohz.needs_update)) flags |= NOHZ_NEXT_KICK; @@ -12832,17 +12824,13 @@ static void nohz_balancer_kick(struct rq *rq) static void set_cpu_sd_state_busy(int cpu) { struct sched_domain *sd; - - rcu_read_lock(); sd = rcu_dereference_all(per_cpu(sd_llc, cpu)); if (!sd || !sd->nohz_idle) - goto unlock; + return; sd->nohz_idle = 0; atomic_inc(&sd->shared->nr_busy_cpus); -unlock: - rcu_read_unlock(); } void nohz_balance_exit_idle(struct rq *rq) @@ -12861,17 +12849,13 @@ void nohz_balance_exit_idle(struct rq *rq) static void set_cpu_sd_state_idle(int cpu) { struct sched_domain *sd; - - rcu_read_lock(); sd = rcu_dereference_all(per_cpu(sd_llc, cpu)); if (!sd || sd->nohz_idle) - goto unlock; + return; sd->nohz_idle = 1; atomic_dec(&sd->shared->nr_busy_cpus); -unlock: - rcu_read_unlock(); } /* -- 2.54.0