From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from MW6PR02CU001.outbound.protection.outlook.com (mail-westus2azon11012009.outbound.protection.outlook.com [52.101.48.9]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F3D1075809 for ; Sat, 9 May 2026 18:03:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.48.9 ARC-Seal:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778349801; cv=fail; b=FeV+otReaf61Myw7GAsvCu4bkw1KzxOHti5BlP2079H2KNmEWzP0JUQbIWkibgRZji2XE5V6zR9d6B0mv3E+K7gE+h5owj3DY0r4QvPMpcxVUrMxNHmMW/Q8bqDRRePX0h+Lda2c/k7QP0g7GQhvG9j61NIOuyYRFIvpjmttE4o= ARC-Message-Signature:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778349801; c=relaxed/simple; bh=NdUoCOMIo85ungML7yHXVApozm7tqzVyq7RkrBBWKp4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=jzDlMGkRvE9dBCf0dzZIkIQljJ9pwfgSbfAf3UfQOjfO10Vi2lJzo70wkOVMoULdecnAtN0hnC7wlHFjo2G7w4MZxK3mzJw0MyjaDiGrpo0y4Y3ftnCoSMgKShyStsT6NYRd3ydRWRzNR9rDLy+tMHaFxgBuqfzXhlvrJu5+JWU= ARC-Authentication-Results:i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=tPnXjsQ2; arc=fail smtp.client-ip=52.101.48.9 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="tPnXjsQ2" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=lrUwB1HIPq4H0WyWrvbPQ0X+5Dn8iPaLiCfYUYEQBSyMMtrDPr6rJMCqdTxSUjLJzjjmaLY53ZSRYs5h+B+P/066j40iDBQqtk+rSUBWraxsYcFJ3BqUFNkDaEr8+JdEaYokd9X+3b/fcSmcoXrNzsd0MqSyPD09fbxKe8sd73WIaFBw32D4OauTq989eTq6DhfIRY5KIPbG2k5Bp1HyEbaFOKokXeHfqAH8WV/5xzGrhLKqTqydtMBFngBimjFv6ePMjPF0NN6Y9DUK374yNO0zS6kq1Nx4RDyRp7vGdIQpTw6Llr89t8j+8hqzyDOetQuAjretEDndzIggXA8PNQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=C22u1z1KvbBxI2BfQK3lulG8ioKyClZYVSYhkp6ntXs=; b=W8ME2PGsJ+xBb0HJgOwzndfdUDsq9/VAIE4Hi0QSCyskmfHkKHoiSfewyjnWxbFJ6GcQ0PAyz1Bx6SiDfng5vHsFIg1qdqovbTQrjp0lAedByPJJz52Ah3zZhI6huehLtjc4cOJilpgieJnc67pl+lVufrmi/BIXYjf/7k9HBBkR+t+ourZF3WadLsNytnU75jU0aGkio+3CPcDAwZBvzNNlxv+s2m666LxWFRxjDMA5qWnGXFwznIGQVsFGHk9n+WxFdzE54d7fekdq2REzQORJ4HdaFgtmBUkRA5Hb3jw8/6xl8LWk1xi8ACLSnofcxsHKzGpQsNxdRM+VGqX/nQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=C22u1z1KvbBxI2BfQK3lulG8ioKyClZYVSYhkp6ntXs=; b=tPnXjsQ2H5m6YNPPAMgBt1UL7toG1A5nEW3HcQv2KVctQDHk4zHsb3/qZyGfxfwhaSGJmfonJ/EZuIMVyjOYxv1IcItOYiJPIZJ2G0CAGwllxZay5VlXQf7RJLUgpDHQ5/wzvJpYdjd4qXpn4CREtc1vkH8mFyW064mupbUTyXsnT3PoQ1TWjR7JuD3IPZbrDJMB2hfZskGgstnXXo52a+7U/QlaJJwO3FtXdR7qOzaIU1/dyeTHK1t2DqDK9Y6XzfDm1Btincqxllm4YPLcwQJkT8tGd02Tej+QRojqBkPbpwfRQmWugdawPcRhSlBw+qLu3D6/WMbJ5XBkbmJlQg== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from LV8PR12MB9620.namprd12.prod.outlook.com (2603:10b6:408:2a1::19) by DM4PR12MB6039.namprd12.prod.outlook.com (2603:10b6:8:aa::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9891.21; Sat, 9 May 2026 18:03:14 +0000 Received: from LV8PR12MB9620.namprd12.prod.outlook.com ([fe80::299d:f5e0:3550:1528]) by LV8PR12MB9620.namprd12.prod.outlook.com ([fe80::299d:f5e0:3550:1528%5]) with mapi id 15.20.9891.019; Sat, 9 May 2026 18:03:14 +0000 From: Andrea Righi To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot Cc: Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider , K Prateek Nayak , Christian Loehle , Phil Auld , Koba Ko , Felix Abecassis , Balbir Singh , Joel Fernandes , Shrikanth Hegde , linux-kernel@vger.kernel.org Subject: [PATCH 1/5] sched/fair: Drop redundant RCU read lock in NOHZ kick path Date: Sat, 9 May 2026 20:01:21 +0200 Message-ID: <20260509180302.1839122-2-arighi@nvidia.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260509180302.1839122-1-arighi@nvidia.com> References: <20260509180302.1839122-1-arighi@nvidia.com> Content-Transfer-Encoding: 8bit Content-Type: text/plain X-ClientProxiedBy: MI2PEPF00000B77.ITAP293.PROD.OUTLOOK.COM (2603:10a6:298:1::40b) To LV8PR12MB9620.namprd12.prod.outlook.com (2603:10b6:408:2a1::19) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: LV8PR12MB9620:EE_|DM4PR12MB6039:EE_ X-MS-Office365-Filtering-Correlation-Id: 2dff2da2-357d-429d-e05a-08deadf53d9a X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|1800799024|7416014|376014|22082099003|56012099003|18002099003; X-Microsoft-Antispam-Message-Info: QjOpOxtAz/IbCczkkdo6j59ZSg3y1dLbWMQ77LXAn2AeMqMtFIy1xK8c5haCoGE6D7o3nNgO4EdeBxIwfAB1vl70EN41uR8hHTvezduzNUJuYryUCWajx0elYd5MW/r8CoD/SVOTAD8JcwpUbbCYl/GT5n1WtiUrR1JDM1w/9ow3+VCjW9XUkqSaumDwiwPTjad/CY0Z/C6TX2SGrklFARIedm/C8VgAvDDFEE67UW5gE1W7lMKChhLTk3/s8KX6bg+UJnK6jJ9gcL3Guz4R8WVYzELFPv7ht2hTYo+qm6nnfha8EDirdQvgJZ4np0yo26z5Yyei3a7Yrslh+zeskTe+zFCivOHE8nSOY8/wNbat3QzchDqU9ep5IzOzOPrCDQXL65YeruRnbdAWJ0VPAdxm2/3KZeRql4kDISPvsDX9MtGaM2jPUB3ou0PLp1oMEIpJZZI9NSVM7gB93MrX+JMrwu3R+4nDQtwkT/1FknaBaoWCiy88cjwGIYfx72nPLxAkKZFCCOfe1zofvbg4ASjAILw+wXaCenwbepFRLlZfBvw83JKlPtTJrHPNKnf/al/dCBSmpZQA3yDj9A73G/8Mr7QOS9AfdF6vZF329fCWigPP45+XDSVuWNQuzkYEpwOpPq80y3YZ2IDvt7+Fh7/hNh3l8t3Ec0hOwRK6tZV8d+i8L1ukZHwA2Dljj3cN X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:LV8PR12MB9620.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(1800799024)(7416014)(376014)(22082099003)(56012099003)(18002099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?2eNDIfVeHnPHcu6fnPiwIVHfCPv9NW+KaAjzRqnzvoEISWfyivDqa/+nT6nA?= =?us-ascii?Q?oHkXHbAy6GGIx1wptooAY+H+7bfzl9GLLG68bhBBfYx4tVqJF05mkSbYkthD?= =?us-ascii?Q?TPQuTTYm/LAFcZqSvbPEc4AjmVeLrPTEvumI+Ko18XxXz1jJwYFK/JvjrS9l?= =?us-ascii?Q?6tKsl2Y8JjllKLuLQxHiA/uGLAeGGy/IyTBOps+08UGOsKRm8TNktShgwb9u?= =?us-ascii?Q?xAsXh+yQINtsakJWGLKQSihNjmTvSLBa+jGDpIjVoJ+1Jr5hjzweWSnACoAk?= =?us-ascii?Q?H+/LW5T1dJ7OqsyNL051VZRHh/rCjgsOmAwxIby1CmVJJFtZhPjnRq74ghgx?= =?us-ascii?Q?WonQ92KVniDlONHmPmKwIYvkBAL4wYfiI81fveRwqDrsFcePXzLVgw6qv66C?= =?us-ascii?Q?1cYocbLUytOhU+1HR5Vfq3HTwnGrIThflA1D777+zMe0/9xAi1bHIPlYkFUV?= =?us-ascii?Q?djA7EFYOrDOHHIcVWTF551+wo9Ob8Qo2PJro3FtGNauT7pHl2+UcDNbW9YGR?= =?us-ascii?Q?6nZ2xVH5gsv5zPWGDNdUM7loahaEgXEfpAgt3W2OeHrCsW1ebq/pFq4tmUzW?= =?us-ascii?Q?vUCv7ShqjFetrouX0/Ae1pXtbD7gvHkrB3IzdrRaB8cTZ0agOBXcdgtkiE5l?= =?us-ascii?Q?I/SwhbxthhnRlMlFiSpfNKUxeNiCljl0vI2gcLLjma67wdjh2OaStVsKZDjZ?= =?us-ascii?Q?oWSvcQDnB3dCdE9PFPeusuWqnbUQa5tLSMdVAdeTD29cbrynl0JoyrSxMaCF?= =?us-ascii?Q?luHKwwA0BtRhVrAquCIBd2OJnfQbxy9YsugaSFrasK4wHO/9HbonQT3HReGK?= =?us-ascii?Q?ue3djBuDrYKsgMxV+fsSWix19kIbQ/GN2gWKAqR1kNZHVdzURYFBC2zqwIMl?= =?us-ascii?Q?l6QBKXlsMDHHHOPqL35lLuiugLUuzAgzzlzxIzWnhV//hwRsZV0bIRbcNu56?= =?us-ascii?Q?O9JcsmAmigVyV8KfD61Dm1nubuXfM1odazbJ9bCpuWZBe51eCsnXpdOaA69p?= =?us-ascii?Q?5hzIVp5sCBW9uZZa6PSnQtYima1YW9mVST0cR+7Z9vK1pj6NYHbmAJtpKHZo?= =?us-ascii?Q?rMffFxan7Ni4FCmd7x9M9dBFUkYYUQPEZqO+k6L0rXpBOw6hz81pg7jd/Q3M?= =?us-ascii?Q?8Vh7sjni3YVcs+2CucYEWSXekCDVdaURQtjmp3BQcAJ3i9Emt6WmJLX/LUvg?= =?us-ascii?Q?bi+UIyjAQN7N3RtMvcpHIeAVFzXSfMFAOfVsmlFzS2wAKG3u1RslTt5hmyk0?= =?us-ascii?Q?reo8Yda5lgbljXrmU4cG7iijJsnc2R0Elahw6CSB0b/4/pstz8SDvuDUFH5m?= =?us-ascii?Q?opJRxrWgkXLTJ6q1m1poj1HWhZqO20TU/T4YUolJlw1XJdZ+iQY+hrr8QZ/+?= =?us-ascii?Q?7AV8YYvwmNqUASLAjz9CRRVV5Zfj1y6qCbRSuAzthumzmvA4lW8ccb9zq+jy?= =?us-ascii?Q?vaL1kIasJckjtvCJFbE+tUYWYvWHJr8EvzeLefxJ7qttcg8oRj15NwZ4ewIL?= =?us-ascii?Q?9CK0XKdf+5bclvJfJW6XBT3ScR/YKQ0K9TviqMMtz/SKGAyz1hVmddWrvITL?= =?us-ascii?Q?RfYXn96IWGorMGZivqoJIA45zOi1hYAUqbMys2/a9YK6fYL42/XDYZuodrF+?= =?us-ascii?Q?58V8FU/hhBe/GKvZGQ3cUeznM3YPfD5v6rHK4H/z7LOrIrUaqR+Ut86doZkp?= =?us-ascii?Q?C3wPZcz/DmNP006qTi0QrmcI/NP4ujJpRu5Dd3EhEQe/NKNpMfU1Z7FjHs0m?= =?us-ascii?Q?uClKrwx1Fg=3D=3D?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 2dff2da2-357d-429d-e05a-08deadf53d9a X-MS-Exchange-CrossTenant-AuthSource: LV8PR12MB9620.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 May 2026 18:03:14.7759 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: QGoiYA9reO7DWgmPSDI6C3mqxi5u9C9EIijGktQUwlt3Rw/NZ3Uvc5PX+ow/Yest1NaHaHMutd7EADfmir/Xaw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB6039 nohz_balancer_kick() is reached from sched_balance_trigger(), which is called from sched_tick(). sched_tick() runs with IRQs disabled, so the additional rcu_read_lock/unlock() used around sched_domain accesses in this path is redundant. Rely on the existing IRQ-disabled context (and the rcu_dereference_all() checking) instead. The same applies to set_cpu_sd_state_idle(), called from the idle entry path with IRQs disabled, and to set_cpu_sd_state_busy(), reachable via nohz_balance_exit_idle() from two contexts: nohz_balancer_kick() (IRQs disabled, as above) and sched_cpu_deactivate() (the CPUHP_AP_ACTIVE teardown, which runs under cpus_write_lock(), so it cannot race with sched-domain rebuilds). In both cases the rcu_dereference_all() validation is sufficient. No functional change intended. Cc: Vincent Guittot Cc: Dietmar Eggemann Suggested-by: K Prateek Nayak Reviewed-by: K Prateek Nayak Signed-off-by: Andrea Righi --- kernel/sched/fair.c | 38 +++++++++++--------------------------- 1 file changed, 11 insertions(+), 27 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 3ebec186f9823..6b059ee80b631 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -12785,8 +12785,6 @@ static void nohz_balancer_kick(struct rq *rq) goto out; } - rcu_read_lock(); - sd = rcu_dereference_all(rq->sd); if (sd) { /* @@ -12794,8 +12792,8 @@ static void nohz_balancer_kick(struct rq *rq) * capacity, kick the ILB to see if there's a better CPU to run on: */ if (rq->cfs.h_nr_runnable >= 1 && check_cpu_capacity(rq, sd)) { - flags = NOHZ_STATS_KICK | NOHZ_BALANCE_KICK; - goto unlock; + flags |= NOHZ_STATS_KICK | NOHZ_BALANCE_KICK; + goto out; } } @@ -12811,8 +12809,8 @@ static void nohz_balancer_kick(struct rq *rq) */ for_each_cpu_and(i, sched_domain_span(sd), nohz.idle_cpus_mask) { if (sched_asym(sd, i, cpu)) { - flags = NOHZ_STATS_KICK | NOHZ_BALANCE_KICK; - goto unlock; + flags |= NOHZ_STATS_KICK | NOHZ_BALANCE_KICK; + goto out; } } } @@ -12823,10 +12821,8 @@ static void nohz_balancer_kick(struct rq *rq) * When ASYM_CPUCAPACITY; see if there's a higher capacity CPU * to run the misfit task on. */ - if (check_misfit_status(rq)) { - flags = NOHZ_STATS_KICK | NOHZ_BALANCE_KICK; - goto unlock; - } + if (check_misfit_status(rq)) + flags |= NOHZ_STATS_KICK | NOHZ_BALANCE_KICK; /* * For asymmetric systems, we do not want to nicely balance @@ -12835,7 +12831,7 @@ static void nohz_balancer_kick(struct rq *rq) * * Skip the LLC logic because it's not relevant in that case. */ - goto unlock; + goto out; } sds = rcu_dereference_all(per_cpu(sd_llc_shared, cpu)); @@ -12850,13 +12846,9 @@ static void nohz_balancer_kick(struct rq *rq) * like this LLC domain has tasks we could move. */ nr_busy = atomic_read(&sds->nr_busy_cpus); - if (nr_busy > 1) { - flags = NOHZ_STATS_KICK | NOHZ_BALANCE_KICK; - goto unlock; - } + if (nr_busy > 1) + flags |= NOHZ_STATS_KICK | NOHZ_BALANCE_KICK; } -unlock: - rcu_read_unlock(); out: if (READ_ONCE(nohz.needs_update)) flags |= NOHZ_NEXT_KICK; @@ -12868,17 +12860,13 @@ static void nohz_balancer_kick(struct rq *rq) static void set_cpu_sd_state_busy(int cpu) { struct sched_domain *sd; - - rcu_read_lock(); sd = rcu_dereference_all(per_cpu(sd_llc, cpu)); if (!sd || !sd->nohz_idle) - goto unlock; + return; sd->nohz_idle = 0; atomic_inc(&sd->shared->nr_busy_cpus); -unlock: - rcu_read_unlock(); } void nohz_balance_exit_idle(struct rq *rq) @@ -12897,17 +12885,13 @@ void nohz_balance_exit_idle(struct rq *rq) static void set_cpu_sd_state_idle(int cpu) { struct sched_domain *sd; - - rcu_read_lock(); sd = rcu_dereference_all(per_cpu(sd_llc, cpu)); if (!sd || sd->nohz_idle) - goto unlock; + return; sd->nohz_idle = 1; atomic_dec(&sd->shared->nr_busy_cpus); -unlock: - rcu_read_unlock(); } /* -- 2.54.0