From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from CY7PR03CU001.outbound.protection.outlook.com (mail-westcentralusazon11010019.outbound.protection.outlook.com [40.93.198.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CF6803822A1 for ; Sat, 9 May 2026 18:10:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.93.198.19 ARC-Seal:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778350210; cv=fail; b=OAPgOlBT0ibN+9wVNVcFaR/uZB4LFOizXGQb7CFXB2Ucrsbtdq+jKwJwqDhSl43N/AkwSxEmRpF7wND85wQL28spCIZPLnDtV4H1e8NscpKcjAmL5JWYjOu338AmFPwPZf8h9my/ze1AL8++wlygyjBckOsCtoHlXY+wY0CxL3s= ARC-Message-Signature:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778350210; c=relaxed/simple; bh=NdUoCOMIo85ungML7yHXVApozm7tqzVyq7RkrBBWKp4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=OtokFyN+WJvCe4sZ+YrBBUAo9vtFiJcfErzgBPBbevFC+oplCa/aVvky71hVHlNFa/2fvIQMGwKmndn95uTolh3orVePXbl3Ea8wsfrYxFHbwNUyIS7zB6SoHPwt7T+qQ3EWikrlKI2WsGjGFDWZDmqy5I/ls2npjiCY0cmc0bg= ARC-Authentication-Results:i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=uaY4JFMU; arc=fail smtp.client-ip=40.93.198.19 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="uaY4JFMU" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=kq97avuoI/Pf4ov93mcDSkbjrnhY1MfLeSO/6ewirdn7LjaUnWRiOI0SBKfviUezj/gFz5nlCyBEcga0Sq01hln7AgTRCoMQtxkCVwtu0On1H7pne4L2oqXwGbQjxly1dkoibXj7Dgpi9HOIj5Mm4t16qbY9EenC3ONWyw1iqS1h10CdbSF1Gh3ARsShGAXH98m2/yjGdzKZn25RqMyGnwYovLYxJdtCuwofUOZ5wEXeuYTv/W/HRfQYOvMCYmblUz40Ow+RWAFsFxFL6AnKJ+wMMVClUYxHLUjmuwzkfOK+Ej1VoLG5/F8bn7f2Ee+/nK/mdxX+n1T1j6OSA+znVg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=C22u1z1KvbBxI2BfQK3lulG8ioKyClZYVSYhkp6ntXs=; b=XhwPXPGgw2woj/2tFqpi5XiuKdzb26rRd29IjfBXCtpqxe6DtgTJ7a5EZ1Mz+PrnnXMyueq7vNXU/MLAP6raiT3n6qjt1uALuho03eYyO7DD1VevKZbQpk7ExPk/VD+afl19tvdBF00kWm+M4yj2Gt5oUSzjtAyMbGCXl4kYdUo3pShCMbKxZVJaxHNZO41/WwhNLl2Vyl76xPYEpSUhcMzx84SuA8xdKbLmB8a8/zNe4N5yYZczZmzogttNFbWsdzmvNigcobRClTLCdOAXa2YSSnlcBmVNAynBqNJny3erJuh5YethltMtlRE4GvssiJyimFQPEVI/ObGdKA0s8w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=C22u1z1KvbBxI2BfQK3lulG8ioKyClZYVSYhkp6ntXs=; b=uaY4JFMUX3HA16iXqXF6Bdfrb6hNE97fEGlIYZwxJT8W7V8LjA8xfmTnNho/dRtF23bVTHy6gqezEiAy7t7zt2pJ0kjMeCDvcCjJRdtyqk5p36DH7iXNxgPw9MQmsuenqdlzyyxjR3BzzedH/+oM8OOggOoHQ1fPzNhhuxI3YiDG+thU4c7hMdFj68Rwdc9TlTScGYNwd4WVrcfT1qOwZGuxJOLmC0lZbAetxaz1Ki9rAlCygH6wUkc2S81qI1gXEjGDpHYs5vjyUS1qn4ZG2gX7eWsjQeA0Nt3UsHZnlyJAPsz1boiXDWsKLQI20tEB3cHwLrx4chhndUCm++hSWw== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from LV8PR12MB9620.namprd12.prod.outlook.com (2603:10b6:408:2a1::19) by SA5PPF9BB0D8619.namprd12.prod.outlook.com (2603:10b6:80f:fc04::8d8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9870.25; Sat, 9 May 2026 18:10:04 +0000 Received: from LV8PR12MB9620.namprd12.prod.outlook.com ([fe80::299d:f5e0:3550:1528]) by LV8PR12MB9620.namprd12.prod.outlook.com ([fe80::299d:f5e0:3550:1528%5]) with mapi id 15.20.9891.019; Sat, 9 May 2026 18:10:04 +0000 From: Andrea Righi To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot Cc: Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider , K Prateek Nayak , Christian Loehle , Phil Auld , Koba Ko , Felix Abecassis , Balbir Singh , Joel Fernandes , Shrikanth Hegde , linux-kernel@vger.kernel.org Subject: [PATCH 1/5] sched/fair: Drop redundant RCU read lock in NOHZ kick path Date: Sat, 9 May 2026 20:07:25 +0200 Message-ID: <20260509180955.1840064-2-arighi@nvidia.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260509180955.1840064-1-arighi@nvidia.com> References: <20260509180955.1840064-1-arighi@nvidia.com> Content-Transfer-Encoding: 8bit Content-Type: text/plain X-ClientProxiedBy: MI2P293CA0007.ITAP293.PROD.OUTLOOK.COM (2603:10a6:290:45::7) To LV8PR12MB9620.namprd12.prod.outlook.com (2603:10b6:408:2a1::19) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: LV8PR12MB9620:EE_|SA5PPF9BB0D8619:EE_ X-MS-Office365-Filtering-Correlation-Id: dfdb5e47-5886-4191-996e-08deadf63186 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|376014|7416014|366016|56012099003|22082099003|18002099003; X-Microsoft-Antispam-Message-Info: p5SLUpZk8Hrji2PRpoPiU5ZEayYKVqpuYOXGYBVdboVTR1EOgcQe8bJF1iEN3WBYvMnGvjQJEFxao7Xa9FJX2JawtoPESqNC+ElMPZa2pL+tU0Od6T+F7QrXnk77sJdCaDAyTGgnnXhs09bzk9ArgecULL3sEODoVOuynI5Ej6dNpFm1TFzm9bdzUU2+JPyFywJVW4Qn+6mPEgpXhJCeJzdXDY5KCGKSGvAz3c0SWTTSWsRQt1/VQwrydU2c2iMbuchsVY9Jm5GUVe6ijl7KIp9YHnMeAJ/FjeJ9Ol+Kp0GtGOcQEE0MxhST9zRoPAHqpB6zLLu/G26cepYdRKUh7ECzk1bWfTI55Bn3QAnGM7S3uoYy4z5Y5vDKYuVnqbjvmW2XrdzAwLNjPFkxrKx3eAIKRG78l24OZNHGSv4AUCX7VTsBh4wnypQmSBiQnYH8aWX7YZc1qaOP2WmOO8XI+A+KtIaETr0QWVRHXCuTXIXugBnrKfb64ZY7IccFLz6Zsne18JdVeZIJ2MqnCxuAHiLcfgDzwlJ8+9nlQWQXp0i3MN4FcicVFxOnkINoTFC/2ziQDVwEEDijX/Mx5Fk09wqJxhFov5rFa8bqekpbh0szhx9vDqtvYZt2IkHl0L5W+bWmFgCTtHWxRQWzEDu34uRYk7e7KH/yMHwCrUtFdoqAKjdiN4h/F1slprjaZW1x X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:LV8PR12MB9620.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(376014)(7416014)(366016)(56012099003)(22082099003)(18002099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?M3Jsa0vLCvBUz978xLtND1xExqxAm7Kl/dvxkJcvlkC0dS2PgmLyxilQ8lq8?= =?us-ascii?Q?H15uPbYRESDPby9OY+r9iaFTdJrzcj7dllLsZA2CfLOnX7ZnPN/Ticz64WMl?= =?us-ascii?Q?UVvvSN0Q1sVgZeM+8HTyTkWAQoxKxAPIutsvqTxVJsnwYV4Q5XP26d4GKxSx?= =?us-ascii?Q?YoVoIg03DS7xXkFGowkXZkAGIrDDuJEeEtbHXFJslPSiDp/0ySA/0u80qute?= =?us-ascii?Q?09aocoHgwY6FpPoD98vFvaFct5/9rv4KVeeq7hlCnayy8viNqHRgVWk7XmWQ?= =?us-ascii?Q?NsG/qJdGfaM5x4GyUqK2jsoaQrUEPA5IqktswrNfxsx6893Lnvtx76m4CPbi?= =?us-ascii?Q?d7KeeBq9Wvqzz2BmOdmnyr8rk/fN2oEgMfsWSMzsP5OPul+xJpJv/fiorNP+?= =?us-ascii?Q?9i7mnjT9Xlh5By6lDR74EWHynQzMxtyR2mzC64ED6oLTsQOUkrd/8Wvy3CjJ?= =?us-ascii?Q?o/qX4+VC0UvGco8E1Kr/ewjDZFDRvc3EZu73hQ4mJFB16rTsCAlZe1sAHjyi?= =?us-ascii?Q?77lwKV3+B/ynPi4DmYUA2W2M2fc2NH9c4WybDNvxqx7BuoceyiYZECaRIRmo?= =?us-ascii?Q?dgHvW16oha30vKdyTiuT/yjGL7EWWQI7++bnOJOW8+MhPlEst2b6/zAL78ne?= =?us-ascii?Q?Qc8NOBDQYa9syYvsMhYayB3kJsMrLYhOisio2FTe5+Tyj5i4CC5iDkX/0bw+?= =?us-ascii?Q?mBMbYyMpek19z0rU41TktEewgjV9U/OQbzqc6Q7sadsZmEe0jW76PzQk9OkX?= =?us-ascii?Q?T819QuHzjc7+oYgL2V9fxP85YlHWnjlH2IOX2q8Va+9O0yWN6nMiMP+/LWGN?= =?us-ascii?Q?mKsz8F/KcCgpNlwa3DpRyYv8RMCFE5ryoagBRRTiZvqtlCYh9pimDRpQX17H?= =?us-ascii?Q?+Is9zILlE85UV49cRN5fQtbJ4E3j40M3BcNiyOh0ZdSIKLfEqODqN0tOyWsX?= =?us-ascii?Q?EyC7PeXY1F79zVIDKpSunsJEM1PBWRy27f+nqzI7+hkDNC1rNNwFtmyrleNQ?= =?us-ascii?Q?4oOKGh8KiPkNvkRUYBQIJdnpn1wmuHmDj9Ooh+tgsvCdBTMs62NON+tBlXQc?= =?us-ascii?Q?hko1H+WN3oZBbzs440CJ6s+LdbjBMI1Ww7QkzeTe3gs9Dl0w/HyIxvRH1Kne?= =?us-ascii?Q?frFdKPR5WIcg1DG7uS1IRXldc7Shvq3NKCcMjpW+qnfgZPSZ+VTdR8772Nzd?= =?us-ascii?Q?/hCqbCUD9tw0awjkY7fMp6HwLrYGbPPIVoZewKM3PydtIAs16RD1wsLoEi2Z?= =?us-ascii?Q?SHFgR8ALCflK/AbniYDDfVh/CYK8OvhrUtCCAQf5ll1lUdL3GZzhLLOp/5iV?= =?us-ascii?Q?bQO520a3eGaNQJTb4BUBe/j6unR2Cv7Q4glogKpbcRHWnoL+ew4kNngsVXop?= =?us-ascii?Q?m6MPrEuwHzFEX4iIenOQmMNErw2D3T1VRj/VwLvdp/lQPLd6kIcbIVDSciIP?= =?us-ascii?Q?xgJ/6O1f2OzN9zaaaczGC4+VNXOlDaUzd/D0sA6L5Wye0iy+dRjqzkzT1+qQ?= =?us-ascii?Q?KqKqOaqm8efSz/NwClSTaF59nGyoPcHD2iJeGhFPGRk3uLunG6LgqEUwky7m?= =?us-ascii?Q?X5JpEL5tNxVLjewjltaS2ihGTusPUzywrQMrpB8tb2brO2J37YwgT1l74xpq?= =?us-ascii?Q?gfpG5VtySIIw3UX8DwLEBK1knPObluJ7Yio/7hx6lcWKLRu3Pp0orZZpm1R2?= =?us-ascii?Q?3W/fVcvqua1H6qIBlJcKT7OCERAS+jGgiA80ARIH583GkEV84825OPtDtOvn?= =?us-ascii?Q?z/oPkm81Vg=3D=3D?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: dfdb5e47-5886-4191-996e-08deadf63186 X-MS-Exchange-CrossTenant-AuthSource: LV8PR12MB9620.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 May 2026 18:10:04.0924 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 01wHSX+GHOphLbgXH1tweHfKHa6JVs7OqwILmio07lZuAERsWv6XY4CxOpJxYzveklq2y5hJk7LRlmDA+RqKzQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA5PPF9BB0D8619 nohz_balancer_kick() is reached from sched_balance_trigger(), which is called from sched_tick(). sched_tick() runs with IRQs disabled, so the additional rcu_read_lock/unlock() used around sched_domain accesses in this path is redundant. Rely on the existing IRQ-disabled context (and the rcu_dereference_all() checking) instead. The same applies to set_cpu_sd_state_idle(), called from the idle entry path with IRQs disabled, and to set_cpu_sd_state_busy(), reachable via nohz_balance_exit_idle() from two contexts: nohz_balancer_kick() (IRQs disabled, as above) and sched_cpu_deactivate() (the CPUHP_AP_ACTIVE teardown, which runs under cpus_write_lock(), so it cannot race with sched-domain rebuilds). In both cases the rcu_dereference_all() validation is sufficient. No functional change intended. Cc: Vincent Guittot Cc: Dietmar Eggemann Suggested-by: K Prateek Nayak Reviewed-by: K Prateek Nayak Signed-off-by: Andrea Righi --- kernel/sched/fair.c | 38 +++++++++++--------------------------- 1 file changed, 11 insertions(+), 27 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 3ebec186f9823..6b059ee80b631 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -12785,8 +12785,6 @@ static void nohz_balancer_kick(struct rq *rq) goto out; } - rcu_read_lock(); - sd = rcu_dereference_all(rq->sd); if (sd) { /* @@ -12794,8 +12792,8 @@ static void nohz_balancer_kick(struct rq *rq) * capacity, kick the ILB to see if there's a better CPU to run on: */ if (rq->cfs.h_nr_runnable >= 1 && check_cpu_capacity(rq, sd)) { - flags = NOHZ_STATS_KICK | NOHZ_BALANCE_KICK; - goto unlock; + flags |= NOHZ_STATS_KICK | NOHZ_BALANCE_KICK; + goto out; } } @@ -12811,8 +12809,8 @@ static void nohz_balancer_kick(struct rq *rq) */ for_each_cpu_and(i, sched_domain_span(sd), nohz.idle_cpus_mask) { if (sched_asym(sd, i, cpu)) { - flags = NOHZ_STATS_KICK | NOHZ_BALANCE_KICK; - goto unlock; + flags |= NOHZ_STATS_KICK | NOHZ_BALANCE_KICK; + goto out; } } } @@ -12823,10 +12821,8 @@ static void nohz_balancer_kick(struct rq *rq) * When ASYM_CPUCAPACITY; see if there's a higher capacity CPU * to run the misfit task on. */ - if (check_misfit_status(rq)) { - flags = NOHZ_STATS_KICK | NOHZ_BALANCE_KICK; - goto unlock; - } + if (check_misfit_status(rq)) + flags |= NOHZ_STATS_KICK | NOHZ_BALANCE_KICK; /* * For asymmetric systems, we do not want to nicely balance @@ -12835,7 +12831,7 @@ static void nohz_balancer_kick(struct rq *rq) * * Skip the LLC logic because it's not relevant in that case. */ - goto unlock; + goto out; } sds = rcu_dereference_all(per_cpu(sd_llc_shared, cpu)); @@ -12850,13 +12846,9 @@ static void nohz_balancer_kick(struct rq *rq) * like this LLC domain has tasks we could move. */ nr_busy = atomic_read(&sds->nr_busy_cpus); - if (nr_busy > 1) { - flags = NOHZ_STATS_KICK | NOHZ_BALANCE_KICK; - goto unlock; - } + if (nr_busy > 1) + flags |= NOHZ_STATS_KICK | NOHZ_BALANCE_KICK; } -unlock: - rcu_read_unlock(); out: if (READ_ONCE(nohz.needs_update)) flags |= NOHZ_NEXT_KICK; @@ -12868,17 +12860,13 @@ static void nohz_balancer_kick(struct rq *rq) static void set_cpu_sd_state_busy(int cpu) { struct sched_domain *sd; - - rcu_read_lock(); sd = rcu_dereference_all(per_cpu(sd_llc, cpu)); if (!sd || !sd->nohz_idle) - goto unlock; + return; sd->nohz_idle = 0; atomic_inc(&sd->shared->nr_busy_cpus); -unlock: - rcu_read_unlock(); } void nohz_balance_exit_idle(struct rq *rq) @@ -12897,17 +12885,13 @@ void nohz_balance_exit_idle(struct rq *rq) static void set_cpu_sd_state_idle(int cpu) { struct sched_domain *sd; - - rcu_read_lock(); sd = rcu_dereference_all(per_cpu(sd_llc, cpu)); if (!sd || sd->nohz_idle) - goto unlock; + return; sd->nohz_idle = 1; atomic_dec(&sd->shared->nr_busy_cpus); -unlock: - rcu_read_unlock(); } /* -- 2.54.0