From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from SN4PR0501CU005.outbound.protection.outlook.com (mail-southcentralusazon11011038.outbound.protection.outlook.com [40.93.194.38]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 67CB232A3EC for ; Sat, 16 May 2026 05:45:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.93.194.38 ARC-Seal:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778910337; cv=fail; b=rrSXCk5T/jQQWMxrhgb1Eeso3MN9Ex13GHdZgRymLb2L2I/UOqH+ax0kFNLFA/uA+3Z/UMC2dux2TkyopT5Uch13ZdJCjZR82ba2ID/alikdKmNthla+vYmvR6YIVIQxBwmPL5QUE3VppGuEhl2YlM5FZ3djD5eyh530v5cTJKg= ARC-Message-Signature:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778910337; c=relaxed/simple; bh=jlbADCZgO5E4uC4TBq+ZF/yX2/nJSNBWmcEvzRFSfyA=; h=Date:From:To:Cc:Subject:Message-ID:References:Content-Type: Content-Disposition:In-Reply-To:MIME-Version; b=j9ILRq8JL4fl9k9m8oA1r6pQfr5WI2ab/qASZJGSIggHjQp/us5VHBeXV3W4kUTWQzZUkiSFqaI7zmlp7sLEI6BCziR/SFn4JsmYSLDGmvgNyT+OxXslTvZmU3kKI+gc1JBQ04Qww3L07G7t89xUhWvLc1Nll2kAaOYetnO4wU4= ARC-Authentication-Results:i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=iyZ8efD+; arc=fail smtp.client-ip=40.93.194.38 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="iyZ8efD+" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=fnGvgsGhPYZSljugNb4eq6PgM2L4ZjHh/7j4JnVZXh06VhrJd9j6ZrQLeegVTr8sKsSBFOPDKTkZi41XgOJslbWCXm/lSy0KrSZXg3JuDEQJ6AlpJcGZq6bbgLrh5u6Zwehf3WtcIMWRI0XmYrpsQvm4pRqi3/Q27k8LhXegB1i/VbkFvsSbAPrP42iRO7V0zwb006dJEzSFjrLtMewA3kKZVGKMIuvJVFKLVPi+siZEed3W9TnUws3hBTkB+9lhFn5HSPtPwTeQPf6WLyvfkBA1bOoPZQ3sEcQhZByGOKGtPRbpJ/isQOoci+8yiSzKiRw6rj8rWzUsZu0LhZPrkg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=JdolhwEA3t7YQe7po81QQ3izS7g44fQQ8pU3MmdmneA=; b=bH4NbGzVQ7pM/AGIGogW2A5gSKOMfyyJRExo6D/Hf919RtTvcOCKeQcYwlQj5R3ItG2vhYyynQcrvNpIESkTvBax0P9mJW6IMehAZSAMPI8aPtLo7zN2SuQjxqm2CBrYg7igAQBaRyiB9omgkS/AlRBctBLbwlkf7RrbPu07dAraGL+BK+8ptn83bdjKqpYkP/N+4fyLeLvi4hnHFC+A3FfW4/ZayL/G3nbeX6nTKYegiAQkbkifNzAB6VjJlktyuialqiMg8DvPJD4A7/B3Qz2fpmWoyl3deVURT/xOkuohkte/YlYEFyWkzNYrN2Z6Lnsu7dwbjNAfsoWEsa+hWg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=JdolhwEA3t7YQe7po81QQ3izS7g44fQQ8pU3MmdmneA=; b=iyZ8efD+ZXJW/HSM1fAzgHXjQNkzC6CHD+uRbCUdHmzLXbvm0ePlidSZn3j0QNIy+HQfg7HZK2bw1bEZ/2gxO6IWqWLPH34VhMIKuEiVds4emehdSVF/TWq0N+A6b/yssIDTOOkGpj+vA/FilnVIGAiRbp57DxlSh6GGK4QIEdKlKHDl/3gRTp7KsisjH0iPy85SJ+YFuwRazM5lcQntoin9Ei0AWIxqDJW/04CDNy2kdSKC4zmwqKi8OPXn3ClM4UlwJDTNjQ3VUV+GDkDfDj90XoXByHWZLGcjqjNjQGzTuyHHbY5yQvulZPowRfQo1FoCV9RzVagUHg3JpBEtig== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from DM6PR12MB4827.namprd12.prod.outlook.com (2603:10b6:5:1d6::14) by PH7PR12MB9103.namprd12.prod.outlook.com (2603:10b6:510:2f5::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.25.19; Sat, 16 May 2026 05:45:31 +0000 Received: from DM6PR12MB4827.namprd12.prod.outlook.com ([fe80::6261:3040:864b:159c]) by DM6PR12MB4827.namprd12.prod.outlook.com ([fe80::6261:3040:864b:159c%4]) with mapi id 15.20.9913.009; Sat, 16 May 2026 05:45:30 +0000 Date: Sat, 16 May 2026 07:45:21 +0200 From: Andrea Righi To: Shrikanth Hegde Cc: K Prateek Nayak , Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider , Christian Loehle , Phil Auld , Koba Ko , Felix Abecassis , Balbir Singh , Joel Fernandes , linux-kernel@vger.kernel.org Subject: Re: [PATCH 1/5] sched/fair: Drop redundant RCU read lock in NOHZ kick path Message-ID: References: <20260509180955.1840064-1-arighi@nvidia.com> <20260509180955.1840064-2-arighi@nvidia.com> <4b04aade-8474-4e37-991e-16f2faedaf0c@linux.ibm.com> Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4b04aade-8474-4e37-991e-16f2faedaf0c@linux.ibm.com> X-ClientProxiedBy: MI1P293CA0020.ITAP293.PROD.OUTLOOK.COM (2603:10a6:290:3::18) To DM6PR12MB4827.namprd12.prod.outlook.com (2603:10b6:5:1d6::14) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6PR12MB4827:EE_|PH7PR12MB9103:EE_ X-MS-Office365-Filtering-Correlation-Id: 8772a1ee-8ee0-401b-dd84-08deb30e570c X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|7416014|1800799024|376014|18002099003|22082099003|56012099003|11063799003|4143699003; X-Microsoft-Antispam-Message-Info: /NwNfd4b9oVEbEsZ5bWIgR1iKmRQGyiEt+FA4ojPid9gZ/KN1erFW7eSEE6OuHZ78hyA4UmG/zT5nk7u3Ttv9aSh4cNPm/GTYaJAdHFJ14WnX5XZ1GWl8H85VU81QifIhI0h8cTmz8h41pG19oCOKI3ENJ8R2PnTrrBKr6YyWoGacZBRqId7B019FbnwzIIf+VUohdrq0+/NI5uTDSN+b1eM3kWvkq89m/FePlPd7tMPmzfoYIObGi61Yws9kBAJIARtEWk3fKD5S3t/3/e0eWJtT+jtBFDOsfP7va42RXT297FVfdPCYzon+HNnPDbOiDjbVVrYVdO7L+OU4gCDEl6kYpi7vk5RjVzj5jnuIwsAV6EBBPteI/cHLc/H3bxTwWnVQ76iAoTr8qd/uJ+hlDnbhX6pYrc/RCRxQjSOinRYGdNtYyF8pqiWbUUorQ1HX09l8aFljcMvK9DS6IvqqriJORvzUM+i6TjhOq40nQVhyr0fO6wYuE90RHDY4jTb/+fOcQWjy/RSs1glZn9tJzJMyxgo6QnQrrBHp9YFUduXeavA6QTSpyiMmQDZ23Xp1+hyfvllwuVmoB5amZ/EXiApN+ZynCi0/si5Dli4PSldbRxRnFeZpfEaS8l3qD8J71g6DBB2nBu3B6T05NOsNPy4hElUc/U8InknT0BmhNCpQ0IzucqbyuCek/2EZ18b X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM6PR12MB4827.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(7416014)(1800799024)(376014)(18002099003)(22082099003)(56012099003)(11063799003)(4143699003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?srw1oCDU9ylzcOfaSOmclSi1t/zvUJWB/sQTfXmMfXFHvtWYSFH2RRbRn1G0?= =?us-ascii?Q?VFwazby+329/JA+RS7zZgpgx2ppJjrJeDYtq4gaKZIN0g+8NxVijjVrHcBXs?= =?us-ascii?Q?JGJ4jAvnc9mwMGH03rqmRYg0wxdUq9QKDMf83q6tSV8mdSOS/tStbVaxlfrD?= =?us-ascii?Q?coUAEhdAqBQ4e+IJ+hdPgaPgRuYWcZSXE0i4It1/kTEfxhkgCtMZkjvP/Hbw?= =?us-ascii?Q?1D+k97qcXVmHrTFJ8ImQ2yjiiIEuGF6YehnfukTFU7HhjWYlPp3mLqI5pggJ?= =?us-ascii?Q?+IOGHwEQTWa3xVb2EoroQ30Sb5qrlpKutwm5epWjdBlflBIpjegTWb1uoP0p?= =?us-ascii?Q?RZtEI2BtK4BjFYomnNTxRxUbjMddEncxphqk46zy2pFBjLU7FVXNhYaLL6Hh?= =?us-ascii?Q?l9D17hSdB8ZCv5v1/lB6dIkRvvzOGeWH+bkn/jXy/tG1kDUu1mQv708gtn89?= =?us-ascii?Q?V3ktqC5cQw79b356Ik3kIuaDDpypDOSq/agiad9udW910USWSAItP1a20dOS?= =?us-ascii?Q?vUdYmkwEiiuxEYd9cHWr6Z7HuZrskbveiGLgJHp5kQ3lki4B2D98YyuMiQtv?= =?us-ascii?Q?mI/7FHqT0GJfrwu/ku8XmjI6LcsuK29feBzaqbJ5DdoUZ6ZK2FQfv4BGRUG3?= =?us-ascii?Q?CivDxgGEr4LS1XaCZ79dcjxwoP8vdqTYbawboMf447TRGizGZsPO0UjSdueF?= =?us-ascii?Q?boyc0sph63QC9dZhJj7UXHfQSNeLWUdKpahW7jg0Vu0ZDEuVk+VV2or+W7Cs?= =?us-ascii?Q?DXOHadj64kk2dFE5UmjxcmLkqTjpX/iNMfRhL3913UMBXZFL14DMNBMeH8F5?= =?us-ascii?Q?hYH7uN2n8/bd02Q9uFjvdobKgGN6gzHIeflpURmIc5lAOT3wemAEZxUq2Oxv?= =?us-ascii?Q?9fmcF+1LI/SsSNM8zWaUBfmDZaDghRR106YTsM1qX3pL3yx28+6q3LEqtEDv?= =?us-ascii?Q?Umcqnd4zuWMCfkdr6OVB60hFnGK68n9jA9lThG+lo5oCKsvrHy8OduhFP0hH?= =?us-ascii?Q?foEgUn+mXkhWa+5T43njnIr/Kb05mfG4dNZJYOnaycZQrUMXzQFw8WPgqs2R?= =?us-ascii?Q?PVFW+VE/AozeOlsnEfR5OukhL3oes+QCEsVts8jXrOc7Dib6oqSWKO6ZDGtE?= =?us-ascii?Q?Ivt+AMCQ4osrArKNvF9pAKUp/JDWb00xtqRA59/41jWV5ihzuWNjeE4edtwH?= =?us-ascii?Q?k9nZO4eDaVeOl0b5BXL9dtrUjHDYD4zqLZGlxzTtw4+qAohm18qRjq55mH5U?= =?us-ascii?Q?rtqyVwCWhhTj06/zntaX5Bab/FNY4vvlEjuVf9L5rYYCH7iAgzzu365wBxpr?= =?us-ascii?Q?C5GROG8PGWe5bV0dIjb76YjYoqUM7w0cF16Iuw0D7lyOX5m6qRSmpYBKN8pT?= =?us-ascii?Q?JF+b+D4EUg+1iX7p0nymWZOSxbOnvr5+/ZKk+7b4ASfu5+dOnnsS1Pz0ORXo?= =?us-ascii?Q?I4CFDwN8cBstRwZCa1hiOcyJSK17EE7T0+4IXFVD6jVDmdU/fujqxblzTHxB?= =?us-ascii?Q?csikEDILjAZcVt7CId+tyCgeZKAIJ6wFk2upITRE4HM5g3XGSX39zcv7RzKS?= =?us-ascii?Q?ZnkpaZAzoH2QAYRitY7ZykZPQ0SrBapS58jR30l8K1Lzdq+4M/2PbxLBnZe9?= =?us-ascii?Q?W69ViqwEbVbhx1wMRFPO0D/QXNlehaHM0WJlmqtmqWzqwmQpuMtvOuGKMTBP?= =?us-ascii?Q?gTVydOC/ackDXEipNwC/r/CcoDJeuEC59ErN1bABG6zg0gJc?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 8772a1ee-8ee0-401b-dd84-08deb30e570c X-MS-Exchange-CrossTenant-AuthSource: DM6PR12MB4827.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 May 2026 05:45:30.6952 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 6xNi3n9Ql70O9s8pJ+7TgaZ6ReDzi4s+3U9PHgphPhJg29UkaufkOrHhqO3HgvZcS1eoLvAa5luboBzz9lERuA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB9103 Hi Shrikanth, On Fri, May 15, 2026 at 12:19:16PM +0530, Shrikanth Hegde wrote: > > > On 5/9/26 11:37 PM, Andrea Righi wrote: > > nohz_balancer_kick() is reached from sched_balance_trigger(), which is > > called from sched_tick(). sched_tick() runs with IRQs disabled, so the > > additional rcu_read_lock/unlock() used around sched_domain accesses in > > this path is redundant. Rely on the existing IRQ-disabled context (and > > the rcu_dereference_all() checking) instead. > > > > The same applies to set_cpu_sd_state_idle(), called from the idle entry > > path with IRQs disabled, and to set_cpu_sd_state_busy(), reachable via > > nohz_balance_exit_idle() from two contexts: nohz_balancer_kick() (IRQs > > disabled, as above) and sched_cpu_deactivate() (the CPUHP_AP_ACTIVE > > teardown, which runs under cpus_write_lock(), so it cannot race with > > sched-domain rebuilds). In both cases the rcu_dereference_all() > > validation is sufficient. > > > > No functional change intended. > > > > For this patch, few more comments below. > > Reviewed-by: Shrikanth Hegde > > > Cc: Vincent Guittot > > Cc: Dietmar Eggemann > > Suggested-by: K Prateek Nayak > > Reviewed-by: K Prateek Nayak > > Signed-off-by: Andrea Righi > > > > @@ -12868,17 +12860,13 @@ static void nohz_balancer_kick(struct rq *rq) > > static void set_cpu_sd_state_busy(int cpu) > > { > > struct sched_domain *sd; > > - > > - rcu_read_lock(); > > sd = rcu_dereference_all(per_cpu(sd_llc, cpu)); > > if (!sd || !sd->nohz_idle) > > - goto unlock; > > + return; > > sd->nohz_idle = 0; > > atomic_inc(&sd->shared->nr_busy_cpus); > > -unlock: > > - rcu_read_unlock(); > > } > > void nohz_balance_exit_idle(struct rq *rq) > > @@ -12897,17 +12885,13 @@ void nohz_balance_exit_idle(struct rq *rq) > > static void set_cpu_sd_state_idle(int cpu) > > { > > struct sched_domain *sd; > > - > > - rcu_read_lock(); > > sd = rcu_dereference_all(per_cpu(sd_llc, cpu)); > > if (!sd || sd->nohz_idle) > > - goto unlock; > > + return; > > sd->nohz_idle = 1; > > atomic_dec(&sd->shared->nr_busy_cpus); > > -unlock: > > - rcu_read_unlock(); > > } > > /* > > I was looking at other users of sd_llc, i.e test_idle_core and set_idle_core. > They have rcu_dereference_all. So callers need not call rcu_read_lock/unlock if > the irq disabled/preempt_disabled. > > One more place would be update_idle_core. I think it is called with interrupt disabled > in __schedule path. Good point, __update_idle_core() reaches set_next_task_idle() via pick_next_task() in __schedule(), and __schedule() disables IRQs before that path. Since set_idle_cores()/test_idle_cores() use rcu_dereference_all(), the rcu_read_lock/unlock() pair in __update_idle_core() is indeed redundant. I can send a follow-up patch for this. > > And in sched_ext, scx_idle_update_selcpu_topology, It seems to be tied to cpu hotplug and > by same logic of cpus_write_lock held, one could remove redundant rcu_read_lock there as well. > > No? For scx_idle_update_selcpu_topology() it's a bit more nuanced, if I'm not missing anything: - the helpers it uses (llc_weight/llc_span/numa_weight/numa_span) use plain rcu_dereference(), so simply dropping rcu_read_lock() in the caller would trip the lockdep check. They'd need to be converted to rcu_dereference_all() first; - the two call sites have different protection: - handle_hotplug() runs from a CPU hotplug callback, so cpus_write_lock() is held, serializes against sched-domain rebuilds, - scx_enable() only holds cpus_read_lock(), which doesn't on its own prevent cpuset sched-domain rebuilds (those run under cpus_read_lock() too). I think this one needs a separate, more careful patch. Maybe we should keep this series scoped to the NOHZ kick path and address those as follow-ups? Thanks, -Andrea