From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from SA9PR02CU001.outbound.protection.outlook.com (mail-southcentralusazon11023116.outbound.protection.outlook.com [40.93.196.116]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 331F012CDBE for ; Thu, 12 Feb 2026 19:36:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.93.196.116 ARC-Seal:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1770924979; cv=fail; b=j2dU18Iau1MyPDyZKtaWYcpvtnt3MEIC8L1TTsu5mAnUcQxDhC+1opuIXsL5g9qTVTSLaXtnuV9uf8rai0/Y6egxMSxblBi4n3MWYeYmhMiRqno7efbeW0T6cplNZVJVqQUrimPFXwwWGkaNyPrxpeh9F+Fl36WEM4H/1CpMqKw= ARC-Message-Signature:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1770924979; c=relaxed/simple; bh=S5T49z9jrDAF11MogNS96x6xcSdQLSx6uwVmOdTa/o4=; h=Date:From:To:cc:Subject:In-Reply-To:Message-ID:References: Content-Type:MIME-Version; b=JQybItN3QJKLi+jiCAmfAHSOBFQrBkmdUnxM6THFCEuI5N03AqiR7vInTQ4IVTxGAOjiyLgI/NJJL/Q4+AHdsSuFXUFef4ElPX/xgjX/onscxL+LncQFRaMdp1BDu0hhbMnLiHaIGHlQ0rMCuCsEe0kJjVYFav7BHRKNcW1Tq1o= ARC-Authentication-Results:i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=os.amperecomputing.com; spf=pass smtp.mailfrom=os.amperecomputing.com; dkim=pass (1024-bit key) header.d=os.amperecomputing.com header.i=@os.amperecomputing.com header.b=X0tMZ5Go; arc=fail smtp.client-ip=40.93.196.116 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=os.amperecomputing.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=os.amperecomputing.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=os.amperecomputing.com header.i=@os.amperecomputing.com header.b="X0tMZ5Go" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=RSLhN51EL7alCwT/TXpi1Qrk7LnDzrZlEpdFHvsJVKNuK7v4F+54CQ7XXzF0q/CZ0ltnNrKinDIwqV7Fu9DGd94TgXseaC3cTU5Ha5KDrUeTY6d0FTHh2cJ7S7qBE+WEFI0bv0wwnR91x8StpVsGQSChv55tPzHnZY9HPhcjg1r3ZJU03ws67aSPJ6yVotyAp5TwyhfxtK+PnVc8CxLb/cO3tG1XS/k6gFyRCAN9GFUCxkCSo9QL5oDaj6X1TRrNtXoOz9fD8YQsW/nkzMBQgLRapb0ZYtZCCZTyP4z/lhNqUhveSkH3tGjB8wO/uBybRkzcNKSF6aMGBc7L93wRpQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=i6kTF0Tud9GN6txP4fBXpZ8zww8x7qdI3uWuMLOgfqI=; b=e8tKxsiJ4VGvTf6nM4orakOBbikyIDcPEkrNHMCOIcJLsYOQj+oyNrjiYn9clKnEwlpXmJIXydK3i093mHGRrSVfoYUq2olpR7AEMi1+Jhh3BVlWjhSu6e/Xk0srmFTr3QvmEwpYHXJK2iVR3LrvdKPxaI27I/FBXDOLJnJ7zp/0KfGvyPaxtkgkYizUPTmmoC3Ke4X3qIQGxnKfH2nVTZDYK3aRdaOnBcrtgHySrSEXgqaI+1fKeWUTLxxBAoumDOToS9Apc6ky+MuQEpk1YnVE3znYKzUat37nARhrybGknSBeJT51yqhmqA6Z1cXzGHps0VPRYJjL9+x5Y5ydow== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=os.amperecomputing.com; dmarc=pass action=none header.from=os.amperecomputing.com; dkim=pass header.d=os.amperecomputing.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=os.amperecomputing.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=i6kTF0Tud9GN6txP4fBXpZ8zww8x7qdI3uWuMLOgfqI=; b=X0tMZ5Gome6qOOevd/njKLjTuEFAeBukqAFW2GmGjFzceNrJkTePHWrWZdRkd8nMwcIBmCT9T1xsgawmActzMRqg1X+pu/+DKdVeS1zMRa1DAicADwb2y3ozCeFu5TAaBUjiJ9VUQZIyF8IFYmocBUnYHzy0KNvP1EDhPaLtspU= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=os.amperecomputing.com; Received: from DM2PR01MB9464.prod.exchangelabs.com (2603:10b6:8:2e4::17) by PH7PR01MB7774.prod.exchangelabs.com (2603:10b6:510:1d9::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9611.10; Thu, 12 Feb 2026 19:36:09 +0000 Received: from DM2PR01MB9464.prod.exchangelabs.com ([fe80::8b41:b64c:97d4:ed48]) by DM2PR01MB9464.prod.exchangelabs.com ([fe80::8b41:b64c:97d4:ed48%4]) with mapi id 15.20.9611.008; Thu, 12 Feb 2026 19:36:09 +0000 Date: Thu, 12 Feb 2026 11:36:06 -0800 (PST) From: Shubhang Kaushik To: Frederic Weisbecker cc: Anna-Maria Behnsen , Ingo Molnar , Thomas Gleixner , Vincent Guittot , Valentin Schneider , dietmar.eggemann@arm.com, bsegall@google.com, mgorman@suse.de, rostedt@goodmis.org, Christoph Lameter , linux-kernel@vger.kernel.org, Adam Li Subject: Re: [RESEND PATCH] tick/nohz: Fix wrong NOHZ idle CPU state In-Reply-To: Message-ID: References: <20260203-fix-nohz-idle-v1-1-ad05a5872080@os.amperecomputing.com> Content-Type: text/plain; charset=US-ASCII; format=flowed X-ClientProxiedBy: CY5PR15CA0112.namprd15.prod.outlook.com (2603:10b6:930:7::15) To DM2PR01MB9464.prod.exchangelabs.com (2603:10b6:8:2e4::17) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM2PR01MB9464:EE_|PH7PR01MB7774:EE_ X-MS-Office365-Filtering-Correlation-Id: 9fc65641-e5b0-49b8-2041-08de6a6df8fb X-MS-Exchange-AtpMessageProperties: SA X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|1800799024|7416014|366016; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?HvBNYNn9WmzZYhHc6dnOqijCwLIZpv7r3zQGIvFe0Tzue2ZUdpYPKHhhhfmW?= =?us-ascii?Q?2cQuYyFwsM0+55jyg9jh5kXn6INq0CXYmmw9XrmQP8Cvl7Q/FbhxmghmfTcn?= =?us-ascii?Q?wl89ykPw/fNJqCVnim0ZQ6iZAtHcspSzEIMfPUcHi/EIAdsYS2idhjmmEyCj?= =?us-ascii?Q?Awc82bs/tc+/9F48FtUYjleCn7UPrVNgXWLNa3UJEi5IJbycT8Cjpp8RXiuY?= =?us-ascii?Q?Rm3dCJom3PtwRf9+cYYmaA3XOlvvRLruOvyVZU2h4CwX2tXPBXDUmiMiU15X?= =?us-ascii?Q?DoA0B5A+h1jCw0ATjWBgEAMKbR7eA7Ldr8MTROgs7paaebY5o+QW6mrai2mm?= =?us-ascii?Q?RO97ef+H6pfaKwcJshAGmSM172Gp5SltQRhZoUu8K4mgbG7AyMPcfP2CHFGW?= =?us-ascii?Q?Q6C2EGiGI30qHJmh1JgNq4rj7n/lvkkIqIuq7fUa9BmOrkPfdXMLc0FUz/kE?= =?us-ascii?Q?1+lyqUq5JpngPB1yTj6HASNRzytwYr4j21unAOukAc1XGvwL2uo2Z7Wun/J0?= =?us-ascii?Q?wgaY53EoJEsaDX7D0/RKSS4rrxrfCZlpgUnolNYIzrbC9lWv1HbdFs5wymS/?= =?us-ascii?Q?KJKUpuWQrpOR0jKTtCCfnL7iRJTeVVZJMJB4E3QuRbNk/NY5rQHWD+VSAqT0?= =?us-ascii?Q?9KsDBmuMEEGLoAXIr0hT/FHaoXhfv/Dq9vJgeeSC+R3DAmfdX3rMYkQGIuCF?= =?us-ascii?Q?eg9IcFYjhHOkZxDzHVwlSvl79kPyYkRqNSMp9Aa8Dv9mHQ+pDAnio0zvZRZq?= =?us-ascii?Q?C801tCf6iRgMtFJprtL5tICnxLKBi2T45WHjqXCyBF5HM0Il327s6iJqK/IL?= =?us-ascii?Q?dsMf8eHT3n4VaNNxl6eB4WALlY98bL+eTEvRjJUAaEKP5H/2BSJr83Xech1o?= =?us-ascii?Q?mETItrZ3/OY65kirbL0avDOoGi3qS6cqE+csVbUkLdspFZMlqJQLSS73NoGZ?= =?us-ascii?Q?2cLMcjGmMOUEs40b/LkAFPTf5gxOBlP9P+1GpGRf7GiSdWBxkck+j7mEY0vk?= =?us-ascii?Q?E1Yht4R9Jm2HgLdLQbelqz5tY0if9c4p3Ahi+2RaekxKvCCUybrOT/ZlCjGl?= =?us-ascii?Q?IPoVUgWdeVUSs2eykIsxAfsY5kDQ5IMwqtX61cNhgmOJcioBdmTA85BvNY0X?= =?us-ascii?Q?xZ7y/uQJH2cj5loWirkP8TXN7dHQ/L4txmKrMAoL5osTmxNA4cw1Fl8NrN/K?= =?us-ascii?Q?/xHjqT3uDPiOdaIV7GCEKEJpqS6gb+zYBXlkSftXp1rvoA/7NJVQLyPtCFgy?= =?us-ascii?Q?M2sKZeUkECpiV4SxGdkzu+Lc1ZgsuLDJS9lk6YQ2yvUa4icsiKsrf+e080x0?= =?us-ascii?Q?V9Kjk4d/cF082Vm3M1yqNk2qfLL4Aijf/gXqLbiZhPmCJtgbMGTlsbaPioPg?= =?us-ascii?Q?r633SJGYPCYknn4JN9U16FgrXmDtOZlhYGqEm7oFvH0nTXkCa/waRQlattJA?= =?us-ascii?Q?KHhA+B7K+V1xIFgBdYLEyfsEZeqbiL6oxjLdGCy4X+m0DXvQf/B8MPVzHiER?= =?us-ascii?Q?rgwdis8j+K8rxWgud36VkkDOOY5UKQCijQUQ2mK34sDdhXdSRvki1PKNFFk6?= =?us-ascii?Q?qJv0ywO6PjEuY7gvGmM=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM2PR01MB9464.prod.exchangelabs.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(1800799024)(7416014)(366016);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?DmV2fQ3pIBfAYRwNvsEChVCYSJuBVGo5QWzqO+uuyVkRJar3MFI13DJJeGAd?= =?us-ascii?Q?xSlDtO5NQGWoueseGpK9kjQCnOKBH9+aVDdGSB+Re004P/DKjtGTy7nkW2lz?= =?us-ascii?Q?uIaSwxRSp1Mo4Jc9J9/uJmxN6xEhkh1A1al0nauDSA7m0UqiAjsVAU644KLD?= =?us-ascii?Q?o+Wizn35nJacE6ufiZYVa0IUETMFCdji+Q+Riou6RZXJGya3Un5tMRFbKpGU?= =?us-ascii?Q?ixBQcT9molReGNb5DVu5SbWp9meQqj4q6taJekx5a/aMfJDuVfRVYWu9MxpO?= =?us-ascii?Q?lDJoQlT30BF0hvuN+e7A8xPES6lcm315sz2GpkU+6pW5F5YfpsXADUCTWaQe?= =?us-ascii?Q?JuP1GfzZksvSloJTQ1jDuhc1mT6m2l1O2vSoRnX2Sk75CvPwBg4CKFlq5wJ2?= =?us-ascii?Q?20pHmo66cveW4/hjgomEKsbFl1F3dslt+sMYAFC1hcmvCd/kT3cY9UsTWOiB?= =?us-ascii?Q?okYoMxBxiqjb8NR1vUS2YjWA/VQ1AQvUoG5eSJJCrzktB/D1Kv6GCx6cYgdS?= =?us-ascii?Q?O+KuHzwHKSBncPDGUlK24VwWyH5STLmK5w4jmR1WODXk/prPrj+aNcu5n6/D?= =?us-ascii?Q?8JOdki2GmL+SHmWGUb6PB5fPRvvP5zKpz5mdB0NwMcMMA+z7wbIye+VwdMz6?= =?us-ascii?Q?xFi10JO2Etl1ZEoj3FmsKgrkCBmF3xyY1l2aiY3rnHeOS2b/doW208LX62cz?= =?us-ascii?Q?a+uwxoAIFJborZLTh7ku9oFDy/RlX+0me6FNtalLcl5Ofb3f+D6GUIoHmWeO?= =?us-ascii?Q?U+fZlym0hq30vgkHYr3e4DZI+SuChIbhtmhEVEz1GqWsXYU1AYIN29ub6Axf?= =?us-ascii?Q?wyswKBF9XCm31peFnJ4AOxNHtzKp3zWUmiw+du3KazUwO8jdBX1PHHxDJv+i?= =?us-ascii?Q?Cx3BISB0d5ZwRAUbXLPdm0+1E3/kl7+huJACdK1BoAtuAzykty9/InvjXS9j?= =?us-ascii?Q?6/EWhlqlsoItQ7siXgv9SLn8fxwKJ5blB3YzaikANhhqz/C1VTIBqkDUrT3E?= =?us-ascii?Q?BWQScA7cP+IgNheCgu5y1Pft5OmiFtoSNpAAnCy1aaOHm0azpCrcs2NH2jJ/?= =?us-ascii?Q?uk//oH2hn9kCBqaS/PKZc0EHKimZ+QTlL3fmMwy+8BBybgExr6IJtTUn78sz?= =?us-ascii?Q?cG+vnCXSS46zhanASg89PE8kDaiAtwTYiEGzXhvEztX83SWWLYTPUWCLqcQd?= =?us-ascii?Q?wuhf+qZn+LAbuBoJqXPjL3l0+e04IVAqrWbDgcfyBA2ASjwRlEub2k1sTsnh?= =?us-ascii?Q?PM7yMNORxdQOyH2dhJSTNNbrN8MqbSKKl/aWbkTPmLrjUBSNaQ/At97yeNn3?= =?us-ascii?Q?iy4QNlYQMOE0nqa0Aur9RlQA6sllqK/oXJTjGyfQI8UH8sAxY6aKYAorXYPy?= =?us-ascii?Q?S4H2iaEq/JH6WFwwnfWr8J6d0BFgLuFMHd5TmLdH/HOIWNIQTLy5kFqUMzPh?= =?us-ascii?Q?7k2dP8nRmCYQEZ7pPXmFpD8OzFJ69G3VzfUJ7I74xyZgpczwC8WM+fOMMQcT?= =?us-ascii?Q?HPgtZ1D/D8m9Uq0CfbJ7Syw9MY7pZjwaLxPWYHa1bNN6JOOiY12VUSC1fUzM?= =?us-ascii?Q?EIxKqjC0wKtng430OI6WfBNLJGQVXkxc1ofNVJEFzinrRMwn6E6SJsHI7mAF?= =?us-ascii?Q?o8drRPYh21vZ5Kbcdlx8dQhDqPeOozzZitlbIzeQfViLNwcgdhrpi2g9p4L1?= =?us-ascii?Q?j+1I7Y4mj2LF0sBstECrvuEpotT3QG2Dn/uuLSMxu4WJB/bEvAEKUmnn3GtN?= =?us-ascii?Q?5beHYuGmTfzjBRGQv4FigBs7ITe30pye9fdoscpxDI+H3ig5h8Mm?= X-OriginatorOrg: os.amperecomputing.com X-MS-Exchange-CrossTenant-Network-Message-Id: 9fc65641-e5b0-49b8-2041-08de6a6df8fb X-MS-Exchange-CrossTenant-AuthSource: DM2PR01MB9464.prod.exchangelabs.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Feb 2026 19:36:09.5582 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3bc2b170-fd94-476d-b0ce-4229bdc904a7 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: +x3aqGvrQLnon0ncItfVOXAsoWRaS6eozVX/9b3m3HvzjRtKpbtX397okSL93V4tSFyE8u5o/T/t3Et0rlOekNdVR3D5Q1n3PzDTKYMNplfBgG1xkUyl3QHIZSNzvrqF X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR01MB7774 Hi Frederic, On Thu, 12 Feb 2026, Frederic Weisbecker wrote: >> >> Tested on Ampere Altra on 6.19.0-rc8 with CONFIG_NO_HZ_FULL enabled: >> - This change improves load distribution by ensuring that tickless idle >> CPUs are visible to NOHZ idle load balancing. In llama-batched-bench, >> throughput improves by up to ~14% across multiple thread counts. >> - Hackbench single-process results improve by 5% and multi-process >> results improve by up to ~26%, consistent with reduced scheduler >> jitter and earlier utilization of fully idle cores. >> No regressions observed. > > Because you rely on dynamic placement of isolated tasks throughout isolated > CPUs by the scheduler. > > But nohz_full is designed for running only one task per isolated CPU without > any disturbance. And migration is a significant disturbance. This is why > nohz_full tries not to be too smart and assumes that task placement is entirely > within the hands of the user. > > So I have to ask, what prevents you from using static task placement in your > workload? Actually, the llama-batched-bench results I shared already included static affinity testing via numactl -C. Even with static placement, we observe this ~14% throughput improvement. This suggests that the issue isn't about the scheduler trying to be smart with task migration, but rather about the side effects of an idle CPU being absent from nohz.idle_cpus_mask. When nohz_full CPUs enter idle but aren't correctly accounted for in the idle mask, it appears to cause unnecessary overhead or interference in the NOHZ load balancing logic for the CPUs that are still running tasks. By ensuring the idle state is correctly tracked, we're not encouraging migration, but rather ensuring the scheduler's global state accurately reflects reality. AFAICT this seems to be a case where correcting the bookkeeping benefits HPC throughput even when the user handles all task placement manually. Regards, Shubhang Kaushik > > I'm not saying it's undesirable or impossible to do adaptive userspace dyntick > for users that don't rely on ultra low latency but rather on high CPU-bound > performance. In fact the initial purpose of nohz_full was for HPC and not > real-time. Turns out that real time is all the usecase I have seen so far and > you're the first HPC one. But adapting nohz_full dynamically for that will involve > much more than just load balancing. Now the static affinity should work for > everyone. > > Thanks. > > >> >> Signed-off-by: Shubhang Kaushik >> Signed-off-by: Adam Li >> Reviewed-by: Christoph Lameter (Ampere) >> Reviewed-by: Shubhang Kaushik >> --- >> This is a resend of the original patch to ensure visibility. >> Previous resend: https://lkml.org/lkml/2025/8/21/170 >> Original thread: https://lkml.org/lkml/2025/8/21/171 >> >> The patch addresses a performance regression in NOHZ idle load balancing >> observed under CONFIG_NO_HZ_FULL, where idle CPUs were becoming >> invisible to the balancer. >> --- >> kernel/time/tick-sched.c | 5 +++-- >> 1 file changed, 3 insertions(+), 2 deletions(-) >> >> diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c >> index 2f8a7923fa279409ffe950f770ff2eac868f6ece..eee6fcebe78c2f8d93464a55fe332e12fe9c164e 100644 >> --- a/kernel/time/tick-sched.c >> +++ b/kernel/time/tick-sched.c >> @@ -1250,8 +1250,9 @@ void tick_nohz_idle_stop_tick(void) >> ts->idle_sleeps++; >> ts->idle_expires = expires; >> >> - if (!was_stopped && tick_sched_flag_test(ts, TS_FLAG_STOPPED)) { >> - ts->idle_jiffies = ts->last_jiffies; >> + if (tick_sched_flag_test(ts, TS_FLAG_STOPPED)) { >> + if (!was_stopped) >> + ts->idle_jiffies = ts->last_jiffies; >> nohz_balance_enter_idle(cpu); >> } >> } else { >> >> --- >> base-commit: 18f7fcd5e69a04df57b563360b88be72471d6b62 >> change-id: 20260203-fix-nohz-idle-b2838276cb91 >> >> Best regards, >> -- >> Shubhang Kaushik >> > > -- > Frederic Weisbecker > SUSE Labs >