From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from SA9PR02CU001.outbound.protection.outlook.com (mail-southcentralusazon11013030.outbound.protection.outlook.com [40.93.196.30]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EF37F286AC for ; Fri, 3 Jul 2026 14:52:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.93.196.30 ARC-Seal:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1783090352; cv=fail; b=B6mBCMsLWoiB06rpcj56CGiCiv1OOwP1GpAgndMDiT4etG8Opf1J8e6ckcWLFClL2N4u/gt25LV9XCbtixV2Hbc0tbRLPopFjG3fQRwaTsj0l4VJpmU9DJ0707ONxud7m/08CzxS+0KOn5k/+BGcLFuLX/ht0QlClWLUOicHpjQ= ARC-Message-Signature:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1783090352; c=relaxed/simple; bh=dCPKDRfqfiUuve27KA9rWSzykTa7LsAcE8SUPZUIMf4=; h=Date:From:To:Cc:Subject:Message-ID:References:Content-Type: Content-Disposition:In-Reply-To:MIME-Version; b=XrQaEt3zgSOne3k1sgy/zI4VTFiYDOdrb9WmtVCMPcKvGenFbg0fgxT05FWH0qKGapeaGgbAOyy/7o+chIUOdS0gYHt5SgqNCaSLxIHcEu6OOjnAhrxOWDIPDaD/rLEU1C3Z1N2E2GHNwR9dOomL+xrwRTGKlo/1jT5G9cW19Qo= ARC-Authentication-Results:i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=YRbBy1CD; arc=fail smtp.client-ip=40.93.196.30 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="YRbBy1CD" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=v99B31U+A3pp69gXRah11VDe3zfRDho8/tvJim/0TxH/BC9NIwMuuYwoEy/5Aql0NBAdWDZ3cwspeKKEe7ZIcTc0xh1xKvr0hUWKERyDdbuJaBy1Wf7Y/h8pow4ns6pF+DpS9aYc2X01GHPJloz+Wb9FVsjCLR4zzr9dGrpcL2EIMyPgjx7YBzzg5ElU8tdhzw/WEyR1MID/dCRwgdv6M/sfsXwTv9LrrnjvijMP1kGP1xsZsB0VcgDyI0AEOgbRuRsxZ3OA01PMfBVBQAoljX8cLfGv5Eli9a2Mm/6LbiIg4I2KPMIr5jn7XSAlm0e2BazC6HlCme1QmNdogNqrfQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=1u6ByXkoNuyC2/0rexn48fsjJjzGRvjgiS52PfjQATE=; b=VuTFtKkyavmeL0Lk+LpmSSaSqylsRCaNJDSt69BQ5HNY+8eXXDilIT3azlTSxr8MQJsDYlBHfBLGY1WJNqZqCXXa/O6Tj144Jk4Ki+xJ3btRmvUtCaTg6wjWgj2sRIy18Gv27XEM0OrL0/NqnQ9uGyxV5YJVCg9EbkeyG8Z10ZwQ9j5b9Ju6y2zH5Wx93mozplrvPvQ5FlrT4lhOTPPbwkcl4UpO7kIYBuRi1DvY+6MZl8GgDArVUpH1lZVEO9Pk7Jx6mhVnctAqiDAfs06dvgoMoGXWExEG5xEUpkHB2m5i5ho31GPSkfS5S7L1jkcm4gdHO6qK0TMgkxGk9Jg+Fw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=1u6ByXkoNuyC2/0rexn48fsjJjzGRvjgiS52PfjQATE=; b=YRbBy1CDxNyf0SjOvOBMotKWYhYILI74yyHMmkPztQGGIBpOL7O0Zk7pnBeLqTpTcjaWRJqHrfU+nEuKGs+2VxGI1ljJeBjuZXM+jXWupimwaxkPSTwvvFC4fDGPpyfA6/rFpbcBGtX3GgVdouOTTga/cncW4oTPLefNULbmUOumamNncpVUAOrZ2cDraP7I03U6SEy3QJQaQKJPLjVNyQvJocNrZoBX9IgU3AqXXf5+qtXVaHllE9gA+rE3iiAe3ZO7TRGOGIJnLB6xWWNaiCV+PQq12bm9n9fmryc55omGVLzsSdteTX/OVkHhH55Kzdhsds7yjJ5QVU8+pIIUnw== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from DM6PR12MB4827.namprd12.prod.outlook.com (2603:10b6:5:1d6::14) by DS4PR12MB9564.namprd12.prod.outlook.com (2603:10b6:8:27e::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.181.10; Fri, 3 Jul 2026 14:52:25 +0000 Received: from DM6PR12MB4827.namprd12.prod.outlook.com ([fe80::6261:3040:864b:159c]) by DM6PR12MB4827.namprd12.prod.outlook.com ([fe80::6261:3040:864b:159c%3]) with mapi id 15.21.0181.010; Fri, 3 Jul 2026 14:52:25 +0000 Date: Fri, 3 Jul 2026 16:52:17 +0200 From: Andrea Righi To: Christian Loehle Cc: K Prateek Nayak , Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider , Ricardo Neri , Shrikanth Hegde , Felix Abecassis , Joel Fernandes , Phil Auld , linux-kernel@vger.kernel.org, Julia Lawall Subject: Re: [PATCH] sched/fair: Stabilize idle SMT core selection with asym-capacity Message-ID: References: <20260630152747.128746-1-arighi@nvidia.com> <2ed258a2-ac9f-413b-aa39-59a59cdee1fe@arm.com> Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <2ed258a2-ac9f-413b-aa39-59a59cdee1fe@arm.com> X-ClientProxiedBy: MI0P293CA0001.ITAP293.PROD.OUTLOOK.COM (2603:10a6:290:44::16) To DM6PR12MB4827.namprd12.prod.outlook.com (2603:10b6:5:1d6::14) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6PR12MB4827:EE_|DS4PR12MB9564:EE_ X-MS-Office365-Filtering-Correlation-Id: b22d1c8b-7b16-4117-6408-08ded912b224 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|366016|376014|7416014|23010399003|56012099006|11063799006|4143699003|18002099003|22082099003; X-Microsoft-Antispam-Message-Info: dxZ/5VdPEgu+z3tYI8IYuR7TrXDjC0A+QA9NTOpk0jPkeLNdnYyUk3plKlPTGFYoC0xuAYO3j9KEu6dgFvyuIKO96JCvHF4cgZ1R+p8GeibuL3oP6YUBq0e1+NUhhZGKio3X0/sPcEMuOhUl0KZBwf3O2DBKvdGdxN41aJA83JtLyQvY3UJsA7sYVdGRvs9xkqfCXgE6FFXl9H8rCjoZwIsookPwvhd7eb1x+oV7O34CGmiTFZ/e0Lds/PbrBPFZyBsxND7r3w9ayoDS61qSUaa3tdPfZ1Q73okffP7LShE//MnAPTRHUn5ixvpHNwg8Qf4+iSFE51IpjmJJ/jkfmEfNzN7H5Bit97s9ydOYK9VzncEoDcvbBT1ZahzURTIkvhtMPsC3CQSbfVjYZUewntGDP4d7k7uxX733eCd+wndVGCzfKSVl66/QoPNYariVL4XEvXFPePr2oOdaakFzNl/ywYINHzOlNjYaIadZQOiBlzeITQ5jwWUDqfi/Ngsw6lK2Mh1AyPzw4r+d13M6noyC7600pzzksnDM59i1Vbwkcn/1Ar9IyVYbGCFA5ZP+hd8Ytp5hvS9+kurlQWvGZvY3n296vvosNak1hwymVxXOaIeeb2i7vJ6UaLe2d687aE/awG+oQkZHGuAI45c/wtNgQaAd07YHctCu6v82i8I= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM6PR12MB4827.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(366016)(376014)(7416014)(23010399003)(56012099006)(11063799006)(4143699003)(18002099003)(22082099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?TDE0c0I2eGp5WCtZVnU3ODNHUTk3VzJ6OW1vMWdVbUp5OW5tQm8rSkEzdUhV?= =?utf-8?B?cHBjVVVYek5mQmlzT2pXSTBRUU5xT2ZXNEVQamZEL0w2NWxQaVhrTjltRm85?= =?utf-8?B?YkpEZitqRVl0dHhNRTRxSkE1RTMxOVM3UmpRTGwwcHQ4VG9pQzNqeUJ5aDdh?= =?utf-8?B?QXhDYmIrWDhST1BGWXdwOWpXU016Q3EwN1UwenJKOVE4b05NcXJ1eUloV1BZ?= =?utf-8?B?Y0JEZGZZem95UnJzcHFEa2RHUy9DM0ZhVi9UN3phUXB4MHU3RnBzb0tySFNx?= =?utf-8?B?dDlhbks4YXN5M29NQjZ1OXBlcVdYVWxxd2c2TDJsbzRpN004Qmo3Sm1zVDJi?= =?utf-8?B?VFVrbkYyblNwWk5WL0JMQjl5LzM4MjFaSEMvVjdMbkZLM2t5ZkZ1SUR1NGx2?= =?utf-8?B?aFFaWU56RFk3bkRUbld1OGU2M3BPSkxPaCtmVS9ZTEhZdnJRYlg1UkdLZE5k?= =?utf-8?B?Wld1ZFNZcWJra3A1dXc1VlVTWCtzeVZ3MmpzTlp6OVE1UERyeWpLc25RNGdi?= =?utf-8?B?d21WUFpuak9OeFU4M2xVbkIzUkJiY2xnQjF3T213T2NQL2JMYzdadVdpOVJF?= =?utf-8?B?eWx2bm9reHBWMDkxOCt4NFpWWmhLckZlYWpWeFpTWGRQc0VOcW05bWVoWWN0?= =?utf-8?B?UmdjUGdnU2hsR2Q0S0ZkRGFnVXBkSFdrZWRRcjIvbTRQMEhqZi9rSzI2OWg3?= =?utf-8?B?eURQRHJUVjJwbW9qeEpiK0tHQUgwQXh2VkN4ZmNKV0VFQXZNRnorZFg5THc2?= =?utf-8?B?Z0FHcjdLT0xpZXdOaDB0ZnY1bWFOTFZPWVA1bVJFR2FSOHZNeFJURXozS1NJ?= =?utf-8?B?OEh1VjlNeVBPZnU3K2lIdGQzWHF6WWd3eTd3V0dSRWI1dDl3YlVmTm5kcURO?= =?utf-8?B?Rm5tYlQzYjBSSUVkQ2p2WExVcTdCR3YxMXF1c2JMU0xKQzRyZmN3STFoY1dC?= =?utf-8?B?QlZyd3RkNXhneUp5UFVjU2RUY1l5anNIRy9OYTBYRVRWL3pIdlltVktHWFlZ?= =?utf-8?B?SGFrK3RrYVVtcS9wNnpqRTViaTZhdGxxNEsydUFRaGlTZTdKT3F4Slk4Q0Jv?= =?utf-8?B?Ylo2RXM4UnMyYWt5TGkrVU5kdXQvUFRqZDcvSDJSU0dUK254NXZSYU5NWEsy?= =?utf-8?B?VG54dXFuNGlrNmI0MGZhVU93NEhsSVc0eG8wTjBnTEZNSGlMT0ZlWmg2bFRN?= =?utf-8?B?STVKQlNaRUUzSlBadG5QaTFQVEdHZTBFTC8xc2trdmFOdXBpdTFyZW8vWnh4?= =?utf-8?B?bWh0eFR0WTFLOGczVjVOMEtrakhrN3NnUjRaM3VhWkQrMnRoSnFGNzgwVjBq?= =?utf-8?B?NGtaNGVIMmtNVXhWV1VJM0tiWXJSS1pab0V2Nm5HNlRONHZIRTFYR1BYT0lJ?= =?utf-8?B?eVpxQVpPOHRzWGhiOU5oM09LcnJKYU1IbWNKd1djMHpBMzJLc3JDZ1Z5cFgv?= =?utf-8?B?dStqL2ROZWxqNHhtMmZvNVVtZzJOL2hBeWgxa293a29ZVkNZOHRGMkZ1YXdE?= =?utf-8?B?UXpENlNNakw0dVJKTUlDYkF6cDFTVmhyc0FsdXNPbjdtT3NwQ2EvbEZaakU2?= =?utf-8?B?RkRxZmtuNmNreXEvSlNQNmFKek1lMXRhTG43MFZTQkYwcUdKOWFyNHh6YURs?= =?utf-8?B?MWJCQkpQdnMzcnR2blVicUpyd1FzQ3Z5QmF5bk8rcVRKM01SV0FDZlEzTVR5?= =?utf-8?B?dmxBQWRkcHU1c2dIS00xeWx6WDBoNUVQaUdMZ2VNcE1CYzRpczBmSythN2RB?= =?utf-8?B?TVBtWU0ydlpGcGNLOEJTaDJtanlhdXRyN3Bnd1FDdTlKZkhsTEhiRjdjMXFY?= =?utf-8?B?eHNaTVM5OW9WN0Q3NEpJYjhnZFczWThlZjQ1NG1vY2hEbVhxYm1sRnlPU0N2?= =?utf-8?B?TGRaVm5kSXhyalY2S0l5REx1UFBMOHlMRkNJeU5vcDErZnAvVWhITUtKUUlS?= =?utf-8?B?ckw4a3RtaWJnbHpoOHovQStsam9LaXJRM1lNbmFYcFFDbTB2K2dLYU14WFlP?= =?utf-8?B?L1JNTjZ1aVJJS01RWEM1bVIzeHNWMzFtWWltbnlibHI1S3VhVGpuNDNtQjRy?= =?utf-8?B?UTliTUZKNm95aTJMYXlranFFRUVNd3lIQkJieUJyNmNNaW9VU0l0eFc1SFZW?= =?utf-8?B?M25sandaT1huWERuT3lWSk9Odnl1WnUybTJRQjlsamUwSXNISnl6M01RR2Fv?= =?utf-8?B?alJWMitJOFVKWTc1U1haUHpQa29MVExaVFdTd29OK05YaFdnc3VucFIvT0l3?= =?utf-8?B?SHl1K1k2QkQzcTlndzFlNGhWN2tSVGhtZ1UwUEl4T0VpbENEbDVWV0VsTWVE?= =?utf-8?B?MGttMzJza1FlTGlQdDJvNkdyYmJwRkRYeWtMeFEyb201UFRscTBZZz09?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: b22d1c8b-7b16-4117-6408-08ded912b224 X-MS-Exchange-CrossTenant-AuthSource: DM6PR12MB4827.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Jul 2026 14:52:25.6470 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: XHCrjbKAiNjJUc9EQ/FcbzifTHz12MvF5rKEwfUhXL0+0TKlpTzlm4Rzn2BVY27HbQtkyqDLvOWGK14zeyPQpQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS4PR12MB9564 Hi Christian, On Fri, Jul 03, 2026 at 11:00:23AM +0100, Christian Loehle wrote: ... > > I think the key here is that temporary runqueue stacking is preferable to > > consuming both SMT siblings when fully-idle SMT cores are available, more than > > having benfits from the stacking itself. > > > I'm curious now, as a not at all SMT expert, this is super counterintuitive to me, > am I missing something? How does this happen? > The SMT-switch should be way less overhead than the task context-switch, no? As mentioned in my other email, I found a surprising asymmetry on this machine: pinning one worker per core to the first SMT siblings gives substantially better performance than pinning them to the second siblings, despite firmware advertising identical capacity and frequency for both. Since this change uses cpumask_first_and() as the stable representative, it also strongly biases placement toward the faster first siblings. That may explain much of the observed improvement independently of whether temporary stacking helps the load balancer. I haven't yet established that stacking two tasks on sibling 0 is better than running one task on each sibling simultaneously. Also, the latter is not really an SMT “switch”: both threads run concurrently and compete for shared execution and memory resources, whereas stacking involves normal scheduler time-sharing and context switches. Once I figure out exactly why this machine has SMT asymmetry, I'll share the details. :) Thanks, -Andrea