From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from BL0PR03CU003.outbound.protection.outlook.com (mail-eastusazon11012032.outbound.protection.outlook.com [52.101.53.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 167C339FCBA for ; Tue, 21 Apr 2026 09:01:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.53.32 ARC-Seal:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776762106; cv=fail; b=J3bPP/f78/PpN0CIBdJWagBKkqx8vqcIbpaQRAwXcSGaTWYO7hMhCY8awOQ2Hp3fnTeqZgi+duklj9DmsMomf8B2Bi2lSTQ0PUc4CST8idUtzZYrZTbU4KL4yw+WoRDcLRedSYH3qhMWDsMgipWBn7K6yHUeR+i2VOZeD0EzEO8= ARC-Message-Signature:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776762106; c=relaxed/simple; bh=+rbAfBZu40+5jI54iRYWV5dfzatIUMH63haDhBsL2JY=; h=Date:From:To:Cc:Subject:Message-ID:References:Content-Type: Content-Disposition:In-Reply-To:MIME-Version; b=CKRbCdFFWAfN6OndcJZK+cOgHAPlhU/3yUcJJADu1u7O7k4BkC/0EFX/G/dCgR7nlO0VFsd3mSmoqeGidCLFP5isQ9tXf8lObeBXXXNQHBt798q2rBqoWp+f9TfWVxETvr/sxl4bFMw6WgoKvYRlOuX+cOvfD3Z2izOc8aWLL0I= ARC-Authentication-Results:i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=q3neXQ6K; arc=fail smtp.client-ip=52.101.53.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="q3neXQ6K" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=NRCcYvxCKKnmq8znbMQvLU87eTyqHUuwC1HtNsSq2OZHLLfOU2ALjd+uuWrXFtzCaV4R6MY/8rILu3VVRNE5+jc7GYi9xvoa0BWGad/sYP69UNK+hCtKnvs2vR4e5+wkk2qtIt1Hx5Qcigz3/pGWt/EgyoLgh96b4gvpNs98YKAsGkp8bJcoezc+rzOfXvBQJ3wK90yIH9xFJl0ahBURWB+iVtJZdfIhpfrjpkjJbRq59ZFcgwNQGD9eJQwR0oKtFE6MlAA5tJNaTqwUV/vflzkAIZv1JqjRZ8SJz2cQDo2WT6ZKmt1d2aQd1RpbhMbtqqjxyILhN8Y0qWqrZyG79w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=zQNhmuwUsAF7nVjx06Bz2wqjdFfaT2YkN3VoVNT42Lc=; b=M+zdou/HRA7pCOC9GGNMwjKeiahIC37VneR9Z4bCuE/sDB8vAwPX7+/LPFXR4xIbeiwsZ8WohswEK924tnQg7ofgKPUFBhce6Fx7ga6M+G/es4oXrTp/dvx5xSeN3PLX3ZFBMpo1bPNz4pLxHX3FeI6jd7PxmCCBhR+g03r4Ssu4up7WFD6MdFQkWZxAQNcWJq3lywoMrkeyw/s45G9f58jvQYnnMqcIYKfvCbIs45SVm/vcZjyYjubNVBGI1y8XYiGYtKuTHS61GZNfUbXCVM6SrGH6Ytagw1B7h23e8Ak69fAepJWpeO3nGv4uWUrfcBW4np+kOA35AXp3SC0axg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=zQNhmuwUsAF7nVjx06Bz2wqjdFfaT2YkN3VoVNT42Lc=; b=q3neXQ6KMJAmPKmOUq7K7dCoohmBaHZHWw98jKFkc7Fn7XFwDTbixc/XGw9vWs6yXsQ1HFywkHT6g2J1I+Q2coRLtEaitdfMD1D8G7pwCJ3PQI2KO1uhPvd1dn6ZVMgqgP1ORWjV6aWJpqptuZb9KUiu/mzQb/JozFcfA/WYKfwZG2roLpJgZiU9K1X9+Yc0lFj1AQv/TTE0YCCeqgYksvKjZgAbgWgiQvWaZK2EsgP15Wy36AZRyXNXHztKFSdv9tQhq3oKd2XIFeIZuLLpCkBsyenZ2BfsqVr/iwmaCoDuhQ4jYmgwv/9C9TSxfimybpkCWVmxSdsO0rH2S33iTw== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from LV8PR12MB9620.namprd12.prod.outlook.com (2603:10b6:408:2a1::19) by BN7PPF49208036B.namprd12.prod.outlook.com (2603:10b6:40f:fc02::6cf) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9818.20; Tue, 21 Apr 2026 09:01:41 +0000 Received: from LV8PR12MB9620.namprd12.prod.outlook.com ([fe80::299d:f5e0:3550:1528]) by LV8PR12MB9620.namprd12.prod.outlook.com ([fe80::299d:f5e0:3550:1528%5]) with mapi id 15.20.9846.016; Tue, 21 Apr 2026 09:01:40 +0000 Date: Tue, 21 Apr 2026 11:01:31 +0200 From: Andrea Righi To: K Prateek Nayak Cc: Dietmar Eggemann , Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider , Christian Loehle , Koba Ko , Felix Abecassis , Balbir Singh , Shrikanth Hegde , linux-kernel@vger.kernel.org Subject: Re: [PATCH 1/2] sched/fair: Prefer fully-idle SMT cores in asym-capacity idle selection Message-ID: References: <20260403053654.1559142-1-arighi@nvidia.com> <20260403053654.1559142-2-arighi@nvidia.com> <64fe32e0-d428-42bb-beb4-2656d8781b0f@arm.com> <7313ba07-7b87-447c-9c48-2f6b2b53ac94@amd.com> <1230f5df-470a-4e59-8c8e-fa159a6fc093@amd.com> Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-ClientProxiedBy: MI1PEPF000008D5.ITAP293.PROD.OUTLOOK.COM (2603:10a6:298:1::430) To LV8PR12MB9620.namprd12.prod.outlook.com (2603:10b6:408:2a1::19) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: LV8PR12MB9620:EE_|BN7PPF49208036B:EE_ X-MS-Office365-Filtering-Correlation-Id: 26440d2c-ab89-48f2-a308-08de9f849a1a X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|376014|366016|7416014|18002099003|22082099003|56012099003; X-Microsoft-Antispam-Message-Info: +IttiZlbD/97PM8PF3s0QePyt9XgiiicrurdFxBcwJeADK1jFcCCwJNXvOfiyVrRzWhca9miEpnu+QCyUtSAV1xSphS6GvPjrWgK2QQUc2TzTKRlrx1w/hDaMdpsfvps8PoUvgK4cEY2Zyd9wSb8NbbZJ9QMV84AFVzkbDNN6birzq+o3I6xmRvZcEb1FDqeExVgqlsMSPs0RSk8WPJyByaTh74+DHdrOKT1E+1VBlxi3KmeIojgwPDnZhaJdvdaNoxt4TYFiqQChrGvyO62hix8YMQXKBVayk/3HU1GD87E06lXyK6guppbyWoIvdFi5mSf/W6GLDZjtjRQs2nriaz88dNJohCa2t02xqmPXxXh47CtnvzfgajsaOGFW5RG52XgIVI8HSJA9UpnoYX8nPFJukyNRvyd0P5eqDvIRnzbNKe6uu1dUPPHQUjElV7iJ0V9Rqrxn64OPIemGv96Mo215AL9IC5jniFWKVRIVLq/4L0P8JCyf/dTjWoXXMwK6flZTGNBn7w9TsUmI994zHVElpowwvmlBuaDw24CSiElIKZZmfCzo/GBz4bU5sMcWzdtFp74UcYO8lcKUXs2I6VSX4H3jcb8VCvlWtfdZbiruF0aZqK7wE9WU7ug0QRS+bI+BlFYKV/ZWM1aHHRqod64KBh+XfeY/R9arnadhWefsmoqRnPorDJ0zhVkpbFKyyDgIupMePrBl8F8XtCoD1xHwAumE6pP51lu2s/uchw= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:LV8PR12MB9620.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(376014)(366016)(7416014)(18002099003)(22082099003)(56012099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?+s0k+JMr02PQ/oAZzyQsNbW4y6nelMjG/UKbuj9W1a6Z9XaLEbwgWcKae4Yr?= =?us-ascii?Q?Y6D+gopBi1B3G9AnxdlRf382+lgsEIbDtdPADTzjkY2Q8UHokm9UX9RIaQBc?= =?us-ascii?Q?LY0NovNofN4CVV+0G/GDnSI+Zfn9KNY01ImtbOvn8r5n8dxegoef1TWMdsW9?= =?us-ascii?Q?MTx8P1zaS3/z7UYGKZwqKC1/qdYaU4hKrrZxUprsjWSQwfDoyeYwTVneTQse?= =?us-ascii?Q?HfszPVl4Wm6Zn6GAyLHtiaOC4r87DMo01gmW3JoHlycpdmDWYo78wr+4A/nG?= =?us-ascii?Q?gW5ZpOmW/1SsWuztFwLWeZ9F70whoyPvQ7NrdvBQsRC2zs9dbHLSsub9kZ6j?= =?us-ascii?Q?nARRTLRS5WpXpbxzQVATalKWWzW7K3vnm5OCKBLIyAO0Cc9R4MDmOWKPMkOA?= =?us-ascii?Q?vkFKEUib2GPfvmRe2jxM8r47yF/4E7Lfj7xPJ82kEXbo1kNbyoWvZ/sN1R5a?= =?us-ascii?Q?S1TXFXHy74Y2i3BkAH7cEFU0o2om5lexufySr8aen+7zYavBqKNFm5arBeyQ?= =?us-ascii?Q?vtxGaLV2HA2wFfXrG2nD6A80Le+vBZS05i6A4FAPPm0P+nqRGlTuVfALMTj4?= =?us-ascii?Q?7PxwczLU5vvbuPc9TJkrMNKUpoLsXAxdGhsa+qzfTkKlXfh2y36Ex7zJETaV?= =?us-ascii?Q?0awC0YYGajqctxeeMcrdIXxBtVoSvG4C/FmbenE0TYzrRqAFAanjLchzzoUE?= =?us-ascii?Q?j9ZZU1az2ouWjsdrqc9EsjE+rsIDUBB990hKMJmLZ1Ebbg1XNE8sKt9u3Hiv?= =?us-ascii?Q?Lowk5HZ4wvGA9JMC/KHJ6vokywVHnTcKQojwX1AorZsV9i7XAszb28/Otq2r?= =?us-ascii?Q?7MQl0eBnhjhd+e6HRs78K7FoA1t294+DnEaUiMUGkEPIvdAzaEu7SY3f+KWY?= =?us-ascii?Q?j8tNKmoP4iRc+tUs9T8AtB3GUi+jRasFpFSuLUAbD7f3yo8aBaxF/xTrkP3i?= =?us-ascii?Q?10oNju5xhjcbDG6dRrQOdjOgH6yRUIvciol/9WC8jY+MYmiqq+VoL/L99R6j?= =?us-ascii?Q?NFx21K3sPlKvhDqDvMQXZhIN4X+iGkOvku39xfvaXnYYzuxGOllYmAQynWM2?= =?us-ascii?Q?4V6DvFuH1k9OWgekvcn7lxlQcbYKeKxoDc5u6XnIQlUgOkbZ324O1Ey8UpyD?= =?us-ascii?Q?sv7rpNMC2RzcDJTsqxWJI/YFqYBjO2K0XD6b+2AuTPTJYh8z46BWFMrj9kQF?= =?us-ascii?Q?G8Od9DhJSkIW2Nop43bOR5X+CyNe2COOp2eTCGrrSd7kH6Mdjw79tfvOZ9sV?= =?us-ascii?Q?8dIwR1GyA16hwGhSi8tVl/tlXR4zXKw1U1s7dcfRtijGZ94U0oGI+Lwg+is8?= =?us-ascii?Q?m5Z/IC2rXEU4qIcpw2z8j0v7hgJXyusMaHrfV5Y5q6ErYmj4iz9jvtkef9ae?= =?us-ascii?Q?2qZxgCmTG8ixr7ncNCBS1VakZTwYuFvtxBr7bmnjY+9elMg5pHQ4+6QkcVhS?= =?us-ascii?Q?1S/yOsadNdM2DiJ3lqa+wMqJG/GT2Zts1zPKSMacKAQsQgALqQA3skHEq7re?= =?us-ascii?Q?LJqaXIpuUWAajM4bQJouimailwulqmo1E0sCppF+pK6pvisIiAj7OpXrkJpZ?= =?us-ascii?Q?QuOe/emU+EDui6CJyr4i9BEiLckdqknCq1sNv/G5TB/dRdLJ/DmGN8QS0OFR?= =?us-ascii?Q?VFwBuHF3cAOGtbeO5dB7uC85m5sGnW5sCi9XQErZSHCZcqkNHryEH98EjTQE?= =?us-ascii?Q?Co/dMwVGsQneJr3Tc86nD5xBw4W7AO9u7XVM4djqaKWuxX4XIFYWKDq3gbeZ?= =?us-ascii?Q?3DIOkW/ypQ=3D=3D?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 26440d2c-ab89-48f2-a308-08de9f849a1a X-MS-Exchange-CrossTenant-AuthSource: LV8PR12MB9620.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Apr 2026 09:01:40.8548 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: SFIlmqSe4EfsOudk3C9M/vwXazd6Q5gs0bfZpCIvvzdZwcB+TacUc/TMLG4kZhS89I7L7nb/boW/KoofFw2OKQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN7PPF49208036B Hi Prateek, On Mon, Apr 20, 2026 at 11:42:23PM +0200, Andrea Righi wrote: ... > > >> I still have one question: Can first SD_ASYM_CPUCAPACITY_FULL be set at > > >> a SD_NUMA? > > >> > > >> We'll need to deal with overlapping domains then but seems like it could > > >> be possible with weird cpusets :-( > > >> > > >> But in that case, do we even want to search CPUs outside the NUMA in > > >> select_idle_capacity()? I don't think anything stops this currently but > > >> I might be wrong. > > > > > > My $0.02 on this. > > > > > > In theory it could happen with unusual topologies or constrained cpusets, > > > although it should be quite rare. That said, select_idle_capacity() already > > > operates on the span of sd_asym_cpucapacity, so if that domain crosses NUMA > > > boundaries, we're already scanning across NUMA today. This patch doesn't > > > fundamentally alter this behavior. > > > > Ack! I was just thinking loud from the topology standpoint since > > sd->shared is not designed to handle the overlapping domains like > > sg->sgc does but we can probably figure some way to make it work. > > > > Using the ring topology example from topology.c: > > > > 0 ----- 1 > > | | > > | | > > | | > > 3 ----- 2 > > > > Consider NUMA-1 below gets the SD_ASYM_CPUCAPACITY_FULL flag: > > > > NUMA-2 0-3 0-3 0-3 0-3 > > groups: {0-1,3},{1-3} {0-2},{0,2-3} {1-3},{0-1,3} {0,2-3},{0-2} > > > > NUMA-1 0-1,3 0-2 1-3 0,2-3 > > groups: {0},{1},{3} {0},{1},{2} {1},{2},{3} {0},{2},{3} > > > > NUMA-0 0 1 2 3 > > > > > > The "sd->shared" assignments at NUMA-1 will put first, second, and the > > last domain in the same "shared" range by today's logic since the first > > CPU in their span is the same although their spans are slightly > > different. > > > > The third will be standalone since the first CPU of the domain span > > will be different. > > Yeah, makes sense. I'm wondering if we should attach the shared blob to > sd_asym_cpucapacity only when asym is a non-overlapping domain, otherwise > fallback to sd_llc and, in this case, ignore has_idle_cores in > select_idle_capacity(). This might be not the best in terms of efficiency on > those exotic topologies, but it'd eliminate the overlap/aliasing risk, while > still being correct. What do you think? I slightly changed your patch adding this logic on top, I'll send an updated patch series, so it's easier to review/comment. Thanks, -Andrea