From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from SN4PR2101CU001.outbound.protection.outlook.com (mail-southcentralusazon11012033.outbound.protection.outlook.com [40.93.195.33]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 17BFC3890F6 for ; Mon, 20 Apr 2026 09:39:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.93.195.33 ARC-Seal:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776677968; cv=fail; b=R2G8f9GO+0qaATszbDhuMTr0QXQn1Iw+qnUU02lDZUISLbZKI3KhfpZkXYKgQ5Be/U2keXmFmi1K+4AXQVASTiHGpNR+yQ7JcwmIRGhtKnrmoWmYzJ+KTv6WzeNBbUoxzHgfIVeHbifBM6Th6+3H6EAL5VfAuf5avKBT/voA1RA= ARC-Message-Signature:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776677968; c=relaxed/simple; bh=eXQ9pNm9EFbDPPW3LFpdqtDk4nYyI0TDHoFH9F/ONBg=; h=Message-ID:Date:MIME-Version:Subject:To:CC:References:From: In-Reply-To:Content-Type; b=V96iZ63W604Q4IzCMNask8KzOqXLPYKS33k/o/M3j57evxloJ6RsoKDBLXB0Zi7+73Tw7KOIbLZutuS0borJLfl/w1IQFxEm2emwbvXaUzw9nXeccAffRB05v48+Od+6HfrI0hLxi6LCrBVyuNg60DX9ElnUX+X853vbS8HLOOg= ARC-Authentication-Results:i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=251mSISX; arc=fail smtp.client-ip=40.93.195.33 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="251mSISX" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=jB30+rbOatHjvxsATfvqGzyXvXqD0AQx1CGX198EbX1f/rQXh1V28HrwXIN8axVwRUg2xJuZFqRffJiNs+l57f3r074D6oWpJ/hwpdNIp32msyLlHeHqsc7V5waHlmeDVQmrrbb8i7swMvXWtWyQOWhfBsp0BKdzTs7MkcyQV22c6/400q15bpG3ZZL429F7qTe2+m9hfOM5YemwnDPv1qv+V+a45sKL9q5Lm0r+Ugi6vYgKrDwDMRMgS39gbT9vmcaHA67YyMOYBjsvfpexsKheE3XveVlx2XwKRwCch9itSICn3AcYMtGFvlCDGhCsQPmWdIRO1Effn4m2QDO7aQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=OCQKS3VH+tPWfBgRQcMAjT4FPZgOLlQmkWFqcsQxuzY=; b=qbOgQkAtUOBjnj8UeGaqllJ7sxwPxSL236XGwtWj0sZHvWBmXZ4aWFQPTwiliMpPztRHqW7NGmWe6peh92H44yyrR/sCy/L3nFYGOZuu206Z9vHkf5n0WQ6J3SU2vUnvA+ngHm23Ccqdw+3FxEFzH/+GM4OokulV0+9TqNgOVcyKukkciI7cF7LGF7VzGCV8P+9dlYd5o4w90sNIsVC3bndfkbn6IjYPkis3acxxGb3R013boIg1zvr6i5NQUNlqx3BNLxUlvLmnfFIaCp2oGbeigpZ0t8lmWjK/rTcrYHuVOQSinoeUHmqRckXr4VvSn5DIUiwZQVlPSXRKM3ml/A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=nvidia.com smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=OCQKS3VH+tPWfBgRQcMAjT4FPZgOLlQmkWFqcsQxuzY=; b=251mSISXA8e97zCzqsjQDznjnA2FB40Y0iYK3n2rw4Pup/jgVIt2IjoXoG5anZnQi4ZFqhWYMfdu1l98MROe2Cfl9cUFKJdHUcSIlOSzvZ0dLtnRfOr3r1K0XyNUrcyouEdAJJWSNWFlVgO02SFI0NAt58pPKyTxKIWbhrjNQ2I= Received: from PH8P220CA0038.NAMP220.PROD.OUTLOOK.COM (2603:10b6:510:2d9::21) by MN0PR12MB6104.namprd12.prod.outlook.com (2603:10b6:208:3c8::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9818.20; Mon, 20 Apr 2026 09:39:23 +0000 Received: from CY4PEPF0000E9D4.namprd03.prod.outlook.com (2603:10b6:510:2d9:cafe::2d) by PH8P220CA0038.outlook.office365.com (2603:10b6:510:2d9::21) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9791.48 via Frontend Transport; Mon, 20 Apr 2026 09:39:22 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=satlexmb08.amd.com; pr=C Received: from satlexmb08.amd.com (165.204.84.17) by CY4PEPF0000E9D4.mail.protection.outlook.com (10.167.241.139) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9791.48 via Frontend Transport; Mon, 20 Apr 2026 09:39:22 +0000 Received: from satlexmb08.amd.com (10.181.42.217) by satlexmb08.amd.com (10.181.42.217) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.17; Mon, 20 Apr 2026 04:39:22 -0500 Received: from [10.136.34.119] (10.180.168.240) by satlexmb08.amd.com (10.181.42.217) with Microsoft SMTP Server id 15.2.2562.17 via Frontend Transport; Mon, 20 Apr 2026 04:39:18 -0500 Message-ID: <1230f5df-470a-4e59-8c8e-fa159a6fc093@amd.com> Date: Mon, 20 Apr 2026 15:09:17 +0530 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH 1/2] sched/fair: Prefer fully-idle SMT cores in asym-capacity idle selection To: Andrea Righi CC: Dietmar Eggemann , Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider , Christian Loehle , Koba Ko , Felix Abecassis , Balbir Singh , Shrikanth Hegde , References: <20260403053654.1559142-1-arighi@nvidia.com> <20260403053654.1559142-2-arighi@nvidia.com> <64fe32e0-d428-42bb-beb4-2656d8781b0f@arm.com> <7313ba07-7b87-447c-9c48-2f6b2b53ac94@amd.com> Content-Language: en-US From: K Prateek Nayak In-Reply-To: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CY4PEPF0000E9D4:EE_|MN0PR12MB6104:EE_ X-MS-Office365-Filtering-Correlation-Id: f730610a-1f10-4edb-1337-08de9ec0b42c X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|36860700016|82310400026|1800799024|7416014|376014|22082099003|56012099003|18002099003; X-Microsoft-Antispam-Message-Info: foe3igc04oCDWjNjEU+oki83OJlgZy0Joj2HUSuprPqXSeOind6KHTMFXjq4ce80VbVfYuvXNHWrPOIydcOGsOUd8dnaIFDgvIzpBLHh6cM/UV3pskdh98IkQdXh5AhX+YnpkJQ2h82m4outshfrl0j0y5pnKZAz7sfLKjI5o9qz5LtnZ+AMcBuJGKWjAUSa9fr8LcZc+P1CWQ16kB4WdChCh0VLeKXKX04ySmn9UE6ycQSMq4Tr3Ol+fgenO3ynwntG97xvqcAWkWtteWjP2ysrkYoVasLgMY084rZGl3+VFQyiiGKBUIZz9up3YzZk6gp5XYN8lj3wJp6UJ8YV+eoVjQZp9ippllu1UPmSSgtrT00qrCz0iSERZxMt5xpR+cFd0jcuRo3ePs2oThhBJgdsq0vMDDX2DZX8+qCicy+VUv45exwP75680Mv+LKFhNV3jGv/CGzfUHUNYFSZh0NwVoZm9RTcqCKiDeL0QVoEHChPJ2YqO2p/9N4BFm9Vinw+HrbAh1idREY/kLMbC0ymCBX3BlWmMca0g8ddmvHxFj2ZaZU/aIJNcdVcoFQVVfSbRO5UY4GO9YkEHa2IS9ZMqYAZRzwSu2dYzplbZR/79BjCMdhPx/kvZTxDGykpfVKr/eCWsa0nZxc73us+Fk0YBaLKBH9GOILA4o9DKQHwF/z7u5QVn3gcSWuZCk8Lh5oB98Ac3NIE5lbvh5UmCYQKFGbZxKueKlXwpH8Klj+FX8ty2vCWZPCWZsSqaoDlPainmk6G6PE8mK3vPCQF35g== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:satlexmb08.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(36860700016)(82310400026)(1800799024)(7416014)(376014)(22082099003)(56012099003)(18002099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: KLNjLRp9yL1J26N7nAibaKGEdgWQHZsyH5Z4CSM7Pie3m8cjJeRZOz+UmDNS5/9f1CnVz50tefIAIfQlrS5EgYKwAXnAeJMBGXsYgHOhTRdyp+8hb5TOr931V7VTSiRgbA3dKImv9/9y3rrdonTLK0C8ZpYQwgTv6rYUq854oK9Ic6JSxLMWvhHeEPS7nfpnaDz8TV7zaXwQU2cLVHNIjONy7pnVkcnNkSHYhBFAGWqWxwpe75GKqE9TBAE2bJckXi5hzJ9Q3kQw5BzJ7hBjyhhJQNJBkYH3rOrYFJcOKCO5yo7B2KG83/KVVJ6tWW2Rsvz2iMBy9GI0ubNhxnKyTG/2EFwG/Apoin4NNv9prdTInBbY+euxPxnz8o7xuhtp5ef25XWylCAqk2MQqTDx/gLejsN95wv5UKXCsStmUqfFrFn7CfgnkN57QG547KU8 X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Apr 2026 09:39:22.5991 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: f730610a-1f10-4edb-1337-08de9ec0b42c X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[satlexmb08.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CY4PEPF0000E9D4.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN0PR12MB6104 Hello Andrea, On 4/20/2026 2:06 PM, Andrea Righi wrote: >> With your changes, only two places actually care about test_idle_cores(): >> >> - select_idle_capacity() >> - select_idle_cpu() >> >> If we go into select_idle_capacity(), we don't do select_idle_cpu() so >> the two paths are mutually exclusive. >> >> In nohz_balancer_kick(), if we find, sd_asym_cpucapacity, we simply >> don't care about the sd_llc_shared->nr_busy_cpus during balancing so >> that begs the question if we can simply track idle_cores at >> sd_asym_cpucapacity for these systems? > > Yeah, makes sense to me. I was planning to test something similar, so thanks for > sharing this patch. :) I'll give it a try and report back. Thank you for taking it for a spin! [..snip..] >> I still have one question: Can first SD_ASYM_CPUCAPACITY_FULL be set at >> a SD_NUMA? >> >> We'll need to deal with overlapping domains then but seems like it could >> be possible with weird cpusets :-( >> >> But in that case, do we even want to search CPUs outside the NUMA in >> select_idle_capacity()? I don't think anything stops this currently but >> I might be wrong. > > My $0.02 on this. > > In theory it could happen with unusual topologies or constrained cpusets, > although it should be quite rare. That said, select_idle_capacity() already > operates on the span of sd_asym_cpucapacity, so if that domain crosses NUMA > boundaries, we're already scanning across NUMA today. This patch doesn't > fundamentally alter this behavior. Ack! I was just thinking loud from the topology standpoint since sd->shared is not designed to handle the overlapping domains like sg->sgc does but we can probably figure some way to make it work. Using the ring topology example from topology.c: 0 ----- 1 | | | | | | 3 ----- 2 Consider NUMA-1 below gets the SD_ASYM_CPUCAPACITY_FULL flag: NUMA-2 0-3 0-3 0-3 0-3 groups: {0-1,3},{1-3} {0-2},{0,2-3} {1-3},{0-1,3} {0,2-3},{0-2} NUMA-1 0-1,3 0-2 1-3 0,2-3 groups: {0},{1},{3} {0},{1},{2} {1},{2},{3} {0},{2},{3} NUMA-0 0 1 2 3 The "sd->shared" assignments at NUMA-1 will put first, second, and the last domain in the same "shared" range by today's logic since the first CPU in their span is the same although their spans are slightly different. The third will be standalone since the first CPU of the domain span will be different. > If we think cross-NUMA scanning is undesirable, that's probably a more general > issue in select_idle_capacity(), rather than something specific to this change > and we can address this later. Ack! That is a tangential problem but may require some looking at if we decide to extend the sd->shared object to SD_NUMA domains. I guess if anyone is running such setup, this bit will be the least of their worries. -- Thanks and Regards, Prateek