From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from DM5PR21CU001.outbound.protection.outlook.com (mail-centralusazon11011068.outbound.protection.outlook.com [52.101.62.68]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F19BE2153D8 for ; Wed, 1 Apr 2026 13:12:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.62.68 ARC-Seal:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775049152; cv=fail; b=f0rGUc5GHwWtpnmw9pDS9AR0ITHGkZxvF8EdlLxYsFs5l5kx6ois2McTsLhYMkx9DaBER7fYyQymhxAHnkZDuHSMMnNyB3FoZgvqopiftDJ1J6oKwJk40luvu7v9c2TRehGY7GohdhMx0mPeDCw2jAqquDLTjbLBD40/AyWl8/s= ARC-Message-Signature:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775049152; c=relaxed/simple; bh=DT5iRh7K5VETG1pIZ/Y4vTzfBdVV+hhUI63mz0XB6lQ=; h=Date:From:To:Cc:Subject:Message-ID:References:Content-Type: Content-Disposition:In-Reply-To:MIME-Version; b=msC7eXt6DDEwP3P0EhIRaztTenc4CdVJ3GS0p3NhOpn9fw+eepDLUv+Y+UT8Lw4eMqOpsmGL/791YpLUV/S/kV0L0tALcjPotgik7RfLRhX32vK3dC1DHTeftMvgmNnyBuNw0ImvFrF2dA+eN5l66GMnQm+5z7zoKlPvDhdjd9s= ARC-Authentication-Results:i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=uRnXA0bj; arc=fail smtp.client-ip=52.101.62.68 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="uRnXA0bj" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=DgTggH6a2ophHme7Ni3GYqvkh41BXOGggJr9reYLmO+KK7ZsJ1APlIp5gI6UXycrLPSYvg1sTCQ+LabLiQCG/lPif/anCtVAIgYqFr7peHsLG6xjKOjWy3UzN0Ku0SphAcinUaykTJRKOJ7X68TDTUxG4iWC6hw1ssGMtahXxG+KJkfHr1jgfVY4YCiHvp+JvgF/7TApRl6CWLCkFf17nWNI8b83jgFbnKAn9y7anKooDaijNsMjUb+/SvgqzQilQRaIQE/ukXnj5WMzO3AVW/w2P9umLBWN5k7ttVWBjDiGupVY5P408hM8vbSKOXAVQtWJvehkRlZ5YFiQ3YX5Jw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=+HT7piBOy4F20w+ZmcC/mteCaNkeHstgvdicfxey6Gg=; b=jFspBB0X+eKG/YWZd3dobfIgjzxz7qFDGJm3zdhuiPQvtNnFdpWzyHsQ34pX6VVp2YCQiwdTDlN/oJQXyWOCF2svGZ+ENA6emf/uddPXfgAxck95So89/lL3U6AylNohgqaaD4S0Ea5uj+31oC8yVeCEfdYD1G/J0tJYf0rTbqW8jT4YQCqD8wHBvn7wgG6vmJGvE/zCKreS56ToxJnfsB0cyPLwv+Thy6N/mHEqTW4R7pcNmxPu9y5DzQhVs22NOJbqLDF4ofErD0UFz5EDLTbE+d1QuuC5YzMHvHHT8CWBXr1L7N6cLfvMedmXCpqSciR4jCXDwCXgvYzr4lOo5Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=+HT7piBOy4F20w+ZmcC/mteCaNkeHstgvdicfxey6Gg=; b=uRnXA0bjrkuT53Ch7BrXkD6AGk1BtqsQ1ikmj+jprbUUPOMHxBQZZSOmq29PNemm8JhdGXw/jvLL5sOD5oeWmGnetII6DCZCgjcohUK2o0klTOy5Exuj6v+yVPv5QKuGZjlYzg2I7YAr9wH9mSr5qJfuiZZJ8TWNZFhw5U8EwQkGuAKmV3IY9YZlhQoZQRvxTnRBsqRsg90DLKntyUOmkw+ME/FvWyzWkPEUdAsViIWLIXv0oo+1pzonuEUqq/bsnV0tm81fLaB7LgBXCO6VG/xkjSf70Zl7P0B88HfdAx4CE/pd5XvRnm0PnhHXj1kKb9xWGjRWO9lO/8NMCEyrDA== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from LV8PR12MB9620.namprd12.prod.outlook.com (2603:10b6:408:2a1::19) by LV5PR12MB9780.namprd12.prod.outlook.com (2603:10b6:408:304::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9769.16; Wed, 1 Apr 2026 13:12:25 +0000 Received: from LV8PR12MB9620.namprd12.prod.outlook.com ([fe80::299d:f5e0:3550:1528]) by LV8PR12MB9620.namprd12.prod.outlook.com ([fe80::299d:f5e0:3550:1528%5]) with mapi id 15.20.9769.016; Wed, 1 Apr 2026 13:12:25 +0000 Date: Wed, 1 Apr 2026 15:12:17 +0200 From: Andrea Righi To: Vincent Guittot Cc: Dietmar Eggemann , Ingo Molnar , Peter Zijlstra , Juri Lelli , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider , Christian Loehle , Koba Ko , Felix Abecassis , Balbir Singh , linux-kernel@vger.kernel.org Subject: Re: [PATCH 0/4] sched/fair: SMT-aware asymmetric CPU capacity Message-ID: References: <20260326151211.1862600-1-arighi@nvidia.com> <193f1cd1-ced3-4b37-83af-ea43a7e5e3d0@arm.com> Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-ClientProxiedBy: ZR0P278CA0065.CHEP278.PROD.OUTLOOK.COM (2603:10a6:910:21::16) To LV8PR12MB9620.namprd12.prod.outlook.com (2603:10b6:408:2a1::19) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: LV8PR12MB9620:EE_|LV5PR12MB9780:EE_ X-MS-Office365-Filtering-Correlation-Id: 197838d2-9356-419e-9f18-08de8ff05151 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|376014|366016|1800799024|18002099003|56012099003|22082099003; X-Microsoft-Antispam-Message-Info: DbPmDBSXX7/EG2gFKtp5NTGWhVjMpKg6O1rC1mMawEGFWOmUme6MbV3bM42u/A64dE0Z/lN0n4Anz3DGaQv60gDKLzzu4INUxCGTJxkGENilZe1tPe6EzmDwi5NF56CViIzyguWzHLF+XY4d9WCAbOLVx63QDL1TgD2eDPWSbk1qqfrKE8VN/ZwAKHf7OU6hciReJ+SrxccMXhmBGGLBTv4nrj8WTSngaIxP9VE2LiPpEv7Ijcgzq7MjBXredWEDn0cKGj90mcyv4TvrxX3TN316o5X672Jutnk57DSSVoYq12cq9C3eAceGMRvH0rJk7NHsp2rvzXAEWBxrFcToqhaXIB0iXYp99NNlzhGJDzf3Qt8K87bIQAg4xUWhHqkzdb2esv14iHNmHLwOtXJrLJHSK9RFuEwVxPN5xvzHMyUs1pMYOghrD66TnRpJW/xky4gEUrj1XH6NJiFBSsVOLIplKH33+NN+7yZUf3Y9MlVNrDcjqq7U62zYm/Ir0bMMYeofN56GevFP1uvdW3DYmAWjjZFyLEdA/LGtpbynLi0P9cke3Oy0iTL/IQefA0I1/P2PMTG5F945GIBRASaSIe7GgJup+K+1+K+GmOnX+MPoph3gfCk27ZfdcMnKhh7TEcx7Aq8BpOkGPsZMjmrLIV8i3sLlAPt8BJXT19+kiLldDxdsDnp+KU1o/FT33gY+3JralUmcFh5ycS1etayby+XZeYBm1zKntF28mVrR1lA= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:LV8PR12MB9620.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(7416014)(376014)(366016)(1800799024)(18002099003)(56012099003)(22082099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?tNkfG1wJ2jIKcCW3AG1N+Xtv6rarmX0sDhIsU72bDFAVVh2R2/PFM/+4Z7pD?= =?us-ascii?Q?Bo4GHQLEMjQS5pRX7k24mOlSkXPI0Nw60dsZIIOgTvDc7gfOVs0r0I896VZn?= =?us-ascii?Q?FonG7U7ouwgIti1THekBFuZhvFai5MAX7MJLb15WyPV5TloBchWC75Tk81W+?= =?us-ascii?Q?QuP9P5ljsQaxU+MfKpZAKu80JpX+aabAJI3TVVIRweiJhsbtJBwsB2snle7Z?= =?us-ascii?Q?AEgihI/Ig+e1o9ce5R44akwpYqoJqGFXfKf2yrKNjbcxdE+9cwKwZ6YUf61b?= =?us-ascii?Q?1sEiEoCC7P1GNVhDkhluUsVpV9NDXjwcx2+z6NOZNoJf9Akz+aCqh7xH9ymb?= =?us-ascii?Q?52VdRDuFLuCIFZU0Fzmu2Cyj8bQBglvJ/BnPo7/ynSbCGdPgH2UrKRtRpy70?= =?us-ascii?Q?7mUlzZGZIWwjmMI6797ZI3euq5v1KSVOY5+YBkBX0TlB0keJDK0gNoix8YoJ?= =?us-ascii?Q?EJu+KgrTl9O1lj0gzuj2Rc6hO3U4obwr2gWMon8hBF9SgkcZV3+spsIWNcoL?= =?us-ascii?Q?b0dOiG2JQIEJNn5uOly1Na3Nwsumt50OBgv1bu1cIaPdFCYCTfWSAGKRIUw6?= =?us-ascii?Q?qoZWVue03/vPD4g8a+duHGrVzw6+KIZ5mh0c5GjF8wrKy2ZI0lcXMZtykryr?= =?us-ascii?Q?CsKCKSQE8IoGKyi6QQU9gxOzuONKAQBPnhwIq4+Tv3qjVCY+6vLUkSlxCT4Y?= =?us-ascii?Q?e/DUDpW4Eu9/ktQ1vZA58ulmJKBZd3J8y8CglkCLpToaHbunJNTLyuAMYgSc?= =?us-ascii?Q?HLxTCilgesWBHkA6NT3ESQuVbdDCiHY5MmFUTMkvz7e2/TWctFTg3wObdDE9?= =?us-ascii?Q?gW4qaYVX9LFtWKi1LphSt2oKojFjMi9NQxc6dobpCOasQRNd32hlE1m6F7q7?= =?us-ascii?Q?W9DfQy0JQLvFwwgaGhsvDCC1C+Q8lLv5gij7RysSQX/HS1r4EAROVPXlqUlW?= =?us-ascii?Q?mztyP8tK2hrDWLiQPJr+5Ps+iN5IGemoE1CiA3hSqgtJdMUmLVC6aLMTKI8y?= =?us-ascii?Q?TXU8SWKIryVshOZEXSTJfSwPB8cLBs47p9edrGKcFvcopUuUerA8I6AsT6lt?= =?us-ascii?Q?qWgpmkgk0V0Ypo9ZcjtINeJnwHEIz6JK74HOUDsVN/YuFmdLLZaCScQwTmOY?= =?us-ascii?Q?XirzJeE0uzPuHnloXOF63qhU85LEExX1Pf/+9C8h//iKi6R3pOHpDQqOZdhw?= =?us-ascii?Q?EnW7wgohBnH5HHU4IRbXGa2ey5qVqBqiqu+vgJereQVyGzazUmsyHW5bo+He?= =?us-ascii?Q?hfrszDusYdMHwusWS7YGd88HBvmtVQh4nBOysMy7j+7Dntt0f4eLEjHGEzMW?= =?us-ascii?Q?LvpKWtjYaF7puSqXggHjJj/fXkho4MdanWUiBNqWCkiB36MbZjYtMbpq+aJM?= =?us-ascii?Q?IinIlCAt/k6l7JByOXwKBk4BkG3h1pQTkqD+VNyl8L/jmgnalZROrV8IuDxA?= =?us-ascii?Q?og6IGsxSebKu8kC1pciav/ld/qTUzuYvZaOCG6wzRWSCLYGFKVkE6zql4sih?= =?us-ascii?Q?g/kUKPQE9nm0nWBelzoPgsQe9m67UuOdfXSbQlUlE7UQx5Cm3iZJbtoVeI1f?= =?us-ascii?Q?XuZZnNEAm7PoiPTMHoAhVUD+jiNF1mzEZhIUu/KzIBI2YnxO9LAslC7A1Amj?= =?us-ascii?Q?kvLA3TobTC5aZ3vjHq1CX7rCDnL6XrS8mFEW69UJX4k0izWFJZgkQaqz2Ril?= =?us-ascii?Q?6x+gY9nMCxJ3Ynz1sS1RimS7QqVnhodefNo2Cegb1uxVSgaF7HWoALgMjcoj?= =?us-ascii?Q?IusSbFkvcg=3D=3D?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 197838d2-9356-419e-9f18-08de8ff05151 X-MS-Exchange-CrossTenant-AuthSource: LV8PR12MB9620.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Apr 2026 13:12:25.3812 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: TBckvcTw2Adj9E37tN/DLWgeXqNpDMhs9npTGvsUZJdh8PPXxZXFWTMvjE+9wNiNf+KUR5PMCUHfJKxvcloiuQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: LV5PR12MB9780 On Wed, Apr 01, 2026 at 02:42:34PM +0200, Andrea Righi wrote: > On Wed, Apr 01, 2026 at 02:08:27PM +0200, Vincent Guittot wrote: > > On Wed, 1 Apr 2026 at 13:57, Dietmar Eggemann wrote: > > > > > > On 31.03.26 11:04, Andrea Righi wrote: > > > > Hi Dietmar, > > > > > > > > On Tue, Mar 31, 2026 at 12:30:55AM +0200, Dietmar Eggemann wrote: > > > >> Hi Andrea, > > > >> > > > >> On 26.03.26 16:02, Andrea Righi wrote: > > > > > > [...] > > > > > > >> So does (2) with NO_SIS_UTIL performs worse than (1) with your smt > > > >> related add-ons in sic()? > > > > > > > > Thanks for running these experiments and sharing the data, this is very > > > > useful! > > > > > > > > I did a quick test on Vera using the NVBLAS benchmark, comparing NO > > > > ASYM_CPUCAPACITY with and without SIS_UTIL, but the difference seems to be > > > > within error range. I'll also run DCPerf MediaWiki with all the different > > > > > > I'm not familiar with the NVBLAS benchmark. Does it drive your system > > > into 'sd->shared->nr_idle_scan = 0' state? > > It's something internally unfortunately... it's just running a single > CPU-intensive task for each SMT core (in practice half of the CPUs tasks). > I don't think we're hitting sd->shared->nr_idle_scan == 0 in this case. Just finished running some tests with DCPerf MediaWiki on Vera as well (sorry, it took a while, I did mutliple runs to rule out potential flukes): +---------------------------------+--------+--------+--------+--------+ | Configuration | rps | p50 | p95 | p99 | +---------------------------------+--------+--------+--------+--------+ | NO ASYM + SIS_UTIL | 8113 | 0.067 | 0.184 | 0.225 | | NO ASYM + NO_SIS_UTIL | 8093 | 0.068 | 0.184 | 0.223 | | | | | | | | ASYM + SMT + SIS_UTIL | 8129 | 0.076 | 0.149 | 0.188 | | ASYM + SMT + NO_SIS_UTIL | 8138 | 0.076 | 0.148 | 0.186 | | | | | | | | ASYM + ILB SMT + SIS_UTIL | 8189 | 0.075 | 0.150 | 0.189 | | ASYM + SMT + ILB SMT + SIS_UTIL | 8185 | 0.076 | 0.151 | 0.190 | +---------------------------------+--------+--------+--------+--------+ Looking at the data: - SIS_UTIL doesn't seem relevant in this case (differences are within error range), - ASYM_CPU_CAPACITY seems to provide a small throughput gain, but it seems more beneficial for tail latency reduction, - the ILB SMT patch seems to slightly improve throughput, but the biggest benefit is still coming from ASYM_CPU_CAPACITY. Overall, also in this case it seems beneficial to use ASYM_CPU_CAPACITY rather than equalizing the capacities. That said, I'm still not sure why ASYM is helping. The frequency asymmetry is really small (~2%), so the latency improvements are unlikely to come from prioritizing the faster cores, as that should mainly affect throughput rather than tail latency and likely to a smaller extent. Thanks, -Andrea