From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from BL2PR02CU003.outbound.protection.outlook.com (mail-eastusazon11011057.outbound.protection.outlook.com [52.101.52.57]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 296D11DE4E0 for ; Fri, 3 Jul 2026 17:07:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.52.57 ARC-Seal:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1783098452; cv=fail; b=Tfoo+IHD9PWS59g8zMGFoakJvSoM8JxISUBSWMcRM2rHOuRUAW0LAwFPW5HRmj3W/wuHg5tn63jVnocD4yuuDgFUPNtn56ORwOemVGNnhcHPxupMoteEL8jmC7gFW0Nl3J01kPCwCjdM1aOBa95uk8sgmJniiy32bxOMwPtZxzA= ARC-Message-Signature:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1783098452; c=relaxed/simple; bh=faC8DWb8Gj6uiRJcAv0ggGMgoZ38aJNEgExO8yreGxk=; h=Date:From:To:Cc:Subject:Message-ID:References:Content-Type: Content-Disposition:In-Reply-To:MIME-Version; b=awqAD6dfH8YOsvWhbM1FjW9v6rrTfc5YuBFErU+x76vGpXx+b+NBet3zyS+c+n2cAcPu3a4P768zXAtLr5khdIu/+ujaJP/cmFDtRTlEMOhHRlY/TIjZ/7B8gIdrp4f+ObUiGy0V+MLQf8XM7ssPyMZyy5RlIeuevCoWsBt4SW8= ARC-Authentication-Results:i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=eNG1bRjk; arc=fail smtp.client-ip=52.101.52.57 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="eNG1bRjk" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=wsghvjaQzaJmeVvl1wBfm2cTDhalEHMDOkPZw7iF8KH8M++dWTRGllpxIDOoBb7EW4P4KEYaEG7a4NmNIiBdWxCL+KCiLOJtIbN1FH5pa7CctcyxUNkzh1ho0xNAktqQsW+/jB5UPa/RUrAt/LviZEIQbHpY47afzu/S8/heNpZCPXEU1gWMEenV+TnS170yrNFMY60XXMr126tS7NpGnftGS/B12eyv7Zht1jMSrwyEuICgsBX2z8QKKMjwbwFBMZ0rP8w7xIU6mrxNGa2d9oFAPlJI75RlOgMtw+8OUVSbmJr0ZDIwq5CRpaN+xM3gExDifTJCIivpNp4r4K/4fg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=I6XeZ34q749lY/leJtrejARKlJXEmCojSsnoioTt8Cc=; b=w6hW0H841Cne2hWur7+iobUU4Z5TrR02PERHU+lO5G/OwlMQNT7SiM/Vc94GhAx93ciS4ylqjU1FOI04G0CXmVQVxKFWho0aHF85658zB6JARXnY1O3667Y7WTMiuoQDQBUZ9bytX0CdWyvB4z9paIm1w2FpDsN1iOxtdkiSJUoI01F80JlPIemwW29E33ylvb6jVdLfWIsU4c/2Oc2ywp4NWi8OflSbL/I5AL/tRjx6PTSq31gRBQ58lzixtTJNN/7SWoNLoN1jUF/5anFRCcQwcH3ucBiL7+PqU2rorceMQ4uoqYduu18QaOO3NknvnGgpFWLx+EFo4B81M5eeNg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=I6XeZ34q749lY/leJtrejARKlJXEmCojSsnoioTt8Cc=; b=eNG1bRjkUGG4B+p0TJyg8teqIEHKYBKSe23RUyrQ5PUMHdQs1S8XUTYVX/Dwgu7I3AJs8uRu+KUMhqRKDdiWyt7wttrbl2+VD2KCK2QSIiR7dNsKMJNnUpirpCbKGvG5WZ3EfI0YatsY7xXNXAGUY+Zcjp0hvcL9RTTyyKShRcLmVN0+iv9QcJTH5UjK9hckUMBaQq8tmtwwaoV/Rmc6k6YTNrQqVD5zbQwrWU0uGKBBtH8y9WcXjHPbzPKa1BoCR/xFmePaw+tjEP+KbzcqITwTq0guwbzrz+5mkaZ2H2aBoidPOCm8H3zPOv+VUYFLDRku0DtfQPkplpIU1eYuRQ== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from DM6PR12MB4827.namprd12.prod.outlook.com (2603:10b6:5:1d6::14) by LV3PR12MB9402.namprd12.prod.outlook.com (2603:10b6:408:213::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.181.10; Fri, 3 Jul 2026 17:07:25 +0000 Received: from DM6PR12MB4827.namprd12.prod.outlook.com ([fe80::6261:3040:864b:159c]) by DM6PR12MB4827.namprd12.prod.outlook.com ([fe80::6261:3040:864b:159c%3]) with mapi id 15.21.0181.010; Fri, 3 Jul 2026 17:07:24 +0000 Date: Fri, 3 Jul 2026 19:07:13 +0200 From: Andrea Righi To: Peter Zijlstra Cc: Christian Loehle , K Prateek Nayak , Ingo Molnar , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider , Ricardo Neri , Shrikanth Hegde , Felix Abecassis , Joel Fernandes , Phil Auld , linux-kernel@vger.kernel.org, Julia Lawall Subject: Re: [PATCH] sched/fair: Stabilize idle SMT core selection with asym-capacity Message-ID: References: <20260630152747.128746-1-arighi@nvidia.com> <2ed258a2-ac9f-413b-aa39-59a59cdee1fe@arm.com> <20260703165414.GR751831@noisy.programming.kicks-ass.net> Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20260703165414.GR751831@noisy.programming.kicks-ass.net> X-ClientProxiedBy: MI3PEPF00004E9E.ITAP293.PROD.OUTLOOK.COM (2603:10a6:298:1::455) To DM6PR12MB4827.namprd12.prod.outlook.com (2603:10b6:5:1d6::14) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6PR12MB4827:EE_|LV3PR12MB9402:EE_ X-MS-Office365-Filtering-Correlation-Id: 76e1361d-2f62-4722-fbc9-08ded9258da1 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|23010399003|376014|7416014|1800799024|366016|56012099006|4143699003|6133799003|11063799006|18002099003|22082099003; X-Microsoft-Antispam-Message-Info: Kc6LGBMXck0oFZY4LcPQlIITc9U+K7Myqu5tqDFlWTP83eycEdoWxpN2WnugNFTx/kO7u+Lu4SNi/SzI6KeRzHSE26jb63IIpCYrZJUGs0yilzLaBboWfghNrfr/w6fIWMtzdPx51ZfJNb5avZUJSae4WXcWbdMmNdXuYsJCjXC/EnSPM3SwwcAwVPYyMCa3Unjxz0DrLhpb1IZQxilyfBLyWJwv9AxCTZGVOEW0Er4El+SQU0vtaXySMhEecgTVX7lRI38Ph08b4BOIO+84jYpStFmWXffYv5T9Y/pvd8+EUx5jLMiqyLqLTP7Hm1j3sxeNXLjwfkT/ZOng5ztA4Z/Z83Aqz2Q7Lu9NEasuvz+R4IVuM/MR3RPK+HLSs4k+qN3L6q7alVSvyW8aA3vQCIPq35d6wMKEj7tB88skXToRG+EvyuB5FyI2r5UjzIQBZs6VkhUH3KpQ39s44gwItbZwenkG/tTczsFt2w+5Oyicv2Kw+YKmPZm75P1gl769zOqUIYuKSDY1es2/o+lcrRm5EzrsOyXqDKXaT7K80JctOuuDWyc2LTWxjFY4OCMt1fB4CDWffhjCwekpIVX4tU4MZSiXRvE5s6mOtsH2qrwlIUKJ35ywa8CfV9yVckIG25o6lZixZhwNOFdVc5J/5k2OaNCZH5iomflEKqa4XRI= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM6PR12MB4827.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(23010399003)(376014)(7416014)(1800799024)(366016)(56012099006)(4143699003)(6133799003)(11063799006)(18002099003)(22082099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?sq3BXnE+WGOTjZa/XfpptOA7ztNoWkSCIBoeROFTJBwHHA9szkkBKYBtxsTY?= =?us-ascii?Q?x3gDS64V8d8metmUx3Z6D3SSwu7U3itiuiaRzGCH8azneXYw5qEpRNBvpIQH?= =?us-ascii?Q?TsAjmHGmrCcKWWtNSr1LjKSaIfapbcDtDHljqZZuuBvaoTH4RtWMvuNmdiU+?= =?us-ascii?Q?aATNGvZVhGLjU/cqmJ6sBXJ4RqMWx9/FDLl4XljevUfPF1Aw0k1ICr+bpkH5?= =?us-ascii?Q?ZKAJOQvfpPGw64ra96PVLIGdW9FpigcsFPHfEquelUFe5dPqvUwhcfFL1yjU?= =?us-ascii?Q?uFcDM8Bb908cb37IwMsVpioOW82FEyft7jjnkeCPJ/0ZhU61qEvjwrb3FKTa?= =?us-ascii?Q?SvUU9EP2cuy1ImRFOtfx3hRUC5fGTYA2JJX3Dg9qIUkvCZcZo0yZ4ykVKbcZ?= =?us-ascii?Q?LwiGc2Nx7lPbyUZ3/pow/M4i+yscXGTe/xYMLLETs4yB84mMHz7avbD//f80?= =?us-ascii?Q?B13jxSheRucOjXuzb20WX5nlZ5jLo3O1053VYs9XDri0eFj/08LnODQ/sl1y?= =?us-ascii?Q?KgQFYmb7QmqQeLb5uz21g7CU82gapRiRckE2PXju3RBSO0KkISret2rlU1oC?= =?us-ascii?Q?NKw6GolOOmbkQ/wpDVqul4g2640+yD/gami7ozIv8eA58kkEBmmtAVmxbiQN?= =?us-ascii?Q?0CIeJRKRb7QcILRAtKNt7SGLk59EuOrL2a3IaYXiIaMQGmeHz0IM8ICVUGWh?= =?us-ascii?Q?+gyl9RpXLNV8Vwu0uyxw6zqPiO+4LwDd6MTK4nzjjhvhxFEdqFznEydGpf3R?= =?us-ascii?Q?SjyMHM0faNEA64IPKKSCIjHW3FaBYoPwNklvuxDafidXuo0h2oz5PSungULl?= =?us-ascii?Q?UYcNDM05YZdOa9goE2wWbX+A8kcoPzChVGi/snh/DB3u9w9zu94bQ+qa3FJH?= =?us-ascii?Q?nICmac93OIQ57GxVg8E59OIts3l+SLktyLlLyhIz8TUuYDyTIjfoxT+l16ee?= =?us-ascii?Q?hCHjJBy66h9SLUDxmDgqbywY8zp25oaS617qM/O5TUgZb+p2G+dxx/SZhtAA?= =?us-ascii?Q?MBreulM7ORgi7EvNddPSLqkteMEIVa08maeLgaAxDhewCAw4TXm2TAtugB/Z?= =?us-ascii?Q?tOisSrpflWNG7+aHieQutxe2XTFWXyd4jjZUt5ddP2A/EIl2bxtDZJhcaC1O?= =?us-ascii?Q?NFvwyqmnkbBgcbi3t0ivmhUJzMlEc5pPOGZMW36xoCH+MSQqf587hrj2Ax8g?= =?us-ascii?Q?AVldbCzBTTbF4zieBt8nzosJgFsx4n8QzZM/vTFRiV/WqQSjmv6AgTeTcsyk?= =?us-ascii?Q?nRsap33wV6oQ9YDMX7aHYUa8wbSIXXjj+2Yb2tE1hlPN6/SIUbBFHOovVtcw?= =?us-ascii?Q?Wdh/yjRH9cygUzY+rEe2pWJFOc3KF/R0MIwqgaLqeGgDXj5CFpde7+0BXFgZ?= =?us-ascii?Q?SOcEZQJUBEfGSuoOPoYexuvQwJbCQZrgRl8kGkrTOSpp/S0WNjyx8qjPul3Q?= =?us-ascii?Q?AZX0FYrMJiYvm5u5ITNiNXPkD3mxpetQVSnAUlqy7HakHEDU7OMizZAWd5pG?= =?us-ascii?Q?0Q3sNC4X+JIsfrD/jTdZowgDKyWxeSEysvy72/NQdnL7RYjHkrUPTJTiMJcX?= =?us-ascii?Q?9rU+2sD7opN8V4WxwKdg2T9tImS7tdOTNU0ol4OxuAOfrxsnx4CRtTsSr7Iu?= =?us-ascii?Q?r8X5HsCbrtjPdu7rSoqLRuC5F3NYXoSVreyzWbRornYTtr2NE03Pb9uVSvZY?= =?us-ascii?Q?yTO+58SEVF26dUPEcSI/OLCmODFy+RnCwRwoGqHL46A3RNC/uCVOW7gwhJmK?= =?us-ascii?Q?sZA5zishjQ=3D=3D?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 76e1361d-2f62-4722-fbc9-08ded9258da1 X-MS-Exchange-CrossTenant-AuthSource: DM6PR12MB4827.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Jul 2026 17:07:24.8088 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: QYuBc0ZlBUFWROmUzgOxF3z0iTJ6SIpeF+KqvgY5RD+JcqbdRg535cEcH8ps+OvCLlcdntZ8o3T12iLa21lKfA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: LV3PR12MB9402 On Fri, Jul 03, 2026 at 06:54:14PM +0200, Peter Zijlstra wrote: > On Fri, Jul 03, 2026 at 04:52:17PM +0200, Andrea Righi wrote: > > > As mentioned in my other email, I found a surprising asymmetry on this machine: > > pinning one worker per core to the first SMT siblings gives substantially better > > performance than pinning them to the second siblings, despite firmware > > advertising identical capacity and frequency for both. > > Cute, that's something that Power7 also had. That's where > SD_ASYM_PACKING originated from. Yep, I'm actually experimenting with a patch that mimics the Power7 and it seems to work. :) But I'm using a quirk to detect the particular CPU implementation to set SD_ASYM_PACKING on the SMT domain and assign a higher arch_asym_cpu_priority() to the first sibling, which is not the best... So I'm checking with the firmware folks whether they can expose the relative SMT thread priority explicitly, so that the kernel can discover the asymmetry and the preferred sibling, instead of relying on CPU type and enumeration order (considering that there are also multiple SMT configurations that can alter this asymmetry... it's not just SMT on/off). That said, we can ignore this patch for now. I'll come up with a better solution, hopefully. Thanks, -Andrea