From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from SN4PR0501CU005.outbound.protection.outlook.com (mail-southcentralusazon11011025.outbound.protection.outlook.com [40.93.194.25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3BDB23F0740 for ; Wed, 25 Mar 2026 16:50:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.93.194.25 ARC-Seal:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774457424; cv=fail; b=Ku0lAXEu1M7gVbTw0HtItnoTR1hpJ+2GJvTqngGYK4EW4D7mJUPJcr2xbsHuUDCZu5wZ0Vkwslr557cVGOxA2miGZ4YYnxiVOd4y1o6f8sOXI2MIrRIGWzRDx8ULmxl/eoUzq9mi6txR43VIfV8Nocef02AZl44i9iR2DQ1mPzU= ARC-Message-Signature:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774457424; c=relaxed/simple; bh=I4MBrOwEiMsL/K0VHRSiWATWcCvXVBAK7nxq8ut4Gj8=; h=Date:From:To:Cc:Subject:Message-ID:References:Content-Type: Content-Disposition:In-Reply-To:MIME-Version; b=FT0+iufNh4bDjZGtHcotnel2c+Qir3FVjl8MdYF8Gw0Fg6SSUifQEGVbYlvAvgPOpTUIzXInqrjuMXIgrdt3/7Q0eF+G2VnZlM32vAk3ui5ERSP4WBmjxdHJyPsTkwMhOBf6QJpf/wMxGJHyWmPYL0p/X6hJ3aebwIa+ppVaRKI= ARC-Authentication-Results:i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=B6ec7TPp; arc=fail smtp.client-ip=40.93.194.25 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="B6ec7TPp" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=GyPJkF3/PeyKzj7t6BfZ4Ui00lpyyd0eoudoqB30D7HlCDY3XdHXNznT4/esh2rIhp54+xeksKapl4Mr72Cr5IapCGhqUdOmKUFa1Pd6Zj3l+dWErFgq0ovAtN73K+cmZd/BrLGCYOVNB4cYNJ2auHYDZ3acSy3SykHKtU/i5TukxhcEzbGhtIzGAi75HQTRJKlT1GMfnbCYBVTGcbdhhFYUpE1tJD391EnkHISfi49lsjXuor/v4wXvI4B36+jrTLnWH3q2yguz+YDKGe00Yd4wFliLYBvb/83P2vyC2ycwGhgWpSiuv3E0D0LCFW5hoSLkq1F9yigexm+T4mlmiQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=uHJ8HbmxQPZu+Q32pjvdIDxNxmrIPF4VFqBC0xXHH54=; b=LSO+wboc+4eE3lk2KUVhs9mIimCey4qAcsdWhEWf9+LEGayMQ5b0dj7ao5wtAqzgwbg2t/X+QWnV0AQfZcnvbLP7y4TwvrR5HOsEgjYTJAmk2R0yXwcz/YzvhHmYY1fvn4aRIfFPHIYZtwT5tCDhlGgJDJwLZZLRrTJVnyrQGlEWVWuSk1HPKMCrSV+nMtBGy9ar/3f9ySi+BTkS2EHYfxgHEqDYZH943Cpgu0IrkW5SY+qtilyq0D0TKFEuT0DntiGzj1M3gPdsRnDbigNlJbRCcn8aMHvgkIVS3XhjUEH9MQCOVk6bLGXP/GxkbSzpZDPjcoSFC+zGSef3RfwMQA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=uHJ8HbmxQPZu+Q32pjvdIDxNxmrIPF4VFqBC0xXHH54=; b=B6ec7TPpkkghNQHOgtZu90xaOBvzGpqbEttZkCVO+hYX7z9PL6EY1BVuxWGKPqg+jBfO7/TNHFs9TfB6hf5xoAfhFhi5vVufO+RSgpjaZIyz+vxxmedYqqxXKnEWpPm54s8H12a0az7AHM8TWal8LQDv3aeE7FqGsw5t1sMAvdVbm2hnbrSHP7SlHHIIDw+13xLeMNHI7BQXT8FKg25aDVoNAEJoX7JdXjWakpY9y+XC15nBLmkWf7lIKTfz8rLhtpvEBgo2BCI6Bg44GgD/ip7HRRg6zoRn/OwbjgXLt4FHg7r5sQNWNqpH3xDB1nmv3IwLc19vyhqMZmW7NHeqzw== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from LV8PR12MB9620.namprd12.prod.outlook.com (2603:10b6:408:2a1::19) by PH0PR12MB7885.namprd12.prod.outlook.com (2603:10b6:510:28f::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9745.20; Wed, 25 Mar 2026 16:50:14 +0000 Received: from LV8PR12MB9620.namprd12.prod.outlook.com ([fe80::299d:f5e0:3550:1528]) by LV8PR12MB9620.namprd12.prod.outlook.com ([fe80::299d:f5e0:3550:1528%5]) with mapi id 15.20.9745.019; Wed, 25 Mar 2026 16:50:14 +0000 Date: Wed, 25 Mar 2026 17:50:06 +0100 From: Andrea Righi To: Dietmar Eggemann Cc: Christian Loehle , Vincent Guittot , Ingo Molnar , Peter Zijlstra , Juri Lelli , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider , linux-kernel@vger.kernel.org, Felix Abecassis Subject: Re: [PATCH] sched/topology: Avoid spurious asymmetry from CPU capacity noise Message-ID: References: <0fb05951-1f2f-474f-9f7c-9f0f15a5f675@arm.com> <86cb3979-02cd-4171-80fd-df20cb3430cb@arm.com> <1d9b4abf-4b70-4775-92b8-924ced316578@arm.com> <9102f6e4-2360-4999-a036-c92cd039e5d4@arm.com> Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <9102f6e4-2360-4999-a036-c92cd039e5d4@arm.com> X-ClientProxiedBy: MI2P293CA0003.ITAP293.PROD.OUTLOOK.COM (2603:10a6:290:45::16) To LV8PR12MB9620.namprd12.prod.outlook.com (2603:10b6:408:2a1::19) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: LV8PR12MB9620:EE_|PH0PR12MB7885:EE_ X-MS-Office365-Filtering-Correlation-Id: a4b9df7b-4005-4795-e7d8-08de8a8e95d5 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|1800799024|376014|366016|18002099003|56012099003|22082099003; X-Microsoft-Antispam-Message-Info: SzoZjnByM11d/dRfLOl2LOIYOzPsaDMzXLSCCl+jXrwbDiRpzS0Orc1+Vn6MrQ3N3ovCVEoA3yTS2BhkemrbE/u/OaRHCqaqzU1xV3qWfIlQ7+sOQobA8v72I05QBSlyio1wiTos0ukMHrD8KCHL6sgx/todDA+942WK9hR2ytk09cbHYa5ymg0rdLUttn7Rh5iU8V/3wKs583sYAOk6Ekqm7/tlveEdSY9crbzrUxbpOtFXw9pN9Dhytm7eEajEFKpCbzUBZzFjzxRUyPE3Ju41usWm3JcIsNEuJQB9zSSQLgO8poIPZySox0LNPZN2Z24QFCiRixlGnv0a4IFH9/QWo/YnSltuATA+tJlGC9lJBeHZFwHu3oT+1Gr0ZbLmURHL6NcW5mN/qdIwq1oeU9cSts916Rpm9CnFpt5vBdVyT+vDkZ0WogcukD4Nz/3761hfR0vuqxiM8yjMBcSmBr7N60vYtmSspm2MVjPgkmnvEOzIplkl8+tM4sRa/GK7UUYZ9z7XlZV2JFv2N0pE/FltcnccYueLuk4qGEXu92iVAAgtSFFLxAxKBmnltMuFukfdedQ86HscASZd7lSEmS1w95Pk+zrZdM8GADSkFn2+yJtrbJJW2eceQWa5UCFd96Srz+OzCh0+Yk22v/z+LeHieEdfhuUzps8ZulQ/qjtRDYhEx+unylD9aaKx64/arwsmJKIsXMvI+Zb1842keARdxIbkGwC6i2MTJoWvXK4= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:LV8PR12MB9620.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(7416014)(1800799024)(376014)(366016)(18002099003)(56012099003)(22082099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?cHgxMFErUE0yQU05YlJaMXFIYy9LL0p5STZhWUJEZWFkamZNVllucXdwMVMy?= =?utf-8?B?Q09raW4vT3g2eEhkNHA5MERjVUxlaXh5M1R6NHFnb3Y5eHFFVnhqVndxZGpl?= =?utf-8?B?UnhibmZzWlJkV3U0VWxpTE52anJVT3N3RWVsbnd5ekl4dUlLa0VYaHBQQXds?= =?utf-8?B?dDJ6RGFlTzk5K1lXUG9vNTBxVXBMT0UzSURpcHpzSlhGYnJlbno1cmhNcURG?= =?utf-8?B?d2UyTmZqNEhURzdLeE1Dc2FLbVB1Zkk5azVRWDRWRGVKZWZjcnZTdENkajNq?= =?utf-8?B?blpOZmVOQnowaDlUZHBNanZqeGsvT1dwWTZoRjhyeTYxQWJubVRhbThvVUFi?= =?utf-8?B?R1hJWlhWOVU3aWdTNFBuTHZSaFFUbHFwbjR4eXBEVXVxZFlsRjNPOHJTU2wz?= =?utf-8?B?UzltUUoyV3VubmFodk1US0JlTGVoWVliNVpTNWUvMzNwWUZuR1JQY05pclJz?= =?utf-8?B?clVqNDlBVHJMd0lmbXNFdFdrNzZjQXBjRGJSaW1BOXlOYm10SkV0VVQvNXRY?= =?utf-8?B?N3E5aDlwWHpZRnpoSFhoeDVublRTWFFNc0RXUHVkY1RxYWZ4aEdmWDYxMG5a?= =?utf-8?B?VUJBeENrbDI0V1c3aW1NUWpldk5SSEFOM1dPcVpHZjBseVUyejBSN0RGd1FJ?= =?utf-8?B?RnlyMTBIbjNzOGNoNHpVb1ljOXF1YktYQ0ZzSitBbWlhTFBDZzhsd2h1b2tL?= =?utf-8?B?eVdZbEEvU0tvMjhEYVdkTTRkUFdsRi9tTlZtMDJrNk0wRjZNVmd2UmtjSlhO?= =?utf-8?B?Ulg1RjEvSkJETSthc0pzSmJjdVBMc0lHeDVYeGlmR1NhcERYRDJKOEsxWHEr?= =?utf-8?B?bkJmYm11OU9GZWJsODhmZ3N4dmROcXdFTVRleGp5ck1kUlRUODl6dVk2STNr?= =?utf-8?B?MkJUYldpTkJEY3MrUGtncWN2S0hYdHNtVjI1ZEtkTXZKYjNoZmwyQWtqcEtH?= =?utf-8?B?SXE1TTlER1FZcFU4SEl0Z29LZkxXVUVBOWxXWm1kUi9zd1FWMGFKMEw2cHdF?= =?utf-8?B?cmk2SGZyNUtkSVdZSWZBWjV6OW85cHkxRHpxNGd3MUNDUGVDZnhMOVIzSUll?= =?utf-8?B?amY3YngvRUl6a1FZVlVDR1VkRVAzNjN1eU1FMzZXcnV0VThEV3Rjak91a0x5?= =?utf-8?B?U0szSFNCaHdxWUc1RUh2WnBJUktXY09DVHY2YTE1cEowTEZoUUV2WWphM3h3?= =?utf-8?B?ZERuaGcvbVBHU3VLU1JUdEtldW4yMC94bFVONVU1bTNPQmljTU9FSkljVFF1?= =?utf-8?B?amRDQitQR0dNUFU1NHFUNXdJTFYzVFdleEY0a2V6WE5tUHNzamtCRFpDV2p6?= =?utf-8?B?c01qRWh4N2VkYVI3WUtSUU9vWWJsQmkrZy9pM1o2TXRTYXlZYXE4OG5ZT0RZ?= =?utf-8?B?OTBaY1dvMExFeDZLYlpEakxYZjc1M29kWW1wNDNVeHArQjFZRzlPRGZWRVNP?= =?utf-8?B?aHhqeU9CUmJCb0o2aGZQWmJPNVo3eXV6dkZ1d0Y1ajNRcnRyVmJEWURVd2F2?= =?utf-8?B?aHYwZmkzMDFTbmpSb2tMeGpDU3JiVVZqbDJuN1JxbVNpL0kybXUvK3RaTHFq?= =?utf-8?B?Ym9xY2NQaWRibHU5ZlNXMi92cUlJT1ZKZ1lrdmkreVRucmpncGFtVTRUdDFx?= =?utf-8?B?enh5bTJESVVOTkhpbzF5ejhaaFEvczNSeHA5UmJTN3VHMGFGNVBGY0duMnN2?= =?utf-8?B?b1NpRzRzay81WVBUaW5QYjlNS2ViUGh1c0liZTlwREVmTGxyQlg2UnJZaHIz?= =?utf-8?B?MUowYW9BeGttMkMxOXF5TGJtZUZJWTBtamNSSldiMDhaMCtwMjJJcXBUaExV?= =?utf-8?B?NmUvSVl1d0dWNUxxZzNtdkxlQVdaczlEUHZpbENFeEpoMTd0NnJ0Z0RkZDd6?= =?utf-8?B?NVlZR256dUI4NTdQaEJpMi9NYnBwZ1NWTTZyTDk4K1RCUGxUTjRNZnM2NExK?= =?utf-8?B?V05GUWhSTTUxcG1RNFlXMDc2TGtRZ3dJcjE5THZmRk1mT3ZoWkhTWjh0Y2Ry?= =?utf-8?B?TExrRWJaQm0rZHlkQWJZbENoWEIvNWtCWHNnUVdkeENYeVdOcjBFN0xYWmdS?= =?utf-8?B?MXk1Z0dUOVErdXJVM2o4ckNTZXVBWllxeFdmL1R6V1I1WHI1MW5CVXBwTHl4?= =?utf-8?B?SURSU283S01Hb1VHYUxzcjJKTXlHMHM2RkVRcjZZWlRITmxLa0lzb2dmU0Jx?= =?utf-8?B?eEdVZG9rVk9CaFBRQ1RMWnd4N0lnT2YyUmllaUpPWWFRdVVuRitVc3Vsb3U2?= =?utf-8?B?eUFjUmRNdElERytlcmRQVENhaWMwRjlHdGNzYnJmcGhqZkpLRy9WVzUyNDNR?= =?utf-8?B?bi9EWDdtRW9Ebk84SE9ZeXYzMFRwRmtSQ3k3ZmlCVTV4cFBjRTJCUT09?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: a4b9df7b-4005-4795-e7d8-08de8a8e95d5 X-MS-Exchange-CrossTenant-AuthSource: LV8PR12MB9620.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Mar 2026 16:50:14.1804 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: oPOgl8N9EiHdSlv/e7oOzRprO6HI3FU+6JI7SticMsotcaXltBZ00ZIdPu85AlfcdD1KHhLq3ccj23ierkiX5A== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR12MB7885 On Wed, Mar 25, 2026 at 04:26:44PM +0100, Dietmar Eggemann wrote: > On 25.03.26 13:25, Andrea Righi wrote: > > On Wed, Mar 25, 2026 at 12:16:59PM +0100, Dietmar Eggemann wrote: > >> On 25.03.26 10:32, Andrea Righi wrote: > >>> On Wed, Mar 25, 2026 at 10:23:09AM +0100, Dietmar Eggemann wrote: > >>>> On 24.03.26 12:01, Andrea Righi wrote: > >>>>> Hi Dietmar, > >>>>> > >>>>> On Tue, Mar 24, 2026 at 11:29:24AM +0100, Dietmar Eggemann wrote: > >>>>>> On 24.03.26 10:46, Andrea Righi wrote: > >>>>>>> Hi Christian, > >>>>>>> > >>>>>>> On Tue, Mar 24, 2026 at 08:08:22AM +0000, Christian Loehle wrote: > >>>>>>>> On 3/24/26 07:55, Christian Loehle wrote: > >>>>>>>>> On 3/24/26 07:39, Vincent Guittot wrote: > >>>>>>>>>> On Tue, 24 Mar 2026 at 01:55, Andrea Righi wrote: > > [...] > > > Exactly, we already prefer fully-idle cores over partially-idle cores with > > asym-capacity disabled, but in that case the idle selection logic stays in > > a world of idle bits, without cap/util math, so it's a bit easier. And it's > > probably fine also when we have both asym-capacity + SMT (at least it seems > > better than what we have now, ignoring the SMT part). > > > > Essentially having somethig like the following (which already gives better > > performance on Vera): > > > > kernel/sched/fair.c | 2 +- > > 1 file changed, 1 insertion(+), 1 deletion(-) > > > > diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c > > index d57c02e82f3a1..534634f813fca 100644 > > --- a/kernel/sched/fair.c > > +++ b/kernel/sched/fair.c > > @@ -8086,7 +8086,7 @@ static int select_idle_sibling(struct task_struct *p, int prev, int target) > > * For asymmetric CPU capacity systems, our domain of interest is > > * sd_asym_cpucapacity rather than sd_llc. > > */ > > - if (sched_asym_cpucap_active()) { > > + if (sched_asym_cpucap_active() && !sched_smt_active()) { > > sd = rcu_dereference_all(per_cpu(sd_asym_cpucapacity, target)); > > /* > > * On an asymmetric CPU capacity system where an exclusive > > Ah, I thought we were talking !sched_asym_cpucap_active() case, either > by letting CPPC return the same value for all CPUs or by introducing > this 20%/5% threshold into asym_cpu_capacity_scan(). Sure, we can also equalize capacity via CPPC, but I tought we were worried about potential regressions with other systems that don't have SMT and may actually benefit from the asym-capacity logic. Moreover, if any other platform with SMT enables asym CPU by slightly exceeding the 5% margin, we may face the same issue again. > > ASYM_CPUCAP + SHARE_CPUCAP vs SHARE_CPUCAP would still behave slightly > differently because of asym_fits_cpu() in all those early bailout > conditions (1) in sis(). > > select_idle_sibling() > > if (choose_idle_cpu(target, p) && > asym_fits_cpu(task_util, util_min, util_max, target)) <-- (1) > return target; > > ... Ah yes, this also needs to be changed... > > And you would still have misfit_task load balance enabled. Correct, in fact to get the optimal performance on Vera with asym-capacity enabled, I also need to fix the misfit logic to prioritize fully-idle SMT cores. Same with find_new_ilb() and potentially other places. With these I get almost 2x improvement in some cases, which is pretty big. But I get similar results also disabling asym-capacity via the 5% threshold. > > Those subtle differences may influence behavior compared to a simpler > homogeneous CPU capacity model, but it’s unclear whether they justify > introducing yet another variant alongside the existing homogeneous and > fully heterogeneous (non-SMT) approaches. > > IMHO, we should only consider allowing this if there is clear evidence > of significant benefits across a representative range of benchmarks and > workloads. Totally agree. But there's still the fact that select_idle_capacity() is not compatible with SMT, so it should be avoided when SMT is enabled, in a way or another. Thanks, -Andrea