From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from BN1PR04CU002.outbound.protection.outlook.com (mail-eastus2azon11010044.outbound.protection.outlook.com [52.101.56.44]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4164B37C0F0 for ; Fri, 24 Apr 2026 23:29:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.56.44 ARC-Seal:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777073365; cv=fail; b=L+xRlBDbd5GaH+MD/vGZQQWtGIHa36ZaIMQjfR75jUs0p0ZjgSRopP+cpz+NpVtb9YY3aVpvQduYj6YPBoGl48UbXucTeGuotTpr7cCrR+lvpLhZQcCx05pKnxsGG6rymrnLmlIJguF9zqLjGAr6KXuduAeFvvIWAGRFRW6QwG4= ARC-Message-Signature:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777073365; c=relaxed/simple; bh=1S+/PoNf4J9mj9XfxOvbvs8vriN2VSvCx4efzKSHnjI=; h=Date:From:To:Cc:Subject:Message-ID:References:Content-Type: Content-Disposition:In-Reply-To:MIME-Version; b=J0LZjbw2m2mhc3hFPhArg9q7ki93dzZuH8N8FrYFbwhL+9uhTEcSwMrn1cFqe5tagn87YkZHNzNYDDzUwDHyVg1m1dbV9JQlkJS+EXvAwEAkkCAbuoKvBzsm8fJHhXpSVnZqA4VhOMmb43s0jFoSjcevMykVIFB1iZ6gpTn8Ctc= ARC-Authentication-Results:i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=oL2Jm/NU; arc=fail smtp.client-ip=52.101.56.44 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="oL2Jm/NU" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=ukiqjLfkX+tCQdid5Xz3zI17UYHnAiRzRTIRvuO86sDczOMMfVv/odPrhbX8SyQNnOjPNjROthQx7cm4G1qq0p9Qm9mhjnuTJoGa0b6JlAwxi2xT7r1TpuZIGZTsHtHbwlNCsN+v+Ih+8HshwvAWCKnToiaCZZjsIEX/CBioJnNTM6LPfVFXx3Omu2jNYhZk2cJ+Vn/+lx+pwDvckEZ95nSXluykg+Z6cn6avJUpJMknsaLOHgKXrLoUKiw4wlatJ2zLL/OGZK3/jkfF2S6nbZL+XbgzKUECa25aMG09ptbPHXDDh7biy++jbvBBRXA638GAFRryvT3h1G/Y3lmj0A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=IZ9A9DdwCxpFMU2trs+gAoRFOjttoBhefyjPdrV+2Xg=; b=DWPzpHT2HKfdwcBThhbTZHzsOPItytFi74c055H66qrXbrmf2lqGJnSTJeySip/Ft8mfxde0Xl77HPIWzd54+JRxbYYHMo0eU55l6QR1zrehoYNSfCktyJNhjq6ODepKjllCtBAaNda4q+CL8uSv8Pz2qBARIJ3D8KZ0Hch6nZYHQJWZULixb+wSEPxe90RsAbT3+uDq4IuFiW9fuVHPUnsnarNXqtsE0ujiProSUJJAky0dQr1ojfH2Nh8R9SN0JZe90a+DWxTumTPewGdlLuBhnLnwk2/OUmStWFH5tms4I+JWMaG7hiV2gRSjDto+oRmVuhMkBHElkj0tSI5jfg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=IZ9A9DdwCxpFMU2trs+gAoRFOjttoBhefyjPdrV+2Xg=; b=oL2Jm/NU2NGZ1y9O86cCarQwLZjnN3oqAU8rnxBQZWcpvltel2IjxiW1LkKHytJTyhAsbfzJHse7fN4BxPorggFgIigKPy5bTL0r65c2AYuQcOfzg6nINBoXGrSx0g7tcgXMs69FvZByv0DhEs2ASNLz2o9zUbvoz1vasEs7rPCSwJInfEyFIH2BGWC3a2F4B2n6JwD7kvx+qeMsCy48VcxHBvGJN9vtjHszjcWEIsG04Ziw1Lu/fxKPlEWOZ7kaYhul+beUwyRI5+ybC6KxWlo+DcVFiJX6MB16JOwTUw4uHZAQ8dA2Nl3Dw806O7d4cUEmFfV+6tyu8Tdc0j5sBQ== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from LV8PR12MB9620.namprd12.prod.outlook.com (2603:10b6:408:2a1::19) by CH3PR12MB8483.namprd12.prod.outlook.com (2603:10b6:610:15c::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9846.21; Fri, 24 Apr 2026 23:29:18 +0000 Received: from LV8PR12MB9620.namprd12.prod.outlook.com ([fe80::299d:f5e0:3550:1528]) by LV8PR12MB9620.namprd12.prod.outlook.com ([fe80::299d:f5e0:3550:1528%5]) with mapi id 15.20.9846.019; Fri, 24 Apr 2026 23:29:18 +0000 Date: Sat, 25 Apr 2026 01:29:10 +0200 From: Andrea Righi To: K Prateek Nayak Cc: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider , Christian Loehle , Koba Ko , Felix Abecassis , Balbir Singh , Joel Fernandes , Shrikanth Hegde , linux-kernel@vger.kernel.org Subject: Re: [PATCH 1/5] sched/fair: Attach sched_domain_shared to sd_asym_cpucapacity Message-ID: References: <20260423074135.380390-1-arighi@nvidia.com> <20260423074135.380390-2-arighi@nvidia.com> <75cf4fd1-2e80-4167-9113-954015ba63e1@amd.com> Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-ClientProxiedBy: MI3PEPF00004EA9.ITAP293.PROD.OUTLOOK.COM (2603:10a6:298:1::44d) To LV8PR12MB9620.namprd12.prod.outlook.com (2603:10b6:408:2a1::19) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: LV8PR12MB9620:EE_|CH3PR12MB8483:EE_ X-MS-Office365-Filtering-Correlation-Id: 88754e77-9a94-4e1c-41c6-08dea2594e3f X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|376014|1800799024|366016|18002099003|22082099003|56012099003; X-Microsoft-Antispam-Message-Info: GIRrZdjxgI4J25kGoLUlg+SycUgmlOGPckTLGPC7Wrj3g454xj32CmEoyymVxJvsnMyZo8S89BZrfXGDrdZsupd9z/i2aAJgCkhjL7bSiO6yJL2vSdxtvZcOhdyDTwp7Qih7W1T740taWnQCtGkVl85ScxCmKJwKV+hpBM8FMutiOsqJxAORgg7Va8XGyIkqp7Ayj0rnf1HRVZNa3r4aqK/LSFZCgrCq737rtEVQNZb8Mzt676E9gRgNptzPQG/72V/OgHnPunloydA2qiriSVwCaIGib58MvJbdY4cZPjzCBJk+HTuFizbJD56fIgugJfvpYBVsif+GCAtntSW+OSDJpmRevKmCyfIz5uRXYuSArLVppomEZ1eztvITlTYcacAxNjR8LQ4vf2uU5b2kv1xUSicQPUSRbaxCGmXVhGouSdwbs0y3bgwoDBZ+vqACvLoMQIjz4bzhMYHrl1gje48JPYIcaZnOv2doADk+ZEGHkV2/pQ/xT9Msot7ahkNo7P5BZxC5Kr9NY/QCdr+r8Fm0SRo+F/se0Gb2Zr1IHAr3sGbu2AlD1c8IMmGYAlOLDUC06smwv3225lYm1XFwX1zK0XVfB/YxaIougudAbuTbi8ECQOouItrST+I3ASpIAnmIYjFT0bzwXdtKDDkmqnN81/SkK01QPxQ7QnZhAEFnE83QZZJ95GTcL7Vmk/fYJ/rnK5x6auVa57GCmgEZEt+bd9xRJNFdIY7KELjkUUA= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:LV8PR12MB9620.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(7416014)(376014)(1800799024)(366016)(18002099003)(22082099003)(56012099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?w6927f35zm0OeXpZjq53dp8bvCQDoGM31SezC+5XbBiN67pOIK0e+6JzytRO?= =?us-ascii?Q?k6yXh1lK4XijArnOVYdcT5tV3HAR+dBDVGsOx+3UsK6uKJjUYDXSV3exNwQK?= =?us-ascii?Q?RRfmMm09qcSYqYQiJ/aSOzALCoHeA+JUVqT7ajl21HX6u4RFCpEtbRxhxLOA?= =?us-ascii?Q?qFInXYmpwjdg+DxJS66xF2B9ueaITV6F1TEbY9MlYHIjhPSfpfWrN6SbbAHA?= =?us-ascii?Q?gpRz+a1ywSF0Com/zlCSgCMv1rgj96oEDy9nT74WDSnioOqNA3ZKXhmo7grN?= =?us-ascii?Q?oYt6d7FPoGyDada1+0a20PaikRFMm4nqrVvMP9LwyMYrCXD+b3+DnSvw4JQe?= =?us-ascii?Q?IIJ9hyghSrKa6xV+sKwmNuz90EijF87Q0vn1Qj3cv4Vtw7+FhTIlhlmV9wwX?= =?us-ascii?Q?6j81+k0XDGRFHjsYws8I23s52it+IeEtYI/GfaocIoCpFNdS6Kg1Y1cDzlc8?= =?us-ascii?Q?EUD1gKapaP8Xn1av5FEhuwFEGCnI2JgEQeIJTs9cTuASRXBUdKrYHHKMAHkI?= =?us-ascii?Q?/czN7KGk163qxPBGH6hh3I4ER5zvocawMPJqqTNosXFR74WNuTPBRtMsLA42?= =?us-ascii?Q?HROkMaKP+SNDq1BJkBsZxYUGtdGtJpdQIHdkRbIgWWKpjCoyG+ZQUbTROdGS?= =?us-ascii?Q?cBeBAL/yPqfGiBVxC16G/26ZmQL8Mqj/A8Al05BPHfdJx+dWAU6WIrYGujgg?= =?us-ascii?Q?rBUGCHhDOz3itdB0nNfZ862HC5D5kz/cukyK+AaNqP3/+KfDRoqDtcrR71UF?= =?us-ascii?Q?3uPWZmwqqT5m4Ci2gCg+BOQ/dKmB4/Zs+raRHjyjSIN1ERN+4Bo0wji5era3?= =?us-ascii?Q?X5CpAmQjBB9yCmUoNzJuuvysHn2ObIBOxhLleP4wgtaAJOCZx663hd3VnYrM?= =?us-ascii?Q?1I0b9Nf4izt+hK28dygqRNcuFNGubsmaAeIBEgPLVAzFLeo6CCuQiym6WNzI?= =?us-ascii?Q?waXoOcBgUtFS1TW/H3zZCk5ButeQ96zTV7qaPC4IWdysZlELsqvnGqBajee7?= =?us-ascii?Q?CdNDNFT4cY1uCpBz56AP3hg44ceTH0sehYN68tHC4eTMnocA1/DImwshAV7+?= =?us-ascii?Q?6iAa0rkjK3/+CcHq0jsaggTgbVZvS5K5gi7f9DuWOBAfhy0YVrQy7fHHSdnq?= =?us-ascii?Q?Wwa7LjjFnqI1FjE2/UW9FoKCBa1hPsXwIxKPZnVTY0m3iGgHlrsc8gwrkgU3?= =?us-ascii?Q?pRnd99w+wNi5Jk256hiBEyDv/C9aqK4pNquLuC+7cpxm3hQ1XlneGLEiDLYD?= =?us-ascii?Q?yAYJsA2T5mdaB7KgaarUxCukqCpUxlIFeKI2IHFZUIcQcnEh6asML761J8vp?= =?us-ascii?Q?lsXT6v4jO1B8G9+798bY2bBOMHOMsiSt7NDBtmjyz4RfIELSzBMRIxCZ80DR?= =?us-ascii?Q?8Fay7Y5gnImLD0ZIKGnqBzdnCg0LAP8ToG8ZVR9Bw7uUVTrandWf0Zr0O01K?= =?us-ascii?Q?NbBHdp04MGYEC6OWuqcY6QU+rmnQ9W2OjA+kZIac8faLGkDsx8ntCNxAdY7F?= =?us-ascii?Q?15cng21XZk6bmKM4dxxSSte4KniKHpPPgGZNb6njHHmZhlCn0CD2e4r7L7MX?= =?us-ascii?Q?GELcNJXEZfuSTH5n8RpA42NhtUz8yTpXPtAALIkTXxa0O4BTgxHdmqm9vfwH?= =?us-ascii?Q?FeYg7tctY88u/ceWR/QuhKNFC0P65Ct2dYysVlwahF0CwEqfy0uU75m5WdIT?= =?us-ascii?Q?ziFvsyUEOBgPm6/R0ZUufK1X6w61O7qGfclbY3dkfZ6sk5XtoK9kK9lFHLD9?= =?us-ascii?Q?zGYmLoAfhw=3D=3D?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 88754e77-9a94-4e1c-41c6-08dea2594e3f X-MS-Exchange-CrossTenant-AuthSource: LV8PR12MB9620.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Apr 2026 23:29:18.3224 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: CnhtiaG/n+PQQI91YdalPtwUYf0hMr/6qYIXf+YfQchzudHj+rzfhCS1UPnmWDYfii9zICRmqTxsmet27rP8nQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH3PR12MB8483 Hi Prateek, On Fri, Apr 24, 2026 at 04:48:59PM +0530, K Prateek Nayak wrote: ... > >>> if (sd->flags & SD_SHARE_LLC) { > >>> - int sd_id = cpumask_first(sched_domain_span(sd)); > >>> - > >>> - sd->shared = *per_cpu_ptr(d.sds, sd_id); > >>> - atomic_set(&sd->shared->nr_busy_cpus, sd->span_weight); > >>> - atomic_inc(&sd->shared->ref); > >>> + /* > >>> + * Initialize the sd->shared for SD_SHARE_LLC unless > >>> + * the asym path above already claimed it. > >>> + */ > >>> + if (!asym_claimed) > >>> + init_sched_domain_shared(&d, sd); > >> > >> Tbh, if "has_asym" is true, we probabaly don't even need this since the > >> nr_busy_cpus accounting gets us nothing. > >> > >> Might save a little overhead and space on those systems but I would > >> love to hear if there are any concerns if we just drop the > >> sd_llc->shared when we detect asym capacities. > > > > Hm... but "has_asym" is global, we may still need LLC-owned shared for symmetric > > islands and NUMA-overlap cases, no? > > "has_asym" is local to build_sched_domains() right? So it should operate > per cpuset partition since we call build_sched_domains() for every > "cpu_map". Yeah sorry, I meant has_asym is aggregated across the whole cpu_map (not global machine-wide). Just to make sure I understand correctly, one CPU in the map can hit the asym flags while another never claims asym shared and still needs the sd_llc->shared, but it'd be unused, so why allocate/initialize it, right? In that case, it'd make sense, but it sounds a bit bug prone, is the extra risk worth the overhead/space saved? Also, what if we decide to wire nr_busy_cpus through the asym-capacity path in the future? Thanks, -Andrea