From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from CH4PR04CU002.outbound.protection.outlook.com (mail-northcentralusazon11013025.outbound.protection.outlook.com [40.107.201.25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 535EA155757 for ; Mon, 18 May 2026 21:31:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.201.25 ARC-Seal:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779139879; cv=fail; b=c/I220edlEf3Mge9AJLW9iAOLMl3f9gV5WYmslRzMo8UYVw53PHOoHiwLru6zE+PtRYEORpl63IOVIQPnwr0Yf0XF6MWbWaNJH3qZSzBvOxKOYUwpIBN4+9NHEIkaNnpZb7F41UqA28oxi0OrEN0y+Lt96t0MIn4bO6gV0zzGz8= ARC-Message-Signature:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779139879; c=relaxed/simple; bh=0gHujH4ZfSJ4kApv71epokJ8sH0y6FM9TSHX0x07uuE=; h=Date:From:To:Cc:Subject:Message-ID:References:Content-Type: Content-Disposition:In-Reply-To:MIME-Version; b=Be+Gv+oOt1vbwqHf+NiYEDBKPlML4BKU6VpW9+yyN5rDWIN2I/XWdPAH5zClBj6Jmh0lfrhc64gGiGoepmonP1glhsqVMXFOAUsBK3RvGfV45yWn7ZFLJWnSZKsLz8j6nZy1JBMlaQPB/yrZ0PnOWq0FC64AFt9VcbJjNU3/mzw= ARC-Authentication-Results:i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=c4oInJby; arc=fail smtp.client-ip=40.107.201.25 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="c4oInJby" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=FtBq/fjiVCXK2uhwAeEEd3ITmKdxJEMsA5DWWYd81okqHhIBc33BuLVZpP+sDcseynimsGhloZRahk0UA/M6P3berqf9taQW5t8xQ1yScFbVZOlAKKsnjMqVWuJzC1HWnYYKWPvxY881z2NBADnnHgbpIm7+iMsoxi64kNt0Wuov9JTIlxvfNmeEL50jOAvYW5qRDI/v/gSroAsBhMkrC/eLUWakxAF8cjUeDkh1jb0kT3ASInTVMkc1QsWCgo8J1XyADBU5s6xp2cPRq31SZr5sxAWGOUNdsgsUDU9L7z2kUFl9temdlhcJe3jfQufVSp6VJWT4gHAVaySFlLwmpQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=oumr4nGKRzeyYV54fIfRBDqBe1m3g23mumOTCelygqA=; b=lV++2U7DtEG4S7sx+iM2TnvpzszqDv5xYkF2+IvAx8nZ6RGZ8gvLGKA0r4kSXhWyd+wr+c0GdGmDqpOZF6glm2hKbPfiM6n2vXOrf0hZT42v2AohnOPc9KIZZbtvsGjphN4WWVLtsGQHG+h48F9l2U/elxZt1iMjQRad4RthiarA2IKMnxM+FwKJaCq3gzHic9esJubK7VOWxc4Mr/bvNrZhQ/IX38Xp+6mj2baw9m7c43CkAer2Y2p3Y81dAy/vhk+qNWiSLkqWqfy0R/1lsDkznI4gSji2H/YRYhthceBUIKZ0qWMygu4MmlRbErxD4lqDXhct5a8irKcFKb8LIA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=oumr4nGKRzeyYV54fIfRBDqBe1m3g23mumOTCelygqA=; b=c4oInJbykucvwvOCJ1y20kXwe3GzzmDbP9MwUhfZ6fdm0ZTEKINzxnSu36MArvM2McozJbM+vh4uAlwgtV9SaontqOi3e0QSTXVix5XKvoEDqzAK9aHOroMtZRJ7mlxWBxHddf9I25XiNvRikfieSy3x8EA0RyZQcdRg6iHG5FFQ0FvHado1rVps0BriRTej2aHfoqKCvwY8gtmAznJS8N8SE6JydmiqaLFnkOvAW/6N44dQjvzT7Vtjb3lNE/G/wX7+e9HA+YbS/MI7yKRRkI5+i/T963jS+SexnsVflaIyJ1bZxwK7fRheEQHt5JetYSC0dl8pf7LHIJohvghApA== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from DM6PR12MB4827.namprd12.prod.outlook.com (2603:10b6:5:1d6::14) by MW4PR12MB7191.namprd12.prod.outlook.com (2603:10b6:303:22d::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.25.21; Mon, 18 May 2026 21:31:13 +0000 Received: from DM6PR12MB4827.namprd12.prod.outlook.com ([fe80::6261:3040:864b:159c]) by DM6PR12MB4827.namprd12.prod.outlook.com ([fe80::6261:3040:864b:159c%4]) with mapi id 15.20.9913.009; Mon, 18 May 2026 21:31:13 +0000 Date: Mon, 18 May 2026 23:31:03 +0200 From: Andrea Righi To: Peter Zijlstra Cc: Ingo Molnar , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider , K Prateek Nayak , Christian Loehle , Phil Auld , Koba Ko , Felix Abecassis , Balbir Singh , Joel Fernandes , Shrikanth Hegde , linux-kernel@vger.kernel.org, tim.c.chen@linux.intel.com, yu.c.chen@intel.com Subject: Re: [PATCH v2 2/5] sched/fair: Attach sched_domain_shared to sd_asym_cpucapacity Message-ID: References: <20260516055850.1345932-1-arighi@nvidia.com> <20260518205859.GY3126523@noisy.programming.kicks-ass.net> Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20260518205859.GY3126523@noisy.programming.kicks-ass.net> X-ClientProxiedBy: MI2P293CA0011.ITAP293.PROD.OUTLOOK.COM (2603:10a6:290:45::11) To DM6PR12MB4827.namprd12.prod.outlook.com (2603:10b6:5:1d6::14) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6PR12MB4827:EE_|MW4PR12MB7191:EE_ X-MS-Office365-Filtering-Correlation-Id: 56effe34-7d70-43ae-0f44-08deb524c881 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|366016|376014|7416014|11063799003|56012099003|4143699003|22082099003|18002099003; X-Microsoft-Antispam-Message-Info: sZb75uw5xcekcYz988aYiFxBnmMGeijQrYpa9f5sg4LjY5WHwcnwj7t5+5qVUN8a3kl0m7m7k1ikr5rHoEgtPXAfRlEl0qGEABIDb7SUyGNx/9rdrFaCf5PPp0qiMgVB/LmEXsxX45La+w0jbcZk0SvTvdrvTVxHUhmhap75zq5R681pm+7riVOMJTR57VWjokKZId0u6+ia7yKgzZhx0Kumwp3BDU5Je7z/TZheROTeivQxfelVDhNPQq+M0mYbrG/uPrlvJf3WzWVbBiku2kSLfU0qgSCh+Q7MJAiTTVaOvHV/65cGmJbR3F6RxspsTFrU+xLtPCypC2wEZKvwED++1hVMEdCDN35oyBaVW7dYchNDLwjSDB71idRKhHKK+/hYi8agLD16eiNXIZmQgli7IqnSnZbPB7SLyHjh1QoicCnWve3PMhK03RcJiOMfr+l9vRzCX7ouzsJBdszrtD9XML2eg16qMBEcZjZ5biXVNMSGNrSADd3ofcQbW5rbcE849vP3oPdMlYTRpnGDLrPDcObTX6HUzFv9iSJL1DLieT5rg32BxGFK42CpffWTZrpGOoOH3WiSbqlAte+uNiLkx/TBc6amUrwyXxEQxv1hYwVzR/DETvBkz9gbWzW533Dv4JFEyKh5IBkadA3UkfNdtDcVaCxw4lW9bZEV72uh4/UvsI7MfbYtzpUSAXYS X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM6PR12MB4827.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(366016)(376014)(7416014)(11063799003)(56012099003)(4143699003)(22082099003)(18002099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?73kHfKCW6xPI7OgTbMXWkEhUlh1NNnnr3qn6W71ZF2YM2CKzq7paMhtb0bST?= =?us-ascii?Q?Sm2XvG735f/JcpHjORcKsnn6CrmtOXhY6+fYvwO4NjxM6ED89YTNn2j6QrMW?= =?us-ascii?Q?5iFiKAS1i5/AEndKoVDuLQBVql3LCZurRMXD1cleQ06vrZ/pLgghpxc51t2j?= =?us-ascii?Q?t4vIS+Mmr/YixKUTEH0b1V+rIez1nTFBkUWmct7ZNzAriCaQ9lkd1SMaCrtj?= =?us-ascii?Q?U0Idr0WuDCO4KEV7dAAgyiLZ+Hk3aVXOQwKbZrU3f7g67XVJuMaDu30ffK6P?= =?us-ascii?Q?KaPB2RJAzkrk06f+LmWhBiIq++eU50GEdzml44Rsx9gtBHumrThvBgq2WxXr?= =?us-ascii?Q?IGRRcH+FHmaw2v3C9IxUUrnMvqIiWfZbW6FehPI682W02Tg0A6qilC3ddDpV?= =?us-ascii?Q?ZVrlIlE/6lO0J3v8o86jS0Z2QaIUCnGdJwzka62vvaTtn2W2sb+iimZ8x8jo?= =?us-ascii?Q?KaBvfZxTDfiH2AvN+utEFRTP7Riq1fpxCAg5ZPvVu2nTGteQJ3RLvzfGCIzU?= =?us-ascii?Q?GnTjOzX5jmZDBDvo3Oddp1KQBTGWm3AqrrDOBaJX7XPHZXYLN3khytrQLLXs?= =?us-ascii?Q?VO9FzcS6x979XYjFk3sxqFYV8ZmyMMjn+rqVjAZFPo3cwwcv4eevMTQo6LRS?= =?us-ascii?Q?NUHZYQUPvtAPpClNi1ksOrmRCxyJ7danvkrlpWyrR8Vyo3nzfjDX6FkfsXD3?= =?us-ascii?Q?A15R1aRc9vfjyjWKk2M3ahFFocdBwSBerGzMf/Eq05rThvOLTT/0pIe78lRf?= =?us-ascii?Q?xqe94SrEHUmwbHEk823i8pmLeBknlSWhVNGopELpmP6xRXZytptm9ZaH/FDI?= =?us-ascii?Q?qOlqXmZWmRxJpeLwr5/ppzM4hrwu42H08/ZWhtOwhBIkw9yxhnYC+oGNPTW6?= =?us-ascii?Q?a9gR2ZNb2wzYwBhQAzXXnB9ejj4JvbVnYX25ADEGuEGlf32AY/d6JxtAXJsD?= =?us-ascii?Q?Q+snxkPReVxKmO42Onm8W7zv/sQsMJVHhvRm7z6uDBOmeJBQ4UQZWhYKbQ2W?= =?us-ascii?Q?MmTl+0GTq77l7ZPdgtQDJshGXYxqzYVnsrdIrG94M+n27qy0JDi8ljzdfuOf?= =?us-ascii?Q?z+y/gWoDwSjrGm6oS5qcG1Fe4NGqQ0HPhM2qElNNo5bV8N62/Riaotomh/F7?= =?us-ascii?Q?YDQ+HDqxa1QA4XaYyCC7zDBVrADSVr2EKeQk5THFZNtmY0Rlqz/4gkBh9znI?= =?us-ascii?Q?DjAwf7mQs9fL2CGNLLsFMSl4xrIwnw2z24Pi27S36NWW+tXOVTcSveRH58Hb?= =?us-ascii?Q?zJ9rhiqwKi03aVXHlwJgXJJnLsfevbKWxfwHHSa5RFpBhce0gE+uh6c221TE?= =?us-ascii?Q?SCqNH9jm7HjVpuo50MG9Mf6VTCsNdba1tokmYwGVcq9TU4/YbP4lXiGSeoMd?= =?us-ascii?Q?12B24p/38ZgJzCIg7DNa/vNjWczcq8SME6Xmwxr4nAgyxpYjx0Rh3REnqrLr?= =?us-ascii?Q?ejnodqr/5Gu3OxYTl3JY8SzDUQ7uUCt+EAYsdYKkMCvlZy1jL0Vkx67jU//9?= =?us-ascii?Q?dSOWRA7lsA508htY0oPyUmKE0NHEG6hmqqxyh6I0whuxIVD84u0Rl2yzlT2s?= =?us-ascii?Q?nkKnFQOGY4muqkIp5WU0Wi/jkDBYJ8xEYL1hC5LQMA1kukmThtRFvj4nvY6a?= =?us-ascii?Q?U28T29skDz5Xq1Y3yGDzY4FWxAzRmpGb/EBcG+omK651DaASCS5+H1im+V7/?= =?us-ascii?Q?zgwMuQa9tFvJWVJ3zvPKH7/9Cf68pHbNfSWVg9zw5gzTwX4yFvmHths5rgSE?= =?us-ascii?Q?9FhPD1Q3jA=3D=3D?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 56effe34-7d70-43ae-0f44-08deb524c881 X-MS-Exchange-CrossTenant-AuthSource: DM6PR12MB4827.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 May 2026 21:31:13.6211 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: WRVGJMZ9ocl8JmWydzQvyknuANK3VkJdQyygEX7OKhSZ4yyLd0wKt/8NDQJhjZCr2EhTWgj+KHVLGq8opAYH2A== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW4PR12MB7191 Hi Peter, On Mon, May 18, 2026 at 10:58:59PM +0200, Peter Zijlstra wrote: > On Sat, May 16, 2026 at 07:58:50AM +0200, Andrea Righi wrote: ... > Right, so I just merged a branch that has this series with a branch that > has the cache aware load balancing stuff on, and the result ain't > pretty. > > That cache aware thing really wants sd_llc_shared. Now, I imagine that > for now the intersection between ASYM and SCHED_CACHE is not that > interesting, but at the same time, I'm fairly sure that is something > people will end up looking at. > > For now, I've stomped on things and the merge holds the below. It > builds, not tested much beyond that. > > I've pushed out the whole pile into queue/sched/core. Conceptually makes sense to me. IIUC cache-aware code necessarily needs per-LLC util_avg/capacity, while the asym path needs has_idle_cores at asym span, so you basically restored sd_llc_shared alongside sd_balance_shared. I'll re-run my tests with your sched/core branch and report back. Thanks! -Andrea > > diff --cc kernel/sched/topology.c > index f96d50131495,e47a3f72eb72..000000000000 > --- a/kernel/sched/topology.c > +++ b/kernel/sched/topology.c > @@@ -663,9 -670,9 +670,10 @@@ static void destroy_sched_domains(struc > */ > DEFINE_PER_CPU(struct sched_domain __rcu *, sd_llc); > DEFINE_PER_CPU(int, sd_llc_size); > - DEFINE_PER_CPU(int, sd_llc_id); > + DEFINE_PER_CPU(int, sd_llc_id) = -1; > DEFINE_PER_CPU(int, sd_share_id); > + DEFINE_PER_CPU(struct sched_domain_shared __rcu *, sd_llc_shared); > +DEFINE_PER_CPU(struct sched_domain_shared __rcu *, sd_balance_shared); > DEFINE_PER_CPU(struct sched_domain __rcu *, sd_numa); > DEFINE_PER_CPU(struct sched_domain __rcu *, sd_asym_packing); > DEFINE_PER_CPU(struct sched_domain __rcu *, sd_asym_cpucapacity); > @@@ -729,6 -717,9 +718,20 @@@ static void update_top_cache_domain(in > > sd = highest_flag_domain(cpu, SD_ASYM_PACKING); > rcu_assign_pointer(per_cpu(sd_asym_packing, cpu), sd); > + > + sd = lowest_flag_domain(cpu, SD_ASYM_CPUCAPACITY_FULL); > ++ /* > ++ * The shared object is attached to sd_asym_cpucapacity only when the > ++ * asym domain is non-overlapping (i.e., not built from SD_NUMA). > ++ * On overlapping (NUMA) asym domains we fall back to letting the > ++ * SD_SHARE_LLC path own the shared object, so sd->shared may be NULL > ++ * here. > ++ */ > ++ if (sd && sd->shared) > ++ sds = sd->shared; > ++ > + rcu_assign_pointer(per_cpu(sd_asym_cpucapacity, cpu), sd); > ++ rcu_assign_pointer(per_cpu(sd_balance_shared, cpu), sds); > } > > /* > @@@ -2663,54 -2906,61 +2916,109 @@@ static void adjust_numa_imbalance(struc > } > } > > +static void init_sched_domain_shared(struct s_data *d, struct sched_domain *sd) > +{ > + int sd_id = cpumask_first(sched_domain_span(sd)); > + > + sd->shared = *per_cpu_ptr(d->sds, sd_id); > + /* > + * nr_busy_cpus is consumed only by the NOHZ kick path via > + * sd_balance_shared; on the asym-capacity path it is initialized but > + * never read. > + */ > + atomic_set(&sd->shared->nr_busy_cpus, sd->span_weight); > + atomic_inc(&sd->shared->ref); > +} > + > +/* > + * For asymmetric CPU capacity, attach sched_domain_shared on the innermost > + * SD_ASYM_CPUCAPACITY_FULL ancestor of @cpu's base domain when that ancestor is > + * not an overlapping NUMA-built domain (then LLC should claim shared). > + * > + * A CPU may lack any FULL ancestor (e.g., exclusive cpuset symmetric island), > + * then LLC must claim shared instead. > + * > + * Note: SD_ASYM_CPUCAPACITY_FULL is only set when all CPU capacity values > + * are present in the domain span, so the asym domain we attach to cannot > + * degenerate into a single-capacity group. The relevant edge cases are instead > + * covered by the caveats above. > + * > + * Return true if this CPU's asym path claimed sd->shared, false otherwise. > + */ > +static bool claim_asym_sched_domain_shared(struct s_data *d, int cpu) > +{ > + struct sched_domain *sd = *per_cpu_ptr(d->sd, cpu); > + struct sched_domain *sd_asym; > + > + if (!sd) > + return false; > + > + sd_asym = sd; > + while (sd_asym && !(sd_asym->flags & SD_ASYM_CPUCAPACITY_FULL)) > + sd_asym = sd_asym->parent; > + > + if (!sd_asym || (sd_asym->flags & SD_NUMA)) > + return false; > + > + init_sched_domain_shared(d, sd_asym); > + return true; > +} > + > + static int __sched_domains_alloc_llc_id(void) > + { > + int lid, max; > + > + lockdep_assert_held(&sched_domains_mutex); > + > + lid = cpumask_first_zero(sched_domains_llc_id_allocmask); > + /* > + * llc_id space should never grow larger than the > + * possible number of CPUs in the system. > + */ > + if (lid >= nr_cpu_ids) > + return -1; > + > + __cpumask_set_cpu(lid, sched_domains_llc_id_allocmask); > + max = cpumask_last(sched_domains_llc_id_allocmask); > + if (max > max_lid) > + max_lid = max; > + > + return lid; > + } > + > + static void __sched_domains_free_llc_id(int cpu) > + { > + int i, lid, max; > + > + lockdep_assert_held(&sched_domains_mutex); > + > + lid = per_cpu(sd_llc_id, cpu); > + if (lid == -1 || lid >= nr_cpu_ids) > + return; > + > + per_cpu(sd_llc_id, cpu) = -1; > + > + for_each_cpu(i, llc_mask(cpu)) { > + /* An online CPU owns the llc_id. */ > + if (per_cpu(sd_llc_id, i) == lid) > + return; > + } > + > + __cpumask_clear_cpu(lid, sched_domains_llc_id_allocmask); > + > + max = cpumask_last(sched_domains_llc_id_allocmask); > + /* shrink max lid to save memory */ > + if (max < max_lid) > + max_lid = max; > + } > + > + void sched_domains_free_llc_id(int cpu) > + { > + sched_domains_mutex_lock(); > + __sched_domains_free_llc_id(cpu); > + sched_domains_mutex_unlock(); > + } > + > /* > * Build sched domains for a given set of CPUs and attach the sched domains > * to the individual CPUs > @@@ -2775,20 -3049,16 +3107,15 @@@ build_sched_domains(const struct cpumas > if (!sd) > continue; > > + if (has_asym) > - asym_claimed = claim_asym_sched_domain_shared(&d, i); > ++ claim_asym_sched_domain_shared(&d, i); > + > /* First, find the topmost SD_SHARE_LLC domain */ > while (sd->parent && (sd->parent->flags & SD_SHARE_LLC)) > sd = sd->parent; > > if (sd->flags & SD_SHARE_LLC) { > - /* > - * Initialize the sd->shared for SD_SHARE_LLC unless > - * the asym path above already claimed it. > - */ > - if (!asym_claimed) > - init_sched_domain_shared(&d, sd); > - int sd_id = cpumask_first(sched_domain_span(sd)); > - > - sd->shared = *per_cpu_ptr(d.sds, sd_id); > - atomic_set(&sd->shared->nr_busy_cpus, sd->span_weight); > - atomic_inc(&sd->shared->ref); > ++ init_sched_domain_shared(&d, sd); > > /* > * In presence of higher domains, adjust the > diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h > index 9dd4a94801c9..300320b0248a 100644 > --- a/kernel/sched/sched.h > +++ b/kernel/sched/sched.h > @@ -2191,6 +2191,7 @@ DECLARE_PER_CPU(struct sched_domain __rcu *, sd_llc); > DECLARE_PER_CPU(int, sd_llc_size); > DECLARE_PER_CPU(int, sd_llc_id); > DECLARE_PER_CPU(int, sd_share_id); > +DECLARE_PER_CPU(struct sched_domain_shared __rcu *, sd_llc_shared); > DECLARE_PER_CPU(struct sched_domain_shared __rcu *, sd_balance_shared); > DECLARE_PER_CPU(struct sched_domain __rcu *, sd_numa); > DECLARE_PER_CPU(struct sched_domain __rcu *, sd_asym_packing);