From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C2304480327 for ; Thu, 22 Jan 2026 08:12:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769069562; cv=none; b=JW/IDIolQXxnmar6m6WH+/hF5ijpM7DlB1rpNuBmLSwihjkDvG9sIwGSTm0968F4kDvvULU0LwGljdxvDMMsA9do9A2WUIpS+Wxx3M7PPXGIUfyQW/7EeK4LT43fxt5znOWNikqLq1xdz6a6GG1Kqb9Izc00WXQlt9GdKB00kCM= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769069562; c=relaxed/simple; bh=4cnDmwhxJyq4LnX3ynL8YjZRCAHxbbvc/NQCLM5Zizk=; h=Message-ID:Date:MIME-Version:Subject:To:Cc:References:From: In-Reply-To:Content-Type; b=j7nC6jN36NG1BrrBCD/xh/0s2W7uJwVzl1/lppFwbe0eKxAjkbK/h+wB5x7pYtL32Qnc7yPnCuDsMfvAut9VCwD72d2eKDohyP+EEziuxNWwnoy/Uy6MoDG/pfEAk0QTm0AGdGVPxaLmuDeVUe5OOU3Mz3ctqM4k/jAxmQPmJ7Y= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=FWl85WKy; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="FWl85WKy" Received: from pps.filterd (m0360072.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 60M0P3uM021452; Thu, 22 Jan 2026 08:12:19 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=pp1; bh=IjDECe YDupgR6P0OI9/BzcbX0gvN/azKYEyvan++4yA=; b=FWl85WKyUALX7p6uxoGm6t MB/jeaMWW8CMEQAnuKUja6yephfxUIp+9rTwIfAyo+1EWHV26vPbdY+Mwg6c+1ih NynaPo2YppilXDPo033OzodS5jzidCWlHpFqt6Y5V8WwclaUpFyW9Qd3WKT1svSk 3o7zUaxGUiz3aZeiVxftpcUTwPPVBxFlnS55UiwqY9x2t5qXsrB8Gt0PGscHr77B DVUWNwaY2d84WNzVVePPKiCDwaT7UFiQUrQEd4gfaRG2NQ6lt22nMBKLhr0Ymc3h e2S4fkuKNkW6KNu8n4wElY0ZncLn9SMgFlpMLbfeXJ5/Ob+Gwvd6qPlqoLKVt0dA == Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4bt612bhn6-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 22 Jan 2026 08:12:18 +0000 (GMT) Received: from m0360072.ppops.net (m0360072.ppops.net [127.0.0.1]) by pps.reinject (8.18.1.12/8.18.0.8) with ESMTP id 60M86BTe014985; Thu, 22 Jan 2026 08:12:18 GMT Received: from ppma11.dal12v.mail.ibm.com (db.9e.1632.ip4.static.sl-reverse.com [50.22.158.219]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4bt612bhn3-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 22 Jan 2026 08:12:18 +0000 (GMT) Received: from pps.filterd (ppma11.dal12v.mail.ibm.com [127.0.0.1]) by ppma11.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 60M7Q29X006424; Thu, 22 Jan 2026 08:12:17 GMT Received: from smtprelay07.fra02v.mail.ibm.com ([9.218.2.229]) by ppma11.dal12v.mail.ibm.com (PPS) with ESMTPS id 4brqf1rnbc-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 22 Jan 2026 08:12:17 +0000 Received: from smtpav02.fra02v.mail.ibm.com (smtpav02.fra02v.mail.ibm.com [10.20.54.101]) by smtprelay07.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 60M8CFkJ44237158 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 22 Jan 2026 08:12:15 GMT Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 7A3382004B; Thu, 22 Jan 2026 08:12:15 +0000 (GMT) Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 29B4620043; Thu, 22 Jan 2026 08:12:13 +0000 (GMT) Received: from [9.109.215.252] (unknown [9.109.215.252]) by smtpav02.fra02v.mail.ibm.com (Postfix) with ESMTP; Thu, 22 Jan 2026 08:12:12 +0000 (GMT) Message-ID: Date: Thu, 22 Jan 2026 13:42:12 +0530 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v3 3/8] sched/topology: Switch to assigning "sd->shared" from s_data To: K Prateek Nayak Cc: Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider , Chen Yu , "Gautham R. Shenoy" , Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , linux-kernel@vger.kernel.org References: <20260120113246.27987-1-kprateek.nayak@amd.com> <20260120113246.27987-4-kprateek.nayak@amd.com> Content-Language: en-US From: Shrikanth Hegde In-Reply-To: <20260120113246.27987-4-kprateek.nayak@amd.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-TM-AS-GCONF: 00 X-Proofpoint-GUID: AWkKacK-hChGCeaZzVnBUtlnttuBzG-Z X-Authority-Analysis: v=2.4 cv=LaIxKzfi c=1 sm=1 tr=0 ts=6971dbe2 cx=c_pps a=aDMHemPKRhS1OARIsFnwRA==:117 a=aDMHemPKRhS1OARIsFnwRA==:17 a=IkcTkHD0fZMA:10 a=vUbySO9Y5rIA:10 a=VkNPw1HP01LnGYTKEx00:22 a=VwQbUJbxAAAA:8 a=zd2uoN0lAAAA:8 a=X6NZ1-IAEnJzT8Q8dDsA:9 a=QEXdDO2ut3YA:10 X-Proofpoint-ORIG-GUID: AT-QWGlD6QY_I440WLCw_OXTnxP6y-aK X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwMTIyMDA1MCBTYWx0ZWRfXxskS529dfevW drE6rgyZfDdW2Lw8frx7W931rasB8GSndCjlcF+WiwpnkU/m2oWa5Fb861767qnVPtNk3uqsglG nMZ0BtHeCISgTv1uG1onqv5ROth55Gbm6KDu2gL+tNtR09MTpiSEygHqFkCZIKqZtqsL/UYvq8C YlJzGyajEFSTU6BepXHquWVpb0oBoNFFYTLSPwGtkVHK17dx199wvQQE3vO0GkgcYiwqip94/4G Qm7bIX66ZRcuCJVbt+dCFJJHSi8siDMe62q3TXb/CJ2nT8niWBkTjwcTBindTkxGFkrLDfToz/w WbVUGKLDhyRhx1j1yUiyPV8f+wHVOpnM+oFtKxrVWiwhDmUtPLAg9GKYjkTnSc3kI/+P/QL+52x vqkHk31Zh2hbkgxpvYsyuqoczti8rcd5dyrAxreuoLMvajdKyyxPiTcRzQkCccYdR1c62nOpC84 wi4bdrrMS34q5Xh/BWA== X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.20,FMLib:17.12.100.49 definitions=2026-01-21_04,2026-01-20_01,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 bulkscore=0 adultscore=0 phishscore=0 priorityscore=1501 lowpriorityscore=0 suspectscore=0 clxscore=1015 impostorscore=0 spamscore=0 malwarescore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2601150000 definitions=main-2601220050 On 1/20/26 5:02 PM, K Prateek Nayak wrote: > Use the "sched_domain_shared" object allocated in s_data for > "sd->shared" assignments. Assign "sd->shared" for the topmost > SD_SHARE_LLC domain before degeneration and rely on the degeneration > path to correctly pass down the shared object to "sd_llc". > > sd_degenerate_parent() ensures degenerating domains must have the same > sched_domain_span() which ensures 1:1 passing down of the shared object. > If the topmost SD_SHARE_LLC domain degenerates, the shared object is > freed from destroy_sched_domain() when the last reference is dropped. > > build_sched_domains() NULLs out the objects that have been assigned as > "sd->shared" and the unassigned ones are freed from the __sds_free() > path. > > Post cpu_attach_domain(), all reclaims of "sd->shared" are handled via > call_rcu() on the sched_domain object via destroy_sched_domains_rcu(). > > Signed-off-by: K Prateek Nayak > --- > Changelog rfc v2..v3: > > o Broke off from a single large patch. Previously > https://lore.kernel.org/lkml/20251208092744.32737-3-kprateek.nayak@amd.com/ > --- > kernel/sched/topology.c | 34 ++++++++++++++++++++++++---------- > 1 file changed, 24 insertions(+), 10 deletions(-) > > diff --git a/kernel/sched/topology.c b/kernel/sched/topology.c > index 623e8835d322..0f56462fef6f 100644 > --- a/kernel/sched/topology.c > +++ b/kernel/sched/topology.c > @@ -679,6 +679,9 @@ static void update_top_cache_domain(int cpu) > if (sd) { > id = cpumask_first(sched_domain_span(sd)); > size = cpumask_weight(sched_domain_span(sd)); > + > + /* If sd_llc exists, sd_llc_shared should exist too. */ > + WARN_ON_ONCE(!sd->shared); > sds = sd->shared; > } > > @@ -727,6 +730,13 @@ cpu_attach_domain(struct sched_domain *sd, struct root_domain *rd, int cpu) > if (sd_parent_degenerate(tmp, parent)) { > tmp->parent = parent->parent; > > + /* Pick reference to parent->shared. */ > + if (parent->shared) { > + WARN_ON_ONCE(tmp->shared); > + tmp->shared = parent->shared; > + parent->shared = NULL; > + } > + > if (parent->parent) { > parent->parent->child = tmp; > parent->parent->groups->flags = tmp->flags; > @@ -1732,16 +1742,6 @@ sd_init(struct sched_domain_topology_level *tl, > sd->cache_nice_tries = 1; > } > > - /* > - * For all levels sharing cache; connect a sched_domain_shared > - * instance. > - */ > - if (sd->flags & SD_SHARE_LLC) { > - sd->shared = *per_cpu_ptr(sdd->sds, sd_id); > - atomic_inc(&sd->shared->ref); > - atomic_set(&sd->shared->nr_busy_cpus, sd_weight); > - } > - > sd->private = sdd; > > return sd; > @@ -2655,8 +2655,19 @@ build_sched_domains(const struct cpumask *cpu_map, struct sched_domain_attr *att > unsigned int imb_span = 1; > > for (sd = *per_cpu_ptr(d.sd, i); sd; sd = sd->parent) { > + struct sched_domain *parent = sd->parent; > struct sched_domain *child = sd->child; > > + /* Attach sd->shared to the topmost SD_SHARE_LLC domain. */ > + if ((sd->flags & SD_SHARE_LLC) && > + (!parent || !(parent->flags & SD_SHARE_LLC))) { > + int llc_id = cpumask_first(sched_domain_span(sd)); > + > + sd->shared = *per_cpu_ptr(d.sds, llc_id); > + atomic_set(&sd->shared->nr_busy_cpus, sd->span_weight); > + atomic_inc(&sd->shared->ref); > + } > + > if (!(sd->flags & SD_SHARE_LLC) && child && > (child->flags & SD_SHARE_LLC)) { > struct sched_domain __rcu *top_p; > @@ -2709,6 +2720,9 @@ build_sched_domains(const struct cpumask *cpu_map, struct sched_domain_attr *att > if (!cpumask_test_cpu(i, cpu_map)) > continue; > > + if (atomic_read(&(*per_cpu_ptr(d.sds, i))->ref)) > + *per_cpu_ptr(d.sds, i) = NULL; > + Can we do this claim_allocations only? sdt_alloc and free is complicated already. > for (sd = *per_cpu_ptr(d.sd, i); sd; sd = sd->parent) { > claim_allocations(i, sd); > init_sched_groups_capacity(i, sd);