From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751304AbdE3Rzl (ORCPT ); Tue, 30 May 2017 13:55:41 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:37844 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751246AbdE3Rzj (ORCPT ); Tue, 30 May 2017 13:55:39 -0400 Subject: Re: [PATCH V4 1/2] powerpc/numa: Update CPU topology when VPHN enabled To: Michael Bringmann , linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org Cc: Tyrel Datwyler , Andrew Donnellan , Sahil Mehta , Rashmica Gupta , Reza Arbab , Ingo Molnar , John Allen , Paul Mackerras , Bharata B Rao , Shailendra Singh , Thomas Gleixner , Sebastian Andrzej Siewior , "Aneesh Kumar K.V" References: <89d48cc9-d730-b299-c81e-83d77e22100d@linux.vnet.ibm.com> From: Nathan Fontenot Date: Tue, 30 May 2017 12:55:31 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.1.1 MIME-Version: 1.0 In-Reply-To: <89d48cc9-d730-b299-c81e-83d77e22100d@linux.vnet.ibm.com> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-TM-AS-GCONF: 00 x-cbid: 17053017-0028-0000-0000-000007B339FE X-IBM-SpamModules-Scores: X-IBM-SpamModules-Versions: BY=3.00007146; HX=3.00000241; KW=3.00000007; PH=3.00000004; SC=3.00000212; SDB=6.00867819; UDB=6.00431210; IPR=6.00647698; BA=6.00005387; NDR=6.00000001; ZLA=6.00000005; ZF=6.00000009; ZB=6.00000000; ZP=6.00000000; ZH=6.00000000; ZU=6.00000002; MB=3.00015648; XFM=3.00000015; UTC=2017-05-30 17:55:37 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 17053017-0029-0000-0000-000035FD672B Message-Id: X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:,, definitions=2017-05-30_12:,, signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 suspectscore=0 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1703280000 definitions=main-1705300330 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 05/26/2017 04:29 PM, Michael Bringmann wrote: > > powerpc/numa: Correct the currently broken capability to set the > topology for shared CPUs in LPARs. At boot time for shared CPU > lpars, the topology for each shared CPU is set to node zero, however, > this is now updated correctly using the Virtual Processor Home Node > (VPHN) capabilities information provided by the pHyp. The VPHN handling > in Linux is disabled, if PRRN handling is present. I'm still not sure this is what we want. Looking at the topology updating code, we only enable VPHN if PRRN is not present. My understanding of the current situation is that the node for partitions with shared cpus are not set. The reason for this is that the device tree presented to a partition using shared cpus at boot puts all cpus in node zero and then uses the VPHN capability to inform the partition which node each cpu really belongs to. Additionally, I think this is how DLPAR of shared cpu partitions work. After the cpu is DLPAR added we should get a VPHN notification to inform us of the true node that the cpu belongs to. When the PRRN capability was introduced it was thought to be a follow-on to the VPHN capability and so the code to start topology updating only enables VPHN if PRRN is not present. I think what we need to do is always enable VPHN for shared cpu partitions. -Nathan > > Signed-off-by: Michael Bringmann > --- > Changes in V4: > -- Fix conditional compile bug. > --- > arch/powerpc/mm/numa.c | 19 ++++++++++++++++++- > arch/powerpc/platforms/pseries/dlpar.c | 2 ++ > 2 files changed, 20 insertions(+), 1 deletion(-) > > diff --git a/arch/powerpc/mm/numa.c b/arch/powerpc/mm/numa.c > index 371792e..afcee3f 100644 > --- a/arch/powerpc/mm/numa.c > +++ b/arch/powerpc/mm/numa.c > @@ -29,6 +29,7 @@ > #include > #include > #include > +#include > #include > #include > #include > @@ -1153,6 +1154,8 @@ struct topology_update_data { > static int vphn_enabled; > static int prrn_enabled; > static void reset_topology_timer(void); > +static int topology_inited; > +static int topology_update_needed; > > /* > * Store the current values of the associativity change counters in the > @@ -1321,8 +1324,11 @@ int arch_update_cpu_topology(void) > struct device *dev; > int weight, new_nid, i = 0; > > - if (!prrn_enabled && !vphn_enabled) > + if (!prrn_enabled && !vphn_enabled) { > + if (!topology_inited) > + topology_update_needed = 1; > return 0; > + } > > weight = cpumask_weight(&cpu_associativity_changes_mask); > if (!weight) > @@ -1361,6 +1367,8 @@ int arch_update_cpu_topology(void) > cpumask_andnot(&cpu_associativity_changes_mask, > &cpu_associativity_changes_mask, > cpu_sibling_mask(cpu)); > + pr_info("Assoc chg gives same node %d for cpu%d\n", > + new_nid, cpu); > cpu = cpu_last_thread_sibling(cpu); > continue; > } > @@ -1377,6 +1385,9 @@ int arch_update_cpu_topology(void) > cpu = cpu_last_thread_sibling(cpu); > } > > + if (i) > + updates[i-1].next = NULL; > + > pr_debug("Topology update for the following CPUs:\n"); > if (cpumask_weight(&updated_cpus)) { > for (ud = &updates[0]; ud; ud = ud->next) { > @@ -1423,6 +1434,7 @@ int arch_update_cpu_topology(void) > > out: > kfree(updates); > + topology_update_needed = 0; > return changed; > } > > @@ -1600,6 +1612,11 @@ static int topology_update_init(void) > if (!proc_create("powerpc/topology_updates", 0644, NULL, &topology_ops)) > return -ENOMEM; > > + topology_inited = 1; > + if (topology_update_needed) > + bitmap_fill(cpumask_bits(&cpu_associativity_changes_mask), > + nr_cpumask_bits); > + > return 0; > } > device_initcall(topology_update_init); > diff --git a/arch/powerpc/platforms/pseries/dlpar.c b/arch/powerpc/platforms/pseries/dlpar.c > index bda18d8..5106263 100644 > --- a/arch/powerpc/platforms/pseries/dlpar.c > +++ b/arch/powerpc/platforms/pseries/dlpar.c > @@ -592,6 +592,8 @@ static ssize_t dlpar_show(struct class *class, struct class_attribute *attr, > > static int __init pseries_dlpar_init(void) > { > + arch_update_cpu_topology(); > + > pseries_hp_wq = alloc_workqueue("pseries hotplug workqueue", > WQ_UNBOUND, 1); > return sysfs_create_file(kernel_kobj, &class_attr_dlpar.attr); >