[PATCH] powerpc/papr_scm: Update the NUMA distance table for the target node

[PATCH] powerpc/papr_scm: Update the NUMA distance table for the target node

[Aneesh Kumar K.V]

platform device helper routines won't update the NUMA distance table
while creating a platform device, even if the device is present on
a NUMA node that doesn't have memory or CPU. This is especially true
for pmem devices. If the target node of the pmem device is not online, we
find the nearest online node to the device and associate the pmem
device with that online node. To find the nearest online node, we should
have the numa distance table updated correctly. Update the distance
information during the device probe.

distance_lookup_table value for distance_ref_points_depth = 2 before and after
fix is below
node 3 distance depth 0  - 0
node 3 distance depth 1  - 0
node 4 distance depth 0  - 4
node 4 distance depth 1  - 2
node 5 distance depth 0  - 5
node 5 distance depth 1  - 1

after fix
node 3 distance depth 0  - 3
node 3 distance depth 1  - 1
node 4 distance depth 0  - 4
node 4 distance depth 1  - 2
node 5 distance depth 0  - 5
node 5 distance depth 1  - 1

Without the fix, the nearest numa node to the pmem device will be picked as 4.
After the fix, we get the correct numa node which is 5.

Fixes: da1115fdbd6e ("powerpc/nvdimm: Pick nearby online node if the device node is not online")
Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.ibm.com>
---
 arch/powerpc/platforms/pseries/papr_scm.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/arch/powerpc/platforms/pseries/papr_scm.c b/arch/powerpc/platforms/pseries/papr_scm.c
index 2f8385523a13..5bef75714bd5 100644
--- a/arch/powerpc/platforms/pseries/papr_scm.c
+++ b/arch/powerpc/platforms/pseries/papr_scm.c
@@ -1428,6 +1428,10 @@ static int papr_scm_probe(struct platform_device *pdev)
 		return -ENODEV;
 	}
 
+	/*
+	 * of platform device create won't update the numa distance table
+	 */
+	update_numa_distance(dn);
 
 	p = kzalloc(sizeof(*p), GFP_KERNEL);
 	if (!p)
-- 
2.39.2

Re: [PATCH] powerpc/papr_scm: Update the NUMA distance table for the target node

[Aneesh Kumar K.V]

"Aneesh Kumar K.V" <aneesh.kumar@linux.ibm.com> writes:

> platform device helper routines won't update the NUMA distance table
> while creating a platform device, even if the device is present on
> a NUMA node that doesn't have memory or CPU. This is especially true
> for pmem devices. If the target node of the pmem device is not online, we
> find the nearest online node to the device and associate the pmem
> device with that online node. To find the nearest online node, we should
> have the numa distance table updated correctly. Update the distance
> information during the device probe.
>
> distance_lookup_table value for distance_ref_points_depth = 2 before and after
> fix is below
> node 3 distance depth 0  - 0
> node 3 distance depth 1  - 0
> node 4 distance depth 0  - 4
> node 4 distance depth 1  - 2
> node 5 distance depth 0  - 5
> node 5 distance depth 1  - 1
>
> after fix
> node 3 distance depth 0  - 3
> node 3 distance depth 1  - 1
> node 4 distance depth 0  - 4
> node 4 distance depth 1  - 2
> node 5 distance depth 0  - 5
> node 5 distance depth 1  - 1
>
> Without the fix, the nearest numa node to the pmem device will be picked as 4.
> After the fix, we get the correct numa node which is 5.
>
> Fixes: da1115fdbd6e ("powerpc/nvdimm: Pick nearby online node if the device node is not online")
> Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.ibm.com>
> ---
>  arch/powerpc/platforms/pseries/papr_scm.c | 4 ++++
>  1 file changed, 4 insertions(+)
>
> diff --git a/arch/powerpc/platforms/pseries/papr_scm.c b/arch/powerpc/platforms/pseries/papr_scm.c
> index 2f8385523a13..5bef75714bd5 100644
> --- a/arch/powerpc/platforms/pseries/papr_scm.c
> +++ b/arch/powerpc/platforms/pseries/papr_scm.c
> @@ -1428,6 +1428,10 @@ static int papr_scm_probe(struct platform_device *pdev)
>  		return -ENODEV;
>  	}
>  
> +	/*
> +	 * of platform device create won't update the numa distance table
> +	 */
> +	update_numa_distance(dn);
>  
>  	p = kzalloc(sizeof(*p), GFP_KERNEL);
>  	if (!p)
> -- 
> 2.39.2

This also requires export of update_numa_distance()

diff --git a/arch/powerpc/mm/numa.c b/arch/powerpc/mm/numa.c
index b44ce71917d7..16cfe56be05b 100644
--- a/arch/powerpc/mm/numa.c
+++ b/arch/powerpc/mm/numa.c
@@ -366,6 +366,7 @@ void update_numa_distance(struct device_node *node)
 	WARN(numa_distance_table[nid][nid] == -1,
 	     "NUMA distance details for node %d not provided\n", nid);
 }
+EXPORT_SYMBOL_GPL(update_numa_distance);
 
 /*
  * ibm,numa-lookup-index-table= {N, domainid1, domainid2, ..... domainidN}