[PATCH] ia64: export node_distance function

[PATCH] ia64: export node_distance function

[Matias Bjørling]

The numa_slit variable used by node_distance is available to a
module as long as it is linked at compile-time. However, it is
not available to loadable modules. Leading to errors such as:

  ERROR: "numa_slit" [drivers/nvme/host/nvme-core.ko] undefined!

The error above is caused by the nvme multipath code that makes
use of node_distance for its path calculation. When the patch was
added, the lightnvm subsystem would select nvme and always compile
it in, leading to the node_distance call to always succeed.
However, when this requirement was removed, nvme could be compiled
in as a module, which exposed this bug.

This patch extracts node_distance to a function and exports it.
Since ACPI is depending on node_distance being a simple lookup to
numa_slit, the previous behavior is exposed as slit_distance and its
users updated.

Fixes: f333444708f82 "nvme: take node locality into account when selecting a path"
Fixes: 73569e11032f "lightnvm: remove dependencies on BLK_DEV_NVME and PCI"
Signed-off-by: Matias Bj√∏ring <mb@lightnvm.io>
---
 arch/ia64/include/asm/numa.h | 4 +++-
 arch/ia64/kernel/acpi.c      | 6 +++---
 arch/ia64/mm/numa.c          | 6 ++++++
 3 files changed, 12 insertions(+), 4 deletions(-)

diff --git a/arch/ia64/include/asm/numa.h b/arch/ia64/include/asm/numa.h
index ebef7f40aabb..c5c253cb9bd6 100644
--- a/arch/ia64/include/asm/numa.h
+++ b/arch/ia64/include/asm/numa.h
@@ -59,7 +59,9 @@ extern struct node_cpuid_s node_cpuid[NR_CPUS];
  */
 
 extern u8 numa_slit[MAX_NUMNODES * MAX_NUMNODES];
-#define node_distance(from,to) (numa_slit[(from) * MAX_NUMNODES + (to)])
+#define slit_distance(from,to) (numa_slit[(from) * MAX_NUMNODES + (to)])
+extern int __node_distance(int from, int to);
+#define node_distance(from,to) __node_distance(from, to)
 
 extern int paddr_to_nid(unsigned long paddr);
 
diff --git a/arch/ia64/kernel/acpi.c b/arch/ia64/kernel/acpi.c
index 1dacbf5e9e09..41eb281709da 100644
--- a/arch/ia64/kernel/acpi.c
+++ b/arch/ia64/kernel/acpi.c
@@ -578,8 +578,8 @@ void __init acpi_numa_fixup(void)
 	if (!slit_table) {
 		for (i = 0; i < MAX_NUMNODES; i++)
 			for (j = 0; j < MAX_NUMNODES; j++)
-				node_distance(i, j) = i = j ? LOCAL_DISTANCE :
-							REMOTE_DISTANCE;
+				slit_distance(i, j) = i = j ?
+					LOCAL_DISTANCE : REMOTE_DISTANCE;
 		return;
 	}
 
@@ -592,7 +592,7 @@ void __init acpi_numa_fixup(void)
 			if (!pxm_bit_test(j))
 				continue;
 			node_to = pxm_to_node(j);
-			node_distance(node_from, node_to) +			slit_distance(node_from, node_to)  			    slit_table->entry[i * slit_table->locality_count + j];
 		}
 	}
diff --git a/arch/ia64/mm/numa.c b/arch/ia64/mm/numa.c
index aa19b7ac8222..5769d4b21270 100644
--- a/arch/ia64/mm/numa.c
+++ b/arch/ia64/mm/numa.c
@@ -36,6 +36,12 @@ struct node_cpuid_s node_cpuid[NR_CPUS]   */
 u8 numa_slit[MAX_NUMNODES * MAX_NUMNODES];
 
+int __node_distance(int from, int to)
+{
+	return slit_distance(from, to);
+}
+EXPORT_SYMBOL(__node_distance);
+
 /* Identify which cnode a physical address resides on */
 int
 paddr_to_nid(unsigned long paddr)
-- 
2.17.1

Re: [PATCH] ia64: export node_distance function

[Matias Bjørling]

On 11/03/2018 07:37 PM, Matias Bjørling wrote:
> The numa_slit variable used by node_distance is available to a
> module as long as it is linked at compile-time. However, it is
> not available to loadable modules. Leading to errors such as:
> 
>    ERROR: "numa_slit" [drivers/nvme/host/nvme-core.ko] undefined!
> 
> The error above is caused by the nvme multipath code that makes
> use of node_distance for its path calculation. When the patch was
> added, the lightnvm subsystem would select nvme and always compile
> it in, leading to the node_distance call to always succeed.
> However, when this requirement was removed, nvme could be compiled
> in as a module, which exposed this bug.
> 
> This patch extracts node_distance to a function and exports it.
> Since ACPI is depending on node_distance being a simple lookup to
> numa_slit, the previous behavior is exposed as slit_distance and its
> users updated.
> 
> Fixes: f333444708f82 "nvme: take node locality into account when selecting a path"
> Fixes: 73569e11032f "lightnvm: remove dependencies on BLK_DEV_NVME and PCI"
> Signed-off-by: Matias Bjøring <mb@lightnvm.io>
> ---
>   arch/ia64/include/asm/numa.h | 4 +++-
>   arch/ia64/kernel/acpi.c      | 6 +++---
>   arch/ia64/mm/numa.c          | 6 ++++++
>   3 files changed, 12 insertions(+), 4 deletions(-)
> 
> diff --git a/arch/ia64/include/asm/numa.h b/arch/ia64/include/asm/numa.h
> index ebef7f40aabb..c5c253cb9bd6 100644
> --- a/arch/ia64/include/asm/numa.h
> +++ b/arch/ia64/include/asm/numa.h
> @@ -59,7 +59,9 @@ extern struct node_cpuid_s node_cpuid[NR_CPUS];
>    */
>   
>   extern u8 numa_slit[MAX_NUMNODES * MAX_NUMNODES];
> -#define node_distance(from,to) (numa_slit[(from) * MAX_NUMNODES + (to)])
> +#define slit_distance(from,to) (numa_slit[(from) * MAX_NUMNODES + (to)])
> +extern int __node_distance(int from, int to);
> +#define node_distance(from,to) __node_distance(from, to)
>   
>   extern int paddr_to_nid(unsigned long paddr);
>   
> diff --git a/arch/ia64/kernel/acpi.c b/arch/ia64/kernel/acpi.c
> index 1dacbf5e9e09..41eb281709da 100644
> --- a/arch/ia64/kernel/acpi.c
> +++ b/arch/ia64/kernel/acpi.c
> @@ -578,8 +578,8 @@ void __init acpi_numa_fixup(void)
>   	if (!slit_table) {
>   		for (i = 0; i < MAX_NUMNODES; i++)
>   			for (j = 0; j < MAX_NUMNODES; j++)
> -				node_distance(i, j) = i = j ? LOCAL_DISTANCE :
> -							REMOTE_DISTANCE;
> +				slit_distance(i, j) = i = j ?
> +					LOCAL_DISTANCE : REMOTE_DISTANCE;
>   		return;
>   	}
>   
> @@ -592,7 +592,7 @@ void __init acpi_numa_fixup(void)
>   			if (!pxm_bit_test(j))
>   				continue;
>   			node_to = pxm_to_node(j);
> -			node_distance(node_from, node_to) > +			slit_distance(node_from, node_to) >   			    slit_table->entry[i * slit_table->locality_count + j];
>   		}
>   	}
> diff --git a/arch/ia64/mm/numa.c b/arch/ia64/mm/numa.c
> index aa19b7ac8222..5769d4b21270 100644
> --- a/arch/ia64/mm/numa.c
> +++ b/arch/ia64/mm/numa.c
> @@ -36,6 +36,12 @@ struct node_cpuid_s node_cpuid[NR_CPUS] >    */
>   u8 numa_slit[MAX_NUMNODES * MAX_NUMNODES];
>   
> +int __node_distance(int from, int to)
> +{
> +	return slit_distance(from, to);
> +}
> +EXPORT_SYMBOL(__node_distance);
> +
>   /* Identify which cnode a physical address resides on */
>   int
>   paddr_to_nid(unsigned long paddr)
> 

Tony and Fenghua, could you please take a look at the above patch? 
kbuild has been bugging me for a fix.

[PATCH] ia64: export node_distance function

[Tony Luck]

From: Matias Bjørling <mb@lightnvm.io>

The numa_slit variable used by node_distance is available to a
module as long as it is linked at compile-time. However, it is
not available to loadable modules. Leading to errors such as:

  ERROR: "numa_slit" [drivers/nvme/host/nvme-core.ko] undefined!

The error above is caused by the nvme multipath code that makes
use of node_distance for its path calculation. When the patch was
added, the lightnvm subsystem would select nvme and always compile
it in, leading to the node_distance call to always succeed.
However, when this requirement was removed, nvme could be compiled
in as a module, which exposed this bug.

This patch extracts node_distance to a function and exports it.
Since ACPI is depending on node_distance being a simple lookup to
numa_slit, the previous behavior is exposed as slit_distance and its
users updated.

Fixes: f333444708f82 "nvme: take node locality into account when selecting a path"
Fixes: 73569e11032f "lightnvm: remove dependencies on BLK_DEV_NVME and PCI"
Signed-off-by: Matias Bjøring <mb@lightnvm.io>
Signed-off-by: Tony Luck <tony.luck@intel.com>
---
 arch/ia64/include/asm/numa.h | 4 +++-
 arch/ia64/kernel/acpi.c      | 6 +++---
 arch/ia64/mm/numa.c          | 6 ++++++
 3 files changed, 12 insertions(+), 4 deletions(-)

diff --git a/arch/ia64/include/asm/numa.h b/arch/ia64/include/asm/numa.h
index ebef7f40aabb..c5c253cb9bd6 100644
--- a/arch/ia64/include/asm/numa.h
+++ b/arch/ia64/include/asm/numa.h
@@ -59,7 +59,9 @@ extern struct node_cpuid_s node_cpuid[NR_CPUS];
  */
 
 extern u8 numa_slit[MAX_NUMNODES * MAX_NUMNODES];
-#define node_distance(from,to) (numa_slit[(from) * MAX_NUMNODES + (to)])
+#define slit_distance(from,to) (numa_slit[(from) * MAX_NUMNODES + (to)])
+extern int __node_distance(int from, int to);
+#define node_distance(from,to) __node_distance(from, to)
 
 extern int paddr_to_nid(unsigned long paddr);
 
diff --git a/arch/ia64/kernel/acpi.c b/arch/ia64/kernel/acpi.c
index 1dacbf5e9e09..41eb281709da 100644
--- a/arch/ia64/kernel/acpi.c
+++ b/arch/ia64/kernel/acpi.c
@@ -578,8 +578,8 @@ void __init acpi_numa_fixup(void)
 	if (!slit_table) {
 		for (i = 0; i < MAX_NUMNODES; i++)
 			for (j = 0; j < MAX_NUMNODES; j++)
-				node_distance(i, j) = i = j ? LOCAL_DISTANCE :
-							REMOTE_DISTANCE;
+				slit_distance(i, j) = i = j ?
+					LOCAL_DISTANCE : REMOTE_DISTANCE;
 		return;
 	}
 
@@ -592,7 +592,7 @@ void __init acpi_numa_fixup(void)
 			if (!pxm_bit_test(j))
 				continue;
 			node_to = pxm_to_node(j);
-			node_distance(node_from, node_to) +			slit_distance(node_from, node_to)  			    slit_table->entry[i * slit_table->locality_count + j];
 		}
 	}
diff --git a/arch/ia64/mm/numa.c b/arch/ia64/mm/numa.c
index 3861d6e32d5f..a03803506b0c 100644
--- a/arch/ia64/mm/numa.c
+++ b/arch/ia64/mm/numa.c
@@ -36,6 +36,12 @@ struct node_cpuid_s node_cpuid[NR_CPUS]   */
 u8 numa_slit[MAX_NUMNODES * MAX_NUMNODES];
 
+int __node_distance(int from, int to)
+{
+	return slit_distance(from, to);
+}
+EXPORT_SYMBOL(__node_distance);
+
 /* Identify which cnode a physical address resides on */
 int
 paddr_to_nid(unsigned long paddr)
-- 
2.19.1

Re: [PATCH] ia64: export node_distance function

[Linus Torvalds]

On Mon, Nov 26, 2018 at 11:27 AM Tony Luck <tony.luck@intel.com> wrote:
>
> The numa_slit variable used by node_distance is available to a
> module as long as it is linked at compile-time. However, it is
> not available to loadable modules. Leading to errors such as:
[...]

Tony: Just checking - you aren't planning on doing a git pull request,
and want me to apply this directly as a patch?

             Linus

Re: [PATCH] ia64: export node_distance function

[Luck, Tony]

Correct. Unless you’d prefer it as a git pull.

Sent from my iPhone

> On Nov 26, 2018, at 18:17, Linus Torvalds <torvalds@linux-foundation.org> wrote:
> 
>> On Mon, Nov 26, 2018 at 11:27 AM Tony Luck <tony.luck@intel.com> wrote:
>> 
>> The numa_slit variable used by node_distance is available to a
>> module as long as it is linked at compile-time. However, it is
>> not available to loadable modules. Leading to errors such as:
> [...]
> 
> Tony: Just checking - you aren't planning on doing a git pull request,
> and want me to apply this directly as a patch?
> 
>             Linus

Re: [PATCH] ia64: export node_distance function

[Linus Torvalds]

On Mon, Nov 26, 2018 at 6:28 PM Luck, Tony <tony.luck@intel.com> wrote:
>
> Correct. Unless you’d prefer it as a git pull.

I have applied it as a patch - it's just that if you're somebody I
know sends me git pull requests _too_, then it's generally best to
tell me explicitly that this patch is supposed to be applied as a
patch.

Otherwise it's ambiguous - am I cc'd for some informational reason and
I'll get it later in a git pull request, or did you send it to me
explicitly because you're _not_ planning on sending it as a git pull..

Put another way: if I'm not sure what I'm supposed to do about a
patch, mistakes happen (ie I can go "ok, I saw it, I'll archive it and
get it later through git").

                  Linus