RE: [Discontig-devel] RE: Cleanup of NUMA support in ACPI

RE: [Discontig-devel] RE: Cleanup of NUMA support in ACPI

[Grover, Andrew]

> From: KOCHI, Takayoshi [mailto:t-kouchi-dPjYVeZdYcz+G+EEi5ephHgSJqDPrsil@public.gmane.org] 
> Here's the patch.
> This is generated against 2.5.30 kernel's driver.
> For 2.4.18 + 20020725, I saw faiures against Config.in and Makefile...
> perhaps it's easy to merge.
> 
> 
> For discontig/numa developers:
> 
> If this is applied, both i386 and ia64 have to provide their own
> functions in architecture-dependent numa support (only when 
> CONFIG_NUMA and
> CONFIG_ACPI_NUMA are both Y):

The strange thing is that ia32 doesn't have CONFIG_NUMA, it has
CONFIG_MULTIQUAD. Are these equivalent, or more importantly, do they imply
the use of SLIT and SRAT?

For now I'll just apply this, and we can sort out the details later. :)

Regards -- Andy


-------------------------------------------------------
This sf.net email is sponsored by: OSDN - Tired of that same old
cell phone?  Get a new here for FREE!
https://www.inphonic.com/r.asp?r=sourceforge1&refcode1=vs3390

RE: [Discontig-devel] RE: Cleanup of NUMA support in ACPI

[Martin J. Bligh]

> The strange thing is that ia32 doesn't have CONFIG_NUMA, it has
> CONFIG_MULTIQUAD. Are these equivalent, or more importantly, do they imply
> the use of SLIT and SRAT?

We do have CONFIG_NUMA when we do discontigmem support (see the
patches in akpm's tree). CONFIG_MULTIQUAD is just h/w stuff for the NUMA-Q
but we do not really do NUMA aware mem in the main tree.

NUMA-Q does not use SLIT/SRAT, but x440 will use them in conjunction with
CONFIG_NUMA (it's another ia32 NUMA platform). John Stultz (cc'ed) is
working on that.

M.




-------------------------------------------------------
This sf.net email is sponsored by: OSDN - Tired of that same old
cell phone?  Get a new here for FREE!
https://www.inphonic.com/r.asp?r=sourceforge1&refcode1=vs3390

RE: Cleanup of NUMA support in ACPI

[Grover, Andrew]

> From: KOCHI, Takayoshi [mailto:t-kouchi-dPjYVeZdYcz+G+EEi5ephHgSJqDPrsil@public.gmane.org] 
> As I promised, I cleaned up NUMA support of ACPI and divided into
> architecture-dependent part and architecture-independent part.
> 
> In this patch, I introduced a new config parameter 'CONFIG_ACPI_NUMA'.
> At this moment, it is independent of CONFIG_NUMA but can be
> tristated.  The division is because I'd like to distinguish
> ACPI-dependent NUMA initialization stuff and
> others (like SGI or phoney_srat/slit).

Nice.

My only comment is that instead of the ifdef CONFIG_ACPI_NUMA chunk in
tables.c, this should just go in a separate file, say, drivers/acpi/numa.c.

There are a bunch of other ACPI config options that should be split out like
this as well, so if you don't want to redo the patch, I'll just take it and
it can get fixed in a sweep with the others, later.

BTW I was thinking I should only take the arch-neutral changes, and let you
push the rest through David. Does that sound right?

Regards -- Andy


-------------------------------------------------------
This sf.net email is sponsored by: Dice - The leading online job board
for high-tech professionals. Search and apply for tech jobs today!
http://seeker.dice.com/seeker.epl?rel_code=31

Re: [Discontig-devel] RE: Cleanup of NUMA support in ACPI

[KOCHI, Takayoshi]

Hi Andy,

On Wed, 14 Aug 2002 10:54:53 -0700
"Grover, Andrew" <andrew.grover-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org> wrote:

> > From: KOCHI, Takayoshi [mailto:t-kouchi-dPjYVeZdYcz+G+EEi5ephHgSJqDPrsil@public.gmane.org] 
> > As I promised, I cleaned up NUMA support of ACPI and divided into
> > architecture-dependent part and architecture-independent part.
> > 
> > In this patch, I introduced a new config parameter 'CONFIG_ACPI_NUMA'.
> > At this moment, it is independent of CONFIG_NUMA but can be
> > tristated.  The division is because I'd like to distinguish
> > ACPI-dependent NUMA initialization stuff and
> > others (like SGI or phoney_srat/slit).
> 
> Nice.
> 
> My only comment is that instead of the ifdef CONFIG_ACPI_NUMA chunk in
> tables.c, this should just go in a separate file, say, drivers/acpi/numa.c.

My only concern was that as parsing of MADT and SRAT is mostly common,
we have to change common functions from static to extern to
split numa functions to other file.
But I don't see any problem because these function names are
not invasive to others.

> There are a bunch of other ACPI config options that should be split out like
> this as well, so if you don't want to redo the patch, I'll just take it and
> it can get fixed in a sweep with the others, later.

Erich has worked to clean up some portions and sent to LKML.
I'll split ACPI part from the patch and send to you this afternoon.

> BTW I was thinking I should only take the arch-neutral changes, and let you
> push the rest through David. Does that sound right?

Yes.
It's the dependency that if you won't accept the changes, David won't...

Thanks,
-- 
KOCHI, Takayoshi <t-kouchi-f7IHDacdhdx8UrSeD/g0lQ@public.gmane.org/t-kouchi>



-------------------------------------------------------
This sf.net email is sponsored by: Dice - The leading online job board
for high-tech professionals. Search and apply for tech jobs today!
http://seeker.dice.com/seeker.epl?rel_code=31

Re: [Discontig-devel] RE: Cleanup of NUMA support in ACPI

[KOCHI, Takayoshi]

Hi Andy,

> > There are a bunch of other ACPI config options that should be split out like
> > this as well, so if you don't want to redo the patch, I'll just take it and
> > it can get fixed in a sweep with the others, later.
> 
> Erich has worked to clean up some portions and sent to LKML.
> I'll split ACPI part from the patch and send to you this afternoon.

Here's the patch.
This is generated against 2.5.30 kernel's driver.
For 2.4.18 + 20020725, I saw faiures against Config.in and Makefile...
perhaps it's easy to merge.


For discontig/numa developers:

If this is applied, both i386 and ia64 have to provide their own
functions in architecture-dependent numa support (only when CONFIG_NUMA and
CONFIG_ACPI_NUMA are both Y):

void __init acpi_numa_slit_init (struct acpi_table_slit *slit);
void __init acpi_numa_processor_affinity_init (struct acpi_table_processor_affinity *pa);
void __init acpi_numa_memory_affinity_init (struct acpi_table_memory_affinity *ma);
void __init acpi_numa_arch_fixup(void);

Also, acpi_numa_init() has to be called in very early stage of
initialization of architecture-dependent setup routine
so that you can use the SRAT/SLIT information for constructing
numa-related structures.

--- linux-2.5.30-orig/drivers/acpi/Config.in	Thu Aug  1 14:16:34 2002
+++ linux-2.5.30-numa/drivers/acpi/Config.in	Wed Aug 14 17:00:37 2002
@@ -31,6 +31,9 @@
       tristate     '  Fan'		CONFIG_ACPI_FAN
       tristate     '  Processor'	CONFIG_ACPI_PROCESSOR
       dep_tristate '  Thermal Zone' 	CONFIG_ACPI_THERMAL $CONFIG_ACPI_PROCESSOR
+      if [ "$CONFIG_NUMA" = "y" ]; then
+        bool       '  NUMA support'     CONFIG_ACPI_NUMA $CONFIG_NUMA
+      fi
       tristate     '  Toshiba Laptop Extras'	CONFIG_ACPI_TOSHIBA
       bool         '  Debug Statements'	CONFIG_ACPI_DEBUG
     fi
@@ -58,6 +61,7 @@
     define_bool CONFIG_ACPI_FAN		n
     define_bool CONFIG_ACPI_PROCESSOR	n
     define_bool CONFIG_ACPI_THERMAL	n
+    define_bool CONFIG_ACPI_NUMA	y
     endmenu
   fi
 
@@ -77,7 +81,10 @@
     tristate     '  Button'		CONFIG_ACPI_BUTTON
     tristate     '  Fan'		CONFIG_ACPI_FAN
     tristate     '  Processor'		CONFIG_ACPI_PROCESSOR
-    dep_tristate '  Thermal Zone' CONFIG_ACPI_THERMAL $CONFIG_ACPI_PROCESSOR
+    dep_tristate '  Thermal Zone'       CONFIG_ACPI_THERMAL $CONFIG_ACPI_PROCESSOR
+    if [ "$CONFIG_NUMA" = "y" ]; then
+      bool       '  NUMA support'       CONFIG_ACPI_NUMA $CONFIG_NUMA
+    fi
     bool         '  Debug Statements' 	CONFIG_ACPI_DEBUG
     endmenu
   fi
--- linux-2.5.30-orig/drivers/acpi/Makefile	Thu Aug  1 14:16:40 2002
+++ linux-2.5.30-numa/drivers/acpi/Makefile	Wed Aug 14 16:27:15 2002
@@ -43,6 +43,7 @@
 obj-$(CONFIG_ACPI_PROCESSOR)	+= processor.o
 obj-$(CONFIG_ACPI_THERMAL)	+= thermal.o
 obj-$(CONFIG_ACPI_SYSTEM)	+= system.o
+obj-$(CONFIG_ACPI_NUMA)		+= numa.o
 obj-$(CONFIG_ACPI_TOSHIBA)	+= toshiba_acpi.o
 
 include $(TOPDIR)/Rules.make
--- linux-2.5.30-orig/drivers/acpi/tables.c	Thu Aug  1 14:16:21 2002
+++ linux-2.5.30-numa/drivers/acpi/tables.c	Wed Aug 14 16:36:26 2002
@@ -224,11 +224,13 @@
 
 
 int __init
-acpi_table_parse_madt (
+acpi_table_parse_madt_family (
 	enum acpi_table_id	id,
+	unsigned long		madt_size,
+	int			entry_id,
 	acpi_madt_entry_handler	handler)
 {
-	struct acpi_table_madt	*madt = NULL;
+	void			*madt = NULL;
 	acpi_table_entry_header	*entry = NULL;
 	unsigned long		count = 0;
 	unsigned long		madt_end = 0;
@@ -240,19 +242,21 @@
 	/* Locate the MADT (if exists). There should only be one. */
 
 	for (i = 0; i < sdt.count; i++) {
-		if (sdt.entry[i].id != ACPI_APIC)
+		if (sdt.entry[i].id != id)
 			continue;
-		madt = (struct acpi_table_madt *)
+		madt = (void *)
 			__acpi_map_table(sdt.entry[i].pa, sdt.entry[i].size);
 		if (!madt) {
-			printk(KERN_WARNING PREFIX "Unable to map MADT\n");
+			printk(KERN_WARNING PREFIX "Unable to map %s\n",
+			       acpi_table_signatures[id]);
 			return -ENODEV;
 		}
 		break;
 	}
 
 	if (!madt) {
-		printk(KERN_WARNING PREFIX "MADT not present\n");
+		printk(KERN_WARNING PREFIX "%s not present\n",
+		       acpi_table_signatures[id]);
 		return -ENODEV;
 	}
 
@@ -261,10 +265,10 @@
 	/* Parse all entries looking for a match. */
 
 	entry = (acpi_table_entry_header *)
-		((unsigned long) madt + sizeof(struct acpi_table_madt));
+		((unsigned long) madt + madt_size);
 
 	while (((unsigned long) entry) < madt_end) {
-		if (entry->type == id) {
+		if (entry->type == entry_id) {
 			count++;
 			handler(entry);
 		}
@@ -276,6 +280,16 @@
 }
 
 
+int __init
+acpi_table_parse_madt (
+	enum acpi_madt_entry_id	id,
+	acpi_madt_entry_handler	handler)
+{
+	return acpi_table_parse_madt_family(ACPI_APIC, sizeof(struct acpi_table_madt),
+					    id, handler);
+}
+
+
 int __init
 acpi_table_parse (
 	enum acpi_table_id	id,
--- linux-2.5.30-orig/drivers/acpi/numa.c	Wed Dec 31 16:00:00 1969
+++ linux-2.5.30-numa/drivers/acpi/numa.c	Wed Aug 14 17:02:02 2002
@@ -0,0 +1,187 @@
+/*
+ *  acpi_numa.c - ACPI NUMA support
+ *
+ *  Copyright (C) 2002 Takayoshi Kochi <t-kouchi-f7IHDacdhdx8UrSeD/g0lQ@public.gmane.org>
+ *
+ * ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ *
+ *  This program is free software; you can redistribute it and/or modify
+ *  it under the terms of the GNU General Public License as published by
+ *  the Free Software Foundation; either version 2 of the License, or
+ *  (at your option) any later version.
+ *
+ *  This program is distributed in the hope that it will be useful,
+ *  but WITHOUT ANY WARRANTY; without even the implied warranty of
+ *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ *  GNU General Public License for more details.
+ *
+ *  You should have received a copy of the GNU General Public License
+ *  along with this program; if not, write to the Free Software
+ *  Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
+ *
+ * ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ *
+ */
+
+#include <linux/config.h>
+#include <linux/init.h>
+#include <linux/kernel.h>
+#include <linux/types.h>
+#include <linux/errno.h>
+#include <linux/acpi.h>
+
+#define PREFIX			"ACPI: "
+
+extern int __init acpi_table_parse_madt_family (enum acpi_table_id id, unsigned long madt_size, int entry_id, acpi_madt_entry_handler handler);
+
+void __init
+acpi_table_print_srat_entry (
+	acpi_table_entry_header	*header)
+{
+	if (!header)
+		return;
+
+	switch (header->type) {
+
+	case ACPI_SRAT_PROCESSOR_AFFINITY:
+	{
+		struct acpi_table_processor_affinity *p =
+			(struct acpi_table_processor_affinity*) header;
+		printk(KERN_INFO PREFIX "SRAT Processor (id[0x%02x] eid[0x%02x]) in proximity domain %d %s\n",
+		       p->apic_id, p->lsapic_eid, p->proximity_domain,
+		       p->flags.enabled?"enabled":"disabled");
+	}
+		break;
+
+	case ACPI_SRAT_MEMORY_AFFINITY:
+	{
+		struct acpi_table_memory_affinity *p =
+			(struct acpi_table_memory_affinity*) header;
+		printk(KERN_INFO PREFIX "SRAT Memory (0x%08x%08x length 0x%08x%08x type 0x%x) in proximity domain %d %s%s\n",
+		       p->base_addr_hi, p->base_addr_lo, p->length_hi, p->length_lo,
+		       p->memory_type, p->proximity_domain,
+		       p->flags.enabled ? "enabled" : "disabled",
+		       p->flags.hot_pluggable ? " hot-pluggable" : "");
+	}
+		break;
+
+	default:
+		printk(KERN_WARNING PREFIX "Found unsupported SRAT entry (type = 0x%x)\n",
+			header->type);
+		break;
+	}
+}
+
+
+static int __init
+acpi_parse_slit (unsigned long phys_addr, unsigned long size)
+{
+	struct acpi_table_slit	*slit;
+	u32			localities;
+
+	if (!phys_addr || !size)
+		return -EINVAL;
+
+	slit = (struct acpi_table_slit *) __va(phys_addr);
+
+	/* downcast just for %llu vs %lu for i386/ia64  */
+	localities = (u32) slit->localities;
+
+	printk(KERN_INFO PREFIX "SLIT localities %ux%u\n", localities, localities);
+
+	acpi_numa_slit_init(slit);
+
+	return 0;
+}
+
+
+static int __init
+acpi_parse_processor_affinity (acpi_table_entry_header *header)
+{
+	struct acpi_table_processor_affinity *processor_affinity = NULL;
+
+	processor_affinity = (struct acpi_table_processor_affinity*) header;
+	if (!processor_affinity)
+		return -EINVAL;
+
+	acpi_table_print_srat_entry(header);
+
+	/* let architecture-dependent part to do it */
+	acpi_numa_processor_affinity_init(processor_affinity);
+
+	return 0;
+}
+
+
+static int __init
+acpi_parse_memory_affinity (acpi_table_entry_header *header)
+{
+	struct acpi_table_memory_affinity *memory_affinity = NULL;
+
+	memory_affinity = (struct acpi_table_memory_affinity*) header;
+	if (!memory_affinity)
+		return -EINVAL;
+
+	acpi_table_print_srat_entry(header);
+
+	/* let architecture-dependent part to do it */
+	acpi_numa_memory_affinity_init(memory_affinity);
+
+	return 0;
+}
+
+
+static int __init
+acpi_parse_srat (unsigned long phys_addr, unsigned long size)
+{
+	struct acpi_table_srat	*srat = NULL;
+
+	if (!phys_addr || !size)
+		return -EINVAL;
+
+	srat = (struct acpi_table_srat *) __va(phys_addr);
+
+	printk(KERN_INFO PREFIX "SRAT revision %d\n", srat->table_revision);
+
+	return 0;
+}
+
+
+int __init
+acpi_table_parse_srat (
+	enum acpi_srat_entry_id	id,
+	acpi_madt_entry_handler	handler)
+{
+	return acpi_table_parse_madt_family(ACPI_SRAT, sizeof(struct acpi_table_srat),
+					    id, handler);
+}
+
+
+int __init
+acpi_numa_init()
+{
+	int			result;
+
+	/* SRAT: Static Resource Affinity Table */
+	result = acpi_table_parse(ACPI_SRAT, acpi_parse_srat);
+
+	if (result > 0) {
+		result = acpi_table_parse_srat(ACPI_SRAT_PROCESSOR_AFFINITY,
+					       acpi_parse_processor_affinity);
+		result = acpi_table_parse_srat(ACPI_SRAT_MEMORY_AFFINITY,
+					       acpi_parse_memory_affinity);
+	} else {
+		/* FIXME */
+		printk("Warning: acpi_table_parse(ACPI_SRAT) returned %d!\n",result);
+	}
+
+	/* SLIT: System Locality Information Table */
+	result = acpi_table_parse(ACPI_SLIT, acpi_parse_slit);
+	if (result < 1) {
+		/* FIXME */
+		printk("Warning: acpi_table_parse(ACPI_SLIT) returned %d!\n",result);
+	}
+
+	acpi_numa_arch_fixup();
+	return 0;
+}
--- linux-2.5.30-orig/include/linux/acpi.h	Thu Aug  1 14:16:01 2002
+++ linux-2.5.30-numa/include/linux/acpi.h	Wed Aug 14 16:05:05 2002
@@ -336,12 +336,21 @@
 char * __acpi_map_table (unsigned long phys_addr, unsigned long size);
 unsigned long acpi_find_rsdp (void);
 int acpi_boot_init (char *cmdline);
+int acpi_numa_init (void);
 
 int acpi_table_init (char *cmdline);
 int acpi_table_parse (enum acpi_table_id, acpi_table_handler);
-int acpi_table_parse_madt (enum acpi_table_id, acpi_madt_entry_handler);
+int acpi_table_parse_madt (enum acpi_madt_entry_id, acpi_madt_entry_handler);
+int acpi_table_parse_srat (enum acpi_srat_entry_id, acpi_madt_entry_handler);
 void acpi_table_print (struct acpi_table_header *, unsigned long);
 void acpi_table_print_madt_entry (acpi_table_entry_header *);
+void acpi_table_print_srat_entry (acpi_table_entry_header *);
+
+/* the following four functions are architecture-dependent */
+extern void __init acpi_numa_slit_init (struct acpi_table_slit *slit);
+extern void __init acpi_numa_processor_affinity_init (struct acpi_table_processor_affinity *pa);
+extern void __init acpi_numa_memory_affinity_init (struct acpi_table_memory_affinity *ma);
+extern void __init acpi_numa_arch_fixup(void);
 
 extern int acpi_mp_config;
 



Thanks,
-- 
KOCHI, Takayoshi <t-kouchi-f7IHDacdhdx8UrSeD/g0lQ@public.gmane.org/t-kouchi>



-------------------------------------------------------
This sf.net email is sponsored by: Dice - The leading online job board
for high-tech professionals. Search and apply for tech jobs today!
http://seeker.dice.com/seeker.epl?rel_code=31

Re: [Discontig-devel] RE: Cleanup of NUMA support in ACPI

[Erich Focht]

[-- Attachment #1: Type: text/plain, Size: 1709 bytes --]

Hi Tak,

looks good! Now the parts are finally splitted and there doesn't seem
to be any reason not to include the arch-independent part into the
mainline.

Here comes the rest, the ia64 arch-dependent part. It applies over the
latest Linus tree and should apply to David's 2.5.30 + Kimi's startup
fix, too.

Regards,
Erich


On Thursday 15 August 2002 02:21, KOCHI, Takayoshi wrote:
> Hi Andy,
>
> > > There are a bunch of other ACPI config options that should be split out
> > > like this as well, so if you don't want to redo the patch, I'll just
> > > take it and it can get fixed in a sweep with the others, later.
> >
> > Erich has worked to clean up some portions and sent to LKML.
> > I'll split ACPI part from the patch and send to you this afternoon.
>
> Here's the patch.
> This is generated against 2.5.30 kernel's driver.
> For 2.4.18 + 20020725, I saw faiures against Config.in and Makefile...
> perhaps it's easy to merge.
>
>
> For discontig/numa developers:
>
> If this is applied, both i386 and ia64 have to provide their own
> functions in architecture-dependent numa support (only when CONFIG_NUMA and
> CONFIG_ACPI_NUMA are both Y):
>
> void __init acpi_numa_slit_init (struct acpi_table_slit *slit);
> void __init acpi_numa_processor_affinity_init (struct
> acpi_table_processor_affinity *pa); void __init
> acpi_numa_memory_affinity_init (struct acpi_table_memory_affinity *ma);
> void __init acpi_numa_arch_fixup(void);
>
> Also, acpi_numa_init() has to be called in very early stage of
> initialization of architecture-dependent setup routine
> so that you can use the SRAT/SLIT information for constructing
> numa-related structures.
>

[-- Attachment #2: acpi-numa-2.5.31-ia64.patch --]
[-- Type: text/x-diff, Size: 12425 bytes --]

# This is a BitKeeper generated patch for the following project:
# Project Name: Linux kernel tree
# This patch format is intended for GNU patch command version 2.5 or higher.
# This patch includes the following deltas:
#	           ChangeSet	1.520   -> 1.521  
#	arch/ia64/kernel/setup.c	1.22    -> 1.23   
#	arch/ia64/mm/Makefile	1.1     -> 1.2    
#	include/asm-ia64/acpi.h	1.3     -> 1.4    
#	arch/ia64/kernel/acpi.c	1.16    -> 1.17   
#	               (new)	        -> 1.1     arch/ia64/mm/numa.c
#	               (new)	        -> 1.1     include/asm-ia64/numa.h
#
# The following is the BitKeeper ChangeSet Log
# --------------------------------------------
# 02/08/15	focht-3YFNcdZb8i1z/Fpu909jNA@public.gmane.org	1.521
#   Architecture dependent part (IA64) of ACPI SRAT (Static Resource
#   Affinity Table) and SLIT (System Locality Information Table)
#   parsing code. Tables are parsed if CONFIG_ACPI_NUMA=y.
#   
#   The ACPI SRAT and SLIT parsing routines are actually quite
#   arch-independent, but currently this setup is used only on IA64.
#   The NUMA related variables touched are:
#    node_memblk : physical memory blocks, node to which they belong
#    node_cpuid : hardware cpu IDs and nodes to which they belong
#    numa_slit : locality matrix with "distances" between nodes.
#   
#   This setup is needed for CONFIG_DISCONTIGMEM on IA64 but can be
#   used for other NUMA purposes, too.
# --------------------------------------------
#
diff -Nru a/arch/ia64/kernel/acpi.c b/arch/ia64/kernel/acpi.c
--- a/arch/ia64/kernel/acpi.c	Thu Aug 15 17:48:17 2002
+++ b/arch/ia64/kernel/acpi.c	Thu Aug 15 17:48:17 2002
@@ -8,6 +8,9 @@
  *  Copyright (C) 2000 Intel Corp.
  *  Copyright (C) 2000,2001 J.I. Lee <jung-ik.lee-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
  *  Copyright (C) 2001 Paul Diefenbaugh <paul.s.diefenbaugh-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
+ *  Copyright (C) 2001 Jenna Hall <jenna.s.hall-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
+ *  Copyright (C) 2001 Takayoshi Kochi <t-kouchi-f7IHDacdhdx8UrSeD/g0lQ@public.gmane.org>
+ *  Copyright (C) 2002 Erich Focht <efocht-+HQ0pkNQ8fyELgA04lAiVw@public.gmane.org>
  *
  * ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  *
@@ -43,6 +46,7 @@
 #include <asm/machvec.h>
 #include <asm/page.h>
 #include <asm/system.h>
+#include <asm/numa.h>
 
 
 #define PREFIX			"ACPI: "
@@ -440,6 +444,168 @@
 	return 0;
 }
 
+#ifdef CONFIG_ACPI_NUMA
+
+#define SRAT_DEBUG
+#define SLIT_DEBUG
+
+#define PXM_FLAG_LEN ((MAX_PXM_DOMAINS + 1)/32)
+
+static int __initdata srat_num_cpus = 0;		/* number of cpus */
+static u32 __initdata pxm_flag[PXM_FLAG_LEN] = { [0 ... PXM_FLAG_LEN-1] = 0};
+#define PXM_BIT_SET(bit)	(set_bit(bit,(void *)pxm_flag))
+#define PXM_BIT_CLEAR(bit)	(clear_bit(bit,(void *)pxm_flag))
+#define PXM_BIT_TEST(bit)	(test_bit(bit,(void *)pxm_flag))
+/* maps to convert between proximity domain and logical node ID */
+int pxm_to_nid_map[MAX_PXM_DOMAINS] = { [0 ... MAX_PXM_DOMAINS-1] = -1};
+int nid_to_pxm_map[NR_NODES] = { [0 ... NR_NODES-1] = -1};
+
+/*
+ * ACPI 2.0 SLIT (System Locality Information Table)
+ * http://devresource.hp.com/devresource/Docs/TechPapers/IA64/
+ */
+void __init
+acpi_numa_slit_init (struct acpi_table_slit *slit)
+{
+	int i, j, node_from, node_to;
+	u32 len;
+
+	len = sizeof(struct acpi_table_header) + 8 
+		+ slit->localities * slit->localities;
+	if (slit->header.length != len) {
+		printk("ACPI 2.0 SLIT: size mismatch: %d expected, %d actual\n",
+		      len, slit->header.length);
+		memset(numa_slit, 10, sizeof(numa_slit));
+		return;
+	}
+
+	memset(numa_slit, -1, sizeof(numa_slit));
+	for (i=0; i<slit->localities; i++) {
+		if (!PXM_BIT_TEST(i))
+			continue;
+		node_from = pxm_to_nid_map[i];
+		for (j=0; j<slit->localities; j++) {
+			if (!PXM_BIT_TEST(j))
+				continue;
+			node_to = pxm_to_nid_map[j];
+			node_distance(node_from, node_to) = 
+				slit->entry[i*slit->localities + j];
+		}
+	}
+
+#ifdef SLIT_DEBUG
+	printk("ACPI 2.0 SLIT locality table:\n");
+	for (i = 0; i < numnodes; i++) {
+		for (j = 0; j < numnodes; j++)
+			printk("%03d ", node_distance(i,j));
+		printk("\n");
+	}
+#endif
+}
+
+void __init
+acpi_numa_processor_affinity_init (struct acpi_table_processor_affinity *pa)
+{
+	/* record this node in proximity bitmap */
+	PXM_BIT_SET(pa->proximity_domain);
+
+	node_cpuid[srat_num_cpus].phys_id = (pa->apic_id << 8) | (pa->lsapic_eid);
+	/* nid should be overridden as logical node id later */
+	node_cpuid[srat_num_cpus].nid = pa->proximity_domain;
+	srat_num_cpus++;
+
+#ifdef SRAT_DEBUG
+	printk("CPU %x in proximity domain %x %s\n",
+	       pa->apic_id, pa->proximity_domain,
+	       pa->flags.enabled ? "enabled" : "disabled");
+#endif
+}
+
+void __init
+acpi_numa_memory_affinity_init (struct acpi_table_memory_affinity *ma)
+{
+	unsigned long paddr, size;
+	u8 pxm;
+	struct node_memblk_s *p, *q, *pend;
+
+	pxm = ma->proximity_domain;
+
+	/* record this node in proximity bitmap */
+	PXM_BIT_SET(pxm);
+
+	/* fill node memory chunk structure */
+	paddr = ma->base_addr_hi;
+	paddr = (paddr << 32) | ma->base_addr_lo;
+	size = ma->length_hi;
+	size = (size << 32) | ma->length_lo;
+
+	if (num_memblks >= NR_MEMBLKS) {
+		printk("Too many mem chunks in SRAT. Ignoring %ld MBytes at %lx\n",
+			size/(1024*1024), paddr);
+		return;
+	}
+
+	/* Insertion sort based on base address */
+	pend = &node_memblk[num_memblks];
+	for (p = &node_memblk[0]; p < pend; p++) {
+		if (paddr < p->start_paddr)
+			break;
+	}
+	if (p < pend) {
+		for (q = pend; q >= p; q--)
+			*(q + 1) = *q;
+	}
+	p->start_paddr = paddr;
+	p->size = size;
+	p->nid = pxm;
+	num_memblks++;
+
+#ifdef SRAT_DEBUG
+	printk("Memory range 0x%lx to 0x%lx (type %x) in proximity domain %x %s\n",
+	       paddr, paddr + size - 1,
+	       ma->memory_type, ma->proximity_domain,
+	       ma->flags.enabled ? (ma->flags.hot_pluggable ? 
+				    "enabled and removable" : "enabled" )
+	       : "disabled");
+#endif
+}
+
+void __init
+acpi_numa_arch_fixup(void)
+{
+	int i, j;
+
+	/* calculate total number of nodes in system from PXM bitmap */
+	numnodes = 0;		/* init total nodes in system */
+	for (i = 0; i < MAX_PXM_DOMAINS; i++) {
+		if (PXM_BIT_TEST(i)) {
+			pxm_to_nid_map[i] = numnodes;
+			nid_to_pxm_map[numnodes++] = i;
+		}
+	}
+
+	/* set logical node id in memory chunk structure */
+	for (i = 0; i < num_memblks; i++)
+		node_memblk[i].nid = pxm_to_nid_map[node_memblk[i].nid];
+
+	/* assign memory bank numbers for each chunk on each node */
+	for (i = 0; i < numnodes; i++) {
+		int bank;
+
+		bank = 0;
+		for (j = 0; j < num_memblks; j++)
+			if (node_memblk[j].nid == i)
+				node_memblk[j].bank = bank++;
+	}
+
+	/* set logical node id in cpu structure */
+	for (i = 0; i < srat_num_cpus; i++)
+		node_cpuid[i].nid = pxm_to_nid_map[node_cpuid[i].nid];
+
+	printk("Number of logical nodes in system = %d\n", numnodes);
+	printk("Number of memory chunks in system = %d\n", num_memblks);
+}
+#endif /* CONFIG_ACPI_NUMA */
 static int __init
 acpi_parse_fadt (unsigned long phys_addr, unsigned long size)
 {
@@ -535,12 +701,6 @@
 int __init
 acpi_boot_init (char *cmdline)
 {
-	int result;
-
-	/* Initialize the ACPI boot-time table parser */
-	result = acpi_table_init(cmdline);
-	if (result)
-		return result;
 
 	/*
 	 * MADT
diff -Nru a/arch/ia64/kernel/setup.c b/arch/ia64/kernel/setup.c
--- a/arch/ia64/kernel/setup.c	Thu Aug 15 17:48:17 2002
+++ b/arch/ia64/kernel/setup.c	Thu Aug 15 17:48:17 2002
@@ -297,6 +297,16 @@
 
 	efi_init();
 
+#ifdef CONFIG_ACPI_BOOT
+	/* Initialize the ACPI boot-time table parser */
+	acpi_table_init(*cmdline_p);
+
+#ifdef CONFIG_ACPI_NUMA
+	acpi_numa_init();
+#endif
+
+#endif /* CONFIG_APCI_BOOT */
+
 	find_memory();
 
 #if 0
diff -Nru a/arch/ia64/mm/Makefile b/arch/ia64/mm/Makefile
--- a/arch/ia64/mm/Makefile	Thu Aug 15 17:48:17 2002
+++ b/arch/ia64/mm/Makefile	Thu Aug 15 17:48:17 2002
@@ -10,5 +10,6 @@
 O_TARGET := mm.o
 
 obj-y	 := init.o fault.o tlb.o extable.o
+obj-$(CONFIG_NUMA) += numa.o
 
 include $(TOPDIR)/Rules.make
diff -Nru a/arch/ia64/mm/numa.c b/arch/ia64/mm/numa.c
--- /dev/null	Wed Dec 31 16:00:00 1969
+++ b/arch/ia64/mm/numa.c	Thu Aug 15 17:48:17 2002
@@ -0,0 +1,46 @@
+/*
+ * This file is subject to the terms and conditions of the GNU General Public
+ * License.  See the file "COPYING" in the main directory of this archive
+ * for more details.
+ *
+ * This file contains NUMA specific variables and functions which can
+ * be split away from DISCONTIGMEM and are used on NUMA machines with
+ * contiguous memory.
+ * 
+ *                         2002/08/07 Erich Focht <efocht-+HQ0pkNQ8fyELgA04lAiVw@public.gmane.org>
+ */
+
+#include <linux/config.h>
+#include <linux/kernel.h>
+#include <linux/mm.h>
+#include <linux/init.h>
+#include <linux/bootmem.h>
+#include <linux/mmzone.h>
+#include <asm/numa.h>
+
+/*
+ * The following structures are usually initialized by ACPI or
+ * similar mechanisms and describe the NUMA characteristics of the machine.
+ */
+int num_memblks = 0;
+struct node_memblk_s node_memblk[NR_MEMBLKS];
+struct node_cpuid_s node_cpuid[NR_CPUS];
+/*
+ * This is a matrix with "distances" between nodes, they should be
+ * proportional to the memory access latency ratios.
+ */
+u8 numa_slit[NR_NODES * NR_NODES];
+
+/* Identify which cnode a physical address resides on */
+int
+paddr_to_nid(unsigned long paddr)
+{
+	int	i;
+
+	for (i = 0; i < num_memblks; i++)
+		if (paddr >= node_memblk[i].start_paddr &&
+		    paddr < node_memblk[i].start_paddr + node_memblk[i].size)
+			break;
+
+	return (i < num_memblks) ? node_memblk[i].nid : -1;
+}
diff -Nru a/include/asm-ia64/acpi.h b/include/asm-ia64/acpi.h
--- a/include/asm-ia64/acpi.h	Thu Aug 15 17:48:17 2002
+++ b/include/asm-ia64/acpi.h	Thu Aug 15 17:48:17 2002
@@ -97,16 +97,16 @@
 	} while (0)
 
 const char *acpi_get_sysname (void);
-int acpi_boot_init (char *cdline);
 int acpi_request_vector (u32 int_type);
 int acpi_get_prt (struct pci_vector_struct **vectors, int *count);
 int acpi_get_interrupt_model(int *type);
 
-#ifdef CONFIG_DISCONTIGMEM
-#define NODE_ARRAY_INDEX(x)	((x) / 8)	/* 8 bits/char */
-#define NODE_ARRAY_OFFSET(x)	((x) % 8)	/* 8 bits/char */
-#define MAX_PXM_DOMAINS		(256)
-#endif /* CONFIG_DISCONTIGMEM */
+#ifdef CONFIG_ACPI_NUMA
+/* Proximity bitmap length; _PXM is at most 255 (8 bit)*/
+#define MAX_PXM_DOMAINS (256)
+extern int pxm_to_nid_map[MAX_PXM_DOMAINS];
+extern int nid_to_pxm_map[NR_NODES];
+#endif
 
 #endif /*__KERNEL__*/
 
diff -Nru a/include/asm-ia64/numa.h b/include/asm-ia64/numa.h
--- /dev/null	Wed Dec 31 16:00:00 1969
+++ b/include/asm-ia64/numa.h	Thu Aug 15 17:48:17 2002
@@ -0,0 +1,64 @@
+/*
+ * This file is subject to the terms and conditions of the GNU General Public
+ * License.  See the file "COPYING" in the main directory of this archive
+ * for more details.
+ *
+ * This file contains NUMA specific prototypes and definitions.
+ * 
+ * 2002/08/05 Erich Focht <efocht-+HQ0pkNQ8fyELgA04lAiVw@public.gmane.org>
+ *
+ */
+#ifndef _ASM_IA64_NUMA_H
+#define _ASM_IA64_NUMA_H
+
+#ifdef CONFIG_NUMA
+
+#ifdef CONFIG_DISCONTIGMEM
+# include <asm/mmzone.h>
+# define NR_NODES     (PLAT_MAX_COMPACT_NODES)
+# define NR_MEMBLKS   (PLAT_MAXCLUMPS)
+#else
+# define NR_NODES     (8)
+# define NR_MEMBLKS   (NR_NODES * 8)
+#endif
+
+/* Stuff below this line could be architecture independent */
+
+extern int num_memblks;		/* total number of memory chunks */
+
+/*
+ * List of node memory chunks. Filled when parsing SRAT table to
+ * obtain information about memory nodes.
+*/
+
+struct node_memblk_s {
+	unsigned long start_paddr;
+	unsigned long size;
+	int nid;		/* which logical node contains this chunk? */
+	int bank;		/* which mem bank on this node */
+};
+
+struct node_cpuid_s {
+	u16	phys_id;	/* id << 8 | eid */
+	int	nid;		/* logical node containing this CPU */
+};
+
+extern struct node_memblk_s node_memblk[NR_MEMBLKS];
+extern struct node_cpuid_s node_cpuid[NR_CPUS];
+
+/*
+ * ACPI 2.0 SLIT (System Locality Information Table)
+ * http://devresource.hp.com/devresource/Docs/TechPapers/IA64/
+ *
+ * This is a matrix with "distances" between nodes, they should be
+ * proportional to the memory access latency ratios.
+ */
+
+extern u8 numa_slit[NR_NODES * NR_NODES];
+#define node_distance(from,to) (numa_slit[from * numnodes + to])
+
+extern int paddr_to_nid(unsigned long paddr);
+
+#endif /* CONFIG_NUMA */
+
+#endif /* _ASM_IA64_NUMA_H */

Re: [Discontig-devel] RE: Cleanup of NUMA support in ACPI

[KOCHI, Takayoshi]

Erich,

The "ifdef SRAT_DEBUG"'s in arch/ia64/kernel/acpi.c arn't
necessary because arch-independent acpi_table_print_srat_entry()
function does this.

> +#ifdef SRAT_DEBUG
> +	printk("CPU %x in proximity domain %x %s\n",
> +	       pa->apic_id, pa->proximity_domain,
> +	       pa->flags.enabled ? "enabled" : "disabled");
> +#endif


> +#ifdef SRAT_DEBUG
> +	printk("Memory range 0x%lx to 0x%lx (type %x) in proximity domain %x %s\n",
> +	       paddr, paddr + size - 1,
> +	       ma->memory_type, ma->proximity_domain,
> +	       ma->flags.enabled ? (ma->flags.hot_pluggable ? 
> +				    "enabled and removable" : "enabled" )
> +	       : "disabled");
> +#endif

Thanks,
-- 
KOCHI, Takayoshi <t-kouchi-f7IHDacdhdx8UrSeD/g0lQ@public.gmane.org/t-kouchi>



-------------------------------------------------------
This sf.net email is sponsored by: OSDN - Tired of that same old
cell phone?  Get a new here for FREE!
https://www.inphonic.com/r.asp?r=sourceforge1&refcode1=vs3390