[PATCH] fix topology init take 2

[PATCH] fix topology init take 2

[Jesse Barnes]

Populate topology directories correctly now that NUMA kernels work.

Jesse

diff -Nru a/arch/ia64/mm/numa.c b/arch/ia64/mm/numa.c
--- a/arch/ia64/mm/numa.c	Fri Oct 17 11:24:49 2003
+++ b/arch/ia64/mm/numa.c	Fri Oct 17 11:24:49 2003
@@ -11,12 +11,19 @@
  */
 
 #include <linux/config.h>
+#include <linux/cpu.h>
 #include <linux/kernel.h>
+#include <linux/memblk.h>
 #include <linux/mm.h>
+#include <linux/node.h>
 #include <linux/init.h>
 #include <linux/bootmem.h>
 #include <asm/numa.h>
 
+static struct memblk *sysfs_memblks;
+static struct node *sysfs_nodes;
+static struct cpu *sysfs_cpus;
+
 /*
  * The following structures are usually initialized by ACPI or
  * similar mechanisms and describe the NUMA characteristics of the machine.
@@ -43,3 +50,49 @@
 
 	return (i < num_memblks) ? node_memblk[i].nid : (num_memblks ? -1 : 0);
 }
+
+static int __init topology_init(void)
+{
+	int i, err = 0;
+
+	sysfs_nodes = kmalloc(sizeof(struct node) * numnodes, GFP_KERNEL);
+	if (!sysfs_nodes) {
+		err = -ENOMEM;
+		goto out;
+	}
+
+	sysfs_memblks = kmalloc(sizeof(struct memblk) * num_memblks,
+				GFP_KERNEL);
+	if (!sysfs_memblks) {
+		kfree(sysfs_nodes);
+		err = -ENOMEM;
+		goto out;
+	}
+
+	sysfs_cpus = kmalloc(sizeof(struct cpu) * NR_CPUS, GFP_KERNEL);
+	if (!sysfs_cpus) {
+		kfree(sysfs_memblks);
+		kfree(sysfs_nodes);
+		err = -ENOMEM;
+		goto out;
+	}
+
+	for (i = 0; i < numnodes; i++)
+		if ((err = register_node(&sysfs_nodes[i], i, 0)))
+			goto out;
+	
+	for (i = 0; i < num_memblks; i++)
+		if ((err = register_memblk(&sysfs_memblks[i], i,
+					   &sysfs_nodes[memblk_to_node(i)])))
+			goto out;
+	
+	for (i = 0; i < NR_CPUS; i++)
+		if (cpu_online(i))
+			if((err = register_cpu(&sysfs_cpus[i], i,
+					       &sysfs_nodes[cpu_to_node(i)])))
+				goto out;
+ out:
+	return err;
+}
+
+__initcall(topology_init);

Re: [PATCH] fix topology init take 2

[Xavier Bru]

Jesse Barnes writes:
 > Populate topology directories correctly now that NUMA kernels work.
 > 
 > Jesse
 > 

Jesse,

Thanks for the patch. Just one question about it:

Are there relevant informations that can be used apart from:
    node*/meminfo
    node*/cpumap

We find some node*/memblk* and cpu/cpu* that are empty directories.

Should they sometimes provide some informations like LID (for cpu*) or 
physical block addr & length (for memblk) ? I suppose this is part of
the discussion on libnuma & hwgfs ?

-- 

 Sincères salutations.
_____________________________________________________________________
 
Xavier BRU                 BULL ISD/R&D/INTEL office:     FREC B1-422
tel : +33 (0)4 76 29 77 45                    http://www-frec.bull.fr
fax : +33 (0)4 76 29 77 70                 mailto:Xavier.Bru@bull.net
addr: BULL, 1 rue de Provence, BP 208, 38432 Echirolles Cedex, FRANCE
_____________________________________________________________________

Re: [PATCH] fix topology init take 2

[Jesse Barnes]

On Mon, Oct 20, 2003 at 05:17:29PM +0200, Xavier Bru wrote:
> 
> Jesse Barnes writes:
>  > Populate topology directories correctly now that NUMA kernels work.
>  > 
>  > Jesse
>  > 
> 
> Jesse,
> 
> Thanks for the patch. Just one question about it:
> 
> Are there relevant informations that can be used apart from:
>     node*/meminfo
>     node*/cpumap
> 
> We find some node*/memblk* and cpu/cpu* that are empty directories.
> 
> Should they sometimes provide some informations like LID (for cpu*) or 
> physical block addr & length (for memblk) ? I suppose this is part of
> the discussion on libnuma & hwgfs ?

Those seem like reasonable things to export in those directories, is
there anything else that would be nice to have?

Jesse

Re: [PATCH] fix topology init take 2

[Xavier Bru]

Jesse Barnes writes:
 > On Mon, Oct 20, 2003 at 05:17:29PM +0200, Xavier Bru wrote:
 > > 
 > > Jesse Barnes writes:
 > >  > Populate topology directories correctly now that NUMA kernels work.
 > >  > 
 > >  > Jesse
 > >  > 
 > > 
 > > Jesse,
 > > 
 > > Thanks for the patch. Just one question about it:
 > > 
 > > Are there relevant informations that can be used apart from:
 > >     node*/meminfo
 > >     node*/cpumap
 > > 
 > > We find some node*/memblk* and cpu/cpu* that are empty directories.
 > > 
 > > Should they sometimes provide some informations like LID (for cpu*) or 
 > > physical block addr & length (for memblk) ? I suppose this is part of
 > > the discussion on libnuma & hwgfs ?
 > 
 > Those seem like reasonable things to export in those directories, is
 > there anything else that would be nice to have?
 > 
 > Jesse
. as evoqued earlier some physical location information like a
position in the rack ( acpi ?).
. information on node status (available/not) for future  hot plug,
unless the presence of the node<x> directory provides the information.
. what about infos already present in /proc/cpuinfo, /proc/pal ? 

-- 

 Sincères salutations.
_____________________________________________________________________
 
Xavier BRU                 BULL ISD/R&D/INTEL office:     FREC B1-422
tel : +33 (0)4 76 29 77 45                    http://www-frec.bull.fr
fax : +33 (0)4 76 29 77 70                 mailto:Xavier.Bru@bull.net
addr: BULL, 1 rue de Provence, BP 208, 38432 Echirolles Cedex, FRANCE
_____________________________________________________________________

Re: [PATCH] fix topology init take 2

[Andreas Schwab]

jbarnes@sgi.com (Jesse Barnes) writes:

> @@ -43,3 +50,49 @@
>  
>  	return (i < num_memblks) ? node_memblk[i].nid : (num_memblks ? -1 : 0);
>  }
> +
> +static int __init topology_init(void)
> +{
> +	int i, err = 0;
> +
> +	sysfs_nodes = kmalloc(sizeof(struct node) * numnodes, GFP_KERNEL);
> +	if (!sysfs_nodes) {
> +		err = -ENOMEM;
> +		goto out;
> +	}
> +
> +	sysfs_memblks = kmalloc(sizeof(struct memblk) * num_memblks,
> +				GFP_KERNEL);
> +	if (!sysfs_memblks) {
> +		kfree(sysfs_nodes);
> +		err = -ENOMEM;
> +		goto out;
> +	}
> +
> +	sysfs_cpus = kmalloc(sizeof(struct cpu) * NR_CPUS, GFP_KERNEL);
> +	if (!sysfs_cpus) {
> +		kfree(sysfs_memblks);
> +		kfree(sysfs_nodes);
> +		err = -ENOMEM;
> +		goto out;
> +	}
> +
> +	for (i = 0; i < numnodes; i++)
> +		if ((err = register_node(&sysfs_nodes[i], i, 0)))
> +			goto out;
> +	
> +	for (i = 0; i < num_memblks; i++)
> +		if ((err = register_memblk(&sysfs_memblks[i], i,
> +					   &sysfs_nodes[memblk_to_node(i)])))
> +			goto out;
> +	
> +	for (i = 0; i < NR_CPUS; i++)
> +		if (cpu_online(i))
> +			if((err = register_cpu(&sysfs_cpus[i], i,
> +					       &sysfs_nodes[cpu_to_node(i)])))
> +				goto out;

You need to clear all kmalloc'd memory before you can pass it to
register_{node,memblk,cpu}, otherwise sys_device_register may trip over
the uninitialized sysdev.

--- linux-2.6.0-test8/arch/ia64/mm/numa.c.~1~	2003-10-20 21:25:23.000000000 +0200
+++ linux-2.6.0-test8/arch/ia64/mm/numa.c	2003-10-20 22:00:21.000000000 +0200
@@ -76,6 +76,9 @@ static int __init topology_init(void)
 		err = -ENOMEM;
 		goto out;
 	}
+	memset(sysfs_nodes, 0, sizeof(struct node) * numnodes);
+	memset(sysfs_memblks, 0, sizeof(struct memblk) * num_memblks);
+	memset(sysfs_cpus, 0, sizeof(struct cpu) * NR_CPUS);
 
 	for (i = 0; i < numnodes; i++)
 		if ((err = register_node(&sysfs_nodes[i], i, 0)))

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux AG, Deutschherrnstr. 15-19, D-90429 Nürnberg
Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."