Re: [patch 00/28] cpu alloc v1: Optimize by removing arrays of pointers to per cpu objects

[Martin Schwidefsky <schwidefsky@de.ibm.com>]

On Tue, 2007-11-06 at 11:51 -0800, Christoph Lameter wrote:
> TODO:
> - Currently only i386, ia64 and x86_64 arch definitions are provided.
>   Other arches fall back to 64k static configurations.
> - Cpu hotplug support. Current we simply allocate for all possible processors.
>   We could reduce this to only online processors if we could allocate the
>   cpu area for the new processor before the callbacks are run and if we could
>   free the cpu areas for a processor going down after all the callbacks for
>   that were run.
> - There are various modifications to exotic configurations that still need
>   some testing (f.e. s/390 iucv--whatever that is--) etc. Tests were
>   done on UP(i386) SMP(i386, x86_64) and NUMA (x86_64, ia64)

IUCV = Inter-User-Communication-Vehicle. Nice name, isn't it?

It is a z/VM hypervisor interface that allows the different guests to
communicate between each other. net/iucv.c is the base code,
net/af_iucv.c implements the socket interface. 

The patch you provided for iucv has a few bugs which are corrected with
the patch below (please merge with patch #25). With it new cpu alloc
code works fine on s390. We likely want to switch to a virtual
configuration as well. For now we can live with the static
configuration.

-- 
blue skies,
  Martin.

"Reality continues to ruin my life." - Calvin.

---

 net/iucv/iucv.c |   18 ++++++++++--------
 1 file changed, 10 insertions(+), 8 deletions(-)

diff --git a/net/iucv/iucv.c b/net/iucv/iucv.c
index 0d77dba..7698f6c 100644
--- a/net/iucv/iucv.c
+++ b/net/iucv/iucv.c
@@ -566,7 +566,6 @@ static int __cpuinit iucv_cpu_notify(struct notifier_block *self,
 				     GFP_KERNEL|GFP_DMA, cpu_to_node(cpu));
 		if (!iucv_param[cpu])
 			return NOTIFY_BAD;
-		}
 		break;
 	case CPU_UP_CANCELED:
 	case CPU_UP_CANCELED_FROZEN:
@@ -1622,19 +1621,21 @@ static int __init iucv_init(void)
 	}
 
 	for_each_online_cpu(cpu) {
-			/* Note: GFP_DMA used to get memory below 2G */
+		/* Note: GFP_DMA used to get memory below 2G */
 		iucv_irq_data[cpu] = kmalloc_node(sizeof(struct iucv_irq_data),
 				     GFP_KERNEL|GFP_DMA, cpu_to_node(cpu));
 		if (!iucv_irq_data[cpu]) {
 			rc = -ENOMEM;
-			goto out_root;
+			goto out_free;
+		}
 
 		/* Allocate parameter blocks. */
 		iucv_param[cpu] = kmalloc_node(sizeof(union iucv_param),
 				  GFP_KERNEL|GFP_DMA, cpu_to_node(cpu));
 		if (!iucv_param[cpu]) {
 			rc = -ENOMEM;
-			goto out_extint;
+			goto out_free;
+		}
 	}
 	register_hotcpu_notifier(&iucv_cpu_notifier);
 	ASCEBC(iucv_error_no_listener, 16);
@@ -1643,12 +1644,13 @@ static int __init iucv_init(void)
 	iucv_available = 1;
 	return 0;
 
-out_extint:
-	for_each_cpu(cpu) {
+out_free:
+	for_each_possible_cpu(cpu) {
+		kfree(iucv_param[cpu]);
+		iucv_param[cpu] = NULL;
 		kfree(iucv_irq_data[cpu]);
 		iucv_irq_data[cpu] = NULL;
 	}
-out_root:
 	s390_root_dev_unregister(iucv_root);
 out_bus:
 	bus_unregister(&iucv_bus);
@@ -1675,7 +1677,7 @@ static void __exit iucv_exit(void)
 		kfree(p);
 	spin_unlock_irq(&iucv_queue_lock);
 	unregister_hotcpu_notifier(&iucv_cpu_notifier);
-	for_each_cpu(cpu) {
+	for_each_possible_cpu(cpu) {
 		kfree(iucv_param[cpu]);
 		iucv_param[cpu] = NULL;
 		kfree(iucv_irq_data[cpu]);