[PATCH] cpu_up: hold zonelists_mutex when build_all_zonelists

[PATCH] cpu_up: hold zonelists_mutex when build_all_zonelists

[Haicheng Li]

akpm@linux-foundation.org wrote:
 > The patch titled
 >      mem-hotplug-avoid-multiple-zones-sharing-same-boot-strapping-boot_pageset-fix
 > has been added to the -mm tree.  Its filename is
 >      mem-hotplug-avoid-multiple-zones-sharing-same-boot-strapping-boot_pageset-fix.patch
 > ------------------------------------------------------
> Subject: mem-hotplug-avoid-multiple-zones-sharing-same-boot-strapping-boot_pageset-fix
> From: Andrew Morton <akpm@linux-foundation.org>
> 
> Cc: Andi Kleen <andi.kleen@intel.com>
> Cc: Christoph Lameter <cl@linux-foundation.org>
> Cc: Haicheng Li <haicheng.li@linux.intel.com>
> Cc: Mel Gorman <mel@csn.ul.ie>
> Cc: Tejun Heo <tj@kernel.org>
> Cc: Wu Fengguang <fengguang.wu@intel.com>
> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
> ---
> 
>  kernel/cpu.c |    2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff -puN kernel/cpu.c~mem-hotplug-avoid-multiple-zones-sharing-same-boot-strapping-boot_pageset-fix kernel/cpu.c
> --- a/kernel/cpu.c~mem-hotplug-avoid-multiple-zones-sharing-same-boot-strapping-boot_pageset-fix
> +++ a/kernel/cpu.c
> @@ -358,7 +358,7 @@ int __cpuinit cpu_up(unsigned int cpu)
>  	}
>  
>  	if (pgdat->node_zonelists->_zonerefs->zone == NULL)
> -		build_all_zonelists();
> +		build_all_zonelists(NULL);
>  #endif
>  
>  	cpu_maps_update_begin();

Andrew,

Here is another issue, we should always hold zonelists_mutex when calling build_all_zonelists
unless system_state == SYSTEM_BOOTING.

We need another patch to fix it, which should be applied after 
mem-hotplug-fix-potential-race-while-building-zonelist-for-new-populated-zone.patch

---
 From 5f547a85e3b331f7ef2c004c93b674f9698c5531 Mon Sep 17 00:00:00 2001
From: Haicheng Li <haicheng.li@linux.intel.com>
Date: Thu, 20 May 2010 11:17:01 +0800
Subject: [PATCH] cpu_up: hold zonelists_mutex when build_all_zonelists

Signed-off-by: Haicheng Li <haicheng.li@linux.intel.com>
---
  kernel/cpu.c |    5 ++++-
  1 files changed, 4 insertions(+), 1 deletions(-)

diff --git a/kernel/cpu.c b/kernel/cpu.c
index 3e8b3ba..124ad9d 100644
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -357,8 +357,11 @@ int __cpuinit cpu_up(unsigned int cpu)
                 return -ENOMEM;
         }

-       if (pgdat->node_zonelists->_zonerefs->zone == NULL)
+       if (pgdat->node_zonelists->_zonerefs->zone == NULL) {
+               mutex_lock(&zonelists_mutex);
                 build_all_zonelists(NULL);
+               mutex_unlock(&zonelists_mutex);
+       }
  #endif

         cpu_maps_update_begin();
-- 
1.5.6.1


-haicheng

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH] cpu_up: hold zonelists_mutex when build_all_zonelists

[Andrew Morton]

On Thu, 20 May 2010 11:23:16 +0800
Haicheng Li <haicheng.li@linux.intel.com> wrote:

> 
> Here is another issue, we should always hold zonelists_mutex when calling build_all_zonelists
> unless system_state == SYSTEM_BOOTING.

Taking a global mutex in the cpu-hotplug code is worrisome.  Perhaps
because of the two years spent weeding out strange deadlocks between
cpu-hotplug and cpufreq.

Has this change been carefully and fully tested with lockdep enabled
(please)?

> --- a/kernel/cpu.c
> +++ b/kernel/cpu.c
> @@ -357,8 +357,11 @@ int __cpuinit cpu_up(unsigned int cpu)
>                  return -ENOMEM;
>          }
> 
> -       if (pgdat->node_zonelists->_zonerefs->zone == NULL)
> +       if (pgdat->node_zonelists->_zonerefs->zone == NULL) {
> +               mutex_lock(&zonelists_mutex);
>                  build_all_zonelists(NULL);
> +               mutex_unlock(&zonelists_mutex);
> +       }

Your email client is performing space-stuffing and it replaces tabs
with spaces.  This requires me to edit the patches rather a lot,
which is dull.


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH] cpu_up: hold zonelists_mutex when build_all_zonelists

[Haicheng Li]

Andrew Morton wrote:
> On Thu, 20 May 2010 11:23:16 +0800
> Haicheng Li <haicheng.li@linux.intel.com> wrote:
> 
>> Here is another issue, we should always hold zonelists_mutex when calling build_all_zonelists
>> unless system_state == SYSTEM_BOOTING.
> 
> Taking a global mutex in the cpu-hotplug code is worrisome.  Perhaps
> because of the two years spent weeding out strange deadlocks between
> cpu-hotplug and cpufreq.
> 
> Has this change been carefully and fully tested with lockdep enabled
> (please)?
Yes, Andrew. I've tested it with lockdep enabled, and there was *no*
issue found for this change in my testing.

My test box: CPUs on node 1~3 are all offlined (16 cpus per node).
Here are my test steps:
on tty0:
# cd /sys/devices/system/node/node1
# for i in cpu*; do echo 1 > $i/online; done

on tty1:
# cd /sys/devices/system/node/node2
# for i in cpu*; do echo 1 > $i/online; done

on tty2:
# cd /sys/devices/system/node/node3
# for i in cpu*; do echo 1 > $i/online; done

on tty3:
# cat zonelist

	#! /bin/bash
	set -x
	while ((1)); do
		echo n > /proc/sys/vm/numa_zonelist_order
		sleep 10
		echo z > /proc/sys/vm/numa_zonelist_order
		sleep 10
	done

# ./zonelist

Besides, I also ran some cpu online/offline tests from LTP/cpu_hotplug test suites.
They worked fine too.

>> --- a/kernel/cpu.c
>> +++ b/kernel/cpu.c
>> @@ -357,8 +357,11 @@ int __cpuinit cpu_up(unsigned int cpu)
>>                  return -ENOMEM;
>>          }
>>
>> -       if (pgdat->node_zonelists->_zonerefs->zone == NULL)
>> +       if (pgdat->node_zonelists->_zonerefs->zone == NULL) {
>> +               mutex_lock(&zonelists_mutex);
>>                  build_all_zonelists(NULL);
>> +               mutex_unlock(&zonelists_mutex);
>> +       }
> 
> Your email client is performing space-stuffing and it replaces tabs
> with spaces.  This requires me to edit the patches rather a lot,
> which is dull.

Really sorry for the inconvenience to you. I'll pay more attention
next time. thank you!

-haicheng

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>