+ mem-hotplug-avoid-multiple-zones-sharing-same-boot-strapping-boot_pageset-fix.patch added to -mm tree

+ mem-hotplug-avoid-multiple-zones-sharing-same-boot-strapping-boot_pageset-fix.patch added to -mm tree

[akpm]

The patch titled
     mem-hotplug-avoid-multiple-zones-sharing-same-boot-strapping-boot_pageset-fix
has been added to the -mm tree.  Its filename is
     mem-hotplug-avoid-multiple-zones-sharing-same-boot-strapping-boot_pageset-fix.patch

Before you just go and hit "reply", please:
   a) Consider who else should be cc'ed
   b) Prefer to cc a suitable mailing list as well
   c) Ideally: find the original patch on the mailing list and do a
      reply-to-all to that, adding suitable additional cc's

*** Remember to use Documentation/SubmitChecklist when testing your code ***

See http://userweb.kernel.org/~akpm/stuff/added-to-mm.txt to find
out what to do about this

The current -mm tree may be found at http://userweb.kernel.org/~akpm/mmotm/

------------------------------------------------------
Subject: mem-hotplug-avoid-multiple-zones-sharing-same-boot-strapping-boot_pageset-fix
From: Andrew Morton <akpm@linux-foundation.org>

Cc: Andi Kleen <andi.kleen@intel.com>
Cc: Christoph Lameter <cl@linux-foundation.org>
Cc: Haicheng Li <haicheng.li@linux.intel.com>
Cc: Mel Gorman <mel@csn.ul.ie>
Cc: Tejun Heo <tj@kernel.org>
Cc: Wu Fengguang <fengguang.wu@intel.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 kernel/cpu.c |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff -puN kernel/cpu.c~mem-hotplug-avoid-multiple-zones-sharing-same-boot-strapping-boot_pageset-fix kernel/cpu.c
--- a/kernel/cpu.c~mem-hotplug-avoid-multiple-zones-sharing-same-boot-strapping-boot_pageset-fix
+++ a/kernel/cpu.c
@@ -358,7 +358,7 @@ int __cpuinit cpu_up(unsigned int cpu)
 	}
 
 	if (pgdat->node_zonelists->_zonerefs->zone == NULL)
-		build_all_zonelists();
+		build_all_zonelists(NULL);
 #endif
 
 	cpu_maps_update_begin();
_

Patches currently in -mm which might be from akpm@linux-foundation.org are

origin.patch
linux-next.patch
next-remove-localversion.patch
i-need-old-gcc.patch
include-linux-fsh-complete-hexification-of-fmode_-constants.patch
drivers-pci-intel-iommuc-errors-with-smaller-iommu-widths-fix.patch
x86-platform-driver-intelligent-power-sharing-driver-checkpatch-fixes.patch
compal-laptop-added-jhl90-battery-hwmon-interface.patch
dib3000mc-reduce-large-stack-usage-fix.patch
fs-fscache-object-listc-fix-warning-on-32-bit.patch
of-gpio-implement-gpiolib-notifier-hooks.patch
hpet-factor-timer-allocate-from-open.patch
led-driver-for-the-soekris-net5501-board.patch
leds-route-kbd-leds-through-the-generic-leds-layer.patch
3x59x-fix-pci-resource-management.patch
backlight-add-s6e63m0-amoled-lcd-panel-driver-checkpatch-fixes.patch
fix-stop_machine-reimplement-using-cpu_stop.patch
scsi-remove-private-bit-macros.patch
vfs-use-kmalloc-to-allocate-fdmem-if-possible-fix.patch
vfs-use-kmalloc-to-allocate-fdmem-if-possible-fix-2.patch
cpusetmm-fix-no-node-to-alloc-memory-when-changing-cpusets-mems-spelling.patch
mm-allow-config_migration-to-be-set-without-config_numa-or-memory-hot-remove.patch
mm-export-unusable-free-space-index-via-debugfs.patch
mmcompaction-direct-compact-when-a-high-order-allocation-fails.patch
buffer_head-remove-redundant-test-from-wait_on_buffer-fix.patch
wait_on_buffer-remove-the-buffer_locked-test.patch
mm-introduce-free_pages_prepare-fix.patch
cpu-mem-hotplug-enable-cpus-online-before-local-memory-online-fix.patch
cpu-mem-hotplug-enable-cpus-online-before-local-memory-online-checkpatch-fixes.patch
highmem-remove-unneeded-ifdef-config_trace_irqflags_support-for-debug_kmap_atomic.patch
mem-hotplug-avoid-multiple-zones-sharing-same-boot-strapping-boot_pageset-fix.patch
frv-extend-gdbstub-to-support-more-features-of-gdb-fix.patch
frv-duplicate-output_buffer-of-e03-checkpatch-fixes.patch
kernel-wide-replace-ushort_max-short_max-and-short_min-with-ushrt_max-shrt_max-and-shrt_min-fix.patch
kernel-wide-replace-ushort_max-short_max-and-short_min-with-ushrt_max-shrt_max-and-shrt_min-fix-fix.patch
include-asm-generic-kmap_typesh-add-helpful-reminder.patch
dynamic_debug-fix-documentation-errors-checkpatch-fixes.patch
lib-introduce-common-method-to-convert-hex-digits-fix.patch
lib-introduce-common-method-to-convert-hex-digits-fix-fix.patch
davinci-mmc-updates-to-suspend-resume-implementation-checkpatch-fixes.patch
checkpatch-add-check-for-too-short-kconfig-descriptions-checkpatch-fixes.patch
crc32-major-optimization-checkpatch-fixes.patch
hwmon-add-ti-ads7871-a-d-converter-driver-checkpatch-fixes.patch
rtc-mxc-remove-unnecessary-clock-source-for-rtc-subsystem.patch
gpiolib-make-names-array-and-its-values-const-fix.patch
s3c-fb-add-s5pv210-support-and-some-features-checkpatch-fixes.patch
memcg-oom-kill-disable-and-oom-status-update-checkpatch-fixes.patch
memcg-move-charge-of-file-pages-fix.patch
memcg-move-charge-of-file-pages-fix-4.patch
cpusets-randomize-node-rotor-used-in-cpuset_mem_spread_node-fix.patch
proc-turn-signal_struct-count-into-int-nr_threads-checkpatch-fixes.patch
fault-injection-add-cpu-notifier-error-injection-module-fix.patch
ipc-semc-optimize-update_queue-for-bulk-wakeup-calls-fix.patch
ipc-semc-move-wake_up_process-out-of-the-spinlock-section-fix.patch
ipc-semc-move-wake_up_process-out-of-the-spinlock-section-fix-2.patch
ipmi-update-driver-to-use-dev_printk-and-its-constructs-checkpatch-fixes.patch
pids-increase-pid_max-based-on-num_possible_cpus-fix.patch
delay-accounting-re-implement-c-for-getdelaysc-to-report-information-on-a-target-command-checkpatch-fixes.patch
drivers-edac-convert-logging-messages-direct-uses-of-__file__-to-%s-__file-checkpatch-fixes.patch
panic-allow-taint-flag-for-warnings-to-be-changed-from-taint_warn-checkpatch-fixes.patch
aio-fix-the-compat-vectored-operations-checkpatch-fixes.patch
aio-fix-the-compat-vectored-operations-fix.patch
kfifo-add-example-files-to-the-kernel-sample-directory-checkpatch-fixes.patch
kfifo-samples-dont-compile.patch
x86_32-use-asm-generic-scatterlisth.patch
numa-introduce-numa_mem_id-effective-local-memory-node-id-fix.patch
reiser4-export-remove_from_page_cache-fix.patch
reiser4.patch
reiser4-writeback_inodes-implementation-fix.patch
reiser4-fixups.patch
reiser4-broke.patch
slab-leaks3-default-y.patch
put_bh-debug.patch
getblk-handle-2tb-devices.patch
getblk-handle-2tb-devices-fix.patch

[PATCH] cpu_up: hold zonelists_mutex when build_all_zonelists

[Haicheng Li]

akpm@linux-foundation.org wrote:
 > The patch titled
 >      mem-hotplug-avoid-multiple-zones-sharing-same-boot-strapping-boot_pageset-fix
 > has been added to the -mm tree.  Its filename is
 >      mem-hotplug-avoid-multiple-zones-sharing-same-boot-strapping-boot_pageset-fix.patch
 > ------------------------------------------------------
> Subject: mem-hotplug-avoid-multiple-zones-sharing-same-boot-strapping-boot_pageset-fix
> From: Andrew Morton <akpm@linux-foundation.org>
> 
> Cc: Andi Kleen <andi.kleen@intel.com>
> Cc: Christoph Lameter <cl@linux-foundation.org>
> Cc: Haicheng Li <haicheng.li@linux.intel.com>
> Cc: Mel Gorman <mel@csn.ul.ie>
> Cc: Tejun Heo <tj@kernel.org>
> Cc: Wu Fengguang <fengguang.wu@intel.com>
> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
> ---
> 
>  kernel/cpu.c |    2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff -puN kernel/cpu.c~mem-hotplug-avoid-multiple-zones-sharing-same-boot-strapping-boot_pageset-fix kernel/cpu.c
> --- a/kernel/cpu.c~mem-hotplug-avoid-multiple-zones-sharing-same-boot-strapping-boot_pageset-fix
> +++ a/kernel/cpu.c
> @@ -358,7 +358,7 @@ int __cpuinit cpu_up(unsigned int cpu)
>  	}
>  
>  	if (pgdat->node_zonelists->_zonerefs->zone == NULL)
> -		build_all_zonelists();
> +		build_all_zonelists(NULL);
>  #endif
>  
>  	cpu_maps_update_begin();

Andrew,

Here is another issue, we should always hold zonelists_mutex when calling build_all_zonelists
unless system_state == SYSTEM_BOOTING.

We need another patch to fix it, which should be applied after 
mem-hotplug-fix-potential-race-while-building-zonelist-for-new-populated-zone.patch

---
 From 5f547a85e3b331f7ef2c004c93b674f9698c5531 Mon Sep 17 00:00:00 2001
From: Haicheng Li <haicheng.li@linux.intel.com>
Date: Thu, 20 May 2010 11:17:01 +0800
Subject: [PATCH] cpu_up: hold zonelists_mutex when build_all_zonelists

Signed-off-by: Haicheng Li <haicheng.li@linux.intel.com>
---
  kernel/cpu.c |    5 ++++-
  1 files changed, 4 insertions(+), 1 deletions(-)

diff --git a/kernel/cpu.c b/kernel/cpu.c
index 3e8b3ba..124ad9d 100644
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -357,8 +357,11 @@ int __cpuinit cpu_up(unsigned int cpu)
                 return -ENOMEM;
         }

-       if (pgdat->node_zonelists->_zonerefs->zone == NULL)
+       if (pgdat->node_zonelists->_zonerefs->zone == NULL) {
+               mutex_lock(&zonelists_mutex);
                 build_all_zonelists(NULL);
+               mutex_unlock(&zonelists_mutex);
+       }
  #endif

         cpu_maps_update_begin();
-- 
1.5.6.1


-haicheng

[PATCH] cpu_up: hold zonelists_mutex when build_all_zonelists

[Haicheng Li]

akpm@linux-foundation.org wrote:
 > The patch titled
 >      mem-hotplug-avoid-multiple-zones-sharing-same-boot-strapping-boot_pageset-fix
 > has been added to the -mm tree.  Its filename is
 >      mem-hotplug-avoid-multiple-zones-sharing-same-boot-strapping-boot_pageset-fix.patch
 > ------------------------------------------------------
> Subject: mem-hotplug-avoid-multiple-zones-sharing-same-boot-strapping-boot_pageset-fix
> From: Andrew Morton <akpm@linux-foundation.org>
> 
> Cc: Andi Kleen <andi.kleen@intel.com>
> Cc: Christoph Lameter <cl@linux-foundation.org>
> Cc: Haicheng Li <haicheng.li@linux.intel.com>
> Cc: Mel Gorman <mel@csn.ul.ie>
> Cc: Tejun Heo <tj@kernel.org>
> Cc: Wu Fengguang <fengguang.wu@intel.com>
> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
> ---
> 
>  kernel/cpu.c |    2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff -puN kernel/cpu.c~mem-hotplug-avoid-multiple-zones-sharing-same-boot-strapping-boot_pageset-fix kernel/cpu.c
> --- a/kernel/cpu.c~mem-hotplug-avoid-multiple-zones-sharing-same-boot-strapping-boot_pageset-fix
> +++ a/kernel/cpu.c
> @@ -358,7 +358,7 @@ int __cpuinit cpu_up(unsigned int cpu)
>  	}
>  
>  	if (pgdat->node_zonelists->_zonerefs->zone == NULL)
> -		build_all_zonelists();
> +		build_all_zonelists(NULL);
>  #endif
>  
>  	cpu_maps_update_begin();

Andrew,

Here is another issue, we should always hold zonelists_mutex when calling build_all_zonelists
unless system_state == SYSTEM_BOOTING.

We need another patch to fix it, which should be applied after 
mem-hotplug-fix-potential-race-while-building-zonelist-for-new-populated-zone.patch

---
 From 5f547a85e3b331f7ef2c004c93b674f9698c5531 Mon Sep 17 00:00:00 2001
From: Haicheng Li <haicheng.li@linux.intel.com>
Date: Thu, 20 May 2010 11:17:01 +0800
Subject: [PATCH] cpu_up: hold zonelists_mutex when build_all_zonelists

Signed-off-by: Haicheng Li <haicheng.li@linux.intel.com>
---
  kernel/cpu.c |    5 ++++-
  1 files changed, 4 insertions(+), 1 deletions(-)

diff --git a/kernel/cpu.c b/kernel/cpu.c
index 3e8b3ba..124ad9d 100644
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -357,8 +357,11 @@ int __cpuinit cpu_up(unsigned int cpu)
                 return -ENOMEM;
         }

-       if (pgdat->node_zonelists->_zonerefs->zone == NULL)
+       if (pgdat->node_zonelists->_zonerefs->zone == NULL) {
+               mutex_lock(&zonelists_mutex);
                 build_all_zonelists(NULL);
+               mutex_unlock(&zonelists_mutex);
+       }
  #endif

         cpu_maps_update_begin();
-- 
1.5.6.1


-haicheng

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH] cpu_up: hold zonelists_mutex when build_all_zonelists

[Andrew Morton]

On Thu, 20 May 2010 11:23:16 +0800
Haicheng Li <haicheng.li@linux.intel.com> wrote:

> 
> Here is another issue, we should always hold zonelists_mutex when calling build_all_zonelists
> unless system_state == SYSTEM_BOOTING.

Taking a global mutex in the cpu-hotplug code is worrisome.  Perhaps
because of the two years spent weeding out strange deadlocks between
cpu-hotplug and cpufreq.

Has this change been carefully and fully tested with lockdep enabled
(please)?

> --- a/kernel/cpu.c
> +++ b/kernel/cpu.c
> @@ -357,8 +357,11 @@ int __cpuinit cpu_up(unsigned int cpu)
>                  return -ENOMEM;
>          }
> 
> -       if (pgdat->node_zonelists->_zonerefs->zone == NULL)
> +       if (pgdat->node_zonelists->_zonerefs->zone == NULL) {
> +               mutex_lock(&zonelists_mutex);
>                  build_all_zonelists(NULL);
> +               mutex_unlock(&zonelists_mutex);
> +       }

Your email client is performing space-stuffing and it replaces tabs
with spaces.  This requires me to edit the patches rather a lot,
which is dull.

Re: [PATCH] cpu_up: hold zonelists_mutex when build_all_zonelists

[Andrew Morton]

On Thu, 20 May 2010 11:23:16 +0800
Haicheng Li <haicheng.li@linux.intel.com> wrote:

> 
> Here is another issue, we should always hold zonelists_mutex when calling build_all_zonelists
> unless system_state == SYSTEM_BOOTING.

Taking a global mutex in the cpu-hotplug code is worrisome.  Perhaps
because of the two years spent weeding out strange deadlocks between
cpu-hotplug and cpufreq.

Has this change been carefully and fully tested with lockdep enabled
(please)?

> --- a/kernel/cpu.c
> +++ b/kernel/cpu.c
> @@ -357,8 +357,11 @@ int __cpuinit cpu_up(unsigned int cpu)
>                  return -ENOMEM;
>          }
> 
> -       if (pgdat->node_zonelists->_zonerefs->zone == NULL)
> +       if (pgdat->node_zonelists->_zonerefs->zone == NULL) {
> +               mutex_lock(&zonelists_mutex);
>                  build_all_zonelists(NULL);
> +               mutex_unlock(&zonelists_mutex);
> +       }

Your email client is performing space-stuffing and it replaces tabs
with spaces.  This requires me to edit the patches rather a lot,
which is dull.


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH] cpu_up: hold zonelists_mutex when build_all_zonelists

[Haicheng Li]

Andrew Morton wrote:
> On Thu, 20 May 2010 11:23:16 +0800
> Haicheng Li <haicheng.li@linux.intel.com> wrote:
> 
>> Here is another issue, we should always hold zonelists_mutex when calling build_all_zonelists
>> unless system_state == SYSTEM_BOOTING.
> 
> Taking a global mutex in the cpu-hotplug code is worrisome.  Perhaps
> because of the two years spent weeding out strange deadlocks between
> cpu-hotplug and cpufreq.
> 
> Has this change been carefully and fully tested with lockdep enabled
> (please)?
Yes, Andrew. I've tested it with lockdep enabled, and there was *no*
issue found for this change in my testing.

My test box: CPUs on node 1~3 are all offlined (16 cpus per node).
Here are my test steps:
on tty0:
# cd /sys/devices/system/node/node1
# for i in cpu*; do echo 1 > $i/online; done

on tty1:
# cd /sys/devices/system/node/node2
# for i in cpu*; do echo 1 > $i/online; done

on tty2:
# cd /sys/devices/system/node/node3
# for i in cpu*; do echo 1 > $i/online; done

on tty3:
# cat zonelist

	#! /bin/bash
	set -x
	while ((1)); do
		echo n > /proc/sys/vm/numa_zonelist_order
		sleep 10
		echo z > /proc/sys/vm/numa_zonelist_order
		sleep 10
	done

# ./zonelist

Besides, I also ran some cpu online/offline tests from LTP/cpu_hotplug test suites.
They worked fine too.

>> --- a/kernel/cpu.c
>> +++ b/kernel/cpu.c
>> @@ -357,8 +357,11 @@ int __cpuinit cpu_up(unsigned int cpu)
>>                  return -ENOMEM;
>>          }
>>
>> -       if (pgdat->node_zonelists->_zonerefs->zone == NULL)
>> +       if (pgdat->node_zonelists->_zonerefs->zone == NULL) {
>> +               mutex_lock(&zonelists_mutex);
>>                  build_all_zonelists(NULL);
>> +               mutex_unlock(&zonelists_mutex);
>> +       }
> 
> Your email client is performing space-stuffing and it replaces tabs
> with spaces.  This requires me to edit the patches rather a lot,
> which is dull.

Really sorry for the inconvenience to you. I'll pay more attention
next time. thank you!

-haicheng

Re: [PATCH] cpu_up: hold zonelists_mutex when build_all_zonelists

[Haicheng Li]

Andrew Morton wrote:
> On Thu, 20 May 2010 11:23:16 +0800
> Haicheng Li <haicheng.li@linux.intel.com> wrote:
> 
>> Here is another issue, we should always hold zonelists_mutex when calling build_all_zonelists
>> unless system_state == SYSTEM_BOOTING.
> 
> Taking a global mutex in the cpu-hotplug code is worrisome.  Perhaps
> because of the two years spent weeding out strange deadlocks between
> cpu-hotplug and cpufreq.
> 
> Has this change been carefully and fully tested with lockdep enabled
> (please)?
Yes, Andrew. I've tested it with lockdep enabled, and there was *no*
issue found for this change in my testing.

My test box: CPUs on node 1~3 are all offlined (16 cpus per node).
Here are my test steps:
on tty0:
# cd /sys/devices/system/node/node1
# for i in cpu*; do echo 1 > $i/online; done

on tty1:
# cd /sys/devices/system/node/node2
# for i in cpu*; do echo 1 > $i/online; done

on tty2:
# cd /sys/devices/system/node/node3
# for i in cpu*; do echo 1 > $i/online; done

on tty3:
# cat zonelist

	#! /bin/bash
	set -x
	while ((1)); do
		echo n > /proc/sys/vm/numa_zonelist_order
		sleep 10
		echo z > /proc/sys/vm/numa_zonelist_order
		sleep 10
	done

# ./zonelist

Besides, I also ran some cpu online/offline tests from LTP/cpu_hotplug test suites.
They worked fine too.

>> --- a/kernel/cpu.c
>> +++ b/kernel/cpu.c
>> @@ -357,8 +357,11 @@ int __cpuinit cpu_up(unsigned int cpu)
>>                  return -ENOMEM;
>>          }
>>
>> -       if (pgdat->node_zonelists->_zonerefs->zone == NULL)
>> +       if (pgdat->node_zonelists->_zonerefs->zone == NULL) {
>> +               mutex_lock(&zonelists_mutex);
>>                  build_all_zonelists(NULL);
>> +               mutex_unlock(&zonelists_mutex);
>> +       }
> 
> Your email client is performing space-stuffing and it replaces tabs
> with spaces.  This requires me to edit the patches rather a lot,
> which is dull.

Really sorry for the inconvenience to you. I'll pay more attention
next time. thank you!

-haicheng

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>