[ RESEND PATCH v3] Memory-Hotplug: Fix the bug on interface /dev/mem for 64-bit kernel

[ RESEND PATCH v3] Memory-Hotplug: Fix the bug on interface /dev/mem for 64-bit kernel

[Zheng, Shaohui]

[-- Attachment #1: Type: text/plain, Size: 2845 bytes --]

Resend the v3 patch after reviewed by KAMEZAWA Hiroyuki. We still keep the 
Old e820map, update variable max_pfn, max_low_pfn and high_memory only. 
It is dependent on Fenguang's page_is_ram patch.

Memory-Hotplug: Fix the bug on interface /dev/mem for 64-bit kernel

The new added memory can not be access by interface /dev/mem, because we do not
 update the variable high_memory, max_pfn and max_low_pfn.

Memory hotplug still has critical issues for 32-bit kernel, and it is more 
important for 64-bit kernel, we fix it on 64-bit first. We add a function 
update_end_of_memory_vars in file arch/x86/mm/init.c to update these variables.

CC: Andi Kleen <ak@linux.intel.com>
CC: Li Haicheng <haicheng.li@intel.com>
Reviewed-by: Wu Fengguang <fengguang.wu@intel.com>
Reviewed-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
Signed-off-by: Shaohui Zheng <shaohui.zheng@intel.com>
diff --git a/arch/x86/mm/init.c b/arch/x86/mm/init.c
index d406c52..b6a85cc 100644
--- a/arch/x86/mm/init.c
+++ b/arch/x86/mm/init.c
@@ -1,6 +1,7 @@
 #include <linux/initrd.h>
 #include <linux/ioport.h>
 #include <linux/swap.h>
+#include <linux/bootmem.h>
 
 #include <asm/cacheflush.h>
 #include <asm/e820.h>
@@ -386,3 +387,24 @@ void free_initrd_mem(unsigned long start, unsigned long end)
 	free_init_pages("initrd memory", start, end);
 }
 #endif
+
+/**
+ * After memory hotplug, the variable max_pfn, max_low_pfn and high_memory will
+ * be affected, it will be updated in this function. Memory hotplug still has
+ * critical issues on 32-bit kennel, it was more important on 64-bit kernel,
+ * so we update the variables for 64-bit kernel first, fix me in future for
+ * 32-bit kenrel.
+ */
+void __meminit __attribute__((weak)) update_end_of_memory_vars(u64 start,
+		u64 size)
+{
+#ifdef CONFIG_X86_64
+	unsigned long start_pfn = start >> PAGE_SHIFT;
+	unsigned long end_pfn = PFN_UP(start + size);
+
+	if (end_pfn > max_pfn) {
+		max_low_pfn = max_pfn = end_pfn;
+		high_memory = (void *)__va(max_pfn * PAGE_SIZE - 1) + 1;
+	}
+#endif /* CONFIG_X86_64 */
+}
diff --git a/include/linux/bootmem.h b/include/linux/bootmem.h
index b10ec49..84533a5 100644
--- a/include/linux/bootmem.h
+++ b/include/linux/bootmem.h
@@ -13,6 +13,7 @@
 
 extern unsigned long max_low_pfn;
 extern unsigned long min_low_pfn;
+extern void update_end_of_memory_vars(u64 start, u64 size);
 
 /*
  * highest page
diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c
index 030ce8a..3e94b23 100644
--- a/mm/memory_hotplug.c
+++ b/mm/memory_hotplug.c
@@ -523,6 +523,9 @@ int __ref add_memory(int nid, u64 start, u64 size)
 		BUG_ON(ret);
 	}
 
+	/* update max_pfn, max_low_pfn and high_memory */
+	update_end_of_memory_vars(start, size);
+
 	goto out;
 
 error:

Thanks & Regards,
Shaohui




[-- Attachment #2: memory-hotplug-fix-the-bug-on-interface-dev-mem-v3.patch --]
[-- Type: application/octet-stream, Size: 2529 bytes --]

Memory-Hotplug: Fix the bug on interface /dev/mem for 64-bit kernel

The new added memory can not be access by interface /dev/mem, because we do not
 update the variable high_memory, max_pfn and max_low_pfn.

Memory hotplug still has critical issues for 32-bit kernel, and it is more 
important for 64-bit kernel, we fix it on 64-bit first. We add a function 
update_end_of_memory_vars in file arch/x86/mm/init.c to update these variables.

Signed-off-by: Shaohui Zheng <shaohui.zheng@intel.com>
CC: Andi Kleen <ak@linux.intel.com>
CC: Li Haicheng <haicheng.li@intel.com>
Reviewed-by: Wu Fengguang <fengguang.wu@intel.com>
Reviewed-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
diff --git a/arch/x86/mm/init.c b/arch/x86/mm/init.c
index d406c52..b6a85cc 100644
--- a/arch/x86/mm/init.c
+++ b/arch/x86/mm/init.c
@@ -1,6 +1,7 @@
 #include <linux/initrd.h>
 #include <linux/ioport.h>
 #include <linux/swap.h>
+#include <linux/bootmem.h>
 
 #include <asm/cacheflush.h>
 #include <asm/e820.h>
@@ -386,3 +387,24 @@ void free_initrd_mem(unsigned long start, unsigned long end)
 	free_init_pages("initrd memory", start, end);
 }
 #endif
+
+/**
+ * After memory hotplug, the variable max_pfn, max_low_pfn and high_memory will
+ * be affected, it will be updated in this function. Memory hotplug still has
+ * critical issues on 32-bit kennel, it was more important on 64-bit kernel,
+ * so we update the variables for 64-bit kernel first, fix me in future for
+ * 32-bit kenrel.
+ */
+void __meminit __attribute__((weak)) update_end_of_memory_vars(u64 start,
+		u64 size)
+{
+#ifdef CONFIG_X86_64
+	unsigned long start_pfn = start >> PAGE_SHIFT;
+	unsigned long end_pfn = PFN_UP(start + size);
+
+	if (end_pfn > max_pfn) {
+		max_low_pfn = max_pfn = end_pfn;
+		high_memory = (void *)__va(max_pfn * PAGE_SIZE - 1) + 1;
+	}
+#endif /* CONFIG_X86_64 */
+}
diff --git a/include/linux/bootmem.h b/include/linux/bootmem.h
index b10ec49..84533a5 100644
--- a/include/linux/bootmem.h
+++ b/include/linux/bootmem.h
@@ -13,6 +13,7 @@
 
 extern unsigned long max_low_pfn;
 extern unsigned long min_low_pfn;
+extern void update_end_of_memory_vars(u64 start, u64 size);
 
 /*
  * highest page
diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c
index 030ce8a..3e94b23 100644
--- a/mm/memory_hotplug.c
+++ b/mm/memory_hotplug.c
@@ -523,6 +523,9 @@ int __ref add_memory(int nid, u64 start, u64 size)
 		BUG_ON(ret);
 	}
 
+	/* update max_pfn, max_low_pfn and high_memory */
+	update_end_of_memory_vars(start, size);
+
 	goto out;
 
 error:

Re: [ RESEND PATCH v3] Memory-Hotplug: Fix the bug on interface /dev/mem for 64-bit kernel

[KAMEZAWA Hiroyuki]

On Tue, 12 Jan 2010 15:45:54 +0800
"Zheng, Shaohui" <shaohui.zheng@intel.com> wrote:

> Resend the v3 patch after reviewed by KAMEZAWA Hiroyuki. We still keep the 
> Old e820map, update variable max_pfn, max_low_pfn and high_memory only. 
> It is dependent on Fenguang's page_is_ram patch.
> 
> Memory-Hotplug: Fix the bug on interface /dev/mem for 64-bit kernel
> 
> The new added memory can not be access by interface /dev/mem, because we do not
>  update the variable high_memory, max_pfn and max_low_pfn.
> 
> Memory hotplug still has critical issues for 32-bit kernel, and it is more 
> important for 64-bit kernel, we fix it on 64-bit first. We add a function 
> update_end_of_memory_vars in file arch/x86/mm/init.c to update these variables.
> 
> CC: Andi Kleen <ak@linux.intel.com>
> CC: Li Haicheng <haicheng.li@intel.com>
> Reviewed-by: Wu Fengguang <fengguang.wu@intel.com>
> Reviewed-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
> Signed-off-by: Shaohui Zheng <shaohui.zheng@intel.com>

3 points...
1. I think this patch cannot be compiled in archs other than x86. Right ?
   IOW, please add static inline dummy...

2. pgdat->[start,end], totalram_pages etc...are updated at memory hotplug.
   Please place the hook nearby them.

3. I recommend you yo use memory hotplug notifier.
   If it's allowed, it will be cleaner.
   It seems there are no strict ordering to update parameters this patch touches.

Thanks,
-Kame





> diff --git a/arch/x86/mm/init.c b/arch/x86/mm/init.c
> index d406c52..b6a85cc 100644
> --- a/arch/x86/mm/init.c
> +++ b/arch/x86/mm/init.c
> @@ -1,6 +1,7 @@
>  #include <linux/initrd.h>
>  #include <linux/ioport.h>
>  #include <linux/swap.h>
> +#include <linux/bootmem.h>
>  
>  #include <asm/cacheflush.h>
>  #include <asm/e820.h>
> @@ -386,3 +387,24 @@ void free_initrd_mem(unsigned long start, unsigned long end)
>  	free_init_pages("initrd memory", start, end);
>  }
>  #endif
> +
> +/**
> + * After memory hotplug, the variable max_pfn, max_low_pfn and high_memory will
> + * be affected, it will be updated in this function. Memory hotplug still has
> + * critical issues on 32-bit kennel, it was more important on 64-bit kernel,
> + * so we update the variables for 64-bit kernel first, fix me in future for
> + * 32-bit kenrel.
> + */
> +void __meminit __attribute__((weak)) update_end_of_memory_vars(u64 start,
> +		u64 size)
> +{
> +#ifdef CONFIG_X86_64
> +	unsigned long start_pfn = start >> PAGE_SHIFT;
> +	unsigned long end_pfn = PFN_UP(start + size);
> +
> +	if (end_pfn > max_pfn) {
> +		max_low_pfn = max_pfn = end_pfn;
> +		high_memory = (void *)__va(max_pfn * PAGE_SIZE - 1) + 1;
> +	}
> +#endif /* CONFIG_X86_64 */
> +}
> diff --git a/include/linux/bootmem.h b/include/linux/bootmem.h
> index b10ec49..84533a5 100644
> --- a/include/linux/bootmem.h
> +++ b/include/linux/bootmem.h
> @@ -13,6 +13,7 @@
>  
>  extern unsigned long max_low_pfn;
>  extern unsigned long min_low_pfn;
> +extern void update_end_of_memory_vars(u64 start, u64 size);
>  
>  /*
>   * highest page
> diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c
> index 030ce8a..3e94b23 100644
> --- a/mm/memory_hotplug.c
> +++ b/mm/memory_hotplug.c
> @@ -523,6 +523,9 @@ int __ref add_memory(int nid, u64 start, u64 size)
>  		BUG_ON(ret);
>  	}
>  
> +	/* update max_pfn, max_low_pfn and high_memory */
> +	update_end_of_memory_vars(start, size);
> +
>  	goto out;
>  
>  error:
> 
> Thanks & Regards,
> Shaohui
> 
> 
> 

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [ RESEND PATCH v3] Memory-Hotplug: Fix the bug on interface /dev/mem for 64-bit kernel

[KAMEZAWA Hiroyuki]

On Tue, 12 Jan 2010 16:57:40 +0800
"Zheng, Shaohui" <shaohui.zheng@intel.com> wrote:

> 
> 3 points...
> 1. I think this patch cannot be compiled in archs other than x86. Right ?
>    IOW, please add static inline dummy...
> [Zheng, Shaohui] Agree, I will add a static dummy function
> 
> 2. pgdat->[start,end], totalram_pages etc...are updated at memory hotplug.
>    Please place the hook nearby them.
> [Zheng, Shaohui] Agree.
> 
> 3. I recommend you yo use memory hotplug notifier.
>    If it's allowed, it will be cleaner.
>    It seems there are no strict ordering to update parameters this patch touches.
> 
> [Zheng, Shaohui] Kame, do you means put the hook into function slab_mem_going_online_callback, it seems a good idea. If we select this method, we will need not to update these variable in function add_memory explicitly.
> 
yes. I think callback is the best.

Thanks,
-Kame

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

RE: [ RESEND PATCH v3] Memory-Hotplug: Fix the bug on interface /dev/mem for 64-bit kernel

[Zheng, Shaohui]

3 points...
1. I think this patch cannot be compiled in archs other than x86. Right ?
   IOW, please add static inline dummy...
[Zheng, Shaohui] Agree, I will add a static dummy function

2. pgdat->[start,end], totalram_pages etc...are updated at memory hotplug.
   Please place the hook nearby them.
[Zheng, Shaohui] Agree.

3. I recommend you yo use memory hotplug notifier.
   If it's allowed, it will be cleaner.
   It seems there are no strict ordering to update parameters this patch touches.

[Zheng, Shaohui] Kame, do you means put the hook into function slab_mem_going_online_callback, it seems a good idea. If we select this method, we will need not to update these variable in function add_memory explicitly.

Thanks,
-Kame



--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

RE: [ RESEND PATCH v3] Memory-Hotplug: Fix the bug on interface /dev/mem for 64-bit kernel

[Zheng, Shaohui]

> 
> 3 points...
> 1. I think this patch cannot be compiled in archs other than x86. Right ?
>    IOW, please add static inline dummy...
> [Zheng, Shaohui] Agree, I will add a static dummy function
> 
> 2. pgdat->[start,end], totalram_pages etc...are updated at memory hotplug.
>    Please place the hook nearby them.
> [Zheng, Shaohui] Agree.
> 
> 3. I recommend you yo use memory hotplug notifier.
>    If it's allowed, it will be cleaner.
>    It seems there are no strict ordering to update parameters this patch touches.
> 
> [Zheng, Shaohui] Kame, do you means put the hook into function slab_mem_going_online_callback, it seems a good idea. If we select this method, we will need not to update these variable in function add_memory explicitly.
> 
yes. I think callback is the best.
[Zheng, Shaohui] it is okay for me, I will rewrite my patch and test it in local, thanks Kame :).

Thanks,
-Kame


Thanks & Regards,
Shaohui

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

RE: [ RESEND PATCH v3] Memory-Hotplug: Fix the bug on interface /dev/mem for 64-bit kernel

[Zheng, Shaohui]

> 
> 3 points...
> 1. I think this patch cannot be compiled in archs other than x86. Right ?
>    IOW, please add static inline dummy...
> [Zheng, Shaohui] Agree, I will add a static dummy function
> 
> 2. pgdat->[start,end], totalram_pages etc...are updated at memory hotplug.
>    Please place the hook nearby them.
> [Zheng, Shaohui] Agree.
> 
> 3. I recommend you yo use memory hotplug notifier.
>    If it's allowed, it will be cleaner.
>    It seems there are no strict ordering to update parameters this patch touches.
> 
> [Zheng, Shaohui] Kame, do you means put the hook into function slab_mem_going_online_callback, it seems a good idea. If we select this method, we will need not to update these variable in function add_memory explicitly.
> 
yes. I think callback is the best.
[Zheng, Shaohui] Kame, I check the code carefully and write a simple patch, I found the callback will never be executed. The only entrance of this callback is in function online_pages/offline_pages, but I did not found explicit call to these 2 functions. Do we need to add a memory_notify to trigger this callback in function add_memory? 


Thanks,
-Kame

Thanks & Regards,
Shaohui

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [ RESEND PATCH v3] Memory-Hotplug: Fix the bug on interface /dev/mem for 64-bit kernel

[Wu Fengguang]

Kame,

On Tue, Jan 12, 2010 at 05:04:33PM +0900, KAMEZAWA Hiroyuki wrote:

> 3 points...
> 1. I think this patch cannot be compiled in archs other than x86. Right ?
>    IOW, please add static inline dummy...

Good catch!

> 2. pgdat->[start,end], totalram_pages etc...are updated at memory hotplug.
>    Please place the hook nearby them.

arch/x86/mm/init_64.c:arch_add_memory() updates max_pfn_mapped, in
this sense it's equally OK to update max_pfn/max_low_pfn etc before
the call to arch_add_memory() ;)

> 3. I recommend you yo use memory hotplug notifier.
>    If it's allowed, it will be cleaner.

Hmm, notifier is for _outsider_ subsystems. It smells a bit
overkill to do notifier _inside_ the hotplug code.

>    It seems there are no strict ordering to update parameters this patch touches.

I tend to agree. That said, it does help keep our mind straight if we do it
in some logical order: max_pfn => max_pfn_mapped => totalram_pages etc.

Thanks,
Fengguang

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [ RESEND PATCH v3] Memory-Hotplug: Fix the bug on interface /dev/mem for 64-bit kernel

[Wu Fengguang]

> > 2. pgdat->[start,end], totalram_pages etc...are updated at memory hotplug.
> >    Please place the hook nearby them.
> 
> arch/x86/mm/init_64.c:arch_add_memory() updates max_pfn_mapped, in
> this sense it's equally OK to update max_pfn/max_low_pfn etc before
> the call to arch_add_memory() ;)

Shaohui, I'd suggest to update max_pfn/max_low_pfn/high_memory in
arch/x86/mm/init_64.c:arch_add_memory() now, for X86_64.

Later on we can add code to arch/x86/mm/init_32.c:arch_add_memory()
for X86_32.

The code cannot be shared anyway.

Thanks,
Fengguang

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>