[PATCH 0/2] memory-hotplug : notification of memoty block's state

[PATCH 0/2] memory-hotplug : notification of memoty block's state

[Yasuaki Ishimatsu]

We are trying to implement a physical memory hot removing function as
following thread.

https://lkml.org/lkml/2012/9/5/201

But there is not enough review to merge into linux kernel.

I think there are following blockades.
  1. no physical memory hot removable system
  2. huge patch-set

If you have a KVM system, we can get rid of 1st blockade. Because
applying following patch, we can create memory hot removable system
on KVM guest.

http://lists.gnu.org/archive/html/qemu-devel/2012-07/msg01389.html

2nd blockade is own problem. So we try to divide huge patch into
a small patch in each function as follows: 

 - bug fix
 - acpi framework
 - kernel core

We had already sent bug fix patches.

https://lkml.org/lkml/2012/9/27/39

And the patch fixes following bug.

remove_memory() offlines memory. And it is called by following two cases:

1. echo offline >/sys/devices/system/memory/memoryXX/state
2. hot remove a memory device

In the 1st case, the memory block's state is changed and the notification
that memory block's state changed is sent to userland after calling
offline_memory(). So user can notice memory block is changed.

But in the 2nd case, the memory block's state is not changed and the
notification is not also sent to userspcae even if calling offline_memory().
So user cannot notice memory block is changed.

We should also notify to userspace at 2nd case.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

[Patch 1/2] memory-hotplug : Preparation to notify memory block's state at memory hot remove

[Yasuaki Ishimatsu]

From: Wen Congyang <wency@cn.fujitsu.com>

remove_memory() is called in two cases:
1. echo offline >/sys/devices/system/memory/memoryXX/state
2. hot remove a memory device

In the 1st case, the memory block's state is changed and the notification
that memory block's state changed is sent to userland after calling
remove_memory(). So user can notice memory block is changed.

But in the 2nd case, the memory block's state is not changed and the
notification is not also sent to userspcae even if calling remove_memory().
So user cannot notice memory block is changed.

For adding the notification at memory hot remove, the patch just prepare
as follows:
1st case uses offline_pages() for offlining memory.
2nd case uses remove_memory() for offlining memory and changing memory block's
state and notifing the information.

The patch does not implement notification to remove_memory().

CC: David Rientjes <rientjes@google.com>
CC: Jiang Liu <liuj97@gmail.com>
CC: Len Brown <len.brown@intel.com>
CC: Christoph Lameter <cl@linux.com>
Cc: Minchan Kim <minchan.kim@gmail.com>
CC: Andrew Morton <akpm@linux-foundation.org>
CC: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com> 
Signed-off-by: Wen Congyang <wency@cn.fujitsu.com>
Signed-off-by: Yasuaki Ishimatsu <isimatu.yasuaki@jp.fujitsu.com>
---
 drivers/base/memory.c          |    9 +++------
 include/linux/memory_hotplug.h |    1 +
 mm/memory_hotplug.c            |   13 +++++++++++--
 3 files changed, 15 insertions(+), 8 deletions(-)

Index: linux-3.6/drivers/base/memory.c
===================================================================
--- linux-3.6.orig/drivers/base/memory.c	2012-10-02 16:01:46.000000000 +0900
+++ linux-3.6/drivers/base/memory.c	2012-10-02 16:07:08.278081232 +0900
@@ -248,26 +248,23 @@ static bool pages_correctly_reserved(uns
 static int
 memory_block_action(unsigned long phys_index, unsigned long action)
 {
-	unsigned long start_pfn, start_paddr;
+	unsigned long start_pfn;
 	unsigned long nr_pages = PAGES_PER_SECTION * sections_per_block;
 	struct page *first_page;
 	int ret;
 
 	first_page = pfn_to_page(phys_index << PFN_SECTION_SHIFT);
+	start_pfn = page_to_pfn(first_page);
 
 	switch (action) {
 		case MEM_ONLINE:
-			start_pfn = page_to_pfn(first_page);
-
 			if (!pages_correctly_reserved(start_pfn, nr_pages))
 				return -EBUSY;
 
 			ret = online_pages(start_pfn, nr_pages);
 			break;
 		case MEM_OFFLINE:
-			start_paddr = page_to_pfn(first_page) << PAGE_SHIFT;
-			ret = remove_memory(start_paddr,
-					    nr_pages << PAGE_SHIFT);
+			ret = offline_pages(start_pfn, nr_pages);
 			break;
 		default:
 			WARN(1, KERN_WARNING "%s(%ld, %ld) unknown action: "
Index: linux-3.6/include/linux/memory_hotplug.h
===================================================================
--- linux-3.6.orig/include/linux/memory_hotplug.h	2012-10-02 16:01:46.000000000 +0900
+++ linux-3.6/include/linux/memory_hotplug.h	2012-10-02 16:07:08.281081235 +0900
@@ -233,6 +233,7 @@ static inline int is_mem_section_removab
 extern int mem_online_node(int nid);
 extern int add_memory(int nid, u64 start, u64 size);
 extern int arch_add_memory(int nid, u64 start, u64 size);
+extern int offline_pages(unsigned long start_pfn, unsigned long nr_pages);
 extern int remove_memory(u64 start, u64 size);
 extern int sparse_add_one_section(struct zone *zone, unsigned long start_pfn,
 								int nr_pages);
Index: linux-3.6/mm/memory_hotplug.c
===================================================================
--- linux-3.6.orig/mm/memory_hotplug.c	2012-10-02 16:01:46.000000000 +0900
+++ linux-3.6/mm/memory_hotplug.c	2012-10-02 16:07:08.279081233 +0900
@@ -870,7 +870,7 @@ check_pages_isolated(unsigned long start
 	return offlined;
 }
 
-static int __ref offline_pages(unsigned long start_pfn,
+static int __ref __offline_pages(unsigned long start_pfn,
 		  unsigned long end_pfn, unsigned long timeout)
 {
 	unsigned long pfn, nr_pages, expire;
@@ -998,15 +998,24 @@ out:
 	return ret;
 }
 
+int offline_pages(unsigned long start_pfn, unsigned long nr_pages)
+{
+	return __offline_pages(start_pfn, start_pfn + nr_pages, 120 * HZ);
+}
+
 int remove_memory(u64 start, u64 size)
 {
 	unsigned long start_pfn, end_pfn;
 
 	start_pfn = PFN_DOWN(start);
 	end_pfn = start_pfn + PFN_DOWN(size);
-	return offline_pages(start_pfn, end_pfn, 120 * HZ);
+	return __offline_pages(start_pfn, end_pfn, 120 * HZ);
 }
 #else
+int offline_pages(unsigned long start_pfn, unsigned long nr_pages)
+{
+	return -EINVAL;
+}
 int remove_memory(u64 start, u64 size)
 {
 	return -EINVAL;

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

[Patch 2/2] memory-hotplug : update memory block's state and notfy theinformation to userspace

[Yasuaki Ishimatsu]

From: Wen Congyang <wency@cn.fujitsu.com>

The function remove_memory() will be called when hot removing a
memory device. But even if offlining memory, we cannot notice it.
So the patch update memory block's state and notify to userspace.

Additionally, the memory device may contain more than one memory
block. If the memory block has been offlined, __offline_pages()
will fail. So we should try to offline one memory block at a
time.

Thus the function remove_memory() also check each memory block's
state. So there is no need to check the memory block's state
before calling remove_memory().

CC: David Rientjes <rientjes@google.com>
CC: Jiang Liu <liuj97@gmail.com>
CC: Len Brown <len.brown@intel.com>
CC: Christoph Lameter <cl@linux.com>
Cc: Minchan Kim <minchan.kim@gmail.com>
CC: Andrew Morton <akpm@linux-foundation.org>
CC: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com> 
Signed-off-by: Wen Congyang <wency@cn.fujitsu.com>
Signed-off-by: Yasuaki Ishimatsu <isimatu.yasuaki@jp.fujitsu.com>
---
 drivers/base/memory.c          |   31 +++++++++++++++++++++++++++----
 include/linux/memory_hotplug.h |    2 ++
 mm/memory_hotplug.c            |   33 ++++++++++++++++++++++++++++++++-
 3 files changed, 61 insertions(+), 5 deletions(-)

Index: linux-3.6/drivers/base/memory.c
===================================================================
--- linux-3.6.orig/drivers/base/memory.c	2012-10-02 16:03:43.000000000 +0900
+++ linux-3.6/drivers/base/memory.c	2012-10-02 16:06:27.786081495 +0900
@@ -275,13 +275,11 @@ memory_block_action(unsigned long phys_i
 	return ret;
 }
 
-static int memory_block_change_state(struct memory_block *mem,
+static int __memory_block_change_state(struct memory_block *mem,
 		unsigned long to_state, unsigned long from_state_req)
 {
 	int ret = 0;
 
-	mutex_lock(&mem->state_mutex);
-
 	if (mem->state != from_state_req) {
 		ret = -EINVAL;
 		goto out;
@@ -309,10 +307,20 @@ static int memory_block_change_state(str
 		break;
 	}
 out:
-	mutex_unlock(&mem->state_mutex);
 	return ret;
 }
 
+static int memory_block_change_state(struct memory_block *mem,
+		unsigned long to_state, unsigned long from_state_req)
+{
+	int ret;
+
+	mutex_lock(&mem->state_mutex);
+	ret = __memory_block_change_state(mem, to_state, from_state_req);
+	mutex_unlock(&mem->state_mutex);
+
+	return ret;
+}
 static ssize_t
 store_mem_state(struct device *dev,
 		struct device_attribute *attr, const char *buf, size_t count)
@@ -653,6 +661,21 @@ int unregister_memory_section(struct mem
 }
 
 /*
+ * offline one memory block. If the memory block has been offlined, do nothing.
+ */
+int offline_memory_block(struct memory_block *mem)
+{
+	int ret = 0;
+
+	mutex_lock(&mem->state_mutex);
+	if (mem->state != MEM_OFFLINE)
+		ret = __memory_block_change_state(mem, MEM_OFFLINE, MEM_ONLINE);
+	mutex_unlock(&mem->state_mutex);
+
+	return ret;
+}
+
+/*
  * Initialize the sysfs support for memory devices...
  */
 int __init memory_dev_init(void)
Index: linux-3.6/include/linux/memory_hotplug.h
===================================================================
--- linux-3.6.orig/include/linux/memory_hotplug.h	2012-10-02 16:04:19.000000000 +0900
+++ linux-3.6/include/linux/memory_hotplug.h	2012-10-02 16:07:02.118080769 +0900
@@ -10,6 +10,7 @@ struct page;
 struct zone;
 struct pglist_data;
 struct mem_section;
+struct memory_block;
 
 #ifdef CONFIG_MEMORY_HOTPLUG
 
@@ -234,6 +235,7 @@ extern int mem_online_node(int nid);
 extern int add_memory(int nid, u64 start, u64 size);
 extern int arch_add_memory(int nid, u64 start, u64 size);
 extern int offline_pages(unsigned long start_pfn, unsigned long nr_pages);
+extern int offline_memory_block(struct memory_block *mem);
 extern int remove_memory(u64 start, u64 size);
 extern int sparse_add_one_section(struct zone *zone, unsigned long start_pfn,
 								int nr_pages);
Index: linux-3.6/mm/memory_hotplug.c
===================================================================
--- linux-3.6.orig/mm/memory_hotplug.c	2012-10-02 16:05:13.000000000 +0900
+++ linux-3.6/mm/memory_hotplug.c	2012-10-02 16:06:27.794081501 +0900
@@ -1005,11 +1005,42 @@ int offline_pages(unsigned long start_pf
 
 int remove_memory(u64 start, u64 size)
 {
+	struct memory_block *mem = NULL;
+	struct mem_section *section;
 	unsigned long start_pfn, end_pfn;
+	unsigned long pfn, section_nr;
+	int ret;
 
 	start_pfn = PFN_DOWN(start);
 	end_pfn = start_pfn + PFN_DOWN(size);
-	return __offline_pages(start_pfn, end_pfn, 120 * HZ);
+
+	for (pfn = start_pfn; pfn < end_pfn; pfn += PAGES_PER_SECTION) {
+		section_nr = pfn_to_section_nr(pfn);
+		if (!present_section_nr(section_nr))
+			continue;
+
+		section = __nr_to_section(section_nr);
+		/* same memblock? */
+		if (mem)
+			if ((section_nr >= mem->start_section_nr) &&
+			    (section_nr <= mem->end_section_nr))
+				continue;
+
+		mem = find_memory_block_hinted(section, mem);
+		if (!mem)
+			continue;
+
+		ret = offline_memory_block(mem);
+		if (ret) {
+			kobject_put(&mem->dev.kobj);
+			return ret;
+		}
+	}
+
+	if (mem)
+		kobject_put(&mem->dev.kobj);
+
+	return 0;
 }
 #else
 int offline_pages(unsigned long start_pfn, unsigned long nr_pages)

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 0/2] memory-hotplug : notification of memoty block's state

[Ni zhan Chen]

On 10/02/2012 04:25 PM, Yasuaki Ishimatsu wrote:
> We are trying to implement a physical memory hot removing function as
> following thread.
>
> https://lkml.org/lkml/2012/9/5/201
>
> But there is not enough review to merge into linux kernel.
>
> I think there are following blockades.
>   1. no physical memory hot removable system
>   2. huge patch-set
>
> If you have a KVM system, we can get rid of 1st blockade. Because
> applying following patch, we can create memory hot removable system
> on KVM guest.
>
> http://lists.gnu.org/archive/html/qemu-devel/2012-07/msg01389.html
>
> 2nd blockade is own problem. So we try to divide huge patch into
> a small patch in each function as follows: 
>
>  - bug fix
>  - acpi framework
>  - kernel core
>
> We had already sent bug fix patches.
>
> https://lkml.org/lkml/2012/9/27/39
>
> And the patch fixes following bug.
>
> remove_memory() offlines memory. And it is called by following two cases:
>
> 1. echo offline >/sys/devices/system/memory/memoryXX/state
> 2. hot remove a memory device
>
> In the 1st case, the memory block's state is changed and the notification
> that memory block's state changed is sent to userland after calling
> offline_memory(). So user can notice memory block is changed.,

Hi Yasuaki,

Thanks for splitting the patchset, it's more easier to review this time.
One question:

How can notify userspace? you mean function node_memory_callback or
...., but
this function basically do nothing.

>
> But in the 2nd case, the memory block's state is not changed and the
> notification is not also sent to userspcae even if calling offline_memory().
> So user cannot notice memory block is changed.
>
> We should also notify to userspace at 2nd case.
>
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 0/2] memory-hotplug : notification of memoty block's state

[Yasuaki Ishimatsu]

2012/10/02 18:42, Ni zhan Chen wrote:
> On 10/02/2012 04:25 PM, Yasuaki Ishimatsu wrote:
>> We are trying to implement a physical memory hot removing function as
>> following thread.
>>
>> https://lkml.org/lkml/2012/9/5/201
>>
>> But there is not enough review to merge into linux kernel.
>>
>> I think there are following blockades.
>>    1. no physical memory hot removable system
>>    2. huge patch-set
>>
>> If you have a KVM system, we can get rid of 1st blockade. Because
>> applying following patch, we can create memory hot removable system
>> on KVM guest.
>>
>> http://lists.gnu.org/archive/html/qemu-devel/2012-07/msg01389.html
>>
>> 2nd blockade is own problem. So we try to divide huge patch into
>> a small patch in each function as follows:
>>
>>   - bug fix
>>   - acpi framework
>>   - kernel core
>>
>> We had already sent bug fix patches.
>>
>> https://lkml.org/lkml/2012/9/27/39
>>
>> And the patch fixes following bug.
>>
>> remove_memory() offlines memory. And it is called by following two cases:
>>
>> 1. echo offline >/sys/devices/system/memory/memoryXX/state
>> 2. hot remove a memory device
>>
>> In the 1st case, the memory block's state is changed and the notification
>> that memory block's state changed is sent to userland after calling
>> offline_memory(). So user can notice memory block is changed.,
> 
> Hi Yasuaki,
> 
> Thanks for splitting the patchset, it's more easier to review this time.
> One question:
> 
> How can notify userspace? you mean function node_memory_callback or
> ...., but

When calling memory_block_change_state(), it calls kobject_uevent().
This function notifies userspace of the online/offline notification.

Thanks,
Yasuaki Ishimatsu


> this function basically do nothing.
> 
>>
>> But in the 2nd case, the memory block's state is not changed and the
>> notification is not also sent to userspcae even if calling offline_memory().
>> So user cannot notice memory block is changed.
>>
>> We should also notify to userspace at 2nd case.
>>
>> --
>> To unsubscribe, send a message with 'unsubscribe linux-mm' in
>> the body to majordomo@kvack.org.  For more info on Linux MM,
>> see: http://www.linux-mm.org/ .
>> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
>>
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-acpi" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 0/2] memory-hotplug : notification of memoty block's state

[Andrew Morton]

On Tue, 2 Oct 2012 17:25:06 +0900
Yasuaki Ishimatsu <isimatu.yasuaki@jp.fujitsu.com> wrote:

> remove_memory() offlines memory. And it is called by following two cases:
> 
> 1. echo offline >/sys/devices/system/memory/memoryXX/state
> 2. hot remove a memory device
> 
> In the 1st case, the memory block's state is changed and the notification
> that memory block's state changed is sent to userland after calling
> offline_memory(). So user can notice memory block is changed.
> 
> But in the 2nd case, the memory block's state is not changed and the
> notification is not also sent to userspcae even if calling offline_memory().
> So user cannot notice memory block is changed.
> 
> We should also notify to userspace at 2nd case.

These two little patches look reasonable to me.

There's a lot of recent activity with memory hotplug!  We're in the 3.7
merge window now so it is not a good time to be merging new material. 
Also there appear to be two teams working on it and it's unclear to me
how well coordinated this work is?

However these two patches are pretty simple and do fix a problem, so I
added them to the 3.7 MM queue.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 0/2] memory-hotplug : notification of memoty block's state

[Yasuaki Ishimatsu]

Hi Andrew,

2012/10/03 6:42, Andrew Morton wrote:
> On Tue, 2 Oct 2012 17:25:06 +0900
> Yasuaki Ishimatsu <isimatu.yasuaki@jp.fujitsu.com> wrote:
>
>> remove_memory() offlines memory. And it is called by following two cases:
>>
>> 1. echo offline >/sys/devices/system/memory/memoryXX/state
>> 2. hot remove a memory device
>>
>> In the 1st case, the memory block's state is changed and the notification
>> that memory block's state changed is sent to userland after calling
>> offline_memory(). So user can notice memory block is changed.
>>
>> But in the 2nd case, the memory block's state is not changed and the
>> notification is not also sent to userspcae even if calling offline_memory().
>> So user cannot notice memory block is changed.
>>
>> We should also notify to userspace at 2nd case.
>
> These two little patches look reasonable to me.
>
> There's a lot of recent activity with memory hotplug!  We're in the 3.7
> merge window now so it is not a good time to be merging new material.

> Also there appear to be two teams working on it and it's unclear to me
> how well coordinated this work is?

As you know, there are two teams for developing the memory hotplug.
   - Wen's patch-set
     https://lkml.org/lkml/2012/9/5/201

   - Lai's patch-set
     https://lkml.org/lkml/2012/9/10/180

Wen's patch-set is for removing physical memory. Now, I'm splitting the
patch-set for reviewing more easy. If the patch-set is merged into
linux kernel, I believe that linux on x86 can hot remove a physical
memory device.

But it is not enough since we cannot remove a memory which has kernel
memory. If we guarantee the memory hot remove, the memory must belong
to ZONE_MOVABLE.

So Lai's patch-set tries to create a movable node that the all memory
belongs to ZONE_MOVABLE.

I think there are two chances for creating the movable node.
   - boot time
   - after hot add memory

- boot time

For creating a movable memory, linux has two kernel parameters
(kernelcore and movablecore). But it is not enough, since even if we
set the kernel paramter, the movable memory is distributed evenly in
each node. So we introduce the kernelcore_max_addr boot parameter.
The parameter limits the range of the memory used as a kernel memory.

For example, the system has following nodes.

     node0 : 0x40000000 - 0x80000000
     node1 : 0x80000000 - 0xc0000000

And when I want to hot remove a node1, we set "kernelcore_max_addr=0x80000000".
In doing so, kernel memory is limited within 0x80000000 and node1's
memory belongs to ZONE_MOEVALBE. As a result, we can guarantee that
node1 is a movable node and we always hot remove node1.

- after hot add memory

When hot adding memory, the memory belongs to ZONE_NORMAL and is offline.
If we online the memory, the memory may have kernel memory. In this case,
we cannot hot remove the memory. So we introduce the online_movable
function. If we use the function as follow, the memory belongs to
ZONE_MOVABLE.

echo online_movable > /sys/devices/system/node/nodeX/memoryX/state

So when new node is hot added and I echo "online_movale" to all hot added
memory, the node's memory belongs to ZONE_MOVABLE. As a result, we can
guarantee that the node is a movable node and we always hot remove node.

# I hope to help your understanding about our works by the information.

Thanks,
Yasuaki Ishimatsu

>
> However these two patches are pretty simple and do fix a problem, so I
> added them to the 3.7 MM queue.
>


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 0/2] memory-hotplug : notification of memoty block's state

[Ni zhan Chen]

On 10/03/2012 09:21 AM, Yasuaki Ishimatsu wrote:
> Hi Andrew,
>
> 2012/10/03 6:42, Andrew Morton wrote:
>> On Tue, 2 Oct 2012 17:25:06 +0900
>> Yasuaki Ishimatsu <isimatu.yasuaki@jp.fujitsu.com> wrote:
>>
>>> remove_memory() offlines memory. And it is called by following two 
>>> cases:
>>>
>>> 1. echo offline >/sys/devices/system/memory/memoryXX/state
>>> 2. hot remove a memory device
>>>
>>> In the 1st case, the memory block's state is changed and the 
>>> notification
>>> that memory block's state changed is sent to userland after calling
>>> offline_memory(). So user can notice memory block is changed.
>>>
>>> But in the 2nd case, the memory block's state is not changed and the
>>> notification is not also sent to userspcae even if calling 
>>> offline_memory().
>>> So user cannot notice memory block is changed.
>>>
>>> We should also notify to userspace at 2nd case.
>>
>> These two little patches look reasonable to me.
>>
>> There's a lot of recent activity with memory hotplug!  We're in the 3.7
>> merge window now so it is not a good time to be merging new material.
>
>> Also there appear to be two teams working on it and it's unclear to me
>> how well coordinated this work is?
>
> As you know, there are two teams for developing the memory hotplug.
>   - Wen's patch-set
>     https://lkml.org/lkml/2012/9/5/201
>
>   - Lai's patch-set
>     https://lkml.org/lkml/2012/9/10/180
>
> Wen's patch-set is for removing physical memory. Now, I'm splitting the
> patch-set for reviewing more easy. If the patch-set is merged into
> linux kernel, I believe that linux on x86 can hot remove a physical
> memory device.
>
> But it is not enough since we cannot remove a memory which has kernel
> memory. If we guarantee the memory hot remove, the memory must belong
> to ZONE_MOVABLE.
>
> So Lai's patch-set tries to create a movable node that the all memory
> belongs to ZONE_MOVABLE.
>
> I think there are two chances for creating the movable node.
>   - boot time
>   - after hot add memory
>
> - boot time
>
> For creating a movable memory, linux has two kernel parameters
> (kernelcore and movablecore). But it is not enough, since even if we
> set the kernel paramter, the movable memory is distributed evenly in
> each node. So we introduce the kernelcore_max_addr boot parameter.
> The parameter limits the range of the memory used as a kernel memory.
>
> For example, the system has following nodes.
>
>     node0 : 0x40000000 - 0x80000000
>     node1 : 0x80000000 - 0xc0000000
>
> And when I want to hot remove a node1, we set 
> "kernelcore_max_addr=0x80000000".
> In doing so, kernel memory is limited within 0x80000000 and node1's
> memory belongs to ZONE_MOEVALBE. As a result, we can guarantee that
> node1 is a movable node and we always hot remove node1.
>
> - after hot add memory
>
> When hot adding memory, the memory belongs to ZONE_NORMAL and is offline.
> If we online the memory, the memory may have kernel memory. In this case,
> we cannot hot remove the memory. So we introduce the online_movable
> function. If we use the function as follow, the memory belongs to
> ZONE_MOVABLE.
>
> echo online_movable > /sys/devices/system/node/nodeX/memoryX/state
>
> So when new node is hot added and I echo "online_movale" to all hot added
> memory, the node's memory belongs to ZONE_MOVABLE. As a result, we can Y
> guarantee that the node is a movable node and we always hot remove node.

Hi Yasuaki,

This time can kernel memory allocated from ZONE_MOVABLE ?

>
> # I hope to help your understanding about our works by the information.
>
> Thanks,
> Yasuaki Ishimatsu
>
>>
>> However these two patches are pretty simple and do fix a problem, so I
>> added them to the 3.7 MM queue.
>>
>
>
> -- 
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 0/2] memory-hotplug : notification of memoty block's state

[Yasuaki Ishimatsu]

Hi Chen,

2012/10/03 11:12, Ni zhan Chen wrote:
> On 10/03/2012 09:21 AM, Yasuaki Ishimatsu wrote:
>> Hi Andrew,
>>
>> 2012/10/03 6:42, Andrew Morton wrote:
>>> On Tue, 2 Oct 2012 17:25:06 +0900
>>> Yasuaki Ishimatsu <isimatu.yasuaki@jp.fujitsu.com> wrote:
>>>
>>>> remove_memory() offlines memory. And it is called by following two cases:
>>>>
>>>> 1. echo offline >/sys/devices/system/memory/memoryXX/state
>>>> 2. hot remove a memory device
>>>>
>>>> In the 1st case, the memory block's state is changed and the notification
>>>> that memory block's state changed is sent to userland after calling
>>>> offline_memory(). So user can notice memory block is changed.
>>>>
>>>> But in the 2nd case, the memory block's state is not changed and the
>>>> notification is not also sent to userspcae even if calling offline_memory().
>>>> So user cannot notice memory block is changed.
>>>>
>>>> We should also notify to userspace at 2nd case.
>>>
>>> These two little patches look reasonable to me.
>>>
>>> There's a lot of recent activity with memory hotplug!  We're in the 3.7
>>> merge window now so it is not a good time to be merging new material.
>>
>>> Also there appear to be two teams working on it and it's unclear to me
>>> how well coordinated this work is?
>>
>> As you know, there are two teams for developing the memory hotplug.
>>   - Wen's patch-set
>>     https://lkml.org/lkml/2012/9/5/201
>>
>>   - Lai's patch-set
>>     https://lkml.org/lkml/2012/9/10/180
>>
>> Wen's patch-set is for removing physical memory. Now, I'm splitting the
>> patch-set for reviewing more easy. If the patch-set is merged into
>> linux kernel, I believe that linux on x86 can hot remove a physical
>> memory device.
>>
>> But it is not enough since we cannot remove a memory which has kernel
>> memory. If we guarantee the memory hot remove, the memory must belong
>> to ZONE_MOVABLE.
>>
>> So Lai's patch-set tries to create a movable node that the all memory
>> belongs to ZONE_MOVABLE.
>>
>> I think there are two chances for creating the movable node.
>>   - boot time
>>   - after hot add memory
>>
>> - boot time
>>
>> For creating a movable memory, linux has two kernel parameters
>> (kernelcore and movablecore). But it is not enough, since even if we
>> set the kernel paramter, the movable memory is distributed evenly in
>> each node. So we introduce the kernelcore_max_addr boot parameter.
>> The parameter limits the range of the memory used as a kernel memory.
>>
>> For example, the system has following nodes.
>>
>>     node0 : 0x40000000 - 0x80000000
>>     node1 : 0x80000000 - 0xc0000000
>>
>> And when I want to hot remove a node1, we set "kernelcore_max_addr=0x80000000".
>> In doing so, kernel memory is limited within 0x80000000 and node1's
>> memory belongs to ZONE_MOEVALBE. As a result, we can guarantee that
>> node1 is a movable node and we always hot remove node1.
>>
>> - after hot add memory
>>
>> When hot adding memory, the memory belongs to ZONE_NORMAL and is offline.
>> If we online the memory, the memory may have kernel memory. In this case,
>> we cannot hot remove the memory. So we introduce the online_movable
>> function. If we use the function as follow, the memory belongs to
>> ZONE_MOVABLE.
>>
>> echo online_movable > /sys/devices/system/node/nodeX/memoryX/state
>>
>> So when new node is hot added and I echo "online_movale" to all hot added
>> memory, the node's memory belongs to ZONE_MOVABLE. As a result, we can Y
>> guarantee that the node is a movable node and we always hot remove node.
>
> Hi Yasuaki,
>
> This time can kernel memory allocated from ZONE_MOVABLE ?

No. In this case, the memory cannot be used as kernel memory.

Thanks,
Yasuaki Ishimatsu

>
>>
>> # I hope to help your understanding about our works by the information.
>>
>> Thanks,
>> Yasuaki Ishimatsu
>>
>>>
>>> However these two patches are pretty simple and do fix a problem, so I
>>> added them to the 3.7 MM queue.
>>>
>>
>>
>> --
>> To unsubscribe, send a message with 'unsubscribe linux-mm' in
>> the body to majordomo@kvack.org.  For more info on Linux MM,
>> see: http://www.linux-mm.org/ .
>> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
>>
>


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>