[PATCH v4 0/6] Add movablecore_map boot option with SRAT support.

[PATCH v4 0/6] Add movablecore_map boot option with SRAT support.

[Tang Chen]

[What we are doing]
This patchset provide a boot option for user to specify ZONE_MOVABLE memory
map for each node in the system.

movablecore_map=nn[KMG]@ss[KMG] or movablecore_map=acpi

movablecore_map=nn[KMG]@ss[KMG]:
This option makes sure memory range from ss to ss+nn is movable memory.

movablecore_map=acpi:
This option informs the kernel to use Hot Pluggable bit in SRAT from ACPI BIOS
to determine which memory device could be hotplugged. Users don't need to
take part in.


[Why we do this]
If we hot remove a memroy, the memory cannot have kernel memory,
because Linux cannot migrate kernel memory currently. Therefore,
we have to guarantee that the hot removed memory has only movable
memory.

Linux has two boot options, kernelcore= and movablecore=, for
creating movable memory. These boot options can specify the amount
of memory use as kernel or movable memory. Using them, we can
create ZONE_MOVABLE which has only movable memory.

But it does not fulfill a requirement of memory hot remove, because
even if we specify the boot options, movable memory is distributed
in each node evenly. So when we want to hot remove memory which
memory range is 0x80000000-0c0000000, we have no way to specify
the memory as movable memory.

So we proposed a new feature which specifies memory range to use as
movable memory.


[Ways to do this]
There may be 2 ways to specify movable memory.
 1. use firmware information
 2. use boot option

1. use firmware information
  According to ACPI spec 5.0, SRAT table has memory affinity structure
  and the structure has Hot Pluggable Filed. See "5.2.16.2 Memory
  Affinity Structure". If we use the information, we might be able to
  specify movable memory by firmware. For example, if Hot Pluggable
  Filed is enabled, Linux sets the memory as movable memory.

2. use boot option
  This is our proposal. New boot option can specify memory range to use
  as movable memory.


[How we do this]
We chose second way at first, because if we use first way, users cannot change
memory range to use as movable memory easily. We think if we create
movable memory, performance regression may occur by NUMA. In this case,
user can turn off the feature easily if we prepare the boot option.
And if we prepare the boot optino, the user can select which memory
to use as movable memory easily.

And from v4, we also support the first way, using Hot Pluggable bit.


[How to use]
Specify the following boot option:
movablecore_map=nn[KMG]@ss[KMG] or movablecore_map=acpi.

1. If user want to specify hotpluggable memory ranges by himself, then specify
   as the following:
        movablecore_map=nn[KMG]@ss[KMG]
   In this way, the kernel will check if the ranges are hotpluggable with info
   from SRAT from ACPI BIOS.
   - If a range is hotpluggable, then from ss to the end of the corresponding
     node will be ZONE_MOVABLE.
   - If a range is not hotpluggable, then the range will be ignored.

2. If user want to leave the configuration work to the kernel, then specify
   as the following:
        movablecore_map=acpi
   In this way, the kernel will get hotplug info from SRAT in ACPI BIOS, and
   auto decide which memory devices could be hotplugged. And all the memory
   on these devices will be in ZONE_MOVABLE.

3. If user didn't specify this option, then the kernel will use all the
   memory on all the nodes evenly. And there is no performance drawback.

And the following points should be considered.

1) If the range is involved in a single node, then from ss to the end of
   the node will be ZONE_MOVABLE.
2) If the range covers two or more nodes, then from ss to the end of
   the 1st node will be ZONE_MOVABLE, and all the other nodes will only
   have ZONE_MOVABLE.
3) If no range is in the node, then the node will have no ZONE_MOVABLE
   unless kernelcore or movablecore is specified.
4) This option could be specified at most MAX_NUMNODES times.
5) If kernelcore or movablecore is also specified, movablecore_map will have
   higher priority to be satisfied.
6) This option has no conflict with memmap option.


Change log:

v3 -> v4:
1) patch2: Add new function remove_movablecore_map() to remove a range from
           movablecore_map.map[].
2) patch2: Add movablecore_map=acpi logic to allow user to skip the physical
           address config. If this option is specified, movablecore_map.map[]
           will be clear at first, and add all the hotpluggable memory ranges
           into it when parsing SRAT.
3) patch3: New patch, add logic to check the Hot Pluggable bit when parsing SRAT.
           If user also specifies a memory range, the logic will check if it is
           hotpluggable and remove it from movablecore_map.map[] if not.

v2 -> v3:
1) Use memblock_alloc_try_nid() instead of memblock_alloc_nid() to allocate
   memory twice if a whole node is ZONE_MOVABLE.
2) Add DMA, DMA32 addresses check, make sure ZONE_MOVABLE won't use these addresses.
   Suggested by Wu Jianguo <wujianguo@huawei.com>
3) Add lowmem addresses check, when the system has highmem, make sure ZONE_MOVABLE
   won't use lowmem. Suggested by Liu Jiang <jiang.liu@huawei.com>
4) Fix misuse of pfns in movablecore_map.map[] as physical addresses.



Tang Chen (5):
  page_alloc: add movablecore_map kernel parameter
  ACPI: Restructure movablecore_map with memory info from SRAT.
  page_alloc: Introduce zone_movable_limit[] to keep movable limit for
    nodes
  page_alloc: Make movablecore_map has higher priority
  page_alloc: Bootmem limit with movablecore_map

Yasuaki Ishimatsu (1):
  x86: get pg_data_t's memory from other node

 Documentation/kernel-parameters.txt |   29 ++++
 arch/x86/mm/numa.c                  |    5 +-
 arch/x86/mm/srat.c                  |   38 ++++-
 include/linux/memblock.h            |    1 +
 include/linux/mm.h                  |   17 ++
 mm/memblock.c                       |   18 ++-
 mm/page_alloc.c                     |  318 ++++++++++++++++++++++++++++++++++-
 7 files changed, 415 insertions(+), 11 deletions(-)

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

[PATCH v4 1/6] x86: get pg_data_t's memory from other node

[Tang Chen]

From: Yasuaki Ishimatsu <isimatu.yasuaki@jp.fujitsu.com>

If system can create movable node which all memory of the
node is allocated as ZONE_MOVABLE, setup_node_data() cannot
allocate memory for the node's pg_data_t.
So, use memblock_alloc_try_nid() instead of memblock_alloc_nid()
to retry when the first allocation fails.

Signed-off-by: Yasuaki Ishimatsu <isimatu.yasuaki@jp.fujitsu.com>
Signed-off-by: Lai Jiangshan <laijs@cn.fujitsu.com>
Signed-off-by: Tang Chen <tangchen@cn.fujitsu.com>
Signed-off-by: Jiang Liu <jiang.liu@huawei.com>
---
 arch/x86/mm/numa.c |    5 ++---
 1 files changed, 2 insertions(+), 3 deletions(-)

diff --git a/arch/x86/mm/numa.c b/arch/x86/mm/numa.c
index 2d125be..db939b6 100644
--- a/arch/x86/mm/numa.c
+++ b/arch/x86/mm/numa.c
@@ -222,10 +222,9 @@ static void __init setup_node_data(int nid, u64 start, u64 end)
 		nd_pa = __pa(nd);
 		remapped = true;
 	} else {
-		nd_pa = memblock_alloc_nid(nd_size, SMP_CACHE_BYTES, nid);
+		nd_pa = memblock_alloc_try_nid(nd_size, SMP_CACHE_BYTES, nid);
 		if (!nd_pa) {
-			pr_err("Cannot find %zu bytes in node %d\n",
-			       nd_size, nid);
+			pr_err("Cannot find %zu bytes in any node\n", nd_size);
 			return;
 		}
 		nd = __va(nd_pa);
-- 
1.7.1

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

[PATCH v4 2/6] page_alloc: add movablecore_map kernel parameter

[Tang Chen]

[What are we doing]
This patch adds a new kernel boot option named movablecore_map to allow
user to specify the range of ZONE_MOVABLE of each node.


[Why do we do this]
The memory used by kernel, such as direct mapping pages, could not be
migrated. As a result, the corresponding memory device could not be
hotplugged. So in order to implement a whole node hotplug feature, we
need to limit the node to only contain movable memory (ZONE_MOVABLE).

This option provide 3 ways to control which memory could be hotplugged.
1. User could specify one or more physical address ranges to inform kernel
   these ranges should be in ZONE_MOVABLE.
   (using movablecore_map=nn[KMG]@ss[KMG])

2. Using the Hot Pluggable bit in flags from SRAT from BIOS, user could just
   leave the configuration work to the kernel.
   (using movablecore_map=acpi)

3. Users with feature known as "hardware memory migration" could migrate
   kernel pages transparently to OS, and these users may just don't use it.

NOTE: We do not use node ids because they could change on each boot.


[How to use]
The option can be used as following:

1. If user want to specify hotpluggable memory ranges by himself, then specify
   as the following:
	movablecore_map=nn[KMG]@ss[KMG]
   In this way, the kernel will check if the ranges are hotpluggable with the
   Hot Pluggable bit in SRAT from ACPI BIOS.
   - If a range is hotpluggable, then from ss to the end of the corresponding
     node will be ZONE_MOVABLE.
   - If a range is not hotpluggable, then the range will be ignored.

2. If user want to leave the configuration work to the kernel, then specify
   as the following:
	movablecore_map=acpi
   In this way, the kernel will get hotplug info from SRAT in ACPI BIOS, and
   auto decide which memory devices could be hotplugged. And all the memory
   on these devices will be in ZONE_MOVABLE.

3. If user didn't specify this option, then the kernel will use all the
   memory on all the nodes evenly. And there is no performance drawback.


This patch also adds functions to parse movablecore_map boot option.
Since the option could be specified more then once, all the maps will
be stored in the global variable movablecore_map.map array.

And also, we keep the array in monotonic increasing order by start_pfn.
And merge all overlapped ranges.

Signed-off-by: Tang Chen <tangchen@cn.fujitsu.com>
Signed-off-by: Lai Jiangshan <laijs@cn.fujitsu.com>
Reviewed-by: Wen Congyang <wency@cn.fujitsu.com>
Tested-by: Lin Feng <linfeng@cn.fujitsu.com>
Tested-by: Gu Zheng <guz.fnst@cn.fujitsu.com>
---
 Documentation/kernel-parameters.txt |   29 +++++
 include/linux/mm.h                  |   17 +++
 mm/page_alloc.c                     |  211 +++++++++++++++++++++++++++++++++++
 3 files changed, 257 insertions(+), 0 deletions(-)

diff --git a/Documentation/kernel-parameters.txt b/Documentation/kernel-parameters.txt
index ea8e5b4..af6cce0 100644
--- a/Documentation/kernel-parameters.txt
+++ b/Documentation/kernel-parameters.txt
@@ -1642,6 +1642,35 @@ bytes respectively. Such letter suffixes can also be entirely omitted.
 			that the amount of memory usable for all allocations
 			is not too small.
 
+	movablecore_map=acpi
+			[KNL,X86,IA-64,PPC] This parameter is similar to
+                        memmap except it specifies the memory map of
+                        ZONE_MOVABLE.
+			This option inform the kernel to use Hot Pluggable bit
+			in flags from SRAT from ACPI BIOS to determine which
+			memory devices could be hotplugged. The corresponding
+			memory ranges will be set as ZONE_MOVABLE.
+
+	movablecore_map=nn[KMG]@ss[KMG]
+			[KNL,X86,IA-64,PPC] The kernel will check if the range
+			is hotpluggable with Hot Pluggable bit in SRAT. If not,
+			the range will be ingored. If yes, do the following:
+			- If more ranges are all within one node, then from
+			  lowest ss to the end of the node will be ZONE_MOVABLE.
+			- If a range is within a node, then from ss to the end
+			  of the node will be ZONE_MOVABLE.
+			- If a range covers two or more nodes, then from ss to
+			  the end of the 1st node will be ZONE_MOVABLE, and all
+			  the rest nodes will only have ZONE_MOVABLE.
+			If memmap is specified at the same time, the
+			movablecore_map will be limited within the memmap
+			areas.
+			If kernelcore or movablecore is also specified,
+			movablecore_map will have higher priority to be
+			satisfied. So the administrator should be careful that
+			the amount of movablecore_map areas are not too large.
+			Otherwise kernel won't have enough memory to start.
+
 	MTD_Partition=	[MTD]
 			Format: <name>,<region-number>,<size>,<offset>
 
diff --git a/include/linux/mm.h b/include/linux/mm.h
index 7f4f906..3ab1bdb 100644
--- a/include/linux/mm.h
+++ b/include/linux/mm.h
@@ -1361,6 +1361,23 @@ extern void free_bootmem_with_active_regions(int nid,
 						unsigned long max_low_pfn);
 extern void sparse_memory_present_with_active_regions(int nid);
 
+#define MOVABLECORE_MAP_MAX MAX_NUMNODES
+struct movablecore_entry {
+	unsigned long start_pfn;    /* start pfn of memory segment */
+	unsigned long end_pfn;      /* end pfn of memory segment */
+};
+
+struct movablecore_map {
+	bool acpi;
+	int nr_map;
+	struct movablecore_entry map[MOVABLECORE_MAP_MAX];
+};
+
+extern int insert_movablecore_map(unsigned long start_pfn,
+				  unsigned long end_pfn);
+extern void remove_movablecore_map(unsigned long start_pfn,
+				   unsigned long end_pfn);
+
 #endif /* CONFIG_HAVE_MEMBLOCK_NODE_MAP */
 
 #if !defined(CONFIG_HAVE_MEMBLOCK_NODE_MAP) && \
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index d037c8b..40d2f4b 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -201,6 +201,12 @@ static unsigned long __meminitdata nr_all_pages;
 static unsigned long __meminitdata dma_reserve;
 
 #ifdef CONFIG_HAVE_MEMBLOCK_NODE_MAP
+/* Movable memory ranges, will also be used by memblock subsystem. */
+struct movablecore_map movablecore_map = {
+	.acpi = false,
+	.nr_map = 0,
+};
+
 static unsigned long __meminitdata arch_zone_lowest_possible_pfn[MAX_NR_ZONES];
 static unsigned long __meminitdata arch_zone_highest_possible_pfn[MAX_NR_ZONES];
 static unsigned long __initdata required_kernelcore;
@@ -5082,6 +5088,211 @@ static int __init cmdline_parse_movablecore(char *p)
 early_param("kernelcore", cmdline_parse_kernelcore);
 early_param("movablecore", cmdline_parse_movablecore);
 
+/**
+ * insert_movablecore_map - Insert a memory range into movablecore_map.map.
+ * @start_pfn: start pfn of the range
+ * @end_pfn: end pfn of the range
+ *
+ * Return 0 on success, -1 if array overflows.
+ *
+ * This function will also merge the overlapped ranges, and sort the array
+ * by start_pfn in monotonic increasing order.
+ */
+int __init insert_movablecore_map(unsigned long start_pfn,
+				  unsigned long end_pfn)
+{
+	int pos, overlap;
+
+	/*
+	 * pos will be at the 1st overlapped range, or the position
+	 * where the element should be inserted.
+	 */
+	for (pos = 0; pos < movablecore_map.nr_map; pos++)
+		if (start_pfn <= movablecore_map.map[pos].end_pfn)
+			break;
+
+	/* If there is no overlapped range, just insert the element. */
+	if (pos == movablecore_map.nr_map ||
+	    end_pfn < movablecore_map.map[pos].start_pfn) {
+		/* No more insertion if array overflows. */
+		if (movablecore_map.nr_map >= ARRAY_SIZE(movablecore_map.map))
+			return -1;
+
+		/*
+		 * If pos is not the end of array, we need to move all
+		 * the rest elements backward.
+		 */
+		if (pos < movablecore_map.nr_map)
+			memmove(&movablecore_map.map[pos+1],
+				&movablecore_map.map[pos],
+				sizeof(struct movablecore_entry) *
+				(movablecore_map.nr_map - pos));
+		movablecore_map.map[pos].start_pfn = start_pfn;
+		movablecore_map.map[pos].end_pfn = end_pfn;
+		movablecore_map.nr_map++;
+		return 0;
+	}
+
+	/* overlap will be at the last overlapped range */
+	for (overlap = pos + 1; overlap < movablecore_map.nr_map; overlap++)
+		if (end_pfn < movablecore_map.map[overlap].start_pfn)
+			break;
+
+	/*
+	 * If there are more ranges overlapped, we need to merge them,
+	 * and move the rest elements forward.
+	 */
+	overlap--;
+	movablecore_map.map[pos].start_pfn = min(start_pfn,
+					movablecore_map.map[pos].start_pfn);
+	movablecore_map.map[pos].end_pfn = max(end_pfn,
+					movablecore_map.map[overlap].end_pfn);
+
+	if (pos != overlap && overlap + 1 != movablecore_map.nr_map)
+		memmove(&movablecore_map.map[pos+1],
+			&movablecore_map.map[overlap+1],
+			sizeof(struct movablecore_entry) *
+			(movablecore_map.nr_map - overlap - 1));
+
+	movablecore_map.nr_map -= overlap - pos;
+	return 0;
+}
+
+/**
+ * remove_movablecore_map - Remove all the parts overlapped with
+ * [start_pfn, end_pfn) from movablecore_map.map.
+ * @start_pfn: start pfn of the range
+ * @end_pfn: end pfn of the range
+ */
+void __init remove_movablecore_map(unsigned long start_pfn,
+				   unsigned long end_pfn)
+{
+	int pos, overlap;
+
+	/*
+	 * If no item in movablecore_map.map, or using SRAT info,
+	 * any removement is forbidden.
+	 */
+	if (!movablecore_map.nr_map || movablecore_map.acpi)
+		return;
+
+	/*
+	 * pos will be at the 1st overlapped range, or the end
+	 * of the array.
+	 */
+	for (pos = 0; pos < movablecore_map.nr_map; pos++)
+		if (start_pfn < movablecore_map.map[pos].end_pfn)
+			break;
+
+	/* If there is no overlapped range, just return. */
+	if (pos == movablecore_map.nr_map ||
+	    end_pfn < movablecore_map.map[pos].start_pfn)
+		return;
+
+	/* overlap will be at the last overlapped range */
+	for (overlap = pos + 1; overlap < movablecore_map.nr_map; overlap++)
+		if (end_pfn <= movablecore_map.map[overlap].start_pfn)
+			break;
+	overlap--;
+
+	/* pos will be at the 1st removed range */
+	if (start_pfn > movablecore_map.map[pos].start_pfn) {
+		movablecore_map.map[pos].end_pfn = start_pfn;
+		pos++;
+	}
+
+	/* overlap will be at the one next to the last removed range */
+	if (end_pfn < movablecore_map.map[overlap].end_pfn)
+		movablecore_map.map[overlap].start_pfn = end_pfn;
+	else
+		overlap++;
+
+	if (overlap != movablecore_map.nr_map)
+		memmove(&movablecore_map.map[pos],
+			&movablecore_map.map[overlap],
+			sizeof(struct movablecore_entry) *
+			(movablecore_map.nr_map - overlap));
+
+	movablecore_map.nr_map -= overlap - pos;
+}
+
+/**
+ * movablecore_map_add_region - Add a memory range into movablecore_map.
+ * @start: physical start address of range
+ * @end: physical end address of range
+ *
+ * This function transform the physical address into pfn, and then add the
+ * range into movablecore_map by calling insert_movablecore_map().
+ */
+static void __init movablecore_map_add_region(u64 start, u64 size)
+{
+	unsigned long start_pfn, end_pfn;
+	int ret;
+
+	/* In case size == 0 or start + size overflows */
+	if (start + size <= start)
+		return;
+
+	start_pfn = PFN_DOWN(start);
+	end_pfn = PFN_UP(start + size);
+	ret = insert_movablecore_map(start_pfn, end_pfn);
+	if (ret)
+		pr_err("movablecore_map: too many entries;"
+			" ignoring [mem %#010llx-%#010llx]\n",
+			(unsigned long long) start,
+			(unsigned long long) (start + size - 1));
+}
+
+/*
+ * movablecore_map=nn[KMG]@ss[KMG] sets the region of memory to be used as
+ * movable memory.
+ * movablecore_map=acpi means using the SRAT info from BISO to specify which
+ * memory is movable.
+ */
+static int __init cmdline_parse_movablecore_map(char *p)
+{
+	char *oldp;
+	u64 start_at, mem_size;
+
+	if (!p)
+		goto err;
+
+	if (!strncmp(p, "acpi", 4))
+		movablecore_map.acpi = true;
+
+	/*
+	 * If user decide to use info from BIOS, all the other user specified
+	 * ranges will be ingored.
+	 */
+	if (movablecore_map.acpi) {
+		if (movablecore_map.nr_map) {
+			memset(movablecore_map.map, 0,
+				sizeof(struct movablecore_entry)
+				* movablecore_map.nr_map);
+			movablecore_map.nr_map = 0;
+		}
+		return 0;
+	}
+
+	oldp = p;
+	mem_size = memparse(p, &p);
+	if (p == oldp)
+		goto err;
+
+	if (*p == '@') {
+		oldp = ++p;
+		start_at = memparse(p, &p);
+		if (p == oldp || *p != '\0')
+			goto err;
+
+		movablecore_map_add_region(start_at, mem_size);
+		return 0;
+	}
+err:
+	return -EINVAL;
+}
+early_param("movablecore_map", cmdline_parse_movablecore_map);
+
 #endif /* CONFIG_HAVE_MEMBLOCK_NODE_MAP */
 
 /**
-- 
1.7.1

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

[PATCH v4 3/6] ACPI: Restructure movablecore_map with memory info from SRAT.

[Tang Chen]

The Hot Plugable bit in SRAT flags specifys if the memory range
could be hotplugged.

If user specified movablecore_map=nn[KMG]@ss[KMG], reset
movablecore_map.map to the intersection of hotpluggable ranges from
SRAT and old movablecore_map.map.
Else if user specified movablecore_map=acpi, just use the hotpluggable
ranges from SRAT.
Otherwise, do nothing. The kernel will use all the memory in all nodes
evenly.

The idea "getting info from SRAT" was from Liu Jiang <jiang.liu@huawei.com>.
And the idea "do more limit for memblock" was from Wu Jianguo <wujianguo@huawei.com>

Signed-off-by: Tang Chen <tangchen@cn.fujitsu.com>
Tested-by: Gu Zheng <guz.fnst@cn.fujitsu.com>
---
 arch/x86/mm/srat.c |   38 +++++++++++++++++++++++++++++++++++---
 1 files changed, 35 insertions(+), 3 deletions(-)

diff --git a/arch/x86/mm/srat.c b/arch/x86/mm/srat.c
index 4ddf497..947a2b5 100644
--- a/arch/x86/mm/srat.c
+++ b/arch/x86/mm/srat.c
@@ -146,7 +146,12 @@ int __init
 acpi_numa_memory_affinity_init(struct acpi_srat_mem_affinity *ma)
 {
 	u64 start, end;
+	u32 hotpluggable;
 	int node, pxm;
+#ifdef CONFIG_HAVE_MEMBLOCK_NODE_MAP
+	int overlap;
+	unsigned long start_pfn, end_pfn;
+#endif /* CONFIG_HAVE_MEMBLOCK_NODE_MAP */
 
 	if (srat_disabled())
 		return -1;
@@ -157,8 +162,10 @@ acpi_numa_memory_affinity_init(struct acpi_srat_mem_affinity *ma)
 	if ((ma->flags & ACPI_SRAT_MEM_ENABLED) == 0)
 		return -1;
 
-	if ((ma->flags & ACPI_SRAT_MEM_HOT_PLUGGABLE) && !save_add_info())
+	hotpluggable = ma->flags & ACPI_SRAT_MEM_HOT_PLUGGABLE;
+	if (hotpluggable && !save_add_info())
 		return -1;
+
 	start = ma->base_address;
 	end = start + ma->length;
 	pxm = ma->proximity_domain;
@@ -178,9 +185,34 @@ acpi_numa_memory_affinity_init(struct acpi_srat_mem_affinity *ma)
 
 	node_set(node, numa_nodes_parsed);
 
-	printk(KERN_INFO "SRAT: Node %u PXM %u [mem %#010Lx-%#010Lx]\n",
+	printk(KERN_INFO "SRAT: Node %u PXM %u [mem %#010Lx-%#010Lx] %s\n",
 	       node, pxm,
-	       (unsigned long long) start, (unsigned long long) end - 1);
+	       (unsigned long long) start, (unsigned long long) end - 1,
+	       hotpluggable ? "Hot Pluggable": "");
+
+#ifdef CONFIG_HAVE_MEMBLOCK_NODE_MAP
+	start_pfn = PFN_DOWN(start);
+	end_pfn = PFN_UP(end);
+
+	if (!hotpluggable) {
+		/* Clear the range overlapped in movablecore_map.map */
+		remove_movablecore_map(start_pfn, end_pfn);
+		goto out;
+	}
+
+	/* If not using SRAT, don't modify user configuration. */
+	if (!movablecore_map.acpi)
+		goto out;
+
+	/* If user chose to use SRAT info, insert the range anyway. */
+	if (insert_movablecore_map(start_pfn, end_pfn))
+		pr_err("movablecore_map: too many entries;"
+			" ignoring [mem %#010llx-%#010llx]\n",
+			(unsigned long long) start,
+			(unsigned long long) (end - 1));
+
+out:
+#endif /* CONFIG_HAVE_MEMBLOCK_NODE_MAP */
 	return 0;
 }
 
-- 
1.7.1

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

[PATCH v4 3/6] ACPI: Restructure movablecore_map with memory info from SRAT.

[Tang Chen]

The Hot Plugable bit in SRAT flags specifys if the memory range
could be hotplugged.

If user specified movablecore_map=nn[KMG]@ss[KMG], reset
movablecore_map.map to the intersection of hotpluggable ranges from
SRAT and old movablecore_map.map.
Else if user specified movablecore_map=acpi, just use the hotpluggable
ranges from SRAT.
Otherwise, do nothing. The kernel will use all the memory in all nodes
evenly.

The idea "getting info from SRAT" was from Liu Jiang <jiang.liu@huawei.com>.
And the idea "do more limit for memblock" was from Wu Jianguo <wujianguo@huawei.com>

Signed-off-by: Tang Chen <tangchen@cn.fujitsu.com>
Tested-by: Gu Zheng <guz.fnst@cn.fujitsu.com>
---
 arch/x86/mm/srat.c |   55 +++++++++++++++++++++++++++++++++++++++++++++++++--
 1 files changed, 52 insertions(+), 3 deletions(-)

diff --git a/arch/x86/mm/srat.c b/arch/x86/mm/srat.c
index 4ddf497..a8856d2 100644
--- a/arch/x86/mm/srat.c
+++ b/arch/x86/mm/srat.c
@@ -146,7 +146,12 @@ int __init
 acpi_numa_memory_affinity_init(struct acpi_srat_mem_affinity *ma)
 {
 	u64 start, end;
+	u32 hotpluggable;
 	int node, pxm;
+#ifdef CONFIG_HAVE_MEMBLOCK_NODE_MAP
+	int overlap;
+	unsigned long start_pfn, end_pfn;
+#endif /* CONFIG_HAVE_MEMBLOCK_NODE_MAP */
 
 	if (srat_disabled())
 		return -1;
@@ -157,8 +162,10 @@ acpi_numa_memory_affinity_init(struct acpi_srat_mem_affinity *ma)
 	if ((ma->flags & ACPI_SRAT_MEM_ENABLED) == 0)
 		return -1;
 
-	if ((ma->flags & ACPI_SRAT_MEM_HOT_PLUGGABLE) && !save_add_info())
+	hotpluggable = ma->flags & ACPI_SRAT_MEM_HOT_PLUGGABLE;
+	if (hotpluggable && !save_add_info())
 		return -1;
+
 	start = ma->base_address;
 	end = start + ma->length;
 	pxm = ma->proximity_domain;
@@ -178,9 +185,51 @@ acpi_numa_memory_affinity_init(struct acpi_srat_mem_affinity *ma)
 
 	node_set(node, numa_nodes_parsed);
 
-	printk(KERN_INFO "SRAT: Node %u PXM %u [mem %#010Lx-%#010Lx]\n",
+	printk(KERN_INFO "SRAT: Node %u PXM %u [mem %#010Lx-%#010Lx] %s\n",
 	       node, pxm,
-	       (unsigned long long) start, (unsigned long long) end - 1);
+	       (unsigned long long) start, (unsigned long long) end - 1,
+	       hotpluggable ? "Hot Pluggable": "");
+
+#ifdef CONFIG_HAVE_MEMBLOCK_NODE_MAP
+	start_pfn = PFN_DOWN(start);
+	end_pfn = PFN_UP(end);
+
+	if (!hotpluggable) {
+		/* Clear the range overlapped in movablecore_map.map */
+		remove_movablecore_map(start_pfn, end_pfn);
+		goto out;
+	}
+
+	if (!movablecore_map.acpi) {
+		for (overlap = 0; overlap < movablecore_map.nr_map; overlap++) {
+			if (start_pfn < movablecore_map.map[overlap].end_pfn)
+				break;
+		}
+
+		/*
+		 * If there is no overlapped range, or the end of the overlapped
+		 * range is higher than end_pfn, then insert nothing.
+		 */
+		if (end_pfn <= movablecore_map.map[overlap].end_pfn)
+			goto out;
+
+		/*
+		 * Otherwise, insert the rest of this range to prevent memblock
+		 * from allocating memory in it.
+		 */
+		start_pfn = movablecore_map.map[overlap].end_pfn;
+		start = start_pfn >> PAGE_SHIFT;
+	}
+
+	/* If user chose to use SRAT info, insert the range anyway. */
+	if (insert_movablecore_map(start_pfn, end_pfn))
+		pr_err("movablecore_map: too many entries;"
+			" ignoring [mem %#010llx-%#010llx]\n",
+			(unsigned long long) start,
+			(unsigned long long) (end - 1));
+
+out:
+#endif /* CONFIG_HAVE_MEMBLOCK_NODE_MAP */
 	return 0;
 }
 
-- 
1.7.1

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH v4 3/6] ACPI: Restructure movablecore_map with memory info from SRAT.

[Yasuaki Ishimatsu]

Hi Tang,

I don't think it can work well.
The patch gets memory range of hotpluggable memory by
acpi_numa_memory_affinity_init(). But it too late.
For example, if we use log_buf_len boot options, memblock allocator
runs before getting SRAT information. In this case, this movablecore_map
boot option does not work well.

Thanks,
Yasuaki Ishimatsu

2012/12/19 18:11, Tang Chen wrote:
> The Hot Plugable bit in SRAT flags specifys if the memory range
> could be hotplugged.
> 
> If user specified movablecore_map=nn[KMG]@ss[KMG], reset
> movablecore_map.map to the intersection of hotpluggable ranges from
> SRAT and old movablecore_map.map.
> Else if user specified movablecore_map=acpi, just use the hotpluggable
> ranges from SRAT.
> Otherwise, do nothing. The kernel will use all the memory in all nodes
> evenly.
> 
> The idea "getting info from SRAT" was from Liu Jiang <jiang.liu@huawei.com>.
> And the idea "do more limit for memblock" was from Wu Jianguo <wujianguo@huawei.com>
> 
> Signed-off-by: Tang Chen <tangchen@cn.fujitsu.com>
> Tested-by: Gu Zheng <guz.fnst@cn.fujitsu.com>
> ---
>   arch/x86/mm/srat.c |   55 +++++++++++++++++++++++++++++++++++++++++++++++++--
>   1 files changed, 52 insertions(+), 3 deletions(-)
> 
> diff --git a/arch/x86/mm/srat.c b/arch/x86/mm/srat.c
> index 4ddf497..a8856d2 100644
> --- a/arch/x86/mm/srat.c
> +++ b/arch/x86/mm/srat.c
> @@ -146,7 +146,12 @@ int __init
>   acpi_numa_memory_affinity_init(struct acpi_srat_mem_affinity *ma)
>   {
>   	u64 start, end;
> +	u32 hotpluggable;
>   	int node, pxm;
> +#ifdef CONFIG_HAVE_MEMBLOCK_NODE_MAP
> +	int overlap;
> +	unsigned long start_pfn, end_pfn;
> +#endif /* CONFIG_HAVE_MEMBLOCK_NODE_MAP */
>   
>   	if (srat_disabled())
>   		return -1;
> @@ -157,8 +162,10 @@ acpi_numa_memory_affinity_init(struct acpi_srat_mem_affinity *ma)
>   	if ((ma->flags & ACPI_SRAT_MEM_ENABLED) == 0)
>   		return -1;
>   
> -	if ((ma->flags & ACPI_SRAT_MEM_HOT_PLUGGABLE) && !save_add_info())
> +	hotpluggable = ma->flags & ACPI_SRAT_MEM_HOT_PLUGGABLE;
> +	if (hotpluggable && !save_add_info())
>   		return -1;
> +
>   	start = ma->base_address;
>   	end = start + ma->length;
>   	pxm = ma->proximity_domain;
> @@ -178,9 +185,51 @@ acpi_numa_memory_affinity_init(struct acpi_srat_mem_affinity *ma)
>   
>   	node_set(node, numa_nodes_parsed);
>   
> -	printk(KERN_INFO "SRAT: Node %u PXM %u [mem %#010Lx-%#010Lx]\n",
> +	printk(KERN_INFO "SRAT: Node %u PXM %u [mem %#010Lx-%#010Lx] %s\n",
>   	       node, pxm,
> -	       (unsigned long long) start, (unsigned long long) end - 1);
> +	       (unsigned long long) start, (unsigned long long) end - 1,
> +	       hotpluggable ? "Hot Pluggable": "");
> +
> +#ifdef CONFIG_HAVE_MEMBLOCK_NODE_MAP
> +	start_pfn = PFN_DOWN(start);
> +	end_pfn = PFN_UP(end);
> +
> +	if (!hotpluggable) {
> +		/* Clear the range overlapped in movablecore_map.map */
> +		remove_movablecore_map(start_pfn, end_pfn);
> +		goto out;
> +	}
> +
> +	if (!movablecore_map.acpi) {
> +		for (overlap = 0; overlap < movablecore_map.nr_map; overlap++) {
> +			if (start_pfn < movablecore_map.map[overlap].end_pfn)
> +				break;
> +		}
> +
> +		/*
> +		 * If there is no overlapped range, or the end of the overlapped
> +		 * range is higher than end_pfn, then insert nothing.
> +		 */
> +		if (end_pfn <= movablecore_map.map[overlap].end_pfn)
> +			goto out;
> +
> +		/*
> +		 * Otherwise, insert the rest of this range to prevent memblock
> +		 * from allocating memory in it.
> +		 */
> +		start_pfn = movablecore_map.map[overlap].end_pfn;
> +		start = start_pfn >> PAGE_SHIFT;
> +	}
> +
> +	/* If user chose to use SRAT info, insert the range anyway. */
> +	if (insert_movablecore_map(start_pfn, end_pfn))
> +		pr_err("movablecore_map: too many entries;"
> +			" ignoring [mem %#010llx-%#010llx]\n",
> +			(unsigned long long) start,
> +			(unsigned long long) (end - 1));
> +
> +out:
> +#endif /* CONFIG_HAVE_MEMBLOCK_NODE_MAP */
>   	return 0;
>   }
>   
> 


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH v4 3/6] ACPI: Restructure movablecore_map with memory info from SRAT.

[Tang Chen]

On 12/26/2012 02:53 PM, Yasuaki Ishimatsu wrote:
> Hi Tang,
> 
> I don't think it can work well.
> The patch gets memory range of hotpluggable memory by
> acpi_numa_memory_affinity_init(). But it too late.
> For example, if we use log_buf_len boot options, memblock allocator
> runs before getting SRAT information. In this case, this movablecore_map
> boot option does not work well.
> 

Hi Ishimatsu-san,

Yes, you are right. After finish_e820_parsing() in setup_arch(),
memblock allocator could start to work. So we need to reserve
the hotpluggable memory before it. But SRAT parsing is far behind.
So for now, I didn't work out a suitable way to do this.

I think we need to move ACPI table parsing logic before memblock is
ready. But we need to solve how to allocate memory for ACPI table
parsing logic.

I'll try to figure out a way and start a discussion soon.

Thanks. :)

> Thanks,
> Yasuaki Ishimatsu
> 
> 2012/12/19 18:11, Tang Chen wrote:
>> The Hot Plugable bit in SRAT flags specifys if the memory range
>> could be hotplugged.
>>
>> If user specified movablecore_map=nn[KMG]@ss[KMG], reset
>> movablecore_map.map to the intersection of hotpluggable ranges from
>> SRAT and old movablecore_map.map.
>> Else if user specified movablecore_map=acpi, just use the hotpluggable
>> ranges from SRAT.
>> Otherwise, do nothing. The kernel will use all the memory in all nodes
>> evenly.
>>
>> The idea "getting info from SRAT" was from Liu Jiang<jiang.liu@huawei.com>.
>> And the idea "do more limit for memblock" was from Wu Jianguo<wujianguo@huawei.com>
>>
>> Signed-off-by: Tang Chen<tangchen@cn.fujitsu.com>
>> Tested-by: Gu Zheng<guz.fnst@cn.fujitsu.com>
>> ---
>>    arch/x86/mm/srat.c |   55 +++++++++++++++++++++++++++++++++++++++++++++++++--
>>    1 files changed, 52 insertions(+), 3 deletions(-)
>>
>> diff --git a/arch/x86/mm/srat.c b/arch/x86/mm/srat.c
>> index 4ddf497..a8856d2 100644
>> --- a/arch/x86/mm/srat.c
>> +++ b/arch/x86/mm/srat.c
>> @@ -146,7 +146,12 @@ int __init
>>    acpi_numa_memory_affinity_init(struct acpi_srat_mem_affinity *ma)
>>    {
>>    	u64 start, end;
>> +	u32 hotpluggable;
>>    	int node, pxm;
>> +#ifdef CONFIG_HAVE_MEMBLOCK_NODE_MAP
>> +	int overlap;
>> +	unsigned long start_pfn, end_pfn;
>> +#endif /* CONFIG_HAVE_MEMBLOCK_NODE_MAP */
>>
>>    	if (srat_disabled())
>>    		return -1;
>> @@ -157,8 +162,10 @@ acpi_numa_memory_affinity_init(struct acpi_srat_mem_affinity *ma)
>>    	if ((ma->flags&  ACPI_SRAT_MEM_ENABLED) == 0)
>>    		return -1;
>>
>> -	if ((ma->flags&  ACPI_SRAT_MEM_HOT_PLUGGABLE)&&  !save_add_info())
>> +	hotpluggable = ma->flags&  ACPI_SRAT_MEM_HOT_PLUGGABLE;
>> +	if (hotpluggable&&  !save_add_info())
>>    		return -1;
>> +
>>    	start = ma->base_address;
>>    	end = start + ma->length;
>>    	pxm = ma->proximity_domain;
>> @@ -178,9 +185,51 @@ acpi_numa_memory_affinity_init(struct acpi_srat_mem_affinity *ma)
>>
>>    	node_set(node, numa_nodes_parsed);
>>
>> -	printk(KERN_INFO "SRAT: Node %u PXM %u [mem %#010Lx-%#010Lx]\n",
>> +	printk(KERN_INFO "SRAT: Node %u PXM %u [mem %#010Lx-%#010Lx] %s\n",
>>    	       node, pxm,
>> -	       (unsigned long long) start, (unsigned long long) end - 1);
>> +	       (unsigned long long) start, (unsigned long long) end - 1,
>> +	       hotpluggable ? "Hot Pluggable": "");
>> +
>> +#ifdef CONFIG_HAVE_MEMBLOCK_NODE_MAP
>> +	start_pfn = PFN_DOWN(start);
>> +	end_pfn = PFN_UP(end);
>> +
>> +	if (!hotpluggable) {
>> +		/* Clear the range overlapped in movablecore_map.map */
>> +		remove_movablecore_map(start_pfn, end_pfn);
>> +		goto out;
>> +	}
>> +
>> +	if (!movablecore_map.acpi) {
>> +		for (overlap = 0; overlap<  movablecore_map.nr_map; overlap++) {
>> +			if (start_pfn<  movablecore_map.map[overlap].end_pfn)
>> +				break;
>> +		}
>> +
>> +		/*
>> +		 * If there is no overlapped range, or the end of the overlapped
>> +		 * range is higher than end_pfn, then insert nothing.
>> +		 */
>> +		if (end_pfn<= movablecore_map.map[overlap].end_pfn)
>> +			goto out;
>> +
>> +		/*
>> +		 * Otherwise, insert the rest of this range to prevent memblock
>> +		 * from allocating memory in it.
>> +		 */
>> +		start_pfn = movablecore_map.map[overlap].end_pfn;
>> +		start = start_pfn>>  PAGE_SHIFT;
>> +	}
>> +
>> +	/* If user chose to use SRAT info, insert the range anyway. */
>> +	if (insert_movablecore_map(start_pfn, end_pfn))
>> +		pr_err("movablecore_map: too many entries;"
>> +			" ignoring [mem %#010llx-%#010llx]\n",
>> +			(unsigned long long) start,
>> +			(unsigned long long) (end - 1));
>> +
>> +out:
>> +#endif /* CONFIG_HAVE_MEMBLOCK_NODE_MAP */
>>    	return 0;
>>    }
>>
>>
> 
> 
> 

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH v4 3/6] ACPI: Restructure movablecore_map with memory info from SRAT.

[Yasuaki Ishimatsu]

Hi Tang,

2012/12/26 16:39, Tang Chen wrote:
> On 12/26/2012 02:53 PM, Yasuaki Ishimatsu wrote:
>> Hi Tang,
>>
>> I don't think it can work well.
>> The patch gets memory range of hotpluggable memory by
>> acpi_numa_memory_affinity_init(). But it too late.
>> For example, if we use log_buf_len boot options, memblock allocator
>> runs before getting SRAT information. In this case, this movablecore_map
>> boot option does not work well.
>>
> 
> Hi Ishimatsu-san,
> 
> Yes, you are right. After finish_e820_parsing() in setup_arch(),
> memblock allocator could start to work. So we need to reserve
> the hotpluggable memory before it. But SRAT parsing is far behind.
> So for now, I didn't work out a suitable way to do this.
> 
> I think we need to move ACPI table parsing logic before memblock is
> ready. But we need to solve how to allocate memory for ACPI table
> parsing logic.
> 
> I'll try to figure out a way and start a discussion soon.

There are two discussion parts:
 - interface
   Interface part is how to specify memory range.
 - core
   Core part is how to allocate movable memory.

I think using SRAT information is long term development. So at first,
we should develop boot option. In this case, core part should be
implemented to usable from both ways, boot option  and firmware information.

Fortunately, I think that current implementation of core part can deal
with both ways.

Thanks,
Yasuaki Ishimatsu




> 
> Thanks. :)
> 
>> Thanks,
>> Yasuaki Ishimatsu
>>
>> 2012/12/19 18:11, Tang Chen wrote:
>>> The Hot Plugable bit in SRAT flags specifys if the memory range
>>> could be hotplugged.
>>>
>>> If user specified movablecore_map=nn[KMG]@ss[KMG], reset
>>> movablecore_map.map to the intersection of hotpluggable ranges from
>>> SRAT and old movablecore_map.map.
>>> Else if user specified movablecore_map=acpi, just use the hotpluggable
>>> ranges from SRAT.
>>> Otherwise, do nothing. The kernel will use all the memory in all nodes
>>> evenly.
>>>
>>> The idea "getting info from SRAT" was from Liu Jiang<jiang.liu@huawei.com>.
>>> And the idea "do more limit for memblock" was from Wu Jianguo<wujianguo@huawei.com>
>>>
>>> Signed-off-by: Tang Chen<tangchen@cn.fujitsu.com>
>>> Tested-by: Gu Zheng<guz.fnst@cn.fujitsu.com>
>>> ---
>>>     arch/x86/mm/srat.c |   55 +++++++++++++++++++++++++++++++++++++++++++++++++--
>>>     1 files changed, 52 insertions(+), 3 deletions(-)
>>>
>>> diff --git a/arch/x86/mm/srat.c b/arch/x86/mm/srat.c
>>> index 4ddf497..a8856d2 100644
>>> --- a/arch/x86/mm/srat.c
>>> +++ b/arch/x86/mm/srat.c
>>> @@ -146,7 +146,12 @@ int __init
>>>     acpi_numa_memory_affinity_init(struct acpi_srat_mem_affinity *ma)
>>>     {
>>>     	u64 start, end;
>>> +	u32 hotpluggable;
>>>     	int node, pxm;
>>> +#ifdef CONFIG_HAVE_MEMBLOCK_NODE_MAP
>>> +	int overlap;
>>> +	unsigned long start_pfn, end_pfn;
>>> +#endif /* CONFIG_HAVE_MEMBLOCK_NODE_MAP */
>>>
>>>     	if (srat_disabled())
>>>     		return -1;
>>> @@ -157,8 +162,10 @@ acpi_numa_memory_affinity_init(struct acpi_srat_mem_affinity *ma)
>>>     	if ((ma->flags&  ACPI_SRAT_MEM_ENABLED) == 0)
>>>     		return -1;
>>>
>>> -	if ((ma->flags&  ACPI_SRAT_MEM_HOT_PLUGGABLE)&&  !save_add_info())
>>> +	hotpluggable = ma->flags&  ACPI_SRAT_MEM_HOT_PLUGGABLE;
>>> +	if (hotpluggable&&  !save_add_info())
>>>     		return -1;
>>> +
>>>     	start = ma->base_address;
>>>     	end = start + ma->length;
>>>     	pxm = ma->proximity_domain;
>>> @@ -178,9 +185,51 @@ acpi_numa_memory_affinity_init(struct acpi_srat_mem_affinity *ma)
>>>
>>>     	node_set(node, numa_nodes_parsed);
>>>
>>> -	printk(KERN_INFO "SRAT: Node %u PXM %u [mem %#010Lx-%#010Lx]\n",
>>> +	printk(KERN_INFO "SRAT: Node %u PXM %u [mem %#010Lx-%#010Lx] %s\n",
>>>     	       node, pxm,
>>> -	       (unsigned long long) start, (unsigned long long) end - 1);
>>> +	       (unsigned long long) start, (unsigned long long) end - 1,
>>> +	       hotpluggable ? "Hot Pluggable": "");
>>> +
>>> +#ifdef CONFIG_HAVE_MEMBLOCK_NODE_MAP
>>> +	start_pfn = PFN_DOWN(start);
>>> +	end_pfn = PFN_UP(end);
>>> +
>>> +	if (!hotpluggable) {
>>> +		/* Clear the range overlapped in movablecore_map.map */
>>> +		remove_movablecore_map(start_pfn, end_pfn);
>>> +		goto out;
>>> +	}
>>> +
>>> +	if (!movablecore_map.acpi) {
>>> +		for (overlap = 0; overlap<  movablecore_map.nr_map; overlap++) {
>>> +			if (start_pfn<  movablecore_map.map[overlap].end_pfn)
>>> +				break;
>>> +		}
>>> +
>>> +		/*
>>> +		 * If there is no overlapped range, or the end of the overlapped
>>> +		 * range is higher than end_pfn, then insert nothing.
>>> +		 */
>>> +		if (end_pfn<= movablecore_map.map[overlap].end_pfn)
>>> +			goto out;
>>> +
>>> +		/*
>>> +		 * Otherwise, insert the rest of this range to prevent memblock
>>> +		 * from allocating memory in it.
>>> +		 */
>>> +		start_pfn = movablecore_map.map[overlap].end_pfn;
>>> +		start = start_pfn>>  PAGE_SHIFT;
>>> +	}
>>> +
>>> +	/* If user chose to use SRAT info, insert the range anyway. */
>>> +	if (insert_movablecore_map(start_pfn, end_pfn))
>>> +		pr_err("movablecore_map: too many entries;"
>>> +			" ignoring [mem %#010llx-%#010llx]\n",
>>> +			(unsigned long long) start,
>>> +			(unsigned long long) (end - 1));
>>> +
>>> +out:
>>> +#endif /* CONFIG_HAVE_MEMBLOCK_NODE_MAP */
>>>     	return 0;
>>>     }
>>>
>>>
>>
>>
>>
> 


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH v4 3/6] ACPI: Restructure movablecore_map with memory info from SRAT.

[Tang Chen]

On 12/19/2012 04:15 PM, Tang Chen wrote:
> The Hot Plugable bit in SRAT flags specifys if the memory range
> could be hotplugged.
>
> If user specified movablecore_map=nn[KMG]@ss[KMG], reset
> movablecore_map.map to the intersection of hotpluggable ranges from
> SRAT and old movablecore_map.map.
> Else if user specified movablecore_map=acpi, just use the hotpluggable
> ranges from SRAT.
> Otherwise, do nothing. The kernel will use all the memory in all nodes
> evenly.
>
> The idea "getting info from SRAT" was from Liu Jiang<jiang.liu@huawei.com>.
> And the idea "do more limit for memblock" was from Wu Jianguo<wujianguo@huawei.com>
>
> Signed-off-by: Tang Chen<tangchen@cn.fujitsu.com>
> Tested-by: Gu Zheng<guz.fnst@cn.fujitsu.com>
> ---
>   arch/x86/mm/srat.c |   38 +++++++++++++++++++++++++++++++++++---
>   1 files changed, 35 insertions(+), 3 deletions(-)
>
> diff --git a/arch/x86/mm/srat.c b/arch/x86/mm/srat.c
> index 4ddf497..947a2b5 100644
> --- a/arch/x86/mm/srat.c
> +++ b/arch/x86/mm/srat.c
> @@ -146,7 +146,12 @@ int __init
>   acpi_numa_memory_affinity_init(struct acpi_srat_mem_affinity *ma)
>   {
>   	u64 start, end;
> +	u32 hotpluggable;
>   	int node, pxm;
> +#ifdef CONFIG_HAVE_MEMBLOCK_NODE_MAP
> +	int overlap;
> +	unsigned long start_pfn, end_pfn;
> +#endif /* CONFIG_HAVE_MEMBLOCK_NODE_MAP */
>
>   	if (srat_disabled())
>   		return -1;
> @@ -157,8 +162,10 @@ acpi_numa_memory_affinity_init(struct acpi_srat_mem_affinity *ma)
>   	if ((ma->flags&  ACPI_SRAT_MEM_ENABLED) == 0)
>   		return -1;
>
> -	if ((ma->flags&  ACPI_SRAT_MEM_HOT_PLUGGABLE)&&  !save_add_info())
> +	hotpluggable = ma->flags&  ACPI_SRAT_MEM_HOT_PLUGGABLE;
> +	if (hotpluggable&&  !save_add_info())
>   		return -1;
> +
>   	start = ma->base_address;
>   	end = start + ma->length;
>   	pxm = ma->proximity_domain;
> @@ -178,9 +185,34 @@ acpi_numa_memory_affinity_init(struct acpi_srat_mem_affinity *ma)
>
>   	node_set(node, numa_nodes_parsed);
>
> -	printk(KERN_INFO "SRAT: Node %u PXM %u [mem %#010Lx-%#010Lx]\n",
> +	printk(KERN_INFO "SRAT: Node %u PXM %u [mem %#010Lx-%#010Lx] %s\n",
>   	       node, pxm,
> -	       (unsigned long long) start, (unsigned long long) end - 1);
> +	       (unsigned long long) start, (unsigned long long) end - 1,
> +	       hotpluggable ? "Hot Pluggable": "");
> +
> +#ifdef CONFIG_HAVE_MEMBLOCK_NODE_MAP
> +	start_pfn = PFN_DOWN(start);
> +	end_pfn = PFN_UP(end);
> +
> +	if (!hotpluggable) {
> +		/* Clear the range overlapped in movablecore_map.map */
> +		remove_movablecore_map(start_pfn, end_pfn);
> +		goto out;
> +	}
> +
> +	/* If not using SRAT, don't modify user configuration. */
> +	if (!movablecore_map.acpi)
> +		goto out;

Here, forgot to do some check. Please see the new resent one.

Thanks. :)

> +
> +	/* If user chose to use SRAT info, insert the range anyway. */
> +	if (insert_movablecore_map(start_pfn, end_pfn))
> +		pr_err("movablecore_map: too many entries;"
> +			" ignoring [mem %#010llx-%#010llx]\n",
> +			(unsigned long long) start,
> +			(unsigned long long) (end - 1));
> +
> +out:
> +#endif /* CONFIG_HAVE_MEMBLOCK_NODE_MAP */
>   	return 0;
>   }
>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

[PATCH v4 4/6] page_alloc: Introduce zone_movable_limit[] to keep movable limit for nodes

[Tang Chen]

This patch introduces a new array zone_movable_limit[] to store the
ZONE_MOVABLE limit from movablecore_map boot option for all nodes.
The function sanitize_zone_movable_limit() will find out to which
node the ranges in movable_map.map[] belongs, and calculates the
low boundary of ZONE_MOVABLE for each node.

Signed-off-by: Tang Chen <tangchen@cn.fujitsu.com>
Signed-off-by: Liu Jiang <jiang.liu@huawei.com>
Reviewed-by: Wen Congyang <wency@cn.fujitsu.com>
Reviewed-by: Lai Jiangshan <laijs@cn.fujitsu.com>
Tested-by: Lin Feng <linfeng@cn.fujitsu.com>
---
 mm/page_alloc.c |   79 ++++++++++++++++++++++++++++++++++++++++++++++++++++++-
 1 files changed, 78 insertions(+), 1 deletions(-)

diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 40d2f4b..128d28a 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -212,6 +212,7 @@ static unsigned long __meminitdata arch_zone_highest_possible_pfn[MAX_NR_ZONES];
 static unsigned long __initdata required_kernelcore;
 static unsigned long __initdata required_movablecore;
 static unsigned long __meminitdata zone_movable_pfn[MAX_NUMNODES];
+static unsigned long __meminitdata zone_movable_limit[MAX_NUMNODES];
 
 /* movable_zone is the "real" zone pages in ZONE_MOVABLE are taken from */
 int movable_zone;
@@ -4385,6 +4386,77 @@ static unsigned long __meminit zone_absent_pages_in_node(int nid,
 	return __absent_pages_in_range(nid, zone_start_pfn, zone_end_pfn);
 }
 
+/**
+ * sanitize_zone_movable_limit - Sanitize the zone_movable_limit array.
+ *
+ * zone_movable_limit is initialized as 0. This function will try to get
+ * the first ZONE_MOVABLE pfn of each node from movablecore_map, and
+ * assigne them to zone_movable_limit.
+ * zone_movable_limit[nid] == 0 means no limit for the node.
+ *
+ * Note: Each range is represented as [start_pfn, end_pfn)
+ */
+static void __meminit sanitize_zone_movable_limit(void)
+{
+	int map_pos = 0, i, nid;
+	unsigned long start_pfn, end_pfn;
+
+	if (!movablecore_map.nr_map)
+		return;
+
+	/* Iterate all ranges from minimum to maximum */
+	for_each_mem_pfn_range(i, MAX_NUMNODES, &start_pfn, &end_pfn, &nid) {
+		/*
+		 * If we have found lowest pfn of ZONE_MOVABLE of the node
+		 * specified by user, just go on to check next range.
+		 */
+		if (zone_movable_limit[nid])
+			continue;
+
+#ifdef CONFIG_ZONE_DMA
+		/* Skip DMA memory. */
+		if (start_pfn < arch_zone_highest_possible_pfn[ZONE_DMA])
+			start_pfn = arch_zone_highest_possible_pfn[ZONE_DMA];
+#endif
+
+#ifdef CONFIG_ZONE_DMA32
+		/* Skip DMA32 memory. */
+		if (start_pfn < arch_zone_highest_possible_pfn[ZONE_DMA32])
+			start_pfn = arch_zone_highest_possible_pfn[ZONE_DMA32];
+#endif
+
+#ifdef CONFIG_HIGHMEM
+		/* Skip lowmem if ZONE_MOVABLE is highmem. */
+		if (zone_movable_is_highmem() &&
+		    start_pfn < arch_zone_lowest_possible_pfn[ZONE_HIGHMEM])
+			start_pfn = arch_zone_lowest_possible_pfn[ZONE_HIGHMEM];
+#endif
+
+		if (start_pfn >= end_pfn)
+			continue;
+
+		while (map_pos < movablecore_map.nr_map) {
+			if (end_pfn <= movablecore_map.map[map_pos].start_pfn)
+				break;
+
+			if (start_pfn >= movablecore_map.map[map_pos].end_pfn) {
+				map_pos++;
+				continue;
+			}
+
+			/*
+			 * The start_pfn of ZONE_MOVABLE is either the minimum
+			 * pfn specified by movablecore_map, or 0, which means
+			 * the node has no ZONE_MOVABLE.
+			 */
+			zone_movable_limit[nid] = max(start_pfn,
+					movablecore_map.map[map_pos].start_pfn);
+
+			break;
+		}
+	}
+}
+
 #else /* CONFIG_HAVE_MEMBLOCK_NODE_MAP */
 static inline unsigned long __meminit zone_spanned_pages_in_node(int nid,
 					unsigned long zone_type,
@@ -4403,6 +4475,10 @@ static inline unsigned long __meminit zone_absent_pages_in_node(int nid,
 	return zholes_size[zone_type];
 }
 
+static void __meminit sanitize_zone_movable_limit(void)
+{
+}
+
 #endif /* CONFIG_HAVE_MEMBLOCK_NODE_MAP */
 
 static void __meminit calculate_node_totalpages(struct pglist_data *pgdat,
@@ -4846,7 +4922,6 @@ static void __init find_zone_movable_pfns_for_nodes(void)
 		goto out;
 
 	/* usable_startpfn is the lowest possible pfn ZONE_MOVABLE can be at */
-	find_usable_zone_for_movable();
 	usable_startpfn = arch_zone_lowest_possible_pfn[movable_zone];
 
 restart:
@@ -5005,6 +5080,8 @@ void __init free_area_init_nodes(unsigned long *max_zone_pfn)
 
 	/* Find the PFNs that ZONE_MOVABLE begins at in each node */
 	memset(zone_movable_pfn, 0, sizeof(zone_movable_pfn));
+	find_usable_zone_for_movable();
+	sanitize_zone_movable_limit();
 	find_zone_movable_pfns_for_nodes();
 
 	/* Print out the zone ranges */
-- 
1.7.1

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

[PATCH v4 5/6] page_alloc: Make movablecore_map has higher priority

[Tang Chen]

If kernelcore or movablecore is specified at the same time
with movablecore_map, movablecore_map will have higher
priority to be satisfied.
This patch will make find_zone_movable_pfns_for_nodes()
calculate zone_movable_pfn[] with the limit from
zone_movable_limit[].

Signed-off-by: Tang Chen <tangchen@cn.fujitsu.com>
Reviewed-by: Wen Congyang <wency@cn.fujitsu.com>
Reviewed-by: Lai Jiangshan <laijs@cn.fujitsu.com>
Reviewed-by: Wu Jianguo <wujianguo@huawei.com>
Tested-by: Lin Feng <linfeng@cn.fujitsu.com>
---
 mm/page_alloc.c |   28 +++++++++++++++++++++++++---
 1 files changed, 25 insertions(+), 3 deletions(-)

diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 128d28a..6ab1ffe 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -4917,9 +4917,17 @@ static void __init find_zone_movable_pfns_for_nodes(void)
 		required_kernelcore = max(required_kernelcore, corepages);
 	}
 
-	/* If kernelcore was not specified, there is no ZONE_MOVABLE */
-	if (!required_kernelcore)
+	/*
+	 * If neither kernelcore/movablecore nor movablecore_map is specified,
+	 * there is no ZONE_MOVABLE. But if movablecore_map is specified, the
+	 * start pfn of ZONE_MOVABLE has been stored in zone_movable_limit[].
+	 */
+	if (!required_kernelcore) {
+		if (movablecore_map.nr_map)
+			memcpy(zone_movable_pfn, zone_movable_limit,
+				sizeof(zone_movable_pfn));
 		goto out;
+	}
 
 	/* usable_startpfn is the lowest possible pfn ZONE_MOVABLE can be at */
 	usable_startpfn = arch_zone_lowest_possible_pfn[movable_zone];
@@ -4949,10 +4957,24 @@ restart:
 		for_each_mem_pfn_range(i, nid, &start_pfn, &end_pfn, NULL) {
 			unsigned long size_pages;
 
+			/*
+			 * Find more memory for kernelcore in
+			 * [zone_movable_pfn[nid], zone_movable_limit[nid]).
+			 */
 			start_pfn = max(start_pfn, zone_movable_pfn[nid]);
 			if (start_pfn >= end_pfn)
 				continue;
 
+			if (zone_movable_limit[nid]) {
+				end_pfn = min(end_pfn, zone_movable_limit[nid]);
+				/* No range left for kernelcore in this node */
+				if (start_pfn >= end_pfn) {
+					zone_movable_pfn[nid] =
+							zone_movable_limit[nid];
+					break;
+				}
+			}
+
 			/* Account for what is only usable for kernelcore */
 			if (start_pfn < usable_startpfn) {
 				unsigned long kernel_pages;
@@ -5012,12 +5034,12 @@ restart:
 	if (usable_nodes && required_kernelcore > usable_nodes)
 		goto restart;
 
+out:
 	/* Align start of ZONE_MOVABLE on all nids to MAX_ORDER_NR_PAGES */
 	for (nid = 0; nid < MAX_NUMNODES; nid++)
 		zone_movable_pfn[nid] =
 			roundup(zone_movable_pfn[nid], MAX_ORDER_NR_PAGES);
 
-out:
 	/* restore the node_state */
 	node_states[N_MEMORY] = saved_node_state;
 }
-- 
1.7.1

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

[PATCH v4 6/6] page_alloc: Bootmem limit with movablecore_map

[Tang Chen]

This patch make sure bootmem will not allocate memory from areas that
may be ZONE_MOVABLE. The map info is from movablecore_map boot option.

Signed-off-by: Tang Chen <tangchen@cn.fujitsu.com>
Signed-off-by: Lai Jiangshan <laijs@cn.fujitsu.com>
Reviewed-by: Wen Congyang <wency@cn.fujitsu.com>
Tested-by: Lin Feng <linfeng@cn.fujitsu.com>
---
 include/linux/memblock.h |    1 +
 mm/memblock.c            |   18 +++++++++++++++++-
 2 files changed, 18 insertions(+), 1 deletions(-)

diff --git a/include/linux/memblock.h b/include/linux/memblock.h
index d452ee1..6e25597 100644
--- a/include/linux/memblock.h
+++ b/include/linux/memblock.h
@@ -42,6 +42,7 @@ struct memblock {
 
 extern struct memblock memblock;
 extern int memblock_debug;
+extern struct movablecore_map movablecore_map;
 
 #define memblock_dbg(fmt, ...) \
 	if (memblock_debug) printk(KERN_INFO pr_fmt(fmt), ##__VA_ARGS__)
diff --git a/mm/memblock.c b/mm/memblock.c
index 6259055..197c3be 100644
--- a/mm/memblock.c
+++ b/mm/memblock.c
@@ -101,6 +101,7 @@ phys_addr_t __init_memblock memblock_find_in_range_node(phys_addr_t start,
 {
 	phys_addr_t this_start, this_end, cand;
 	u64 i;
+	int curr = movablecore_map.nr_map - 1;
 
 	/* pump up @end */
 	if (end == MEMBLOCK_ALLOC_ACCESSIBLE)
@@ -114,13 +115,28 @@ phys_addr_t __init_memblock memblock_find_in_range_node(phys_addr_t start,
 		this_start = clamp(this_start, start, end);
 		this_end = clamp(this_end, start, end);
 
-		if (this_end < size)
+restart:
+		if (this_end <= this_start || this_end < size)
 			continue;
 
+		for (; curr >= 0; curr--) {
+			if ((movablecore_map.map[curr].start_pfn << PAGE_SHIFT)
+			    < this_end)
+				break;
+		}
+
 		cand = round_down(this_end - size, align);
+		if (curr >= 0 &&
+		    cand < movablecore_map.map[curr].end_pfn << PAGE_SHIFT) {
+			this_end = movablecore_map.map[curr].start_pfn
+				   << PAGE_SHIFT;
+			goto restart;
+		}
+
 		if (cand >= this_start)
 			return cand;
 	}
+
 	return 0;
 }
 
-- 
1.7.1

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>