[Patch 0/8] V3 Implement crashkernel=auto

[Patch 0/8] V3 Implement crashkernel=auto

[Amerigo Wang]

V2 -> V3:
 - Use more clever way to calculate reserved memory size, especially for IA64.
 - Add that patch that implements shrinking reserved memory

V1 -> V2:
 - Use include/asm-generic/kexec.h, suggested by Neil.
 - Rename a local variable, suggested by Fenghua.
 - Fix some style problems found by checkpatch.pl.
 - Unify the Kconfig docs.

This series of patch implements automatically reserved memory for crashkernel,
by introducing a new boot option "crashkernel=auto". This idea is from Neil.

In case of breaking user-space applications, it modifies this boot option after
it decides how much memory should be reserved.

On different arch, the threshold and reserved memory size is different. Please
refer patch 7/8 which contains an update for the documentation.

Patch 8/8 implements shrinking reserved memory at run-time.

Note: This patchset was only tested on x86_64 with differernt memory sizes.

Signed-off-by: WANG Cong <amwang@redhat.com>
Cc: Neil Horman <nhorman@redhat.com>
Cc: Eric W. Biederman <ebiederm@xmission.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Tony Luck <tony.luck@intel.com>
Cc: Anton Vorontsov <avorontsov@ru.mvista.com>
Cc: Andi Kleen <andi@firstfloor.org>
Cc: Fenghua Yu <fenghua.yu@intel.com>
Cc: Bernhard Walle <bernhard.walle@gmx.de>

---
 Documentation/kdump/kdump.txt    |   28 ++++++++++
 arch/ia64/Kconfig                |   14 +++++
 arch/ia64/include/asm/kexec.h    |   23 ++++++++
 arch/powerpc/Kconfig             |   11 ++++
 arch/powerpc/include/asm/kexec.h |    8 +++
 arch/x86/Kconfig                 |   13 ++++
 arch/x86/include/asm/kexec.h     |    1 
 include/asm-generic/kexec.h      |   42 +++++++++++++++
 include/linux/kexec.h            |    3 +
 kernel/kexec.c                   |  104 +++++++++++++++++++++++++++++++++++++++
 kernel/ksysfs.c                  |   21 +++++++
 11 files changed, 268 insertions(+)

[Patch 1/8] x86: add CONFIG_KEXEC_AUTO_RESERVE

[Amerigo Wang]

Introduce a new config option KEXEC_AUTO_RESERVE for x86.

Signed-off-by: WANG Cong <amwang@redhat.com>
Acked-by: Neil Horman <nhorman@tuxdriver.com>

---

Index: linux-2.6/arch/x86/Kconfig
===================================================================
--- linux-2.6.orig/arch/x86/Kconfig
+++ linux-2.6/arch/x86/Kconfig
@@ -1482,6 +1482,19 @@ config KEXEC
 	  support.  As of this writing the exact hardware interface is
 	  strongly in flux, so no good recommendation can be made.
 
+config KEXEC_AUTO_RESERVE
+	bool "automatically reserve memory for kexec kernel"
+	depends on KEXEC
+	default y
+	---help---
+	  Automatically reserve memory for a kexec kernel, so that you don't
+	  need to specify numbers for the "crashkernel=X@Y" boot option,
+	  instead you can use "crashkernel=auto". To make this work, you need
+	  to have more than 4G memory.
+
+	  On x86_32, 128M is reserved, on x86_64 1/32 of your memory is
+	  reserved, but it will not exceed 1T/32.
+
 config CRASH_DUMP
 	bool "kernel crash dumps"
 	depends on X86_64 || (X86_32 && HIGHMEM)

[Patch 2/8] x86: implement crashkernel=auto

[Amerigo Wang]

Implement "crashkernel=auto" for x86 first, other arch will be added in the
following patches.

Signed-off-by: WANG Cong <amwang@redhat.com>

---

Index: linux-2.6/kernel/kexec.c
===================================================================
--- linux-2.6.orig/kernel/kexec.c
+++ linux-2.6/kernel/kexec.c
@@ -37,6 +37,7 @@
 #include <asm/io.h>
 #include <asm/system.h>
 #include <asm/sections.h>
+#include <asm/setup.h>
 
 /* Per cpu memory for storing cpu states in case of system crash. */
 note_buf_t* crash_notes;
@@ -1297,6 +1298,39 @@ int __init parse_crashkernel(char 		 *cm
 
 	ck_cmdline += 12; /* strlen("crashkernel=") */
 
+#ifdef CONFIG_KEXEC_AUTO_RESERVE
+	if (strncmp(ck_cmdline, "auto", 4) == 0) {
+		unsigned long long size;
+		int len;
+		char tmp[32];
+
+		size = arch_default_crash_size(system_ram);
+		if (size != 0) {
+			*crash_size = size;
+			*crash_base = arch_default_crash_base();
+			len = scnprintf(tmp, sizeof(tmp), "%luM@%luM",
+					(unsigned long)(*crash_size)>>20,
+					(unsigned long)(*crash_base)>>20);
+			/* 'len' can't be <= 4. */
+			if (likely((len - 4 + strlen(cmdline))
+					< COMMAND_LINE_SIZE - 1)) {
+				memmove(ck_cmdline + len, ck_cmdline + 4,
+					strlen(cmdline) - (ck_cmdline + 4 - cmdline) + 1);
+				memcpy(ck_cmdline, tmp, len);
+			}
+			return 0;
+		} else {
+			/*
+			 * We can't reserve memory auotmatcally,
+			 * remove "crashkernel=auto" from cmdline.
+			 */
+			ck_cmdline += 4; /* strlen("auto") */
+			memmove(ck_cmdline - 16, ck_cmdline,
+				strlen(cmdline) - (ck_cmdline - cmdline) + 1);
+			return -ENOMEM;
+		}
+	}
+#endif
 	/*
 	 * if the commandline contains a ':', then that's the extended
 	 * syntax -- if not, it must be the classic syntax
Index: linux-2.6/arch/x86/include/asm/kexec.h
===================================================================
--- linux-2.6.orig/arch/x86/include/asm/kexec.h
+++ linux-2.6/arch/x86/include/asm/kexec.h
@@ -23,6 +23,7 @@
 
 #include <asm/page.h>
 #include <asm/ptrace.h>
+#include <asm-generic/kexec.h>
 
 /*
  * KEXEC_SOURCE_MEMORY_LIMIT maximum page get_free_page can return.
Index: linux-2.6/include/asm-generic/kexec.h
===================================================================
--- /dev/null
+++ linux-2.6/include/asm-generic/kexec.h
@@ -0,0 +1,42 @@
+#ifndef _ASM_GENERIC_KEXEC_H
+#define _ASM_GENERIC_KEXEC_H
+
+#ifdef CONFIG_KEXEC_AUTO_RESERVE
+
+#ifndef KEXEC_AUTO_RESERVED_SIZE
+#define KEXEC_AUTO_RESERVED_SIZE (1ULL<<27) /* 128M */
+#endif
+#ifndef KEXEC_AUTO_THRESHOLD
+#define KEXEC_AUTO_THRESHOLD (1ULL<<32) /* 4G */
+#endif
+
+#ifndef ARCH_HAS_DEFAULT_CRASH_SIZE
+static inline
+unsigned long long arch_default_crash_size(unsigned long long total_size)
+{
+	if (total_size < KEXEC_AUTO_THRESHOLD)
+		return 0;
+	else {
+#ifdef CONFIG_64BIT
+		if (total_size > 1ULL<<40) /* 1TB */
+			return KEXEC_AUTO_RESERVED_SIZE
+				* (1ULL<<40 / KEXEC_AUTO_THRESHOLD);
+		return 1ULL<<ilog2(roundup(total_size/32, 1ULL<<21));
+#else
+		return KEXEC_AUTO_RESERVED_SIZE;
+#endif
+	}
+}
+#endif
+#ifndef ARCH_HAS_DEFAULT_CRASH_BASE
+static inline
+unsigned long long arch_default_crash_base(void)
+{
+	/* 0 means find the base address automatically. */
+	return 0;
+}
+#endif
+
+#endif /* CONFIG_KEXEC_AUTO_RESERVE */
+
+#endif

[Patch 3/8] ia64: add CONFIG_KEXEC_AUTO_RESERVE

[Amerigo Wang]

Introduce a new config option KEXEC_AUTO_RESERVE for ia64.

Signed-off-by: WANG Cong <amwang@redhat.com>
Acked-by: Neil Horman <nhorman@tuxdriver.com>

---

Index: linux-2.6/arch/ia64/Kconfig
===================================================================
--- linux-2.6.orig/arch/ia64/Kconfig
+++ linux-2.6/arch/ia64/Kconfig
@@ -582,6 +582,20 @@ config KEXEC
 	  support.  As of this writing the exact hardware interface is
 	  strongly in flux, so no good recommendation can be made.
 
+config KEXEC_AUTO_RESERVE
+	bool "automatically reserve memory for kexec kernel"
+	depends on KEXEC
+	default y
+	---help---
+	  Automatically reserve memory for a kexec kernel, so that you don't
+	  need to specify numbers for the "crashkernel=X@Y" boot option,
+	  instead you can use "crashkernel=auto". To make this work, you need
+	  to have more than 4G memory.
+
+	  The reserved memory size is different depends on how much memory
+	  you actually have. Please check Documentation/kdump/kdump.txt.
+	  If you doubt, say N.
+
 config CRASH_DUMP
 	  bool "kernel crash dumps"
 	  depends on IA64_MCA_RECOVERY && !IA64_HP_SIM && (!SMP || HOTPLUG_CPU)

[Patch 4/8] ia64: implement crashkernel=auto

[Amerigo Wang]

Since in patch 2/8 we already implement the generic part, this will
add the rest part for ia64.

Signed-off-by: WANG Cong <amwang@redhat.com>

---

Index: linux-2.6/arch/ia64/include/asm/kexec.h
===================================================================
--- linux-2.6.orig/arch/ia64/include/asm/kexec.h
+++ linux-2.6/arch/ia64/include/asm/kexec.h
@@ -19,6 +19,29 @@
                 flush_icache_range(page_addr, page_addr + PAGE_SIZE); \
         } while(0)
 
+#ifdef CONFIG_KEXEC_AUTO_RESERVE
+#define ARCH_HAS_DEFAULT_CRASH_SIZE
+static inline
+unsigned long long arch_default_crash_size(unsigned long long total_size)
+{
+	if (total_size >= 4ULL<<30 && total_size < 12ULL<<30)
+		return 1ULL<<28;
+	else if (total_size >= 12ULL<<30 && total_size < 128ULL<<30)
+		return 1ULL<<29;
+	else if (total_size >= 128ULL<<30 && total_size < 256ULL<<30)
+		return 3ULL<<28;
+	else if (total_size >= 256ULL<<30 && total_size < 378ULL<<30)
+		return 1ULL<<30;
+	else if (total_size >= 318ULL<<30 && total_size < 512ULL<<30)
+		return 3ULL<<29;
+	else if (total_size >= 512ULL<<30 && total_size < 768ULL<<30)
+		return 2ULL<<30;
+	else if (total_size >= 768ULL<<30)
+		return 3ULL<<30;
+}
+#include <asm-generic/kexec.h>
+#endif
+
 extern struct kimage *ia64_kimage;
 extern const unsigned int relocate_new_kernel_size;
 extern void relocate_new_kernel(unsigned long, unsigned long,

[Patch 5/8] powerpc: add CONFIG_KEXEC_AUTO_RESERVE

[Amerigo Wang]

Introduce a new config option KEXEC_AUTO_RESERVE for powerpc.

Signed-off-by: WANG Cong <amwang@redhat.com>
Acked-by: Neil Horman <nhorman@tuxdriver.com>

---

Index: linux-2.6/arch/powerpc/Kconfig
===================================================================
--- linux-2.6.orig/arch/powerpc/Kconfig
+++ linux-2.6/arch/powerpc/Kconfig
@@ -346,6 +346,17 @@ config KEXEC
 	  support.  As of this writing the exact hardware interface is
 	  strongly in flux, so no good recommendation can be made.
 
+config KEXEC_AUTO_RESERVE
+	bool "automatically reserve memory for kexec kernel"
+	depends on KEXEC
+	default y
+	---help---
+	  Automatically reserve memory for a kexec kernel, so that you don't
+	  need to specify numbers for the "crashkernel=X@Y" boot option,
+	  instead you can use "crashkernel=auto". To make this work, you need
+	  to have more than 4G memory. On PPC 256M is reserved, 1/32 memory
+	  on PPC64, but it will not exceed 1T/32.
+
 config CRASH_DUMP
 	bool "Build a kdump crash kernel"
 	depends on PPC64 || 6xx

[Patch 6/8] powerpc: implement crashkernel=auto

[Amerigo Wang]

Since in patch 2/8 we already implement the generic part, this will
add the rest part for powerpc.

Signed-off-by: WANG Cong <amwang@redhat.com>

---

Index: linux-2.6/arch/powerpc/include/asm/kexec.h
===================================================================
--- linux-2.6.orig/arch/powerpc/include/asm/kexec.h
+++ linux-2.6/arch/powerpc/include/asm/kexec.h
@@ -39,6 +39,14 @@ typedef void (*crash_shutdown_t)(void);
 
 #ifdef CONFIG_KEXEC
 
+#ifdef CONFIG_KEXEC_AUTO_RESERVE
+#ifdef KEXEC_AUTO_RESERVED_SIZE
+#undef KEXEC_AUTO_RESERVED_SIZE
+#endif
+#define KEXEC_AUTO_RESERVED_SIZE (1ULL<<28) /* 256M */
+#include <asm-generic/kexec.h>
+#endif
+
 /*
  * This function is responsible for capturing register states if coming
  * via panic or invoking dump using sysrq-trigger.

[Patch 7/8] doc: update the kdump document

[Amerigo Wang]

Update the document for kdump.

Signed-off-by: WANG Cong <amwang@redhat.com>

---

Index: linux-2.6/Documentation/kdump/kdump.txt
===================================================================
--- linux-2.6.orig/Documentation/kdump/kdump.txt
+++ linux-2.6/Documentation/kdump/kdump.txt
@@ -147,6 +147,15 @@ System kernel config options
    analysis tools require a vmlinux with debug symbols in order to read
    and analyze a dump file.
 
+4) Enable "automatically reserve memory for kexec kernel" in
+   "Processor type and features."
+
+   CONFIG_KEXEC_AUTO_RESERVE=y
+
+   This will let you to use "crashkernel=auto", instead of specifying
+   numbers for "crashkernel=". Note, you need to have enough memory.
+   The threshold and reserved memory size are arch-dependent.
+
 Dump-capture kernel config options (Arch Independent)
 -----------------------------------------------------
 
@@ -266,6 +275,25 @@ This would mean:
     2) if the RAM size is between 512M and 2G (exclusive), then reserve 64M
     3) if the RAM size is larger than 2G, then reserve 128M
 
+Or you can use:
+
+    crashkernel=auto
+
+if you have enough memory. The threshold is 4G, below which this won't work.
+
+The automatically reserved memory size would be 128M on x86_32, 256M on
+ppc, 1/32 of your physical memory size on x86_64 and ppc64 (but it will not
+exceed 1TB/32 if you have more). IA64 has its own policy, shown below:
+
+	Memory size	Reserved memory
+	===========	===============
+	[4G, 12G)	256M
+	[12G, 128G)	512M
+	[128G, 256G)	768M
+	[256G, 378G)	1024M
+	[378G, 512G)	1536M
+	[512G, 768G)	2048M
+	[768G, )	3072M
 
 
 Boot into System Kernel

[Patch 8/8] kexec: allow to shrink reserved memory

[Amerigo Wang]

This patch implements shrinking the reserved memory for crash kernel,
if it is more than enough.

For example, if you have already reserved 128M, now you just want 100M,
you can do:

# echo $((100*1024*1024)) > /sys/kernel/kexec_crash_size

Signed-off-by: WANG Cong <amwang@redhat.com>
Cc: Neil Horman <nhorman@redhat.com>
Cc: Eric W. Biederman <ebiederm@xmission.com>
Cc: Andi Kleen <andi@firstfloor.org>

---

Index: linux-2.6/include/linux/kexec.h
===================================================================
--- linux-2.6.orig/include/linux/kexec.h
+++ linux-2.6/include/linux/kexec.h
@@ -206,6 +206,9 @@ extern size_t vmcoreinfo_max_size;
 
 int __init parse_crashkernel(char *cmdline, unsigned long long system_ram,
 		unsigned long long *crash_size, unsigned long long *crash_base);
+int shrink_crash_memory(unsigned long new_size);
+int kexec_crash_kernel_loaded(void);
+size_t get_crash_memory_size(void);
 
 #else /* !CONFIG_KEXEC */
 struct pt_regs;
Index: linux-2.6/kernel/kexec.c
===================================================================
--- linux-2.6.orig/kernel/kexec.c
+++ linux-2.6/kernel/kexec.c
@@ -1083,6 +1083,76 @@ void crash_kexec(struct pt_regs *regs)
 	}
 }
 
+int kexec_crash_kernel_loaded(void)
+{
+	int ret;
+	if (!mutex_trylock(&kexec_mutex))
+		return 1;
+	ret = kexec_crash_image != NULL;
+	mutex_unlock(&kexec_mutex);
+	return ret;
+}
+
+size_t get_crash_memory_size(void)
+{
+	size_t size;
+	if (!mutex_trylock(&kexec_mutex))
+		return 1;
+	size = crashk_res.end - crashk_res.start + 1;
+	mutex_unlock(&kexec_mutex);
+	return size;
+}
+
+int shrink_crash_memory(unsigned long new_size)
+{
+	struct page **pages;
+	int ret = 0;
+	int  npages, i;
+	unsigned long addr;
+	unsigned long start, end;
+	void *vaddr;
+
+	if (!mutex_trylock(&kexec_mutex))
+		return -EBUSY;
+
+	start = crashk_res.start;
+	end = crashk_res.end;
+
+	if (new_size >= end - start + 1) {
+		ret = -EINVAL;
+		if (new_size == end - start + 1)
+			ret = 0;
+		goto unlock;
+	}
+
+	start = roundup(start, PAGE_SIZE);
+	end = roundup(start + new_size, PAGE_SIZE) - 1;
+	npages = (end + 1 - start ) / PAGE_SIZE;
+
+	pages = kmalloc(sizeof(struct page *) * npages, GFP_KERNEL);
+	if (!pages) {
+		ret = -ENOMEM;
+		goto unlock;
+	}
+	for (i = 0; i < npages; i++) {
+		addr = end + 1 + i * PAGE_SIZE;
+		pages[i] = virt_to_page(addr);
+	}
+
+	vaddr = vm_map_ram(pages, npages, 0, PAGE_KERNEL);
+	if (!vaddr) {
+		ret = -ENOMEM;
+		goto free;
+	}
+	crashk_res.end = end;
+
+free:
+	kfree(pages);
+unlock:
+	mutex_unlock(&kexec_mutex);
+	return ret;
+}
+
 static u32 *append_elf_note(u32 *buf, char *name, unsigned type, void *data,
 			    size_t data_len)
 {
Index: linux-2.6/kernel/ksysfs.c
===================================================================
--- linux-2.6.orig/kernel/ksysfs.c
+++ linux-2.6/kernel/ksysfs.c
@@ -100,6 +100,26 @@ static ssize_t kexec_crash_loaded_show(s
 }
 KERNEL_ATTR_RO(kexec_crash_loaded);
 
+static ssize_t kexec_crash_size_show(struct kobject *kobj,
+				       struct kobj_attribute *attr, char *buf)
+{
+	return sprintf(buf, "%lu\n", get_crash_memory_size());
+}
+static ssize_t kexec_crash_size_store(struct kobject *kobj,
+				   struct kobj_attribute *attr,
+				   const char *buf, size_t count)
+{
+	unsigned long cnt;
+	int ret;
+
+	if (kexec_crash_kernel_loaded())
+		return -ENOENT;
+	cnt = simple_strtoul(buf, NULL, 10);
+	ret = shrink_crash_memory(cnt);
+	return ret < 0 ? ret: count;
+}
+KERNEL_ATTR_RW(kexec_crash_size);
+
 static ssize_t vmcoreinfo_show(struct kobject *kobj,
 			       struct kobj_attribute *attr, char *buf)
 {
@@ -147,6 +167,7 @@ static struct attribute * kernel_attrs[]
 #ifdef CONFIG_KEXEC
 	&kexec_loaded_attr.attr,
 	&kexec_crash_loaded_attr.attr,
+	&kexec_crash_size_attr.attr,
 	&vmcoreinfo_attr.attr,
 #endif
 	NULL

Re: [Patch 8/8] kexec: allow to shrink reserved memory

[Eric W. Biederman]

Amerigo Wang <amwang@redhat.com> writes:

> This patch implements shrinking the reserved memory for crash kernel,
> if it is more than enough.
>
> For example, if you have already reserved 128M, now you just want 100M,
> you can do:
>
> # echo $((100*1024*1024)) > /sys/kernel/kexec_crash_size

Getting closer (comments inline)

Semantically this patch is non-contriversial and pretty
simple, but still needs a fair amount of review.  Can
you put this patch at the front of your patch set.

> Signed-off-by: WANG Cong <amwang@redhat.com>
> Cc: Neil Horman <nhorman@redhat.com>
> Cc: Eric W. Biederman <ebiederm@xmission.com>
> Cc: Andi Kleen <andi@firstfloor.org>
>
> ---
>
> Index: linux-2.6/include/linux/kexec.h
> ===================================================================
> --- linux-2.6.orig/include/linux/kexec.h
> +++ linux-2.6/include/linux/kexec.h
> @@ -206,6 +206,9 @@ extern size_t vmcoreinfo_max_size;
>  
>  int __init parse_crashkernel(char *cmdline, unsigned long long system_ram,
>  		unsigned long long *crash_size, unsigned long long *crash_base);
> +int shrink_crash_memory(unsigned long new_size);
> +int kexec_crash_kernel_loaded(void);
> +size_t get_crash_memory_size(void);
>  
>  #else /* !CONFIG_KEXEC */
>  struct pt_regs;
> Index: linux-2.6/kernel/kexec.c
> ===================================================================
> --- linux-2.6.orig/kernel/kexec.c
> +++ linux-2.6/kernel/kexec.c
> @@ -1083,6 +1083,76 @@ void crash_kexec(struct pt_regs *regs)
>  	}
>  }
>  
> +int kexec_crash_kernel_loaded(void)
> +{
> +	int ret;
> +	if (!mutex_trylock(&kexec_mutex))
> +		return 1;

We don't need trylock on this code path 

> +	ret = kexec_crash_image != NULL;
> +	mutex_unlock(&kexec_mutex);
> +	return ret;
> +}
> +
> +size_t get_crash_memory_size(void)
> +{
> +	size_t size;
> +	if (!mutex_trylock(&kexec_mutex))
> +		return 1;

We don't need trylock on this code path 

> +	size = crashk_res.end - crashk_res.start + 1;
> +	mutex_unlock(&kexec_mutex);
> +	return size;
> +}
> +
> +int shrink_crash_memory(unsigned long new_size)
> +{
> +	struct page **pages;
> +	int ret = 0;
> +	int  npages, i;
> +	unsigned long addr;
> +	unsigned long start, end;
> +	void *vaddr;
> +
> +	if (!mutex_trylock(&kexec_mutex))
> +		return -EBUSY;

We don't need trylock on this code path 

We are missing the check to see if the crash_kernel is loaded
under this lock instance. So I please move the kexec_crash_image != NULL
test inline here and kill the kexec_crash_kernel_loaded function.

> +	start = crashk_res.start;
> +	end = crashk_res.end;
> +
> +	if (new_size >= end - start + 1) {
> +		ret = -EINVAL;
> +		if (new_size == end - start + 1)
> +			ret = 0;
> +		goto unlock;
> +	}
> +
> +	start = roundup(start, PAGE_SIZE);
> +	end = roundup(start + new_size, PAGE_SIZE) - 1;
> +	npages = (end + 1 - start ) / PAGE_SIZE;
> +
> +	pages = kmalloc(sizeof(struct page *) * npages, GFP_KERNEL);
> +	if (!pages) {
> +		ret = -ENOMEM;
> +		goto unlock;
> +	}
> +	for (i = 0; i < npages; i++) {
> +		addr = end + 1 + i * PAGE_SIZE;
> +		pages[i] = virt_to_page(addr);
> +	}
> +
> +	vaddr = vm_map_ram(pages, npages, 0, PAGE_KERNEL);

This is the wrong kernel call to use.  I expect this needs to look
like a memory hotplug event.  This does not put the pages into the
free page pool.

> +	if (!vaddr) {
> +		ret = -ENOMEM;
> +		goto free;
> +	}
> +	crashk_res.end = end;
> +
> +free:
> +	kfree(pages);
> +unlock:
> +	mutex_unlock(&kexec_mutex);
> +	return ret;
> +}
> +
>  static u32 *append_elf_note(u32 *buf, char *name, unsigned type, void *data,
>  			    size_t data_len)
>  {
> Index: linux-2.6/kernel/ksysfs.c
> ===================================================================
> --- linux-2.6.orig/kernel/ksysfs.c
> +++ linux-2.6/kernel/ksysfs.c
> @@ -100,6 +100,26 @@ static ssize_t kexec_crash_loaded_show(s
>  }
>  KERNEL_ATTR_RO(kexec_crash_loaded);
>  
> +static ssize_t kexec_crash_size_show(struct kobject *kobj,
> +				       struct kobj_attribute *attr, char *buf)
> +{
> +	return sprintf(buf, "%lu\n", get_crash_memory_size());
> +}
> +static ssize_t kexec_crash_size_store(struct kobject *kobj,
> +				   struct kobj_attribute *attr,
> +				   const char *buf, size_t count)
> +{
> +	unsigned long cnt;
> +	int ret;
> +
> +	if (kexec_crash_kernel_loaded())
> +		return -ENOENT;
> +	cnt = simple_strtoul(buf, NULL, 10);
> +	ret = shrink_crash_memory(cnt);
> +	return ret < 0 ? ret: count;
> +}
> +KERNEL_ATTR_RW(kexec_crash_size);
> +
>  static ssize_t vmcoreinfo_show(struct kobject *kobj,
>  			       struct kobj_attribute *attr, char *buf)
>  {
> @@ -147,6 +167,7 @@ static struct attribute * kernel_attrs[]
>  #ifdef CONFIG_KEXEC
>  	&kexec_loaded_attr.attr,
>  	&kexec_crash_loaded_attr.attr,
> +	&kexec_crash_size_attr.attr,
>  	&vmcoreinfo_attr.attr,
>  #endif
>  	NULL

Re: [Patch 8/8] kexec: allow to shrink reserved memory

[Amerigo Wang]

Eric W. Biederman wrote:
> Amerigo Wang <amwang@redhat.com> writes:
>
>   
>> This patch implements shrinking the reserved memory for crash kernel,
>> if it is more than enough.
>>
>> For example, if you have already reserved 128M, now you just want 100M,
>> you can do:
>>
>> # echo $((100*1024*1024)) > /sys/kernel/kexec_crash_size
>>     
>
> Getting closer (comments inline)
>
> Semantically this patch is non-contriversial and pretty
> simple, but still needs a fair amount of review.  Can
> you put this patch at the front of your patch set.
>
>   

Sure, I will do it when I resend them next time.

I add mm people into Cc.
>> Index: linux-2.6/kernel/kexec.c
>> ===================================================================
>> --- linux-2.6.orig/kernel/kexec.c
>> +++ linux-2.6/kernel/kexec.c
>> @@ -1083,6 +1083,76 @@ void crash_kexec(struct pt_regs *regs)
>>  	}
>>  }
>>  
>> +int kexec_crash_kernel_loaded(void)
>> +{
>> +	int ret;
>> +	if (!mutex_trylock(&kexec_mutex))
>> +		return 1;
>>     
>
> We don't need trylock on this code path 
>   

OK.

>   
>> +	ret = kexec_crash_image != NULL;
>> +	mutex_unlock(&kexec_mutex);
>> +	return ret;
>> +}
>> +
>> +size_t get_crash_memory_size(void)
>> +{
>> +	size_t size;
>> +	if (!mutex_trylock(&kexec_mutex))
>> +		return 1;
>>     
>
> We don't need trylock on this code path 
>
>   

Hmm, crashk_res is a global struct, so other process can also
change it... but currently no process does that, right?

>> +	size = crashk_res.end - crashk_res.start + 1;
>> +	mutex_unlock(&kexec_mutex);
>> +	return size;
>> +}
>> +
>> +int shrink_crash_memory(unsigned long new_size)
>> +{
>> +	struct page **pages;
>> +	int ret = 0;
>> +	int  npages, i;
>> +	unsigned long addr;
>> +	unsigned long start, end;
>> +	void *vaddr;
>> +
>> +	if (!mutex_trylock(&kexec_mutex))
>> +		return -EBUSY;
>>     
>
> We don't need trylock on this code path 
>
> We are missing the check to see if the crash_kernel is loaded
> under this lock instance. So I please move the kexec_crash_image != NULL
> test inline here and kill the kexec_crash_kernel_loaded function.
>   

Ok, no problem.

>   
>> +	start = crashk_res.start;
>> +	end = crashk_res.end;
>> +
>> +	if (new_size >= end - start + 1) {
>> +		ret = -EINVAL;
>> +		if (new_size == end - start + 1)
>> +			ret = 0;
>> +		goto unlock;
>> +	}
>> +
>> +	start = roundup(start, PAGE_SIZE);
>> +	end = roundup(start + new_size, PAGE_SIZE) - 1;
>> +	npages = (end + 1 - start ) / PAGE_SIZE;
>> +
>> +	pages = kmalloc(sizeof(struct page *) * npages, GFP_KERNEL);
>> +	if (!pages) {
>> +		ret = -ENOMEM;
>> +		goto unlock;
>> +	}
>> +	for (i = 0; i < npages; i++) {
>> +		addr = end + 1 + i * PAGE_SIZE;
>> +		pages[i] = virt_to_page(addr);
>> +	}
>> +
>> +	vaddr = vm_map_ram(pages, npages, 0, PAGE_KERNEL);
>>     
>
> This is the wrong kernel call to use.  I expect this needs to look
> like a memory hotplug event.  This does not put the pages into the
> free page pool.
>   

Well, I also wanted to use an memory-hotplug API, but that will make the 
code depend on memory-hotplug, which certainly is not what we want...

I checked the mm code, actually what I need is an API which is similar 
to add_active_range(), but add_active_range() can't be used here since 
it is marked as "__init".

Do we have that kind of API in mm? I can't find one.

Thanks!

Re: [Patch 8/8] kexec: allow to shrink reserved memory

[Eric W. Biederman]

Amerigo Wang <amwang@redhat.com> writes:

> Eric W. Biederman wrote:
>> Amerigo Wang <amwang@redhat.com> writes:
>>
>>   
>>> This patch implements shrinking the reserved memory for crash kernel,
>>> if it is more than enough.
>>>
>>> For example, if you have already reserved 128M, now you just want 100M,
>>> you can do:
>>>
>>> # echo $((100*1024*1024)) > /sys/kernel/kexec_crash_size
>>>     
>>
>> Getting closer (comments inline)
>>
>> Semantically this patch is non-contriversial and pretty
>> simple, but still needs a fair amount of review.  Can
>> you put this patch at the front of your patch set.
>>
>>   
>
> Sure, I will do it when I resend them next time.
>
> I add mm people into Cc.
>>> Index: linux-2.6/kernel/kexec.c
>>> ===================================================================
>>> --- linux-2.6.orig/kernel/kexec.c
>>> +++ linux-2.6/kernel/kexec.c
>>> @@ -1083,6 +1083,76 @@ void crash_kexec(struct pt_regs *regs)
>>>  	}
>>>  }
>>>  +int kexec_crash_kernel_loaded(void)
>>> +{
>>> +	int ret;
>>> +	if (!mutex_trylock(&kexec_mutex))
>>> +		return 1;
>>>     
>>
>> We don't need trylock on this code path   
>
> OK.
>
>>   
>>> +	ret = kexec_crash_image != NULL;
>>> +	mutex_unlock(&kexec_mutex);
>>> +	return ret;
>>> +}
>>> +
>>> +size_t get_crash_memory_size(void)
>>> +{
>>> +	size_t size;
>>> +	if (!mutex_trylock(&kexec_mutex))
>>> +		return 1;
>>>     
>>
>> We don't need trylock on this code path 
>>
>>   
>
> Hmm, crashk_res is a global struct, so other process can also
> change it... but currently no process does that, right?
>

We still need the lock.  Just doing trylock doesn't instead
of just sleeping doesn't seem to make any sense on these
code paths.

>>> +	size = crashk_res.end - crashk_res.start + 1;
>>> +	mutex_unlock(&kexec_mutex);
>>> +	return size;
>>> +}
>>> +
>>> +int shrink_crash_memory(unsigned long new_size)
>>> +{
>>> +	struct page **pages;
>>> +	int ret = 0;
>>> +	int  npages, i;
>>> +	unsigned long addr;
>>> +	unsigned long start, end;
>>> +	void *vaddr;
>>> +
>>> +	if (!mutex_trylock(&kexec_mutex))
>>> +		return -EBUSY;
>>>     
>>
>> We don't need trylock on this code path 
>>
>> We are missing the check to see if the crash_kernel is loaded
>> under this lock instance. So I please move the kexec_crash_image != NULL
>> test inline here and kill the kexec_crash_kernel_loaded function.
>>   
>
> Ok, no problem.
>
>>   
>>> +	start = crashk_res.start;
>>> +	end = crashk_res.end;
>>> +
>>> +	if (new_size >= end - start + 1) {
>>> +		ret = -EINVAL;
>>> +		if (new_size == end - start + 1)
>>> +			ret = 0;
>>> +		goto unlock;
>>> +	}
>>> +
>>> +	start = roundup(start, PAGE_SIZE);
>>> +	end = roundup(start + new_size, PAGE_SIZE) - 1;
>>> +	npages = (end + 1 - start ) / PAGE_SIZE;
>>> +
>>> +	pages = kmalloc(sizeof(struct page *) * npages, GFP_KERNEL);
>>> +	if (!pages) {
>>> +		ret = -ENOMEM;
>>> +		goto unlock;
>>> +	}
>>> +	for (i = 0; i < npages; i++) {
>>> +		addr = end + 1 + i * PAGE_SIZE;
>>> +		pages[i] = virt_to_page(addr);
>>> +	}
>>> +
>>> +	vaddr = vm_map_ram(pages, npages, 0, PAGE_KERNEL);
>>>     
>>
>> This is the wrong kernel call to use.  I expect this needs to look
>> like a memory hotplug event.  This does not put the pages into the
>> free page pool.
>>   
>
> Well, I also wanted to use an memory-hotplug API, but that will make the code
> depend on memory-hotplug, which certainly is not what we want...
>
> I checked the mm code, actually what I need is an API which is similar to
> add_active_range(), but add_active_range() can't be used here since it is marked
> as "__init".
>
> Do we have that kind of API in mm? I can't find one.

Perhaps we will need to remove __init from add_active_range.  I know the logic
but I'm not up to speed on the mm pieces at the moment.

Eric

Re: [Patch 8/8] kexec: allow to shrink reserved memory

[Amerigo Wang]

Eric W. Biederman wrote:
> Amerigo Wang <amwang@redhat.com> writes:
>
>   
>>>   
>>>       
>>>> +	ret = kexec_crash_image != NULL;
>>>> +	mutex_unlock(&kexec_mutex);
>>>> +	return ret;
>>>> +}
>>>> +
>>>> +size_t get_crash_memory_size(void)
>>>> +{
>>>> +	size_t size;
>>>> +	if (!mutex_trylock(&kexec_mutex))
>>>> +		return 1;
>>>>     
>>>>         
>>> We don't need trylock on this code path 
>>>
>>>   
>>>       
>> Hmm, crashk_res is a global struct, so other process can also
>> change it... but currently no process does that, right?
>>
>>     
>
> We still need the lock.  Just doing trylock doesn't instead
> of just sleeping doesn't seem to make any sense on these
> code paths.
>
>   

Ok, got it.

>>>   
>>>       
>>>> +	start = crashk_res.start;
>>>> +	end = crashk_res.end;
>>>> +
>>>> +	if (new_size >= end - start + 1) {
>>>> +		ret = -EINVAL;
>>>> +		if (new_size == end - start + 1)
>>>> +			ret = 0;
>>>> +		goto unlock;
>>>> +	}
>>>> +
>>>> +	start = roundup(start, PAGE_SIZE);
>>>> +	end = roundup(start + new_size, PAGE_SIZE) - 1;
>>>> +	npages = (end + 1 - start ) / PAGE_SIZE;
>>>> +
>>>> +	pages = kmalloc(sizeof(struct page *) * npages, GFP_KERNEL);
>>>> +	if (!pages) {
>>>> +		ret = -ENOMEM;
>>>> +		goto unlock;
>>>> +	}
>>>> +	for (i = 0; i < npages; i++) {
>>>> +		addr = end + 1 + i * PAGE_SIZE;
>>>> +		pages[i] = virt_to_page(addr);
>>>> +	}
>>>> +
>>>> +	vaddr = vm_map_ram(pages, npages, 0, PAGE_KERNEL);
>>>>     
>>>>         
>>> This is the wrong kernel call to use.  I expect this needs to look
>>> like a memory hotplug event.  This does not put the pages into the
>>> free page pool.
>>>   
>>>       
>> Well, I also wanted to use an memory-hotplug API, but that will make the code
>> depend on memory-hotplug, which certainly is not what we want...
>>
>> I checked the mm code, actually what I need is an API which is similar to
>> add_active_range(), but add_active_range() can't be used here since it is marked
>> as "__init".
>>
>> Do we have that kind of API in mm? I can't find one.
>>     
>
> Perhaps we will need to remove __init from add_active_range.  I know the logic
> but I'm not up to speed on the mm pieces at the moment.
>   

Not that simple, marking it as "__init" means it uses some "__init" data 
which will be dropped after initialization.

Thanks.

Re: [Patch 8/8] kexec: allow to shrink reserved memory

[Eric W. Biederman]

Amerigo Wang <amwang@redhat.com> writes:

> Not that simple, marking it as "__init" means it uses some "__init" data which
> will be dropped after initialization.

If we start with the assumption that we will be reserving to much and
will free the memory once we know how much we really need I see a very
simple way to go about this. We ensure that the reservation of crash
kernel memory is done through a normal allocation so that we have
struct page entries for every page.  On 32bit x86 that is an extra 1MB
for a 128MB allocation.

Then when it comes time to release that memory we clear whatever magic
flags we have on the page (like PG_reserve) and call free_page.

Eric

Re: [Patch 8/8] kexec: allow to shrink reserved memory

[Amerigo Wang]

Eric W. Biederman wrote:
> Amerigo Wang <amwang@redhat.com> writes:
>
>   
>> Not that simple, marking it as "__init" means it uses some "__init" data which
>> will be dropped after initialization.
>>     
>
> If we start with the assumption that we will be reserving to much and
> will free the memory once we know how much we really need I see a very
> simple way to go about this. We ensure that the reservation of crash
> kernel memory is done through a normal allocation so that we have
> struct page entries for every page.  On 32bit x86 that is an extra 1MB
> for a 128MB allocation.
>
> Then when it comes time to release that memory we clear whatever magic
> flags we have on the page (like PG_reserve) and call free_page.
>   

Hmm, my MM knowledge is not good enough to judge if this works...
I need to check more MM source code.

Can any MM people help?

Thanks.

Re: [Patch 8/8] kexec: allow to shrink reserved memory

[KAMEZAWA Hiroyuki]

On Mon, 17 Aug 2009 17:50:21 +0800
Amerigo Wang <amwang@redhat.com> wrote:

> Eric W. Biederman wrote:
> > Amerigo Wang <amwang@redhat.com> writes:
> >
> >   
> >> Not that simple, marking it as "__init" means it uses some "__init" data which
> >> will be dropped after initialization.
> >>     
> >
> > If we start with the assumption that we will be reserving to much and
> > will free the memory once we know how much we really need I see a very
> > simple way to go about this. We ensure that the reservation of crash
> > kernel memory is done through a normal allocation so that we have
> > struct page entries for every page.  On 32bit x86 that is an extra 1MB
> > for a 128MB allocation.
> >
> > Then when it comes time to release that memory we clear whatever magic
> > flags we have on the page (like PG_reserve) and call free_page.
> >   
> 
> Hmm, my MM knowledge is not good enough to judge if this works...
> I need to check more MM source code.
> 
> Can any MM people help?
> 
Hm, memory-hotplug guy is here.

Can I have a question ?

  - How crash kernel's memory is preserved at boot ?
    It's hidden from the system before mem_init() ?

Thanks,
-Kame

Re: [Patch 8/8] kexec: allow to shrink reserved memory

[Amerigo Wang]

KAMEZAWA Hiroyuki wrote:
> On Mon, 17 Aug 2009 17:50:21 +0800
> Amerigo Wang <amwang@redhat.com> wrote:
>
>   
>> Eric W. Biederman wrote:
>>     
>>> Amerigo Wang <amwang@redhat.com> writes:
>>>
>>>   
>>>       
>>>> Not that simple, marking it as "__init" means it uses some "__init" data which
>>>> will be dropped after initialization.
>>>>     
>>>>         
>>> If we start with the assumption that we will be reserving to much and
>>> will free the memory once we know how much we really need I see a very
>>> simple way to go about this. We ensure that the reservation of crash
>>> kernel memory is done through a normal allocation so that we have
>>> struct page entries for every page.  On 32bit x86 that is an extra 1MB
>>> for a 128MB allocation.
>>>
>>> Then when it comes time to release that memory we clear whatever magic
>>> flags we have on the page (like PG_reserve) and call free_page.
>>>   
>>>       
>> Hmm, my MM knowledge is not good enough to judge if this works...
>> I need to check more MM source code.
>>
>> Can any MM people help?
>>
>>     
> Hm, memory-hotplug guy is here.
>   

Hi, thank you!
> Can I have a question ?
>
>   - How crash kernel's memory is preserved at boot ?
>   

Use bootmem, I think.

>     It's hidden from the system before mem_init() ?
>   

Not sure, but probably yes. It is reserved in setup_arch() which is 
before mm_init() which calls mem_init().

Do you have any advice to free that reserved memory after boot? :)

Thanks.

Re: [Patch 8/8] kexec: allow to shrink reserved memory

[KAMEZAWA Hiroyuki]

On Tue, 18 Aug 2009 14:31:23 +0800
Amerigo Wang <amwang@redhat.com> wrote:
> Hi, thank you!
> > Can I have a question ?
> >
> >   - How crash kernel's memory is preserved at boot ?
> >   
> 
> Use bootmem, I think.
> 
I see.

In x86,
 
  setup_arch()
	-> reserve_crashkernel()
		-> find_and_reserve_crashkernel()
			-> reserve_bootmem_generic()

Then, all "active range" is already registered and there are memmap.


> >     It's hidden from the system before mem_init() ?
> >   
> 
> Not sure, but probably yes. It is reserved in setup_arch() which is 
> before mm_init() which calls mem_init().
> 
> Do you have any advice to free that reserved memory after boot? :)
> 

Let's see arch/x86/mm/init.c::free_initmem()

Maybe it's all you want.

	- ClearPageReserved()
	- init_page_count()
	- free_page()
	- totalram_pages++

But it has no argumetns. Maybe you need your own function or modification.
online_pages() does very similar. But, hmm,.. writing something open coded one
for crashkernel is not very bad, I think.

Thanks,
-Kame

Re: [Patch 8/8] kexec: allow to shrink reserved memory

[Amerigo Wang]

KAMEZAWA Hiroyuki wrote:
> On Tue, 18 Aug 2009 14:31:23 +0800
> Amerigo Wang <amwang@redhat.com> wrote:
>   
>> Hi, thank you!
>>     
>>> Can I have a question ?
>>>
>>>   - How crash kernel's memory is preserved at boot ?
>>>   
>>>       
>> Use bootmem, I think.
>>
>>     
> I see.
>
> In x86,
>  
>   setup_arch()
> 	-> reserve_crashkernel()
> 		-> find_and_reserve_crashkernel()
> 			-> reserve_bootmem_generic()
>
> Then, all "active range" is already registered and there are memmap.
>
>
>   
>>>     It's hidden from the system before mem_init() ?
>>>   
>>>       
>> Not sure, but probably yes. It is reserved in setup_arch() which is 
>> before mm_init() which calls mem_init().
>>
>> Do you have any advice to free that reserved memory after boot? :)
>>
>>     
>
> Let's see arch/x86/mm/init.c::free_initmem()
>
> Maybe it's all you want.
>
> 	- ClearPageReserved()
> 	- init_page_count()
> 	- free_page()
> 	- totalram_pages++
>
> But it has no argumetns. Maybe you need your own function or modification.
> online_pages() does very similar. But, hmm,.. writing something open coded one
> for crashkernel is not very bad, I think.
>   

Nice help!

Yeah, I think we can make that be a generic wrapper function so that 
both free_initmem() and shrink_crash_memory() can use it.

Then I will update and resend the whole patchset.

Thank you!

Re: [Patch 8/8] kexec: allow to shrink reserved memory

[Amerigo Wang]

KAMEZAWA Hiroyuki wrote:
> On Tue, 18 Aug 2009 14:31:23 +0800
> Amerigo Wang <amwang@redhat.com> wrote:
>   
>>>     It's hidden from the system before mem_init() ?
>>>   
>>>       
>> Not sure, but probably yes. It is reserved in setup_arch() which is 
>> before mm_init() which calls mem_init().
>>
>> Do you have any advice to free that reserved memory after boot? :)
>>
>>     
>
> Let's see arch/x86/mm/init.c::free_initmem()
>
> Maybe it's all you want.
>
> 	- ClearPageReserved()
> 	- init_page_count()
> 	- free_page()
> 	- totalram_pages++
>   

Just FYI: calling ClearPageReserved() caused an oops: "Unable to handle 
paging request".

I am trying to figure out why...

Re: [Patch 8/8] kexec: allow to shrink reserved memory

[KAMEZAWA Hiroyuki]

On Tue, 18 Aug 2009 18:35:32 +0800
Amerigo Wang <amwang@redhat.com> wrote:

> KAMEZAWA Hiroyuki wrote:
> > On Tue, 18 Aug 2009 14:31:23 +0800
> > Amerigo Wang <amwang@redhat.com> wrote:
> >   
> >>>     It's hidden from the system before mem_init() ?
> >>>   
> >>>       
> >> Not sure, but probably yes. It is reserved in setup_arch() which is 
> >> before mm_init() which calls mem_init().
> >>
> >> Do you have any advice to free that reserved memory after boot? :)
> >>
> >>     
> >
> > Let's see arch/x86/mm/init.c::free_initmem()
> >
> > Maybe it's all you want.
> >
> > 	- ClearPageReserved()
> > 	- init_page_count()
> > 	- free_page()
> > 	- totalram_pages++
> >   
> 
> Just FYI: calling ClearPageReserved() caused an oops: "Unable to handle 
> paging request".
> 
> I am trying to figure out why...
> 
Hmm...then....memmap is not there.
pfn_valid() check will help you. What arch ? x86-64 ?

Thanks,
-Kame

Re: [Patch 8/8] kexec: allow to shrink reserved memory

[Amerigo Wang]

KAMEZAWA Hiroyuki wrote:
> On Tue, 18 Aug 2009 18:35:32 +0800
> Amerigo Wang <amwang@redhat.com> wrote:
>
>   
>> KAMEZAWA Hiroyuki wrote:
>>     
>>> On Tue, 18 Aug 2009 14:31:23 +0800
>>> Amerigo Wang <amwang@redhat.com> wrote:
>>>   
>>>       
>>>>>     It's hidden from the system before mem_init() ?
>>>>>   
>>>>>       
>>>>>           
>>>> Not sure, but probably yes. It is reserved in setup_arch() which is 
>>>> before mm_init() which calls mem_init().
>>>>
>>>> Do you have any advice to free that reserved memory after boot? :)
>>>>
>>>>     
>>>>         
>>> Let's see arch/x86/mm/init.c::free_initmem()
>>>
>>> Maybe it's all you want.
>>>
>>> 	- ClearPageReserved()
>>> 	- init_page_count()
>>> 	- free_page()
>>> 	- totalram_pages++
>>>   
>>>       
>> Just FYI: calling ClearPageReserved() caused an oops: "Unable to handle 
>> paging request".
>>
>> I am trying to figure out why...
>>
>>     
> Hmm...then....memmap is not there.
> pfn_valid() check will help you. What arch ? x86-64 ?
>   

Hmm, yes, x86_64, but this code is arch-independent, I mean it should 
work or not work on all arch, no?

So I am afraid we need to use other API to free it...

Thanks.

Re: [Patch 8/8] kexec: allow to shrink reserved memory

[KAMEZAWA Hiroyuki]

On Wed, 19 Aug 2009 10:41:13 +0800
Amerigo Wang <amwang@redhat.com> wrote:

> KAMEZAWA Hiroyuki wrote:
> > On Tue, 18 Aug 2009 18:35:32 +0800
> > Amerigo Wang <amwang@redhat.com> wrote:
> >
> >   
> >> KAMEZAWA Hiroyuki wrote:
> >>     
> >>> On Tue, 18 Aug 2009 14:31:23 +0800
> >>> Amerigo Wang <amwang@redhat.com> wrote:
> >>>   
> >>>       
> >>>>>     It's hidden from the system before mem_init() ?
> >>>>>   
> >>>>>       
> >>>>>           
> >>>> Not sure, but probably yes. It is reserved in setup_arch() which is 
> >>>> before mm_init() which calls mem_init().
> >>>>
> >>>> Do you have any advice to free that reserved memory after boot? :)
> >>>>
> >>>>     
> >>>>         
> >>> Let's see arch/x86/mm/init.c::free_initmem()
> >>>
> >>> Maybe it's all you want.
> >>>
> >>> 	- ClearPageReserved()
> >>> 	- init_page_count()
> >>> 	- free_page()
> >>> 	- totalram_pages++
> >>>   
> >>>       
> >> Just FYI: calling ClearPageReserved() caused an oops: "Unable to handle 
> >> paging request".
> >>
> >> I am trying to figure out why...
> >>
> >>     
> > Hmm...then....memmap is not there.
> > pfn_valid() check will help you. What arch ? x86-64 ?
> >   
> 
> Hmm, yes, x86_64, but this code is arch-independent, I mean it should 
> work or not work on all arch, no?
> 
> So I am afraid we need to use other API to free it...
> 
The, problem is whether memmap is there or not. That's all.
plz see init sequence and check there are memmap.
If memory-for-crash is obtained via bootmem,
Don't you try to free memory hole ?


Thanks,
-Kame

Re: [Patch 8/8] kexec: allow to shrink reserved memory

[Amerigo Wang]

KAMEZAWA Hiroyuki wrote:
> On Wed, 19 Aug 2009 10:41:13 +0800
> Amerigo Wang <amwang@redhat.com> wrote:
>
>   
>> KAMEZAWA Hiroyuki wrote:
>>     
>>> On Tue, 18 Aug 2009 18:35:32 +0800
>>> Amerigo Wang <amwang@redhat.com> wrote:
>>>
>>>   
>>>       
>>>> KAMEZAWA Hiroyuki wrote:
>>>>     
>>>>         
>>>>> On Tue, 18 Aug 2009 14:31:23 +0800
>>>>> Amerigo Wang <amwang@redhat.com> wrote:
>>>>>   
>>>>>       
>>>>>           
>>>>>>>     It's hidden from the system before mem_init() ?
>>>>>>>   
>>>>>>>       
>>>>>>>           
>>>>>>>               
>>>>>> Not sure, but probably yes. It is reserved in setup_arch() which is 
>>>>>> before mm_init() which calls mem_init().
>>>>>>
>>>>>> Do you have any advice to free that reserved memory after boot? :)
>>>>>>
>>>>>>     
>>>>>>         
>>>>>>             
>>>>> Let's see arch/x86/mm/init.c::free_initmem()
>>>>>
>>>>> Maybe it's all you want.
>>>>>
>>>>> 	- ClearPageReserved()
>>>>> 	- init_page_count()
>>>>> 	- free_page()
>>>>> 	- totalram_pages++
>>>>>   
>>>>>       
>>>>>           
>>>> Just FYI: calling ClearPageReserved() caused an oops: "Unable to handle 
>>>> paging request".
>>>>
>>>> I am trying to figure out why...
>>>>
>>>>     
>>>>         
>>> Hmm...then....memmap is not there.
>>> pfn_valid() check will help you. What arch ? x86-64 ?
>>>   
>>>       
>> Hmm, yes, x86_64, but this code is arch-independent, I mean it should 
>> work or not work on all arch, no?
>>
>> So I am afraid we need to use other API to free it...
>>
>>     
> The, problem is whether memmap is there or not. That's all.
> plz see init sequence and check there are memmap.
> If memory-for-crash is obtained via bootmem,
> Don't you try to free memory hole ?
>   

Yes, I am checking the code. Thanks!

Re: [Patch 8/8] kexec: allow to shrink reserved memory

[Amerigo Wang]

KAMEZAWA Hiroyuki wrote:
> On Wed, 19 Aug 2009 10:41:13 +0800
> Amerigo Wang <amwang@redhat.com> wrote:
>
>   
>> KAMEZAWA Hiroyuki wrote:
>>     
>>> On Tue, 18 Aug 2009 18:35:32 +0800
>>> Amerigo Wang <amwang@redhat.com> wrote:
>>>
>>>   
>>>       
>>>> KAMEZAWA Hiroyuki wrote:
>>>>     
>>>>         
>>>>> On Tue, 18 Aug 2009 14:31:23 +0800
>>>>> Amerigo Wang <amwang@redhat.com> wrote:
>>>>>   
>>>>>       
>>>>>           
>>>>>>>     It's hidden from the system before mem_init() ?
>>>>>>>   
>>>>>>>       
>>>>>>>           
>>>>>>>               
>>>>>> Not sure, but probably yes. It is reserved in setup_arch() which is 
>>>>>> before mm_init() which calls mem_init().
>>>>>>
>>>>>> Do you have any advice to free that reserved memory after boot? :)
>>>>>>
>>>>>>     
>>>>>>         
>>>>>>             
>>>>> Let's see arch/x86/mm/init.c::free_initmem()
>>>>>
>>>>> Maybe it's all you want.
>>>>>
>>>>> 	- ClearPageReserved()
>>>>> 	- init_page_count()
>>>>> 	- free_page()
>>>>> 	- totalram_pages++
>>>>>   
>>>>>       
>>>>>           
>>>> Just FYI: calling ClearPageReserved() caused an oops: "Unable to handle 
>>>> paging request".
>>>>
>>>> I am trying to figure out why...
>>>>
>>>>     
>>>>         
>>> Hmm...then....memmap is not there.
>>> pfn_valid() check will help you. What arch ? x86-64 ?
>>>   
>>>       
>> Hmm, yes, x86_64, but this code is arch-independent, I mean it should 
>> work or not work on all arch, no?
>>
>> So I am afraid we need to use other API to free it...
>>
>>     
> The, problem is whether memmap is there or not. That's all.
> plz see init sequence and check there are memmap.
> If memory-for-crash is obtained via bootmem,
> Don't you try to free memory hole ?
>   

Hi,

It looks like that mem_map has 'struct page' for the reserved memory, I 
checked my "early_node_map[] active PFN ranges" output, the reserved 
memory area for crash kernel is right in one range. Am I missing 
something here?

I don't know why that oops comes out, maybe because of no PTE for thoese 
pages?

Thanks.

Re: [Patch 8/8] kexec: allow to shrink reserved memory

[KAMEZAWA Hiroyuki]

On Thu, 20 Aug 2009 17:15:56 +0800
Amerigo Wang <amwang@redhat.com> wrote:
    
> > The, problem is whether memmap is there or not. That's all.
> > plz see init sequence and check there are memmap.
> > If memory-for-crash is obtained via bootmem,
> > Don't you try to free memory hole ?
> >   
> 
> Hi,
> 
> It looks like that mem_map has 'struct page' for the reserved memory, I 
> checked my "early_node_map[] active PFN ranges" output, the reserved 
> memory area for crash kernel is right in one range. Am I missing 
> something here?
> 
> I don't know why that oops comes out, maybe because of no PTE for thoese 
> pages?
> 
Hmm ? Could you show me the code you use ?

Thanks,
-Kame

Re: [Patch 8/8] kexec: allow to shrink reserved memory

[Amerigo Wang]

On Fri, Aug 21, 2009 at 09:34:52AM +0900, KAMEZAWA Hiroyuki wrote:
>On Thu, 20 Aug 2009 17:15:56 +0800
>Amerigo Wang <amwang@redhat.com> wrote:
>    
>> > The, problem is whether memmap is there or not. That's all.
>> > plz see init sequence and check there are memmap.
>> > If memory-for-crash is obtained via bootmem,
>> > Don't you try to free memory hole ?
>> >   
>> 
>> Hi,
>> 
>> It looks like that mem_map has 'struct page' for the reserved memory, I 
>> checked my "early_node_map[] active PFN ranges" output, the reserved 
>> memory area for crash kernel is right in one range. Am I missing 
>> something here?
>> 
>> I don't know why that oops comes out, maybe because of no PTE for thoese 
>> pages?
>> 
>Hmm ? Could you show me the code you use ?

(Sorry that I reply to you with my gmail, my work email can't send out
this message, probably because one of the destinations is broken...
I am the same person, don't be confused. :-)

Sure. Below is it:

+    for (addr = end + 1; addr < crashk_res.end; addr += PAGE_SIZE) {
+        printk(KERN_DEBUG "PFN is valid? %d\n", pfn_valid(addr>>PAGE_SHIFT));
+        ClearPageReserved(virt_to_page(addr));
+        init_page_count(virt_to_page(addr));
+        free_page(addr);
+        totalram_pages++;
+    }


pfn_valid() returns 1, and oops happens at ClearPageReserved().
('addr' is right between crashk_res.start and crashk_res.end)

Thank you!

Re: [Patch 8/8] kexec: allow to shrink reserved memory

[KAMEZAWA Hiroyuki]

On Fri, 21 Aug 2009 09:59:27 +0800
Amerigo Wang <xiyou.wangcong@gmail.com> wrote:

> On Fri, Aug 21, 2009 at 09:34:52AM +0900, KAMEZAWA Hiroyuki wrote:
> >On Thu, 20 Aug 2009 17:15:56 +0800
> >Amerigo Wang <amwang@redhat.com> wrote:
> >    
> >> > The, problem is whether memmap is there or not. That's all.
> >> > plz see init sequence and check there are memmap.
> >> > If memory-for-crash is obtained via bootmem,
> >> > Don't you try to free memory hole ?
> >> >   
> >> 
> >> Hi,
> >> 
> >> It looks like that mem_map has 'struct page' for the reserved memory, I 
> >> checked my "early_node_map[] active PFN ranges" output, the reserved 
> >> memory area for crash kernel is right in one range. Am I missing 
> >> something here?
> >> 
> >> I don't know why that oops comes out, maybe because of no PTE for thoese 
> >> pages?
> >> 
> >Hmm ? Could you show me the code you use ?
> 
> (Sorry that I reply to you with my gmail, my work email can't send out
> this message, probably because one of the destinations is broken...
> I am the same person, don't be confused. :-)
> 
> Sure. Below is it:
> 
> +    for (addr = end + 1; addr < crashk_res.end; addr += PAGE_SIZE) {
> +        printk(KERN_DEBUG "PFN is valid? %d\n", pfn_valid(addr>>PAGE_SHIFT));
> +        ClearPageReserved(virt_to_page(addr));
> +        init_page_count(virt_to_page(addr));
> +        free_page(addr);
> +        totalram_pages++;
> +    }
> 
> 
> pfn_valid() returns 1, and oops happens at ClearPageReserved().
> ('addr' is right between crashk_res.start and crashk_res.end)

Confused, 
  if    pfn_valid(addr >> PAGE_SHIFT) == true 

you should do
	ClearPageReserved(pfn_to_page(addr >> PAGE_SHIFT));

because addr is physical address, not virtual.
I guess crashk_res.end is physical address....

Thanks,
-Kame


> 
> Thank you!
> 

Re: [Patch 8/8] kexec: allow to shrink reserved memory

[Amerigo Wang]

On Fri, Aug 21, 2009 at 11:03:09AM +0900, KAMEZAWA Hiroyuki wrote:
>On Fri, 21 Aug 2009 09:59:27 +0800
>Amerigo Wang <xiyou.wangcong@gmail.com> wrote:
>
>> On Fri, Aug 21, 2009 at 09:34:52AM +0900, KAMEZAWA Hiroyuki wrote:
>> >On Thu, 20 Aug 2009 17:15:56 +0800
>> >Amerigo Wang <amwang@redhat.com> wrote:
>> >    
>> >> > The, problem is whether memmap is there or not. That's all.
>> >> > plz see init sequence and check there are memmap.
>> >> > If memory-for-crash is obtained via bootmem,
>> >> > Don't you try to free memory hole ?
>> >> >   
>> >> 
>> >> Hi,
>> >> 
>> >> It looks like that mem_map has 'struct page' for the reserved memory, I 
>> >> checked my "early_node_map[] active PFN ranges" output, the reserved 
>> >> memory area for crash kernel is right in one range. Am I missing 
>> >> something here?
>> >> 
>> >> I don't know why that oops comes out, maybe because of no PTE for thoese 
>> >> pages?
>> >> 
>> >Hmm ? Could you show me the code you use ?
>> 
>> (Sorry that I reply to you with my gmail, my work email can't send out
>> this message, probably because one of the destinations is broken...
>> I am the same person, don't be confused. :-)
>> 
>> Sure. Below is it:
>> 
>> +    for (addr = end + 1; addr < crashk_res.end; addr += PAGE_SIZE) {
>> +        printk(KERN_DEBUG "PFN is valid? %d\n", pfn_valid(addr>>PAGE_SHIFT));
>> +        ClearPageReserved(virt_to_page(addr));
>> +        init_page_count(virt_to_page(addr));
>> +        free_page(addr);
>> +        totalram_pages++;
>> +    }
>> 
>> 
>> pfn_valid() returns 1, and oops happens at ClearPageReserved().
>> ('addr' is right between crashk_res.start and crashk_res.end)
>
>Confused, 
>  if    pfn_valid(addr >> PAGE_SHIFT) == true 
>
>you should do
>	ClearPageReserved(pfn_to_page(addr >> PAGE_SHIFT));
>
>because addr is physical address, not virtual.
>I guess crashk_res.end is physical address....

Excellent! You are right!

In fact, when I read the kexec code at the first time, I thought
'crashk_res' should hold physical address too, but after reading
more code I dropped that idea, so I am wrong. :-/

I will resend the whole patchset soon. It works now!

Thanks for your nice help, Hiroyuki!

Re: [Patch 0/8] V3 Implement crashkernel=auto

[Bernhard Walle]

* Amerigo Wang <amwang@redhat.com> [2009-08-12 10:15]:
> 
> This series of patch implements automatically reserved memory for crashkernel,
> by introducing a new boot option "crashkernel=auto". This idea is from Neil.

Honestly I don't see why everything is guarded by
CONFIG_KEXEC_AUTO_RESERVE. We do we need that new configuration
option? I mean, if I don't specify 'crashkernel=auto', then the patch
does nothing, right? Then the option CONFIG_KEXEC_AUTO_RESERVE would
only be needed so save some bytes of code. Is that really worth it?


Regards,
Bernhard

Re: [Patch 0/8] V3 Implement crashkernel=auto

[Amerigo Wang]

Bernhard Walle wrote:
> * Amerigo Wang <amwang@redhat.com> [2009-08-12 10:15]:
>   
>> This series of patch implements automatically reserved memory for crashkernel,
>> by introducing a new boot option "crashkernel=auto". This idea is from Neil.
>>     
>
> Honestly I don't see why everything is guarded by
> CONFIG_KEXEC_AUTO_RESERVE. We do we need that new configuration
> option? I mean, if I don't specify 'crashkernel=auto', then the patch
> does nothing, right? Then the option CONFIG_KEXEC_AUTO_RESERVE would
> only be needed so save some bytes of code. Is that really worth it?
>   

Hi,  CONFIG_KEXEC_AUTO_RESERVE is not for saving bytes, it just provides 
a choice for the user to decide to enable it or not.

Thanks.

Re: [Patch 0/8] V3 Implement crashkernel=auto

[Bernhard Walle]

Hi,

* Amerigo Wang <amwang@redhat.com> [2009-08-13 04:49]:
> Bernhard Walle wrote:
> >
> >Honestly I don't see why everything is guarded by
> >CONFIG_KEXEC_AUTO_RESERVE. We do we need that new configuration
> >option? I mean, if I don't specify 'crashkernel=auto', then the patch
> >does nothing, right? Then the option CONFIG_KEXEC_AUTO_RESERVE would
> >only be needed so save some bytes of code. Is that really worth it?
> 
> Hi,  CONFIG_KEXEC_AUTO_RESERVE is not for saving bytes, it just
> provides a choice for the user to decide to enable it or not.

Still, I don't understand it. When I don't say "crashkernel=auto" on
command line, then nothing is done, right? So the choice for the user
is the "crashkernel=auto". Why do we need CONFIG_KEXEC_AUTO_RESERVE
then? Maybe I just missed something in my logic ...


Regards,
Bernhard

Re: [Patch 0/8] V3 Implement crashkernel=auto

[Amerigo Wang]

Bernhard Walle wrote:
> Hi,
>
> * Amerigo Wang <amwang@redhat.com> [2009-08-13 04:49]:
>   
>> Bernhard Walle wrote:
>>     
>>> Honestly I don't see why everything is guarded by
>>> CONFIG_KEXEC_AUTO_RESERVE. We do we need that new configuration
>>> option? I mean, if I don't specify 'crashkernel=auto', then the patch
>>> does nothing, right? Then the option CONFIG_KEXEC_AUTO_RESERVE would
>>> only be needed so save some bytes of code. Is that really worth it?
>>>       
>> Hi,  CONFIG_KEXEC_AUTO_RESERVE is not for saving bytes, it just
>> provides a choice for the user to decide to enable it or not.
>>     
>
> Still, I don't understand it. When I don't say "crashkernel=auto" on
> command line, then nothing is done, right? So the choice for the user
> is the "crashkernel=auto". Why do we need CONFIG_KEXEC_AUTO_RESERVE
> then? Maybe I just missed something in my logic ...
>   
Sure.

But if we disable CONFIG_KEXEC_AUTO_RESERVE, that means crashkernel=auto 
will be invalid, this is the same as it is now.

Re: [Patch 0/8] V3 Implement crashkernel=auto

[Bernhard Walle]

* Amerigo Wang <amwang@redhat.com> [2009-08-13 10:19]:
> Sure.
> 
> But if we disable CONFIG_KEXEC_AUTO_RESERVE, that means
> crashkernel=auto will be invalid, this is the same as it is now.

Ok, but since 'crashkernel=auto' is not used today, nobody has
'crashkernel=auto' in the bootloader configuration. So I don't see any
practial advantage of that config option.

Eric, what's your opinion on that, do we need a config option
CONFIG_KEXEC_AUTO_RESERVE or could we just implement that feature
unconditionally (if CONFIG_KEXEC is enabled, of course).


Regards,
Bernhard

Re: [Patch 0/8] V3 Implement crashkernel=auto

[Eric W. Biederman]

Bernhard Walle <bernhard.walle@gmx.de> writes:

> * Amerigo Wang <amwang@redhat.com> [2009-08-13 10:19]:
>> Sure.
>> 
>> But if we disable CONFIG_KEXEC_AUTO_RESERVE, that means
>> crashkernel=auto will be invalid, this is the same as it is now.
>
> Ok, but since 'crashkernel=auto' is not used today, nobody has
> 'crashkernel=auto' in the bootloader configuration. So I don't see any
> practial advantage of that config option.
>
> Eric, what's your opinion on that, do we need a config option
> CONFIG_KEXEC_AUTO_RESERVE or could we just implement that feature
> unconditionally (if CONFIG_KEXEC is enabled, of course).

The only reason I can see the option going away would be
a dependency on CONFIG_HOTPLUG_MEM.

Eric

Re: [Patch 0/8] V3 Implement crashkernel=auto

[Amerigo Wang]

Eric W. Biederman wrote:
> Bernhard Walle <bernhard.walle@gmx.de> writes:
>
>   
>> * Amerigo Wang <amwang@redhat.com> [2009-08-13 10:19]:
>>     
>>> Sure.
>>>
>>> But if we disable CONFIG_KEXEC_AUTO_RESERVE, that means
>>> crashkernel=auto will be invalid, this is the same as it is now.
>>>       
>> Ok, but since 'crashkernel=auto' is not used today, nobody has
>> 'crashkernel=auto' in the bootloader configuration. So I don't see any
>> practial advantage of that config option.
>>
>> Eric, what's your opinion on that, do we need a config option
>> CONFIG_KEXEC_AUTO_RESERVE or could we just implement that feature
>> unconditionally (if CONFIG_KEXEC is enabled, of course).
>>     
>
> The only reason I can see the option going away would be
> a dependency on CONFIG_HOTPLUG_MEM.
>   

I am not sure if I understand you correctly, but it doesn't and won't 
depend on CONFIG_MEMORY_HOTPLUG.

Thanks.