[RFC]Kexec based crash dumping

[RFC]Kexec based crash dumping

[Hariprasad Nellitheertha]

Hi,

The patches that follow contain the initial implementation for kexec based
crash dumping that we are working on. I had sent this to the fastboot mailing 
list a couple of weeks ago and this set of patches includes the changes made as
per feedback from Andrew, Eric and others.

Main Idea

- Whenever a panic occurs, reboot to a new kernel using kexec using a small
  amount of memory (16MB). The rest of the memory is preserved across the
  reboot.
- In the second kernel, the memory contents from the failed kernel is 
  available as an ELF format file for write-out.

Details on the design and implementation and on how to setup this facility
are available in the first of the patches that follow. The patches have
been made for the 2.6.8.1 kernel.

Kindly review these patches and provide feedback.

Thanks to Martin Bligh and Suparna for the design ideas and to Adam Litke
who hacked up most of the memory preserving reboot code and the dump
device abstraction code.

Regards, Hari
-- 
Hariprasad Nellitheertha
Linux Technology Center
India Software Labs
IBM India, Bangalore

Re: [PATCH][1/6]Documentation

[Hariprasad Nellitheertha]

[-- Attachment #1: Type: text/plain, Size: 108 bytes --]

Regards, Hari
-- 
Hariprasad Nellitheertha
Linux Technology Center
India Software Labs
IBM India, Bangalore

[-- Attachment #2: kd-doc-268.patch --]
[-- Type: text/plain, Size: 4190 bytes --]



This patch contains the documentation for the kexec based crash dump tool.

Signed off by Hariprasad Nellitheertha <hari@in.ibm.com>



---

 linux-2.6.8.1-hari/Documentation/kdump.txt |  101 +++++++++++++++++++++++++++++
 1 files changed, 101 insertions(+)

diff -puN /dev/null Documentation/kdump.txt
--- /dev/null	2003-01-30 15:54:37.000000000 +0530
+++ linux-2.6.8.1-hari/Documentation/kdump.txt	2004-08-17 17:03:19.000000000 +0530
@@ -0,0 +1,101 @@
+Documentation for kdump - the kexec based crash dumping solution
+================================================================
+
+DESIGN
+======
+
+We use kexec to reboot to a second kernel whenever a dump needs to be taken.
+This second kernel is booted with with very little memory (current
+implementation has this at 16MB). The contents of the 16MB of memory that
+the second kernel uses is copied onto a reserved area, before rebooting. So,
+we have the entire memory image of the previous kernel preserved.
+
+In the second kernel, this "old memory" can be accessed in two ways. The
+first one is through a device interface. We can create a /dev/hmem or
+whatever and write out the memory in raw format. The second interface is
+through /proc/vmcore. This exports the dump as an ELF format file which
+can be written out using any file copy command (cp, scp, etc). Further, gdb
+can be used to perform some minimal debugging on the dump file.
+
+Note that the two approaches are independent and the patches
+can be used depending on the functionality needed. More details on the
+patches below.
+
+PATCHES
+=======
+
+We currently have 6 patches.
+
+1) kd-doc-<version>.patch - Contains basic documentation (this document!!)
+2) kd-reb-<version>.patch - This patch ensures we do a kexec reboot upon panic
+   and also saves the first 16MB of memory into a backup area
+3) kd-copy-<version>.patch - This contains the code for reading the dump pages
+   in the second kernel.
+4) kd-reg-<version>.patch - This patch is for snapshotting the register contents
+   of all processors on to the backup area before rebooting.
+5) kd-elf-<version>.patch - This patch provides an ELF format interface to
+   the dump, post-reboot.
+6) kd-hmem-<version>.patch - This patch contains the code to access the dump as
+   an /dev/hmem.
+
+SETUP
+=====
+
+1) Apply the kexec patch on to the appropriate vanilla kernel tree.
+   The latest version of the kexec patch can be obtained from
+   http://www.xmission.com/~ebiederm/files/kexec/2.6.8-rc1-kexec1/
+   This patch should apply on 2.6.8-rc4 as well.
+   Older kexec patches can be found at
+   http://developer.osdl.org/rddunlap/kexec/
+
+2) Apply the crash dump patches.
+
+3) Ensure you have chosen the "kernel crash dumps" option under "Processor
+   type and features" and also the kexec syscall option under kernel hacking
+
+4) Load the second kernel to be booted using
+
+   kexec -l <kernel> --append="root=<root-dev> mem=16M dump init 1"
+
+5) System reboots into the second kernel when a panic occurs.
+   You could write a module to call panic, for testing purposes.
+
+6) Write out the dump file using
+
+   cp /proc/vmcore <dump-file>
+
+You can also access the dump as a device for a linear/raw view. To do this,
+you will need the kd-hmem-<version>.patch built into the kernel. To create
+the device, type
+
+  mknod /dev/hmem c 1 12
+
+Use "dd" with suitable options for count, bs and skip to access specific
+portions of the dump.
+
+ANALYSIS
+========
+
+You can run gdb on the dump file copied out of /proc/vmcore. Use vmlinux built
+with -g and run
+
+  gdb vmlinux <dump-file>
+
+Stack trace for the task on processor 0, register display, memory display
+work fine.
+
+TODO
+====
+
+1) Provide a kernel-pages only view for the dump. This could possibly turn up
+   as /proc/vmcore-kern.
+2) Provide register contents of all processors (similar to what multi-threaded
+   core dumps does).
+3) Modify "crash" to make it recognize this dump.
+4) Make the i386 kernel boot from any location so we can run the second kernel
+   from the reserved location instead of the current approach.
+
+CONTACT
+=======
+
+Hariprasad Nellitheertha - hari at in dot ibm dot com

_

Re: [PATCH][2/6]Memory preserving reboot using kexec

[Hariprasad Nellitheertha]

[-- Attachment #1: Type: text/plain, Size: 108 bytes --]

Regards, Hari
-- 
Hariprasad Nellitheertha
Linux Technology Center
India Software Labs
IBM India, Bangalore

[-- Attachment #2: kd-reb-268.patch --]
[-- Type: text/plain, Size: 8996 bytes --]



This patch contains the code that does the memory preserving reboot. It 
copies over the first CRASH_BACKUP_SIZE amount of memory into a backup
region before handing over to kexec.

Signed off by Hariprasad Nellitheertha <hari@in.ibm.com>
Signed off by Adam Litke <litke@us.ibm.com>


---

 linux-2.6.8.1-hari/arch/i386/Kconfig                |   22 +++++++++++
 linux-2.6.8.1-hari/arch/i386/kernel/machine_kexec.c |   31 +++++++++++++++
 linux-2.6.8.1-hari/arch/i386/kernel/setup.c         |   11 +++++
 linux-2.6.8.1-hari/include/asm-i386/crash_dump.h    |   39 ++++++++++++++++++++
 linux-2.6.8.1-hari/include/linux/bootmem.h          |    1 
 linux-2.6.8.1-hari/include/linux/crash_dump.h       |   27 +++++++++++++
 linux-2.6.8.1-hari/kernel/panic.c                   |    6 +++
 linux-2.6.8.1-hari/mm/bootmem.c                     |    5 ++
 8 files changed, 142 insertions(+)

diff -puN arch/i386/Kconfig~kd-reb-268 arch/i386/Kconfig
--- linux-2.6.8.1/arch/i386/Kconfig~kd-reb-268	2004-08-17 17:04:35.000000000 +0530
+++ linux-2.6.8.1-hari/arch/i386/Kconfig	2004-08-17 17:05:03.000000000 +0530
@@ -865,6 +865,28 @@ config REGPARM
 	generate incorrect output with certain kernel constructs when
 	-mregparm=3 is used.
 
+config CRASH_DUMP
+	bool "kernel crash dumps (EXPERIMENTAL)"
+	depends on KEXEC
+	help
+	  Generate crash dump using kexec.
+
+config BACKUP_BASE
+	int "location of the crash dumps backup region"
+	depends on CRASH_DUMP
+	default 128
+	help
+	Offset of backup region in terms of MB. Change this if you want
+	to modify the location of the crash dumps backup region.
+
+config BACKUP_SIZE
+	int "Size of the crash dumps backup region"
+	depends on CRASH_DUMP
+	range 16 64
+	default 16
+	help
+	The size of the backup region, in MB. This is also the size of the
+	memory that will be used to boot the second kernel after a crash.
 endmenu
 
 
diff -puN arch/i386/kernel/machine_kexec.c~kd-reb-268 arch/i386/kernel/machine_kexec.c
--- linux-2.6.8.1/arch/i386/kernel/machine_kexec.c~kd-reb-268	2004-08-17 17:04:35.000000000 +0530
+++ linux-2.6.8.1-hari/arch/i386/kernel/machine_kexec.c	2004-08-17 17:05:03.000000000 +0530
@@ -9,6 +9,7 @@
 #include <linux/mm.h>
 #include <linux/kexec.h>
 #include <linux/delay.h>
+#include <linux/crash_dump.h>
 #include <asm/pgtable.h>
 #include <asm/pgalloc.h>
 #include <asm/tlbflush.h>
@@ -16,6 +17,7 @@
 #include <asm/io.h>
 #include <asm/apic.h>
 #include <asm/cpufeature.h>
+#include <asm/hw_irq.h>
 
 
 static void set_idt(void *newidt, __u16 limit)
@@ -76,6 +78,28 @@ const extern unsigned int relocate_new_k
 extern void use_mm(struct mm_struct *mm);
 
 /*
+ * We are going to do a memory preserving reboot. So, we copy over the
+ * first xxMB of memory into a backup location.
+ */
+void __relocate_base_mem(unsigned long backup_addr, unsigned long backup_size)
+{
+	unsigned long pfn, pfn_max;
+	void *src_addr, *dest_addr;
+	struct page *page;
+
+	pfn_max = backup_size >> PAGE_SHIFT;
+	for (pfn = 0; pfn < pfn_max; pfn++) {
+		src_addr = phys_to_virt(pfn << PAGE_SHIFT);
+		dest_addr = backup_addr + src_addr;
+		if (!pfn_valid(pfn))
+			continue;
+		page = pfn_to_page(pfn);
+		if (PageReserved(page))
+			copy_page(dest_addr, src_addr);
+	}
+}
+
+/*
  * Do not allocate memory (or fail in any way) in machine_kexec().
  * We are past the point of no return, committed to rebooting now.
  */
@@ -94,6 +118,13 @@ void machine_kexec(struct kimage *image)
 	reboot_code_buffer = page_to_pfn(image->reboot_code_pages) << PAGE_SHIFT;
 	indirection_page = image->head & PAGE_MASK;
 
+	/*
+	 * If we are here to do a crash dump, save the memory from
+	 * 0-16MB before we copy over the kexec kernel image.  Otherwise
+	 * our dump will show the wrong kernel entirely.
+	 */
+	relocate_base_mem();
+
 	/* copy it out */
 	memcpy((void *)reboot_code_buffer, relocate_new_kernel, relocate_new_kernel_size);
 
diff -puN arch/i386/kernel/setup.c~kd-reb-268 arch/i386/kernel/setup.c
--- linux-2.6.8.1/arch/i386/kernel/setup.c~kd-reb-268	2004-08-17 17:04:35.000000000 +0530
+++ linux-2.6.8.1-hari/arch/i386/kernel/setup.c	2004-08-17 17:05:03.000000000 +0530
@@ -39,6 +39,7 @@
 #include <linux/efi.h>
 #include <linux/init.h>
 #include <linux/edd.h>
+#include <linux/crash_dump.h>
 #include <video/edid.h>
 #include <asm/e820.h>
 #include <asm/mpspec.h>
@@ -56,6 +57,7 @@
 unsigned long init_pg_tables_end __initdata = ~0UL;
 
 int disable_pse __initdata = 0;
+unsigned int dump_enabled;
 
 /*
  * Machine setup..
@@ -347,6 +349,9 @@ static void __init limit_regions(unsigne
 			}
 		}
 	}
+	/* If we are doing a crash dump, we still need to know the real mem size*/
+	set_saved_max_pfn();
+
 	for (i = 0; i < e820.nr_map; i++) {
 		if (e820.map[i].type == E820_RAM) {
 			current_addr = e820.map[i].addr + e820.map[i].size;
@@ -809,6 +814,9 @@ static void __init parse_cmdline_early (
 		if (c == ' ' && !memcmp(from, "highmem=", 8))
 			highmem_pages = memparse(from+8, &from) >> PAGE_SHIFT;
 	
+		if (!memcmp(from, "dump", 4))
+			dump_enabled = 1;
+
 		c = *(from++);
 		if (!c)
 			break;
@@ -1082,6 +1090,9 @@ static unsigned long __init setup_memory
 		}
 	}
 #endif
+
+	crash_reserve_bootmem();
+
 	return max_low_pfn;
 }
 #else
diff -puN /dev/null include/asm-i386/crash_dump.h
--- /dev/null	2003-01-30 15:54:37.000000000 +0530
+++ linux-2.6.8.1-hari/include/asm-i386/crash_dump.h	2004-08-17 17:05:03.000000000 +0530
@@ -0,0 +1,39 @@
+/* asm-i386/crash_dump.h */
+#include <linux/bootmem.h>
+
+extern unsigned int dump_enabled;
+
+unsigned long __init find_max_low_pfn(void);
+void __init find_max_pfn(void);
+void __relocate_base_mem(unsigned long, unsigned long);
+
+extern unsigned int crashed;
+
+#ifdef CONFIG_CRASH_DUMP
+#define CRASH_BACKUP_BASE ((unsigned long)CONFIG_BACKUP_BASE * 0x100000)
+#define CRASH_BACKUP_SIZE ((unsigned long)CONFIG_BACKUP_SIZE * 0x100000)
+
+static inline void relocate_base_mem(void)
+{
+	if (crashed)
+		__relocate_base_mem(CRASH_BACKUP_BASE, CRASH_BACKUP_SIZE);
+}
+
+static inline void set_saved_max_pfn(void)
+{
+	find_max_pfn();
+	saved_max_pfn = find_max_low_pfn();
+}
+
+static inline void crash_reserve_bootmem(void)
+{
+	if (!dump_enabled)
+		reserve_bootmem(CRASH_BACKUP_BASE, CRASH_BACKUP_SIZE);
+}
+#else
+#define CRASH_BACKUP_BASE 0x08000000
+#define CRASH_BACKUP_SIZE 0x01000000
+static inline void relocate_base_mem(void) {}
+static inline void set_saved_max_pfn(void) {}
+static inline void crash_reserve_bootmem(void) {}
+#endif
diff -puN include/linux/bootmem.h~kd-reb-268 include/linux/bootmem.h
--- linux-2.6.8.1/include/linux/bootmem.h~kd-reb-268	2004-08-17 17:04:36.000000000 +0530
+++ linux-2.6.8.1-hari/include/linux/bootmem.h	2004-08-17 17:05:03.000000000 +0530
@@ -21,6 +21,7 @@ extern unsigned long min_low_pfn;
  * highest page
  */
 extern unsigned long max_pfn;
+extern unsigned long saved_max_pfn;
 
 /*
  * node_bootmem_map is a map pointer - the bits represent all physical 
diff -puN /dev/null include/linux/crash_dump.h
--- /dev/null	2003-01-30 15:54:37.000000000 +0530
+++ linux-2.6.8.1-hari/include/linux/crash_dump.h	2004-08-17 17:05:03.000000000 +0530
@@ -0,0 +1,27 @@
+#include <linux/kexec.h>
+#include <linux/smp_lock.h>
+#include <linux/device.h>
+#include <asm/crash_dump.h>
+
+#ifdef CONFIG_CRASH_DUMP
+static inline void crash_machine_kexec(void)
+{
+	struct kimage *image;
+
+	if (crashed)
+		return;
+
+        image = xchg(&kexec_image, 0);
+        if (image) {
+		crashed = 1;
+		device_shutdown();
+		printk(KERN_EMERG "kexec: opening parachute\n");
+		machine_kexec(image);
+		while (1);
+        } else {
+		printk(KERN_EMERG "kexec: No kernel image loaded!\n");
+        }
+}
+#else
+static inline void crash_machine_kexec(void) {}
+#endif
diff -puN kernel/panic.c~kd-reb-268 kernel/panic.c
--- linux-2.6.8.1/kernel/panic.c~kd-reb-268	2004-08-17 17:04:36.000000000 +0530
+++ linux-2.6.8.1-hari/kernel/panic.c	2004-08-17 17:05:03.000000000 +0530
@@ -19,10 +19,13 @@
 #include <linux/syscalls.h>
 #include <linux/interrupt.h>
 #include <linux/nmi.h>
+#include <linux/kexec.h>
+#include <linux/crash_dump.h>
 
 int panic_timeout;
 int panic_on_oops;
 int tainted;
+unsigned int crashed;
 
 EXPORT_SYMBOL(panic_timeout);
 
@@ -68,6 +71,9 @@ NORET_TYPE void panic(const char * fmt, 
 		sys_sync();
 	bust_spinlocks(0);
 
+	/* If we have crashed, perform a kexec reboot, for dump write-out */
+	crash_machine_kexec();
+
 #ifdef CONFIG_SMP
 	smp_send_stop();
 #endif
diff -puN mm/bootmem.c~kd-reb-268 mm/bootmem.c
--- linux-2.6.8.1/mm/bootmem.c~kd-reb-268	2004-08-17 17:04:36.000000000 +0530
+++ linux-2.6.8.1-hari/mm/bootmem.c	2004-08-17 17:05:03.000000000 +0530
@@ -27,6 +27,11 @@
 unsigned long max_low_pfn;
 unsigned long min_low_pfn;
 unsigned long max_pfn;
+/*
+ * If we have booted due to a crash, max_pfn will be a very low value. We need
+ * to know the amount of memory that the previous kernel used.
+ */
+unsigned long saved_max_pfn;
 
 EXPORT_SYMBOL(max_pfn);		/* This is exported so
 				 * dma_get_required_mask(), which uses

_

Re: [PATCH][3/6]Interface for copying the dump pages

[Hariprasad Nellitheertha]

[-- Attachment #1: Type: text/plain, Size: 108 bytes --]

Regards, Hari
-- 
Hariprasad Nellitheertha
Linux Technology Center
India Software Labs
IBM India, Bangalore

[-- Attachment #2: kd-copy-268.patch --]
[-- Type: text/plain, Size: 4338 bytes --]



This patch provides the interfaces necessary to read the dump contents,
treating it as a high memory device.

Signed off by Hariprasad Nellitheertha <hari@in.ibm.com>


---

 linux-2.6.8.1-hari/arch/i386/mm/highmem.c     |   18 ++++++++++
 linux-2.6.8.1-hari/drivers/char/mem.c         |   44 ++++++++++++++++++++++++++
 linux-2.6.8.1-hari/include/asm-i386/highmem.h |    1 
 linux-2.6.8.1-hari/include/linux/highmem.h    |    1 
 4 files changed, 64 insertions(+)

diff -puN arch/i386/mm/highmem.c~kd-copy-268 arch/i386/mm/highmem.c
--- linux-2.6.8.1/arch/i386/mm/highmem.c~kd-copy-268	2004-08-17 17:06:21.000000000 +0530
+++ linux-2.6.8.1-hari/arch/i386/mm/highmem.c	2004-08-17 17:06:27.000000000 +0530
@@ -74,6 +74,24 @@ void kunmap_atomic(void *kvaddr, enum km
 	preempt_check_resched();
 }
 
+/* This is the same as kmap_atomic but takes in a pfn instead of a
+ * struct page.
+ */
+void *kmap_atomic_pfn(unsigned long pfn, enum km_type type)
+{
+        enum fixed_addresses idx;
+        unsigned long vaddr;
+
+        inc_preempt_count();
+
+        idx = type + KM_TYPE_NR*smp_processor_id();
+        vaddr = __fix_to_virt(FIX_KMAP_BEGIN + idx);
+        set_pte(kmap_pte-idx, pfn_pte(pfn, kmap_prot));
+        __flush_tlb_one(vaddr);
+
+        return (void*) vaddr;
+}
+
 struct page *kmap_atomic_to_page(void *ptr)
 {
 	unsigned long idx, vaddr = (unsigned long)ptr;
diff -puN drivers/char/mem.c~kd-copy-268 drivers/char/mem.c
--- linux-2.6.8.1/drivers/char/mem.c~kd-copy-268	2004-08-17 17:06:21.000000000 +0530
+++ linux-2.6.8.1-hari/drivers/char/mem.c	2004-08-17 17:06:27.000000000 +0530
@@ -23,9 +23,13 @@
 #include <linux/devfs_fs_kernel.h>
 #include <linux/ptrace.h>
 #include <linux/device.h>
+#include <linux/bootmem.h>
 
 #include <asm/uaccess.h>
 #include <asm/io.h>
+#include <asm/tlbflush.h>
+#include <asm/crash_dump.h>
+#include <asm/highmem.h>
 
 #ifdef CONFIG_IA64
 # include <linux/efi.h>
@@ -38,6 +42,8 @@ extern void fbmem_init(void);
 extern void tapechar_init(void);
 #endif
 
+ssize_t copy_hmem_page(unsigned long, char *, size_t, int);
+
 /*
  * Architectures vary in how they handle caching for addresses
  * outside of main memory.
@@ -219,6 +225,44 @@ static int mmap_mem(struct file * file, 
 	return 0;
 }
 
+/*
+ * Copy a page from "highmem". For this page, there is no pte mapped
+ * in the current kernel. We stitch up a pte, similar to kmap_atomic.
+ */
+ssize_t copy_hmem_page(unsigned long pfn, char *buf, size_t csize, int userbuf)
+{
+	void *page, *vaddr;
+
+	if (!csize)
+		return 0;
+
+	if (pfn == -1) {
+		/* Give them a zeroed page */
+		if (userbuf) {
+			if (clear_user(buf, csize))
+				return -EFAULT;
+		} else
+			memset(buf, 0, csize);
+	} else {
+		page = kmalloc(PAGE_SIZE, GFP_KERNEL);
+
+		vaddr = kmap_atomic_pfn(pfn, KM_PTE0);
+		copy_page(page, vaddr);
+		kunmap_atomic(vaddr, KM_PTE0);
+
+		if (userbuf) {
+			if (copy_to_user(buf, page, csize)) {
+				kfree(page);
+				return -EFAULT;
+			}
+		} else
+			memcpy(buf, page, csize);
+		kfree(page);
+	}
+
+	return 0;
+}
+
 extern long vread(char *buf, char *addr, unsigned long count);
 extern long vwrite(char *buf, char *addr, unsigned long count);
 
diff -puN include/asm-i386/highmem.h~kd-copy-268 include/asm-i386/highmem.h
--- linux-2.6.8.1/include/asm-i386/highmem.h~kd-copy-268	2004-08-17 17:06:21.000000000 +0530
+++ linux-2.6.8.1-hari/include/asm-i386/highmem.h	2004-08-17 17:06:27.000000000 +0530
@@ -61,6 +61,7 @@ void *kmap(struct page *page);
 void kunmap(struct page *page);
 void *kmap_atomic(struct page *page, enum km_type type);
 void kunmap_atomic(void *kvaddr, enum km_type type);
+void *kmap_atomic_pfn(unsigned long pfn, enum km_type type);
 struct page *kmap_atomic_to_page(void *ptr);
 
 #define flush_cache_kmaps()	do { } while (0)
diff -puN include/linux/highmem.h~kd-copy-268 include/linux/highmem.h
--- linux-2.6.8.1/include/linux/highmem.h~kd-copy-268	2004-08-17 17:06:21.000000000 +0530
+++ linux-2.6.8.1-hari/include/linux/highmem.h	2004-08-17 17:06:27.000000000 +0530
@@ -30,6 +30,7 @@ static inline void *kmap(struct page *pa
 
 #define kmap_atomic(page, idx)		page_address(page)
 #define kunmap_atomic(addr, idx)	do { } while (0)
+#define kmap_atomic_pfn(pfn, idx)	page_address(pfn_to_page(pfn))
 #define kmap_atomic_to_page(ptr)	virt_to_page(ptr)
 
 #endif /* CONFIG_HIGHMEM */

_

Re: [PATCH][4/6]Register snapshotting before kexec-boot

[Hariprasad Nellitheertha]

[-- Attachment #1: Type: text/plain, Size: 108 bytes --]

Regards, Hari
-- 
Hariprasad Nellitheertha
Linux Technology Center
India Software Labs
IBM India, Bangalore

[-- Attachment #2: kd-reg-268.patch --]
[-- Type: text/plain, Size: 9739 bytes --]



This patch contains the code for stopping the other cpus and snapshotting
their register values before doing the kexec reboot.

Signed off by Hariprasad Nellitheertha <hari@in.ibm.com>


---

 linux-2.6.8.1-hari/arch/i386/kernel/Makefile                   |    1 
 linux-2.6.8.1-hari/arch/i386/kernel/crash_dump.c               |   98 ++++++++++
 linux-2.6.8.1-hari/arch/i386/kernel/machine_kexec.c            |   14 +
 linux-2.6.8.1-hari/arch/i386/kernel/smp.c                      |   13 +
 linux-2.6.8.1-hari/include/asm-i386/crash_dump.h               |   32 +++
 linux-2.6.8.1-hari/include/asm-i386/mach-default/irq_vectors.h |    1 
 linux-2.6.8.1-hari/include/asm-i386/smp.h                      |    1 
 7 files changed, 156 insertions(+), 4 deletions(-)

diff -puN arch/i386/kernel/Makefile~kd-reg-268 arch/i386/kernel/Makefile
--- linux-2.6.8.1/arch/i386/kernel/Makefile~kd-reg-268	2004-08-17 17:08:08.000000000 +0530
+++ linux-2.6.8.1-hari/arch/i386/kernel/Makefile	2004-08-17 17:08:18.000000000 +0530
@@ -24,6 +24,7 @@ obj-$(CONFIG_X86_MPPARSE)	+= mpparse.o
 obj-$(CONFIG_X86_LOCAL_APIC)	+= apic.o nmi.o
 obj-$(CONFIG_X86_IO_APIC)	+= io_apic.o
 obj-$(CONFIG_KEXEC)		+= machine_kexec.o relocate_kernel.o
+obj-$(CONFIG_CRASH_DUMP)	+= crash_dump.o
 obj-$(CONFIG_X86_NUMAQ)		+= numaq.o
 obj-$(CONFIG_X86_SUMMIT_NUMA)	+= summit.o
 obj-$(CONFIG_MODULES)		+= module.o
diff -puN /dev/null arch/i386/kernel/crash_dump.c
--- /dev/null	2003-01-30 15:54:37.000000000 +0530
+++ linux-2.6.8.1-hari/arch/i386/kernel/crash_dump.c	2004-08-17 17:08:34.000000000 +0530
@@ -0,0 +1,98 @@
+/*
+ * Architecture specific (i386) functions for kexec based crash dumps.
+ *
+ * Created by: Hariprasad Nellitheertha (hari@in.ibm.com)
+ *
+ * Copyright (C) IBM Corporation, 2004. All rights reserved.
+ *
+ */
+
+#include <linux/init.h>
+#include <linux/types.h>
+#include <linux/kernel.h>
+#include <linux/smp.h>
+#include <linux/irq.h>
+
+#include <asm/crash_dump.h>
+#include <asm/processor.h>
+#include <asm/hardirq.h>
+#include <asm/nmi.h>
+#include <asm/hw_irq.h>
+
+#ifdef CONFIG_SMP
+static atomic_t waiting_for_dump_ipi;
+static int crash_dump_expect_ipi[NR_CPUS];
+extern void crash_dump_send_ipi(void);
+extern void stop_this_cpu(void *);
+
+static int crash_dump_nmi_callback(struct pt_regs *regs, int cpu)
+{
+	if (!crash_dump_expect_ipi[cpu])
+		return 0;
+
+	crash_dump_expect_ipi[cpu] = 0;
+	crash_dump_save_this_cpu(regs, cpu);
+	atomic_dec(&waiting_for_dump_ipi);
+
+	stop_this_cpu(NULL);
+
+	return 1;
+}
+
+void __crash_dump_stop_cpus(void)
+{
+	int i, cpu = smp_processor_id();
+	int other_cpus = num_online_cpus()-1;
+
+	if (other_cpus > 0) {
+		atomic_set(&waiting_for_dump_ipi, other_cpus);
+
+		for (i = 0; i < NR_CPUS; i++)
+			crash_dump_expect_ipi[i] = (i != cpu && cpu_online(i));
+
+		set_nmi_callback(crash_dump_nmi_callback);
+		/* Ensure the new callback function is set before sending
+		 * out the IPI
+		 */
+		wmb();
+
+		crash_dump_send_ipi();
+		while (atomic_read(&waiting_for_dump_ipi) > 0)
+			cpu_relax();
+
+		unset_nmi_callback();
+	} else {
+		local_irq_disable();
+		disable_local_APIC();
+		local_irq_enable();
+	}
+}
+#else
+void __crash_dump_stop_cpus(void) {}
+#endif
+
+void crash_get_current_regs(struct pt_regs *regs)
+{
+	__asm__ __volatile__("movl %%ebx,%0" : "=m"(regs->ebx));
+	__asm__ __volatile__("movl %%ecx,%0" : "=m"(regs->ecx));
+	__asm__ __volatile__("movl %%edx,%0" : "=m"(regs->edx));
+	__asm__ __volatile__("movl %%esi,%0" : "=m"(regs->esi));
+	__asm__ __volatile__("movl %%edi,%0" : "=m"(regs->edi));
+	__asm__ __volatile__("movl %%ebp,%0" : "=m"(regs->ebp));
+	__asm__ __volatile__("movl %%eax,%0" : "=m"(regs->eax));
+	__asm__ __volatile__("movl %%esp,%0" : "=m"(regs->esp));
+	__asm__ __volatile__("movw %%ss, %%ax;" :"=a"(regs->xss));
+	__asm__ __volatile__("movw %%cs, %%ax;" :"=a"(regs->xcs));
+	__asm__ __volatile__("movw %%ds, %%ax;" :"=a"(regs->xds));
+	__asm__ __volatile__("movw %%es, %%ax;" :"=a"(regs->xes));
+	__asm__ __volatile__("pushfl; popl %0" :"=m"(regs->eflags));
+
+	regs->eip = (unsigned long)current_text_addr();
+}
+
+void crash_dump_save_this_cpu(struct pt_regs *regs, int cpu)
+{
+	crash_smp_current_task[cpu] = (int)current;
+	crash_smp_regs[cpu] = *regs;
+}
+
diff -puN arch/i386/kernel/machine_kexec.c~kd-reg-268 arch/i386/kernel/machine_kexec.c
--- linux-2.6.8.1/arch/i386/kernel/machine_kexec.c~kd-reg-268	2004-08-17 17:08:08.000000000 +0530
+++ linux-2.6.8.1-hari/arch/i386/kernel/machine_kexec.c	2004-08-17 17:08:18.000000000 +0530
@@ -17,8 +17,9 @@
 #include <asm/io.h>
 #include <asm/apic.h>
 #include <asm/cpufeature.h>
-#include <asm/hw_irq.h>
 
+struct pt_regs crash_smp_regs[NR_CPUS];
+u32 crash_smp_current_task[NR_CPUS];
 
 static void set_idt(void *newidt, __u16 limit)
 {
@@ -97,6 +98,14 @@ void __relocate_base_mem(unsigned long b
 		if (PageReserved(page))
 			copy_page(dest_addr, src_addr);
 	}
+
+	/* Now copy over the saved task pointers and the registers
+	 * to the end of the  reserved area
+	 */
+	dest_addr = (void *)(__va(CRASH_BACKUP_BASE + CRASH_BACKUP_SIZE));
+	memcpy(dest_addr, crash_smp_regs, (sizeof(struct pt_regs)*NR_CPUS));
+	dest_addr += sizeof(struct pt_regs)*NR_CPUS;
+	memcpy(dest_addr, crash_smp_current_task, (sizeof(u32)*NR_CPUS));
 }
 
 /*
@@ -111,7 +120,8 @@ void machine_kexec(struct kimage *image)
 
 	/* switch to an mm where the reboot_code_buffer is identity mapped */
 	use_mm(&init_mm);
-	stop_apics();
+
+	crash_dump_stop_cpus();
 
 	/* Interrupts aren't acceptable while we reboot */
 	local_irq_disable();
diff -puN arch/i386/kernel/smp.c~kd-reg-268 arch/i386/kernel/smp.c
--- linux-2.6.8.1/arch/i386/kernel/smp.c~kd-reg-268	2004-08-17 17:08:08.000000000 +0530
+++ linux-2.6.8.1-hari/arch/i386/kernel/smp.c	2004-08-17 17:08:18.000000000 +0530
@@ -143,6 +143,9 @@ inline void __send_IPI_shortcut(unsigned
 	 */
 	cfg = __prepare_ICR(shortcut, vector);
 
+	if (vector == CRASH_DUMP_VECTOR)
+		cfg = (cfg&~APIC_VECTOR_MASK)|APIC_DM_NMI;
+
 	/*
 	 * Send the IPI. The write to APIC_ICR fires this off.
 	 */
@@ -221,6 +224,9 @@ inline void send_IPI_mask_sequence(cpuma
 			 */
 			cfg = __prepare_ICR(0, vector);
 			
+			if (vector == CRASH_DUMP_VECTOR)
+				cfg = (cfg&~APIC_VECTOR_MASK)|APIC_DM_NMI;
+
 			/*
 			 * Send the IPI. The write to APIC_ICR fires this off.
 			 */
@@ -467,6 +473,11 @@ void flush_tlb_all(void)
 	on_each_cpu(do_flush_tlb_all, NULL, 1, 1);
 }
 
+void crash_dump_send_ipi(void)
+{
+	send_IPI_allbutself(CRASH_DUMP_VECTOR);
+}
+
 /*
  * this function sends a 'reschedule' IPI to another CPU.
  * it goes straight through and wastes no time serializing
@@ -548,7 +559,7 @@ int smp_call_function (void (*func) (voi
 	return 0;
 }
 
-static void stop_this_cpu (void * dummy)
+void stop_this_cpu (void * dummy)
 {
 	/*
 	 * Remove this CPU:
diff -puN include/asm-i386/crash_dump.h~kd-reg-268 include/asm-i386/crash_dump.h
--- linux-2.6.8.1/include/asm-i386/crash_dump.h~kd-reg-268	2004-08-17 17:08:09.000000000 +0530
+++ linux-2.6.8.1-hari/include/asm-i386/crash_dump.h	2004-08-17 17:08:18.000000000 +0530
@@ -1,5 +1,7 @@
 /* asm-i386/crash_dump.h */
 #include <linux/bootmem.h>
+#include <asm/hw_irq.h>
+#include <asm/apic.h>
 
 extern unsigned int dump_enabled;
 
@@ -8,6 +10,11 @@ void __init find_max_pfn(void);
 void __relocate_base_mem(unsigned long, unsigned long);
 
 extern unsigned int crashed;
+extern struct pt_regs crash_smp_regs[NR_CPUS];
+extern u32 crash_smp_current_task[NR_CPUS];
+extern void crash_dump_save_this_cpu(struct pt_regs *, int);
+extern void __crash_dump_stop_cpus(void);
+extern void crash_get_current_regs(struct pt_regs *regs);
 
 #ifdef CONFIG_CRASH_DUMP
 #define CRASH_BACKUP_BASE ((unsigned long)CONFIG_BACKUP_BASE * 0x100000)
@@ -28,7 +35,26 @@ static inline void set_saved_max_pfn(voi
 static inline void crash_reserve_bootmem(void)
 {
 	if (!dump_enabled)
-		reserve_bootmem(CRASH_BACKUP_BASE, CRASH_BACKUP_SIZE);
+		reserve_bootmem(CRASH_BACKUP_BASE, CRASH_BACKUP_SIZE + PAGE_SIZE);
+}
+
+static inline void crash_dump_stop_cpus(void)
+{
+	if (!crashed)
+	stop_apics();
+	else {
+		int cpu = smp_processor_id();
+
+		crash_smp_current_task[cpu] = (int)current;
+		crash_get_current_regs(&crash_smp_regs[cpu]);
+
+		/* This also captures the register states of the other cpus */
+		__crash_dump_stop_cpus();
+#if defined(CONFIG_X86_IO_APIC)
+		disable_IO_APIC();
+#endif
+		disconnect_bsp_APIC();
+	}
 }
 #else
 #define CRASH_BACKUP_BASE 0x08000000
@@ -36,4 +62,8 @@ static inline void crash_reserve_bootmem
 static inline void relocate_base_mem(void) {}
 static inline void set_saved_max_pfn(void) {}
 static inline void crash_reserve_bootmem(void) {}
+static inline void crash_dump_stop_cpus(void)
+{
+	stop_apics();
+}
 #endif
diff -puN include/asm-i386/mach-default/irq_vectors.h~kd-reg-268 include/asm-i386/mach-default/irq_vectors.h
--- linux-2.6.8.1/include/asm-i386/mach-default/irq_vectors.h~kd-reg-268	2004-08-17 17:08:09.000000000 +0530
+++ linux-2.6.8.1-hari/include/asm-i386/mach-default/irq_vectors.h	2004-08-17 17:08:18.000000000 +0530
@@ -48,6 +48,7 @@
 #define INVALIDATE_TLB_VECTOR	0xfd
 #define RESCHEDULE_VECTOR	0xfc
 #define CALL_FUNCTION_VECTOR	0xfb
+#define CRASH_DUMP_VECTOR	0xfa
 
 #define THERMAL_APIC_VECTOR	0xf0
 /*
diff -puN include/asm-i386/smp.h~kd-reg-268 include/asm-i386/smp.h
--- linux-2.6.8.1/include/asm-i386/smp.h~kd-reg-268	2004-08-17 17:08:09.000000000 +0530
+++ linux-2.6.8.1-hari/include/asm-i386/smp.h	2004-08-17 17:08:18.000000000 +0530
@@ -41,6 +41,7 @@ extern void smp_message_irq(int cpl, voi
 extern void smp_invalidate_rcv(void);		/* Process an NMI */
 extern void (*mtrr_hook) (void);
 extern void zap_low_mappings (void);
+extern void stop_this_cpu(void *);
 
 #define MAX_APICID 256
 

_

Re: [PATCH][5/6]ELF format interface for the dump

[Hariprasad Nellitheertha]

[-- Attachment #1: Type: text/plain, Size: 108 bytes --]

Regards, Hari
-- 
Hariprasad Nellitheertha
Linux Technology Center
India Software Labs
IBM India, Bangalore

[-- Attachment #2: kd-elf-268.patch --]
[-- Type: text/plain, Size: 11371 bytes --]



This patch contains the code that provides an ELF format interface to the
previous kernel's memory post kexec reboot.

Signed off by Hariprasad Nellitheertha <hari@in.ibm.com>


---

 linux-2.6.8.1-hari/fs/proc/Makefile           |    1 
 linux-2.6.8.1-hari/fs/proc/kcore.c            |   10 -
 linux-2.6.8.1-hari/fs/proc/proc_misc.c        |   13 +
 linux-2.6.8.1-hari/fs/proc/vmcore.c           |  233 ++++++++++++++++++++++++++
 linux-2.6.8.1-hari/include/linux/crash_dump.h |    6 
 linux-2.6.8.1-hari/include/linux/proc_fs.h    |    2 
 6 files changed, 260 insertions(+), 5 deletions(-)

diff -puN fs/proc/Makefile~kd-elf-268 fs/proc/Makefile
--- linux-2.6.8.1/fs/proc/Makefile~kd-elf-268	2004-08-17 17:10:18.000000000 +0530
+++ linux-2.6.8.1-hari/fs/proc/Makefile	2004-08-17 17:10:25.000000000 +0530
@@ -11,4 +11,5 @@ proc-y       += inode.o root.o base.o ge
 		kmsg.o proc_tty.o proc_misc.o
 
 proc-$(CONFIG_PROC_KCORE)	+= kcore.o
+proc-$(CONFIG_CRASH_DUMP)	+= vmcore.o
 proc-$(CONFIG_PROC_DEVICETREE)	+= proc_devtree.o
diff -puN fs/proc/kcore.c~kd-elf-268 fs/proc/kcore.c
--- linux-2.6.8.1/fs/proc/kcore.c~kd-elf-268	2004-08-17 17:10:18.000000000 +0530
+++ linux-2.6.8.1-hari/fs/proc/kcore.c	2004-08-17 17:10:25.000000000 +0530
@@ -114,7 +114,7 @@ static size_t get_kcore_size(int *nphdr,
 /*
  * determine size of ELF note
  */
-static int notesize(struct memelfnote *en)
+int notesize(struct memelfnote *en)
 {
 	int sz;
 
@@ -129,7 +129,7 @@ static int notesize(struct memelfnote *e
 /*
  * store a note in the header buffer
  */
-static char *storenote(struct memelfnote *men, char *bufp)
+char *storenote(struct memelfnote *men, char *bufp)
 {
 	struct elf_note en;
 
@@ -156,7 +156,7 @@ static char *storenote(struct memelfnote
  * store an ELF coredump header in the supplied buffer
  * nphdr is the number of elf_phdr to insert
  */
-static void elf_kcore_store_hdr(char *bufp, int nphdr, int dataoff)
+void elf_kcore_store_hdr(char *bufp, int nphdr, int dataoff, struct kcore_list *clist)
 {
 	struct elf_prstatus prstatus;	/* NT_PRSTATUS */
 	struct elf_prpsinfo prpsinfo;	/* NT_PRPSINFO */
@@ -208,7 +208,7 @@ static void elf_kcore_store_hdr(char *bu
 	nhdr->p_align	= 0;
 
 	/* setup ELF PT_LOAD program header for every area */
-	for (m=kclist; m; m=m->next) {
+	for (m=clist; m; m=m->next) {
 		phdr = (struct elf_phdr *) bufp;
 		bufp += sizeof(struct elf_phdr);
 		offset += sizeof(struct elf_phdr);
@@ -305,7 +305,7 @@ read_kcore(struct file *file, char __use
 			return -ENOMEM;
 		}
 		memset(elf_buf, 0, elf_buflen);
-		elf_kcore_store_hdr(elf_buf, nphdr, elf_buflen);
+		elf_kcore_store_hdr(elf_buf, nphdr, elf_buflen, kclist);
 		read_unlock(&kclist_lock);
 		if (copy_to_user(buffer, elf_buf + *fpos, tsz)) {
 			kfree(elf_buf);
diff -puN fs/proc/proc_misc.c~kd-elf-268 fs/proc/proc_misc.c
--- linux-2.6.8.1/fs/proc/proc_misc.c~kd-elf-268	2004-08-17 17:10:18.000000000 +0530
+++ linux-2.6.8.1-hari/fs/proc/proc_misc.c	2004-08-17 17:10:25.000000000 +0530
@@ -44,6 +44,8 @@
 #include <linux/jiffies.h>
 #include <linux/sysrq.h>
 #include <linux/vmalloc.h>
+#include <linux/bootmem.h>
+#include <linux/crash_dump.h>
 #include <asm/uaccess.h>
 #include <asm/pgtable.h>
 #include <asm/io.h>
@@ -628,6 +630,7 @@ static struct file_operations proc_sysrq
 #endif
 
 struct proc_dir_entry *proc_root_kcore;
+struct proc_dir_entry *proc_vmcore;
 
 static void create_seq_entry(char *name, mode_t mode, struct file_operations *f)
 {
@@ -689,6 +692,16 @@ void __init proc_misc_init(void)
 				(size_t)high_memory - PAGE_OFFSET + PAGE_SIZE;
 	}
 #endif
+#ifdef CONFIG_CRASH_DUMP
+	if (dump_enabled) {
+		proc_vmcore = create_proc_entry("vmcore", S_IRUSR, NULL);
+		if (proc_vmcore) {
+			proc_vmcore->proc_fops = &proc_vmcore_operations;
+			proc_vmcore->size =
+				(size_t)(saved_max_pfn << PAGE_SHIFT);
+		}
+	}
+#endif
 	if (prof_on) {
 		entry = create_proc_entry("profile", S_IWUSR | S_IRUGO, NULL);
 		if (entry) {
diff -puN /dev/null fs/proc/vmcore.c
--- /dev/null	2003-01-30 15:54:37.000000000 +0530
+++ linux-2.6.8.1-hari/fs/proc/vmcore.c	2004-08-17 17:10:25.000000000 +0530
@@ -0,0 +1,233 @@
+/*
+ *	fs/proc/vmcore.c Interface for accessing the crash
+ * 				 dump from the system's previous life.
+ * 	Heavily borrowed from fs/proc/kcore.c
+ *	Created by: Hariprasad Nellitheertha (hari@in.ibm.com)
+ *	Copyright (C) IBM Corporation, 2004. All rights reserved
+ */
+
+#include <linux/config.h>
+#include <linux/mm.h>
+#include <linux/proc_fs.h>
+#include <linux/user.h>
+#include <linux/a.out.h>
+#include <linux/elf.h>
+#include <linux/elfcore.h>
+#include <linux/vmalloc.h>
+#include <linux/proc_fs.h>
+#include <linux/highmem.h>
+#include <linux/bootmem.h>
+#include <linux/init.h>
+#include <linux/crash_dump.h>
+#include <asm/uaccess.h>
+#include <asm/io.h>
+
+extern unsigned long saved_max_pfn;
+/* This is to re-use the kcore header creation code */
+static struct kcore_list vmcore_mem;
+
+static int open_vmcore(struct inode * inode, struct file * filp)
+{
+	return 0;
+}
+
+static ssize_t read_vmcore(struct file *,char __user *,size_t, loff_t *);
+
+struct file_operations proc_vmcore_operations = {
+	.read		= read_vmcore,
+	.open		= open_vmcore,
+};
+
+#define BACKUP_START CRASH_BACKUP_BASE
+#define BACKUP_END CRASH_BACKUP_BASE + CRASH_BACKUP_SIZE
+#define REG_SIZE sizeof(elf_gregset_t)
+
+struct memelfnote
+{
+	const char *name;
+	int type;
+	unsigned int datasz;
+	void *data;
+};
+
+static size_t get_vmcore_size(int *nphdr, size_t *elf_buflen)
+{
+	size_t size;
+
+	/* We need 1 PT_LOAD segment headers
+	 * In addition, we need one PT_NOTE header
+	 */
+	*nphdr = 2;
+	size = (size_t)(saved_max_pfn << PAGE_SHIFT);
+
+	*elf_buflen =	sizeof(struct elfhdr) +
+			(*nphdr + 2)*sizeof(struct elf_phdr) +
+			3 * sizeof(struct memelfnote) +
+			sizeof(struct elf_prstatus) +
+			sizeof(struct elf_prpsinfo) +
+			sizeof(struct task_struct);
+	*elf_buflen = PAGE_ALIGN(*elf_buflen);
+	return size + *elf_buflen;
+}
+
+/*
+ * Reads from the hmem device from given offset till
+ * given count
+ */
+static ssize_t read_from_hmem(char *buf, size_t count,
+			     loff_t *ppos, int userbuf)
+{
+	unsigned long pfn, p = *ppos;
+
+	if (p == -1) {
+		pfn = -1;
+		p = 0;
+	} else {
+		pfn = p / PAGE_SIZE;
+		p = p % PAGE_SIZE;
+	}
+	if ((pfn != -1) && (pfn > saved_max_pfn))
+		return -EINVAL;
+
+	if (count > PAGE_SIZE - p)
+		count = PAGE_SIZE - p;
+
+	if (copy_hmem_page(pfn, buf, count, userbuf))
+		return -EFAULT;
+
+	*ppos += count;
+	return 0;
+}
+
+/*
+ * store an ELF crash dump header in the supplied buffer
+ * nphdr is the number of elf_phdr to insert
+ */
+static void elf_vmcore_store_hdr(char *bufp, int nphdr, int dataoff)
+{
+	struct elf_prstatus prstatus;	/* NT_PRSTATUS */
+	struct memelfnote notes[1];
+	char reg_buf[REG_SIZE];
+	loff_t reg_ppos;
+	char *buf = bufp;
+
+	vmcore_mem.addr = (unsigned long)__va(0);
+	vmcore_mem.size = saved_max_pfn << PAGE_SHIFT;
+	vmcore_mem.next = NULL;
+
+	/* Re-use the kcore code */
+	elf_kcore_store_hdr(bufp, nphdr, dataoff, &vmcore_mem);
+	buf += sizeof(struct elfhdr) + 2*sizeof(struct elf_phdr);
+
+	/* set up the process status */
+	notes[0].name = "CORE";
+	notes[0].type = NT_PRSTATUS;
+	notes[0].datasz = sizeof(struct elf_prstatus);
+	notes[0].data = &prstatus;
+
+	memset(&prstatus, 0, sizeof(struct elf_prstatus));
+
+	/* 1 - Get the registers from the reserved memory area */
+	reg_ppos = BACKUP_END;
+	read_from_hmem(reg_buf, REG_SIZE, &reg_ppos, 0);
+	elf_core_copy_regs(&prstatus.pr_reg, (struct pt_regs *)reg_buf);
+	buf = storenote(&notes[0], buf);
+}
+
+/*
+ * read from the ELF header and then the crash dump
+ */
+static ssize_t read_vmcore(
+struct file *file, char __user *buffer, size_t buflen, loff_t *fpos)
+{
+	ssize_t acc = 0;
+	size_t size, tsz;
+	size_t elf_buflen;
+	int nphdr;
+	unsigned long start;
+
+	tsz =  get_vmcore_size(&nphdr, &elf_buflen);
+	proc_vmcore->size = size = tsz + elf_buflen;
+	if (buflen == 0 || *fpos >= size) {
+		goto done;
+	}
+
+	/* trim buflen to not go beyond EOF */
+	if (buflen > size - *fpos)
+		buflen = size - *fpos;
+
+	/* construct an ELF core header if we'll need some of it */
+	if (*fpos < elf_buflen) {
+		char * elf_buf;
+
+		tsz = elf_buflen - *fpos;
+		if (buflen < tsz)
+			tsz = buflen;
+		elf_buf = kmalloc(elf_buflen, GFP_ATOMIC);
+		if (!elf_buf) {
+			acc = -ENOMEM;
+			goto done;
+		}
+		memset(elf_buf, 0, elf_buflen);
+		elf_vmcore_store_hdr(elf_buf, nphdr, elf_buflen);
+		if (copy_to_user(buffer, elf_buf + *fpos, tsz)) {
+			kfree(elf_buf);
+			acc = -EFAULT;
+			goto done;
+		}
+		kfree(elf_buf);
+		buflen -= tsz;
+		*fpos += tsz;
+		buffer += tsz;
+		acc += tsz;
+
+		/* leave now if filled buffer already */
+		if (buflen == 0) {
+			goto done;
+		}
+	}
+
+	start = *fpos - elf_buflen;
+	if ((tsz = (PAGE_SIZE - (start & ~PAGE_MASK))) > buflen)
+		tsz = buflen;
+
+	while (buflen) {
+		unsigned long p;
+
+		if ((start < 0) || (start >= size))
+			if (clear_user(buffer, tsz)) {
+				acc = -EFAULT;
+				goto done;
+			}
+
+		/* tsz contains actual len of dump to be read.
+		 * buflen is the total len that was requested.
+		 * This may contain part of ELF header. start
+		 * is the fpos for the hmem region
+		 * If the file position is 0-16M, we need to read
+		 * from 128-144M. If the file position is 128-144M
+		 * we just return zeroes
+		 */
+		p = start;
+		if (p < (max_pfn << PAGE_SHIFT)) {
+			p += CRASH_BACKUP_BASE;
+		} else if ((p >= BACKUP_START) && (p < BACKUP_END)) {
+			p = -1;
+		}
+
+		if (read_from_hmem(buffer, tsz, (loff_t *)&p, 1)) {
+			acc = -EINVAL;
+			goto done;
+		}
+
+		buflen -= tsz;
+		*fpos += tsz;
+		buffer += tsz;
+		acc += tsz;
+		start += tsz;
+		tsz = (buflen > PAGE_SIZE ? PAGE_SIZE : buflen);
+	}
+
+done:
+	return acc;
+}
diff -puN include/linux/crash_dump.h~kd-elf-268 include/linux/crash_dump.h
--- linux-2.6.8.1/include/linux/crash_dump.h~kd-elf-268	2004-08-17 17:10:18.000000000 +0530
+++ linux-2.6.8.1-hari/include/linux/crash_dump.h	2004-08-17 17:10:25.000000000 +0530
@@ -3,6 +3,12 @@
 #include <linux/device.h>
 #include <asm/crash_dump.h>
 
+extern struct memelfnote memelfnote;
+extern int notesize(struct memelfnote *);
+extern char *storenote(struct memelfnote *, char *);
+extern ssize_t copy_hmem_page(unsigned long, char *, size_t, int);
+extern void elf_kcore_store_hdr(char *, int, int, struct kcore_list *);
+
 #ifdef CONFIG_CRASH_DUMP
 static inline void crash_machine_kexec(void)
 {
diff -puN include/linux/proc_fs.h~kd-elf-268 include/linux/proc_fs.h
--- linux-2.6.8.1/include/linux/proc_fs.h~kd-elf-268	2004-08-17 17:10:18.000000000 +0530
+++ linux-2.6.8.1-hari/include/linux/proc_fs.h	2004-08-17 17:10:25.000000000 +0530
@@ -82,6 +82,7 @@ extern struct proc_dir_entry *proc_net;
 extern struct proc_dir_entry *proc_bus;
 extern struct proc_dir_entry *proc_root_driver;
 extern struct proc_dir_entry *proc_root_kcore;
+extern struct proc_dir_entry *proc_vmcore;
 
 extern void proc_root_init(void);
 extern void proc_misc_init(void);
@@ -112,6 +113,7 @@ extern int proc_readdir(struct file *, v
 extern struct dentry *proc_lookup(struct inode *, struct dentry *, struct nameidata *);
 
 extern struct file_operations proc_kcore_operations;
+extern struct file_operations proc_vmcore_operations;
 extern struct file_operations proc_kmsg_operations;
 extern struct file_operations ppc_htab_operations;
 

_

Re: [PATCH][6/6]Device abstraction for linear/raw view of the dump

[Hariprasad Nellitheertha]

[-- Attachment #1: Type: text/plain, Size: 108 bytes --]

Regards, Hari
-- 
Hariprasad Nellitheertha
Linux Technology Center
India Software Labs
IBM India, Bangalore

[-- Attachment #2: kd-hmem-268.patch --]
[-- Type: text/plain, Size: 4534 bytes --]



This patch contains the code that enables us to access the previous kernel's
memory as /dev/hmem.

Signed off by Hariprasad Nellitheertha <hari@in.ibm.com>
Signed off by Adam Litke <litke@us.ibm.com>


---

 linux-2.6.8.1-hari/Documentation/devices.txt |    1 
 linux-2.6.8.1-hari/drivers/char/mem.c        |   72 +++++++++++++++++++++++++++
 2 files changed, 73 insertions(+)

diff -puN Documentation/devices.txt~kd-hmem-268 Documentation/devices.txt
--- linux-2.6.8.1/Documentation/devices.txt~kd-hmem-268	2004-08-17 17:11:26.000000000 +0530
+++ linux-2.6.8.1-hari/Documentation/devices.txt	2004-08-17 17:11:32.000000000 +0530
@@ -100,6 +100,7 @@ Your cooperation is appreciated.
 		  9 = /dev/urandom	Faster, less secure random number gen.
 		 10 = /dev/aio		Asyncronous I/O notification interface
 		 11 = /dev/kmsg		Writes to this come out as printk's
+		 12 = /dev/hmem		Access to kexec-ed crash dump
   1 block	RAM disk
 		  0 = /dev/ram0		First RAM disk
 		  1 = /dev/ram1		Second RAM disk
diff -puN drivers/char/mem.c~kd-hmem-268 drivers/char/mem.c
--- linux-2.6.8.1/drivers/char/mem.c~kd-hmem-268	2004-08-17 17:11:26.000000000 +0530
+++ linux-2.6.8.1-hari/drivers/char/mem.c	2004-08-17 17:11:32.000000000 +0530
@@ -263,6 +263,68 @@ ssize_t copy_hmem_page(unsigned long pfn
 	return 0;
 }
 
+/*
+ * After a crash-induced kexec, our physical memory layout
+ * looks like the following:
+ *
+ *        max_pfn                           End of memory\
+ * /---------|--------------------------------------------\
+ * | Section | Section | Section | Section                |
+ * |    K    |    B    |    A    |    C                   |
+ * \------------------------------------------------------/
+ *
+ * Section K: System memory.  The system is fooled to think it
+ *            can only use memory in this area.
+ * Section A: Backup of 0 - CRASH_BACKUP_SIZE kB from
+ *            the crashed kernel.
+ * Section B: Untouched memory from last kernel.
+ * Section C: Untouched memory from last kernel.
+ *
+ * In order to present a properly-ordered dump via this device,
+ * we must reorder the sections per the following:
+ *
+ * /---------|--------------------------------------------\
+ * | Section | Section |  Empty  | Section                |
+ * |    A    |    B    | Section |    C                   |
+ * \------------------------------------------------------/
+ *
+ */
+static ssize_t read_hmem(struct file * file, char * buf,
+                        size_t count, loff_t *ppos)
+{
+	unsigned long pfn;
+	unsigned curmem_end, backup_start, backup_end;
+	size_t read=0, csize;
+
+	backup_start = CRASH_BACKUP_BASE / PAGE_SIZE;
+	backup_end = backup_start + (CRASH_BACKUP_SIZE / PAGE_SIZE);
+	curmem_end = max_pfn;
+
+	while(count) {
+		pfn = *ppos / PAGE_SIZE;
+
+		/* Perform translation (see comment above) */
+		if (pfn < curmem_end)
+			pfn += backup_start;
+		else if ((pfn >= backup_start) && (pfn < backup_end))
+			pfn = -1; /* Placeholder so we know to zero page later */
+
+		if ((pfn != -1) && (pfn > saved_max_pfn))
+			return 0; /* EOF */
+
+		csize = (count > PAGE_SIZE) ? PAGE_SIZE : count;
+
+		if (copy_hmem_page(pfn, buf, csize, 1))
+			return -EFAULT;
+
+		buf += csize;
+		*ppos += csize;
+		read += csize;
+		count -= csize;
+	}
+	return read;
+}
+
 extern long vread(char *buf, char *addr, unsigned long count);
 extern long vwrite(char *buf, char *addr, unsigned long count);
 
@@ -626,6 +688,7 @@ static int open_port(struct inode * inod
 #define read_full       read_zero
 #define open_mem	open_port
 #define open_kmem	open_mem
+#define open_hmem	open_mem
 
 static struct file_operations mem_fops = {
 	.llseek		= memory_lseek,
@@ -671,6 +734,11 @@ static struct file_operations full_fops 
 	.write		= write_full,
 };
 
+static struct file_operations hmem_fops = {
+        .read           = read_hmem,
+        .open           = open_hmem,
+};
+
 static ssize_t kmsg_write(struct file * file, const char __user * buf,
 			  size_t count, loff_t *ppos)
 {
@@ -725,6 +793,9 @@ static int memory_open(struct inode * in
 		case 11:
 			filp->f_op = &kmsg_fops;
 			break;
+		case 12:
+			filp->f_op = &hmem_fops;
+			break;
 		default:
 			return -ENXIO;
 	}
@@ -754,6 +825,7 @@ static const struct {
 	{8, "random",  S_IRUGO | S_IWUSR,           &random_fops},
 	{9, "urandom", S_IRUGO | S_IWUSR,           &urandom_fops},
 	{11,"kmsg",    S_IRUGO | S_IWUSR,           &kmsg_fops},
+	{12,"hmem",    S_IRUSR | S_IWUSR | S_IRGRP, &hmem_fops},
 };
 
 static struct class_simple *mem_class;

_

Re: [PATCH][2/6]Memory preserving reboot using kexec

[Dave Hansen]

On Tue, 2004-08-17 at 05:07, Hariprasad Nellitheertha wrote:
> Regards, Hari


> +void __relocate_base_mem(unsigned long backup_addr, unsigned long backup_size)
> +{
> +       unsigned long pfn, pfn_max;
> +       void *src_addr, *dest_addr;
> +       struct page *page;
> +
> +       pfn_max = backup_size >> PAGE_SHIFT;
> +       for (pfn = 0; pfn < pfn_max; pfn++) {
> +               src_addr = phys_to_virt(pfn << PAGE_SHIFT);
> +               dest_addr = backup_addr + src_addr;
> +               if (!pfn_valid(pfn))
> +                       continue;
> +               page = pfn_to_page(pfn);
> +               if (PageReserved(page))
> +                       copy_page(dest_addr, src_addr);
> +       }
> +}

You're getting a little sloppy with your types in there.  I know you
probably aren't getting warnings for passing unsigned longs to
copy_page(), but you should probably still be passing pointers to it.  

I think the general convention is to keep physical addresses in unsigned
longs and virtual addresses in pointers.  Just keep that in mind.  

> +#define CRASH_BACKUP_BASE 0x08000000
> +#define CRASH_BACKUP_SIZE 0x01000000

What are these numbers?  Why do you need to define them when your config
option is off?

> +/*
> + * If we have booted due to a crash, max_pfn will be a very low value. We need
> + * to know the amount of memory that the previous kernel used.
> + */
> +unsigned long saved_max_pfn;

I'd probably put that comment next to the place where saved_max_pfn is
used instead of where it is declared.  


-- Dave

Re: [PATCH][3/6]Interface for copying the dump pages

[Dave Hansen]

On Tue, 2004-08-17 at 05:08, Hariprasad Nellitheertha wrote:
> Regards, Hari
> +/* This is the same as kmap_atomic but takes in a pfn instead of a
> + * struct page.
> + */

How about:

/* 
 * This is the same as kmap_atomic() but can map memory that doesn't 
 * necessarily have struct page associated with it.
 */

> +ssize_t copy_hmem_page(unsigned long pfn, char *buf, size_t csize,
> int userbuf)
...
> +       if (pfn == -1) {
> +               /* Give them a zeroed page */
> +               if (userbuf) {
> +                       if (clear_user(buf, csize))
> +                               return -EFAULT;
> +               } else
> +                       memset(buf, 0, csize);
> +       } else {

This is really hackish.  If you want to zero a userspace page, just zero
it in the caller's code, or call a different function.  Zeroing a
userspace page has nothing to do with copying a highmem page, and it
really shouldn't be here.  Just changing it from taking a 0 to a -1 from
the last time you posted it doesn't really make it any better.

Also, that function might belong in the same area as copy_highpage(),
not in some driver.  In any case, it certainly needs a big fat comment
explaining why it is special, and different from copy_highpage().

If it shouldn't be moved and really is isolated to the mem.c driver, it
needs to be declared static.

-- Dave

Re: [PATCH][4/6]Register snapshotting before kexec-boot

[Dave Hansen]

On Tue, 2004-08-17 at 05:09, Hariprasad Nellitheertha wrote:
> Regards, Hari
> +       /* Now copy over the saved task pointers and the registers
> +        * to the end of the  reserved area
> +        */
> +       dest_addr = (void *)(__va(CRASH_BACKUP_BASE + CRASH_BACKUP_SIZE));

Why the cast?  Isn't __va() already returning a void*?

-- Dave

Re: [RFC]Kexec based crash dumping

[Andrew Morton]

Hariprasad Nellitheertha <hari@in.ibm.com> wrote:
>
> The patches that follow contain the initial implementation for kexec based
> crash dumping that we are working on.

It seems to be coming together nicely.

Where do we stand with support for other architectures?  Do you expect that
each architecture will involve a lot of work?

And how much of the i386 implementation do you expect x86_64 can
reuse?

Re: [PATCH][6/6]Device abstraction for linear/raw view of the dump

[Martin J. Bligh]

> This patch contains the code that enables us to access the 
> previous kernel's memory as /dev/hmem.

One of the bits of feedback we got at kernel summit was that nobody
liked the /dev/hmem name ... could we change it to /dev/oldmem, perhaps?

Thanks,

M.

Re: [RFC]Kexec based crash dumping

[Hariprasad Nellitheertha]

Hi Andrew,

On Tue, Aug 17, 2004 at 03:44:36PM -0700, Andrew Morton wrote:
> Hariprasad Nellitheertha <hari@in.ibm.com> wrote:
> >
> > The patches that follow contain the initial implementation for kexec based
> > crash dumping that we are working on.
> 
> It seems to be coming together nicely.
> 
> Where do we stand with support for other architectures?  Do you expect that
> each architecture will involve a lot of work?

I don't think so. The main architecture dependent components are register
snapshotting and silencing of other cpus. These are not new problems to
solve as other projects such as LKCD, KDB have already done this for
most archs.

> 
> And how much of the i386 implementation do you expect x86_64 can
> reuse?

We should be able to re-use most of the code. The x86_64 port should be 
ready pretty quickly once kexec itself is available.

Regards, Hari
-- 
Hariprasad Nellitheertha
Linux Technology Center
India Software Labs
IBM India, Bangalore

Re: [PATCH][6/6]Device abstraction for linear/raw view of the dump

[Hariprasad Nellitheertha]

Hi Martin,

On Tue, Aug 17, 2004 at 03:53:21PM -0700, Martin J. Bligh wrote:
> > This patch contains the code that enables us to access the 
> > previous kernel's memory as /dev/hmem.
> 
> One of the bits of feedback we got at kernel summit was that nobody
> liked the /dev/hmem name ... could we change it to /dev/oldmem, perhaps?

I'll change it to /dev/oldmem in the next iteration of the patches.

> 
> Thanks,
> 
> M.
> 
Regards, Hari
-- 
Hariprasad Nellitheertha
Linux Technology Center
India Software Labs
IBM India, Bangalore

Re: [PATCH][4/6]Register snapshotting before kexec-boot

[Theodore Ts'o]

This patch is needed so that the kexec patches will compile on systems
that don't have APIC's enabled.

						- Ted

# This is a BitKeeper generated diff -Nru style patch.
#
# ChangeSet
#   2004/08/18 09:41:41-04:00 tytso@think.thunk.org 
#   Fix so that kexec patches compile on systems when 
#   CONFIG_X86_LOCAL_APIC is disabled.
# 
# include/asm-i386/crash_dump.h
#   2004/08/18 09:41:29-04:00 tytso@think.thunk.org +2 -0
#   Fix so that kexec patches compile on systems when 
#   CONFIG_X86_LOCAL_APIC is disabled.
# 
diff -Nru a/include/asm-i386/crash_dump.h b/include/asm-i386/crash_dump.h
--- a/include/asm-i386/crash_dump.h	2004-08-18 09:42:15 -04:00
+++ b/include/asm-i386/crash_dump.h	2004-08-18 09:42:15 -04:00
@@ -53,7 +53,9 @@
 #if defined(CONFIG_X86_IO_APIC)
 		disable_IO_APIC();
 #endif
+#if defined(CONFIG_X86_LOCAL_APIC)
 		disconnect_bsp_APIC();
+#endif
 	}
 }
 #else

Re: [PATCH][4/6]Register snapshotting before kexec-boot

[Hariprasad Nellitheertha]

Hi Ted,

On Wed, Aug 18, 2004 at 09:43:59AM -0400, Theodore Ts'o wrote:
> This patch is needed so that the kexec patches will compile on systems
> that don't have APIC's enabled.

Thanks a lot for the patch.

Regards, Hari
-- 
Hariprasad Nellitheertha
Linux Technology Center
India Software Labs
IBM India, Bangalore

Re: [Fastboot] [RFC]Kexec based crash dumping

[Eric W. Biederman]

Hariprasad Nellitheertha <hari@in.ibm.com> writes:

> Hi,
> 
> The patches that follow contain the initial implementation for kexec based
> crash dumping that we are working on. I had sent this to the fastboot mailing 
> list a couple of weeks ago and this set of patches includes the changes made as
> per feedback from Andrew, Eric and others.

One significant change is missing.  You do not separate out the two
use cases of kexec.  So on a system that is configured to use
call sys_reboot(LINUX_REBOOT_CMD_KEXEC) on reboot I will get a core
dump if using your code.

Or alternatively if I call sys_kexec_load and then something in the
shutdown scripts triggers a kernel panic it is not OK to
start boot the new kernel instead of taking normal panic behavior.

Until the two uses of kexec are separated they two uses
of kexec remain mutually exclusive and incompatible, and it I cannot
merge your patches into the existing kexec patchset.

.....

In general I still your kexec on panic code is doing way to much,
and I think a lot of that  is that you don't have a kernel that will
run when loaded at a different physical address.  

To that end I have written a patch that accomplishes exactly that.
Please see my just released kexec patchset.

Eric

Re: [Fastboot] Re: [RFC]Kexec based crash dumping

[Eric W. Biederman]

Andrew Morton <akpm@osdl.org> writes:

> Hariprasad Nellitheertha <hari@in.ibm.com> wrote:
> >
> > The patches that follow contain the initial implementation for kexec based
> > crash dumping that we are working on.
> 
> It seems to be coming together nicely.

Well it is coming.  There is a lot of exploratory work still needed to
find what it takes to reliably get a kernel working in the hostile
environment present after a kernel panic.  That really has 
barely begun and it likely to turn up some unexpected architecture
specific issues.

Eric

Re: [Fastboot] [RFC]Kexec based crash dumping

[Hariprasad Nellitheertha]

On Fri, Aug 20, 2004 at 02:05:54AM -0600, Eric W. Biederman wrote:
> Hariprasad Nellitheertha <hari@in.ibm.com> writes:
> 
> > Hi,
> > 
> > The patches that follow contain the initial implementation for kexec based
> > crash dumping that we are working on. I had sent this to the fastboot mailing 
> > list a couple of weeks ago and this set of patches includes the changes made as
> > per feedback from Andrew, Eric and others.
> 
> One significant change is missing.  You do not separate out the two
> use cases of kexec.  So on a system that is configured to use
> call sys_reboot(LINUX_REBOOT_CMD_KEXEC) on reboot I will get a core
> dump if using your code.

In some ways, whether we are doing a "crash dump" reboot or a "normal"
kexec reboot is determined by the command line parameters that are
passed to the kexec loaded kernel. If we do not restrict the memory
size and do not add the keyword "dump" in the command line, it will do 
a normal kexec reboot. The dump file will not show up in the second kernel.

> 
> Or alternatively if I call sys_kexec_load and then something in the
> shutdown scripts triggers a kernel panic it is not OK to
> start boot the new kernel instead of taking normal panic behavior.

A way to disable kexec reboot from panic code is indeed needed. I will
add the necessary code to do this.

> 
> Until the two uses of kexec are separated they two uses
> of kexec remain mutually exclusive and incompatible, and it I cannot
> merge your patches into the existing kexec patchset.
> 
> .....
> 
> In general I still your kexec on panic code is doing way to much,
> and I think a lot of that  is that you don't have a kernel that will
> run when loaded at a different physical address.  
> 
> To that end I have written a patch that accomplishes exactly that.
> Please see my just released kexec patchset.

Will work on the necessary changes so we can use your patchset.

> 
> Eric

Thanks and Regards, Hari
-- 
Hariprasad Nellitheertha
Linux Technology Center
India Software Labs
IBM India, Bangalore