[PATCH v21 0/8] arm64: kdump support

[PATCH v21 0/8] arm64: kdump support

[AKASHI Takahiro]

This patch series adds kdump support on arm64.

To load a crash-dump kernel to the systems, a series of patches to
kexec-tools [1], which have not yet been merged upstream, are needed.
Please update to the latest if you have been using an older version.

To examine vmcore (/proc/vmcore) on a crash-dump kernel, you can use
  - crash utility (coming v7.1.6 or later) [2]
    (Necessary patches have already been queued in the master.)

[1]  https://git.kernel.org/cgit/linux/kernel/git/geoff/kexec-tools.git
[2]  https://github.com/crash-utility/crash.git

Changes for v21 (July 6, 2016):

  o Remove kexec patches.
  o Rebase to arm64's for-next/core (Linux-4.7-rc4 based).
  o Clarify the description about kvm in kdump.txt.

See the following link [3] for older changes:
[3]  http://lists.infradead.org/pipermail/linux-arm-kernel/2016-June/438780.html

AKASHI Takahiro (7):
  arm64: kdump: reserve memory for crash dump kernel
  arm64: limit memory regions based on DT property, usable-memory
  arm64: kdump: implement machine_crash_shutdown()
  arm64: kdump: add kdump support
  arm64: kdump: add VMCOREINFO's for user-space coredump tools
  arm64: kdump: enable kdump in the arm64 defconfig
  arm64: kdump: update a kernel doc

James Morse (1):
  Documentation: dt: usable-memory and elfcorehdr nodes for arm64 kexec

 Documentation/devicetree/bindings/chosen.txt |  28 +++++
 Documentation/kdump/kdump.txt                |  16 ++-
 arch/arm64/Kconfig                           |  11 ++
 arch/arm64/configs/defconfig                 |   1 +
 arch/arm64/include/asm/hardirq.h             |   2 +-
 arch/arm64/include/asm/kexec.h               |  41 ++++++-
 arch/arm64/include/asm/smp.h                 |   4 +
 arch/arm64/kernel/Makefile                   |   1 +
 arch/arm64/kernel/crash_dump.c               |  71 ++++++++++++
 arch/arm64/kernel/machine_kexec.c            |  67 +++++++++++-
 arch/arm64/kernel/setup.c                    |   7 +-
 arch/arm64/kernel/smp.c                      |  61 +++++++++++
 arch/arm64/mm/init.c                         | 154 +++++++++++++++++++++++++++
 13 files changed, 457 insertions(+), 7 deletions(-)
 create mode 100644 arch/arm64/kernel/crash_dump.c

-- 
2.9.0

[PATCH v21 1/8] arm64: kdump: reserve memory for crash dump kernel

[AKASHI Takahiro]

On the startup of primary kernel, the memory region used by crash dump
kernel must be specified by "crashkernel=" kernel parameter.
reserve_crashkernel() will allocate and reserve the region for later use.

User space tools, like kexec-tools, will be able to find that region marked
as "Crash kernel" in /proc/iomem.

Signed-off-by: AKASHI Takahiro <takahiro.akashi@linaro.org>
Signed-off-by: Mark Salter <msalter@redhat.com>
Signed-off-by: Pratyush Anand <panand@redhat.com>
Reviewed-by: James Morse <james.morse@arm.com>
---
 arch/arm64/kernel/setup.c |  7 +++++-
 arch/arm64/mm/init.c      | 62 +++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 68 insertions(+), 1 deletion(-)

diff --git a/arch/arm64/kernel/setup.c b/arch/arm64/kernel/setup.c
index c1509e6..cb5eee0 100644
--- a/arch/arm64/kernel/setup.c
+++ b/arch/arm64/kernel/setup.c
@@ -31,7 +31,6 @@
 #include <linux/screen_info.h>
 #include <linux/init.h>
 #include <linux/kexec.h>
-#include <linux/crash_dump.h>
 #include <linux/root_dev.h>
 #include <linux/cpu.h>
 #include <linux/interrupt.h>
@@ -222,6 +221,12 @@ static void __init request_standard_resources(void)
 		    kernel_data.end <= res->end)
 			request_resource(res, &kernel_data);
 	}
+
+#ifdef CONFIG_KEXEC_CORE
+	/* User space tools will find "Crash kernel" region in /proc/iomem. */
+	if (crashk_res.end)
+		insert_resource(&iomem_resource, &crashk_res);
+#endif
 }
 
 u64 __cpu_logical_map[NR_CPUS] = { [0 ... NR_CPUS-1] = INVALID_HWID };
diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
index 2ade7a6..e1134fb 100644
--- a/arch/arm64/mm/init.c
+++ b/arch/arm64/mm/init.c
@@ -34,6 +34,7 @@
 #include <linux/dma-contiguous.h>
 #include <linux/efi.h>
 #include <linux/swiotlb.h>
+#include <linux/kexec.h>
 
 #include <asm/boot.h>
 #include <asm/fixmap.h>
@@ -76,6 +77,65 @@ static int __init early_initrd(char *p)
 early_param("initrd", early_initrd);
 #endif
 
+#ifdef CONFIG_KEXEC_CORE
+/*
+ * reserve_crashkernel() - reserves memory for crash kernel
+ *
+ * This function reserves memory area given in "crashkernel=" kernel command
+ * line parameter. The memory reserved is used by dump capture kernel when
+ * primary kernel is crashing.
+ */
+static void __init reserve_crashkernel(void)
+{
+	unsigned long long crash_size = 0, crash_base = 0;
+	int ret;
+
+	ret = parse_crashkernel(boot_command_line, memblock_phys_mem_size(),
+				&crash_size, &crash_base);
+	/* no crashkernel= or invalid value specified */
+	if (ret || !crash_size)
+		return;
+
+	if (crash_base == 0) {
+		/* Current arm64 boot protocol requires 2MB alignment */
+		crash_base = memblock_find_in_range(0,
+				MEMBLOCK_ALLOC_ACCESSIBLE, crash_size, SZ_2M);
+		if (crash_base == 0) {
+			pr_warn("Unable to allocate crashkernel (size:%llx)\n",
+				crash_size);
+			return;
+		}
+		memblock_reserve(crash_base, crash_size);
+
+	} else {
+		/* User specifies base address explicitly. */
+		if (!memblock_is_region_memory(crash_base, crash_size) ||
+			memblock_is_region_reserved(crash_base, crash_size)) {
+			pr_warn("crashkernel has wrong address or size\n");
+			return;
+		}
+
+		if (!IS_ALIGNED(crash_base, SZ_2M)) {
+			pr_warn("crashkernel base address is not 2MB aligned\n");
+			return;
+		}
+
+		memblock_reserve(crash_base, crash_size);
+	}
+
+	pr_info("Reserving %lldMB of memory at %lldMB for crashkernel\n",
+		crash_size >> 20, crash_base >> 20);
+
+	crashk_res.start = crash_base;
+	crashk_res.end = crash_base + crash_size - 1;
+}
+#else
+static void __init reserve_crashkernel(void)
+{
+	;
+}
+#endif /* CONFIG_KEXEC_CORE */
+
 /*
  * Return the maximum physical address for ZONE_DMA (DMA_BIT_MASK(32)). It
  * currently assumes that for memory starting above 4G, 32-bit devices will
@@ -289,6 +349,8 @@ void __init arm64_memblock_init(void)
 	}
 #endif
 
+	reserve_crashkernel();
+
 	early_init_fdt_scan_reserved_mem();
 
 	/* 4GB maximum for 32-bit only capable devices */
-- 
2.9.0

[PATCH v21 1/8] arm64: kdump: reserve memory for crash dump kernel

[Geoff Levand]

On Wed, 2016-07-06 at 16:52 +0900, AKASHI Takahiro wrote:
>  
> +#ifdef CONFIG_KEXEC_CORE
> +/*
> + * reserve_crashkernel() - reserves memory for crash kernel
> + *
> + * This function reserves memory area given in "crashkernel=" kernel command
> + * line parameter. The memory reserved is used by dump capture kernel when
> + * primary kernel is crashing.
> + */
> +static void __init reserve_crashkernel(void)
> +{
> +> 	> unsigned long long crash_size = 0, crash_base = 0;
> +> 	> int ret;
> +
> +> 	> ret = parse_crashkernel(boot_command_line, memblock_phys_mem_size(),
> +> 	> 	> 	> 	> &crash_size, &crash_base);
> +> 	> /* no crashkernel= or invalid value specified */
> +> 	> if (ret || !crash_size)
> +> 	> 	> return;
> +
> +> 	> if (crash_base == 0) {
> +> 	> 	> /* Current arm64 boot protocol requires 2MB alignment */
> +> 	> 	> crash_base = memblock_find_in_range(0,
> +> 	> 	> 	> 	> MEMBLOCK_ALLOC_ACCESSIBLE, crash_size, SZ_2M);
> +> 	> 	> if (crash_base == 0) {
> +> 	> 	> 	> pr_warn("Unable to allocate crashkernel (size:%llx)\n",
> +> 	> 	> 	> 	> crash_size);
> +> 	> 	> 	> return;
> +> 	> 	> }
> +> 	> 	> memblock_reserve(crash_base, crash_size);
> +
> +> 	> } else {
> +> 	> 	> /* User specifies base address explicitly. */
> +> 	> 	> if (!memblock_is_region_memory(crash_base, crash_size) ||
> +> 	> 	> 	> memblock_is_region_reserved(crash_base, crash_size)) {
> +> 	> 	> 	> pr_warn("crashkernel has wrong address or size\n");
> +> 	> 	> 	> return;
> +> 	> 	> }
> +
> +> 	> 	> if (!IS_ALIGNED(crash_base, SZ_2M)) {
> +> 	> 	> 	> pr_warn("crashkernel base address is not 2MB aligned\n");
> +> 	> 	> 	> return;
> +> 	> 	> }
> +
> +> 	> 	> memblock_reserve(crash_base, crash_size);
> +> 	> }
> +
> +> 	> pr_info("Reserving %lldMB of memory at %lldMB for crashkernel\n",
> +> 	> 	> crash_size >> 20, crash_base >> 20);
> +
> +> 	> crashk_res.start = crash_base;
> +> 	> crashk_res.end = crash_base + crash_size - 1;
> +}
> +#else
> +static void __init reserve_crashkernel(void)
> +{
> +> 	> ;
> +}
> +#endif /* CONFIG_KEXEC_CORE */
> +

I think it is a bit cleaner if you move this conditional
into the function and have something like:

static void __init reserve_crashkernel(void)
{
#ifdef CONFIG_KEXEC_CORE
#else
#endif
}

-Geoff

[PATCH v21 1/8] arm64: kdump: reserve memory for crash dump kernel

[AKASHI Takahiro]

Geoff,

Thank you for your reviewing.

On Wed, Jul 06, 2016 at 10:00:18AM -0700, Geoff Levand wrote:
> On Wed, 2016-07-06 at 16:52 +0900, AKASHI Takahiro wrote:
> >  
> > +#ifdef CONFIG_KEXEC_CORE
> > +/*
> > + * reserve_crashkernel() - reserves memory for crash kernel
> > + *
> > + * This function reserves memory area given in "crashkernel=" kernel command
> > + * line parameter. The memory reserved is used by dump capture kernel when
> > + * primary kernel is crashing.
> > + */
> > +static void __init reserve_crashkernel(void)
> > +{
> > +> 	> unsigned long long crash_size = 0, crash_base = 0;
> > +> 	> int ret;
> > +
> > +> 	> ret = parse_crashkernel(boot_command_line, memblock_phys_mem_size(),
> > +> 	> 	> 	> 	> &crash_size, &crash_base);
> > +> 	> /* no crashkernel= or invalid value specified */
> > +> 	> if (ret || !crash_size)
> > +> 	> 	> return;
> > +
> > +> 	> if (crash_base == 0) {
> > +> 	> 	> /* Current arm64 boot protocol requires 2MB alignment */
> > +> 	> 	> crash_base = memblock_find_in_range(0,
> > +> 	> 	> 	> 	> MEMBLOCK_ALLOC_ACCESSIBLE, crash_size, SZ_2M);
> > +> 	> 	> if (crash_base == 0) {
> > +> 	> 	> 	> pr_warn("Unable to allocate crashkernel (size:%llx)\n",
> > +> 	> 	> 	> 	> crash_size);
> > +> 	> 	> 	> return;
> > +> 	> 	> }
> > +> 	> 	> memblock_reserve(crash_base, crash_size);
> > +
> > +> 	> } else {
> > +> 	> 	> /* User specifies base address explicitly. */
> > +> 	> 	> if (!memblock_is_region_memory(crash_base, crash_size) ||
> > +> 	> 	> 	> memblock_is_region_reserved(crash_base, crash_size)) {
> > +> 	> 	> 	> pr_warn("crashkernel has wrong address or size\n");
> > +> 	> 	> 	> return;
> > +> 	> 	> }
> > +
> > +> 	> 	> if (!IS_ALIGNED(crash_base, SZ_2M)) {
> > +> 	> 	> 	> pr_warn("crashkernel base address is not 2MB aligned\n");
> > +> 	> 	> 	> return;
> > +> 	> 	> }
> > +
> > +> 	> 	> memblock_reserve(crash_base, crash_size);
> > +> 	> }
> > +
> > +> 	> pr_info("Reserving %lldMB of memory at %lldMB for crashkernel\n",
> > +> 	> 	> crash_size >> 20, crash_base >> 20);
> > +
> > +> 	> crashk_res.start = crash_base;
> > +> 	> crashk_res.end = crash_base + crash_size - 1;
> > +}
> > +#else
> > +static void __init reserve_crashkernel(void)
> > +{
> > +> 	> ;
> > +}
> > +#endif /* CONFIG_KEXEC_CORE */
> > +

I know that this is just a matter of preference, but
I can see more use cases of the above than the below
that you mentioned under arm64 directory.
So I want to keep the current style unchanged unless
the maintainers don't like it.

Thanks,
-Takahiro AKASHI

> I think it is a bit cleaner if you move this conditional
> into the function and have something like:
> 
> static void __init reserve_crashkernel(void)
> {
> #ifdef CONFIG_KEXEC_CORE
> #else
> #endif
> }
>
> -Geoff
> 

[PATCH v21 1/8] arm64: kdump: reserve memory for crash dump kernel

[Thiago Jung Bauermann]

Am Mittwoch, 06 Juli 2016, 16:52:19 schrieb AKASHI Takahiro:
> On the startup of primary kernel, the memory region used by crash dump
> kernel must be specified by "crashkernel=" kernel parameter.
> reserve_crashkernel() will allocate and reserve the region for later use.
> 
> User space tools, like kexec-tools, will be able to find that region
> marked as "Crash kernel" in /proc/iomem.

On powerpc, userspace tools get everything from the device tree (exposed to 
userspace in /proc/device-tree/), not /proc/iomem.

In the case of the crashkernel reserved region, that information is in 
/chosen/linux,crashkernel-base and /chosen/linux,crashkernel-size.

Either way is fine I think. I'm just mentioning this for reference in case 
you want the ARM implementation to be closer to another arch which is also 
based on the device tree.

-- 
[]'s
Thiago Jung Bauermann
IBM Linux Technology Center

[PATCH v21 1/8] arm64: kdump: reserve memory for crash dump kernel

[AKASHI Takahiro]

On Wed, Jul 06, 2016 at 03:11:45PM -0300, Thiago Jung Bauermann wrote:
> Am Mittwoch, 06 Juli 2016, 16:52:19 schrieb AKASHI Takahiro:
> > On the startup of primary kernel, the memory region used by crash dump
> > kernel must be specified by "crashkernel=" kernel parameter.
> > reserve_crashkernel() will allocate and reserve the region for later use.
> > 
> > User space tools, like kexec-tools, will be able to find that region
> > marked as "Crash kernel" in /proc/iomem.
> 
> On powerpc, userspace tools get everything from the device tree (exposed to 
> userspace in /proc/device-tree/), not /proc/iomem.

I started my kdump patches with arm (not arm64) implementation, and
so /proc/iomem is used for consistency. Since my kexec-tools (for arm64)
have to access /proc/iomem to identify usable memory regions on
*UEFI* systems anyway, I didn't think it was odd.

(Please note that UEFI won't expose memory regions information via
a device tree, but via ACPI table.)

> In the case of the crashkernel reserved region, that information is in 
> /chosen/linux,crashkernel-base and /chosen/linux,crashkernel-size.
> 
> Either way is fine I think. I'm just mentioning this for reference in case 
> you want the ARM implementation to be closer to another arch which is also 
> based on the device tree.

Adding those properties to a device tree is quite easy
(and it won't conflict with /proc/iomem), so I'd like 
to defer to arm64 maintainers.

Thanks,
-Takahiro AKASHI

> -- 
> []'s
> Thiago Jung Bauermann
> IBM Linux Technology Center
> 

[PATCH v21 2/8] arm64: limit memory regions based on DT property, usable-memory

[AKASHI Takahiro]

Crash dump kernel will be run with a limited range of memory as System
RAM.

On arm64, we will use a device-tree property under /chosen,
   linux,usable-memory = <BASE SIZE>
in order for primary kernel either on uefi or non-uefi (device tree only)
system to hand over the information about usable memory region to crash
dump kernel. This property will supercede entries in uefi memory map table
and "memory" nodes in a device tree.

Signed-off-by: AKASHI Takahiro <takahiro.akashi@linaro.org>
---
 arch/arm64/mm/init.c | 38 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 38 insertions(+)

diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
index e1134fb..88fd99a 100644
--- a/arch/arm64/mm/init.c
+++ b/arch/arm64/mm/init.c
@@ -247,10 +247,48 @@ static int __init early_mem(char *p)
 }
 early_param("mem", early_mem);
 
+static int __init early_init_dt_scan_usablemem(unsigned long node,
+		const char *uname, int depth, void *data)
+{
+	struct memblock_region *usablemem = (struct memblock_region *)data;
+	const __be32 *reg;
+	int len;
+
+	usablemem->size = 0;
+
+	if (depth != 1 || strcmp(uname, "chosen") != 0)
+		return 0;
+
+	reg = of_get_flat_dt_prop(node, "linux,usable-memory", &len);
+	if (!reg || (len < (dt_root_addr_cells + dt_root_size_cells)))
+		return 1;
+
+	usablemem->base = dt_mem_next_cell(dt_root_addr_cells, &reg);
+	usablemem->size = dt_mem_next_cell(dt_root_size_cells, &reg);
+
+	return 1;
+}
+
+static void __init fdt_enforce_memory_region(void)
+{
+	struct memblock_region reg;
+
+	of_scan_flat_dt(early_init_dt_scan_usablemem, &reg);
+
+	if (reg.size) {
+		memblock_remove(0, PAGE_ALIGN(reg.base));
+		memblock_remove(round_down(reg.base + reg.size, PAGE_SIZE),
+				ULLONG_MAX);
+	}
+}
+
 void __init arm64_memblock_init(void)
 {
 	const s64 linear_region_size = -(s64)PAGE_OFFSET;
 
+	/* Handle linux,usable-memory property */
+	fdt_enforce_memory_region();
+
 	/*
 	 * Ensure that the linear region takes up exactly half of the kernel
 	 * virtual address space. This way, we can distinguish a linear address
-- 
2.9.0

[PATCH v21 2/8] arm64: limit memory regions based on DT property, usable-memory

[Geoff Levand]

On Wed, 2016-07-06 at 16:52 +0900, AKASHI Takahiro wrote:
> Crash dump kernel will be run with a limited range of memory as System
> RAM.
> 
> On arm64, we will use a device-tree property under /chosen,
>    linux,usable-memory = 
> in order for primary kernel either on uefi or non-uefi (device tree only)
> system to hand over the information about usable memory region to crash
> dump kernel. This property will supercede entries in uefi memory map table
> and "memory" nodes in a device tree.
> 
> Signed-off-by: AKASHI Takahiro <takahiro.akashi@linaro.org>

This all looks OK.

Reviewed-by: Geoff Levand <geoff@infradead.org>

[PATCH v21 3/8] arm64: kdump: implement machine_crash_shutdown()

[AKASHI Takahiro]

Primary kernel calls machine_crash_shutdown() to shut down non-boot cpus
and save registers' status in per-cpu ELF notes before starting crash
dump kernel. See kernel_kexec().
Even if not all secondary cpus have shut down, we do kdump anyway.

As we don't have to make non-boot(crashed) cpus offline (to preserve
correct status of cpus at crash dump) before shutting down, this patch
also adds a variant of smp_send_stop().

Signed-off-by: AKASHI Takahiro <takahiro.akashi@linaro.org>
---
 arch/arm64/include/asm/hardirq.h  |  2 +-
 arch/arm64/include/asm/kexec.h    | 41 +++++++++++++++++++++++++-
 arch/arm64/include/asm/smp.h      |  4 +++
 arch/arm64/kernel/machine_kexec.c | 56 +++++++++++++++++++++++++++++++++--
 arch/arm64/kernel/smp.c           | 61 +++++++++++++++++++++++++++++++++++++++
 5 files changed, 159 insertions(+), 5 deletions(-)

diff --git a/arch/arm64/include/asm/hardirq.h b/arch/arm64/include/asm/hardirq.h
index 8740297..1473fc2 100644
--- a/arch/arm64/include/asm/hardirq.h
+++ b/arch/arm64/include/asm/hardirq.h
@@ -20,7 +20,7 @@
 #include <linux/threads.h>
 #include <asm/irq.h>
 
-#define NR_IPI	6
+#define NR_IPI	7
 
 typedef struct {
 	unsigned int __softirq_pending;
diff --git a/arch/arm64/include/asm/kexec.h b/arch/arm64/include/asm/kexec.h
index 04744dc..a908958 100644
--- a/arch/arm64/include/asm/kexec.h
+++ b/arch/arm64/include/asm/kexec.h
@@ -40,7 +40,46 @@
 static inline void crash_setup_regs(struct pt_regs *newregs,
 				    struct pt_regs *oldregs)
 {
-	/* Empty routine needed to avoid build errors. */
+	if (oldregs) {
+		memcpy(newregs, oldregs, sizeof(*newregs));
+	} else {
+		u64 tmp1, tmp2;
+
+		__asm__ __volatile__ (
+			"stp	 x0,   x1, [%2, #16 *  0]\n"
+			"stp	 x2,   x3, [%2, #16 *  1]\n"
+			"stp	 x4,   x5, [%2, #16 *  2]\n"
+			"stp	 x6,   x7, [%2, #16 *  3]\n"
+			"stp	 x8,   x9, [%2, #16 *  4]\n"
+			"stp	x10,  x11, [%2, #16 *  5]\n"
+			"stp	x12,  x13, [%2, #16 *  6]\n"
+			"stp	x14,  x15, [%2, #16 *  7]\n"
+			"stp	x16,  x17, [%2, #16 *  8]\n"
+			"stp	x18,  x19, [%2, #16 *  9]\n"
+			"stp	x20,  x21, [%2, #16 * 10]\n"
+			"stp	x22,  x23, [%2, #16 * 11]\n"
+			"stp	x24,  x25, [%2, #16 * 12]\n"
+			"stp	x26,  x27, [%2, #16 * 13]\n"
+			"stp	x28,  x29, [%2, #16 * 14]\n"
+			"mov	 %0,  sp\n"
+			"stp	x30,  %0,  [%2, #16 * 15]\n"
+
+			"/* faked current PSTATE */\n"
+			"mrs	 %0, CurrentEL\n"
+			"mrs	 %1, DAIF\n"
+			"orr	 %0, %0, %1\n"
+			"mrs	 %1, NZCV\n"
+			"orr	 %0, %0, %1\n"
+
+			/* pc */
+			"adr	 %1, 1f\n"
+		"1:\n"
+			"stp	 %1, %0,   [%2, #16 * 16]\n"
+			: "=r" (tmp1), "=r" (tmp2), "+r" (newregs)
+			:
+			: "memory"
+		);
+	}
 }
 
 #endif /* __ASSEMBLY__ */
diff --git a/arch/arm64/include/asm/smp.h b/arch/arm64/include/asm/smp.h
index 0226447..6af76f59 100644
--- a/arch/arm64/include/asm/smp.h
+++ b/arch/arm64/include/asm/smp.h
@@ -136,6 +136,10 @@ static inline void cpu_panic_kernel(void)
  */
 bool cpus_are_stuck_in_kernel(void);
 
+#ifdef CONFIG_KEXEC_CORE
+extern void smp_send_crash_stop(void);
+#endif
+
 #endif /* ifndef __ASSEMBLY__ */
 
 #endif /* ifndef __ASM_SMP_H */
diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c
index bc96c8a..8ac9dba8 100644
--- a/arch/arm64/kernel/machine_kexec.c
+++ b/arch/arm64/kernel/machine_kexec.c
@@ -9,6 +9,9 @@
  * published by the Free Software Foundation.
  */
 
+#include <linux/interrupt.h>
+#include <linux/irq.h>
+#include <linux/kernel.h>
 #include <linux/kexec.h>
 #include <linux/smp.h>
 
@@ -22,6 +25,7 @@
 extern const unsigned char arm64_relocate_new_kernel[];
 extern const unsigned long arm64_relocate_new_kernel_size;
 
+bool in_crash_kexec;
 static unsigned long kimage_start;
 
 /**
@@ -148,7 +152,8 @@ void machine_kexec(struct kimage *kimage)
 	/*
 	 * New cpus may have become stuck_in_kernel after we loaded the image.
 	 */
-	BUG_ON(cpus_are_stuck_in_kernel() || (num_online_cpus() > 1));
+	BUG_ON((cpus_are_stuck_in_kernel() || (num_online_cpus() > 1)) &&
+			!WARN_ON(in_crash_kexec));
 
 	reboot_code_buffer_phys = page_to_phys(kimage->control_code_page);
 	reboot_code_buffer = phys_to_virt(reboot_code_buffer_phys);
@@ -200,13 +205,58 @@ void machine_kexec(struct kimage *kimage)
 	 * relocation is complete.
 	 */
 
-	cpu_soft_restart(1, reboot_code_buffer_phys, kimage->head,
+	cpu_soft_restart(!in_crash_kexec, reboot_code_buffer_phys, kimage->head,
 		kimage_start, 0);
 
 	BUG(); /* Should never get here. */
 }
 
+static void machine_kexec_mask_interrupts(void)
+{
+	unsigned int i;
+	struct irq_desc *desc;
+
+	for_each_irq_desc(i, desc) {
+		struct irq_chip *chip;
+		int ret;
+
+		chip = irq_desc_get_chip(desc);
+		if (!chip)
+			continue;
+
+		/*
+		 * First try to remove the active state. If this
+		 * fails, try to EOI the interrupt.
+		 */
+		ret = irq_set_irqchip_state(i, IRQCHIP_STATE_ACTIVE, false);
+
+		if (ret && irqd_irq_inprogress(&desc->irq_data) &&
+		    chip->irq_eoi)
+			chip->irq_eoi(&desc->irq_data);
+
+		if (chip->irq_mask)
+			chip->irq_mask(&desc->irq_data);
+
+		if (chip->irq_disable && !irqd_irq_disabled(&desc->irq_data))
+			chip->irq_disable(&desc->irq_data);
+	}
+}
+
+/**
+ * machine_crash_shutdown - shutdown non-crashing cpus and save registers
+ */
 void machine_crash_shutdown(struct pt_regs *regs)
 {
-	/* Empty routine needed to avoid build errors. */
+	local_irq_disable();
+
+	in_crash_kexec = true;
+
+	/* shutdown non-crashing cpus */
+	smp_send_crash_stop();
+
+	/* for crashing cpu */
+	crash_save_cpu(regs, smp_processor_id());
+	machine_kexec_mask_interrupts();
+
+	pr_info("Starting crashdump kernel...\n");
 }
diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c
index 62ff3c0..51e0b73 100644
--- a/arch/arm64/kernel/smp.c
+++ b/arch/arm64/kernel/smp.c
@@ -37,6 +37,7 @@
 #include <linux/completion.h>
 #include <linux/of.h>
 #include <linux/irq_work.h>
+#include <linux/kexec.h>
 
 #include <asm/alternative.h>
 #include <asm/atomic.h>
@@ -71,6 +72,7 @@ enum ipi_msg_type {
 	IPI_RESCHEDULE,
 	IPI_CALL_FUNC,
 	IPI_CPU_STOP,
+	IPI_CPU_CRASH_STOP,
 	IPI_TIMER,
 	IPI_IRQ_WORK,
 	IPI_WAKEUP
@@ -726,6 +728,7 @@ static const char *ipi_types[NR_IPI] __tracepoint_string = {
 	S(IPI_RESCHEDULE, "Rescheduling interrupts"),
 	S(IPI_CALL_FUNC, "Function call interrupts"),
 	S(IPI_CPU_STOP, "CPU stop interrupts"),
+	S(IPI_CPU_CRASH_STOP, "CPU stop (for crash dump) interrupts"),
 	S(IPI_TIMER, "Timer broadcast interrupts"),
 	S(IPI_IRQ_WORK, "IRQ work interrupts"),
 	S(IPI_WAKEUP, "CPU wake-up interrupts"),
@@ -800,6 +803,28 @@ static void ipi_cpu_stop(unsigned int cpu)
 		cpu_relax();
 }
 
+#ifdef CONFIG_KEXEC_CORE
+static atomic_t waiting_for_crash_ipi;
+
+static void ipi_cpu_crash_stop(unsigned int cpu, struct pt_regs *regs)
+{
+	crash_save_cpu(regs, cpu);
+
+	atomic_dec(&waiting_for_crash_ipi);
+
+	local_irq_disable();
+
+#ifdef CONFIG_HOTPLUG_CPU
+	if (cpu_ops[cpu]->cpu_die)
+		cpu_ops[cpu]->cpu_die(cpu);
+#endif
+
+	/* just in case */
+	while (1)
+		wfi();
+}
+#endif
+
 /*
  * Main handler for inter-processor interrupts
  */
@@ -830,6 +855,14 @@ void handle_IPI(int ipinr, struct pt_regs *regs)
 		irq_exit();
 		break;
 
+#ifdef CONFIG_KEXEC_CORE
+	case IPI_CPU_CRASH_STOP:
+		irq_enter();
+		ipi_cpu_crash_stop(cpu, regs);
+
+		unreachable();
+#endif
+
 #ifdef CONFIG_GENERIC_CLOCKEVENTS_BROADCAST
 	case IPI_TIMER:
 		irq_enter();
@@ -902,6 +935,34 @@ void smp_send_stop(void)
 			   cpumask_pr_args(cpu_online_mask));
 }
 
+#ifdef CONFIG_KEXEC_CORE
+void smp_send_crash_stop(void)
+{
+	cpumask_t mask;
+	unsigned long timeout;
+
+	if (num_online_cpus() == 1)
+		return;
+
+	cpumask_copy(&mask, cpu_online_mask);
+	cpumask_clear_cpu(smp_processor_id(), &mask);
+
+	atomic_set(&waiting_for_crash_ipi, num_online_cpus() - 1);
+
+	pr_crit("SMP: stopping secondary CPUs\n");
+	smp_cross_call(&mask, IPI_CPU_CRASH_STOP);
+
+	/* Wait up to one second for other CPUs to stop */
+	timeout = USEC_PER_SEC;
+	while ((atomic_read(&waiting_for_crash_ipi) > 0) && timeout--)
+		udelay(1);
+
+	if (atomic_read(&waiting_for_crash_ipi) > 0)
+		pr_warning("SMP: failed to stop secondary CPUs %*pbl\n",
+			   cpumask_pr_args(cpu_online_mask));
+}
+#endif
+
 /*
  * not supported here
  */
-- 
2.9.0

[PATCH v21 3/8] arm64: kdump: implement machine_crash_shutdown()

[Geoff Levand]

On Wed, 2016-07-06 at 16:52 +0900, AKASHI Takahiro wrote:
> --- a/arch/arm64/include/asm/smp.h
> +++ b/arch/arm64/include/asm/smp.h
> @@ -136,6 +136,10 @@ static inline void cpu_panic_kernel(void)
>   */
>  bool cpus_are_stuck_in_kernel(void);
>  
> +#ifdef CONFIG_KEXEC_CORE
> +extern void smp_send_crash_stop(void);
> +#endif

It seems this could be simplified by removing these
conditionals, then move the conditionals that are
around the smp_send_crash_stop definition to inside
that definition.

> diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c
> index 62ff3c0..51e0b73 100644
> --- a/arch/arm64/kernel/smp.c
> +++ b/arch/arm64/kernel/smp.c
> @@ -800,6 +803,28 @@ static void ipi_cpu_stop(unsigned int cpu)
>  > 	> 	> cpu_relax();
>  }
>  
> +#ifdef CONFIG_KEXEC_CORE
> +static atomic_t waiting_for_crash_ipi;
> +
> +static void ipi_cpu_crash_stop(unsigned int cpu, struct pt_regs *regs)
> +{
> +> 	> crash_save_cpu(regs, cpu);
> +
> +> 	> atomic_dec(&waiting_for_crash_ipi);
> +
> +> 	> local_irq_disable();
> +
> +#ifdef CONFIG_HOTPLUG_CPU
> +> 	> if (cpu_ops[cpu]->cpu_die)
> +> 	> 	> cpu_ops[cpu]->cpu_die(cpu);
> +#endif
> +
> +> 	> /* just in case */
> +> 	> while (1)
> +> 	> 	> wfi();
> +}
> +#endif

 /*
>   * Main handler for inter-processor interrupts
>   */
> @@ -830,6 +855,14 @@ void handle_IPI(int ipinr, struct pt_regs *regs)
>  > 	> 	> irq_exit();
>  > 	> 	> break;
>  
> +#ifdef CONFIG_KEXEC_CORE
> +> 	> case IPI_CPU_CRASH_STOP:
> +> 	> 	> irq_enter();
> +> 	> 	> ipi_cpu_crash_stop(cpu, regs);
> +
> +> 	> 	> unreachable();
> +#endif

It seems like you could simplifiy with something like:

	case IPI_CPU_CRASH_STOP:
		if (IS_ENABLED(CONFIG_KEXEC_CORE) {
			irq_enter();
			ipi_cpu_crash_stop(cpu, regs);
			unreachable();
		}

Then move the CONFIG_KEXEC_CORE conditional around
ipi_cpu_crash_stop to just around crash_save_cpu().

 -Geoff

[PATCH v21 3/8] arm64: kdump: implement machine_crash_shutdown()

[AKASHI Takahiro]

On Wed, Jul 06, 2016 at 10:28:57AM -0700, Geoff Levand wrote:
> On Wed, 2016-07-06 at 16:52 +0900, AKASHI Takahiro wrote:
> > --- a/arch/arm64/include/asm/smp.h
> > +++ b/arch/arm64/include/asm/smp.h
> > @@ -136,6 +136,10 @@ static inline void cpu_panic_kernel(void)
> >   */
> >  bool cpus_are_stuck_in_kernel(void);
> >  
> > +#ifdef CONFIG_KEXEC_CORE
> > +extern void smp_send_crash_stop(void);
> > +#endif
> 
> It seems this could be simplified by removing these
> conditionals, then move the conditionals that are
> around the smp_send_crash_stop definition to inside
> that definition.

OK.

> > diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c
> > index 62ff3c0..51e0b73 100644
> > --- a/arch/arm64/kernel/smp.c
> > +++ b/arch/arm64/kernel/smp.c
> > @@ -800,6 +803,28 @@ static void ipi_cpu_stop(unsigned int cpu)
> >  > 	> 	> cpu_relax();
> >  }
> >  
> > +#ifdef CONFIG_KEXEC_CORE
> > +static atomic_t waiting_for_crash_ipi;
> > +
> > +static void ipi_cpu_crash_stop(unsigned int cpu, struct pt_regs *regs)
> > +{
> > +> 	> crash_save_cpu(regs, cpu);
> > +
> > +> 	> atomic_dec(&waiting_for_crash_ipi);
> > +
> > +> 	> local_irq_disable();
> > +
> > +#ifdef CONFIG_HOTPLUG_CPU
> > +> 	> if (cpu_ops[cpu]->cpu_die)
> > +> 	> 	> cpu_ops[cpu]->cpu_die(cpu);
> > +#endif
> > +
> > +> 	> /* just in case */
> > +> 	> while (1)
> > +> 	> 	> wfi();
> > +}
> > +#endif
> 
>  /*
> >   * Main handler for inter-processor interrupts
> >   */
> > @@ -830,6 +855,14 @@ void handle_IPI(int ipinr, struct pt_regs *regs)
> >  > 	> 	> irq_exit();
> >  > 	> 	> break;
> >  
> > +#ifdef CONFIG_KEXEC_CORE
> > +> 	> case IPI_CPU_CRASH_STOP:
> > +> 	> 	> irq_enter();
> > +> 	> 	> ipi_cpu_crash_stop(cpu, regs);
> > +
> > +> 	> 	> unreachable();
> > +#endif
> 
> It seems like you could simplifiy with something like:
> 
> 	case IPI_CPU_CRASH_STOP:
> 		if (IS_ENABLED(CONFIG_KEXEC_CORE) {
> 			irq_enter();
> 			ipi_cpu_crash_stop(cpu, regs);
> 			unreachable();
> 		}

OK, I will take this.

> Then move the CONFIG_KEXEC_CORE conditional around
> ipi_cpu_crash_stop to just around crash_save_cpu().

Yeah, we can do that, but I added these conditionals just because
the kernel size can be reduced (a bit, obviously) if kexec is
not needed.
So instead of removing the conditional around ipi_cpu_crash_stop(),
an empty definition will be added for !CONFIG_KEXEC_CORE.

Thanks,
-Takahiro AKASHI

>  -Geoff

[PATCH v21 3/8] arm64: kdump: implement machine_crash_shutdown()

[Geoff Levand]

Hi,

On Thu, 2016-07-07 at 09:43 +0900, AKASHI Takahiro wrote:
> On Wed, Jul 06, 2016 at 10:28:57AM -0700, Geoff Levand wrote:
> > On Wed, 2016-07-06 at 16:52 +0900, AKASHI Takahiro wrote:
> > > @@ -830,6 +855,14 @@ void handle_IPI(int ipinr, struct pt_regs
> > > *regs)
> > >  > > > > 	> > > > > > > 	> > > > irq_exit();
> > >  > > > > 	> > > > > > > 	> > > > break;
> > >  
> > > +#ifdef CONFIG_KEXEC_CORE
> > > +> > > > 	> > > > case IPI_CPU_CRASH_STOP:
> > > +> > > > 	> > > > > > > 	> > > > irq_enter();
> > > +> > > > 	> > > > > > > 	> > > > ipi_cpu_crash_stop(cpu, regs);
> > > +
> > > +> > > > 	> > > > > > > 	> > > > unreachable();
> > > +#endif
> > 
> > It seems like you could simplifiy with something like:
> > 
> > 	> > case IPI_CPU_CRASH_STOP:
> > 	> > 	> > if (IS_ENABLED(CONFIG_KEXEC_CORE) {
> > 	> > 	> > 	> > irq_enter();
> > 	> > 	> > 	> > ipi_cpu_crash_stop(cpu, regs);
> > 	> > 	> > 	> > unreachable();
> > 	> > 	> > }
> 
> OK, I will take this.
> 
> > Then move the CONFIG_KEXEC_CORE conditional around
> > ipi_cpu_crash_stop to just around crash_save_cpu().
> 
> Yeah, we can do that, but I added these conditionals just because
> the kernel size can be reduced (a bit, obviously) if kexec is
> not needed.

The linker will remove the unused ipi_cpu_crash_stop since
it is static.  atomic_t waiting_for_crash_ipi also becomes
unused, and I think it will be removed also.  The motivation
for these changes is to get a build test of this code
even when CONFIG_KEXEC_CORE=n.

-Geoff

[PATCH v21 4/8] arm64: kdump: add kdump support

[AKASHI Takahiro]

On crash dump kernel, all the information about primary kernel's system
memory (core image) is available in elf core header.
The primary kernel will set aside this header with reserve_elfcorehdr()
at boot time and inform crash dump kernel of its location via a new
device-tree property, "linux,elfcorehdr".

Please note that all other architectures use traditional "elfcorehdr="
kernel parameter for this purpose.

Then crash dump kernel will access the primary kernel's memory with
copy_oldmem_page(), which reads one page by ioremap'ing it since it does
not reside in linear mapping on crash dump kernel.

We also need our own elfcorehdr_read() here since the header is placed
within crash dump kernel's usable memory.

Signed-off-by: AKASHI Takahiro <takahiro.akashi@linaro.org>
---
 arch/arm64/Kconfig             | 11 +++++++
 arch/arm64/kernel/Makefile     |  1 +
 arch/arm64/kernel/crash_dump.c | 71 ++++++++++++++++++++++++++++++++++++++++++
 arch/arm64/mm/init.c           | 54 ++++++++++++++++++++++++++++++++
 4 files changed, 137 insertions(+)
 create mode 100644 arch/arm64/kernel/crash_dump.c

diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index 1b196bf..ad2c53e 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -675,6 +675,17 @@ config KEXEC
 	  but it is independent of the system firmware.   And like a reboot
 	  you can start any kernel with it, not just Linux.
 
+config CRASH_DUMP
+	bool "Build kdump crash kernel"
+	help
+	  Generate crash dump after being started by kexec. This should
+	  be normally only set in special crash dump kernels which are
+	  loaded in the main kernel with kexec-tools into a specially
+	  reserved region and then later executed after a crash by
+	  kdump/kexec.
+
+	  For more details see Documentation/kdump/kdump.txt
+
 config XEN_DOM0
 	def_bool y
 	depends on XEN
diff --git a/arch/arm64/kernel/Makefile b/arch/arm64/kernel/Makefile
index 7700c0c..c13d3eb 100644
--- a/arch/arm64/kernel/Makefile
+++ b/arch/arm64/kernel/Makefile
@@ -48,6 +48,7 @@ arm64-obj-$(CONFIG_RANDOMIZE_BASE)	+= kaslr.o
 arm64-obj-$(CONFIG_HIBERNATION)		+= hibernate.o hibernate-asm.o
 arm64-obj-$(CONFIG_KEXEC)		+= machine_kexec.o relocate_kernel.o	\
 					   cpu-reset.o
+arm64-obj-$(CONFIG_CRASH_DUMP)		+= crash_dump.o
 
 obj-y					+= $(arm64-obj-y) vdso/
 obj-m					+= $(arm64-obj-m)
diff --git a/arch/arm64/kernel/crash_dump.c b/arch/arm64/kernel/crash_dump.c
new file mode 100644
index 0000000..2dc54d1
--- /dev/null
+++ b/arch/arm64/kernel/crash_dump.c
@@ -0,0 +1,71 @@
+/*
+ * Routines for doing kexec-based kdump
+ *
+ * Copyright (C) 2014 Linaro Limited
+ * Author: AKASHI Takahiro <takahiro.akashi@linaro.org>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#include <linux/crash_dump.h>
+#include <linux/errno.h>
+#include <linux/io.h>
+#include <linux/memblock.h>
+#include <linux/uaccess.h>
+#include <asm/memory.h>
+
+/**
+ * copy_oldmem_page() - copy one page from old kernel memory
+ * @pfn: page frame number to be copied
+ * @buf: buffer where the copied page is placed
+ * @csize: number of bytes to copy
+ * @offset: offset in bytes into the page
+ * @userbuf: if set, @buf is in a user address space
+ *
+ * This function copies one page from old kernel memory into buffer pointed by
+ * @buf. If @buf is in userspace, set @userbuf to %1. Returns number of bytes
+ * copied or negative error in case of failure.
+ */
+ssize_t copy_oldmem_page(unsigned long pfn, char *buf,
+			 size_t csize, unsigned long offset,
+			 int userbuf)
+{
+	void *vaddr;
+
+	if (!csize)
+		return 0;
+
+	vaddr = ioremap_cache(__pfn_to_phys(pfn), PAGE_SIZE);
+	if (!vaddr)
+		return -ENOMEM;
+
+	if (userbuf) {
+		if (copy_to_user(buf, vaddr + offset, csize)) {
+			iounmap(vaddr);
+			return -EFAULT;
+		}
+	} else {
+		memcpy(buf, vaddr + offset, csize);
+	}
+
+	iounmap(vaddr);
+
+	return csize;
+}
+
+/**
+ * elfcorehdr_read - read from ELF core header
+ * @buf: buffer where the data is placed
+ * @csize: number of bytes to read
+ * @ppos: address in the memory
+ *
+ * This function reads @count bytes from elf core header which exists
+ * on crash dump kernel's memory.
+ */
+ssize_t elfcorehdr_read(char *buf, size_t count, u64 *ppos)
+{
+	memcpy(buf, phys_to_virt((phys_addr_t)*ppos), count);
+	return count;
+}
diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
index 88fd99a..7e5844d 100644
--- a/arch/arm64/mm/init.c
+++ b/arch/arm64/mm/init.c
@@ -35,6 +35,7 @@
 #include <linux/efi.h>
 #include <linux/swiotlb.h>
 #include <linux/kexec.h>
+#include <linux/crash_dump.h>
 
 #include <asm/boot.h>
 #include <asm/fixmap.h>
@@ -136,6 +137,57 @@ static void __init reserve_crashkernel(void)
 }
 #endif /* CONFIG_KEXEC_CORE */
 
+#ifdef CONFIG_CRASH_DUMP
+static int __init early_init_dt_scan_elfcorehdr(unsigned long node,
+		const char *uname, int depth, void *data)
+{
+	const __be32 *reg;
+	int len;
+
+	if (depth != 1 || strcmp(uname, "chosen") != 0)
+		return 0;
+
+	reg = of_get_flat_dt_prop(node, "linux,elfcorehdr", &len);
+	if (!reg || (len < (dt_root_addr_cells + dt_root_size_cells)))
+		return 1;
+
+	elfcorehdr_addr = dt_mem_next_cell(dt_root_addr_cells, &reg);
+	elfcorehdr_size = dt_mem_next_cell(dt_root_size_cells, &reg);
+
+	return 1;
+}
+
+/*
+ * reserve_elfcorehdr() - reserves memory for elf core header
+ *
+ * This function reserves elf core header given in "elfcorehdr=" kernel
+ * command line parameter. This region contains all the information about
+ * primary kernel's core image and is used by a dump capture kernel to
+ * access the system memory on primary kernel.
+ */
+static void __init reserve_elfcorehdr(void)
+{
+	of_scan_flat_dt(early_init_dt_scan_elfcorehdr, NULL);
+
+	if (!elfcorehdr_size)
+		return;
+
+	if (memblock_is_region_reserved(elfcorehdr_addr, elfcorehdr_size)) {
+		pr_warn("elfcorehdr is overlapped\n");
+		return;
+	}
+
+	memblock_reserve(elfcorehdr_addr, elfcorehdr_size);
+
+	pr_info("Reserving %lldKB of memory@0x%llx for elfcorehdr\n",
+		elfcorehdr_size >> 10, elfcorehdr_addr);
+}
+#else
+static void __init reserve_elfcorehdr(void)
+{
+	;
+}
+#endif /* CONFIG_CRASH_DUMP */
 /*
  * Return the maximum physical address for ZONE_DMA (DMA_BIT_MASK(32)). It
  * currently assumes that for memory starting above 4G, 32-bit devices will
@@ -389,6 +441,8 @@ void __init arm64_memblock_init(void)
 
 	reserve_crashkernel();
 
+	reserve_elfcorehdr();
+
 	early_init_fdt_scan_reserved_mem();
 
 	/* 4GB maximum for 32-bit only capable devices */
-- 
2.9.0

[PATCH v21 4/8] arm64: kdump: add kdump support

[Geoff Levand]

On Wed, 2016-07-06 at 16:52 +0900, AKASHI Takahiro wrote:
> --- a/arch/arm64/mm/init.c
> +++ b/arch/arm64/mm/init.c
> @@ -136,6 +137,57 @@ static void __init reserve_crashkernel(void)
>  }
>  #endif /* CONFIG_KEXEC_CORE */
>  
> +#ifdef CONFIG_CRASH_DUMP
> +static int __init early_init_dt_scan_elfcorehdr(unsigned long node,
> +> 	> 	> const char *uname, int depth, void *data)
> +{
> +> 	> const __be32 *reg;
> +> 	> int len;
> +
> +> 	> if (depth != 1 || strcmp(uname, "chosen") != 0)
> +> 	> 	> return 0;
> +
> +> 	> reg = of_get_flat_dt_prop(node, "linux,elfcorehdr", &len);
> +> 	> if (!reg || (len < (dt_root_addr_cells + dt_root_size_cells)))
> +> 	> 	> return 1;
> +
> +> 	> elfcorehdr_addr = dt_mem_next_cell(dt_root_addr_cells, ?);
> +> 	> elfcorehdr_size = dt_mem_next_cell(dt_root_size_cells, ?);
> +
> +> 	> return 1;
> +}
> +
> +/*
> + * reserve_elfcorehdr() - reserves memory for elf core header
> + *
> + * This function reserves elf core header given in "elfcorehdr=" kernel
> + * command line parameter. This region contains all the information about
> + * primary kernel's core image and is used by a dump capture kernel to
> + * access the system memory on primary kernel.
> + */
> +static void __init reserve_elfcorehdr(void)
> +{

It seems like you could get rid of all these conditionals by
just putting this here:

	if (!IS_ENABLED(CONFIG_CRASH_DUMP))
		return;


> +> 	> of_scan_flat_dt(early_init_dt_scan_elfcorehdr, NULL);
> +
> +> 	> if (!elfcorehdr_size)
> +> 	> 	> return;
> +
> +> 	> if (memblock_is_region_reserved(elfcorehdr_addr, elfcorehdr_size)) {
> +> 	> 	> pr_warn("elfcorehdr is overlapped\n");
> +> 	> 	> return;
> +> 	> }
> +
> +> 	> memblock_reserve(elfcorehdr_addr, elfcorehdr_size);
> +
> +> 	> pr_info("Reserving %lldKB of memory at 0x%llx for elfcorehdr\n",
> +> 	> 	> elfcorehdr_size >> 10, elfcorehdr_addr);
> +}
> +#else
> +static void __init reserve_elfcorehdr(void)
> +{
> +> 	> ;
> +}
> +#endif /* CONFIG_CRASH_DUMP */
>  /*
>   * Return the maximum physical address for ZONE_DMA (DMA_BIT_MASK(32)). It
>   * currently assumes that for memory starting above 4G, 32-bit devices will
> @@ -389,6 +441,8 @@ void __init arm64_memblock_init(void)
>  
>  > 	> reserve_crashkernel();
>  
> +> 	> reserve_elfcorehdr();
> +
>  > 	> early_init_fdt_scan_reserved_mem();
>  
>  > 	> /* 4GB maximum for 32-bit only capable devices */

[PATCH v21 4/8] arm64: kdump: add kdump support

[AKASHI Takahiro]

On Wed, Jul 06, 2016 at 10:38:09AM -0700, Geoff Levand wrote:
> On Wed, 2016-07-06 at 16:52 +0900, AKASHI Takahiro wrote:
> > --- a/arch/arm64/mm/init.c
> > +++ b/arch/arm64/mm/init.c
> > @@ -136,6 +137,57 @@ static void __init reserve_crashkernel(void)
> >  }
> >  #endif /* CONFIG_KEXEC_CORE */
> >  
> > +#ifdef CONFIG_CRASH_DUMP
> > +static int __init early_init_dt_scan_elfcorehdr(unsigned long node,
> > +> 	> 	> const char *uname, int depth, void *data)
> > +{
> > +> 	> const __be32 *reg;
> > +> 	> int len;
> > +
> > +> 	> if (depth != 1 || strcmp(uname, "chosen") != 0)
> > +> 	> 	> return 0;
> > +
> > +> 	> reg = of_get_flat_dt_prop(node, "linux,elfcorehdr", &len);
> > +> 	> if (!reg || (len < (dt_root_addr_cells + dt_root_size_cells)))
> > +> 	> 	> return 1;
> > +
> > +> 	> elfcorehdr_addr = dt_mem_next_cell(dt_root_addr_cells, ??);
> > +> 	> elfcorehdr_size = dt_mem_next_cell(dt_root_size_cells, ??);
> > +
> > +> 	> return 1;
> > +}
> > +
> > +/*
> > + * reserve_elfcorehdr() - reserves memory for elf core header
> > + *
> > + * This function reserves elf core header given in "elfcorehdr=" kernel
> > + * command line parameter. This region contains all the information about
> > + * primary kernel's core image and is used by a dump capture kernel to
> > + * access the system memory on primary kernel.
> > + */
> > +static void __init reserve_elfcorehdr(void)
> > +{
> 
> It seems like you could get rid of all these conditionals by
> just putting this here:
> 
> 	if (!IS_ENABLED(CONFIG_CRASH_DUMP))
> 		return;

Add this to reserve_elfcorehdr()?
Unfortunately, it won't work because this function accesses
elfcorehdr_addr & elfcorehdr_size which are defined only
if CONFIG_CRASH_DUMP.

Thanks,
-Takahiro AKASHI

> 
> > +> 	> of_scan_flat_dt(early_init_dt_scan_elfcorehdr, NULL);
> > +
> > +> 	> if (!elfcorehdr_size)
> > +> 	> 	> return;
> > +
> > +> 	> if (memblock_is_region_reserved(elfcorehdr_addr, elfcorehdr_size)) {
> > +> 	> 	> pr_warn("elfcorehdr is overlapped\n");
> > +> 	> 	> return;
> > +> 	> }
> > +
> > +> 	> memblock_reserve(elfcorehdr_addr, elfcorehdr_size);
> > +
> > +> 	> pr_info("Reserving %lldKB of memory at 0x%llx for elfcorehdr\n",
> > +> 	> 	> elfcorehdr_size >> 10, elfcorehdr_addr);
> > +}
> > +#else
> > +static void __init reserve_elfcorehdr(void)
> > +{
> > +> 	> ;
> > +}
> > +#endif /* CONFIG_CRASH_DUMP */
> >  /*
> >   * Return the maximum physical address for ZONE_DMA (DMA_BIT_MASK(32)). It
> >   * currently assumes that for memory starting above 4G, 32-bit devices will
> > @@ -389,6 +441,8 @@ void __init arm64_memblock_init(void)
> >  
> >  > 	> reserve_crashkernel();
> >  
> > +> 	> reserve_elfcorehdr();
> > +
> >  > 	> early_init_fdt_scan_reserved_mem();
> >  
> >  > 	> /* 4GB maximum for 32-bit only capable devices */
> 

[PATCH v21 5/8] arm64: kdump: add VMCOREINFO's for user-space coredump tools

[AKASHI Takahiro]

For the current crash utility, we need to know, at least,
  - kimage_voffset
  - PHYS_OFFSET
to handle the contents of core dump file (/proc/vmcore) correctly due to
the introduction of KASLR (CONFIG_RANDOMIZE_BASE) in v4.6.
This patch puts them as VMCOREINFO's into the file.

  - VA_BITS
is also added for makedumpfile command.
More VMCOREINFO's may be added later.

Signed-off-by: AKASHI Takahiro <takahiro.akashi@linaro.org>
---
 arch/arm64/kernel/machine_kexec.c | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c
index 8ac9dba8..38b4411 100644
--- a/arch/arm64/kernel/machine_kexec.c
+++ b/arch/arm64/kernel/machine_kexec.c
@@ -17,6 +17,7 @@
 
 #include <asm/cacheflush.h>
 #include <asm/cpu_ops.h>
+#include <asm/memory.h>
 #include <asm/mmu_context.h>
 
 #include "cpu-reset.h"
@@ -260,3 +261,13 @@ void machine_crash_shutdown(struct pt_regs *regs)
 
 	pr_info("Starting crashdump kernel...\n");
 }
+
+void arch_crash_save_vmcoreinfo(void)
+{
+	VMCOREINFO_NUMBER(VA_BITS);
+	/* Please note VMCOREINFO_NUMBER() uses "%d", not "%x" */
+	vmcoreinfo_append_str("NUMBER(kimage_voffset)=0x%llx\n",
+						kimage_voffset);
+	vmcoreinfo_append_str("NUMBER(PHYS_OFFSET)=0x%llx\n",
+						PHYS_OFFSET);
+}
-- 
2.9.0

[PATCH v21 6/8] arm64: kdump: enable kdump in the arm64 defconfig

[AKASHI Takahiro]

Signed-off-by: AKASHI Takahiro <takahiro.akashi@linaro.org>
---
 arch/arm64/configs/defconfig | 1 +
 1 file changed, 1 insertion(+)

diff --git a/arch/arm64/configs/defconfig b/arch/arm64/configs/defconfig
index 4ed4756..aa3e1dd 100644
--- a/arch/arm64/configs/defconfig
+++ b/arch/arm64/configs/defconfig
@@ -71,6 +71,7 @@ CONFIG_TRANSPARENT_HUGEPAGE=y
 CONFIG_CMA=y
 CONFIG_XEN=y
 CONFIG_KEXEC=y
+CONFIG_CRASH_DUMP=y
 # CONFIG_CORE_DUMP_DEFAULT_ELF_HEADERS is not set
 CONFIG_COMPAT=y
 CONFIG_CPU_IDLE=y
-- 
2.9.0

[PATCH v21 7/8] arm64: kdump: update a kernel doc

[AKASHI Takahiro]

This patch adds arch specific descriptions about kdump usage on arm64
to kdump.txt.

Signed-off-by: AKASHI Takahiro <takahiro.akashi@linaro.org>
Reviewed-by: Baoquan He <bhe@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
---
 Documentation/kdump/kdump.txt | 16 +++++++++++++++-
 1 file changed, 15 insertions(+), 1 deletion(-)

diff --git a/Documentation/kdump/kdump.txt b/Documentation/kdump/kdump.txt
index 88ff63d..8bf91fd 100644
--- a/Documentation/kdump/kdump.txt
+++ b/Documentation/kdump/kdump.txt
@@ -18,7 +18,7 @@ memory image to a dump file on the local disk, or across the network to
 a remote system.
 
 Kdump and kexec are currently supported on the x86, x86_64, ppc64, ia64,
-s390x and arm architectures.
+s390x, arm and arm64 architectures.
 
 When the system kernel boots, it reserves a small section of memory for
 the dump-capture kernel. This ensures that ongoing Direct Memory Access
@@ -249,6 +249,13 @@ Dump-capture kernel config options (Arch Dependent, arm)
 
     AUTO_ZRELADDR=y
 
+Dump-capture kernel config options (Arch Dependent, arm64)
+----------------------------------------------------------
+
+- Please note that kvm at the dump-capture kernel will not be enabled
+  on non-VHE system even if it is configured. This is because we have
+  no chance to reset CPU state to EL2 on panic.
+
 Extended crashkernel syntax
 ===========================
 
@@ -305,6 +312,8 @@ Boot into System Kernel
    kernel will automatically locate the crash kernel image within the
    first 512MB of RAM if X is not given.
 
+   On arm64, use "crashkernel=Y[@X]".  Note that the start address of
+   the kernel, X if explicitly specified, must be aligned to 2MiB (0x200000).
 
 Load the Dump-capture Kernel
 ============================
@@ -327,6 +336,8 @@ For s390x:
 	- Use image or bzImage
 For arm:
 	- Use zImage
+For arm64:
+	- Use vmlinux or Image
 
 If you are using a uncompressed vmlinux image then use following command
 to load dump-capture kernel.
@@ -370,6 +381,9 @@ For s390x:
 For arm:
 	"1 maxcpus=1 reset_devices"
 
+For arm64:
+	"1 maxcpus=1 reset_devices"
+
 Notes on loading the dump-capture kernel:
 
 * By default, the ELF headers are stored in ELF64 format to support
-- 
2.9.0

[PATCH v21 7/8] arm64: kdump: update a kernel doc

[Geoff Levand]

On Wed, 2016-07-06 at 16:52 +0900, AKASHI Takahiro wrote:
> --- a/Documentation/kdump/kdump.txt
> +++ b/Documentation/kdump/kdump.txt
> @@ -18,7 +18,7 @@ memory image to a dump file on the local disk, or across the network to
>  a remote system.
>  
>  Kdump and kexec are currently supported on the x86, x86_64, ppc64, ia64,
> -s390x and arm architectures.
> +s390x, arm and arm64 architectures.
>  
>  When the system kernel boots, it reserves a small section of memory for
>  the dump-capture kernel. This ensures that ongoing Direct Memory Access
> @@ -249,6 +249,13 @@ Dump-capture kernel config options (Arch Dependent, arm)
>  
>      AUTO_ZRELADDR=y
>  
> +Dump-capture kernel config options (Arch Dependent, arm64)
> +----------------------------------------------------------
> +
> +- Please note that kvm at the dump-capture kernel will not be enabled

kvm of the dump-capture kernel

> +  on non-VHE system even if it is configured. This is because we have

on non-VHE systems

> +  no chance to reset CPU state to EL2 on panic.

This is because the CPU state cannot be reset to EL2 on panic.

[PATCH v21 7/8] arm64: kdump: update a kernel doc

[AKASHI Takahiro]

On Wed, Jul 06, 2016 at 10:44:27AM -0700, Geoff Levand wrote:
> On Wed, 2016-07-06 at 16:52 +0900, AKASHI Takahiro wrote:
> > --- a/Documentation/kdump/kdump.txt
> > +++ b/Documentation/kdump/kdump.txt
> > @@ -18,7 +18,7 @@ memory image to a dump file on the local disk, or across the network to
> >  a remote system.
> >  
> >  Kdump and kexec are currently supported on the x86, x86_64, ppc64, ia64,
> > -s390x and arm architectures.
> > +s390x, arm and arm64 architectures.
> >  
> >  When the system kernel boots, it reserves a small section of memory for
> >  the dump-capture kernel. This ensures that ongoing Direct Memory Access
> > @@ -249,6 +249,13 @@ Dump-capture kernel config options (Arch Dependent, arm)
> >  
> >      AUTO_ZRELADDR=y
> >  
> > +Dump-capture kernel config options (Arch Dependent, arm64)
> > +----------------------------------------------------------
> > +
> > +- Please note that kvm at the dump-capture kernel will not be enabled
> 
> kvm of the dump-capture kernel
> 
> > +  on non-VHE system even if it is configured. This is because we have
> 
> on non-VHE systems
> 
> > +  no chance to reset CPU state to EL2 on panic.
> 
> This is because the CPU state cannot be reset to EL2 on panic.

Thanks :)
-Takahiro AKASHI

> 
> 

[PATCH v21 8/8] Documentation: dt: usable-memory and elfcorehdr nodes for arm64 kexec

[AKASHI Takahiro]

From: James Morse <james.morse@arm.com>

Add documentation for linux,usable-memory and linux,elfcorehdr chosen
nodes used by arm64 kexec to decribe the kdump reserved area, and the
elfcorehdr's location within it.

Signed-off-by: James Morse <james.morse@arm.com>
[akashi: wrap the commit description to avoid a checkpatch warning]
Signed-off-by: AKASHI Takahiro <takahiro.akashi@linaro.org>
---
 Documentation/devicetree/bindings/chosen.txt | 28 ++++++++++++++++++++++++++++
 1 file changed, 28 insertions(+)

diff --git a/Documentation/devicetree/bindings/chosen.txt b/Documentation/devicetree/bindings/chosen.txt
index 6ae9d82..443f88b 100644
--- a/Documentation/devicetree/bindings/chosen.txt
+++ b/Documentation/devicetree/bindings/chosen.txt
@@ -52,3 +52,31 @@ This property is set (currently only on PowerPC, and only needed on
 book3e) by some versions of kexec-tools to tell the new kernel that it
 is being booted by kexec, as the booting environment may differ (e.g.
 a different secondary CPU release mechanism)
+
+linux,usable-memory
+-------------------
+
+This property is set on PowerPC and arm64 by kexec-tools during kdump
+to tell the crash kernel the base address of its reserved area of memory, and
+the size. e.g.
+
+/ {
+	chosen {
+		linux,usable-memory = <0x9 0xf0000000 0x0 0x10000000>;
+	};
+};
+
+linux,elfcorehdr
+----------------
+
+This property is set (currently only on arm64) by kexec-tools during kdump
+to tell the crash kernel the address and size of the elfcorehdr that describes
+the old kernel's memory as an elf file. This memory must reside within the area
+described by 'linux,usable-memory'. e.g.
+
+/ {
+	chosen {
+		linux,usable-memory = <0x9 0xf0000000 0x0 0x10000000>;
+		linux,elfcorehdr = <0x9 0xfffff000 0x0 0x800>;
+	};
+};
-- 
2.9.0

[PATCH v21 8/8] Documentation: dt: usable-memory and elfcorehdr nodes for arm64 kexec

[Geoff Levand]

On Wed, 2016-07-06 at 16:52 +0900, AKASHI Takahiro wrote:
> From: James Morse <james.morse@arm.com>
> 
> Add documentation for linux,usable-memory and linux,elfcorehdr chosen
> nodes used by arm64 kexec to decribe the kdump reserved area, and the
> elfcorehdr's location within it.
> 
> Signed-off-by: James Morse <james.morse@arm.com>
> [akashi: wrap the commit description to avoid a checkpatch warning]
> Signed-off-by: AKASHI Takahiro <takahiro.akashi@linaro.org>

Look OK.

Reviewed-by: Geoff Levand <geoff@infradead.org>

[PATCH v21 8/8] Documentation: dt: usable-memory and elfcorehdr nodes for arm64 kexec

[Thiago Jung Bauermann]

Hi,

Am Mittwoch, 06 Juli 2016, 16:52:26 schrieb AKASHI Takahiro:
> +linux,usable-memory
> +-------------------
> +
> +This property is set on PowerPC and arm64 by kexec-tools during kdump
> +to tell the crash kernel the base address of its reserved area of memory,
> and +the size. e.g.
> +
> +/ {
> +	chosen {
> +		linux,usable-memory = <0x9 0xf0000000 0x0 0x10000000>;
> +	};
> +};

Again, this description is wrong for PowerPC. See messages from myself and 
Michael Ellerman:

https://lists.infradead.org/pipermail/kexec/2016-June/016250.html

https://lists.infradead.org/pipermail/kexec/2016-June/016253.html

IMHO, it would be simpler if ARM used linux,usable-memory in the same way 
that PowerPC does, for consistency.

-- 
[]'s
Thiago Jung Bauermann
IBM Linux Technology Center

[PATCH v21 8/8] Documentation: dt: usable-memory and elfcorehdr nodes for arm64 kexec

[AKASHI Takahiro]

On Wed, Jul 06, 2016 at 04:29:18PM -0300, Thiago Jung Bauermann wrote:
> Hi,
> 
> Am Mittwoch, 06 Juli 2016, 16:52:26 schrieb AKASHI Takahiro:
> > +linux,usable-memory
> > +-------------------
> > +
> > +This property is set on PowerPC and arm64 by kexec-tools during kdump
> > +to tell the crash kernel the base address of its reserved area of memory,
> > and +the size. e.g.
> > +
> > +/ {
> > +	chosen {
> > +		linux,usable-memory = <0x9 0xf0000000 0x0 0x10000000>;
> > +	};
> > +};
> 
> Again, this description is wrong for PowerPC. See messages from myself and 
> Michael Ellerman:
> 
> https://lists.infradead.org/pipermail/kexec/2016-June/016250.html
> 
> https://lists.infradead.org/pipermail/kexec/2016-June/016253.html

Oops, I must have missed your previous comments. Apologies.

Yes, I know that, and I used to implement the same functionality before.
It did work for dtb-based systems, but not for UEFI(ACPI)-based systems
because UEFI doesn't export memory regions information via a device tree,
but rather via ACPI table. So "/memory" node won't appear.
So I went back with "mem=" command line approach, and later this "/chosen/"
approach.

> IMHO, it would be simpler if ARM used linux,usable-memory in the same way 
> that PowerPC does, for consistency.

Well, this property won't conflict with per-"/memory" ones
if we take it that the former, if present, supersedes the latter.
Sophistic?
What about changing the name to usable-memory-limit?
(I know that you have another one, "memory-limit" though.)

Again, I would like to defer to arm64 maintainers.

Thanks,
-Takahiro AKASHI

> -- 
> []'s
> Thiago Jung Bauermann
> IBM Linux Technology Center
> 

[PATCH v21 8/8] Documentation: dt: usable-memory and elfcorehdr nodes for arm64 kexec

[Thiago Jung Bauermann]

Am Donnerstag, 07 Juli 2016, 11:00:25 schrieb AKASHI Takahiro:
> On Wed, Jul 06, 2016 at 04:29:18PM -0300, Thiago Jung Bauermann wrote:
> > Hi,
> > 
> > Am Mittwoch, 06 Juli 2016, 16:52:26 schrieb AKASHI Takahiro:
> > > +linux,usable-memory
> > > +-------------------
> > > +
> > > +This property is set on PowerPC and arm64 by kexec-tools during kdump
> > > +to tell the crash kernel the base address of its reserved area of
> > > memory, and +the size. e.g.
> > > +
> > > +/ {
> > > +	chosen {
> > > +		linux,usable-memory = <0x9 0xf0000000 0x0 0x10000000>;
> > > +	};
> > > +};
> > 
> > Again, this description is wrong for PowerPC. See messages from myself
> > and Michael Ellerman:
> > 
> > https://lists.infradead.org/pipermail/kexec/2016-June/016250.html
> > 
> > https://lists.infradead.org/pipermail/kexec/2016-June/016253.html
> 
> Oops, I must have missed your previous comments. Apologies.

No problem.

> Yes, I know that, and I used to implement the same functionality before.
> It did work for dtb-based systems, but not for UEFI(ACPI)-based systems
> because UEFI doesn't export memory regions information via a device tree,
> but rather via ACPI table. So "/memory" node won't appear.
> So I went back with "mem=" command line approach, and later this
> "/chosen/" approach.

Ah, I didn't realize there could be dtb and UEFI systems.

> > IMHO, it would be simpler if ARM used linux,usable-memory in the same
> > way
> > that PowerPC does, for consistency.
> 
> Well, this property won't conflict with per-"/memory" ones
> if we take it that the former, if present, supersedes the latter.
> Sophistic?
> What about changing the name to usable-memory-limit?
> (I know that you have another one, "memory-limit" though.)
> 
> Again, I would like to defer to arm64 maintainers.

My personal opinion is that having a property with a different name would be 
less confusing, but it's not a strong opinion.

I would suggest calling it usable-memory-range, but I'm fine with whatever 
is decided by the maintainers.

-- 
[]'s
Thiago Jung Bauermann
IBM Linux Technology Center