* Re: [PATCH v17 6/6] powerpc/crash: add crash memory hotplug support
@ 2024-03-02 13:19 Hari Bathini
2024-03-04 8:28 ` Sourabh Jain
0 siblings, 1 reply; 3+ messages in thread
From: Hari Bathini @ 2024-03-02 13:19 UTC (permalink / raw)
To: Sourabh Jain, linuxppc-dev
Cc: David Hildenbrand, Dave Hansen, Mimi Zohar, Boris Ostrovsky,
Valentin Schneider, Baoquan He, x86, Aneesh Kumar K . V,
Laurent Dufour, Dave Young, Vivek Goyal, Naveen N Rao,
Borislav Petkov, Thomas Gleixner, Oscar Salvador,
Greg Kroah-Hartman, kexec, Mahesh Salgaonkar, Akhil Raj,
Andrew Morton
On 26/02/24 2:11 pm, Sourabh Jain wrote:
> Extend the arch crash hotplug handler, as introduced by the patch title
> ("powerpc: add crash CPU hotplug support"), to also support memory
> add/remove events.
>
> Elfcorehdr describes the memory of the crash kernel to capture the
> kernel; hence, it needs to be updated if memory resources change due to
> memory add/remove events. Therefore, arch_crash_handle_hotplug_event()
> is updated to recreate the elfcorehdr and replace it with the previous
> one on memory add/remove events.
>
> The memblock list is used to prepare the elfcorehdr. In the case of
> memory hot remove, the memblock list is updated after the arch crash
> hotplug handler is triggered, as depicted in Figure 1. Thus, the
> hot-removed memory is explicitly removed from the crash memory ranges
> to ensure that the memory ranges added to elfcorehdr do not include the
> hot-removed memory.
>
> Memory remove
> |
> v
> Offline pages
> |
> v
> Initiate memory notify call <----> crash hotplug handler
> chain for MEM_OFFLINE event
> |
> v
> Update memblock list
>
> Figure 1
>
> There are two system calls, `kexec_file_load` and `kexec_load`, used to
> load the kdump image. A few changes have been made to ensure that the
> kernel can safely update the elfcorehdr component of the kdump image for
> both system calls.
>
> For the kexec_file_load syscall, kdump image is prepared in the kernel.
> To support an increasing number of memory regions, the elfcorehdr is
> built with extra buffer space to ensure that it can accommodate
> additional memory ranges in future.
>
> For the kexec_load syscall, the elfcorehdr is updated only if the
> KEXEC_CRASH_HOTPLUG_SUPPORT kexec flag is passed to the kernel by the
> kexec tool. Passing this flag to the kernel indicates that the
> elfcorehdr is built to accommodate additional memory ranges and the
> elfcorehdr segment is not considered for SHA calculation, making it safe
> to update.
>
> The changes related to this feature are kept under the CRASH_HOTPLUG
> config, and it is enabled by default.
>
Overall, the patchset looks good. I tried out the changes too.
Acked-by: Hari Bathini <hbathini@linux.ibm.com>
> Signed-off-by: Sourabh Jain <sourabhjain@linux.ibm.com>
> Cc: Akhil Raj <lf32.dev@gmail.com>
> Cc: Andrew Morton <akpm@linux-foundation.org>
> Cc: Aneesh Kumar K.V <aneesh.kumar@kernel.org>
> Cc: Baoquan He <bhe@redhat.com>
> Cc: Borislav Petkov (AMD) <bp@alien8.de>
> Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com>
> Cc: Christophe Leroy <christophe.leroy@csgroup.eu>
> Cc: Dave Hansen <dave.hansen@linux.intel.com>
> Cc: Dave Young <dyoung@redhat.com>
> Cc: David Hildenbrand <david@redhat.com>
> Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> Cc: Hari Bathini <hbathini@linux.ibm.com>
> Cc: Laurent Dufour <laurent.dufour@fr.ibm.com>
> Cc: Mahesh Salgaonkar <mahesh@linux.ibm.com>
> Cc: Michael Ellerman <mpe@ellerman.id.au>
> Cc: Mimi Zohar <zohar@linux.ibm.com>
> Cc: Naveen N Rao <naveen@kernel.org>
> Cc: Oscar Salvador <osalvador@suse.de>
> Cc: Thomas Gleixner <tglx@linutronix.de>
> Cc: Valentin Schneider <vschneid@redhat.com>
> Cc: Vivek Goyal <vgoyal@redhat.com>
> Cc: kexec@lists.infradead.org
> Cc: x86@kernel.org
> ---
> arch/powerpc/include/asm/kexec.h | 3 +
> arch/powerpc/include/asm/kexec_ranges.h | 1 +
> arch/powerpc/kexec/crash.c | 95 ++++++++++++++++++++++++-
> arch/powerpc/kexec/file_load_64.c | 20 +++++-
> arch/powerpc/kexec/ranges.c | 85 ++++++++++++++++++++++
> 5 files changed, 202 insertions(+), 2 deletions(-)
>
> diff --git a/arch/powerpc/include/asm/kexec.h b/arch/powerpc/include/asm/kexec.h
> index e75970351bcd..95a98b390d62 100644
> --- a/arch/powerpc/include/asm/kexec.h
> +++ b/arch/powerpc/include/asm/kexec.h
> @@ -141,6 +141,9 @@ void arch_crash_handle_hotplug_event(struct kimage *image, void *arg);
>
> int arch_crash_hotplug_support(struct kimage *image, unsigned long kexec_flags);
> #define arch_crash_hotplug_support arch_crash_hotplug_support
> +
> +unsigned int arch_crash_get_elfcorehdr_size(void);
> +#define crash_get_elfcorehdr_size arch_crash_get_elfcorehdr_size
> #endif /* CONFIG_CRASH_HOTPLUG */
>
> extern int crashing_cpu;
> diff --git a/arch/powerpc/include/asm/kexec_ranges.h b/arch/powerpc/include/asm/kexec_ranges.h
> index 8489e844b447..14055896cbcb 100644
> --- a/arch/powerpc/include/asm/kexec_ranges.h
> +++ b/arch/powerpc/include/asm/kexec_ranges.h
> @@ -7,6 +7,7 @@
> void sort_memory_ranges(struct crash_mem *mrngs, bool merge);
> struct crash_mem *realloc_mem_ranges(struct crash_mem **mem_ranges);
> int add_mem_range(struct crash_mem **mem_ranges, u64 base, u64 size);
> +int remove_mem_range(struct crash_mem **mem_ranges, u64 base, u64 size);
> int get_exclude_memory_ranges(struct crash_mem **mem_ranges);
> int get_reserved_memory_ranges(struct crash_mem **mem_ranges);
> int get_crash_memory_ranges(struct crash_mem **mem_ranges);
> diff --git a/arch/powerpc/kexec/crash.c b/arch/powerpc/kexec/crash.c
> index 8938a19af12f..21b193e938a3 100644
> --- a/arch/powerpc/kexec/crash.c
> +++ b/arch/powerpc/kexec/crash.c
> @@ -17,6 +17,7 @@
> #include <linux/irq.h>
> #include <linux/types.h>
> #include <linux/libfdt.h>
> +#include <linux/memory.h>
>
> #include <asm/processor.h>
> #include <asm/machdep.h>
> @@ -25,6 +26,7 @@
> #include <asm/setjmp.h>
> #include <asm/debug.h>
> #include <asm/interrupt.h>
> +#include <asm/kexec_ranges.h>
>
> /*
> * The primary CPU waits a while for all secondary CPUs to enter. This is to
> @@ -398,6 +400,94 @@ void default_machine_crash_shutdown(struct pt_regs *regs)
> #undef pr_fmt
> #define pr_fmt(fmt) "crash hp: " fmt
>
> +/*
> + * Advertise preferred elfcorehdr size to userspace via
> + * /sys/kernel/crash_elfcorehdr_size sysfs interface.
> + */
> +unsigned int arch_crash_get_elfcorehdr_size(void)
> +{
> + unsigned long phdr_cnt;
> +
> + /* A program header for possible CPUs + vmcoreinfo */
> + phdr_cnt = num_possible_cpus() + 1;
> + if (IS_ENABLED(CONFIG_MEMORY_HOTPLUG))
> + phdr_cnt += CONFIG_CRASH_MAX_MEMORY_RANGES;
> +
> + return sizeof(struct elfhdr) + (phdr_cnt * sizeof(Elf64_Phdr));
> +}
> +
> +/**
> + * update_crash_elfcorehdr() - Recreate the elfcorehdr and replace it with old
> + * elfcorehdr in the kexec segment array.
> + * @image: the active struct kimage
> + * @mn: struct memory_notify data handler
> + */
> +static void update_crash_elfcorehdr(struct kimage *image, struct memory_notify *mn)
> +{
> + int ret;
> + struct crash_mem *cmem = NULL;
> + struct kexec_segment *ksegment;
> + void *ptr, *mem, *elfbuf = NULL;
> + unsigned long elfsz, memsz, base_addr, size;
> +
> + ksegment = &image->segment[image->elfcorehdr_index];
> + mem = (void *) ksegment->mem;
> + memsz = ksegment->memsz;
> +
> + ret = get_crash_memory_ranges(&cmem);
> + if (ret) {
> + pr_err("Failed to get crash mem range\n");
> + return;
> + }
> +
> + /*
> + * The hot unplugged memory is part of crash memory ranges,
> + * remove it here.
> + */
> + if (image->hp_action == KEXEC_CRASH_HP_REMOVE_MEMORY) {
> + base_addr = PFN_PHYS(mn->start_pfn);
> + size = mn->nr_pages * PAGE_SIZE;
> + ret = remove_mem_range(&cmem, base_addr, size);
> + if (ret) {
> + pr_err("Failed to remove hot-unplugged memory from crash memory ranges\n");
> + goto out;
> + }
> + }
> +
> + ret = crash_prepare_elf64_headers(cmem, false, &elfbuf, &elfsz);
> + if (ret) {
> + pr_err("Failed to prepare elf header\n");
> + goto out;
> + }
> +
> + /*
> + * It is unlikely that kernel hit this because elfcorehdr kexec
> + * segment (memsz) is built with addition space to accommodate growing
> + * number of crash memory ranges while loading the kdump kernel. It is
> + * Just to avoid any unforeseen case.
> + */
> + if (elfsz > memsz) {
> + pr_err("Updated crash elfcorehdr elfsz %lu > memsz %lu", elfsz, memsz);
> + goto out;
> + }
> +
> + ptr = __va(mem);
> + if (ptr) {
> + /* Temporarily invalidate the crash image while it is replaced */
> + xchg(&kexec_crash_image, NULL);
> +
> + /* Replace the old elfcorehdr with newly prepared elfcorehdr */
> + memcpy((void *)ptr, elfbuf, elfsz);
> +
> + /* The crash image is now valid once again */
> + xchg(&kexec_crash_image, image);
> + }
> +out:
> + kvfree(cmem);
> + if (elfbuf)
> + kvfree(elfbuf);
> +}
> +
> /**
> * get_fdt_index - Loop through the kexec segment array and find
> * the index of the FDT segment.
> @@ -478,6 +568,8 @@ int arch_crash_hotplug_support(struct kimage *image, unsigned long kexec_flags)
> */
> void arch_crash_handle_hotplug_event(struct kimage *image, void *arg)
> {
> + struct memory_notify *mn;
> +
> switch (image->hp_action) {
> case KEXEC_CRASH_HP_REMOVE_CPU:
> return;
> @@ -488,7 +580,8 @@ void arch_crash_handle_hotplug_event(struct kimage *image, void *arg)
>
> case KEXEC_CRASH_HP_REMOVE_MEMORY:
> case KEXEC_CRASH_HP_ADD_MEMORY:
> - pr_info_once("Crash update is not supported for memory hotplug\n");
> + mn = (struct memory_notify *)arg;
> + update_crash_elfcorehdr(image, mn);
> return;
> default:
> pr_warn_once("Unknown hotplug action\n");
> diff --git a/arch/powerpc/kexec/file_load_64.c b/arch/powerpc/kexec/file_load_64.c
> index 2640a804fcdf..925a69ad2468 100644
> --- a/arch/powerpc/kexec/file_load_64.c
> +++ b/arch/powerpc/kexec/file_load_64.c
> @@ -595,6 +595,23 @@ static void update_backup_region_phdr(struct kimage *image, Elf64_Ehdr *ehdr)
> }
> }
>
> +static unsigned int kdump_extra_elfcorehdr_size(struct crash_mem *cmem)
> +{
> +#if defined(CONFIG_CRASH_HOTPLUG) && defined(CONFIG_MEMORY_HOTPLUG)
> + unsigned int extra_sz = 0;
> +
> + if (CONFIG_CRASH_MAX_MEMORY_RANGES > (unsigned int)PN_XNUM)
> + pr_warn("Number of Phdrs %u exceeds max\n", CONFIG_CRASH_MAX_MEMORY_RANGES);
> + else if (cmem->nr_ranges >= CONFIG_CRASH_MAX_MEMORY_RANGES)
> + pr_warn("Configured crash mem ranges may not be enough\n");
> + else
> + extra_sz = (CONFIG_CRASH_MAX_MEMORY_RANGES - cmem->nr_ranges) * sizeof(Elf64_Phdr);
> +
> + return extra_sz;
> +#endif
> + return 0;
> +}
> +
> /**
> * load_elfcorehdr_segment - Setup crash memory ranges and initialize elfcorehdr
> * segment needed to load kdump kernel.
> @@ -626,7 +643,8 @@ static int load_elfcorehdr_segment(struct kimage *image, struct kexec_buf *kbuf)
>
> kbuf->buffer = headers;
> kbuf->mem = KEXEC_BUF_MEM_UNKNOWN;
> - kbuf->bufsz = kbuf->memsz = headers_sz;
> + kbuf->bufsz = headers_sz;
> + kbuf->memsz = headers_sz + kdump_extra_elfcorehdr_size(cmem);
> kbuf->top_down = false;
>
> ret = kexec_add_buffer(kbuf);
> diff --git a/arch/powerpc/kexec/ranges.c b/arch/powerpc/kexec/ranges.c
> index 297b8bc97b9f..00dfd62203bd 100644
> --- a/arch/powerpc/kexec/ranges.c
> +++ b/arch/powerpc/kexec/ranges.c
> @@ -618,4 +618,89 @@ int get_crash_memory_ranges(struct crash_mem **mem_ranges)
> pr_err("Failed to setup crash memory ranges\n");
> return ret;
> }
> +
> +/**
> + * remove_mem_range - Removes the given memory range from the range list.
> + * @mem_ranges: Range list to remove the memory range to.
> + * @base: Base address of the range to remove.
> + * @size: Size of the memory range to remove.
> + *
> + * (Re)allocates memory, if needed.
> + *
> + * Returns 0 on success, negative errno on error.
> + */
> +int remove_mem_range(struct crash_mem **mem_ranges, u64 base, u64 size)
> +{
> + u64 end;
> + int ret = 0;
> + unsigned int i;
> + u64 mstart, mend;
> + struct crash_mem *mem_rngs = *mem_ranges;
> +
> + if (!size)
> + return 0;
> +
> + /*
> + * Memory range are stored as start and end address, use
> + * the same format to do remove operation.
> + */
> + end = base + size - 1;
> +
> + for (i = 0; i < mem_rngs->nr_ranges; i++) {
> + mstart = mem_rngs->ranges[i].start;
> + mend = mem_rngs->ranges[i].end;
> +
> + /*
> + * Memory range to remove is not part of this range entry
> + * in the memory range list
> + */
> + if (!(base >= mstart && end <= mend))
> + continue;
> +
> + /*
> + * Memory range to remove is equivalent to this entry in the
> + * memory range list. Remove the range entry from the list.
> + */
> + if (base == mstart && end == mend) {
> + for (; i < mem_rngs->nr_ranges - 1; i++) {
> + mem_rngs->ranges[i].start = mem_rngs->ranges[i+1].start;
> + mem_rngs->ranges[i].end = mem_rngs->ranges[i+1].end;
> + }
> + mem_rngs->nr_ranges--;
> + goto out;
> + }
> + /*
> + * Start address of the memory range to remove and the
> + * current memory range entry in the list is same. Just
> + * move the start address of the current memory range
> + * entry in the list to end + 1.
> + */
> + else if (base == mstart) {
> + mem_rngs->ranges[i].start = end + 1;
> + goto out;
> + }
> + /*
> + * End address of the memory range to remove and the
> + * current memory range entry in the list is same.
> + * Just move the end address of the current memory
> + * range entry in the list to base - 1.
> + */
> + else if (end == mend) {
> + mem_rngs->ranges[i].end = base - 1;
> + goto out;
> + }
> + /*
> + * Memory range to remove is not at the edge of current
> + * memory range entry. Split the current memory entry into
> + * two half.
> + */
> + else {
> + mem_rngs->ranges[i].end = base - 1;
> + size = mem_rngs->ranges[i].end - end;
> + ret = add_mem_range(mem_ranges, end + 1, size);
> + }
> + }
> +out:
> + return ret;
> +}
> #endif /* CONFIG_CRASH_DUMP */
_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
^ permalink raw reply [flat|nested] 3+ messages in thread* Re: [PATCH v17 6/6] powerpc/crash: add crash memory hotplug support
2024-03-02 13:19 [PATCH v17 6/6] powerpc/crash: add crash memory hotplug support Hari Bathini
@ 2024-03-04 8:28 ` Sourabh Jain
0 siblings, 0 replies; 3+ messages in thread
From: Sourabh Jain @ 2024-03-04 8:28 UTC (permalink / raw)
To: Hari Bathini, linuxppc-dev
Cc: David Hildenbrand, Dave Hansen, Mimi Zohar, Boris Ostrovsky,
Valentin Schneider, Baoquan He, x86, Aneesh Kumar K . V,
Laurent Dufour, Dave Young, Vivek Goyal, Naveen N Rao,
Borislav Petkov, Thomas Gleixner, Oscar Salvador,
Greg Kroah-Hartman, kexec, Mahesh Salgaonkar, Akhil Raj,
Andrew Morton
On 02/03/24 18:49, Hari Bathini wrote:
>
>
> On 26/02/24 2:11 pm, Sourabh Jain wrote:
>> Extend the arch crash hotplug handler, as introduced by the patch title
>> ("powerpc: add crash CPU hotplug support"), to also support memory
>> add/remove events.
>>
>> Elfcorehdr describes the memory of the crash kernel to capture the
>> kernel; hence, it needs to be updated if memory resources change due to
>> memory add/remove events. Therefore, arch_crash_handle_hotplug_event()
>> is updated to recreate the elfcorehdr and replace it with the previous
>> one on memory add/remove events.
>>
>> The memblock list is used to prepare the elfcorehdr. In the case of
>> memory hot remove, the memblock list is updated after the arch crash
>> hotplug handler is triggered, as depicted in Figure 1. Thus, the
>> hot-removed memory is explicitly removed from the crash memory ranges
>> to ensure that the memory ranges added to elfcorehdr do not include the
>> hot-removed memory.
>>
>> Memory remove
>> |
>> v
>> Offline pages
>> |
>> v
>> Initiate memory notify call <----> crash hotplug handler
>> chain for MEM_OFFLINE event
>> |
>> v
>> Update memblock list
>>
>> Figure 1
>>
>> There are two system calls, `kexec_file_load` and `kexec_load`, used to
>> load the kdump image. A few changes have been made to ensure that the
>> kernel can safely update the elfcorehdr component of the kdump image for
>> both system calls.
>>
>> For the kexec_file_load syscall, kdump image is prepared in the kernel.
>> To support an increasing number of memory regions, the elfcorehdr is
>> built with extra buffer space to ensure that it can accommodate
>> additional memory ranges in future.
>>
>> For the kexec_load syscall, the elfcorehdr is updated only if the
>> KEXEC_CRASH_HOTPLUG_SUPPORT kexec flag is passed to the kernel by the
>> kexec tool. Passing this flag to the kernel indicates that the
>> elfcorehdr is built to accommodate additional memory ranges and the
>> elfcorehdr segment is not considered for SHA calculation, making it safe
>> to update.
>>
>> The changes related to this feature are kept under the CRASH_HOTPLUG
>> config, and it is enabled by default.
>>
>
> Overall, the patchset looks good. I tried out the changes too.
>
> Acked-by: Hari Bathini <hbathini@linux.ibm.com>
Hello Hari,
Thanks for trying out the change.
- Sourabh
>
>> Signed-off-by: Sourabh Jain <sourabhjain@linux.ibm.com>
>> Cc: Akhil Raj <lf32.dev@gmail.com>
>> Cc: Andrew Morton <akpm@linux-foundation.org>
>> Cc: Aneesh Kumar K.V <aneesh.kumar@kernel.org>
>> Cc: Baoquan He <bhe@redhat.com>
>> Cc: Borislav Petkov (AMD) <bp@alien8.de>
>> Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com>
>> Cc: Christophe Leroy <christophe.leroy@csgroup.eu>
>> Cc: Dave Hansen <dave.hansen@linux.intel.com>
>> Cc: Dave Young <dyoung@redhat.com>
>> Cc: David Hildenbrand <david@redhat.com>
>> Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
>> Cc: Hari Bathini <hbathini@linux.ibm.com>
>> Cc: Laurent Dufour <laurent.dufour@fr.ibm.com>
>> Cc: Mahesh Salgaonkar <mahesh@linux.ibm.com>
>> Cc: Michael Ellerman <mpe@ellerman.id.au>
>> Cc: Mimi Zohar <zohar@linux.ibm.com>
>> Cc: Naveen N Rao <naveen@kernel.org>
>> Cc: Oscar Salvador <osalvador@suse.de>
>> Cc: Thomas Gleixner <tglx@linutronix.de>
>> Cc: Valentin Schneider <vschneid@redhat.com>
>> Cc: Vivek Goyal <vgoyal@redhat.com>
>> Cc: kexec@lists.infradead.org
>> Cc: x86@kernel.org
>> ---
>> arch/powerpc/include/asm/kexec.h | 3 +
>> arch/powerpc/include/asm/kexec_ranges.h | 1 +
>> arch/powerpc/kexec/crash.c | 95 ++++++++++++++++++++++++-
>> arch/powerpc/kexec/file_load_64.c | 20 +++++-
>> arch/powerpc/kexec/ranges.c | 85 ++++++++++++++++++++++
>> 5 files changed, 202 insertions(+), 2 deletions(-)
>>
>> diff --git a/arch/powerpc/include/asm/kexec.h
>> b/arch/powerpc/include/asm/kexec.h
>> index e75970351bcd..95a98b390d62 100644
>> --- a/arch/powerpc/include/asm/kexec.h
>> +++ b/arch/powerpc/include/asm/kexec.h
>> @@ -141,6 +141,9 @@ void arch_crash_handle_hotplug_event(struct
>> kimage *image, void *arg);
>> int arch_crash_hotplug_support(struct kimage *image, unsigned
>> long kexec_flags);
>> #define arch_crash_hotplug_support arch_crash_hotplug_support
>> +
>> +unsigned int arch_crash_get_elfcorehdr_size(void);
>> +#define crash_get_elfcorehdr_size arch_crash_get_elfcorehdr_size
>> #endif /* CONFIG_CRASH_HOTPLUG */
>> extern int crashing_cpu;
>> diff --git a/arch/powerpc/include/asm/kexec_ranges.h
>> b/arch/powerpc/include/asm/kexec_ranges.h
>> index 8489e844b447..14055896cbcb 100644
>> --- a/arch/powerpc/include/asm/kexec_ranges.h
>> +++ b/arch/powerpc/include/asm/kexec_ranges.h
>> @@ -7,6 +7,7 @@
>> void sort_memory_ranges(struct crash_mem *mrngs, bool merge);
>> struct crash_mem *realloc_mem_ranges(struct crash_mem **mem_ranges);
>> int add_mem_range(struct crash_mem **mem_ranges, u64 base, u64 size);
>> +int remove_mem_range(struct crash_mem **mem_ranges, u64 base, u64
>> size);
>> int get_exclude_memory_ranges(struct crash_mem **mem_ranges);
>> int get_reserved_memory_ranges(struct crash_mem **mem_ranges);
>> int get_crash_memory_ranges(struct crash_mem **mem_ranges);
>> diff --git a/arch/powerpc/kexec/crash.c b/arch/powerpc/kexec/crash.c
>> index 8938a19af12f..21b193e938a3 100644
>> --- a/arch/powerpc/kexec/crash.c
>> +++ b/arch/powerpc/kexec/crash.c
>> @@ -17,6 +17,7 @@
>> #include <linux/irq.h>
>> #include <linux/types.h>
>> #include <linux/libfdt.h>
>> +#include <linux/memory.h>
>> #include <asm/processor.h>
>> #include <asm/machdep.h>
>> @@ -25,6 +26,7 @@
>> #include <asm/setjmp.h>
>> #include <asm/debug.h>
>> #include <asm/interrupt.h>
>> +#include <asm/kexec_ranges.h>
>> /*
>> * The primary CPU waits a while for all secondary CPUs to enter.
>> This is to
>> @@ -398,6 +400,94 @@ void default_machine_crash_shutdown(struct
>> pt_regs *regs)
>> #undef pr_fmt
>> #define pr_fmt(fmt) "crash hp: " fmt
>> +/*
>> + * Advertise preferred elfcorehdr size to userspace via
>> + * /sys/kernel/crash_elfcorehdr_size sysfs interface.
>> + */
>> +unsigned int arch_crash_get_elfcorehdr_size(void)
>> +{
>> + unsigned long phdr_cnt;
>> +
>> + /* A program header for possible CPUs + vmcoreinfo */
>> + phdr_cnt = num_possible_cpus() + 1;
>> + if (IS_ENABLED(CONFIG_MEMORY_HOTPLUG))
>> + phdr_cnt += CONFIG_CRASH_MAX_MEMORY_RANGES;
>> +
>> + return sizeof(struct elfhdr) + (phdr_cnt * sizeof(Elf64_Phdr));
>> +}
>> +
>> +/**
>> + * update_crash_elfcorehdr() - Recreate the elfcorehdr and replace
>> it with old
>> + * elfcorehdr in the kexec segment array.
>> + * @image: the active struct kimage
>> + * @mn: struct memory_notify data handler
>> + */
>> +static void update_crash_elfcorehdr(struct kimage *image, struct
>> memory_notify *mn)
>> +{
>> + int ret;
>> + struct crash_mem *cmem = NULL;
>> + struct kexec_segment *ksegment;
>> + void *ptr, *mem, *elfbuf = NULL;
>> + unsigned long elfsz, memsz, base_addr, size;
>> +
>> + ksegment = &image->segment[image->elfcorehdr_index];
>> + mem = (void *) ksegment->mem;
>> + memsz = ksegment->memsz;
>> +
>> + ret = get_crash_memory_ranges(&cmem);
>> + if (ret) {
>> + pr_err("Failed to get crash mem range\n");
>> + return;
>> + }
>> +
>> + /*
>> + * The hot unplugged memory is part of crash memory ranges,
>> + * remove it here.
>> + */
>> + if (image->hp_action == KEXEC_CRASH_HP_REMOVE_MEMORY) {
>> + base_addr = PFN_PHYS(mn->start_pfn);
>> + size = mn->nr_pages * PAGE_SIZE;
>> + ret = remove_mem_range(&cmem, base_addr, size);
>> + if (ret) {
>> + pr_err("Failed to remove hot-unplugged memory from crash
>> memory ranges\n");
>> + goto out;
>> + }
>> + }
>> +
>> + ret = crash_prepare_elf64_headers(cmem, false, &elfbuf, &elfsz);
>> + if (ret) {
>> + pr_err("Failed to prepare elf header\n");
>> + goto out;
>> + }
>> +
>> + /*
>> + * It is unlikely that kernel hit this because elfcorehdr kexec
>> + * segment (memsz) is built with addition space to accommodate
>> growing
>> + * number of crash memory ranges while loading the kdump kernel.
>> It is
>> + * Just to avoid any unforeseen case.
>> + */
>> + if (elfsz > memsz) {
>> + pr_err("Updated crash elfcorehdr elfsz %lu > memsz %lu",
>> elfsz, memsz);
>> + goto out;
>> + }
>> +
>> + ptr = __va(mem);
>> + if (ptr) {
>> + /* Temporarily invalidate the crash image while it is
>> replaced */
>> + xchg(&kexec_crash_image, NULL);
>> +
>> + /* Replace the old elfcorehdr with newly prepared elfcorehdr */
>> + memcpy((void *)ptr, elfbuf, elfsz);
>> +
>> + /* The crash image is now valid once again */
>> + xchg(&kexec_crash_image, image);
>> + }
>> +out:
>> + kvfree(cmem);
>> + if (elfbuf)
>> + kvfree(elfbuf);
>> +}
>> +
>> /**
>> * get_fdt_index - Loop through the kexec segment array and find
>> * the index of the FDT segment.
>> @@ -478,6 +568,8 @@ int arch_crash_hotplug_support(struct kimage
>> *image, unsigned long kexec_flags)
>> */
>> void arch_crash_handle_hotplug_event(struct kimage *image, void *arg)
>> {
>> + struct memory_notify *mn;
>> +
>> switch (image->hp_action) {
>> case KEXEC_CRASH_HP_REMOVE_CPU:
>> return;
>> @@ -488,7 +580,8 @@ void arch_crash_handle_hotplug_event(struct
>> kimage *image, void *arg)
>> case KEXEC_CRASH_HP_REMOVE_MEMORY:
>> case KEXEC_CRASH_HP_ADD_MEMORY:
>> - pr_info_once("Crash update is not supported for memory
>> hotplug\n");
>> + mn = (struct memory_notify *)arg;
>> + update_crash_elfcorehdr(image, mn);
>> return;
>> default:
>> pr_warn_once("Unknown hotplug action\n");
>> diff --git a/arch/powerpc/kexec/file_load_64.c
>> b/arch/powerpc/kexec/file_load_64.c
>> index 2640a804fcdf..925a69ad2468 100644
>> --- a/arch/powerpc/kexec/file_load_64.c
>> +++ b/arch/powerpc/kexec/file_load_64.c
>> @@ -595,6 +595,23 @@ static void update_backup_region_phdr(struct
>> kimage *image, Elf64_Ehdr *ehdr)
>> }
>> }
>> +static unsigned int kdump_extra_elfcorehdr_size(struct crash_mem
>> *cmem)
>> +{
>> +#if defined(CONFIG_CRASH_HOTPLUG) && defined(CONFIG_MEMORY_HOTPLUG)
>> + unsigned int extra_sz = 0;
>> +
>> + if (CONFIG_CRASH_MAX_MEMORY_RANGES > (unsigned int)PN_XNUM)
>> + pr_warn("Number of Phdrs %u exceeds max\n",
>> CONFIG_CRASH_MAX_MEMORY_RANGES);
>> + else if (cmem->nr_ranges >= CONFIG_CRASH_MAX_MEMORY_RANGES)
>> + pr_warn("Configured crash mem ranges may not be enough\n");
>> + else
>> + extra_sz = (CONFIG_CRASH_MAX_MEMORY_RANGES -
>> cmem->nr_ranges) * sizeof(Elf64_Phdr);
>> +
>> + return extra_sz;
>> +#endif
>> + return 0;
>> +}
>> +
>> /**
>> * load_elfcorehdr_segment - Setup crash memory ranges and
>> initialize elfcorehdr
>> * segment needed to load kdump kernel.
>> @@ -626,7 +643,8 @@ static int load_elfcorehdr_segment(struct kimage
>> *image, struct kexec_buf *kbuf)
>> kbuf->buffer = headers;
>> kbuf->mem = KEXEC_BUF_MEM_UNKNOWN;
>> - kbuf->bufsz = kbuf->memsz = headers_sz;
>> + kbuf->bufsz = headers_sz;
>> + kbuf->memsz = headers_sz + kdump_extra_elfcorehdr_size(cmem);
>> kbuf->top_down = false;
>> ret = kexec_add_buffer(kbuf);
>> diff --git a/arch/powerpc/kexec/ranges.c b/arch/powerpc/kexec/ranges.c
>> index 297b8bc97b9f..00dfd62203bd 100644
>> --- a/arch/powerpc/kexec/ranges.c
>> +++ b/arch/powerpc/kexec/ranges.c
>> @@ -618,4 +618,89 @@ int get_crash_memory_ranges(struct crash_mem
>> **mem_ranges)
>> pr_err("Failed to setup crash memory ranges\n");
>> return ret;
>> }
>> +
>> +/**
>> + * remove_mem_range - Removes the given memory range from the range
>> list.
>> + * @mem_ranges: Range list to remove the memory range to.
>> + * @base: Base address of the range to remove.
>> + * @size: Size of the memory range to remove.
>> + *
>> + * (Re)allocates memory, if needed.
>> + *
>> + * Returns 0 on success, negative errno on error.
>> + */
>> +int remove_mem_range(struct crash_mem **mem_ranges, u64 base, u64 size)
>> +{
>> + u64 end;
>> + int ret = 0;
>> + unsigned int i;
>> + u64 mstart, mend;
>> + struct crash_mem *mem_rngs = *mem_ranges;
>> +
>> + if (!size)
>> + return 0;
>> +
>> + /*
>> + * Memory range are stored as start and end address, use
>> + * the same format to do remove operation.
>> + */
>> + end = base + size - 1;
>> +
>> + for (i = 0; i < mem_rngs->nr_ranges; i++) {
>> + mstart = mem_rngs->ranges[i].start;
>> + mend = mem_rngs->ranges[i].end;
>> +
>> + /*
>> + * Memory range to remove is not part of this range entry
>> + * in the memory range list
>> + */
>> + if (!(base >= mstart && end <= mend))
>> + continue;
>> +
>> + /*
>> + * Memory range to remove is equivalent to this entry in the
>> + * memory range list. Remove the range entry from the list.
>> + */
>> + if (base == mstart && end == mend) {
>> + for (; i < mem_rngs->nr_ranges - 1; i++) {
>> + mem_rngs->ranges[i].start =
>> mem_rngs->ranges[i+1].start;
>> + mem_rngs->ranges[i].end = mem_rngs->ranges[i+1].end;
>> + }
>> + mem_rngs->nr_ranges--;
>> + goto out;
>> + }
>> + /*
>> + * Start address of the memory range to remove and the
>> + * current memory range entry in the list is same. Just
>> + * move the start address of the current memory range
>> + * entry in the list to end + 1.
>> + */
>> + else if (base == mstart) {
>> + mem_rngs->ranges[i].start = end + 1;
>> + goto out;
>> + }
>> + /*
>> + * End address of the memory range to remove and the
>> + * current memory range entry in the list is same.
>> + * Just move the end address of the current memory
>> + * range entry in the list to base - 1.
>> + */
>> + else if (end == mend) {
>> + mem_rngs->ranges[i].end = base - 1;
>> + goto out;
>> + }
>> + /*
>> + * Memory range to remove is not at the edge of current
>> + * memory range entry. Split the current memory entry into
>> + * two half.
>> + */
>> + else {
>> + mem_rngs->ranges[i].end = base - 1;
>> + size = mem_rngs->ranges[i].end - end;
>> + ret = add_mem_range(mem_ranges, end + 1, size);
>> + }
>> + }
>> +out:
>> + return ret;
>> +}
>> #endif /* CONFIG_CRASH_DUMP */
_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
^ permalink raw reply [flat|nested] 3+ messages in thread
* [PATCH v17 0/6] powerpc/crash: Kernel handling of CPU and memory hotplug
@ 2024-02-26 8:41 Sourabh Jain
2024-02-26 8:41 ` [PATCH v17 6/6] powerpc/crash: add crash memory hotplug support Sourabh Jain
0 siblings, 1 reply; 3+ messages in thread
From: Sourabh Jain @ 2024-02-26 8:41 UTC (permalink / raw)
To: linuxppc-dev
Cc: Sourabh Jain, Akhil Raj, Andrew Morton, Aneesh Kumar K . V,
Baoquan He, Borislav Petkov, Boris Ostrovsky, Christophe Leroy,
Dave Hansen, Dave Young, David Hildenbrand, Greg Kroah-Hartman,
Hari Bathini, Laurent Dufour, Mahesh Salgaonkar, Michael Ellerman,
Mimi Zohar, Naveen N Rao, Oscar Salvador, Thomas Gleixner,
Valentin Schneider, Vivek Goyal, kexec, x86
Commit 247262756121 ("crash: add generic infrastructure for crash
hotplug support") added a generic infrastructure that allows
architectures to selectively update the kdump image component during CPU
or memory add/remove events within the kernel itself.
This patch series adds crash hotplug handler for PowerPC and enable
support to update the kdump image on CPU/Memory add/remove events.
Among the 5 patches in this series, the first two patches make changes
to the generic crash hotplug handler to assist PowerPC in adding support
for this feature. The last three patches add support for this feature.
The following section outlines the problem addressed by this patch
series, along with the current solution, its shortcomings, and the
proposed resolution.
Problem:
========
Due to CPU/Memory hotplug or online/offline events the elfcorehdr
(which describes the CPUs and memory of the crashed kernel) and FDT
(Flattened Device Tree) of kdump image becomes outdated. Consequently,
attempting dump collection with an outdated elfcorehdr or FDT can lead
to failed or inaccurate dump collection.
Going forward CPU hotplug or online/offline events are referred as
CPU/Memory add/remove events.
Existing solution and its shortcoming:
======================================
The current solution to address the above issue involves monitoring the
CPU/memory add/remove events in userspace using udev rules and whenever
there are changes in CPU and memory resources, the entire kdump image
is loaded again. The kdump image includes kernel, initrd, elfcorehdr,
FDT, purgatory. Given that only elfcorehdr and FDT get outdated due to
CPU/Memory add/remove events, reloading the entire kdump image is
inefficient. More importantly, kdump remains inactive for a substantial
amount of time until the kdump reload completes.
Proposed solution:
==================
Instead of initiating a full kdump image reload from userspace on
CPU/Memory hotplug and online/offline events, the proposed solution aims
to update only the necessary kdump image component within the kernel
itself.
Git tree for testing:
=====================
https://github.com/sourabhjains/linux/tree/kdump-in-kernel-crash-update-v17
Above tree is rebased on top of linux-next and the below patch series:
https://lore.kernel.org/all/20240213113150.1148276-1-hbathini@linux.ibm.com/
To realize this feature, the kdump udev rule must be updated. On RHEL,
add the following two lines at the top of the
"/usr/lib/udev/rules.d/98-kexec.rules" file.
SUBSYSTEM=="cpu", ATTRS{crash_hotplug}=="1", GOTO="kdump_reload_end"
SUBSYSTEM=="memory", ATTRS{crash_hotplug}=="1", GOTO="kdump_reload_end"
With the above change to the kdump udev rule, kdump reload is avoided
during CPU/Memory add/remove events if this feature is enabled in the
kernel.
Note: only kexec_file_load syscall will work. For kexec_load minor changes
are required in kexec tool.
Changelog:
----------
v17:
- Rebase the patch series on top linux-next tree and below patch series
https://lore.kernel.org/all/20240213113150.1148276-1-hbathini@linux.ibm.com/
- Split 0003 patch from v16 into two patches
1. Move get_crash_memory_ranges() along with other *_memory_ranges()
functions to ranges.c and make them public.
2. Make update_cpus_node function public and take this function
out of file_load_64.c
- Keep arch_crash_hotplug_support in crash.c instead of core_64.c [05/06]
- Use CONFIG_CRASH_MAX_MEMORY_RANGES to find extra elfcorehdr size [06/06]
v16: [https://lore.kernel.org/all/20240217081452.164571-1-sourabhjain@linux.ibm.com/]
- Remove the unused #define `crash_hotplug_cpu_support`
and `crash_hotplug_memory_support` in `arch/x86/include/asm/kexec.h`.
- Document why two kexec flag bits are used in
`arch_crash_hotplug_memory_support` (x86).
- Use a switch case to handle different hotplug operations
in `arch_crash_handle_hotplug_event` for PowerPC.
- Fix a typo in 4/5.
v15:
- Remove the patch that adds a new kexec flag for FDT update.
- Introduce a generic kexec flag bit to share hotplug support
intent between the kexec tool and the kernel for the kexec_load
syscall. (2/5)
- Introduce an architecture-specific handler to process the kexec
flag for crash hotplug support. (2/5)
- Rename the @update_elfcorehdr member of the struct kimage to
@hotplug_support. (2/5)
- Use a common function to advertise hotplug support for both CPU
and Memory. (2/5)
v14:
- Fix build warnings by including necessary header files
- Rebase to v6.7-rc5
v13:
- Fix a build warning, take ranges.c out of CONFIG_KEXEC_FILE
- Rebase to v6.7-rc4
v12:
- A patch to add new kexec flags to support this feature on kexec_load
system call
- Change in the way this feature is advertise to userspace for both
kexec_load syscall
- Rebase to v6.6-rc7
v11:
- Rebase to v6.4-rc6
- The patch that introduced CONFIG_CRASH_HOTPLUG for PowerPC has been
removed. The config is now part of common configuration:
https://lore.kernel.org/all/87ilbpflsk.fsf@mail.lhotse/
v10:
- Drop the patch that adds fdt_index attribute to struct kimage_arch
Find the fdt segment index when needed.
- Added more details into commits messages.
- Rebased onto 6.3.0-rc5
v9:
- Removed patch to prepare elfcorehdr crash notes for possible CPUs.
The patch is moved to generic patch series that introduces generic
infrastructure for in kernel crash update.
- Removed patch to pass the hotplug action type to the arch crash
hotplug handler function. The generic patch series has introduced
the hotplug action type in kimage struct.
- Add detail commit message for better understanding.
v8:
- Restrict fdt_index initialization to machine_kexec_post_load
it work for both kexec_load and kexec_file_load.[3/8] Laurent Dufour
- Updated the logic to find the number of offline core. [6/8]
- Changed the logic to find the elfcore program header to accommodate
future memory ranges due memory hotplug events. [8/8]
v7
- added a new config to configure this feature
- pass hotplug action type to arch specific handler
v6
- Added crash memory hotplug support
v5:
- Replace COFNIG_CRASH_HOTPLUG with CONFIG_HOTPLUG_CPU.
- Move fdt segment identification for kexec_load case to load path
instead of crash hotplug handler
- Keep new attribute defined under kimage_arch to track FDT segment
under CONFIG_HOTPLUG_CPU config.
v4:
- Update the logic to find the additional space needed for hotadd CPUs
post kexec load. Refer "[RFC v4 PATCH 4/5] powerpc/crash hp: add crash
hotplug support for kexec_file_load" patch to know more about the
change.
- Fix a couple of typo.
- Replace pr_err to pr_info_once to warn user about memory hotplug
support.
- In crash hotplug handle exit the for loop if FDT segment is found.
v3
- Move fdt_index and fdt_index_vaild variables to kimage_arch struct.
- Rebase patche on top of
https://lore.kernel.org/lkml/20220303162725.49640-1-eric.devolder@oracle.com/
- Fixed warning reported by checpatch script
v2:
- Use generic hotplug handler introduced by
https://lore.kernel.org/lkml/20220209195706.51522-1-eric.devolder@oracle.com/
a significant change from v1.
Cc: Akhil Raj <lf32.dev@gmail.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Aneesh Kumar K.V <aneesh.kumar@kernel.org>
Cc: Baoquan He <bhe@redhat.com>
Cc: Borislav Petkov (AMD) <bp@alien8.de>
Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Christophe Leroy <christophe.leroy@csgroup.eu>
Cc: Dave Hansen <dave.hansen@linux.intel.com>
Cc: Dave Young <dyoung@redhat.com>
Cc: David Hildenbrand <david@redhat.com>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Hari Bathini <hbathini@linux.ibm.com>
Cc: Laurent Dufour <laurent.dufour@fr.ibm.com>
Cc: Mahesh Salgaonkar <mahesh@linux.ibm.com>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Mimi Zohar <zohar@linux.ibm.com>
Cc: Naveen N Rao <naveen@kernel.org>
Cc: Oscar Salvador <osalvador@suse.de>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Valentin Schneider <vschneid@redhat.com>
Cc: Vivek Goyal <vgoyal@redhat.com>
Cc: kexec@lists.infradead.org
Cc: x86@kernel.org
Sourabh Jain (6):
crash: forward memory_notify arg to arch crash hotplug handler
crash: add a new kexec flag for hotplug support
powerpc/kexec: move *_memory_ranges functions to ranges.c
PowerPC/kexec: make the update_cpus_node() function public
powerpc/crash: add crash CPU hotplug support
powerpc/crash: add crash memory hotplug support
arch/powerpc/Kconfig | 4 +
arch/powerpc/include/asm/kexec.h | 15 ++
arch/powerpc/include/asm/kexec_ranges.h | 20 +-
arch/powerpc/kexec/Makefile | 4 +-
arch/powerpc/kexec/core_64.c | 91 +++++++
arch/powerpc/kexec/crash.c | 196 +++++++++++++++
arch/powerpc/kexec/elf_64.c | 3 +-
arch/powerpc/kexec/file_load_64.c | 314 +++---------------------
arch/powerpc/kexec/ranges.c | 312 ++++++++++++++++++++++-
arch/x86/include/asm/kexec.h | 13 +-
arch/x86/kernel/crash.c | 32 ++-
drivers/base/cpu.c | 2 +-
drivers/base/memory.c | 2 +-
include/linux/crash_core.h | 15 +-
include/linux/kexec.h | 11 +-
include/uapi/linux/kexec.h | 1 +
kernel/crash_core.c | 25 +-
kernel/kexec.c | 4 +-
kernel/kexec_file.c | 5 +
19 files changed, 712 insertions(+), 357 deletions(-)
--
2.43.0
_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
^ permalink raw reply [flat|nested] 3+ messages in thread* [PATCH v17 6/6] powerpc/crash: add crash memory hotplug support
2024-02-26 8:41 [PATCH v17 0/6] powerpc/crash: Kernel handling of CPU and memory hotplug Sourabh Jain
@ 2024-02-26 8:41 ` Sourabh Jain
0 siblings, 0 replies; 3+ messages in thread
From: Sourabh Jain @ 2024-02-26 8:41 UTC (permalink / raw)
To: linuxppc-dev
Cc: Sourabh Jain, Akhil Raj, Andrew Morton, Aneesh Kumar K . V,
Baoquan He, Borislav Petkov, Boris Ostrovsky, Christophe Leroy,
Dave Hansen, Dave Young, David Hildenbrand, Greg Kroah-Hartman,
Hari Bathini, Laurent Dufour, Mahesh Salgaonkar, Michael Ellerman,
Mimi Zohar, Naveen N Rao, Oscar Salvador, Thomas Gleixner,
Valentin Schneider, Vivek Goyal, kexec, x86
Extend the arch crash hotplug handler, as introduced by the patch title
("powerpc: add crash CPU hotplug support"), to also support memory
add/remove events.
Elfcorehdr describes the memory of the crash kernel to capture the
kernel; hence, it needs to be updated if memory resources change due to
memory add/remove events. Therefore, arch_crash_handle_hotplug_event()
is updated to recreate the elfcorehdr and replace it with the previous
one on memory add/remove events.
The memblock list is used to prepare the elfcorehdr. In the case of
memory hot remove, the memblock list is updated after the arch crash
hotplug handler is triggered, as depicted in Figure 1. Thus, the
hot-removed memory is explicitly removed from the crash memory ranges
to ensure that the memory ranges added to elfcorehdr do not include the
hot-removed memory.
Memory remove
|
v
Offline pages
|
v
Initiate memory notify call <----> crash hotplug handler
chain for MEM_OFFLINE event
|
v
Update memblock list
Figure 1
There are two system calls, `kexec_file_load` and `kexec_load`, used to
load the kdump image. A few changes have been made to ensure that the
kernel can safely update the elfcorehdr component of the kdump image for
both system calls.
For the kexec_file_load syscall, kdump image is prepared in the kernel.
To support an increasing number of memory regions, the elfcorehdr is
built with extra buffer space to ensure that it can accommodate
additional memory ranges in future.
For the kexec_load syscall, the elfcorehdr is updated only if the
KEXEC_CRASH_HOTPLUG_SUPPORT kexec flag is passed to the kernel by the
kexec tool. Passing this flag to the kernel indicates that the
elfcorehdr is built to accommodate additional memory ranges and the
elfcorehdr segment is not considered for SHA calculation, making it safe
to update.
The changes related to this feature are kept under the CRASH_HOTPLUG
config, and it is enabled by default.
Signed-off-by: Sourabh Jain <sourabhjain@linux.ibm.com>
Cc: Akhil Raj <lf32.dev@gmail.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Aneesh Kumar K.V <aneesh.kumar@kernel.org>
Cc: Baoquan He <bhe@redhat.com>
Cc: Borislav Petkov (AMD) <bp@alien8.de>
Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Christophe Leroy <christophe.leroy@csgroup.eu>
Cc: Dave Hansen <dave.hansen@linux.intel.com>
Cc: Dave Young <dyoung@redhat.com>
Cc: David Hildenbrand <david@redhat.com>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Hari Bathini <hbathini@linux.ibm.com>
Cc: Laurent Dufour <laurent.dufour@fr.ibm.com>
Cc: Mahesh Salgaonkar <mahesh@linux.ibm.com>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Mimi Zohar <zohar@linux.ibm.com>
Cc: Naveen N Rao <naveen@kernel.org>
Cc: Oscar Salvador <osalvador@suse.de>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Valentin Schneider <vschneid@redhat.com>
Cc: Vivek Goyal <vgoyal@redhat.com>
Cc: kexec@lists.infradead.org
Cc: x86@kernel.org
---
arch/powerpc/include/asm/kexec.h | 3 +
arch/powerpc/include/asm/kexec_ranges.h | 1 +
arch/powerpc/kexec/crash.c | 95 ++++++++++++++++++++++++-
arch/powerpc/kexec/file_load_64.c | 20 +++++-
arch/powerpc/kexec/ranges.c | 85 ++++++++++++++++++++++
5 files changed, 202 insertions(+), 2 deletions(-)
diff --git a/arch/powerpc/include/asm/kexec.h b/arch/powerpc/include/asm/kexec.h
index e75970351bcd..95a98b390d62 100644
--- a/arch/powerpc/include/asm/kexec.h
+++ b/arch/powerpc/include/asm/kexec.h
@@ -141,6 +141,9 @@ void arch_crash_handle_hotplug_event(struct kimage *image, void *arg);
int arch_crash_hotplug_support(struct kimage *image, unsigned long kexec_flags);
#define arch_crash_hotplug_support arch_crash_hotplug_support
+
+unsigned int arch_crash_get_elfcorehdr_size(void);
+#define crash_get_elfcorehdr_size arch_crash_get_elfcorehdr_size
#endif /* CONFIG_CRASH_HOTPLUG */
extern int crashing_cpu;
diff --git a/arch/powerpc/include/asm/kexec_ranges.h b/arch/powerpc/include/asm/kexec_ranges.h
index 8489e844b447..14055896cbcb 100644
--- a/arch/powerpc/include/asm/kexec_ranges.h
+++ b/arch/powerpc/include/asm/kexec_ranges.h
@@ -7,6 +7,7 @@
void sort_memory_ranges(struct crash_mem *mrngs, bool merge);
struct crash_mem *realloc_mem_ranges(struct crash_mem **mem_ranges);
int add_mem_range(struct crash_mem **mem_ranges, u64 base, u64 size);
+int remove_mem_range(struct crash_mem **mem_ranges, u64 base, u64 size);
int get_exclude_memory_ranges(struct crash_mem **mem_ranges);
int get_reserved_memory_ranges(struct crash_mem **mem_ranges);
int get_crash_memory_ranges(struct crash_mem **mem_ranges);
diff --git a/arch/powerpc/kexec/crash.c b/arch/powerpc/kexec/crash.c
index 8938a19af12f..21b193e938a3 100644
--- a/arch/powerpc/kexec/crash.c
+++ b/arch/powerpc/kexec/crash.c
@@ -17,6 +17,7 @@
#include <linux/irq.h>
#include <linux/types.h>
#include <linux/libfdt.h>
+#include <linux/memory.h>
#include <asm/processor.h>
#include <asm/machdep.h>
@@ -25,6 +26,7 @@
#include <asm/setjmp.h>
#include <asm/debug.h>
#include <asm/interrupt.h>
+#include <asm/kexec_ranges.h>
/*
* The primary CPU waits a while for all secondary CPUs to enter. This is to
@@ -398,6 +400,94 @@ void default_machine_crash_shutdown(struct pt_regs *regs)
#undef pr_fmt
#define pr_fmt(fmt) "crash hp: " fmt
+/*
+ * Advertise preferred elfcorehdr size to userspace via
+ * /sys/kernel/crash_elfcorehdr_size sysfs interface.
+ */
+unsigned int arch_crash_get_elfcorehdr_size(void)
+{
+ unsigned long phdr_cnt;
+
+ /* A program header for possible CPUs + vmcoreinfo */
+ phdr_cnt = num_possible_cpus() + 1;
+ if (IS_ENABLED(CONFIG_MEMORY_HOTPLUG))
+ phdr_cnt += CONFIG_CRASH_MAX_MEMORY_RANGES;
+
+ return sizeof(struct elfhdr) + (phdr_cnt * sizeof(Elf64_Phdr));
+}
+
+/**
+ * update_crash_elfcorehdr() - Recreate the elfcorehdr and replace it with old
+ * elfcorehdr in the kexec segment array.
+ * @image: the active struct kimage
+ * @mn: struct memory_notify data handler
+ */
+static void update_crash_elfcorehdr(struct kimage *image, struct memory_notify *mn)
+{
+ int ret;
+ struct crash_mem *cmem = NULL;
+ struct kexec_segment *ksegment;
+ void *ptr, *mem, *elfbuf = NULL;
+ unsigned long elfsz, memsz, base_addr, size;
+
+ ksegment = &image->segment[image->elfcorehdr_index];
+ mem = (void *) ksegment->mem;
+ memsz = ksegment->memsz;
+
+ ret = get_crash_memory_ranges(&cmem);
+ if (ret) {
+ pr_err("Failed to get crash mem range\n");
+ return;
+ }
+
+ /*
+ * The hot unplugged memory is part of crash memory ranges,
+ * remove it here.
+ */
+ if (image->hp_action == KEXEC_CRASH_HP_REMOVE_MEMORY) {
+ base_addr = PFN_PHYS(mn->start_pfn);
+ size = mn->nr_pages * PAGE_SIZE;
+ ret = remove_mem_range(&cmem, base_addr, size);
+ if (ret) {
+ pr_err("Failed to remove hot-unplugged memory from crash memory ranges\n");
+ goto out;
+ }
+ }
+
+ ret = crash_prepare_elf64_headers(cmem, false, &elfbuf, &elfsz);
+ if (ret) {
+ pr_err("Failed to prepare elf header\n");
+ goto out;
+ }
+
+ /*
+ * It is unlikely that kernel hit this because elfcorehdr kexec
+ * segment (memsz) is built with addition space to accommodate growing
+ * number of crash memory ranges while loading the kdump kernel. It is
+ * Just to avoid any unforeseen case.
+ */
+ if (elfsz > memsz) {
+ pr_err("Updated crash elfcorehdr elfsz %lu > memsz %lu", elfsz, memsz);
+ goto out;
+ }
+
+ ptr = __va(mem);
+ if (ptr) {
+ /* Temporarily invalidate the crash image while it is replaced */
+ xchg(&kexec_crash_image, NULL);
+
+ /* Replace the old elfcorehdr with newly prepared elfcorehdr */
+ memcpy((void *)ptr, elfbuf, elfsz);
+
+ /* The crash image is now valid once again */
+ xchg(&kexec_crash_image, image);
+ }
+out:
+ kvfree(cmem);
+ if (elfbuf)
+ kvfree(elfbuf);
+}
+
/**
* get_fdt_index - Loop through the kexec segment array and find
* the index of the FDT segment.
@@ -478,6 +568,8 @@ int arch_crash_hotplug_support(struct kimage *image, unsigned long kexec_flags)
*/
void arch_crash_handle_hotplug_event(struct kimage *image, void *arg)
{
+ struct memory_notify *mn;
+
switch (image->hp_action) {
case KEXEC_CRASH_HP_REMOVE_CPU:
return;
@@ -488,7 +580,8 @@ void arch_crash_handle_hotplug_event(struct kimage *image, void *arg)
case KEXEC_CRASH_HP_REMOVE_MEMORY:
case KEXEC_CRASH_HP_ADD_MEMORY:
- pr_info_once("Crash update is not supported for memory hotplug\n");
+ mn = (struct memory_notify *)arg;
+ update_crash_elfcorehdr(image, mn);
return;
default:
pr_warn_once("Unknown hotplug action\n");
diff --git a/arch/powerpc/kexec/file_load_64.c b/arch/powerpc/kexec/file_load_64.c
index 2640a804fcdf..925a69ad2468 100644
--- a/arch/powerpc/kexec/file_load_64.c
+++ b/arch/powerpc/kexec/file_load_64.c
@@ -595,6 +595,23 @@ static void update_backup_region_phdr(struct kimage *image, Elf64_Ehdr *ehdr)
}
}
+static unsigned int kdump_extra_elfcorehdr_size(struct crash_mem *cmem)
+{
+#if defined(CONFIG_CRASH_HOTPLUG) && defined(CONFIG_MEMORY_HOTPLUG)
+ unsigned int extra_sz = 0;
+
+ if (CONFIG_CRASH_MAX_MEMORY_RANGES > (unsigned int)PN_XNUM)
+ pr_warn("Number of Phdrs %u exceeds max\n", CONFIG_CRASH_MAX_MEMORY_RANGES);
+ else if (cmem->nr_ranges >= CONFIG_CRASH_MAX_MEMORY_RANGES)
+ pr_warn("Configured crash mem ranges may not be enough\n");
+ else
+ extra_sz = (CONFIG_CRASH_MAX_MEMORY_RANGES - cmem->nr_ranges) * sizeof(Elf64_Phdr);
+
+ return extra_sz;
+#endif
+ return 0;
+}
+
/**
* load_elfcorehdr_segment - Setup crash memory ranges and initialize elfcorehdr
* segment needed to load kdump kernel.
@@ -626,7 +643,8 @@ static int load_elfcorehdr_segment(struct kimage *image, struct kexec_buf *kbuf)
kbuf->buffer = headers;
kbuf->mem = KEXEC_BUF_MEM_UNKNOWN;
- kbuf->bufsz = kbuf->memsz = headers_sz;
+ kbuf->bufsz = headers_sz;
+ kbuf->memsz = headers_sz + kdump_extra_elfcorehdr_size(cmem);
kbuf->top_down = false;
ret = kexec_add_buffer(kbuf);
diff --git a/arch/powerpc/kexec/ranges.c b/arch/powerpc/kexec/ranges.c
index 297b8bc97b9f..00dfd62203bd 100644
--- a/arch/powerpc/kexec/ranges.c
+++ b/arch/powerpc/kexec/ranges.c
@@ -618,4 +618,89 @@ int get_crash_memory_ranges(struct crash_mem **mem_ranges)
pr_err("Failed to setup crash memory ranges\n");
return ret;
}
+
+/**
+ * remove_mem_range - Removes the given memory range from the range list.
+ * @mem_ranges: Range list to remove the memory range to.
+ * @base: Base address of the range to remove.
+ * @size: Size of the memory range to remove.
+ *
+ * (Re)allocates memory, if needed.
+ *
+ * Returns 0 on success, negative errno on error.
+ */
+int remove_mem_range(struct crash_mem **mem_ranges, u64 base, u64 size)
+{
+ u64 end;
+ int ret = 0;
+ unsigned int i;
+ u64 mstart, mend;
+ struct crash_mem *mem_rngs = *mem_ranges;
+
+ if (!size)
+ return 0;
+
+ /*
+ * Memory range are stored as start and end address, use
+ * the same format to do remove operation.
+ */
+ end = base + size - 1;
+
+ for (i = 0; i < mem_rngs->nr_ranges; i++) {
+ mstart = mem_rngs->ranges[i].start;
+ mend = mem_rngs->ranges[i].end;
+
+ /*
+ * Memory range to remove is not part of this range entry
+ * in the memory range list
+ */
+ if (!(base >= mstart && end <= mend))
+ continue;
+
+ /*
+ * Memory range to remove is equivalent to this entry in the
+ * memory range list. Remove the range entry from the list.
+ */
+ if (base == mstart && end == mend) {
+ for (; i < mem_rngs->nr_ranges - 1; i++) {
+ mem_rngs->ranges[i].start = mem_rngs->ranges[i+1].start;
+ mem_rngs->ranges[i].end = mem_rngs->ranges[i+1].end;
+ }
+ mem_rngs->nr_ranges--;
+ goto out;
+ }
+ /*
+ * Start address of the memory range to remove and the
+ * current memory range entry in the list is same. Just
+ * move the start address of the current memory range
+ * entry in the list to end + 1.
+ */
+ else if (base == mstart) {
+ mem_rngs->ranges[i].start = end + 1;
+ goto out;
+ }
+ /*
+ * End address of the memory range to remove and the
+ * current memory range entry in the list is same.
+ * Just move the end address of the current memory
+ * range entry in the list to base - 1.
+ */
+ else if (end == mend) {
+ mem_rngs->ranges[i].end = base - 1;
+ goto out;
+ }
+ /*
+ * Memory range to remove is not at the edge of current
+ * memory range entry. Split the current memory entry into
+ * two half.
+ */
+ else {
+ mem_rngs->ranges[i].end = base - 1;
+ size = mem_rngs->ranges[i].end - end;
+ ret = add_mem_range(mem_ranges, end + 1, size);
+ }
+ }
+out:
+ return ret;
+}
#endif /* CONFIG_CRASH_DUMP */
--
2.43.0
_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
^ permalink raw reply related [flat|nested] 3+ messages in thread
end of thread, other threads:[~2024-03-04 8:29 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2024-03-02 13:19 [PATCH v17 6/6] powerpc/crash: add crash memory hotplug support Hari Bathini
2024-03-04 8:28 ` Sourabh Jain
-- strict thread matches above, loose matches on Subject: below --
2024-02-26 8:41 [PATCH v17 0/6] powerpc/crash: Kernel handling of CPU and memory hotplug Sourabh Jain
2024-02-26 8:41 ` [PATCH v17 6/6] powerpc/crash: add crash memory hotplug support Sourabh Jain
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox