[PATCH v1 0/8] s390/kdump: Add crashkernel CMA support

[PATCH v1 0/8] s390/kdump: Add crashkernel CMA support

[Mikhail Zaslonko]

This series adds support for crashkernel=size,cma on s390, as already
available on x86, arm64, riscv and powerpc. In addition to the regular
crashkernel reservation, CMA memory is reserved. The production kernel
uses it for movable allocations, and the kdump kernel uses it as
additional memory. The regular reservation then only needs to hold the
kdump kernel and its initrd.

On s390 the kdump kernel runs in the crashkernel memory, which is
swapped to address 0, and its decompressor limits usable memory to the
crashkernel size. All other memory belongs to the crashed kernel and
makes up the dump. To use the CMA memory, the kdump kernel has to know
where it is, and the decompressor has to handle usable memory above
that limit.

Patches 1-3 reserve the CMA memory with the generic
reserve_crashkernel_cma() and store the ranges in the parmarea of a
kdump kernel loaded with kexec_file_load(). They use 64 of the 72
padding bytes before the command line, enough for the four ranges the
generic code reserves at most. Kernel images have this area zeroed, so
older and newer kernels keep working together, just without CMA memory
for kdump. kexec_load() needs a matching kexec-tools change, which can
read the ranges from /sys/kernel/kexec_crash_cma_ranges.

Patch 4 replaces the decompressor's usable memory limit with a small
sorted array of usable ranges, physmem_info.usable[]. The iterator
returns their intersection with online memory, and the boot allocator
walks only these ranges, top-down from ident_map_size. The identity
mapping, the KASAN shadow and memblock use the same iterator. A normal
boot has a single range from 0 to ident_map_size and behaves as before.

Patch 5 excludes the CMA ranges from /proc/vmcore. They are kept in
oldmem_data and added to oldmem_type next to the crashkernel memory, so
no PT_LOAD headers are created for them.

Patch 6 lets the kdump kernel use the CMA memory. If the parmarea
contains CMA ranges, the decompressor does not limit ident_map_size to
the crashkernel size and adds the crashkernel memory and the CMA ranges
as usable ranges. All other memory up to ident_map_size is treated as
a memory hole. The CMA memory is then mapped, added to memblock and
freed to the buddy allocator like any other memory. Boot allocations can
use it as well: page tables, the KASAN shadow and the initrd.

Patch 7 skips the generic 10 second wait for pending DMA before the
kdump kernel is started. s390 does not need it, as s390_reset_system()
terminates all in-flight I/O before the kdump kernel starts. The patch
makes the timeout overridable in kernel/crash_core.c and sets it to 0
on s390. It is the only patch in the series that touches generic code.

Patch 8 updates the documentation.

Without ",cma", the behavior is the same as before: the decompressor
has a single usable range, and a kdump kernel uses only the crashkernel
memory.

Tested on a 32G z/VM guest and on a 64G LPAR:
- crashkernel=82M crashkernel=1G,cma: the kdump kernel has ~1.1G of
  memory and writes a dump that contains all memory except the
  crashkernel and CMA ranges. With crashkernel=128M alone, the same
  kdump kernel and initrd run out of memory.
- 26G of CMA in four ranges (z/VM, with the two generic fixes below):
  same result.
- With ",cma", the kdump kernel starts without the 10 second wait.
- Without ",cma": same memory layout and dump as before.

Two generic crashkernel CMA fixes are queued in kexec-next. This series
applies without them, but both problems show up on s390:
- kdump: Give crashkernel CMA areas unique names [1]
  When the reservation is split into several CMA areas, the CMA sysfs
  setup fails, and debugfs shows only the first crashkernel area.
- crash: keep crashkernel CMA pages reserved if activation fails [2]
  A CMA area that crosses a zone boundary fails to activate, and its
  pages go to the buddy allocator although the kdump kernel still uses
  that memory. On s390 ZONE_DMA ends at 2G so this can happen when an
  area does not fit above 4G.

Usama Arif's hugetlb CMA v2 [3] changes reserve_crashkernel_cma() for
all architectures. This series needs to adapt the s390 call in patch 2
if hugetlb CMA patch goes in.

[1] https://lore.kernel.org/lkml/20261106104309.3669583-1-zaslonko@linux.ibm.com/
[2] https://lore.kernel.org/lkml/20261104145600.3708886-1-usama.arif@linux.dev/
[3] https://lore.kernel.org/r/20261106165142.2847916-1-usama.arif@linux.dev/

Mikhail Zaslonko (8):
  s390/kdump: Enable crashkernel CMA infrastructure
  s390/kdump: Reserve crashkernel CMA memory
  s390/kexec: Pass crashkernel CMA ranges to the kdump kernel
  s390/boot: Allow additional usable memory ranges
  s390/kdump: Exclude crashkernel CMA memory from vmcore
  s390/kdump: Use crashkernel CMA memory in the kdump kernel
  s390/kdump: Skip the crashkernel CMA DMA wait
  Documentation: kdump: Add s390 to crashkernel CMA

 Documentation/admin-guide/kdump/kdump.rst     |  3 +
 .../admin-guide/kernel-parameters.txt         |  4 +-
 arch/s390/Kconfig                             |  3 +
 arch/s390/boot/boot.h                         |  3 +-
 arch/s390/boot/kaslr.c                        |  4 +-
 arch/s390/boot/physmem_info.c                 | 65 ++++++++++++++++---
 arch/s390/boot/startup.c                      | 30 +++++++--
 arch/s390/boot/vmem.c                         |  7 +-
 arch/s390/include/asm/crash_reserve.h         | 24 +++++++
 arch/s390/include/asm/kexec.h                 |  3 +
 arch/s390/include/asm/physmem_info.h          | 63 +++++++++++++-----
 arch/s390/include/asm/setup.h                 | 10 ++-
 arch/s390/kernel/crash_dump.c                 | 48 +++++++++++---
 arch/s390/kernel/machine_kexec_file.c         |  5 ++
 arch/s390/kernel/setup.c                      | 17 ++++-
 kernel/crash_core.c                           |  2 +
 16 files changed, 241 insertions(+), 50 deletions(-)
 create mode 100644 arch/s390/include/asm/crash_reserve.h


base-commit: 067c5c24cef23d913b67eaaf71ae67e7169a6620
-- 
2.55.0

[PATCH v1 1/8] s390/kdump: Enable crashkernel CMA infrastructure

[Mikhail Zaslonko]

Select ARCH_HAS_GENERIC_CRASHKERNEL_RESERVATION to build the generic
crashkernel CMA code: reserve_crashkernel_cma() and crashk_cma_ranges[].

s390 keeps its own reserve_crashkernel(), the CRASH_* definitions in the
new asm/crash_reserve.h are only there to satisfy the generic header.
arch_add_crash_res_to_iomem() returns false because setup_resources()
already inserts the crashkernel resource.

Signed-off-by: Mikhail Zaslonko <zaslonko@linux.ibm.com>
---
 arch/s390/Kconfig                     |  3 +++
 arch/s390/include/asm/crash_reserve.h | 24 ++++++++++++++++++++++++
 2 files changed, 27 insertions(+)
 create mode 100644 arch/s390/include/asm/crash_reserve.h

diff --git a/arch/s390/Kconfig b/arch/s390/Kconfig
index 7138d923d86f..1a99fcf520bf 100644
--- a/arch/s390/Kconfig
+++ b/arch/s390/Kconfig
@@ -331,6 +331,9 @@ config ARCH_SUPPORTS_CRASH_DUMP
 	  This option also enables s390 zfcpdump.
 	  See also <file:Documentation/arch/s390/zfcpdump.rst>
 
+config ARCH_HAS_GENERIC_CRASHKERNEL_RESERVATION
+	def_bool CRASH_RESERVE
+
 config ARCH_DEFAULT_CRASH_DUMP
 	def_bool y
 
diff --git a/arch/s390/include/asm/crash_reserve.h b/arch/s390/include/asm/crash_reserve.h
new file mode 100644
index 000000000000..7e04db630bc6
--- /dev/null
+++ b/arch/s390/include/asm/crash_reserve.h
@@ -0,0 +1,24 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_S390_CRASH_RESERVE_H
+#define _ASM_S390_CRASH_RESERVE_H
+/*
+ * s390 crashkernel alignment and address limits.
+ */
+
+#define CRASH_ALIGN		HPAGE_SIZE	/* matches KEXEC_CRASH_MEM_ALIGN */
+
+#define CRASH_ADDR_LOW_MAX	(4UL << 30)	/* 4 GB */
+#define CRASH_ADDR_HIGH_MAX	ULONG_MAX
+
+/*
+ * s390 keeps its own reserve_crashkernel() and manages its own
+ * iomem resources via setup_resources(). Prevent the generic
+ * insert_crashkernel_resources() from inserting duplicates.
+ */
+#define arch_add_crash_res_to_iomem arch_add_crash_res_to_iomem
+static inline bool arch_add_crash_res_to_iomem(void)
+{
+	return false;
+}
+
+#endif /* _ASM_S390_CRASH_RESERVE_H */
-- 
2.55.0

Re: [PATCH v1 1/8] s390/kdump: Enable crashkernel CMA infrastructure

[sashiko-bot]

> Select ARCH_HAS_GENERIC_CRASHKERNEL_RESERVATION to build the generic
> crashkernel CMA code: reserve_crashkernel_cma() and crashk_cma_ranges[].
> 
> s390 keeps its own reserve_crashkernel(), the CRASH_* definitions in the
> new asm/crash_reserve.h are only there to satisfy the generic header.
> arch_add_crash_res_to_iomem() returns false because setup_resources()
> already inserts the crashkernel resource.
> 
> Signed-off-by: Mikhail Zaslonko <zaslonko@linux.ibm.com>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261112112105.1169118-1-zaslonko@linux.ibm.com?part=1

[PATCH v1 2/8] s390/kdump: Reserve crashkernel CMA memory

[Mikhail Zaslonko]

Parse the ",cma" suffix of crashkernel= and reserve the CMA memory with
reserve_crashkernel_cma(). A kdump kernel does not reserve any, it uses
the CMA memory of the crashed kernel instead.

Signed-off-by: Mikhail Zaslonko <zaslonko@linux.ibm.com>
---
 arch/s390/kernel/setup.c | 11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)

diff --git a/arch/s390/kernel/setup.c b/arch/s390/kernel/setup.c
index b60284328fe3..0251bd35954a 100644
--- a/arch/s390/kernel/setup.c
+++ b/arch/s390/kernel/setup.c
@@ -598,12 +598,12 @@ static void __init reserve_pgtables(void)
 static void __init reserve_crashkernel(void)
 {
 #ifdef CONFIG_CRASH_DUMP
-	unsigned long long crash_base, crash_size;
+	unsigned long long crash_base, crash_size, cma_size = 0;
 	phys_addr_t low, high;
 	int rc;
 
 	rc = parse_crashkernel(boot_command_line, ident_map_size,
-			       &crash_size, &crash_base, NULL, NULL, NULL);
+			       &crash_size, &crash_base, NULL, &cma_size, NULL);
 
 	crash_base = ALIGN(crash_base, KEXEC_CRASH_MEM_ALIGN);
 	crash_size = ALIGN(crash_size, KEXEC_CRASH_MEM_ALIGN);
@@ -658,6 +658,13 @@ static void __init reserve_crashkernel(void)
 		crash_size >> 20, crash_base >> 20,
 		(unsigned long)memblock.memory.total_size >> 20);
 	os_info_crashkernel_add(crash_base, crash_size);
+
+	/*
+	 * A kdump kernel is handed the CMA ranges of the crashed kernel
+	 * instead of reserving its own.
+	 */
+	if (!oldmem_data.start)
+		reserve_crashkernel_cma(cma_size);
 #endif
 }
 
-- 
2.55.0

Re: [PATCH v1 2/8] s390/kdump: Reserve crashkernel CMA memory

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential
issue to consider:
- [Medium] s390/kdump: reserve_crashkernel() allows CMA allocation in
  [0, crash_size)
--

commit 8b228cd4036f52fccdf6ce96ff9330544194ef3d
Author: Mikhail Zaslonko <zaslonko@linux.ibm.com>

s390/kdump: Reserve crashkernel CMA memory

This commit parses the ",cma" suffix of the crashkernel= command-line
parameter on s390 and reserves the requested CMA memory via
reserve_crashkernel_cma() when not booting as a kdump kernel.

> diff --git a/arch/s390/kernel/setup.c b/arch/s390/kernel/setup.c
> index b60284328fe3d..0251bd35954a8 100644
> --- a/arch/s390/kernel/setup.c
> +++ b/arch/s390/kernel/setup.c
> @@ -598,12 +598,12 @@ static void __init reserve_pgtables(void)
>  static void __init reserve_crashkernel(void)
>  {
>  #ifdef CONFIG_CRASH_DUMP
> -	unsigned long long crash_base, crash_size;
> +	unsigned long long crash_base, crash_size, cma_size = 0;
>  	phys_addr_t low, high;
>  	int rc;
>  
>  	rc = parse_crashkernel(boot_command_line, ident_map_size,
> -			       &crash_size, &crash_base, NULL, NULL, NULL);
> +			       &crash_size, &crash_base, NULL, &cma_size, NULL);
>  
>  	crash_base = ALIGN(crash_base, KEXEC_CRASH_MEM_ALIGN);
>  	crash_size = ALIGN(crash_size, KEXEC_CRASH_MEM_ALIGN);
> @@ -658,6 +658,13 @@ static void __init reserve_crashkernel(void)
>  		crash_size >> 20, crash_base >> 20,
>  		(unsigned long)memblock.memory.total_size >> 20);
>  	os_info_crashkernel_add(crash_base, crash_size);
> +
> +	/*
> +	 * A kdump kernel is handed the CMA ranges of the crashed kernel
> +	 * instead of reserving its own.
> +	 */
> +	if (!oldmem_data.start)
> +		reserve_crashkernel_cma(cma_size);
>  #endif
>  }

[Severity: Medium]
Can reserve_crashkernel_cma() allocate CMA ranges inside [0, low),
overlapping the [0, crash_size) memory region that purgatory swaps with
[crash_base, crash_base + crash_size) on kdump (or the [0, sclp.hsa_size)
HSA region)?

Earlier in reserve_crashkernel(), crash_base is constrained to sit at or
above low:

arch/s390/kernel/setup.c:reserve_crashkernel() {
	...
		/* Find suitable area in free memory */
		low = max_t(unsigned long, crash_size, sclp.hsa_size);
		high = crash_base ? crash_base + crash_size : ULONG_MAX;
	...
		low = crash_base ?: low;
		crash_base = memblock_phys_alloc_range(crash_size,
						       KEXEC_CRASH_MEM_ALIGN,
						       low, high);
}

However, [0, low) is not reserved in memblock when
reserve_crashkernel_cma() runs, and reserve_crashkernel_cma() calls
cma_declare_contiguous() with base = 0.  When free memory above 4 GB (or
above low) is insufficient for cma_size, cma_alloc_mem() allocates from
[0, low).  In the kdump kernel, any CMA range inside [0, crash_size)
merges with the existing [0, oldmem_data.size) primary memory range
instead of providing the requested extra memory, and a CMA range
straddling crash_size corrupts the swapped memory layout.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261112112105.1169118-1-zaslonko@linux.ibm.com?part=2

[PATCH v1 3/8] s390/kexec: Pass crashkernel CMA ranges to the kdump kernel

[Mikhail Zaslonko]

Store the crashkernel CMA ranges in the parmarea of the kdump kernel,
next to oldmem_base and oldmem_size, when it is loaded with
kexec_file_load().

The ranges use 64 of the 72 bytes of pad1, which is enough for the four
ranges the generic code reserves at most. They are sorted by address
and unused entries are zero. Older kernels ignore pad1 and kernel images
have it zeroed, so old and new kernels still work together.

For kexec_load(), kexec-tools would have to fill in the ranges. It can
read them from /sys/kernel/kexec_crash_cma_ranges.

Co-developed-by: Vasily Gorbik <gor@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
Signed-off-by: Mikhail Zaslonko <zaslonko@linux.ibm.com>
---
 arch/s390/include/asm/setup.h         | 9 ++++++++-
 arch/s390/kernel/machine_kexec_file.c | 5 +++++
 arch/s390/kernel/setup.c              | 5 ++++-
 3 files changed, 17 insertions(+), 2 deletions(-)

diff --git a/arch/s390/include/asm/setup.h b/arch/s390/include/asm/setup.h
index cbf60ade741d..7e992f3953b1 100644
--- a/arch/s390/include/asm/setup.h
+++ b/arch/s390/include/asm/setup.h
@@ -23,12 +23,18 @@
 #define STARTUP_KDUMP_OFFSET	0x10010
 
 #define LEGACY_COMMAND_LINE_SIZE	896
+#define CRASHK_CMA_ENTRIES	4
 
 #ifndef __ASSEMBLER__
 
 #include <asm/lowcore.h>
 #include <asm/types.h>
 
+struct crashk_cma_range {
+	unsigned long start;
+	unsigned long size;
+};
+
 struct parmarea {
 	unsigned long ipl_device;			/* 0x10400 */
 	unsigned long initrd_start;			/* 0x10408 */
@@ -37,7 +43,8 @@ struct parmarea {
 	unsigned long oldmem_size;			/* 0x10420 */
 	unsigned long kernel_version;			/* 0x10428 */
 	unsigned long max_command_line_size;		/* 0x10430 */
-	char pad1[0x10480-0x10438];			/* 0x10438 - 0x10480 */
+	struct crashk_cma_range crashk_cma[CRASHK_CMA_ENTRIES]; /* 0x10438 - 0x10478 */
+	char pad1[0x10480-0x10478];			/* 0x10478 - 0x10480 */
 	char command_line[COMMAND_LINE_SIZE];		/* 0x10480 */
 };
 
diff --git a/arch/s390/kernel/machine_kexec_file.c b/arch/s390/kernel/machine_kexec_file.c
index 6f0852d5a3a9..198e0d736aa0 100644
--- a/arch/s390/kernel/machine_kexec_file.c
+++ b/arch/s390/kernel/machine_kexec_file.c
@@ -282,6 +282,11 @@ void *kexec_file_add_components(struct kimage *image,
 	if (image->type == KEXEC_TYPE_CRASH) {
 		data.parm->oldmem_base = crashk_res.start;
 		data.parm->oldmem_size = crashk_res.end - crashk_res.start + 1;
+		BUILD_BUG_ON(ARRAY_SIZE(data.parm->crashk_cma) < CRASHK_CMA_RANGES_MAX);
+		for (int i = 0; i < crashk_cma_cnt; i++) {
+			data.parm->crashk_cma[i].start = crashk_cma_ranges[i].start;
+			data.parm->crashk_cma[i].size = range_len(&crashk_cma_ranges[i]);
+		}
 	}
 #endif
 
diff --git a/arch/s390/kernel/setup.c b/arch/s390/kernel/setup.c
index 0251bd35954a..3c1c66e6c281 100644
--- a/arch/s390/kernel/setup.c
+++ b/arch/s390/kernel/setup.c
@@ -663,8 +663,11 @@ static void __init reserve_crashkernel(void)
 	 * A kdump kernel is handed the CMA ranges of the crashed kernel
 	 * instead of reserving its own.
 	 */
-	if (!oldmem_data.start)
+	if (!oldmem_data.start) {
 		reserve_crashkernel_cma(cma_size);
+		/* the ranges are handed over in ascending order */
+		sort_range(crashk_cma_ranges, crashk_cma_cnt);
+	}
 #endif
 }
 
-- 
2.55.0

Re: [PATCH v1 3/8] s390/kexec: Pass crashkernel CMA ranges to the kdump kernel

[sashiko-bot]

> Store the crashkernel CMA ranges in the parmarea of the kdump kernel,
> next to oldmem_base and oldmem_size, when it is loaded with
> kexec_file_load().
> 
> The ranges use 64 of the 72 bytes of pad1, which is enough for the four
> ranges the generic code reserves at most. They are sorted by address
> and unused entries are zero. Older kernels ignore pad1 and kernel images
> have it zeroed, so old and new kernels still work together.
> 
> For kexec_load(), kexec-tools would have to fill in the ranges. It can
> read them from /sys/kernel/kexec_crash_cma_ranges.
> 
> Co-developed-by: Vasily Gorbik <gor@linux.ibm.com>
> Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
> Signed-off-by: Mikhail Zaslonko <zaslonko@linux.ibm.com>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261112112105.1169118-1-zaslonko@linux.ibm.com?part=3

[PATCH v1 4/8] s390/boot: Allow additional usable memory ranges

[Mikhail Zaslonko]

All online memory above the usable limit is left unused. A kdump kernel
will need a few ranges above that limit for the crashkernel CMA memory
of the crashed kernel.

Add physmem_add_usable_range() to register usable memory ranges. Clip
them to ident_map_size and merge overlapping or adjacent ranges.
for_each_physmem_usable_range() walks their intersection with the
online memory ranges.

Use the same ranges for boot allocations, identity mappings, KASAN
shadow and memblock. Keep the allocation position separate from the
usable ranges.

When no extra ranges are added, nothing changes.

Co-developed-by: Vasily Gorbik <gor@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
Signed-off-by: Mikhail Zaslonko <zaslonko@linux.ibm.com>
---
 arch/s390/boot/boot.h                |  3 +-
 arch/s390/boot/kaslr.c               |  4 +-
 arch/s390/boot/physmem_info.c        | 65 ++++++++++++++++++++++++----
 arch/s390/boot/startup.c             |  3 +-
 arch/s390/boot/vmem.c                |  7 +--
 arch/s390/include/asm/physmem_info.h | 63 ++++++++++++++++++++-------
 arch/s390/kernel/setup.c             |  3 +-
 7 files changed, 116 insertions(+), 32 deletions(-)

diff --git a/arch/s390/boot/boot.h b/arch/s390/boot/boot.h
index 61a205b489fb..39d5fc520c0c 100644
--- a/arch/s390/boot/boot.h
+++ b/arch/s390/boot/boot.h
@@ -44,7 +44,8 @@ struct vmlinux_info {
 void startup_kernel(void);
 unsigned long detect_max_physmem_end(void);
 void detect_physmem_online_ranges(unsigned long max_physmem_end);
-void physmem_set_usable_limit(unsigned long limit);
+void physmem_set_alloc_limit(unsigned long limit);
+void physmem_add_usable_range(unsigned long start, unsigned long end);
 void physmem_reserve(enum reserved_range_type type, unsigned long addr, unsigned long size);
 void physmem_free(enum reserved_range_type type);
 /* for continuous/multiple allocations per type */
diff --git a/arch/s390/boot/kaslr.c b/arch/s390/boot/kaslr.c
index 941f4c9e27cc..f2de79a69031 100644
--- a/arch/s390/boot/kaslr.c
+++ b/arch/s390/boot/kaslr.c
@@ -112,11 +112,11 @@ static unsigned long iterate_valid_positions(unsigned long size, unsigned long a
 	unsigned long start, end, tmp_end, range_pos, pos = 0;
 	struct reserved_range *res_end = res + res_count;
 	struct reserved_range *skip_res;
-	int i;
+	struct physmem_iter iter;
 
 	align = max(align, 8UL);
 	_min = round_up(_min, align);
-	for_each_physmem_usable_range(i, &start, &end) {
+	for_each_physmem_usable_range(iter, &start, &end) {
 		if (_min >= end)
 			continue;
 		start = round_up(start, align);
diff --git a/arch/s390/boot/physmem_info.c b/arch/s390/boot/physmem_info.c
index e5248b637326..c9888d1669e3 100644
--- a/arch/s390/boot/physmem_info.c
+++ b/arch/s390/boot/physmem_info.c
@@ -195,19 +195,44 @@ void detect_physmem_online_ranges(unsigned long max_physmem_end)
 		boot_debug(" online [%d]:   0x%016lx-0x%016lx\n", i, start, end);
 }
 
-void physmem_set_usable_limit(unsigned long limit)
+void physmem_set_alloc_limit(unsigned long limit)
 {
-	physmem_info.usable = limit;
 	physmem_alloc_pos = limit;
-	boot_debug("Usable memory limit: 0x%016lx\n", limit);
+	boot_debug("Upper alloc limit: 0x%016lx\n", limit);
+}
+
+/*
+ * Add usable memory ranges, merging overlapping or adjacent ranges.
+ * The kdump code passes CMA ranges sorted by start address.
+ */
+void physmem_add_usable_range(unsigned long start, unsigned long end)
+{
+	struct physmem_range *range;
+	u32 n = physmem_info.usable_count;
+
+	boot_debug("Usable memory range: 0x%016lx-0x%016lx\n", start, end);
+	end = min(end, ident_map_size);
+	if (start >= end)
+		return;
+	if (n) {
+		range = &physmem_info.usable[n - 1];
+		if (start <= range->end) {
+			range->end = max_t(u64, range->end, end);
+			return;
+		}
+	}
+	range = &physmem_info.usable[n];
+	range->start = start;
+	range->end = end;
+	physmem_info.usable_count++;
 }
 
 static void die_oom(unsigned long size, unsigned long align, unsigned long min, unsigned long max)
 {
 	unsigned long start, end, total_mem = 0, total_reserved_mem = 0;
 	struct reserved_range *range;
+	struct physmem_iter iter;
 	enum reserved_range_type t;
-	int i;
 
 	boot_emerg("Linux version %s\n", kernel_version);
 	if (!is_prot_virt_guest() && early_command_line[0])
@@ -221,7 +246,7 @@ static void die_oom(unsigned long size, unsigned long align, unsigned long min,
 	}
 	boot_emerg("Usable online memory ranges (info source: %s [%d]):\n",
 		   get_physmem_info_source(), physmem_info.info_source);
-	for_each_physmem_usable_range(i, &start, &end) {
+	for_each_physmem_usable_range(iter, &start, &end) {
 		boot_emerg("%016lx %016lx\n", start, end);
 		total_mem += end - start;
 	}
@@ -269,6 +294,26 @@ static bool __physmem_alloc_intersects(unsigned long addr, unsigned long size,
 	return ipl_report_certs_intersects(addr, size, intersection_start);
 }
 
+/* Find the highest usable part of an online range below max. */
+static int __get_physmem_alloc_range(u32 n, unsigned long max,
+				     unsigned long *start, unsigned long *end)
+{
+	unsigned long online_start, online_end;
+	struct physmem_range *range;
+	u32 i = physmem_info.usable_count;
+
+	__get_physmem_online_range(n, &online_start, &online_end);
+	online_end = min(online_end, max);
+	while (i) {
+		range = &physmem_info.usable[--i];
+		*start = max_t(unsigned long, online_start, range->start);
+		*end = min_t(unsigned long, online_end, range->end);
+		if (*start < *end)
+			return 0;
+	}
+	return -1;
+}
+
 static unsigned long __physmem_alloc_range(unsigned long size, unsigned long align,
 					   unsigned long min, unsigned long max,
 					   unsigned int from_ranges, unsigned int *ranges_left,
@@ -281,14 +326,18 @@ static unsigned long __physmem_alloc_range(unsigned long size, unsigned long ali
 
 	align = max(align, 8UL);
 	while (nranges) {
-		__get_physmem_range(nranges - 1, &range_start, &range_end, false);
-		pos = min(range_end, pos);
+		if (__get_physmem_alloc_range(nranges - 1, pos, &range_start, &range_end)) {
+			nranges--;
+			continue;
+		}
+		pos = range_end;
 
 		if (round_up(min, align) + size > pos)
 			break;
 		addr = round_down(pos - size, align);
 		if (range_start > addr) {
-			nranges--;
+			/* A lower usable range may fit within the same online range. */
+			pos = range_start;
 			continue;
 		}
 		if (__physmem_alloc_intersects(addr, size, &intersection_start)) {
diff --git a/arch/s390/boot/startup.c b/arch/s390/boot/startup.c
index 5eb7a3e1f9f1..95543656d1b8 100644
--- a/arch/s390/boot/startup.c
+++ b/arch/s390/boot/startup.c
@@ -545,7 +545,8 @@ void startup_kernel(void)
 	setup_vmalloc_size();
 	asce_limit = setup_kernel_memory_layout(kernel_size);
 	/* got final ident_map_size, physmem allocations could be performed now */
-	physmem_set_usable_limit(ident_map_size);
+	physmem_add_usable_range(0, ident_map_size);
+	physmem_set_alloc_limit(ident_map_size);
 	detect_physmem_online_ranges(max_physmem_end);
 	save_ipl_cert_comp_list();
 	rescue_initrd(safe_addr, ident_map_size);
diff --git a/arch/s390/boot/vmem.c b/arch/s390/boot/vmem.c
index ff6d58a476ba..a4fb8254f156 100644
--- a/arch/s390/boot/vmem.c
+++ b/arch/s390/boot/vmem.c
@@ -99,8 +99,8 @@ static void kasan_populate_shadow(unsigned long kernel_start, unsigned long kern
 	pud_t pud_z = __pud(__pa(kasan_early_shadow_pmd) | _REGION3_ENTRY);
 	p4d_t p4d_z = __p4d(__pa(kasan_early_shadow_pud) | _REGION2_ENTRY);
 	unsigned long memgap_start = 0;
+	struct physmem_iter iter;
 	unsigned long start, end;
-	int i;
 
 	pte_z = __pte(__pa(kasan_early_shadow_page) | pgprot_val(PAGE_KERNEL_RO));
 	crst_table_init((unsigned long *)kasan_early_shadow_p4d, p4d_val(p4d_z));
@@ -112,7 +112,7 @@ static void kasan_populate_shadow(unsigned long kernel_start, unsigned long kern
 	__arch_set_page_dat(kasan_early_shadow_pmd, 1UL << CRST_ALLOC_ORDER);
 	__arch_set_page_dat(kasan_early_shadow_pte, 1);
 
-	for_each_physmem_usable_range(i, &start, &end) {
+	for_each_physmem_usable_range(iter, &start, &end) {
 		kasan_populate((unsigned long)__identity_va(start),
 			       (unsigned long)__identity_va(end),
 			       POPULATE_KASAN_MAP_SHADOW);
@@ -473,6 +473,7 @@ void setup_vmem(unsigned long kernel_start, unsigned long kernel_end, unsigned l
 	unsigned long start, end;
 	unsigned long asce_type;
 	unsigned long asce_bits;
+	struct physmem_iter iter;
 	pgd_t *init_mm_pgd;
 	int i;
 
@@ -518,7 +519,7 @@ void setup_vmem(unsigned long kernel_start, unsigned long kernel_end, unsigned l
 	pgtable_populate(lowcore_address,
 			 lowcore_address + sizeof(struct lowcore),
 			 POPULATE_LOWCORE);
-	for_each_physmem_usable_range(i, &start, &end) {
+	for_each_physmem_usable_range(iter, &start, &end) {
 		/* Do not map lowcore with identity mapping */
 		if (!start)
 			start = sizeof(struct lowcore);
diff --git a/arch/s390/include/asm/physmem_info.h b/arch/s390/include/asm/physmem_info.h
index 7ef3bbec98b0..c6f9f2f5d26b 100644
--- a/arch/s390/include/asm/physmem_info.h
+++ b/arch/s390/include/asm/physmem_info.h
@@ -2,8 +2,10 @@
 #ifndef _ASM_S390_MEM_DETECT_H
 #define _ASM_S390_MEM_DETECT_H
 
+#include <linux/minmax.h>
 #include <linux/types.h>
 #include <asm/page.h>
+#include <asm/setup.h>
 
 enum physmem_info_source {
 	MEM_DETECT_NONE = 0,
@@ -47,21 +49,26 @@ struct reserved_range {
  */
 #define MEM_INLINED_ENTRIES 255 /* (PAGE_SIZE - 16) / 16 */
 
+/* Primary memory and the handed-over crashkernel CMA ranges. */
+#define MEM_USABLE_ENTRIES (1 + CRASHK_CMA_ENTRIES)
+
 struct physmem_info {
 	u32 range_count;
 	u8 info_source;
-	unsigned long usable;
 	struct reserved_range reserved[RR_MAX];
 	struct physmem_range online[MEM_INLINED_ENTRIES];
 	struct physmem_range *online_extended;
+	/* Sorted usable ranges, intersected with online memory when iterating. */
+	u32 usable_count;
+	struct physmem_range usable[MEM_USABLE_ENTRIES];
 };
 
 extern struct physmem_info physmem_info;
 
 void add_physmem_online_range(u64 start, u64 end);
 
-static inline int __get_physmem_range(u32 n, unsigned long *start,
-				      unsigned long *end, bool respect_usable_limit)
+static inline int __get_physmem_online_range(u32 n, unsigned long *start,
+					     unsigned long *end)
 {
 	if (n >= physmem_info.range_count) {
 		*start = 0;
@@ -77,29 +84,53 @@ static inline int __get_physmem_range(u32 n, unsigned long *start,
 		*end = (unsigned long)physmem_info.online_extended[n - MEM_INLINED_ENTRIES].end;
 	}
 
-	if (respect_usable_limit && physmem_info.usable) {
-		if (*start >= physmem_info.usable)
-			return -1;
-		if (*end > physmem_info.usable)
-			*end = physmem_info.usable;
-	}
 	return 0;
 }
 
+struct physmem_iter {
+	u32 online;
+	u32 usable;
+};
+
+static inline int __next_physmem_usable_range(struct physmem_iter *iter,
+					      unsigned long *start, unsigned long *end)
+{
+	unsigned long online_start, online_end, usable_start, usable_end;
+
+	while (iter->usable < physmem_info.usable_count) {
+		if (__get_physmem_online_range(iter->online, &online_start, &online_end))
+			break;
+		usable_start = physmem_info.usable[iter->usable].start;
+		usable_end = physmem_info.usable[iter->usable].end;
+		if (online_end <= usable_end)
+			iter->online++;
+		if (usable_end <= online_end)
+			iter->usable++;
+		*start = max(online_start, usable_start);
+		*end = min(online_end, usable_end);
+		if (*start < *end)
+			return 0;
+	}
+	*start = 0;
+	*end = 0;
+	return -1;
+}
+
 /**
- * for_each_physmem_usable_range - early online memory range iterator
- * @i: an integer used as loop variable
+ * for_each_physmem_usable_range - early usable memory range iterator
+ * @iter: a struct physmem_iter used as loop cursor
  * @p_start: ptr to unsigned long for start address of the range
  * @p_end: ptr to unsigned long for end address of the range
  *
- * Walks over detected online memory ranges below usable limit.
+ * Walks over the intersection of detected online memory and usable ranges.
  */
-#define for_each_physmem_usable_range(i, p_start, p_end)		\
-	for (i = 0; !__get_physmem_range(i, p_start, p_end, true); i++)
+#define for_each_physmem_usable_range(iter, p_start, p_end)		\
+	for (iter = (struct physmem_iter) { 0 };				\
+	     !__next_physmem_usable_range(&iter, p_start, p_end);)
 
-/* Walks over all detected online memory ranges disregarding usable limit. */
+/* Walks over all detected online memory ranges. */
 #define for_each_physmem_online_range(i, p_start, p_end)		\
-	for (i = 0; !__get_physmem_range(i, p_start, p_end, false); i++)
+	for (i = 0; !__get_physmem_online_range(i, p_start, p_end); i++)
 
 static inline const char *get_physmem_info_source(void)
 {
diff --git a/arch/s390/kernel/setup.c b/arch/s390/kernel/setup.c
index 3c1c66e6c281..9ec9dfc06761 100644
--- a/arch/s390/kernel/setup.c
+++ b/arch/s390/kernel/setup.c
@@ -712,6 +712,7 @@ static void __init free_physmem_info(void)
 
 static void __init memblock_add_physmem_info(void)
 {
+	struct physmem_iter iter;
 	unsigned long start, end;
 	int i;
 
@@ -719,7 +720,7 @@ static void __init memblock_add_physmem_info(void)
 		 get_physmem_info_source(), physmem_info.info_source);
 	/* keep memblock lists close to the kernel */
 	memblock_set_bottom_up(true);
-	for_each_physmem_usable_range(i, &start, &end)
+	for_each_physmem_usable_range(iter, &start, &end)
 		memblock_add(start, end - start);
 	for_each_physmem_online_range(i, &start, &end)
 		memblock_physmem_add(start, end - start);
-- 
2.55.0

Re: [PATCH v1 4/8] s390/boot: Allow additional usable memory ranges

[sashiko-bot]

> All online memory above the usable limit is left unused. A kdump kernel
> will need a few ranges above that limit for the crashkernel CMA memory
> of the crashed kernel.
> 
> Add physmem_add_usable_range() to register usable memory ranges. Clip
> them to ident_map_size and merge overlapping or adjacent ranges.
> for_each_physmem_usable_range() walks their intersection with the
> online memory ranges.
> 
> Use the same ranges for boot allocations, identity mappings, KASAN
> shadow and memblock. Keep the allocation position separate from the
> usable ranges.
> 
> When no extra ranges are added, nothing changes.
> 
> Co-developed-by: Vasily Gorbik <gor@linux.ibm.com>
> Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
> Signed-off-by: Mikhail Zaslonko <zaslonko@linux.ibm.com>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261112112105.1169118-1-zaslonko@linux.ibm.com?part=4

[PATCH v1 5/8] s390/kdump: Exclude crashkernel CMA memory from vmcore

[Mikhail Zaslonko]

The kdump kernel will be using the crashkernel CMA memory of the
crashed kernel, so it must not be a part of the dump.

Keep the CMA ranges handed over by the crashed kernel in oldmem_data.
Add them to oldmem_type next to the crashkernel memory, so that no
PT_LOAD headers are created for them.

Merge adjacent regions, such as CMA areas placed back to back or next
to the crashkernel memory. Otherwise for_each_physmem_range() returns an
empty range between them, which ends up as an empty PT_LOAD header.

Use the original ranges even if mem= limits usable memory in the kdump
kernel. Build the list without depending on CONFIG_CMA in that kernel.

Co-developed-by: Vasily Gorbik <gor@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
Signed-off-by: Mikhail Zaslonko <zaslonko@linux.ibm.com>
---
 arch/s390/boot/startup.c      |  1 +
 arch/s390/include/asm/setup.h |  1 +
 arch/s390/kernel/crash_dump.c | 48 ++++++++++++++++++++++++++++-------
 3 files changed, 41 insertions(+), 9 deletions(-)

diff --git a/arch/s390/boot/startup.c b/arch/s390/boot/startup.c
index 95543656d1b8..111b146a3332 100644
--- a/arch/s390/boot/startup.c
+++ b/arch/s390/boot/startup.c
@@ -529,6 +529,7 @@ void startup_kernel(void)
 		physmem_reserve(RR_INITRD, parmarea.initrd_start, parmarea.initrd_size);
 	oldmem_data.start = parmarea.oldmem_base;
 	oldmem_data.size = parmarea.oldmem_size;
+	memcpy(oldmem_data.crashk_cma, parmarea.crashk_cma, sizeof(oldmem_data.crashk_cma));
 
 	read_ipl_report();
 	sclp_early_read_info();
diff --git a/arch/s390/include/asm/setup.h b/arch/s390/include/asm/setup.h
index 7e992f3953b1..7048985ec014 100644
--- a/arch/s390/include/asm/setup.h
+++ b/arch/s390/include/asm/setup.h
@@ -100,6 +100,7 @@ extern void (*_machine_power_off)(void);
 struct oldmem_data {
 	unsigned long start;
 	unsigned long size;
+	struct crashk_cma_range crashk_cma[CRASHK_CMA_ENTRIES];
 };
 extern struct oldmem_data oldmem_data;
 
diff --git a/arch/s390/kernel/crash_dump.c b/arch/s390/kernel/crash_dump.c
index d4839de8ce9d..f2f4a5f40d60 100644
--- a/arch/s390/kernel/crash_dump.c
+++ b/arch/s390/kernel/crash_dump.c
@@ -19,6 +19,7 @@
 #include <linux/uio.h>
 #include <asm/asm-offsets.h>
 #include <asm/os_info.h>
+#include <asm/setup.h>
 #include <asm/elf.h>
 #include <asm/ipl.h>
 #include <asm/sclp.h>
@@ -29,13 +30,14 @@
 #define PTR_SUB(x, y) (((char *) (x)) - ((unsigned long) (y)))
 #define PTR_DIFF(x, y) ((unsigned long)(((char *) (x)) - ((unsigned long) (y))))
 
-static struct memblock_region oldmem_region;
+/* crashkernel memory and crashkernel CMA memory, sorted by address */
+static struct memblock_region oldmem_regions[1 + CRASHK_CMA_ENTRIES];
 
 static struct memblock_type oldmem_type = {
 	.cnt = 1,
-	.max = 1,
+	.max = ARRAY_SIZE(oldmem_regions),
 	.total_size = 0,
-	.regions = &oldmem_region,
+	.regions = oldmem_regions,
 	.name = "oldmem",
 };
 
@@ -478,6 +480,37 @@ static int get_cpu_cnt(void)
 	return cpus;
 }
 
+/* Exclude the original crashkernel and CMA reservations, independent of mem=. */
+static void oldmem_type_init(void)
+{
+	struct memblock_region *regions = oldmem_regions;
+	unsigned long base, size;
+	int i, j;
+
+	regions[0].base = oldmem_data.start;
+	regions[0].size = oldmem_data.size;
+	oldmem_type.cnt = 1;
+	oldmem_type.total_size = oldmem_data.size;
+	for (i = 0; i < ARRAY_SIZE(oldmem_data.crashk_cma) && oldmem_data.crashk_cma[i].size; i++) {
+		base = oldmem_data.crashk_cma[i].start;
+		size = oldmem_data.crashk_cma[i].size;
+		for (j = oldmem_type.cnt; j > 0 && regions[j - 1].base > base; j--)
+			regions[j] = regions[j - 1];
+		regions[j].base = base;
+		regions[j].size = size;
+		oldmem_type.cnt++;
+		oldmem_type.total_size += size;
+	}
+	/* memblock iterators expect separate regions, merge adjacent ones */
+	for (i = 1, j = 0; i < oldmem_type.cnt; i++) {
+		if (regions[j].base + regions[j].size == regions[i].base)
+			regions[j].size += regions[i].size;
+		else
+			regions[++j] = regions[i];
+	}
+	oldmem_type.cnt = j + 1;
+}
+
 /*
  * Return memory chunk count for ELF header (new kernel)
  */
@@ -626,12 +659,9 @@ int elfcorehdr_alloc(unsigned long long *addr, unsigned long long *size)
 	if (is_ipl_type_dump() && !sclp.hsa_size)
 		return -ENODEV;
 
-	/* For kdump, exclude previous crashkernel memory */
-	if (oldmem_data.start) {
-		oldmem_region.base = oldmem_data.start;
-		oldmem_region.size = oldmem_data.size;
-		oldmem_type.total_size = oldmem_data.size;
-	}
+	/* For kdump, exclude memory used by the kdump kernel */
+	if (oldmem_data.start)
+		oldmem_type_init();
 
 	mem_chunk_cnt = get_mem_chunk_cnt();
 	phdr_text_cnt = os_info_has_vm() ? 1 : 0;
-- 
2.55.0

Re: [PATCH v1 5/8] s390/kdump: Exclude crashkernel CMA memory from vmcore

[sashiko-bot]

> The kdump kernel will be using the crashkernel CMA memory of the
> crashed kernel, so it must not be a part of the dump.
> 
> Keep the CMA ranges handed over by the crashed kernel in oldmem_data.
> Add them to oldmem_type next to the crashkernel memory, so that no
> PT_LOAD headers are created for them.
> 
> Merge adjacent regions, such as CMA areas placed back to back or next
> to the crashkernel memory. Otherwise for_each_physmem_range() returns an
> empty range between them, which ends up as an empty PT_LOAD header.
> 
> Use the original ranges even if mem= limits usable memory in the kdump
> kernel. Build the list without depending on CONFIG_CMA in that kernel.
> 
> Co-developed-by: Vasily Gorbik <gor@linux.ibm.com>
> Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
> Signed-off-by: Mikhail Zaslonko <zaslonko@linux.ibm.com>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261112112105.1169118-1-zaslonko@linux.ibm.com?part=5

[PATCH v1 6/8] s390/kdump: Use crashkernel CMA memory in the kdump kernel

[Mikhail Zaslonko]

A kdump kernel limits ident_map_size to the crashkernel size, so it
can't use the crashkernel CMA memory above it.

Don't limit ident_map_size when CMA ranges are passed. Instead, add the
crashkernel memory and the CMA ranges as usable memory ranges. All
other memory is treated as a memory hole.

The CMA memory is then mapped, added to memblock and freed to the buddy
allocator like any other memory. As the boot allocator starts at
ident_map_size, decompressor allocations such as page tables, the KASAN
shadow and the initrd moved by rescue_initrd() come from the CMA memory
first.

Like oldmem_base and oldmem_size, the ranges are used as passed by the
crashed kernel.

Co-developed-by: Vasily Gorbik <gor@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
Signed-off-by: Mikhail Zaslonko <zaslonko@linux.ibm.com>
---
 arch/s390/boot/startup.c | 28 +++++++++++++++++++++++-----
 1 file changed, 23 insertions(+), 5 deletions(-)

diff --git a/arch/s390/boot/startup.c b/arch/s390/boot/startup.c
index 111b146a3332..dba1dfc0ee49 100644
--- a/arch/s390/boot/startup.c
+++ b/arch/s390/boot/startup.c
@@ -197,6 +197,22 @@ static void deploy_kernel(void *output)
 }
 #endif
 
+static void setup_usable_memory(void)
+{
+	unsigned long start;
+	int i;
+
+	if (!IS_ENABLED(CONFIG_CRASH_DUMP) || !oldmem_data.start) {
+		physmem_add_usable_range(0, ident_map_size);
+		return;
+	}
+	physmem_add_usable_range(0, oldmem_data.size);
+	for (i = 0; i < ARRAY_SIZE(oldmem_data.crashk_cma) && oldmem_data.crashk_cma[i].size; i++) {
+		start = oldmem_data.crashk_cma[i].start;
+		physmem_add_usable_range(start, start + oldmem_data.crashk_cma[i].size);
+	}
+}
+
 static void rescue_initrd(unsigned long min, unsigned long max)
 {
 	unsigned long old_addr, addr, size;
@@ -270,8 +286,9 @@ static void kaslr_adjust_got(unsigned long offset)
  * 2. CONFIG_MAX_PHYSMEM_BITS - the maximum size of physical memory the
  *    kernel is able to support.
  * 3. "mem=" kernel command line option which limits physical memory usage.
- * 4. OLDMEM_BASE which is a kdump memory limit when the kernel is executed as
- *    crash kernel.
+ * 4. OLDMEM_SIZE limits the identity map to primary crashkernel memory
+ *    when OLDMEM_BASE is set and no crashkernel CMA ranges are passed.
+ *    With CMA ranges, setup_usable_memory() selects the usable ranges.
  * 5. "hsa" size which is a memory limit when the kernel is executed during
  *    zfcp/nvme dump.
  */
@@ -287,7 +304,9 @@ static void setup_ident_map_size(unsigned long max_physmem_end)
 #ifdef CONFIG_CRASH_DUMP
 	if (oldmem_data.start) {
 		__kaslr_enabled = 0;
-		ident_map_size = min(ident_map_size, oldmem_data.size);
+		/* no CMA ranges handed over to the crashkernel */
+		if (!oldmem_data.crashk_cma[0].size)
+			ident_map_size = min(ident_map_size, oldmem_data.size);
 		boot_debug("kdump memory limit:  0x%016lx\n", oldmem_data.size);
 	} else if (ipl_block_valid && is_ipl_block_dump()) {
 		__kaslr_enabled = 0;
@@ -545,8 +564,7 @@ void startup_kernel(void)
 	setup_ident_map_size(max_physmem_end);
 	setup_vmalloc_size();
 	asce_limit = setup_kernel_memory_layout(kernel_size);
-	/* got final ident_map_size, physmem allocations could be performed now */
-	physmem_add_usable_range(0, ident_map_size);
+	setup_usable_memory();
 	physmem_set_alloc_limit(ident_map_size);
 	detect_physmem_online_ranges(max_physmem_end);
 	save_ipl_cert_comp_list();
-- 
2.55.0

Re: [PATCH v1 6/8] s390/kdump: Use crashkernel CMA memory in the kdump kernel

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 5 potential
issues to consider:
- [Medium] s390/kdump: unmapped hole aborts __set_memory_4k() in
  vmem_map_init()
- [Medium] s390/kdump: uncapped ident_map_size and nokaslr truncate CMA
  ranges
- [Medium] s390/kdump: setup_usable_memory() loses CMA ranges below
  oldmem_data.size
- [Medium] s390/kdump: uncapped ident_map_size lets
  reserve_crashkernel() run in kdump
- [Low] s390/kdump: missing KASAN zero shadow in kasan_populate_shadow()
  holes
--

commit 9e7830ee8f173942437ec5cda601c7a5e6920ddc
Author: Mikhail Zaslonko <zaslonko@linux.ibm.com>

s390/kdump: Use crashkernel CMA memory in the kdump kernel

This commit stops capping ident_map_size to oldmem_data.size when crashkernel
CMA ranges are present and instead registers [0, oldmem_data.size) and the
CMA ranges as usable physical memory ranges in setup_usable_memory().

> diff --git a/arch/s390/boot/startup.c b/arch/s390/boot/startup.c
> index 111b146a3332f..dba1dfc0ee499 100644
> --- a/arch/s390/boot/startup.c
> +++ b/arch/s390/boot/startup.c
> @@ -197,6 +197,22 @@ static void deploy_kernel(void *output)
>  }
>  #endif
>  
> +static void setup_usable_memory(void)
> +{
> +	unsigned long start;
> +	int i;
> +
> +	if (!IS_ENABLED(CONFIG_CRASH_DUMP) || !oldmem_data.start) {
> +		physmem_add_usable_range(0, ident_map_size);
> +		return;
> +	}
> +	physmem_add_usable_range(0, oldmem_data.size);
> +	for (i = 0; i < ARRAY_SIZE(oldmem_data.crashk_cma) && oldmem_data.crashk_cma[i].size; i++) {
> +		start = oldmem_data.crashk_cma[i].start;
> +		physmem_add_usable_range(start, start + oldmem_data.crashk_cma[i].size);
> +	}
> +}

[Severity: Medium]
Can registering discontiguous usable ranges here leave crashk_cma mapped with
large pages when debug_pagealloc is enabled?

In setup_vmem(), pgtable_populate() only populates identity mappings for
for_each_physmem_usable_range(), using 1MB or 2GB large pages while leaving
the hole starting at __va(oldmem_data.size) unmapped.  Later, vmem_map_init()
calls __set_memory_4k(__va(0), absolute_pointer(__va(0)) + ident_map_size),
which walks the range linearly and aborts with -EINVAL in walk_pmd_level()
at the first unmapped entry at __va(oldmem_data.size):

arch/s390/mm/pageattr.c:walk_pmd_level() {
	...
		pmd = pmdp_get(pmdp);
		if (pmd_none(pmd))
			return -EINVAL;
	...
}

As a result, the large-page mappings for the CMA ranges above the hole are
never split into 4KB PTEs.  When mem_init() frees those CMA pages to the
buddy allocator, __kernel_map_pages() calls virt_to_kpte() on leaf PMD/PUD
entries and passes the physical CMA data frames to ipte_range() or set_pte(),
corrupting memory or triggering a program check during kdump boot.

[Severity: Medium]
What happens if a crashk_cma range was allocated below oldmem_data.size in
the primary kernel?

reserve_crashkernel() calls reserve_crashkernel_cma() with a base of 0, so
cma_declare_contiguous() can place CMA ranges below oldmem_data.size (for
example, above 4GB when crash_size > 4GB, or via the top-down fallback when
memory above 4GB is full).  When kdump triggers, purgatory swaps physical
[0, oldmem_data.size) with [oldmem_data.start, oldmem_data.start +
oldmem_data.size), moving any CMA range in that region into the
oldmem_data.start area.

Because setup_usable_memory() passes the unswapped crashk_cma[i].start to
physmem_add_usable_range(), any CMA range with start <= oldmem_data.size is
merged into the [0, oldmem_data.size) range instead of adding the swapped
frames inside [oldmem_data.start, oldmem_data.start + oldmem_data.size),
losing that CMA memory in the kdump kernel.

[ ... ]

> @@ -287,7 +304,9 @@ static void setup_ident_map_size(unsigned long max_physmem_end)
>  #ifdef CONFIG_CRASH_DUMP
>  	if (oldmem_data.start) {
>  		__kaslr_enabled = 0;
> -		ident_map_size = min(ident_map_size, oldmem_data.size);
> +		/* no CMA ranges handed over to the crashkernel */
> +		if (!oldmem_data.crashk_cma[0].size)
> +			ident_map_size = min(ident_map_size, oldmem_data.size);
>  		boot_debug("kdump memory limit:  0x%016lx\n", oldmem_data.size);
>  	} else if (ipl_block_valid && is_ipl_block_dump()) {

[Severity: Medium]
Can leaving ident_map_size uncapped at max_physmem_end while forcing
__kaslr_enabled = 0 cause crashk_cma ranges to be truncated on large-memory
systems?

In setup_kernel_memory_layout(), a non-KASAN kernel with default
CONFIG_KERNEL_IMAGE_BASE only selects 4-level paging (_REGION1_SIZE) when
vsize > _REGION2_SIZE && kaslr_enabled():

arch/s390/boot/startup.c:setup_kernel_memory_layout() {
	...
	if (IS_ENABLED(CONFIG_KASAN) || __NO_KASLR_END_KERNEL > _REGION2_SIZE ||
	    (vsize > _REGION2_SIZE && kaslr_enabled())) {
		asce_limit = _REGION1_SIZE;
	...
	} else {
		asce_limit = _REGION2_SIZE;
		rte_size = _REGION3_SIZE;
	}
}

While the primary kernel with KASLR enabled upgrades to 4-level paging and
can reserve crashk_cma ranges at higher physical addresses, the kdump kernel
has __kaslr_enabled = 0 and stays in 3-level paging (4TB).  Because
ident_map_size is left at max_physmem_end instead of the end of the highest
usable CMA range, vmalloc_size and vmemmap are sized for max_physmem_end,
pushing vmemmap_start down within the 4TB space.  setup_kernel_memory_layout()
then clamps ident_map_size to vmemmap_start, and physmem_add_usable_range()
silently clips and drops any crashk_cma ranges above that limit.

[Severity: Medium]
Does leaving ident_map_size uncapped also allow reserve_crashkernel() to
succeed inside the kdump kernel when crashkernel= is on the kdump command
line?

In reserve_crashkernel(), parse_crashkernel(boot_command_line,
ident_map_size, ...) previously saw ident_map_size capped to oldmem_data.size
and rejected the reservation because crash_size >= system_ram.  With
ident_map_size left at max_physmem_end, parse_crashkernel() succeeds: when
crash_base is 0, reserve_crashkernel() sets crashk_res to
[oldmem_data.start, oldmem_data.start + crash_size - 1], and
setup_resources() adds that range to memblock even though
setup_usable_memory() excluded oldmem_data.start from the identity mapping.

[Severity: Low]
When CONFIG_KASAN is enabled, does leaving ident_map_size above the last
usable CMA range leave KASAN shadow holes unmapped?

In kasan_populate_shadow(), the loop over for_each_physmem_usable_range()
only populates POPULATE_KASAN_ZERO_SHADOW for inter-range gaps when
physmem_info.info_source == MEM_DETECT_DIAG260, and the untracked shadow
population after the loop starts at __identity_va(ident_map_size) rather than
at the end of the last usable range (memgap_start):

arch/s390/boot/vmem.c:kasan_populate_shadow() {
	...
	for_each_physmem_usable_range(iter, &start, &end) {
		kasan_populate((unsigned long)__identity_va(start),
			       (unsigned long)__identity_va(end),
			       POPULATE_KASAN_MAP_SHADOW);
		if (memgap_start && physmem_info.info_source == MEM_DETECT_DIAG260) {
			kasan_populate((unsigned long)__identity_va(memgap_start),
				       (unsigned long)__identity_va(start),
				       POPULATE_KASAN_ZERO_SHADOW);
		}
		memgap_start = end;
	}
	...
	kasan_populate((unsigned long)__identity_va(ident_map_size), VMALLOC_START,
		       POPULATE_KASAN_ZERO_SHADOW);
}

This leaves the identity-map holes between usable ranges (when info_source is
not MEM_DETECT_DIAG260) and between the last CMA range and ident_map_size
without KASAN shadow mappings, which will fault if a DCSS segment is mapped
into those regions.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261112112105.1169118-1-zaslonko@linux.ibm.com?part=6

[PATCH v1 7/8] s390/kdump: Skip the crashkernel CMA DMA wait

[Mikhail Zaslonko]

With a crashkernel CMA reservation, __crash_kexec() waits 10 seconds
before starting the kdump kernel, so that pending DMA into the CMA
memory can finish.

s390 does not need this: before it jumps to the kdump kernel,
s390_reset_system() issues diag 308 subcode 1, which terminates all
in-flight I/O.

Let architectures override the timeout and set it to 0 on s390.

Signed-off-by: Mikhail Zaslonko <zaslonko@linux.ibm.com>
---
 arch/s390/include/asm/kexec.h | 3 +++
 kernel/crash_core.c           | 2 ++
 2 files changed, 5 insertions(+)

diff --git a/arch/s390/include/asm/kexec.h b/arch/s390/include/asm/kexec.h
index 9084b750350d..2e15c4fa70a0 100644
--- a/arch/s390/include/asm/kexec.h
+++ b/arch/s390/include/asm/kexec.h
@@ -97,6 +97,9 @@ void arch_kexec_unprotect_crashkres(void);
 
 bool is_kdump_kernel(void);
 #define is_kdump_kernel is_kdump_kernel
+
+/* s390_reset_system() stops all I/O before the kdump kernel starts */
+#define CMA_DMA_TIMEOUT_SEC 0
 #endif
 
 #ifdef CONFIG_KEXEC_FILE
diff --git a/kernel/crash_core.c b/kernel/crash_core.c
index d0bd2d0cf899..5f60d252f4f1 100644
--- a/kernel/crash_core.c
+++ b/kernel/crash_core.c
@@ -36,7 +36,9 @@ note_buf_t __percpu *crash_notes;
 /* time to wait for possible DMA to finish before starting the kdump kernel
  * when a CMA reservation is used
  */
+#ifndef CMA_DMA_TIMEOUT_SEC
 #define CMA_DMA_TIMEOUT_SEC 10
+#endif
 
 #ifdef CONFIG_CRASH_DUMP
 
-- 
2.55.0

Re: [PATCH v1 7/8] s390/kdump: Skip the crashkernel CMA DMA wait

[sashiko-bot]

> With a crashkernel CMA reservation, __crash_kexec() waits 10 seconds
> before starting the kdump kernel, so that pending DMA into the CMA
> memory can finish.
> 
> s390 does not need this: before it jumps to the kdump kernel,
> s390_reset_system() issues diag 308 subcode 1, which terminates all
> in-flight I/O.
> 
> Let architectures override the timeout and set it to 0 on s390.
> 
> Signed-off-by: Mikhail Zaslonko <zaslonko@linux.ibm.com>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261112112105.1169118-1-zaslonko@linux.ibm.com?part=7

[PATCH v1 8/8] Documentation: kdump: Add s390 to crashkernel CMA

[Mikhail Zaslonko]

Add s390 to the architectures supporting crashkernel=size,cma and
mention that on s390 only kexec_file_load() passes the CMA memory on to
the kdump kernel.

Signed-off-by: Mikhail Zaslonko <zaslonko@linux.ibm.com>
---
 Documentation/admin-guide/kdump/kdump.rst       | 3 +++
 Documentation/admin-guide/kernel-parameters.txt | 4 ++--
 2 files changed, 5 insertions(+), 2 deletions(-)

diff --git a/Documentation/admin-guide/kdump/kdump.rst b/Documentation/admin-guide/kdump/kdump.rst
index 7587caadbae1..7952e8aca67a 100644
--- a/Documentation/admin-guide/kdump/kdump.rst
+++ b/Documentation/admin-guide/kdump/kdump.rst
@@ -332,6 +332,9 @@ crashkernel syntax
 	less reliable and possibly incomplete kdump is preferable to no kdump at
 	all.
 
+	On s390 the CMA memory is passed on only to a crash kernel loaded with
+	kexec_file_load(), e.g. with "kexec -s -p".
+
 Boot into System Kernel
 -----------------------
 1) Update the boot loader (such as grub, yaboot, or lilo) configuration
diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index 0b3c140babd9..1627c934324c 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -1099,8 +1099,8 @@ Kernel parameters
 			It will be ignored when crashkernel=X,high is not used
 			or memory reserved is below 4G.
 	crashkernel=size[KMG],cma
-			[KNL, X86, ARM64, RISCV, PPC] Reserve additional crash
-			kernel memory from CMA. This reservation is usable by
+			[KNL, X86, ARM64, RISCV, PPC, S390] Reserve additional
+			crash kernel memory from CMA. This reservation is usable by
 			the first system's userspace memory and kernel movable
 			allocations (memory balloon, zswap). Pages allocated
 			from this memory range will not be included in the vmcore
-- 
2.55.0

Re: [PATCH v1 8/8] Documentation: kdump: Add s390 to crashkernel CMA

[sashiko-bot]

> Add s390 to the architectures supporting crashkernel=size,cma and
> mention that on s390 only kexec_file_load() passes the CMA memory on to
> the kdump kernel.
> 
> Signed-off-by: Mikhail Zaslonko <zaslonko@linux.ibm.com>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261112112105.1169118-1-zaslonko@linux.ibm.com?part=8