[PATCH 0/5] riscv: kexec: fix kexec_file with CMA-backed segments

[PATCH 0/5] riscv: kexec: fix kexec_file with CMA-backed segments

[Han Gao via B4 Relay]

I ran into these while doing kexec_file reboots in a loop on a Zhihe
A210 board. Things mostly worked, but every few rounds the next kernel
would hang or purgatory would spin, depending on where the segments and
the DTB happened to land. Most of it goes back to segments now being
allocated from CMA, which exposed a few assumptions that no longer hold.
They are separate bugs, so each one gets its own patch.

Patch 1 lets the RISC-V relocation code turn the MMU off even when the
entry list has no indirection page, which is the normal case with CMA.
Patch 2 fixes the purgatory relocations, which assumed a PC-relative low
part always sits right after its high part. Patches 3 and 4 are in the
generic code: CMA quietly caps the alignment below what the RISC-V
kernel needs, and the tail of CMA segment pages was never cleared even
though the purgatory digest expects zeroes there. Patch 5 reserves the
DTB early enough that a CMA pool cannot be placed on top of it.

With all five applied, the board survives repeated kexec_file reboots,
including the layouts where every segment comes from CMA.

---
Han Gao (5):
      riscv: kexec: Disable the MMU before walking the entry list
      riscv: kexec_file: Pair PC-relative low parts with their high parts
      kexec_file: Honor the segment alignment for CMA allocations
      kexec_core: Clear the unused part of CMA segments
      riscv: mm: Reserve the DTB before scanning reserved memory

 arch/riscv/kernel/kexec_relocate.S     | 29 ++++++------
 arch/riscv/kernel/machine_kexec_file.c | 84 ++++++++++++++++++++++++++--------
 arch/riscv/mm/init.c                   | 18 ++++----
 kernel/kexec_core.c                    |  6 +--
 kernel/kexec_file.c                    |  7 +++
 5 files changed, 99 insertions(+), 45 deletions(-)
---
base-commit: c03c1a14507dfaf8b856c2608f9ac216021fc222
change-id: 20261023-riscv-kexec-cma-fixes-9bc69798a5cf

Best regards,
--  
Han Gao <gaohan@iscas.ac.cn>

[PATCH 1/5] riscv: kexec: Disable the MMU before walking the entry list

[Han Gao via B4 Relay]

From: Han Gao <gaohan@iscas.ac.cn>

riscv_kexec_relocate() only switches satp to Bare when it meets an
IND_INDIRECTION entry. Since segments may be allocated from CMA at their
destination, an image can have no indirection pages, and its list then
holds only IND_DONE. The relocation code jumps to the physical entry
point with the MMU still enabled and loops on instruction page faults.

Switch to Bare mode before reading the first entry, converting the list
pointer from its linear-map address.

Fixes: 07d24902977e ("kexec: enable CMA based contiguous allocation")
Signed-off-by: Han Gao <gaohan@iscas.ac.cn>
---
 arch/riscv/kernel/kexec_relocate.S | 29 +++++++++++++----------------
 1 file changed, 13 insertions(+), 16 deletions(-)

diff --git a/arch/riscv/kernel/kexec_relocate.S b/arch/riscv/kernel/kexec_relocate.S
index de0a4b35d01e..419b56600550 100644
--- a/arch/riscv/kernel/kexec_relocate.S
+++ b/arch/riscv/kernel/kexec_relocate.S
@@ -34,24 +34,22 @@ SYM_CODE_START(riscv_kexec_relocate)
 	csrw	CSR_SIP, zero
 
 	/*
-	 * When we switch SATP.MODE to "Bare" we'll only
-	 * play with physical addresses. However the first time
-	 * we try to jump somewhere, the offset on the jump
-	 * will be relative to pc which will still be on VA. To
-	 * deal with this we set stvec to the physical address at
-	 * the start of the loop below so that we jump there in
-	 * any case.
+	 * Switch SATP.MODE to "Bare" before reading any entry. An image
+	 * whose segments were allocated from CMA at their destination has
+	 * no indirection pages, so its list may hold only IND_DONE.
+	 *
+	 * pc is still a VA here: set stvec to the physical address of the
+	 * loop below, so that we get there even if the next instruction
+	 * fetch faults. The entry list starts at image->head, in the
+	 * linear mapping.
 	 */
 	la	s6, 1f
 	sub	s6, s6, s4
+	sub	s0, s0, s4
 	csrw	CSR_STVEC, s6
-
-	/*
-	 * With C-extension, here we get 42 Bytes and the next
-	 * .align directive would pad zeros here up to 44 Bytes.
-	 * So manually put a nop here to avoid zeros padding.
-	*/
-	nop
+	csrw	CSR_SATP, zero
+	sfence.vma
+	jr	s6
 
 	/* Process entries in a loop */
 .align 2
@@ -70,8 +68,7 @@ SYM_CODE_START(riscv_kexec_relocate)
 	andi	t1, t0, 0x2
 	beqz	t1, 2f
 	andi	s0, t0, ~0x2
-	csrw	CSR_SATP, zero
-	jr	s6
+	j	1b
 
 2:
 	/* IND_DONE entry ? -> jump to done label */

-- 
2.55.0

Re: [PATCH 1/5] riscv: kexec: Disable the MMU before walking the entry list

[sashiko-bot]

> From: Han Gao <gaohan@iscas.ac.cn>
> 
> riscv_kexec_relocate() only switches satp to Bare when it meets an
> IND_INDIRECTION entry. Since segments may be allocated from CMA at their
> destination, an image can have no indirection pages, and its list then
> holds only IND_DONE. The relocation code jumps to the physical entry
> point with the MMU still enabled and loops on instruction page faults.
> 
> Switch to Bare mode before reading the first entry, converting the list
> pointer from its linear-map address.
> 
> Fixes: 07d24902977e ("kexec: enable CMA based contiguous allocation")
> Signed-off-by: Han Gao <gaohan@iscas.ac.cn>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261023-riscv-kexec-cma-fixes-v1-0-825b90ea23c4@iscas.ac.cn?part=1

[PATCH 2/5] riscv: kexec_file: Pair PC-relative low parts with their high parts

[Han Gao via B4 Relay]

From: Han Gao <gaohan@iscas.ac.cn>

The purgatory relocation code applies R_RISCV_PCREL_HI20 to the auipc
and to the instruction after it, and ignores R_RISCV_PCREL_LO12_I. This
assumes that the low part immediately follows its high part. Compilers
may schedule several auipc instructions before their users; clang does
so in sha256_init(), which corrupts the hash and leaves purgatory
spinning on a digest mismatch.

Apply the high part to the auipc only. Resolve each R_RISCV_PCREL_LO12_I
and R_RISCV_PCREL_LO12_S through the R_RISCV_PCREL_HI20 at the address
its symbol labels, as the module loader does.

Fixes: 838b3e28488f ("RISC-V: Load purgatory in kexec_file")
Signed-off-by: Han Gao <gaohan@iscas.ac.cn>
---
 arch/riscv/kernel/machine_kexec_file.c | 84 ++++++++++++++++++++++++++--------
 1 file changed, 66 insertions(+), 18 deletions(-)

diff --git a/arch/riscv/kernel/machine_kexec_file.c b/arch/riscv/kernel/machine_kexec_file.c
index 26cd2a8bd0cd..1d7636acd382 100644
--- a/arch/riscv/kernel/machine_kexec_file.c
+++ b/arch/riscv/kernel/machine_kexec_file.c
@@ -104,6 +104,8 @@ static char *setup_kdump_cmdline(struct kimage *image, char *cmdline,
 
 #define ENCODE_ITYPE_IMM(x) \
 	(RV_X(x, 0, 12) << 20)
+#define ENCODE_STYPE_IMM(x) \
+	((RV_X(x, 0, 5) << 7) | (RV_X(x, 5, 7) << 25))
 #define ENCODE_BTYPE_IMM(x) \
 	((RV_X(x, 1, 4) << 8) | (RV_X(x, 5, 6) << 25) | \
 	(RV_X(x, 11, 1) << 7) | (RV_X(x, 12, 1) << 31))
@@ -128,6 +130,49 @@ static char *setup_kdump_cmdline(struct kimage *image, char *cmdline,
 #define CLEAN_IMM(type, x) \
 	((~ENCODE_##type##_IMM((uint64_t)(-1))) & (x))
 
+static int riscv_kexec_rela_value(const struct purgatory_info *pi,
+				  const Elf_Shdr *symtab, const Elf64_Rela *rela,
+				  unsigned long *val)
+{
+	const Elf_Sym *sym = (void *)pi->ehdr + symtab->sh_offset;
+
+	sym += ELF64_R_SYM(rela->r_info);
+	if (sym->st_shndx == SHN_ABS)
+		*val = 0;
+	else if (sym->st_shndx < pi->ehdr->e_shnum)
+		*val = pi->sechdrs[sym->st_shndx].sh_addr;
+	else
+		return -ENOEXEC;
+	*val += sym->st_value + rela->r_addend;
+	return 0;
+}
+
+/*
+ * An R_RISCV_PCREL_LO12_* symbol labels its R_RISCV_PCREL_HI20 instruction,
+ * which compilers need not place immediately before the low part.
+ */
+static int riscv_kexec_pcrel_hi20(const struct purgatory_info *pi,
+				  const Elf_Shdr *section, const Elf_Shdr *relsec,
+				  const Elf_Shdr *symtab, unsigned long hi_addr,
+				  long *offset)
+{
+	const Elf64_Rela *relas = (void *)pi->ehdr + relsec->sh_offset;
+	unsigned long val;
+	int i, ret;
+
+	for (i = 0; i < relsec->sh_size / sizeof(*relas); i++) {
+		if (ELF64_R_TYPE(relas[i].r_info) != R_RISCV_PCREL_HI20 ||
+		    section->sh_addr + relas[i].r_offset != hi_addr)
+			continue;
+		ret = riscv_kexec_rela_value(pi, symtab, &relas[i], &val);
+		if (ret)
+			return ret;
+		*offset = val - hi_addr;
+		return 0;
+	}
+	return -ENOEXEC;
+}
+
 int arch_kexec_apply_relocations_add(struct purgatory_info *pi,
 				     Elf_Shdr *section,
 				     const Elf_Shdr *relsec,
@@ -149,8 +194,8 @@ int arch_kexec_apply_relocations_add(struct purgatory_info *pi,
 		const Elf_Sym *sym;	/* symbol to relocate */
 		unsigned long addr;	/* final location after relocation */
 		unsigned long val;	/* relocated symbol value */
-		unsigned long sec_base;	/* relocated symbol value */
 		void *loc;		/* tmp location to modify */
+		long offset;
 
 		sym = (void *)pi->ehdr + symtab->sh_offset;
 		sym += ELF64_R_SYM(relas[i].r_info);
@@ -164,18 +209,11 @@ int arch_kexec_apply_relocations_add(struct purgatory_info *pi,
 		loc += section->sh_offset;
 		loc += relas[i].r_offset;
 
-		if (sym->st_shndx == SHN_ABS)
-			sec_base = 0;
-		else if (sym->st_shndx >= pi->ehdr->e_shnum) {
+		if (riscv_kexec_rela_value(pi, symtab, &relas[i], &val)) {
 			pr_err("Invalid section %d for symbol %s\n",
 			       sym->st_shndx, name);
 			return -ENOEXEC;
-		} else
-			sec_base = pi->sechdrs[sym->st_shndx].sh_addr;
-
-		val = sym->st_value;
-		val += sec_base;
-		val += relas[i].r_addend;
+		}
 
 		addr = section->sh_addr + relas[i].r_offset;
 
@@ -190,13 +228,25 @@ int arch_kexec_apply_relocations_add(struct purgatory_info *pi,
 			*(u32 *)loc = CLEAN_IMM(JTYPE, *(u32 *)loc) |
 				 ENCODE_JTYPE_IMM(val - addr);
 			break;
-		/*
-		 * With no R_RISCV_PCREL_LO12_S, R_RISCV_PCREL_LO12_I
-		 * sym is expected to be next to R_RISCV_PCREL_HI20
-		 * in purgatory relsec. Handle it like R_RISCV_CALL
-		 * sym, instead of searching the whole relsec.
-		 */
 		case R_RISCV_PCREL_HI20:
+			*(u32 *)loc = CLEAN_IMM(UTYPE, *(u32 *)loc) |
+				 ENCODE_UTYPE_IMM(RISCV_CONST_HIGH_PART(val - addr));
+			break;
+		case R_RISCV_PCREL_LO12_I:
+		case R_RISCV_PCREL_LO12_S:
+			if (riscv_kexec_pcrel_hi20(pi, section, relsec, symtab,
+						   val, &offset)) {
+				pr_err("No R_RISCV_PCREL_HI20 for %s\n", name);
+				return -ENOEXEC;
+			}
+			offset = RISCV_CONST_LOW_PART(offset);
+			if (r_type == R_RISCV_PCREL_LO12_I)
+				*(u32 *)loc = CLEAN_IMM(ITYPE, *(u32 *)loc) |
+					 ENCODE_ITYPE_IMM(offset);
+			else
+				*(u32 *)loc = CLEAN_IMM(STYPE, *(u32 *)loc) |
+					 ENCODE_STYPE_IMM(offset);
+			break;
 		case R_RISCV_CALL_PLT:
 		case R_RISCV_CALL:
 			*(u64 *)loc = CLEAN_IMM(UITYPE, *(u64 *)loc) |
@@ -222,8 +272,6 @@ int arch_kexec_apply_relocations_add(struct purgatory_info *pi,
 		case R_RISCV_SUB32:
 			*(u32 *)loc -= val;
 			break;
-		/* It has been applied by R_RISCV_PCREL_HI20 sym */
-		case R_RISCV_PCREL_LO12_I:
 		case R_RISCV_ALIGN:
 		case R_RISCV_RELAX:
 			break;

-- 
2.55.0

Re: [PATCH 2/5] riscv: kexec_file: Pair PC-relative low parts with their high parts

[sashiko-bot]

> From: Han Gao <gaohan@iscas.ac.cn>
> 
> The purgatory relocation code applies R_RISCV_PCREL_HI20 to the auipc
> and to the instruction after it, and ignores R_RISCV_PCREL_LO12_I. This
> assumes that the low part immediately follows its high part. Compilers
> may schedule several auipc instructions before their users; clang does
> so in sha256_init(), which corrupts the hash and leaves purgatory
> spinning on a digest mismatch.
> 
> Apply the high part to the auipc only. Resolve each R_RISCV_PCREL_LO12_I
> and R_RISCV_PCREL_LO12_S through the R_RISCV_PCREL_HI20 at the address
> its symbol labels, as the module loader does.
> 
> Fixes: 838b3e28488f ("RISC-V: Load purgatory in kexec_file")
> Signed-off-by: Han Gao <gaohan@iscas.ac.cn>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261023-riscv-kexec-cma-fixes-v1-0-825b90ea23c4@iscas.ac.cn?part=2

[PATCH 3/5] kexec_file: Honor the segment alignment for CMA allocations

[Han Gao via B4 Relay]

From: Han Gao <gaohan@iscas.ac.cn>

dma_alloc_from_contiguous() limits the requested alignment to
CONFIG_CMA_ALIGNMENT without reporting it. kexec_alloc_contig() can
therefore place a segment at a less aligned address than buf_align. On
RISC-V with 4 KiB pages, the default 1 MiB limit is below the 2 MiB
kernel alignment, and the new kernel stops at the setup_vm() BUG_ON for
a misaligned physical address.

Release such a block and fall back to the regular memory hole search,
which honors the alignment.

Fixes: 07d24902977e ("kexec: enable CMA based contiguous allocation")
Signed-off-by: Han Gao <gaohan@iscas.ac.cn>
---
 kernel/kexec_file.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/kernel/kexec_file.c b/kernel/kexec_file.c
index 59fb9d71e9d8..6ce10e01414e 100644
--- a/kernel/kexec_file.c
+++ b/kernel/kexec_file.c
@@ -690,6 +690,13 @@ static int kexec_alloc_contig(struct kexec_buf *kbuf)
 
 	mem = page_to_boot_pfn(p) << PAGE_SHIFT;
 
+	/* dma_alloc_from_contiguous() caps the alignment at CONFIG_CMA_ALIGNMENT. */
+	if (!IS_ALIGNED(mem, kbuf->buf_align)) {
+		pr_debug("CMA block 0x%lx is not aligned to 0x%lx\n", mem, kbuf->buf_align);
+		dma_release_from_contiguous(NULL, p, nr_pages);
+		return -EINVAL;
+	}
+
 	if (kimage_is_destination_range(kbuf->image, mem, mem + kbuf->memsz)) {
 		/* Our region is already in use by a statically defined one. Bail out. */
 		pr_debug("CMA overlaps existing mem: 0x%lx+0x%lx\n", mem, kbuf->memsz);

-- 
2.55.0

Re: [PATCH 3/5] kexec_file: Honor the segment alignment for CMA allocations

[sashiko-bot]

> From: Han Gao <gaohan@iscas.ac.cn>
> 
> dma_alloc_from_contiguous() limits the requested alignment to
> CONFIG_CMA_ALIGNMENT without reporting it. kexec_alloc_contig() can
> therefore place a segment at a less aligned address than buf_align. On
> RISC-V with 4 KiB pages, the default 1 MiB limit is below the 2 MiB
> kernel alignment, and the new kernel stops at the setup_vm() BUG_ON for
> a misaligned physical address.
> 
> Release such a block and fall back to the regular memory hole search,
> which honors the alignment.
> 
> Fixes: 07d24902977e ("kexec: enable CMA based contiguous allocation")
> Signed-off-by: Han Gao <gaohan@iscas.ac.cn>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261023-riscv-kexec-cma-fixes-v1-0-825b90ea23c4@iscas.ac.cn?part=3

[PATCH 4/5] kexec_core: Clear the unused part of CMA segments

[Han Gao via B4 Relay]

From: Han Gao <gaohan@iscas.ac.cn>

kimage_load_cma_segment() copies the source data into pages that are not
cleared, but zeroes nothing after it: the final memset() runs once the
loop has consumed all of memsz. The tail of the last source page and
every page beyond bufsz keep stale data, while the purgatory digest
covers them as zeroes.

A kexec_file reboot then stops in purgatory on a digest mismatch, for
instance when the DTB lands in a reused CMA page.

Zero the remainder of each page instead.

Fixes: 07d24902977e ("kexec: enable CMA based contiguous allocation")
Signed-off-by: Han Gao <gaohan@iscas.ac.cn>
---
 kernel/kexec_core.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/kernel/kexec_core.c b/kernel/kexec_core.c
index dc770b9a6d05..0d6e10bea4e4 100644
--- a/kernel/kexec_core.c
+++ b/kernel/kexec_core.c
@@ -780,15 +780,15 @@ static int kimage_load_cma_segment(struct kimage *image, int idx)
 			goto out;
 		}
 
+		/* CMA pages are not cleared: zero the part without source data */
+		memset(ptr + uchunk, 0, mchunk - uchunk);
+
 		ptr    += mchunk;
 		mbytes -= mchunk;
 
 		cond_resched();
 	}
 
-	/* Clear any remainder */
-	memset(ptr, 0, mbytes);
-
 out:
 	return result;
 }

-- 
2.55.0

Re: [PATCH 4/5] kexec_core: Clear the unused part of CMA segments

[sashiko-bot]

> From: Han Gao <gaohan@iscas.ac.cn>
> 
> kimage_load_cma_segment() copies the source data into pages that are not
> cleared, but zeroes nothing after it: the final memset() runs once the
> loop has consumed all of memsz. The tail of the last source page and
> every page beyond bufsz keep stale data, while the purgatory digest
> covers them as zeroes.
> 
> A kexec_file reboot then stops in purgatory on a digest mismatch, for
> instance when the DTB lands in a reused CMA page.
> 
> Zero the remainder of each page instead.
> 
> Fixes: 07d24902977e ("kexec: enable CMA based contiguous allocation")
> Signed-off-by: Han Gao <gaohan@iscas.ac.cn>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261023-riscv-kexec-cma-fixes-v1-0-825b90ea23c4@iscas.ac.cn?part=4

[PATCH 5/5] riscv: mm: Reserve the DTB before scanning reserved memory

[Han Gao via B4 Relay]

From: Han Gao <gaohan@iscas.ac.cn>

setup_bootmem() reserves the DTB only after
early_init_fdt_scan_reserved_mem() has allocated the dynamically placed
reserved-memory regions. A CMA pool described with size and alloc-ranges
can therefore be placed over the DTB, which is later handed to the page
allocator and overwritten.

This happens after kexec_file has put the DTB of the next kernel in CMA:
the next kernel reuses that range for its own pool, and a further
kexec_file_load() fails with -FDT_ERR_BADMAGIC.

Reserve the DTB first.

Fixes: ef69d2559fe9 ("riscv: Move early dtb mapping into the fixmap region")
Signed-off-by: Han Gao <gaohan@iscas.ac.cn>
---
 arch/riscv/mm/init.c | 18 ++++++++++--------
 1 file changed, 10 insertions(+), 8 deletions(-)

diff --git a/arch/riscv/mm/init.c b/arch/riscv/mm/init.c
index fb37b0b67efe..1ca908bcc4c0 100644
--- a/arch/riscv/mm/init.c
+++ b/arch/riscv/mm/init.c
@@ -290,21 +290,23 @@ static void __init setup_bootmem(void)
 
 	reserve_initrd_mem();
 
-	/*
-	 * No allocation should be done before reserving the memory as defined
-	 * in the device tree, otherwise the allocation could end up in a
-	 * reserved region.
-	 */
-	early_init_fdt_scan_reserved_mem();
-
 	/*
 	 * If DTB is built in, no need to reserve its memblock.
 	 * Otherwise, do reserve it but avoid using
 	 * early_init_fdt_reserve_self() since __pa() does
-	 * not work for DTB pointers that are fixmap addresses
+	 * not work for DTB pointers that are fixmap addresses.
+	 * Reserve it before dynamically placed reserved-memory
+	 * regions, such as a CMA pool, are allocated.
 	 */
 	if (!IS_ENABLED(CONFIG_BUILTIN_DTB))
 		memblock_reserve(dtb_early_pa, fdt_totalsize(dtb_early_va));
+
+	/*
+	 * No allocation should be done before reserving the memory as defined
+	 * in the device tree, otherwise the allocation could end up in a
+	 * reserved region.
+	 */
+	early_init_fdt_scan_reserved_mem();
 }
 
 #ifdef CONFIG_RELOCATABLE

-- 
2.55.0

Re: [PATCH 5/5] riscv: mm: Reserve the DTB before scanning reserved memory

[sashiko-bot]

> From: Han Gao <gaohan@iscas.ac.cn>
> 
> setup_bootmem() reserves the DTB only after
> early_init_fdt_scan_reserved_mem() has allocated the dynamically placed
> reserved-memory regions. A CMA pool described with size and alloc-ranges
> can therefore be placed over the DTB, which is later handed to the page
> allocator and overwritten.
> 
> This happens after kexec_file has put the DTB of the next kernel in CMA:
> the next kernel reuses that range for its own pool, and a further
> kexec_file_load() fails with -FDT_ERR_BADMAGIC.
> 
> Reserve the DTB first.
> 
> Fixes: ef69d2559fe9 ("riscv: Move early dtb mapping into the fixmap region")
> Signed-off-by: Han Gao <gaohan@iscas.ac.cn>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261023-riscv-kexec-cma-fixes-v1-0-825b90ea23c4@iscas.ac.cn?part=5