* [PATCH v2 1/2] arm64: kexec: use a typed entry to fix kCFI oops
2026-11-11 23:26 [PATCH v2 0/2] arm64: use typed entries into copied code for kCFI Ben Cressey
@ 2026-11-11 23:26 ` Ben Cressey
2026-11-11 23:42 ` sashiko-bot
2026-11-11 23:26 ` [PATCH v2 2/2] arm64: hibernate: use a typed entry to drop __nocfi Ben Cressey
1 sibling, 1 reply; 5+ messages in thread
From: Ben Cressey @ 2026-11-11 23:26 UTC (permalink / raw)
To: Catalin Marinas, Will Deacon, Mark Rutland, Pasha Tatashin,
Rafael J. Wysocki, Pavel Machek
Cc: linux-arm-kernel, linux-kernel, linux-pm, Kees Cook,
Sami Tolvanen, kexec, llvm, Nathan Chancellor, Zhaoyang Huang,
Jeson Gao, stable
An arm64 Linux kernel built with CONFIG_CFI=y oopses after "Bye!" on
every kexec that relocates the image. Crash images load in place and are
not affected. The oops reproduces under QEMU:
Unable to handle kernel paging request at virtual address 00000000544bfffc
pc : machine_kexec+0x104/0x274
Code: d5182008 d5033fdf aa1303e0 f9418a68 (b85fc110)
b85fc110 is "ldur w16, [x8, #-4]", the kCFI check's load of the type
hash in front of the callee. Commit efc2d0f20a9d ("arm64: kexec: keep
MMU enabled during kexec relocation") made machine_kexec() call the
control page's copy of arm64_relocate_new_kernel() indirectly, while
TTBR0 maps only that page. The copy starts at the page's first byte, so
the load faults. SYM_CODE_START() also gives the routine no hash.
SYM_TYPED_FUNC_START() emits the hash inside .kexec_relocate.text, and
machine_kexec_post_load() already copies that whole section, at most
4 KiB, to the page's start, so the mapped page holds both hash and
entry.
The ASSERT from commit dc4824faa265 ("arm64: avoid executing padding
bytes during kexec / hibernation") kept the entry first, since C
branched to the page's start. Relax it, so the entry need only be in the
section.
Fix by typing the routine with SYM_TYPED_FUNC_START() and a C prototype,
and entering the copy at the symbol's offset in the section.
The code is Kees Cook's, as posted in review, with two comments and the
ASSERT message reworded.
An LLM diagnosed the oops.
Fixes: efc2d0f20a9d ("arm64: kexec: keep MMU enabled during kexec relocation")
Cc: stable@vger.kernel.org
Suggested-by: Kees Cook <kees@kernel.org>
Link: https://lore.kernel.org/linux-arm-kernel/202609241804.BB23BF10@keescook/
Signed-off-by: Ben Cressey <ben@cressey.dev>
---
arch/arm64/include/asm/kexec.h | 2 ++
arch/arm64/kernel/machine_kexec.c | 7 +++++--
arch/arm64/kernel/relocate_kernel.S | 11 +++++++----
arch/arm64/kernel/vmlinux.lds.S | 5 +++--
4 files changed, 17 insertions(+), 8 deletions(-)
diff --git a/arch/arm64/include/asm/kexec.h b/arch/arm64/include/asm/kexec.h
index 892e5bebda95..0be042ec22c1 100644
--- a/arch/arm64/include/asm/kexec.h
+++ b/arch/arm64/include/asm/kexec.h
@@ -100,6 +100,8 @@ void cpu_soft_restart(unsigned long el2_switch, unsigned long entry,
unsigned long arg0, unsigned long arg1,
unsigned long arg2);
+void arm64_relocate_new_kernel(struct kimage *kimage);
+
int machine_kexec_post_load(struct kimage *image);
#define machine_kexec_post_load machine_kexec_post_load
#endif
diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c
index 48f343704cb5..24366b5b3259 100644
--- a/arch/arm64/kernel/machine_kexec.c
+++ b/arch/arm64/kernel/machine_kexec.c
@@ -147,9 +147,12 @@ int machine_kexec_post_load(struct kimage *kimage)
kimage->arch.ttbr1 = __pa(trans_pgd);
kimage->arch.zero_page = __pa_symbol(empty_zero_page);
+ /* Copy the whole section: a kCFI hash may precede the entry point */
reloc_size = __relocate_new_kernel_end - __relocate_new_kernel_start;
memcpy(reloc_code, __relocate_new_kernel_start, reloc_size);
- kimage->arch.kern_reloc = __pa(reloc_code);
+ kimage->arch.kern_reloc = __pa(reloc_code) +
+ ((unsigned long)arm64_relocate_new_kernel -
+ (unsigned long)__relocate_new_kernel_start);
rc = trans_pgd_idmap_page(&info, &kimage->arch.ttbr0,
&kimage->arch.t0sz, reloc_code);
if (rc)
@@ -204,7 +207,7 @@ void machine_kexec(struct kimage *kimage)
restart(is_hyp_nvhe(), kimage->start, kimage->arch.dtb_mem,
0, 0);
} else {
- void (*kernel_reloc)(struct kimage *kimage);
+ typeof(arm64_relocate_new_kernel) *kernel_reloc;
if (is_hyp_nvhe())
__hyp_set_vectors(kimage->arch.el2_vectors);
diff --git a/arch/arm64/kernel/relocate_kernel.S b/arch/arm64/kernel/relocate_kernel.S
index 6cb4209f5dab..064db5856bba 100644
--- a/arch/arm64/kernel/relocate_kernel.S
+++ b/arch/arm64/kernel/relocate_kernel.S
@@ -8,6 +8,7 @@
* Pasha Tatashin <pasha.tatashin@soleen.com>
*/
+#include <linux/cfi_types.h>
#include <linux/kexec.h>
#include <linux/linkage.h>
@@ -32,11 +33,13 @@
* new image to its final location. To assure that the
* arm64_relocate_new_kernel routine which does that copy is not overwritten,
* all code and data needed by arm64_relocate_new_kernel must be between the
- * symbols arm64_relocate_new_kernel and arm64_relocate_new_kernel_end. The
- * machine_kexec() routine will copy arm64_relocate_new_kernel to the kexec
+ * symbols __relocate_new_kernel_start and __relocate_new_kernel_end. The
+ * machine_kexec_post_load() routine will copy that whole region to the kexec
* safe memory that has been set up to be preserved during the copy operation.
+ *
+ * Called indirectly by machine_kexec(), this routine needs a kCFI type hash.
*/
-SYM_CODE_START(arm64_relocate_new_kernel)
+SYM_TYPED_FUNC_START(arm64_relocate_new_kernel)
/*
* The kimage structure isn't allocated specially and may be clobbered
* during relocation. We must load any values we need from it prior to
@@ -98,4 +101,4 @@ SYM_CODE_START(arm64_relocate_new_kernel)
mov x2, xzr
mov x3, xzr
br x28 /* Jumps from el1 */
-SYM_CODE_END(arm64_relocate_new_kernel)
+SYM_FUNC_END(arm64_relocate_new_kernel)
diff --git a/arch/arm64/kernel/vmlinux.lds.S b/arch/arm64/kernel/vmlinux.lds.S
index 1869b44ae190..c95243a8b24e 100644
--- a/arch/arm64/kernel/vmlinux.lds.S
+++ b/arch/arm64/kernel/vmlinux.lds.S
@@ -440,6 +440,7 @@ ASSERT(swapper_pg_dir - tramp_pg_dir == TRAMP_SWAPPER_OFFSET,
ASSERT(__relocate_new_kernel_end - __relocate_new_kernel_start <= SZ_4K,
"kexec relocation code is bigger than 4 KiB")
ASSERT(KEXEC_CONTROL_PAGE_SIZE >= SZ_4K, "KEXEC_CONTROL_PAGE_SIZE is broken")
-ASSERT(__relocate_new_kernel_start == arm64_relocate_new_kernel,
- "kexec control page does not start with arm64_relocate_new_kernel")
+ASSERT(arm64_relocate_new_kernel >= __relocate_new_kernel_start &&
+ arm64_relocate_new_kernel < __relocate_new_kernel_end,
+ "kexec relocation code does not contain arm64_relocate_new_kernel")
#endif
--
2.56.0
^ permalink raw reply related [flat|nested] 5+ messages in thread* [PATCH v2 2/2] arm64: hibernate: use a typed entry to drop __nocfi
2026-11-11 23:26 [PATCH v2 0/2] arm64: use typed entries into copied code for kCFI Ben Cressey
2026-11-11 23:26 ` [PATCH v2 1/2] arm64: kexec: use a typed entry to fix kCFI oops Ben Cressey
@ 2026-11-11 23:26 ` Ben Cressey
2026-11-11 23:43 ` sashiko-bot
1 sibling, 1 reply; 5+ messages in thread
From: Ben Cressey @ 2026-11-11 23:26 UTC (permalink / raw)
To: Catalin Marinas, Will Deacon, Mark Rutland, Pasha Tatashin,
Rafael J. Wysocki, Pavel Machek
Cc: linux-arm-kernel, linux-kernel, linux-pm, Kees Cook,
Sami Tolvanen, kexec, llvm, Nathan Chancellor, Zhaoyang Huang,
Jeson Gao
swsusp_arch_resume() calls the safe page's copy of
swsusp_arch_suspend_exit() indirectly, without a kCFI check. The MMU is
on at the call and TTBR0 maps only that page. The copy starts at the
page's first byte, so the check's load of the type hash in front of it
would fault. SYM_CODE_START() also gives the routine no hash.
Commit e2f8216ca2d8 ("arm64: Set __nocfi on swsusp_arch_resume()")
disabled the check "for now", since the entry had to be first
in .hibernate_exit.text.
SYM_TYPED_FUNC_START() emits the hash inside that section, and
create_safe_exec_page() already copies the whole section, at most 4 KiB,
to the page's start, so the page holds both hash and entry. Relax the
linker script's ASSERT, so the entry need only be in the section.
Fix by typing the routine, with a C prototype, entering the copy at the
symbol's offset in the section, and dropping __nocfi.
Suggested-by: Mark Rutland <mark.rutland@arm.com>
Link: https://lore.kernel.org/linux-arm-kernel/arZSe8hNnRPM5FqY@J2N7QTR9R3.cambridge.arm.com/
Signed-off-by: Ben Cressey <ben@cressey.dev>
---
arch/arm64/include/asm/suspend.h | 5 +++++
arch/arm64/kernel/hibernate-asm.S | 7 +++++--
arch/arm64/kernel/hibernate.c | 14 +++++++++-----
arch/arm64/kernel/vmlinux.lds.S | 5 +++--
4 files changed, 22 insertions(+), 9 deletions(-)
diff --git a/arch/arm64/include/asm/suspend.h b/arch/arm64/include/asm/suspend.h
index e9ce68d50ba4..e75aa3681414 100644
--- a/arch/arm64/include/asm/suspend.h
+++ b/arch/arm64/include/asm/suspend.h
@@ -45,6 +45,11 @@ void _cpu_resume(void);
int swsusp_arch_suspend(void);
int swsusp_arch_resume(void);
+void __noreturn swsusp_arch_suspend_exit(phys_addr_t tmp_pg_dir,
+ phys_addr_t ttbr1_el1,
+ void *reenter_kernel, void *pblist,
+ phys_addr_t hyp_stub_vectors,
+ phys_addr_t zero_page);
int arch_hibernation_header_save(void *addr, unsigned int max_size);
int arch_hibernation_header_restore(void *addr);
diff --git a/arch/arm64/kernel/hibernate-asm.S b/arch/arm64/kernel/hibernate-asm.S
index 2baefe7a82d3..ef8c0b926dcb 100644
--- a/arch/arm64/kernel/hibernate-asm.S
+++ b/arch/arm64/kernel/hibernate-asm.S
@@ -6,6 +6,7 @@
* Author: James Morse <james.morse@arm.com>
*/
#include <linux/linkage.h>
+#include <linux/cfi_types.h>
#include <linux/errno.h>
#include <asm/asm-offsets.h>
@@ -31,6 +32,8 @@
* switches to a copy of the linear map in ttbr1, performs the restore, then
* switches ttbr1 to the original kernel's swapper_pg_dir.
*
+ * Called indirectly by swsusp_arch_resume(), this code needs a kCFI type hash.
+ *
* All of memory gets written to, including code. We need to clean the kernel
* text to the Point of Coherence (PoC) before secondary cores can be booted.
* Because the kernel modules and executable pages mapped to user space are
@@ -45,7 +48,7 @@
* x5: physical address of a zero page that remains zero after resume
*/
.pushsection ".hibernate_exit.text", "ax"
-SYM_CODE_START(swsusp_arch_suspend_exit)
+SYM_TYPED_FUNC_START(swsusp_arch_suspend_exit)
/*
* We execute from ttbr0, change ttbr1 to our copied linear map tables
* with a break-before-make via the zero page
@@ -93,5 +96,5 @@ alternative_insn "dc cvau, x4", "dc civac, x4", ARM64_WORKAROUND_CLEAN_CACHE
mov x0, #HVC_SET_VECTORS
hvc #0
3: ret
-SYM_CODE_END(swsusp_arch_suspend_exit)
+SYM_FUNC_END(swsusp_arch_suspend_exit)
.popsection
diff --git a/arch/arm64/kernel/hibernate.c b/arch/arm64/kernel/hibernate.c
index 8c39863d2d63..d5a066ece8b2 100644
--- a/arch/arm64/kernel/hibernate.c
+++ b/arch/arm64/kernel/hibernate.c
@@ -418,15 +418,15 @@ int swsusp_arch_suspend(void)
* Memory allocated by get_safe_page() will be dealt with by the hibernate code,
* we don't need to free it here.
*/
-int __nocfi swsusp_arch_resume(void)
+int swsusp_arch_resume(void)
{
int rc;
void *zero_page;
size_t exit_size;
pgd_t *tmp_pg_dir;
phys_addr_t el2_vectors;
- void __noreturn (*hibernate_exit)(phys_addr_t, phys_addr_t, void *,
- void *, phys_addr_t, phys_addr_t);
+ phys_addr_t exit_page;
+ typeof(swsusp_arch_suspend_exit) *hibernate_exit;
struct trans_pgd_info trans_info = {
.trans_alloc_page = hibernate_page_alloc,
.trans_alloc_arg = (__force void *)GFP_ATOMIC,
@@ -463,14 +463,18 @@ int __nocfi swsusp_arch_resume(void)
exit_size = __hibernate_exit_text_end - __hibernate_exit_text_start;
/*
* Copy swsusp_arch_suspend_exit() to a safe page. This will generate
- * a new set of ttbr0 page tables and load them.
+ * a new set of ttbr0 page tables and load them. The whole section is
+ * copied: a kCFI hash may precede the entry point.
*/
rc = create_safe_exec_page(__hibernate_exit_text_start, exit_size,
- (phys_addr_t *)&hibernate_exit);
+ &exit_page);
if (rc) {
pr_err("Failed to create safe executable page for hibernate_exit code.\n");
return rc;
}
+ hibernate_exit = (void *)(exit_page +
+ ((unsigned long)swsusp_arch_suspend_exit -
+ (unsigned long)__hibernate_exit_text_start));
/*
* KASLR will cause the el2 vectors to be in a different location in
diff --git a/arch/arm64/kernel/vmlinux.lds.S b/arch/arm64/kernel/vmlinux.lds.S
index c95243a8b24e..641074a6e6ca 100644
--- a/arch/arm64/kernel/vmlinux.lds.S
+++ b/arch/arm64/kernel/vmlinux.lds.S
@@ -412,8 +412,9 @@ ASSERT(__idmap_text_end - (__idmap_text_start & ~(SZ_4K - 1)) <= SZ_4K,
#ifdef CONFIG_HIBERNATION
ASSERT(__hibernate_exit_text_end - __hibernate_exit_text_start <= SZ_4K,
"Hibernate exit text is bigger than 4 KiB")
-ASSERT(__hibernate_exit_text_start == swsusp_arch_suspend_exit,
- "Hibernate exit text does not start with swsusp_arch_suspend_exit")
+ASSERT(swsusp_arch_suspend_exit >= __hibernate_exit_text_start &&
+ swsusp_arch_suspend_exit < __hibernate_exit_text_end,
+ "Hibernate exit text does not contain swsusp_arch_suspend_exit")
#endif
#ifdef CONFIG_UNMAP_KERNEL_AT_EL0
ASSERT((__entry_tramp_text_end - __entry_tramp_text_start) <= 3*PAGE_SIZE,
--
2.56.0
^ permalink raw reply related [flat|nested] 5+ messages in thread