* [PATCH v2 0/2] arm64: sdei: Handle exceptions taken inside the event handler
@ 2026-11-05 12:08 Kiryl Shutsemau
2026-11-05 12:08 ` [PATCH v2 1/2] arm64: sdei: Run the event handler on the kernel vectors Kiryl Shutsemau
2026-11-05 12:08 ` [PATCH v2 2/2] arm64: sdei: Preserve the interrupted context's exception registers Kiryl Shutsemau
0 siblings, 2 replies; 5+ messages in thread
From: Kiryl Shutsemau @ 2026-11-05 12:08 UTC (permalink / raw)
To: Catalin Marinas, Will Deacon, Mark Rutland, James Morse,
Usama Arif
Cc: Kiryl Shutsemau (Meta), Thomas Huth, Vladimir Murzin,
Peter Zijlstra, Marc Zyngier, Jinjie Ruan, kernel-team,
Doug Anderson, linux-kernel, mrigendrachaubey, linux-arm-kernel,
Breno Leitao, Linus Walleij, Dawei Li
From: "Kiryl Shutsemau (Meta)" <kas@kernel.org>
An SDEI event can land anywhere, and a synchronous exception taken by its
handler breaks the context it interrupted:
- With KPTI, an event that lands in user space runs the handler on the
trampoline vectors, whose current-EL half is empty. The exception
loops there and the CPU is lost.
- The exception overwrites ELR, SPSR, ESR and FAR. If the event landed
in exception entry or return, the interrupted context runs on the
handler's values: a task gets SIGILL, or the kernel oopses on a fault
it never took.
Patch 1 runs the handler on the kernel vectors. Patch 2 saves and
restores the exception registers around it.
Handlers take such exceptions in normal operation now that SDEI delivers
NMI backtraces: show_regs() reads through _nofault() accessors that
fault, and a kprobe or WARN anywhere in the path takes a BRK.
Tested under QEMU TCG with TF-A, 4 CPUs, 20 x sysrq-l under load, with a
test-only copy_from_kernel_nofault() of address 0x10 in
sdei_nmi_handler():
- v7.4-rc1, KPTI on: CPUs stuck in the trampoline vectors, RCU stalls.
- v7.4-rc1, KPTI off: 60 "unsafe: exception during handler" warnings,
SIGILL and SIGSEGV in user tasks, and an oops in execve().
- With the series, KPTI on and off: no warnings, no stalls. The same
with an eprobe on irq_disable that reads address 0x10, which faults
inside irqentry_nmi_enter() when the event lands in user space.
An earlier revision of patch 2 was also run on arm64 hardware with
firmware SDEI, where a plain sysrq-l was enough to trigger the warning
without it.
Changes since v1
================
https://lore.kernel.org/all/20261104180936.2981070-1-kas@kernel.org/
- Switch VBAR and save the exception registers in __sdei_handler(),
before irqentry_nmi_enter(), and restore them after
irqentry_nmi_exit(). Both can take an exception too: the eprobe above
wedged v1 with KPTI on (Usama). do_sdei_event() takes the interrupted
VBAR for the IRQ-like resume.
Kiryl Shutsemau (Meta) (2):
arm64: sdei: Run the event handler on the kernel vectors
arm64: sdei: Preserve the interrupted context's exception registers
arch/arm64/include/asm/sdei.h | 3 +-
arch/arm64/kernel/entry-common.c | 40 +++++++++++++++++++++-
arch/arm64/kernel/sdei.c | 59 ++++++++++++++++++++++++++------
3 files changed, 89 insertions(+), 13 deletions(-)
Range-diff against v1:
1: f3e1e7fd09f7 ! 1: 93ef4a287a4c arm64: sdei: Run the event handler on the kernel vectors
@@ Commit message
path does it: a WARN, a kprobe hit, or a fault fixed up by a _nofault()
accessor.
- Install the kernel vectors in do_sdei_event() when VBAR_EL1 holds the
- trampoline vectors, and restore it before returning to firmware,
- including when the handler fails. The IRQ-like resume into user space
- keeps using the interrupted value, as it must go through the trampoline
- to map the kernel.
+ Install the kernel vectors in __sdei_handler() when VBAR_EL1 holds the
+ trampoline vectors, and restore it before returning to firmware. The
+ switch covers irqentry_nmi_enter() and irqentry_nmi_exit() too, as Usama
+ Arif pointed out: their tracepoints and context tracking WARNs can take
+ an exception as well.
+
+ The IRQ-like resume into user space must still go through the
+ trampoline to map the kernel. do_sdei_event() takes the interrupted
+ VBAR_EL1 and keeps basing the resume address on it.
Fixes: 79e9aa59dc29 ("arm64: sdei: Add trampoline code for remapping the kernel")
Cc: stable@vger.kernel.org
Signed-off-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
- ## arch/arm64/kernel/sdei.c ##
+ ## arch/arm64/include/asm/sdei.h ##
+@@ arch/arm64/include/asm/sdei.h: asmlinkage unsigned long __sdei_handler(struct pt_regs *regs,
+ struct sdei_registered_event *arg);
+
+ unsigned long do_sdei_event(struct pt_regs *regs,
+- struct sdei_registered_event *arg);
++ struct sdei_registered_event *arg,
++ unsigned long vbar);
+
+ unsigned long sdei_arch_get_entry_point(int conduit);
+ #define sdei_arch_get_entry_point(x) sdei_arch_get_entry_point(x)
+
+ ## arch/arm64/kernel/entry-common.c ##
@@
- #include <asm/sections.h>
#include <asm/stacktrace.h>
#include <asm/sysreg.h>
+ #include <asm/system_misc.h>
+#include <asm/vectors.h>
- #include <asm/vmap_stack.h>
- unsigned long sdei_exit_mode;
-@@ arch/arm64/kernel/sdei.c: unsigned long __kprobes do_sdei_event(struct pt_regs *regs,
- if (arm64_kernel_unmapped_at_el0())
- clobbered_registers++;
+ /*
+ * Where the context being returned to had an active percpu GPR critical
+@@ arch/arm64/kernel/entry-common.c: asmlinkage void noinstr __noreturn handle_bad_stack(struct pt_regs *regs)
+ asmlinkage noinstr unsigned long
+ __sdei_handler(struct pt_regs *regs, struct sdei_registered_event *arg)
+ {
++ unsigned long vbar;
+ irqentry_state_t state;
+ unsigned long ret;
+
+@@ arch/arm64/kernel/entry-common.c: __sdei_handler(struct pt_regs *regs, struct sdei_registered_event *arg)
+ else if (cpu_has_pan())
+ set_pstate_pan(0);
+ /*
+ * With KPTI, VBAR_EL1 holds the trampoline vectors from the return to
+ * EL0 until the next exception from EL0 maps the kernel. They only
-+ * handle exceptions from EL0: an exception taken by the handler would
-+ * run their empty current-EL half and never come back. Run the handler
-+ * on the kernel vectors. The IRQ-like resume below keeps using the
-+ * interrupted VBAR, so it still goes through the trampoline.
++ * handle exceptions from EL0: an exception taken here would run their
++ * empty current-EL half and never come back. Switch to the kernel
++ * vectors. The IRQ-like resume keeps using the interrupted VBAR, so it
++ * still goes through the trampoline.
+ */
++ vbar = read_sysreg(vbar_el1);
+ if (arm64_kernel_unmapped_at_el0() &&
-+ vbar == (unsigned long)this_cpu_read(this_cpu_vector)) {
++ vbar == (unsigned long)__this_cpu_read(this_cpu_vector)) {
+ write_sysreg((unsigned long)vectors, vbar_el1);
+ isb();
+ }
+
- /* Retrieve the missing registers values */
- for (i = 0; i < clobbered_registers; i++) {
- /* from within the handler, this call always succeeds */
-@@ arch/arm64/kernel/sdei.c: unsigned long __kprobes do_sdei_event(struct pt_regs *regs,
- }
-
- err = sdei_event_handler(regs, arg);
-- if (err)
-- return SDEI_EV_FAILED;
-
- if (elr != read_sysreg(elr_el1)) {
- /*
-@@ arch/arm64/kernel/sdei.c: unsigned long __kprobes do_sdei_event(struct pt_regs *regs,
- pr_warn("unsafe: exception during handler\n");
- }
+ state = irqentry_nmi_enter(regs);
+- ret = do_sdei_event(regs, arg);
++ ret = do_sdei_event(regs, arg, vbar);
+ irqentry_nmi_exit(regs, state);
+ write_sysreg(vbar, vbar_el1);
+
-+ if (err)
-+ return SDEI_EV_FAILED;
-+
- mode = regs->pstate & (PSR_MODE32_BIT | PSR_MODE_MASK);
+ return ret;
+ }
+ #endif /* CONFIG_ARM_SDE_INTERFACE */
+
+ ## arch/arm64/kernel/sdei.c ##
+@@ arch/arm64/kernel/sdei.c: unsigned long sdei_arch_get_entry_point(int conduit)
+ * SDEI_EV_HANDLED - success, return to the interrupted context.
+ * SDEI_EV_FAILED - failure, return this error code to firmware.
+ * virtual-address - success, return to this address.
++ *
++ * vbar is the interrupted context's VBAR_EL1, for the IRQ-like resume.
+ */
+ unsigned long __kprobes do_sdei_event(struct pt_regs *regs,
+- struct sdei_registered_event *arg)
++ struct sdei_registered_event *arg,
++ unsigned long vbar)
+ {
+ u32 mode;
+ int i, err = 0;
+ int clobbered_registers = 4;
+ u64 elr = read_sysreg(elr_el1);
+ u32 kernel_mode = read_sysreg(CurrentEL) | 1; /* +SPSel */
+- unsigned long vbar = read_sysreg(vbar_el1);
- /*
+ if (arm64_kernel_unmapped_at_el0())
+ clobbered_registers++;
2: ba422c8220e5 ! 2: 07d3078373ee arm64: sdei: Preserve the interrupted context's exception registers
@@ Commit message
kworker that is not running a work item, and a single sysrq-l warned
once per such kworker.
- Save ELR, SPSR, ESR and FAR on entry to do_sdei_event() and restore them
- before returning to firmware.
+ Save ELR, SPSR, ESR and FAR in __sdei_handler() before
+ irqentry_nmi_enter(), which can take an exception too, and restore them
+ after irqentry_nmi_exit().
Once they are restored, a nested exception is harmless if its handler is
safe in NMI context. That holds for EL1 data aborts on translation,
@@ Commit message
Cc: stable@vger.kernel.org
Signed-off-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
+ ## arch/arm64/kernel/entry-common.c ##
+@@ arch/arm64/kernel/entry-common.c: asmlinkage void noinstr __noreturn handle_bad_stack(struct pt_regs *regs)
+ asmlinkage noinstr unsigned long
+ __sdei_handler(struct pt_regs *regs, struct sdei_registered_event *arg)
+ {
+- unsigned long vbar;
++ unsigned long elr, spsr, esr, far, vbar;
+ irqentry_state_t state;
+ unsigned long ret;
+
+@@ arch/arm64/kernel/entry-common.c: __sdei_handler(struct pt_regs *regs, struct sdei_registered_event *arg)
+ else if (cpu_has_pan())
+ set_pstate_pan(0);
+
++ /*
++ * Nothing saved the interrupted context's exception registers on the
++ * way in, and it may be in the middle of exception entry or return
++ * (SDEI ignores DAIF). Save them before anything can take an exception
++ * and overwrite them.
++ */
++ elr = read_sysreg(elr_el1);
++ spsr = read_sysreg(spsr_el1);
++ esr = read_sysreg(esr_el1);
++ far = read_sysreg(far_el1);
++
+ /*
+ * With KPTI, VBAR_EL1 holds the trampoline vectors from the return to
+ * EL0 until the next exception from EL0 maps the kernel. They only
+@@ arch/arm64/kernel/entry-common.c: __sdei_handler(struct pt_regs *regs, struct sdei_registered_event *arg)
+ ret = do_sdei_event(regs, arg, vbar);
+ irqentry_nmi_exit(regs, state);
+
++ /*
++ * Put them back unconditionally. An unchanged ELR does not rule out a
++ * nested exception: one may have returned to the same address.
++ */
++ write_sysreg(elr, elr_el1);
++ write_sysreg(spsr, spsr_el1);
++ write_sysreg(esr, esr_el1);
++ write_sysreg(far, far_el1);
+ write_sysreg(vbar, vbar_el1);
+
+ return ret;
+
## arch/arm64/kernel/sdei.c ##
@@ arch/arm64/kernel/sdei.c: unsigned long sdei_arch_get_entry_point(int conduit)
return 0;
}
+/*
-+ * Exceptions an SDEI handler may take without harm once do_sdei_event() has
++ * Exceptions an SDEI handler may take without harm once __sdei_handler() has
+ * restored the interrupted context's exception registers. Their handlers are
+ * safe in NMI context:
+ * - EL1 data aborts on translation, permission and access-flag faults:
@@ arch/arm64/kernel/sdei.c: unsigned long __kprobes do_sdei_event(struct pt_regs *
int i, err = 0;
int clobbered_registers = 4;
u64 elr = read_sysreg(elr_el1);
-+ u64 spsr = read_sysreg(spsr_el1);
-+ u64 esr = read_sysreg(esr_el1);
-+ u64 far = read_sysreg(far_el1);
+ u64 nested_elr, nested_esr, nested_far;
u32 kernel_mode = read_sysreg(CurrentEL) | 1; /* +SPSel */
- unsigned long vbar = read_sysreg(vbar_el1);
+ if (arm64_kernel_unmapped_at_el0())
@@ arch/arm64/kernel/sdei.c: unsigned long __kprobes do_sdei_event(struct pt_regs *regs,
+ }
err = sdei_event_handler(regs, arg);
-
-- if (elr != read_sysreg(elr_el1)) {
-- /*
-- * We took a synchronous exception from the SDEI handler.
-- * This could deadlock, and if you interrupt KVM it will
-- * hyp-panic instead.
-- */
-- pr_warn("unsafe: exception during handler\n");
++
+ /*
-+ * SDEI is not an architectural exception: nothing saved the
-+ * interrupted context's exception registers on the way in, and it may
-+ * be in the middle of exception entry or return (SDEI ignores DAIF).
-+ * Any synchronous exception the handler took has overwritten them.
-+ * Only the last one's syndrome survives; report it if it is not one
-+ * known to be harmless.
++ * A synchronous exception taken by the handler has overwritten the
++ * exception registers; __sdei_handler() puts them back. Only the last
++ * one's syndrome survives; report it if it is not one known to be
++ * harmless.
+ *
+ * If the event interrupted a KVM world switch, VBAR points at the KVM
+ * hyp vectors and any such exception is a hyp panic instead.
@@ arch/arm64/kernel/sdei.c: unsigned long __kprobes do_sdei_event(struct pt_regs *
+ if (nested_elr != elr && !sdei_nested_exception_ok(nested_esr)) {
+ pr_warn_once("unexpected exception during handler: ESR %#llx FAR %#llx near %pS\n",
+ nested_esr, nested_far, (void *)nested_elr);
- }
-
-+ /*
-+ * Put them back unconditionally. An unchanged ELR does not rule out a
-+ * nested exception: one may have returned to the same address.
-+ */
-+ write_sysreg(elr, elr_el1);
-+ write_sysreg(spsr, spsr_el1);
-+ write_sysreg(esr, esr_el1);
-+ write_sysreg(far, far_el1);
- write_sysreg(vbar, vbar_el1);
-
++ }
++
if (err)
+ return SDEI_EV_FAILED;
+
+- if (elr != read_sysreg(elr_el1)) {
+- /*
+- * We took a synchronous exception from the SDEI handler.
+- * This could deadlock, and if you interrupt KVM it will
+- * hyp-panic instead.
+- */
+- pr_warn("unsafe: exception during handler\n");
+- }
+-
+ mode = regs->pstate & (PSR_MODE32_BIT | PSR_MODE_MASK);
+
+ /*
base-commit: 6628e7d4c6b25c8d18f1345e3a20d21829aae7e4
--
2.54.0
^ permalink raw reply [flat|nested] 5+ messages in thread* [PATCH v2 1/2] arm64: sdei: Run the event handler on the kernel vectors
2026-11-05 12:08 [PATCH v2 0/2] arm64: sdei: Handle exceptions taken inside the event handler Kiryl Shutsemau
@ 2026-11-05 12:08 ` Kiryl Shutsemau
2026-11-05 18:37 ` Catalin Marinas
2026-11-05 12:08 ` [PATCH v2 2/2] arm64: sdei: Preserve the interrupted context's exception registers Kiryl Shutsemau
1 sibling, 1 reply; 5+ messages in thread
From: Kiryl Shutsemau @ 2026-11-05 12:08 UTC (permalink / raw)
To: Catalin Marinas, Will Deacon, Mark Rutland, James Morse,
Usama Arif
Cc: Kiryl Shutsemau (Meta), Thomas Huth, Vladimir Murzin,
Peter Zijlstra, Marc Zyngier, Jinjie Ruan, kernel-team,
Doug Anderson, stable, linux-kernel, mrigendrachaubey,
linux-arm-kernel, Breno Leitao, Linus Walleij, Dawei Li
From: "Kiryl Shutsemau (Meta)" <kas@kernel.org>
With KPTI, VBAR_EL1 holds the trampoline vectors from the return to EL0
until the next exception from EL0 maps the kernel and installs the
kernel vectors. The trampoline vectors only handle exceptions from EL0;
their current-EL half is zero-filled.
An SDEI event is dispatched regardless of PSTATE.DAIF, so it lands there
whenever the CPU is in user space. The SDEI entry trampoline maps the
kernel but leaves VBAR_EL1 alone. An exception taken by the handler then
executes the zeros, takes an undefined instruction exception to the same
vector, and loops there forever. The event never completes and the CPU
is lost; RCU stall reports are the first sign of it.
RAS notifications land in user space whenever a task consumes the
error, and NMI backtraces land wherever the CPU happens to be. The GHES
handler takes no exception on its own, but any exception in the handler
path does it: a WARN, a kprobe hit, or a fault fixed up by a _nofault()
accessor.
Install the kernel vectors in __sdei_handler() when VBAR_EL1 holds the
trampoline vectors, and restore it before returning to firmware. The
switch covers irqentry_nmi_enter() and irqentry_nmi_exit() too, as Usama
Arif pointed out: their tracepoints and context tracking WARNs can take
an exception as well.
The IRQ-like resume into user space must still go through the
trampoline to map the kernel. do_sdei_event() takes the interrupted
VBAR_EL1 and keeps basing the resume address on it.
Fixes: 79e9aa59dc29 ("arm64: sdei: Add trampoline code for remapping the kernel")
Cc: stable@vger.kernel.org
Signed-off-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
---
arch/arm64/include/asm/sdei.h | 3 ++-
arch/arm64/kernel/entry-common.c | 21 ++++++++++++++++++++-
arch/arm64/kernel/sdei.c | 6 ++++--
3 files changed, 26 insertions(+), 4 deletions(-)
diff --git a/arch/arm64/include/asm/sdei.h b/arch/arm64/include/asm/sdei.h
index b2248bd3cb58..a78a5337ff85 100644
--- a/arch/arm64/include/asm/sdei.h
+++ b/arch/arm64/include/asm/sdei.h
@@ -44,7 +44,8 @@ asmlinkage unsigned long __sdei_handler(struct pt_regs *regs,
struct sdei_registered_event *arg);
unsigned long do_sdei_event(struct pt_regs *regs,
- struct sdei_registered_event *arg);
+ struct sdei_registered_event *arg,
+ unsigned long vbar);
unsigned long sdei_arch_get_entry_point(int conduit);
#define sdei_arch_get_entry_point(x) sdei_arch_get_entry_point(x)
diff --git a/arch/arm64/kernel/entry-common.c b/arch/arm64/kernel/entry-common.c
index 11d3f888fcd8..8a17e3720f81 100644
--- a/arch/arm64/kernel/entry-common.c
+++ b/arch/arm64/kernel/entry-common.c
@@ -31,6 +31,7 @@
#include <asm/stacktrace.h>
#include <asm/sysreg.h>
#include <asm/system_misc.h>
+#include <asm/vectors.h>
/*
* Where the context being returned to had an active percpu GPR critical
@@ -1040,6 +1041,7 @@ asmlinkage void noinstr __noreturn handle_bad_stack(struct pt_regs *regs)
asmlinkage noinstr unsigned long
__sdei_handler(struct pt_regs *regs, struct sdei_registered_event *arg)
{
+ unsigned long vbar;
irqentry_state_t state;
unsigned long ret;
@@ -1065,10 +1067,27 @@ __sdei_handler(struct pt_regs *regs, struct sdei_registered_event *arg)
else if (cpu_has_pan())
set_pstate_pan(0);
+ /*
+ * With KPTI, VBAR_EL1 holds the trampoline vectors from the return to
+ * EL0 until the next exception from EL0 maps the kernel. They only
+ * handle exceptions from EL0: an exception taken here would run their
+ * empty current-EL half and never come back. Switch to the kernel
+ * vectors. The IRQ-like resume keeps using the interrupted VBAR, so it
+ * still goes through the trampoline.
+ */
+ vbar = read_sysreg(vbar_el1);
+ if (arm64_kernel_unmapped_at_el0() &&
+ vbar == (unsigned long)__this_cpu_read(this_cpu_vector)) {
+ write_sysreg((unsigned long)vectors, vbar_el1);
+ isb();
+ }
+
state = irqentry_nmi_enter(regs);
- ret = do_sdei_event(regs, arg);
+ ret = do_sdei_event(regs, arg, vbar);
irqentry_nmi_exit(regs, state);
+ write_sysreg(vbar, vbar_el1);
+
return ret;
}
#endif /* CONFIG_ARM_SDE_INTERFACE */
diff --git a/arch/arm64/kernel/sdei.c b/arch/arm64/kernel/sdei.c
index 778f2a1faac8..260c13106009 100644
--- a/arch/arm64/kernel/sdei.c
+++ b/arch/arm64/kernel/sdei.c
@@ -200,16 +200,18 @@ unsigned long sdei_arch_get_entry_point(int conduit)
* SDEI_EV_HANDLED - success, return to the interrupted context.
* SDEI_EV_FAILED - failure, return this error code to firmware.
* virtual-address - success, return to this address.
+ *
+ * vbar is the interrupted context's VBAR_EL1, for the IRQ-like resume.
*/
unsigned long __kprobes do_sdei_event(struct pt_regs *regs,
- struct sdei_registered_event *arg)
+ struct sdei_registered_event *arg,
+ unsigned long vbar)
{
u32 mode;
int i, err = 0;
int clobbered_registers = 4;
u64 elr = read_sysreg(elr_el1);
u32 kernel_mode = read_sysreg(CurrentEL) | 1; /* +SPSel */
- unsigned long vbar = read_sysreg(vbar_el1);
if (arm64_kernel_unmapped_at_el0())
clobbered_registers++;
--
2.54.0
^ permalink raw reply related [flat|nested] 5+ messages in thread* Re: [PATCH v2 1/2] arm64: sdei: Run the event handler on the kernel vectors
2026-11-05 12:08 ` [PATCH v2 1/2] arm64: sdei: Run the event handler on the kernel vectors Kiryl Shutsemau
@ 2026-11-05 18:37 ` Catalin Marinas
0 siblings, 0 replies; 5+ messages in thread
From: Catalin Marinas @ 2026-11-05 18:37 UTC (permalink / raw)
To: Kiryl Shutsemau
Cc: Mark Rutland, Kiryl Shutsemau (Meta), Thomas Huth,
Vladimir Murzin, Usama Arif, Peter Zijlstra, Marc Zyngier,
Jinjie Ruan, kernel-team, Doug Anderson, stable, linux-kernel,
James Morse, linux-arm-kernel, Breno Leitao, mrigendrachaubey,
Will Deacon, Linus Walleij, Dawei Li
On Thu, Nov 05, 2026 at 12:08:33PM +0000, Kiryl Shutsemau wrote:
> @@ -1065,10 +1067,27 @@ __sdei_handler(struct pt_regs *regs, struct sdei_registered_event *arg)
> else if (cpu_has_pan())
> set_pstate_pan(0);
>
> + /*
> + * With KPTI, VBAR_EL1 holds the trampoline vectors from the return to
> + * EL0 until the next exception from EL0 maps the kernel. They only
> + * handle exceptions from EL0: an exception taken here would run their
> + * empty current-EL half and never come back. Switch to the kernel
> + * vectors. The IRQ-like resume keeps using the interrupted VBAR, so it
> + * still goes through the trampoline.
> + */
> + vbar = read_sysreg(vbar_el1);
> + if (arm64_kernel_unmapped_at_el0() &&
> + vbar == (unsigned long)__this_cpu_read(this_cpu_vector)) {
> + write_sysreg((unsigned long)vectors, vbar_el1);
> + isb();
> + }
Sashiko had a comment that the above should be raw_cpu_read() since with
preempt debugging and pseudo-NMIs, irqs_disabled() is false here and the
check in __this_cpu_read() will warn.
Thinking about it, further down on the irqentry_nmi_enter() ->
lockdep_hardirqs_off() we'll hit the irqs_disabled() issue again. We
could just do a raw_local_irq_save/restore() around this function.
Unnecessary for DAIF but useful with pseudo-NMIs.
Alternatively, __sdei_asm_handler() could save/restore PMR and change
the priorities similar to the kernel_entry code. Actually, this may be
better since do_sdei_event() also checks regs_irqs_disabled() and we
don't have any regs->pmr populated.
Worth a separate patch to fix SDEI handling with pseudo-NMI on and no
changes to __this_cpu_read() above needed. So, for this patch:
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
(I'll probably queue it for -rc but waiting to see if there are more
comments)
--
Catalin
^ permalink raw reply [flat|nested] 5+ messages in thread
* [PATCH v2 2/2] arm64: sdei: Preserve the interrupted context's exception registers
2026-11-05 12:08 [PATCH v2 0/2] arm64: sdei: Handle exceptions taken inside the event handler Kiryl Shutsemau
2026-11-05 12:08 ` [PATCH v2 1/2] arm64: sdei: Run the event handler on the kernel vectors Kiryl Shutsemau
@ 2026-11-05 12:08 ` Kiryl Shutsemau
2026-11-05 15:41 ` Breno Leitao
1 sibling, 1 reply; 5+ messages in thread
From: Kiryl Shutsemau @ 2026-11-05 12:08 UTC (permalink / raw)
To: Catalin Marinas, Will Deacon, Mark Rutland, James Morse,
Usama Arif
Cc: Kiryl Shutsemau (Meta), Thomas Huth, Vladimir Murzin,
Peter Zijlstra, Marc Zyngier, Jinjie Ruan, kernel-team,
Doug Anderson, stable, linux-kernel, mrigendrachaubey,
linux-arm-kernel, Breno Leitao, Linus Walleij, Dawei Li
From: "Kiryl Shutsemau (Meta)" <kas@kernel.org>
An SDEI event is dispatched by firmware regardless of PSTATE.DAIF, so it
can interrupt the kernel where the exception registers are live:
- exception entry, before kernel_entry has saved ELR_EL1/SPSR_EL1 and
before the C handler has read ESR_EL1/FAR_EL1;
- exception return, after kernel_exit has loaded ELR_EL1/SPSR_EL1 and
before the ERET.
An IRQ or pseudo-NMI cannot arrive there with PSTATE.I set. Firmware
saves only x0-x17, PC and PSTATE around an SDEI event, so a synchronous
exception taken by the handler overwrites the interrupted context's
ELR, SPSR, ESR and FAR. do_sdei_event() only notices that ELR changed,
and warns.
The interrupted context then carries on with the handler's state. A
task interrupted on its way into the kernel can be sent SIGILL for the
handler's kernel data abort. A task interrupted on its way back to user
space ERETs to EL1, at the handler's fixup address, with user registers
loaded.
With SDEI delivering NMI backtraces, this happens in normal operation.
The handler runs show_regs(), which reads through possibly-NULL pointers
with the _nofault() accessors. print_worker_info() faults this way for a
kworker that is not running a work item, and a single sysrq-l warned
once per such kworker.
Save ELR, SPSR, ESR and FAR in __sdei_handler() before
irqentry_nmi_enter(), which can take an exception too, and restore them
after irqentry_nmi_exit().
Once they are restored, a nested exception is harmless if its handler is
safe in NMI context. That holds for EL1 data aborts on translation,
permission and access-flag faults, which never take the mmap lock in NMI
context, and for BRK. Warn once, with the syndrome, about any other
exception class.
Fixes: f5df26961853 ("arm64: kernel: Add arch-specific SDEI entry code and CPU masking")
Cc: stable@vger.kernel.org
Signed-off-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
---
arch/arm64/kernel/entry-common.c | 21 ++++++++++++-
arch/arm64/kernel/sdei.c | 53 ++++++++++++++++++++++++++------
2 files changed, 64 insertions(+), 10 deletions(-)
diff --git a/arch/arm64/kernel/entry-common.c b/arch/arm64/kernel/entry-common.c
index 8a17e3720f81..64d287e1533b 100644
--- a/arch/arm64/kernel/entry-common.c
+++ b/arch/arm64/kernel/entry-common.c
@@ -1041,7 +1041,7 @@ asmlinkage void noinstr __noreturn handle_bad_stack(struct pt_regs *regs)
asmlinkage noinstr unsigned long
__sdei_handler(struct pt_regs *regs, struct sdei_registered_event *arg)
{
- unsigned long vbar;
+ unsigned long elr, spsr, esr, far, vbar;
irqentry_state_t state;
unsigned long ret;
@@ -1067,6 +1067,17 @@ __sdei_handler(struct pt_regs *regs, struct sdei_registered_event *arg)
else if (cpu_has_pan())
set_pstate_pan(0);
+ /*
+ * Nothing saved the interrupted context's exception registers on the
+ * way in, and it may be in the middle of exception entry or return
+ * (SDEI ignores DAIF). Save them before anything can take an exception
+ * and overwrite them.
+ */
+ elr = read_sysreg(elr_el1);
+ spsr = read_sysreg(spsr_el1);
+ esr = read_sysreg(esr_el1);
+ far = read_sysreg(far_el1);
+
/*
* With KPTI, VBAR_EL1 holds the trampoline vectors from the return to
* EL0 until the next exception from EL0 maps the kernel. They only
@@ -1086,6 +1097,14 @@ __sdei_handler(struct pt_regs *regs, struct sdei_registered_event *arg)
ret = do_sdei_event(regs, arg, vbar);
irqentry_nmi_exit(regs, state);
+ /*
+ * Put them back unconditionally. An unchanged ELR does not rule out a
+ * nested exception: one may have returned to the same address.
+ */
+ write_sysreg(elr, elr_el1);
+ write_sysreg(spsr, spsr_el1);
+ write_sysreg(esr, esr_el1);
+ write_sysreg(far, far_el1);
write_sysreg(vbar, vbar_el1);
return ret;
diff --git a/arch/arm64/kernel/sdei.c b/arch/arm64/kernel/sdei.c
index 260c13106009..1b902da8ba43 100644
--- a/arch/arm64/kernel/sdei.c
+++ b/arch/arm64/kernel/sdei.c
@@ -195,6 +195,31 @@ unsigned long sdei_arch_get_entry_point(int conduit)
return 0;
}
+/*
+ * Exceptions an SDEI handler may take without harm once __sdei_handler() has
+ * restored the interrupted context's exception registers. Their handlers are
+ * safe in NMI context:
+ * - EL1 data aborts on translation, permission and access-flag faults:
+ * faulthandler_disabled() is true in NMI context, so do_page_fault() never
+ * takes the mmap lock. This covers the _nofault() accessors used by the
+ * generic dump_stack()/show_regs() code (e.g. print_worker_info()).
+ * - BRK: kprobes, WARN/BUG, kCFI and UBSAN traps, all NMI-capable.
+ * Anything else taken at EL1 is fatal on its own, or unexpected here.
+ */
+static bool sdei_nested_exception_ok(unsigned long esr)
+{
+ switch (ESR_ELx_EC(esr)) {
+ case ESR_ELx_EC_DABT_CUR:
+ return esr_fsc_is_translation_fault(esr) ||
+ esr_fsc_is_permission_fault(esr) ||
+ esr_fsc_is_access_flag_fault(esr);
+ case ESR_ELx_EC_BRK64:
+ return true;
+ default:
+ return false;
+ }
+}
+
/*
* do_sdei_event() returns one of:
* SDEI_EV_HANDLED - success, return to the interrupted context.
@@ -211,6 +236,7 @@ unsigned long __kprobes do_sdei_event(struct pt_regs *regs,
int i, err = 0;
int clobbered_registers = 4;
u64 elr = read_sysreg(elr_el1);
+ u64 nested_elr, nested_esr, nested_far;
u32 kernel_mode = read_sysreg(CurrentEL) | 1; /* +SPSel */
if (arm64_kernel_unmapped_at_el0())
@@ -223,18 +249,27 @@ unsigned long __kprobes do_sdei_event(struct pt_regs *regs,
}
err = sdei_event_handler(regs, arg);
+
+ /*
+ * A synchronous exception taken by the handler has overwritten the
+ * exception registers; __sdei_handler() puts them back. Only the last
+ * one's syndrome survives; report it if it is not one known to be
+ * harmless.
+ *
+ * If the event interrupted a KVM world switch, VBAR points at the KVM
+ * hyp vectors and any such exception is a hyp panic instead.
+ */
+ nested_elr = read_sysreg(elr_el1);
+ nested_esr = read_sysreg(esr_el1);
+ nested_far = read_sysreg(far_el1);
+ if (nested_elr != elr && !sdei_nested_exception_ok(nested_esr)) {
+ pr_warn_once("unexpected exception during handler: ESR %#llx FAR %#llx near %pS\n",
+ nested_esr, nested_far, (void *)nested_elr);
+ }
+
if (err)
return SDEI_EV_FAILED;
- if (elr != read_sysreg(elr_el1)) {
- /*
- * We took a synchronous exception from the SDEI handler.
- * This could deadlock, and if you interrupt KVM it will
- * hyp-panic instead.
- */
- pr_warn("unsafe: exception during handler\n");
- }
-
mode = regs->pstate & (PSR_MODE32_BIT | PSR_MODE_MASK);
/*
--
2.54.0
^ permalink raw reply related [flat|nested] 5+ messages in thread* Re: [PATCH v2 2/2] arm64: sdei: Preserve the interrupted context's exception registers
2026-11-05 12:08 ` [PATCH v2 2/2] arm64: sdei: Preserve the interrupted context's exception registers Kiryl Shutsemau
@ 2026-11-05 15:41 ` Breno Leitao
0 siblings, 0 replies; 5+ messages in thread
From: Breno Leitao @ 2026-11-05 15:41 UTC (permalink / raw)
To: Kiryl Shutsemau
Cc: Mark Rutland, Kiryl Shutsemau (Meta), Thomas Huth,
Vladimir Murzin, Usama Arif, Peter Zijlstra, Catalin Marinas,
Jinjie Ruan, kernel-team, Doug Anderson, stable, linux-kernel,
James Morse, linux-arm-kernel, Marc Zyngier, mrigendrachaubey,
Will Deacon, Linus Walleij, Dawei Li
On Thu, Nov 05, 2026 at 12:08:34PM +0000, Kiryl Shutsemau wrote:
> From: "Kiryl Shutsemau (Meta)" <kas@kernel.org>
>
> An SDEI event is dispatched by firmware regardless of PSTATE.DAIF, so it
> can interrupt the kernel where the exception registers are live:
>
> - exception entry, before kernel_entry has saved ELR_EL1/SPSR_EL1 and
> before the C handler has read ESR_EL1/FAR_EL1;
> - exception return, after kernel_exit has loaded ELR_EL1/SPSR_EL1 and
> before the ERET.
>
> An IRQ or pseudo-NMI cannot arrive there with PSTATE.I set. Firmware
> saves only x0-x17, PC and PSTATE around an SDEI event, so a synchronous
> exception taken by the handler overwrites the interrupted context's
> ELR, SPSR, ESR and FAR. do_sdei_event() only notices that ELR changed,
> and warns.
>
> The interrupted context then carries on with the handler's state. A
> task interrupted on its way into the kernel can be sent SIGILL for the
> handler's kernel data abort. A task interrupted on its way back to user
> space ERETs to EL1, at the handler's fixup address, with user registers
> loaded.
>
> With SDEI delivering NMI backtraces, this happens in normal operation.
> The handler runs show_regs(), which reads through possibly-NULL pointers
> with the _nofault() accessors. print_worker_info() faults this way for a
> kworker that is not running a work item, and a single sysrq-l warned
> once per such kworker.
>
> Save ELR, SPSR, ESR and FAR in __sdei_handler() before
> irqentry_nmi_enter(), which can take an exception too, and restore them
> after irqentry_nmi_exit().
>
> Once they are restored, a nested exception is harmless if its handler is
> safe in NMI context. That holds for EL1 data aborts on translation,
> permission and access-flag faults, which never take the mmap lock in NMI
> context, and for BRK. Warn once, with the syndrome, about any other
> exception class.
>
> Fixes: f5df26961853 ("arm64: kernel: Add arch-specific SDEI entry code and CPU masking")
> Cc: stable@vger.kernel.org
> Signed-off-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
Tested-by: Breno Leitao <leitao@debian.org>
Tested on NVIDIA Grace (firmware SDEI) with next-20261104, KASAN + lockdep,
ARM_SDEI_NMI=y: sysrq-l under stress-ng, with a kprobe on show_regs() and
eprobes on irq_disable/irq_enable that fault on purpose.
Without the series the box crashed (a task interrupted in el0t_64_sync
got the handler's BRK, then lockdep/KASAN fallout and a hard lockup)
With it, KPTI off and kpti=1, ~4000 kprobe hits and ~5500 faults in
irqentry_nmi_enter()/exit() produced no warnings or stalls.
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2026-11-05 18:38 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-11-05 12:08 [PATCH v2 0/2] arm64: sdei: Handle exceptions taken inside the event handler Kiryl Shutsemau
2026-11-05 12:08 ` [PATCH v2 1/2] arm64: sdei: Run the event handler on the kernel vectors Kiryl Shutsemau
2026-11-05 18:37 ` Catalin Marinas
2026-11-05 12:08 ` [PATCH v2 2/2] arm64: sdei: Preserve the interrupted context's exception registers Kiryl Shutsemau
2026-11-05 15:41 ` Breno Leitao
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.