Linux-ARM-Kernel Archive on lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH v2 0/2] arm64: sdei: Handle exceptions taken inside the event handler
@ 2026-11-05 12:08 Kiryl Shutsemau
  2026-11-05 12:08 ` [PATCH v2 1/2] arm64: sdei: Run the event handler on the kernel vectors Kiryl Shutsemau
  2026-11-05 12:08 ` [PATCH v2 2/2] arm64: sdei: Preserve the interrupted context's exception registers Kiryl Shutsemau
  0 siblings, 2 replies; 4+ messages in thread
From: Kiryl Shutsemau @ 2026-11-05 12:08 UTC (permalink / raw)
  To: Catalin Marinas, Will Deacon, Mark Rutland, James Morse,
	Usama Arif
  Cc: Kiryl Shutsemau (Meta), Thomas Huth, Vladimir Murzin,
	Peter Zijlstra, Marc Zyngier, Jinjie Ruan, kernel-team,
	Doug Anderson, linux-kernel, mrigendrachaubey, linux-arm-kernel,
	Breno Leitao, Linus Walleij, Dawei Li

From: "Kiryl Shutsemau (Meta)" <kas@kernel.org>

An SDEI event can land anywhere, and a synchronous exception taken by its
handler breaks the context it interrupted:

 - With KPTI, an event that lands in user space runs the handler on the
   trampoline vectors, whose current-EL half is empty. The exception
   loops there and the CPU is lost.

 - The exception overwrites ELR, SPSR, ESR and FAR. If the event landed
   in exception entry or return, the interrupted context runs on the
   handler's values: a task gets SIGILL, or the kernel oopses on a fault
   it never took.

Patch 1 runs the handler on the kernel vectors. Patch 2 saves and
restores the exception registers around it.

Handlers take such exceptions in normal operation now that SDEI delivers
NMI backtraces: show_regs() reads through _nofault() accessors that
fault, and a kprobe or WARN anywhere in the path takes a BRK.

Tested under QEMU TCG with TF-A, 4 CPUs, 20 x sysrq-l under load, with a
test-only copy_from_kernel_nofault() of address 0x10 in
sdei_nmi_handler():

 - v7.4-rc1, KPTI on: CPUs stuck in the trampoline vectors, RCU stalls.
 - v7.4-rc1, KPTI off: 60 "unsafe: exception during handler" warnings,
   SIGILL and SIGSEGV in user tasks, and an oops in execve().
 - With the series, KPTI on and off: no warnings, no stalls. The same
   with an eprobe on irq_disable that reads address 0x10, which faults
   inside irqentry_nmi_enter() when the event lands in user space.

An earlier revision of patch 2 was also run on arm64 hardware with
firmware SDEI, where a plain sysrq-l was enough to trigger the warning
without it.

Changes since v1
================

  https://lore.kernel.org/all/20261104180936.2981070-1-kas@kernel.org/

 - Switch VBAR and save the exception registers in __sdei_handler(),
   before irqentry_nmi_enter(), and restore them after
   irqentry_nmi_exit(). Both can take an exception too: the eprobe above
   wedged v1 with KPTI on (Usama). do_sdei_event() takes the interrupted
   VBAR for the IRQ-like resume.

Kiryl Shutsemau (Meta) (2):
  arm64: sdei: Run the event handler on the kernel vectors
  arm64: sdei: Preserve the interrupted context's exception registers

 arch/arm64/include/asm/sdei.h    |  3 +-
 arch/arm64/kernel/entry-common.c | 40 +++++++++++++++++++++-
 arch/arm64/kernel/sdei.c         | 59 ++++++++++++++++++++++++++------
 3 files changed, 89 insertions(+), 13 deletions(-)

Range-diff against v1:
1:  f3e1e7fd09f7 ! 1:  93ef4a287a4c arm64: sdei: Run the event handler on the kernel vectors
    @@ Commit message
         path does it: a WARN, a kprobe hit, or a fault fixed up by a _nofault()
         accessor.
     
    -    Install the kernel vectors in do_sdei_event() when VBAR_EL1 holds the
    -    trampoline vectors, and restore it before returning to firmware,
    -    including when the handler fails. The IRQ-like resume into user space
    -    keeps using the interrupted value, as it must go through the trampoline
    -    to map the kernel.
    +    Install the kernel vectors in __sdei_handler() when VBAR_EL1 holds the
    +    trampoline vectors, and restore it before returning to firmware. The
    +    switch covers irqentry_nmi_enter() and irqentry_nmi_exit() too, as Usama
    +    Arif pointed out: their tracepoints and context tracking WARNs can take
    +    an exception as well.
    +
    +    The IRQ-like resume into user space must still go through the
    +    trampoline to map the kernel. do_sdei_event() takes the interrupted
    +    VBAR_EL1 and keeps basing the resume address on it.
     
         Fixes: 79e9aa59dc29 ("arm64: sdei: Add trampoline code for remapping the kernel")
         Cc: stable@vger.kernel.org
         Signed-off-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
     
    - ## arch/arm64/kernel/sdei.c ##
    + ## arch/arm64/include/asm/sdei.h ##
    +@@ arch/arm64/include/asm/sdei.h: asmlinkage unsigned long __sdei_handler(struct pt_regs *regs,
    + 					struct sdei_registered_event *arg);
    + 
    + unsigned long do_sdei_event(struct pt_regs *regs,
    +-			    struct sdei_registered_event *arg);
    ++			    struct sdei_registered_event *arg,
    ++			    unsigned long vbar);
    + 
    + unsigned long sdei_arch_get_entry_point(int conduit);
    + #define sdei_arch_get_entry_point(x)	sdei_arch_get_entry_point(x)
    +
    + ## arch/arm64/kernel/entry-common.c ##
     @@
    - #include <asm/sections.h>
      #include <asm/stacktrace.h>
      #include <asm/sysreg.h>
    + #include <asm/system_misc.h>
     +#include <asm/vectors.h>
    - #include <asm/vmap_stack.h>
      
    - unsigned long sdei_exit_mode;
    -@@ arch/arm64/kernel/sdei.c: unsigned long __kprobes do_sdei_event(struct pt_regs *regs,
    - 	if (arm64_kernel_unmapped_at_el0())
    - 		clobbered_registers++;
    + /*
    +  * Where the context being returned to had an active percpu GPR critical
    +@@ arch/arm64/kernel/entry-common.c: asmlinkage void noinstr __noreturn handle_bad_stack(struct pt_regs *regs)
    + asmlinkage noinstr unsigned long
    + __sdei_handler(struct pt_regs *regs, struct sdei_registered_event *arg)
    + {
    ++	unsigned long vbar;
    + 	irqentry_state_t state;
    + 	unsigned long ret;
    + 
    +@@ arch/arm64/kernel/entry-common.c: __sdei_handler(struct pt_regs *regs, struct sdei_registered_event *arg)
    + 	else if (cpu_has_pan())
    + 		set_pstate_pan(0);
      
     +	/*
     +	 * With KPTI, VBAR_EL1 holds the trampoline vectors from the return to
     +	 * EL0 until the next exception from EL0 maps the kernel. They only
    -+	 * handle exceptions from EL0: an exception taken by the handler would
    -+	 * run their empty current-EL half and never come back. Run the handler
    -+	 * on the kernel vectors. The IRQ-like resume below keeps using the
    -+	 * interrupted VBAR, so it still goes through the trampoline.
    ++	 * handle exceptions from EL0: an exception taken here would run their
    ++	 * empty current-EL half and never come back. Switch to the kernel
    ++	 * vectors. The IRQ-like resume keeps using the interrupted VBAR, so it
    ++	 * still goes through the trampoline.
     +	 */
    ++	vbar = read_sysreg(vbar_el1);
     +	if (arm64_kernel_unmapped_at_el0() &&
    -+	    vbar == (unsigned long)this_cpu_read(this_cpu_vector)) {
    ++	    vbar == (unsigned long)__this_cpu_read(this_cpu_vector)) {
     +		write_sysreg((unsigned long)vectors, vbar_el1);
     +		isb();
     +	}
     +
    - 	/* Retrieve the missing registers values */
    - 	for (i = 0; i < clobbered_registers; i++) {
    - 		/* from within the handler, this call always succeeds */
    -@@ arch/arm64/kernel/sdei.c: unsigned long __kprobes do_sdei_event(struct pt_regs *regs,
    - 	}
    - 
    - 	err = sdei_event_handler(regs, arg);
    --	if (err)
    --		return SDEI_EV_FAILED;
    - 
    - 	if (elr != read_sysreg(elr_el1)) {
    - 		/*
    -@@ arch/arm64/kernel/sdei.c: unsigned long __kprobes do_sdei_event(struct pt_regs *regs,
    - 		pr_warn("unsafe: exception during handler\n");
    - 	}
    + 	state = irqentry_nmi_enter(regs);
    +-	ret = do_sdei_event(regs, arg);
    ++	ret = do_sdei_event(regs, arg, vbar);
    + 	irqentry_nmi_exit(regs, state);
      
     +	write_sysreg(vbar, vbar_el1);
     +
    -+	if (err)
    -+		return SDEI_EV_FAILED;
    -+
    - 	mode = regs->pstate & (PSR_MODE32_BIT | PSR_MODE_MASK);
    + 	return ret;
    + }
    + #endif /* CONFIG_ARM_SDE_INTERFACE */
    +
    + ## arch/arm64/kernel/sdei.c ##
    +@@ arch/arm64/kernel/sdei.c: unsigned long sdei_arch_get_entry_point(int conduit)
    +  *  SDEI_EV_HANDLED -  success, return to the interrupted context.
    +  *  SDEI_EV_FAILED  -  failure, return this error code to firmware.
    +  *  virtual-address -  success, return to this address.
    ++ *
    ++ * vbar is the interrupted context's VBAR_EL1, for the IRQ-like resume.
    +  */
    + unsigned long __kprobes do_sdei_event(struct pt_regs *regs,
    +-				      struct sdei_registered_event *arg)
    ++				      struct sdei_registered_event *arg,
    ++				      unsigned long vbar)
    + {
    + 	u32 mode;
    + 	int i, err = 0;
    + 	int clobbered_registers = 4;
    + 	u64 elr = read_sysreg(elr_el1);
    + 	u32 kernel_mode = read_sysreg(CurrentEL) | 1;	/* +SPSel */
    +-	unsigned long vbar = read_sysreg(vbar_el1);
      
    - 	/*
    + 	if (arm64_kernel_unmapped_at_el0())
    + 		clobbered_registers++;
2:  ba422c8220e5 ! 2:  07d3078373ee arm64: sdei: Preserve the interrupted context's exception registers
    @@ Commit message
         kworker that is not running a work item, and a single sysrq-l warned
         once per such kworker.
     
    -    Save ELR, SPSR, ESR and FAR on entry to do_sdei_event() and restore them
    -    before returning to firmware.
    +    Save ELR, SPSR, ESR and FAR in __sdei_handler() before
    +    irqentry_nmi_enter(), which can take an exception too, and restore them
    +    after irqentry_nmi_exit().
     
         Once they are restored, a nested exception is harmless if its handler is
         safe in NMI context. That holds for EL1 data aborts on translation,
    @@ Commit message
         Cc: stable@vger.kernel.org
         Signed-off-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
     
    + ## arch/arm64/kernel/entry-common.c ##
    +@@ arch/arm64/kernel/entry-common.c: asmlinkage void noinstr __noreturn handle_bad_stack(struct pt_regs *regs)
    + asmlinkage noinstr unsigned long
    + __sdei_handler(struct pt_regs *regs, struct sdei_registered_event *arg)
    + {
    +-	unsigned long vbar;
    ++	unsigned long elr, spsr, esr, far, vbar;
    + 	irqentry_state_t state;
    + 	unsigned long ret;
    + 
    +@@ arch/arm64/kernel/entry-common.c: __sdei_handler(struct pt_regs *regs, struct sdei_registered_event *arg)
    + 	else if (cpu_has_pan())
    + 		set_pstate_pan(0);
    + 
    ++	/*
    ++	 * Nothing saved the interrupted context's exception registers on the
    ++	 * way in, and it may be in the middle of exception entry or return
    ++	 * (SDEI ignores DAIF). Save them before anything can take an exception
    ++	 * and overwrite them.
    ++	 */
    ++	elr = read_sysreg(elr_el1);
    ++	spsr = read_sysreg(spsr_el1);
    ++	esr = read_sysreg(esr_el1);
    ++	far = read_sysreg(far_el1);
    ++
    + 	/*
    + 	 * With KPTI, VBAR_EL1 holds the trampoline vectors from the return to
    + 	 * EL0 until the next exception from EL0 maps the kernel. They only
    +@@ arch/arm64/kernel/entry-common.c: __sdei_handler(struct pt_regs *regs, struct sdei_registered_event *arg)
    + 	ret = do_sdei_event(regs, arg, vbar);
    + 	irqentry_nmi_exit(regs, state);
    + 
    ++	/*
    ++	 * Put them back unconditionally. An unchanged ELR does not rule out a
    ++	 * nested exception: one may have returned to the same address.
    ++	 */
    ++	write_sysreg(elr, elr_el1);
    ++	write_sysreg(spsr, spsr_el1);
    ++	write_sysreg(esr, esr_el1);
    ++	write_sysreg(far, far_el1);
    + 	write_sysreg(vbar, vbar_el1);
    + 
    + 	return ret;
    +
      ## arch/arm64/kernel/sdei.c ##
     @@ arch/arm64/kernel/sdei.c: unsigned long sdei_arch_get_entry_point(int conduit)
      	return 0;
      }
      
     +/*
    -+ * Exceptions an SDEI handler may take without harm once do_sdei_event() has
    ++ * Exceptions an SDEI handler may take without harm once __sdei_handler() has
     + * restored the interrupted context's exception registers. Their handlers are
     + * safe in NMI context:
     + *  - EL1 data aborts on translation, permission and access-flag faults:
    @@ arch/arm64/kernel/sdei.c: unsigned long __kprobes do_sdei_event(struct pt_regs *
      	int i, err = 0;
      	int clobbered_registers = 4;
      	u64 elr = read_sysreg(elr_el1);
    -+	u64 spsr = read_sysreg(spsr_el1);
    -+	u64 esr = read_sysreg(esr_el1);
    -+	u64 far = read_sysreg(far_el1);
     +	u64 nested_elr, nested_esr, nested_far;
      	u32 kernel_mode = read_sysreg(CurrentEL) | 1;	/* +SPSel */
    - 	unsigned long vbar = read_sysreg(vbar_el1);
      
    + 	if (arm64_kernel_unmapped_at_el0())
     @@ arch/arm64/kernel/sdei.c: unsigned long __kprobes do_sdei_event(struct pt_regs *regs,
    + 	}
      
      	err = sdei_event_handler(regs, arg);
    - 
    --	if (elr != read_sysreg(elr_el1)) {
    --		/*
    --		 * We took a synchronous exception from the SDEI handler.
    --		 * This could deadlock, and if you interrupt KVM it will
    --		 * hyp-panic instead.
    --		 */
    --		pr_warn("unsafe: exception during handler\n");
    ++
     +	/*
    -+	 * SDEI is not an architectural exception: nothing saved the
    -+	 * interrupted context's exception registers on the way in, and it may
    -+	 * be in the middle of exception entry or return (SDEI ignores DAIF).
    -+	 * Any synchronous exception the handler took has overwritten them.
    -+	 * Only the last one's syndrome survives; report it if it is not one
    -+	 * known to be harmless.
    ++	 * A synchronous exception taken by the handler has overwritten the
    ++	 * exception registers; __sdei_handler() puts them back. Only the last
    ++	 * one's syndrome survives; report it if it is not one known to be
    ++	 * harmless.
     +	 *
     +	 * If the event interrupted a KVM world switch, VBAR points at the KVM
     +	 * hyp vectors and any such exception is a hyp panic instead.
    @@ arch/arm64/kernel/sdei.c: unsigned long __kprobes do_sdei_event(struct pt_regs *
     +	if (nested_elr != elr && !sdei_nested_exception_ok(nested_esr)) {
     +		pr_warn_once("unexpected exception during handler: ESR %#llx FAR %#llx near %pS\n",
     +			     nested_esr, nested_far, (void *)nested_elr);
    - 	}
    - 
    -+	/*
    -+	 * Put them back unconditionally. An unchanged ELR does not rule out a
    -+	 * nested exception: one may have returned to the same address.
    -+	 */
    -+	write_sysreg(elr, elr_el1);
    -+	write_sysreg(spsr, spsr_el1);
    -+	write_sysreg(esr, esr_el1);
    -+	write_sysreg(far, far_el1);
    - 	write_sysreg(vbar, vbar_el1);
    - 
    ++	}
    ++
      	if (err)
    + 		return SDEI_EV_FAILED;
    + 
    +-	if (elr != read_sysreg(elr_el1)) {
    +-		/*
    +-		 * We took a synchronous exception from the SDEI handler.
    +-		 * This could deadlock, and if you interrupt KVM it will
    +-		 * hyp-panic instead.
    +-		 */
    +-		pr_warn("unsafe: exception during handler\n");
    +-	}
    +-
    + 	mode = regs->pstate & (PSR_MODE32_BIT | PSR_MODE_MASK);
    + 
    + 	/*

base-commit: 6628e7d4c6b25c8d18f1345e3a20d21829aae7e4
-- 
2.54.0



^ permalink raw reply	[flat|nested] 4+ messages in thread
end of thread, other threads:[~2026-11-05 15:41 UTC | newest]

Thread overview: 4+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-11-05 12:08 [PATCH v2 0/2] arm64: sdei: Handle exceptions taken inside the event handler Kiryl Shutsemau
2026-11-05 12:08 ` [PATCH v2 1/2] arm64: sdei: Run the event handler on the kernel vectors Kiryl Shutsemau
2026-11-05 12:08 ` [PATCH v2 2/2] arm64: sdei: Preserve the interrupted context's exception registers Kiryl Shutsemau
2026-11-05 15:41   ` Breno Leitao

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox