From: Jinjie Ruan <ruanjinjie@huawei.com>
To: Vladimir Murzin <vladimir.murzin@arm.com>,
<linux-arm-kernel@lists.infradead.org>
Cc: mark.rutland@arm.com, maz@kernel.org, will@kernel.org,
catalin.marinas@arm.com
Subject: Re: [RFC PATCH 12/36] arm64: entry: replace DAIF helpers with entry helpers
Date: Fri, 10 Jul 2026 17:36:39 +0800 [thread overview]
Message-ID: <5129a523-142f-4834-ba4d-6ff15278d12a@huawei.com> (raw)
In-Reply-To: <20260709121333.23507-13-vladimir.murzin@arm.com>
On 7/9/2026 8:13 PM, Vladimir Murzin wrote:
> From: Ada Couprie Diaz <ada.coupriediaz@arm.com>
>
> Replace all uses of the `local_daif_...` helpers in `entry-common.c`
> with the new entry-specific exception masking helpers.
>
> Also replace `local_irq_disable()` with a switch to NOIRQ context
> using the new helpers. This provides:
>
> - consistency checks for hardware state
>
> - a unified style for managing exception context
>
> Now that exception context is tracked explicitly, involuntary kernel
> preemption can be optimized. We either preempt:
>
> - from PROCESS exception context, or
>
> - when returning from an interrupt handler, where the GIC code
> switches from NONMI to NOIRQ exception context
>
> To support this split, divide `arm64_exit_to_kernel_mode()` into
> preempt, non-preempt, and dispatch parts.
>
> Remove `local_daif_inherit()`, since it is no longer used.
>
> Now that both the irqflags API and entry code handle DAIF and PMR
> correctly, remove `GIC_PRIO_PSR_I_SET` from the PMR value set by
> `kernel_entry` in `entry.S` and by `init_gic_priority_masking()` in
> `kernel/smp.c`.
>
> The `local_daif_...` helpers and other specialized code still use
> `GIC_PRIO_PSR_I_SET` for now; this commit does not change their
> behavior.
>
> Signed-off-by: Ada Couprie Diaz <ada.coupriediaz@arm.com>
> Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
> ---
> arch/arm64/include/asm/daifflags.h | 22 ---
> arch/arm64/kernel/entry-common.c | 272 ++++++++++++++++++++---------
> arch/arm64/kernel/entry.S | 2 +-
> arch/arm64/kernel/smp.c | 2 +-
> 4 files changed, 190 insertions(+), 108 deletions(-)
>
> diff --git a/arch/arm64/include/asm/daifflags.h b/arch/arm64/include/asm/daifflags.h
> index 56341578e7e3..6b14ec4d4dbe 100644
> --- a/arch/arm64/include/asm/daifflags.h
> +++ b/arch/arm64/include/asm/daifflags.h
> @@ -120,28 +120,6 @@ static __always_inline void local_daif_restore(unsigned long flags)
> trace_hardirqs_off();
> }
>
> -/*
> - * Called by synchronous exception handlers to restore the DAIF bits that were
> - * modified by taking an exception.
> - */
> -static __always_inline void local_daif_inherit(struct pt_regs *regs)
> -{
> - unsigned long flags = regs->pstate & DAIF_MASK;
> -
> - if (!regs_irqs_disabled(regs))
> - trace_hardirqs_on();
> -
> - if (system_uses_irq_prio_masking())
> - gic_write_pmr(regs->pmr);
> -
> - /*
> - * We can't use local_daif_restore(regs->pstate) here as
> - * system_has_prio_mask_debugging() won't restore the I bit if it can
> - * use the pmr instead.
> - */
> - write_sysreg(flags, daif);
> -}
> -
> /*
> * During early boot, we unmask PSR.DA before the GIC has been set up.
> * If we use IRQ priority masking, the PMR and PSR will be out of sync
> diff --git a/arch/arm64/kernel/entry-common.c b/arch/arm64/kernel/entry-common.c
> index 2be42d7f4eaa..a13653b228b7 100644
> --- a/arch/arm64/kernel/entry-common.c
> +++ b/arch/arm64/kernel/entry-common.c
> @@ -18,11 +18,11 @@
> #include <linux/thread_info.h>
>
> #include <asm/cpufeature.h>
> -#include <asm/daifflags.h>
> #include <asm/esr.h>
> #include <asm/exception.h>
> #include <asm/fpsimd.h>
> #include <asm/irq_regs.h>
> +#include <asm/interrupts/entry.h>
> #include <asm/kprobes.h>
> #include <asm/mmu.h>
> #include <asm/processor.h>
> @@ -52,12 +52,11 @@ static noinstr irqentry_state_t arm64_enter_from_kernel_mode(struct pt_regs *reg
> * After this function returns it is not safe to call regular kernel code,
> * instrumentable code, or any code which may trigger an exception.
> */
> -static void noinstr arm64_exit_to_kernel_mode(struct pt_regs *regs,
> - irqentry_state_t state)
> +static void noinstr __arm64_exit_to_kernel_mode(struct pt_regs *regs,
> + irqentry_state_t state,
> + arm64_exc_hwstate_t hwstate)
> {
> - local_irq_disable();
> - irqentry_exit_to_kernel_mode_preempt(regs, state);
> - local_daif_mask();
> + arm64_mask_exc_context(hwstate);
> mte_check_tfsr_exit();
> irqentry_exit_to_kernel_mode_after_preempt(regs, state);
> }
> @@ -69,6 +68,30 @@ static __always_inline void arm64_syscall_enter_from_user_mode(struct pt_regs *r
> sme_enter_from_user_mode();
> }
>
> +/*
> + * We are returning from the context which allows involuntary kernel preemption
> + */
> +static void noinstr arm64_exit_to_kernel_mode_preempt(struct pt_regs *regs,
> + irqentry_state_t state,
> + arm64_exc_hwstate_t hwstate)
> +{
> + irqentry_exit_to_kernel_mode_preempt(regs, state);
> + __arm64_exit_to_kernel_mode(regs, state, hwstate);
> +}
> +
> +static void noinstr arm64_exit_to_kernel_mode(struct pt_regs *regs,
> + irqentry_state_t state,
> + arm64_exc_hwstate_t hwstate)
> +{
> + if (!arch_irqs_disabled_flags(hwstate.flags)) {
> + hwstate = arm64_lift_exc_context(hwstate, NOIRQ_CONTEXT);
> + arm64_exit_to_kernel_mode_preempt(regs, state, hwstate);
> + return;
> + }
> +
> + __arm64_exit_to_kernel_mode(regs, state, hwstate);
> +}
> +
> /*
> * Handle IRQ/context state management when entering from user mode.
> * Before this function is called it is not safe to call regular kernel code,
> @@ -82,11 +105,12 @@ static __always_inline void arm64_enter_from_user_mode(struct pt_regs *regs)
> sme_enter_from_user_mode();
> }
>
> -static __always_inline void arm64_syscall_exit_to_user_mode(struct pt_regs *regs)
> +static __always_inline void arm64_syscall_exit_to_user_mode(struct pt_regs *regs,
> + arm64_exc_hwstate_t hwstate)
> {
> - local_irq_disable();
> + hwstate = arm64_lift_exc_context(hwstate, NOIRQ_CONTEXT);
> syscall_exit_to_user_mode_prepare(regs);
> - local_daif_mask();
> + arm64_mask_exc_context(hwstate);
> sme_exit_to_user_mode();
> mte_check_tfsr_exit();
> exit_to_user_mode();
> @@ -97,11 +121,12 @@ static __always_inline void arm64_syscall_exit_to_user_mode(struct pt_regs *regs
> * After this function returns it is not safe to call regular kernel code,
> * instrumentable code, or any code which may trigger an exception.
> */
> -static __always_inline void arm64_exit_to_user_mode(struct pt_regs *regs)
> +static __always_inline void arm64_exit_to_user_mode(struct pt_regs *regs,
> + arm64_exc_hwstate_t hwstate)
> {
> - local_irq_disable();
> + hwstate = arm64_lift_exc_context(hwstate, NOIRQ_CONTEXT);
> irqentry_exit_to_user_mode_prepare(regs);
> - local_daif_mask();
> + arm64_mask_exc_context(hwstate);
> sme_exit_to_user_mode();
> mte_check_tfsr_exit();
> exit_to_user_mode();
> @@ -109,7 +134,10 @@ static __always_inline void arm64_exit_to_user_mode(struct pt_regs *regs)
>
> asmlinkage void noinstr asm_exit_to_user_mode(struct pt_regs *regs)
> {
> - arm64_syscall_exit_to_user_mode(regs);
> + arm64_exc_hwstate_t hwstate = arm64_exc_hwstate_of_context(PROCESS_CONTEXT);
> +
> + arm64_syscall_exit_to_user_mode(regs, hwstate);
> + arm64_debug_exc_context(CRITICAL_CONTEXT);
> }
>
> /*
> @@ -315,63 +343,69 @@ UNHANDLED(el1t, 64, error)
> static void noinstr el1_abort(struct pt_regs *regs, unsigned long esr)
> {
> unsigned long far = read_sysreg(far_el1);
> + arm64_exc_hwstate_t hwstate;
> irqentry_state_t state;
>
> state = arm64_enter_from_kernel_mode(regs);
> - local_daif_inherit(regs);
> + hwstate = arm64_inherit_exc_context(regs);
> do_mem_abort(far, esr, regs);
> - arm64_exit_to_kernel_mode(regs, state);
> + arm64_exit_to_kernel_mode(regs, state, hwstate);
> }
>
> static void noinstr el1_pc(struct pt_regs *regs, unsigned long esr)
> {
> unsigned long far = read_sysreg(far_el1);
> + arm64_exc_hwstate_t hwstate;
> irqentry_state_t state;
>
> state = arm64_enter_from_kernel_mode(regs);
> - local_daif_inherit(regs);
> + hwstate = arm64_inherit_exc_context(regs);
> do_sp_pc_abort(far, esr, regs);
> - arm64_exit_to_kernel_mode(regs, state);
> + arm64_exit_to_kernel_mode(regs, state, hwstate);
> }
>
> static void noinstr el1_undef(struct pt_regs *regs, unsigned long esr)
> {
> + arm64_exc_hwstate_t hwstate;
> irqentry_state_t state;
>
> state = arm64_enter_from_kernel_mode(regs);
> - local_daif_inherit(regs);
> + hwstate = arm64_inherit_exc_context(regs);
local_daif_inherit do not trace_hardirqs_off() but only
trace_hardirqs_off()，but now arm64_inherit_exc_context() do it, is this
a functional change?
> do_el1_undef(regs, esr);
> - arm64_exit_to_kernel_mode(regs, state);
> + arm64_exit_to_kernel_mode(regs, state, hwstate);
> }
>
> static void noinstr el1_bti(struct pt_regs *regs, unsigned long esr)
> {
> + arm64_exc_hwstate_t hwstate;
> irqentry_state_t state;
>
> state = arm64_enter_from_kernel_mode(regs);
> - local_daif_inherit(regs);
> + hwstate = arm64_inherit_exc_context(regs);
> do_el1_bti(regs, esr);
> - arm64_exit_to_kernel_mode(regs, state);
> + arm64_exit_to_kernel_mode(regs, state, hwstate);
> }
>
> static void noinstr el1_gcs(struct pt_regs *regs, unsigned long esr)
> {
> + arm64_exc_hwstate_t hwstate;
> irqentry_state_t state;
>
> state = arm64_enter_from_kernel_mode(regs);
> - local_daif_inherit(regs);
> + hwstate = arm64_inherit_exc_context(regs);
> do_el1_gcs(regs, esr);
> - arm64_exit_to_kernel_mode(regs, state);
> + arm64_exit_to_kernel_mode(regs, state, hwstate);
> }
>
> static void noinstr el1_mops(struct pt_regs *regs, unsigned long esr)
> {
> + arm64_exc_hwstate_t hwstate;
> irqentry_state_t state;
>
> state = arm64_enter_from_kernel_mode(regs);
> - local_daif_inherit(regs);
> + hwstate = arm64_inherit_exc_context(regs);
> do_el1_mops(regs, esr);
> - arm64_exit_to_kernel_mode(regs, state);
> + arm64_exit_to_kernel_mode(regs, state, hwstate);
> }
>
> static void noinstr el1_breakpt(struct pt_regs *regs, unsigned long esr)
> @@ -431,12 +465,13 @@ static void noinstr el1_brk64(struct pt_regs *regs, unsigned long esr)
>
> static void noinstr el1_fpac(struct pt_regs *regs, unsigned long esr)
> {
> + arm64_exc_hwstate_t hwstate;
> irqentry_state_t state;
>
> state = arm64_enter_from_kernel_mode(regs);
> - local_daif_inherit(regs);
> + hwstate = arm64_inherit_exc_context(regs);
> do_el1_fpac(regs, esr);
> - arm64_exit_to_kernel_mode(regs, state);
> + arm64_exit_to_kernel_mode(regs, state, hwstate);
> }
>
> asmlinkage void noinstr el1h_64_sync_handler(struct pt_regs *regs)
> @@ -486,16 +521,20 @@ asmlinkage void noinstr el1h_64_sync_handler(struct pt_regs *regs)
> default:
> __panic_unhandled(regs, "64-bit el1h sync", esr);
> }
> +
> + arm64_debug_exc_context(CRITICAL_CONTEXT);
> }
>
> static __always_inline void __el1_pnmi(struct pt_regs *regs,
> void (*handler)(struct pt_regs *))
> {
> + arm64_exc_hwstate_t hwstate;
> irqentry_state_t state;
>
> state = irqentry_nmi_enter(regs);
> + hwstate = arm64_unmask_exc_context(NONMI_CONTEXT);
> do_interrupt_handler(regs, handler);
> - local_daif_mask();
> + arm64_mask_exc_context(hwstate);
> irqentry_nmi_exit(regs, state);
> }
>
> @@ -506,21 +545,32 @@ static __always_inline void __el1_irq(struct pt_regs *regs,
>
> state = arm64_enter_from_kernel_mode(regs);
>
> + arm64_unmask_exc_context(NONMI_CONTEXT);
> +
> irq_enter_rcu();
> do_interrupt_handler(regs, handler);
> irq_exit_rcu();
>
> - arm64_exit_to_kernel_mode(regs, state);
> + /*
> + * If pseudo-NMIs are enabled and the interrupted context had
> + * IRQs unmasked, the interrupt handler will have cleared DAIF
> + * and switched to PMR masking in order to handle
> + * NMIs. Otherwise it would keep IF. In both cases on return
> + * we effectivly have NOIRQ_CONTEXT - keep track of it
> + */
> + arm64_debug_exc_context(NOIRQ_CONTEXT);
> + arm64_exit_to_kernel_mode_preempt(regs, state, arm64_exc_hwstate_of_context(NOIRQ_CONTEXT));
> }
> +
> static void noinstr el1_interrupt(struct pt_regs *regs,
> void (*handler)(struct pt_regs *))
> {
> - write_sysreg(DAIF_PROCCTX_NOIRQ, daif);
> -
> if (IS_ENABLED(CONFIG_ARM64_PSEUDO_NMI) && regs_irqs_disabled(regs))
> __el1_pnmi(regs, handler);
> else
> __el1_irq(regs, handler);
> +
> + arm64_debug_exc_context(CRITICAL_CONTEXT);
> }
>
> asmlinkage void noinstr el1h_64_irq_handler(struct pt_regs *regs)
> @@ -536,28 +586,31 @@ asmlinkage void noinstr el1h_64_fiq_handler(struct pt_regs *regs)
> asmlinkage void noinstr el1h_64_error_handler(struct pt_regs *regs)
> {
> unsigned long esr = read_sysreg(esr_el1);
> + arm64_exc_hwstate_t hwstate;
> irqentry_state_t state;
>
> - local_daif_restore(DAIF_ERRCTX);
> + hwstate = arm64_unmask_exc_context(ERROR_CONTEXT);
> state = irqentry_nmi_enter(regs);
> do_serror(regs, esr);
> - local_daif_mask();
> + arm64_mask_exc_context(hwstate);
> irqentry_nmi_exit(regs, state);
> }
>
> static void noinstr el0_da(struct pt_regs *regs, unsigned long esr)
> {
> unsigned long far = read_sysreg(far_el1);
> + arm64_exc_hwstate_t hwstate;
>
> arm64_enter_from_user_mode(regs);
> - local_daif_restore(DAIF_PROCCTX);
> + hwstate = arm64_unmask_exc_context(PROCESS_CONTEXT);
> do_mem_abort(far, esr, regs);
> - arm64_exit_to_user_mode(regs);
> + arm64_exit_to_user_mode(regs, hwstate);
> }
>
> static void noinstr el0_ia(struct pt_regs *regs, unsigned long esr)
> {
> unsigned long far = read_sysreg(far_el1);
> + arm64_exc_hwstate_t hwstate;
>
> /*
> * We've taken an instruction abort from userspace and not yet
> @@ -568,114 +621,139 @@ static void noinstr el0_ia(struct pt_regs *regs, unsigned long esr)
> arm64_apply_bp_hardening();
>
> arm64_enter_from_user_mode(regs);
> - local_daif_restore(DAIF_PROCCTX);
> + hwstate = arm64_unmask_exc_context(PROCESS_CONTEXT);
> do_mem_abort(far, esr, regs);
> - arm64_exit_to_user_mode(regs);
> + arm64_exit_to_user_mode(regs, hwstate);
> }
>
> static void noinstr el0_fpsimd_acc(struct pt_regs *regs, unsigned long esr)
> {
> + arm64_exc_hwstate_t hwstate;
> +
> arm64_enter_from_user_mode(regs);
> - local_daif_restore(DAIF_PROCCTX);
> + hwstate = arm64_unmask_exc_context(PROCESS_CONTEXT);
> do_fpsimd_acc(esr, regs);
> - arm64_exit_to_user_mode(regs);
> + arm64_exit_to_user_mode(regs, hwstate);
> }
>
> static void noinstr el0_sve_acc(struct pt_regs *regs, unsigned long esr)
> {
> + arm64_exc_hwstate_t hwstate;
> +
> arm64_enter_from_user_mode(regs);
> - local_daif_restore(DAIF_PROCCTX);
> + hwstate = arm64_unmask_exc_context(PROCESS_CONTEXT);
> do_sve_acc(esr, regs);
> - arm64_exit_to_user_mode(regs);
> + arm64_exit_to_user_mode(regs, hwstate);
> }
>
> static void noinstr el0_sme_acc(struct pt_regs *regs, unsigned long esr)
> {
> + arm64_exc_hwstate_t hwstate;
> +
> arm64_enter_from_user_mode(regs);
> - local_daif_restore(DAIF_PROCCTX);
> + hwstate = arm64_unmask_exc_context(PROCESS_CONTEXT);
> do_sme_acc(esr, regs);
> - arm64_exit_to_user_mode(regs);
> + arm64_exit_to_user_mode(regs, hwstate);
> }
>
> static void noinstr el0_fpsimd_exc(struct pt_regs *regs, unsigned long esr)
> {
> + arm64_exc_hwstate_t hwstate;
> +
> arm64_enter_from_user_mode(regs);
> - local_daif_restore(DAIF_PROCCTX);
> + hwstate = arm64_unmask_exc_context(PROCESS_CONTEXT);
> do_fpsimd_exc(esr, regs);
> - arm64_exit_to_user_mode(regs);
> + arm64_exit_to_user_mode(regs, hwstate);
> }
>
> static void noinstr el0_sys(struct pt_regs *regs, unsigned long esr)
> {
> + arm64_exc_hwstate_t hwstate;
> +
> arm64_enter_from_user_mode(regs);
> - local_daif_restore(DAIF_PROCCTX);
> + hwstate = arm64_unmask_exc_context(PROCESS_CONTEXT);
> do_el0_sys(esr, regs);
> - arm64_exit_to_user_mode(regs);
> + arm64_exit_to_user_mode(regs, hwstate);
> }
>
> static void noinstr el0_pc(struct pt_regs *regs, unsigned long esr)
> {
> unsigned long far = read_sysreg(far_el1);
> + arm64_exc_hwstate_t hwstate;
>
> if (!is_ttbr0_addr(instruction_pointer(regs)))
> arm64_apply_bp_hardening();
>
> arm64_enter_from_user_mode(regs);
> - local_daif_restore(DAIF_PROCCTX);
> + hwstate = arm64_unmask_exc_context(PROCESS_CONTEXT);
> do_sp_pc_abort(far, esr, regs);
> - arm64_exit_to_user_mode(regs);
> + arm64_exit_to_user_mode(regs, hwstate);
> }
>
> static void noinstr el0_sp(struct pt_regs *regs, unsigned long esr)
> {
> + arm64_exc_hwstate_t hwstate;
> +
> arm64_enter_from_user_mode(regs);
> - local_daif_restore(DAIF_PROCCTX);
> + hwstate = arm64_unmask_exc_context(PROCESS_CONTEXT);
> do_sp_pc_abort(regs->sp, esr, regs);
> - arm64_exit_to_user_mode(regs);
> + arm64_exit_to_user_mode(regs, hwstate);
> }
>
> static void noinstr el0_undef(struct pt_regs *regs, unsigned long esr)
> {
> + arm64_exc_hwstate_t hwstate;
> +
> arm64_enter_from_user_mode(regs);
> - local_daif_restore(DAIF_PROCCTX);
> + hwstate = arm64_unmask_exc_context(PROCESS_CONTEXT);
> do_el0_undef(regs, esr);
> - arm64_exit_to_user_mode(regs);
> + arm64_exit_to_user_mode(regs, hwstate);
> }
>
> static void noinstr el0_bti(struct pt_regs *regs)
> {
> + arm64_exc_hwstate_t hwstate;
> +
> arm64_enter_from_user_mode(regs);
> - local_daif_restore(DAIF_PROCCTX);
> + hwstate = arm64_unmask_exc_context(PROCESS_CONTEXT);
> do_el0_bti(regs);
> - arm64_exit_to_user_mode(regs);
> + arm64_exit_to_user_mode(regs, hwstate);
> }
>
> static void noinstr el0_mops(struct pt_regs *regs, unsigned long esr)
> {
> + arm64_exc_hwstate_t hwstate;
> +
> arm64_enter_from_user_mode(regs);
> - local_daif_restore(DAIF_PROCCTX);
> + hwstate = arm64_unmask_exc_context(PROCESS_CONTEXT);
> do_el0_mops(regs, esr);
> - arm64_exit_to_user_mode(regs);
> + arm64_exit_to_user_mode(regs, hwstate);
> }
>
> static void noinstr el0_gcs(struct pt_regs *regs, unsigned long esr)
> {
> + arm64_exc_hwstate_t hwstate;
> +
> arm64_enter_from_user_mode(regs);
> - local_daif_restore(DAIF_PROCCTX);
> + hwstate = arm64_unmask_exc_context(PROCESS_CONTEXT);
> do_el0_gcs(regs, esr);
> - arm64_exit_to_user_mode(regs);
> + arm64_exit_to_user_mode(regs, hwstate);
> }
>
> static void noinstr el0_inv(struct pt_regs *regs, unsigned long esr)
> {
> + arm64_exc_hwstate_t hwstate;
> +
> arm64_enter_from_user_mode(regs);
> - local_daif_restore(DAIF_PROCCTX);
> + hwstate = arm64_unmask_exc_context(PROCESS_CONTEXT);
> bad_el0_sync(regs, 0, esr);
> - arm64_exit_to_user_mode(regs);
> + arm64_exit_to_user_mode(regs, hwstate);
> }
>
> static void noinstr el0_breakpt(struct pt_regs *regs, unsigned long esr)
> {
> + arm64_exc_hwstate_t hwstate;
> +
> if (!is_ttbr0_addr(regs->pc))
> arm64_apply_bp_hardening();
>
> @@ -683,12 +761,13 @@ static void noinstr el0_breakpt(struct pt_regs *regs, unsigned long esr)
> debug_exception_enter(regs);
> do_breakpoint(esr, regs);
> debug_exception_exit(regs);
> - local_daif_restore(DAIF_PROCCTX);
> - arm64_exit_to_user_mode(regs);
> + hwstate = arm64_unmask_exc_context(PROCESS_CONTEXT);
> + arm64_exit_to_user_mode(regs, hwstate);
> }
>
> static void noinstr el0_softstp(struct pt_regs *regs, unsigned long esr)
> {
> + arm64_exc_hwstate_t hwstate;
> bool step_done;
>
> if (!is_ttbr0_addr(regs->pc))
> @@ -702,50 +781,56 @@ static void noinstr el0_softstp(struct pt_regs *regs, unsigned long esr)
> * the single-step is complete.
> */
> step_done = try_step_suspended_breakpoints(regs);
> - local_daif_restore(DAIF_PROCCTX);
> + hwstate = arm64_unmask_exc_context(PROCESS_CONTEXT);
> if (!step_done)
> do_el0_softstep(esr, regs);
> - arm64_exit_to_user_mode(regs);
> + arm64_exit_to_user_mode(regs, hwstate);
> }
>
> static void noinstr el0_watchpt(struct pt_regs *regs, unsigned long esr)
> {
> /* Watchpoints are the only debug exception to write FAR_EL1 */
> unsigned long far = read_sysreg(far_el1);
> + arm64_exc_hwstate_t hwstate;
>
> arm64_enter_from_user_mode(regs);
> debug_exception_enter(regs);
> do_watchpoint(far, esr, regs);
> debug_exception_exit(regs);
> - local_daif_restore(DAIF_PROCCTX);
> - arm64_exit_to_user_mode(regs);
> + hwstate = arm64_unmask_exc_context(PROCESS_CONTEXT);
> + arm64_exit_to_user_mode(regs, hwstate);
> }
>
> static void noinstr el0_brk64(struct pt_regs *regs, unsigned long esr)
> {
> + arm64_exc_hwstate_t hwstate;
> +
> arm64_enter_from_user_mode(regs);
> - local_daif_restore(DAIF_PROCCTX);
> + hwstate = arm64_unmask_exc_context(PROCESS_CONTEXT);
> do_el0_brk64(esr, regs);
> - arm64_exit_to_user_mode(regs);
> + arm64_exit_to_user_mode(regs, hwstate);
> }
>
> static void noinstr el0_svc(struct pt_regs *regs)
> {
> + arm64_exc_hwstate_t hwstate;
> arm64_syscall_enter_from_user_mode(regs);
> cortex_a76_erratum_1463225_svc_handler();
> fpsimd_syscall_enter();
> - local_daif_restore(DAIF_PROCCTX);
> + hwstate = arm64_unmask_exc_context(PROCESS_CONTEXT);
> do_el0_svc(regs);
> - arm64_syscall_exit_to_user_mode(regs);
> + arm64_syscall_exit_to_user_mode(regs, hwstate);
> fpsimd_syscall_exit();
> }
>
> static void noinstr el0_fpac(struct pt_regs *regs, unsigned long esr)
> {
> + arm64_exc_hwstate_t hwstate;
> +
> arm64_enter_from_user_mode(regs);
> - local_daif_restore(DAIF_PROCCTX);
> + hwstate = arm64_unmask_exc_context(PROCESS_CONTEXT);
> do_el0_fpac(regs, esr);
> - arm64_exit_to_user_mode(regs);
> + arm64_exit_to_user_mode(regs, hwstate);
> }
>
> asmlinkage void noinstr el0t_64_sync_handler(struct pt_regs *regs)
> @@ -814,6 +899,8 @@ asmlinkage void noinstr el0t_64_sync_handler(struct pt_regs *regs)
> default:
> el0_inv(regs, esr);
> }
> +
> + arm64_debug_exc_context(CRITICAL_CONTEXT);
> }
>
> static void noinstr el0_interrupt(struct pt_regs *regs,
> @@ -821,7 +908,7 @@ static void noinstr el0_interrupt(struct pt_regs *regs,
> {
> arm64_enter_from_user_mode(regs);
>
> - write_sysreg(DAIF_PROCCTX_NOIRQ, daif);
> + arm64_unmask_exc_context(NONMI_CONTEXT);
>
> if (regs->pc & BIT(55))
> arm64_apply_bp_hardening();
> @@ -830,7 +917,14 @@ static void noinstr el0_interrupt(struct pt_regs *regs,
> do_interrupt_handler(regs, handler);
> irq_exit_rcu();
>
> - arm64_exit_to_user_mode(regs);
> + /*
> + * For the same reason as in el1_irq() we effectivly
> + * have NOIRQ_CONTEXT on return from handler - keep
> + * track of it
> + */
> + arm64_debug_exc_context(NOIRQ_CONTEXT);
> + arm64_exit_to_user_mode(regs, arm64_exc_hwstate_of_context(NOIRQ_CONTEXT));
> + arm64_debug_exc_context(CRITICAL_CONTEXT);
> }
>
> static void noinstr __el0_irq_handler_common(struct pt_regs *regs)
> @@ -856,15 +950,17 @@ asmlinkage void noinstr el0t_64_fiq_handler(struct pt_regs *regs)
> static void noinstr __el0_error_handler_common(struct pt_regs *regs)
> {
> unsigned long esr = read_sysreg(esr_el1);
> + arm64_exc_hwstate_t hwstate;
> irqentry_state_t state;
>
> arm64_enter_from_user_mode(regs);
> - local_daif_restore(DAIF_ERRCTX);
> + hwstate = arm64_unmask_exc_context(ERROR_CONTEXT);
> state = irqentry_nmi_enter(regs);
> do_serror(regs, esr);
> irqentry_nmi_exit(regs, state);
> - local_daif_restore(DAIF_PROCCTX);
> - arm64_exit_to_user_mode(regs);
> + hwstate = arm64_drop_exc_context(hwstate, PROCESS_CONTEXT);
> + arm64_exit_to_user_mode(regs, hwstate);
> + arm64_debug_exc_context(CRITICAL_CONTEXT);
> }
>
> asmlinkage void noinstr el0t_64_error_handler(struct pt_regs *regs)
> @@ -875,27 +971,33 @@ asmlinkage void noinstr el0t_64_error_handler(struct pt_regs *regs)
> #ifdef CONFIG_COMPAT
> static void noinstr el0_cp15(struct pt_regs *regs, unsigned long esr)
> {
> + arm64_exc_hwstate_t hwstate;
> +
> arm64_enter_from_user_mode(regs);
> - local_daif_restore(DAIF_PROCCTX);
> + hwstate = arm64_unmask_exc_context(PROCESS_CONTEXT);
> do_el0_cp15(esr, regs);
> - arm64_exit_to_user_mode(regs);
> + arm64_exit_to_user_mode(regs, hwstate);
> }
>
> static void noinstr el0_svc_compat(struct pt_regs *regs)
> {
> + arm64_exc_hwstate_t hwstate;
> +
> arm64_syscall_enter_from_user_mode(regs);
> cortex_a76_erratum_1463225_svc_handler();
> - local_daif_restore(DAIF_PROCCTX);
> + hwstate = arm64_unmask_exc_context(PROCESS_CONTEXT);
> do_el0_svc_compat(regs);
> - arm64_syscall_exit_to_user_mode(regs);
> + arm64_syscall_exit_to_user_mode(regs, hwstate);
> }
>
> static void noinstr el0_bkpt32(struct pt_regs *regs, unsigned long esr)
> {
> + arm64_exc_hwstate_t hwstate;
> +
> arm64_enter_from_user_mode(regs);
> - local_daif_restore(DAIF_PROCCTX);
> + hwstate = arm64_unmask_exc_context(PROCESS_CONTEXT);
> do_bkpt32(esr, regs);
> - arm64_exit_to_user_mode(regs);
> + arm64_exit_to_user_mode(regs, hwstate);
> }
>
> asmlinkage void noinstr el0t_32_sync_handler(struct pt_regs *regs)
> @@ -946,6 +1048,8 @@ asmlinkage void noinstr el0t_32_sync_handler(struct pt_regs *regs)
> default:
> el0_inv(regs, esr);
> }
> +
> + arm64_debug_exc_context(CRITICAL_CONTEXT);
> }
>
> asmlinkage void noinstr el0t_32_irq_handler(struct pt_regs *regs)
> diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S
> index f63049ac32dc..cb3be770f2d0 100644
> --- a/arch/arm64/kernel/entry.S
> +++ b/arch/arm64/kernel/entry.S
> @@ -316,7 +316,7 @@ alternative_else_nop_endif
>
> mrs_s x20, SYS_ICC_PMR_EL1
> str w20, [sp, #S_PMR]
> - mov x20, #GIC_PRIO_IRQON | GIC_PRIO_PSR_I_SET
> + mov x20, #GIC_PRIO_IRQON
> msr_s SYS_ICC_PMR_EL1, x20
>
> .Lskip_pmr_save\@:
> diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c
> index d46022f72075..b91cf163aac7 100644
> --- a/arch/arm64/kernel/smp.c
> +++ b/arch/arm64/kernel/smp.c
> @@ -185,7 +185,7 @@ static void init_gic_priority_masking(void)
> WARN_ON(!(cpuflags & PSR_I_BIT));
> WARN_ON(!(cpuflags & PSR_F_BIT));
>
> - gic_write_pmr(GIC_PRIO_IRQON | GIC_PRIO_PSR_I_SET);
> + gic_write_pmr(GIC_PRIO_IRQON);
> }
>
> /*
next prev parent reply other threads:[~2026-07-10 9:37 UTC|newest]
Thread overview: 70+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-07-09 12:12 [RFC PATCH 00/36] arm64: Add support for FEAT_NMI Vladimir Murzin
2026-07-09 12:12 ` [RFC PATCH 01/36] arm64: ptrace: Remove INIT_PSTATE_EL2 Vladimir Murzin
2026-07-09 12:36 ` Jinjie Ruan
2026-07-09 12:12 ` [RFC PATCH 02/36] arm64: debug: don't mask DAIF for mdscr_write() Vladimir Murzin
2026-07-09 13:06 ` Jinjie Ruan
2026-07-09 12:13 ` [RFC PATCH 03/36] arm64: hibernate: mask DAIF before restoring hibernated kernel Vladimir Murzin
2026-07-09 13:19 ` Jinjie Ruan
2026-07-10 3:00 ` Jinjie Ruan
2026-07-10 3:28 ` Jinjie Ruan
2026-07-10 3:40 ` Liao, Chang
2026-07-09 12:13 ` [RFC PATCH 04/36] arm64: suspend: rely on daif helpers to handle PMR Vladimir Murzin
2026-07-10 3:41 ` Jinjie Ruan
2026-07-10 4:06 ` Jinjie Ruan
2026-07-09 12:13 ` [RFC PATCH 05/36] arm64: suspend: Initialize PMR on resume Vladimir Murzin
2026-07-09 12:13 ` [RFC PATCH 06/36] arm64: irq: introduce a helper for GIC priority initialization Vladimir Murzin
2026-07-10 4:16 ` Jinjie Ruan
2026-07-10 7:29 ` Jinjie Ruan
2026-07-10 7:44 ` Jinjie Ruan
2026-07-09 12:13 ` [RFC PATCH 07/36] arm64: entry: mask DAIF before returning from C EL1 handlers Vladimir Murzin
2026-07-10 7:57 ` Jinjie Ruan
2026-07-09 12:13 ` [RFC PATCH 08/36] irqchip/gic-v3: make the unmasking of pseudo-NMIs explicit when handling IRQs Vladimir Murzin
2026-07-10 8:04 ` Jinjie Ruan
2026-07-09 12:13 ` [RFC PATCH 09/36] arm64: irqflags: introduce arm64-specific irqflags type Vladimir Murzin
2026-07-10 8:40 ` Jinjie Ruan
2026-07-09 12:13 ` [RFC PATCH 10/36] arm64: irqflags: save and use both DAIF and PMR Vladimir Murzin
2026-07-10 3:53 ` Liao, Chang
2026-07-10 8:11 ` Jinjie Ruan
2026-07-10 8:47 ` Jinjie Ruan
2026-07-10 9:02 ` Jinjie Ruan
2026-07-09 12:13 ` [RFC PATCH 11/36] arm64: interrupts: introduce interrupt masking helpers for entry code Vladimir Murzin
2026-07-10 9:19 ` Jinjie Ruan
2026-07-10 9:39 ` Liao, Chang
2026-07-10 9:39 ` Jinjie Ruan
2026-07-10 9:44 ` Jinjie Ruan
2026-07-09 12:13 ` [RFC PATCH 12/36] arm64: entry: replace DAIF helpers with entry helpers Vladimir Murzin
2026-07-10 9:36 ` Jinjie Ruan [this message]
2026-07-10 10:01 ` Jinjie Ruan
2026-07-09 12:13 ` [RFC PATCH 13/36] arm64: process: Use helper to check exception state Vladimir Murzin
2026-07-10 10:00 ` Jinjie Ruan
2026-07-09 12:13 ` [RFC PATCH 14/36] arm64: interrupts: introduce generic interrupt masking helpers Vladimir Murzin
2026-07-09 12:13 ` [RFC PATCH 15/36] arm64: replace local_daif helpers Vladimir Murzin
2026-07-09 12:13 ` [RFC PATCH 16/36] arm64: cpuidle: use new helpers to bypass interrupt priority masking Vladimir Murzin
2026-07-09 12:13 ` [RFC PATCH 17/36] arm64: remove daifflags.h Vladimir Murzin
2026-07-09 12:13 ` [RFC PATCH 18/36] arm64: gicv3: remove GIC_PRIO_PSR_I_SET Vladimir Murzin
2026-07-09 12:13 ` [RFC PATCH 19/36] arm64: cpufeature: Remove system_has_prio_mask_debugging() Vladimir Murzin
2026-07-09 12:13 ` [RFC PATCH 20/36] arm64: irqflags: Switch to CONFIG_DEBUG_IRQFLAGS Vladimir Murzin
2026-07-09 12:13 ` [RFC PATCH 21/36] arm64: Kconfig: Remove CONFIG_ARM64_DEBUG_PRIORITY_MASKING Vladimir Murzin
2026-07-09 12:13 ` [RFC PATCH 22/36] efi/runtime-wrappers: Permit architectures to override IRQ flags checks Vladimir Murzin
2026-07-09 12:13 ` [RFC PATCH 23/36] arm64/efi: Implement override for " Vladimir Murzin
2026-07-09 12:13 ` [RFC PATCH 24/36] arm64: booting: Document boot requirements for FEAT_NMI Vladimir Murzin
2026-07-10 2:39 ` Jinjie Ruan
2026-07-09 12:13 ` [RFC PATCH 25/36] arm64: sysreg: Add definitions for immediate versions of MSR ALLINT Vladimir Murzin
2026-07-09 12:13 ` [RFC PATCH 26/36] arm64: ptrace: Add PSR_ALLINT_BIT Vladimir Murzin
2026-07-10 2:16 ` Jinjie Ruan
2026-07-09 12:13 ` [RFC PATCH 27/36] arm64: idreg: Add an override for FEAT_NMI Vladimir Murzin
2026-07-10 2:17 ` Jinjie Ruan
2026-07-09 12:13 ` [RFC PATCH 28/36] arm64: cpufeature: Detect PE support " Vladimir Murzin
2026-07-10 2:25 ` Jinjie Ruan
2026-07-09 12:13 ` [RFC PATCH 29/36] arm64: nmi: Manage masking for superpriority interrupts Vladimir Murzin
2026-07-10 10:04 ` Jinjie Ruan
2026-07-10 10:08 ` Jinjie Ruan
2026-07-09 12:13 ` [RFC PATCH 30/36] arm64: irq: Report FEAT_NMI masking local IRQs Vladimir Murzin
2026-07-09 12:13 ` [RFC PATCH 31/36] arm64: nmi: Add handling of superpriority interrupts as NMIs Vladimir Murzin
2026-07-10 10:13 ` Jinjie Ruan
2026-07-09 12:13 ` [RFC PATCH 32/36] arm64: suspend: Always initialise PSTATE.ALLINT Vladimir Murzin
2026-07-09 12:13 ` [RFC PATCH 33/36] arm64/efi: Add ALLINT to IRQ flags checks Vladimir Murzin
2026-07-09 12:13 ` [RFC PATCH 34/36] arm64: kprobes: Disable NMIs Vladimir Murzin
2026-07-09 12:13 ` [RFC PATCH 35/36] arm64: nmi: Add Kconfig for NMI Vladimir Murzin
2026-07-10 2:41 ` Jinjie Ruan
2026-07-09 12:13 ` [RFC PATCH 36/36] irqchip/gic-v3: Implement FEAT_GICv3_NMI support Vladimir Murzin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=5129a523-142f-4834-ba4d-6ff15278d12a@huawei.com \
--to=ruanjinjie@huawei.com \
--cc=catalin.marinas@arm.com \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=mark.rutland@arm.com \
--cc=maz@kernel.org \
--cc=vladimir.murzin@arm.com \
--cc=will@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox