All of lore.kernel.org
 help / color / mirror / Atom feed
From: Will Deacon <will@kernel.org>
To: Breno Leitao <leitao@debian.org>
Cc: Catalin Marinas <catalin.marinas@arm.com>,
	"Peter Zijlstra (Intel)" <peterz@infradead.org>,
	Mark Rutland <mark.rutland@arm.com>,
	Jinjie Ruan <ruanjinjie@huawei.com>,
	linux-arm-kernel@lists.infradead.org,
	linux-kernel@vger.kernel.org, bpf@vger.kernel.org,
	rmikey@meta.com, kernel-team@meta.com, maz@kernel.org,
	ada.coupriediaz@arm.com, vladimir.murzin@arm.com
Subject: Re: [PATCH RFC] arm64: entry: PSTATE_I_SET is leaking on pseudo NMI mode
Date: Fri, 7 Aug 2026 14:40:41 +0100	[thread overview]
Message-ID: <anXgWRmcjwPKG7N5@willie-the-truck> (raw)
In-Reply-To: <20260807-arm64_fix-v1-1-d069ccf9d71b@debian.org>

[+Marc, Ada and Vladimir]

On Fri, Aug 07, 2026 at 04:45:31AM -0700, Breno Leitao wrote:
> Running retsnoop on a kernel with GIC priority masking and
> CONFIG_ARM64_DEBUG_PRIORITY_MASKING=y trips the ICC_PMR_EL1 sanity check
> in __pmr_local_irq_disable():
> 
>      WARNING: ./arch/arm64/include/asm/irqflags.h:63 at arm64_exit_to_kernel_mode+0xb8/0xc0, CPU#40: retsnoop/31805

That's warning because we're trying to disable interrupts in the PMR but
the existing PMR value is not IRQON or IRQOFF. The implication later is
that GIC_PRIO_PSR_I_SET is set.

>      CPU: 40 UID: 0 PID: 31805 Comm: retsnoop Not tainted 7.2.0-rc6-next-20260805 #7 PREEMPTLAZY
>      pstate: 234013c9 (nzCv DAIF +PAN -UAO +TCO +DIT +SSBS BTYPE=--)
>      pc : arm64_exit_to_kernel_mode (arch/arm64/kernel/entry-common.c:63)
>      lr : el1_abort (arch/arm64/kernel/entry-common.c:323)
>      pmr: 000000f0
>      Call trace:
> D)    arm64_exit_to_kernel_mode (arch/arm64/kernel/entry-common.c:63) (P)
>       el1_abort (arch/arm64/kernel/entry-common.c:323)
>       el1h_64_sync_handler (arch/arm64/kernel/entry-common.c:449)
> C)    el1h_64_sync (arch/arm64/kernel/entry.S:589)
>       copy_from_kernel_nofault (mm/maccess.c:52) (P)
>       bpf_probe_read_kernel (kernel/trace/bpf_trace.c:268)
>       bpf_prog_db21a1730c2407e5_calib_exit+0xf0/0x160
>       trace_call_bpf (kernel/trace/bpf_trace.c:147)
>       kretprobe_perf_func (kernel/trace/trace_kprobe.c:1750)
>       kretprobe_dispatcher (kernel/trace/trace_kprobe.c:1875)
>       __kretprobe_trampoline_handler (kernel/kprobes.c:2116)
>       kretprobe_brk_handler (arch/arm64/kernel/probes/kprobes.c:422)
>       call_el1_break_hook (arch/arm64/kernel/debug-monitors.c:244)
>       do_el1_brk64 (arch/arm64/kernel/debug-monitors.c:266)
> B)    el1_brk64 (arch/arm64/kernel/entry-common.c:427)
>       el1h_64_sync_handler (arch/arm64/kernel/entry-common.c:481)
>       el1h_64_sync (arch/arm64/kernel/entry.S:589)
>       invoke_syscall (arch/arm64/kernel/syscall.c:49) (P)
>       do_el0_svc (arch/arm64/kernel/syscall.c:140)
>       el0_svc (arch/arm64/kernel/entry-common.c:736)
>       el0t_64_sync_handler (arch/arm64/kernel/entry-common.c:755)
> A)    el0t_64_sync (arch/arm64/kernel/entry.S:594)
> 
> This is my understand of the current situation:

Just a couple of nits that might help others:

  - The description refers to PSTATE_I_SET, which doesn't exist
  - The steps refer to 'regs->PMR' as though it's always the same memory
    location

> A) A task enters the kernel via a syscall.
> 	* PSTATE_I_SET becomes set on the live PMR
> 	* regs->PMR doesn't have PSR_I_SET set

Hmm. My (possibly incorrect) reading of the code here is that the syscall
entry path will clear GIC_PRIO_PSR_I_SET in the live PMR register when
unmasking interrupts via local_daif_restore(DAIF_PROCCTX) ...

> B) A BRK fires at EL1 and that is what leaves the live PMR
>    with PSR_I_SET.
> 	* At this stage PSTATE_I_SET is set on both on PMR and regs->PMR

... so the EL1 debug handler for the BRK shouldn't save a PMR value with
GIC_PRIO_PSR_I_SET into the regs. We leave interrupts disabled for debug
exceptions, so I agree that the PMR register retains GIC_PRIO_PSR_I_SET.

> C) A nested synchronous exception happens (Not sure why -- BPF related)
> 	* Now both the live PMR and regs->pmr have PSR_I_SET.

That looks like it's the case and local_daif_inherit() won't change
anything. I wonder if we should drop GIC_PRIO_PSR_I_SET when writing
the PMR there?

Will

(retaining the rest of the mail for the folks I've added)

> D) On the way out, local_irq_disable() → __pmr_local_irq_disable() warns.
>    It detects that PMR is different than GIC_PRIO_IRQON and GIC_PRIO_IRQOFF,
>    given live PMR and regs->PMR have PSTATE_I_SET ORed.
>
> How to fix it? I don't know very well.
> 
> I am not certain that we want to have PSTATE_I_SET ever be sent to
> regs->pstate. Do we ever need PSTATE_I_SET in regs->pstate?
> 
> In the current patch, I found that disabling IRQ in case it is disabled,
> would solve the warning, but, this seems more a hack than a proper fix,
> perhaps.
> 
> Fixes: ae654112eac0 ("arm64: entry: Use split preemption logic")
> Signed-off-by: Breno Leitao <leitao@debian.org>
> ---
>  arch/arm64/kernel/entry-common.c | 8 +++++++-
>  1 file changed, 7 insertions(+), 1 deletion(-)
> 
> diff --git a/arch/arm64/kernel/entry-common.c b/arch/arm64/kernel/entry-common.c
> index ceb4eb11232a6..fcce9ccd37108 100644
> --- a/arch/arm64/kernel/entry-common.c
> +++ b/arch/arm64/kernel/entry-common.c
> @@ -55,7 +55,13 @@ static noinstr irqentry_state_t arm64_enter_from_kernel_mode(struct pt_regs *reg
>  static void noinstr arm64_exit_to_kernel_mode(struct pt_regs *regs,
>  					      irqentry_state_t state)
>  {
> -	local_irq_disable();
> +	/*
> +	 * Only irqentry_exit_to_kernel_mode_preempt() needs interrupts masked,
> +	 * and it returns early when regs had them disabled. Skipping the
> +	 * disable avoids clobbering a PMR the irqflags API does not expect.
> +	 */
> +	if (!regs_irqs_disabled(regs))
> +		local_irq_disable();
>  	irqentry_exit_to_kernel_mode_preempt(regs, state);
>  	local_daif_mask();
>  	mte_check_tfsr_exit();
> 
> ---
> base-commit: ea2bff00da89d7767d677bb68470130ba96f4928
> change-id: 20260807-arm64_fix-47cad8fb6323
> 
> Best regards,
> --  
> Breno Leitao <leitao@debian.org>
> 

WARNING: multiple messages have this Message-ID (diff)
From: Will Deacon <will@kernel.org>
To: Breno Leitao <leitao@debian.org>
Cc: Mark Rutland <mark.rutland@arm.com>,
	vladimir.murzin@arm.com, rmikey@meta.com,
	"Peter Zijlstra \(Intel\)" <peterz@infradead.org>,
	Catalin Marinas <catalin.marinas@arm.com>,
	Jinjie Ruan <ruanjinjie@huawei.com>,
	kernel-team@meta.com, linux-kernel@vger.kernel.org,
	maz@kernel.org, bpf@vger.kernel.org,
	linux-arm-kernel@lists.infradead.org
Subject: Re: [PATCH RFC] arm64: entry: PSTATE_I_SET is leaking on pseudo NMI mode
Date: Fri, 7 Aug 2026 14:40:41 +0100	[thread overview]
Message-ID: <anXgWRmcjwPKG7N5@willie-the-truck> (raw)
In-Reply-To: <20260807-arm64_fix-v1-1-d069ccf9d71b@debian.org>

[+Marc, Ada and Vladimir]

On Fri, Aug 07, 2026 at 04:45:31AM -0700, Breno Leitao wrote:
> Running retsnoop on a kernel with GIC priority masking and
> CONFIG_ARM64_DEBUG_PRIORITY_MASKING=y trips the ICC_PMR_EL1 sanity check
> in __pmr_local_irq_disable():
> 
>      WARNING: ./arch/arm64/include/asm/irqflags.h:63 at arm64_exit_to_kernel_mode+0xb8/0xc0, CPU#40: retsnoop/31805

That's warning because we're trying to disable interrupts in the PMR but
the existing PMR value is not IRQON or IRQOFF. The implication later is
that GIC_PRIO_PSR_I_SET is set.

>      CPU: 40 UID: 0 PID: 31805 Comm: retsnoop Not tainted 7.2.0-rc6-next-20260805 #7 PREEMPTLAZY
>      pstate: 234013c9 (nzCv DAIF +PAN -UAO +TCO +DIT +SSBS BTYPE=--)
>      pc : arm64_exit_to_kernel_mode (arch/arm64/kernel/entry-common.c:63)
>      lr : el1_abort (arch/arm64/kernel/entry-common.c:323)
>      pmr: 000000f0
>      Call trace:
> D)    arm64_exit_to_kernel_mode (arch/arm64/kernel/entry-common.c:63) (P)
>       el1_abort (arch/arm64/kernel/entry-common.c:323)
>       el1h_64_sync_handler (arch/arm64/kernel/entry-common.c:449)
> C)    el1h_64_sync (arch/arm64/kernel/entry.S:589)
>       copy_from_kernel_nofault (mm/maccess.c:52) (P)
>       bpf_probe_read_kernel (kernel/trace/bpf_trace.c:268)
>       bpf_prog_db21a1730c2407e5_calib_exit+0xf0/0x160
>       trace_call_bpf (kernel/trace/bpf_trace.c:147)
>       kretprobe_perf_func (kernel/trace/trace_kprobe.c:1750)
>       kretprobe_dispatcher (kernel/trace/trace_kprobe.c:1875)
>       __kretprobe_trampoline_handler (kernel/kprobes.c:2116)
>       kretprobe_brk_handler (arch/arm64/kernel/probes/kprobes.c:422)
>       call_el1_break_hook (arch/arm64/kernel/debug-monitors.c:244)
>       do_el1_brk64 (arch/arm64/kernel/debug-monitors.c:266)
> B)    el1_brk64 (arch/arm64/kernel/entry-common.c:427)
>       el1h_64_sync_handler (arch/arm64/kernel/entry-common.c:481)
>       el1h_64_sync (arch/arm64/kernel/entry.S:589)
>       invoke_syscall (arch/arm64/kernel/syscall.c:49) (P)
>       do_el0_svc (arch/arm64/kernel/syscall.c:140)
>       el0_svc (arch/arm64/kernel/entry-common.c:736)
>       el0t_64_sync_handler (arch/arm64/kernel/entry-common.c:755)
> A)    el0t_64_sync (arch/arm64/kernel/entry.S:594)
> 
> This is my understand of the current situation:

Just a couple of nits that might help others:

  - The description refers to PSTATE_I_SET, which doesn't exist
  - The steps refer to 'regs->PMR' as though it's always the same memory
    location

> A) A task enters the kernel via a syscall.
> 	* PSTATE_I_SET becomes set on the live PMR
> 	* regs->PMR doesn't have PSR_I_SET set

Hmm. My (possibly incorrect) reading of the code here is that the syscall
entry path will clear GIC_PRIO_PSR_I_SET in the live PMR register when
unmasking interrupts via local_daif_restore(DAIF_PROCCTX) ...

> B) A BRK fires at EL1 and that is what leaves the live PMR
>    with PSR_I_SET.
> 	* At this stage PSTATE_I_SET is set on both on PMR and regs->PMR

... so the EL1 debug handler for the BRK shouldn't save a PMR value with
GIC_PRIO_PSR_I_SET into the regs. We leave interrupts disabled for debug
exceptions, so I agree that the PMR register retains GIC_PRIO_PSR_I_SET.

> C) A nested synchronous exception happens (Not sure why -- BPF related)
> 	* Now both the live PMR and regs->pmr have PSR_I_SET.

That looks like it's the case and local_daif_inherit() won't change
anything. I wonder if we should drop GIC_PRIO_PSR_I_SET when writing
the PMR there?

Will

(retaining the rest of the mail for the folks I've added)

> D) On the way out, local_irq_disable() → __pmr_local_irq_disable() warns.
>    It detects that PMR is different than GIC_PRIO_IRQON and GIC_PRIO_IRQOFF,
>    given live PMR and regs->PMR have PSTATE_I_SET ORed.
>
> How to fix it? I don't know very well.
> 
> I am not certain that we want to have PSTATE_I_SET ever be sent to
> regs->pstate. Do we ever need PSTATE_I_SET in regs->pstate?
> 
> In the current patch, I found that disabling IRQ in case it is disabled,
> would solve the warning, but, this seems more a hack than a proper fix,
> perhaps.
> 
> Fixes: ae654112eac0 ("arm64: entry: Use split preemption logic")
> Signed-off-by: Breno Leitao <leitao@debian.org>
> ---
>  arch/arm64/kernel/entry-common.c | 8 +++++++-
>  1 file changed, 7 insertions(+), 1 deletion(-)
> 
> diff --git a/arch/arm64/kernel/entry-common.c b/arch/arm64/kernel/entry-common.c
> index ceb4eb11232a6..fcce9ccd37108 100644
> --- a/arch/arm64/kernel/entry-common.c
> +++ b/arch/arm64/kernel/entry-common.c
> @@ -55,7 +55,13 @@ static noinstr irqentry_state_t arm64_enter_from_kernel_mode(struct pt_regs *reg
>  static void noinstr arm64_exit_to_kernel_mode(struct pt_regs *regs,
>  					      irqentry_state_t state)
>  {
> -	local_irq_disable();
> +	/*
> +	 * Only irqentry_exit_to_kernel_mode_preempt() needs interrupts masked,
> +	 * and it returns early when regs had them disabled. Skipping the
> +	 * disable avoids clobbering a PMR the irqflags API does not expect.
> +	 */
> +	if (!regs_irqs_disabled(regs))
> +		local_irq_disable();
>  	irqentry_exit_to_kernel_mode_preempt(regs, state);
>  	local_daif_mask();
>  	mte_check_tfsr_exit();
> 
> ---
> base-commit: ea2bff00da89d7767d677bb68470130ba96f4928
> change-id: 20260807-arm64_fix-47cad8fb6323
> 
> Best regards,
> --  
> Breno Leitao <leitao@debian.org>
> 


  reply	other threads:[~2026-08-07 13:40 UTC|newest]

Thread overview: 31+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-08-07 11:45 [PATCH RFC] arm64: entry: PSTATE_I_SET is leaking on pseudo NMI mode Breno Leitao
2026-08-07 13:40 ` Will Deacon [this message]
2026-08-07 13:40   ` Will Deacon
2026-08-07 13:57 ` Mark Rutland
2026-08-07 13:57   ` Mark Rutland
2026-08-07 14:49   ` Vladimir Murzin
2026-08-07 14:49     ` Vladimir Murzin
2026-08-10 12:44     ` Jinjie Ruan
2026-08-10 12:44       ` Jinjie Ruan
2026-08-10 13:04       ` Vladimir Murzin
2026-08-10 13:04         ` Vladimir Murzin
2026-08-11  2:47         ` Jinjie Ruan
2026-08-11  2:47           ` Jinjie Ruan
2026-08-11  8:18           ` Vladimir Murzin
2026-08-11  8:18             ` Vladimir Murzin
2026-08-07 14:58   ` Breno Leitao
2026-08-07 14:58     ` Breno Leitao
2026-08-07 16:29     ` Breno Leitao
2026-08-07 16:29       ` Breno Leitao
2026-08-10 11:43       ` Will Deacon
2026-08-10 11:43         ` Will Deacon
2026-08-10 12:42         ` Vladimir Murzin
2026-08-10 12:42           ` Vladimir Murzin
2026-08-10 15:05           ` Will Deacon
2026-08-10 15:05             ` Will Deacon
2026-08-10 16:39             ` Vladimir Murzin
2026-08-10 16:39               ` Vladimir Murzin
2026-08-10 12:51         ` Jinjie Ruan
2026-08-10 12:51           ` Jinjie Ruan
2026-08-10 12:59         ` Breno Leitao
2026-08-10 12:59           ` Breno Leitao

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=anXgWRmcjwPKG7N5@willie-the-truck \
    --to=will@kernel.org \
    --cc=ada.coupriediaz@arm.com \
    --cc=bpf@vger.kernel.org \
    --cc=catalin.marinas@arm.com \
    --cc=kernel-team@meta.com \
    --cc=leitao@debian.org \
    --cc=linux-arm-kernel@lists.infradead.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mark.rutland@arm.com \
    --cc=maz@kernel.org \
    --cc=peterz@infradead.org \
    --cc=rmikey@meta.com \
    --cc=ruanjinjie@huawei.com \
    --cc=vladimir.murzin@arm.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.