[RFC v3 0/2] rm64: kprobes: Fix single-step fault and reentry handling

[RFC v3 0/2] rm64: kprobes: Fix single-step fault and reentry handling

[Pu Hu]

From: Pu Hu <hupu@transsion.com>

This series fixes two arm64 kprobes issues observed when running
simpleperf with preemptirq tracepoints and dwarf callchains while a
kprobe is active on a frequently executed kernel function.

The crash happens in the kprobe debug exception path. While a kprobe is
preparing or executing its XOL single-step instruction, perf/trace code
can run in the same window. That code may either take a fault of its own
or hit another kprobe.

Patch 1 fixes kprobe_fault_handler() so that it only handles a fault
taken in KPROBE_HIT_SS or KPROBE_REENTER state when the faulting PC
points at the current kprobe's XOL instruction. Simulated kprobes,
which have no XOL slot at all, are filtered out at function entry so
that the normal page fault handler (including fixup_exception) can
process any fault without interference.


Patch 2 allows a kprobe hit in KPROBE_HIT_SS to be handled as a
recoverable one-level reentry, instead of treating it as unrecoverable.
This is safe because the reentry save area has not yet been consumed at
that point. Only a hit while already in KPROBE_REENTER remains
unrecoverable. The same patch also extends struct prev_kprobe with a
saved_irqflag field so that the outer kprobe's original DAIF state is
preserved across reentry. Without this, the nested kprobe would
overwrite kcb->saved_irqflag, and the outer kprobe would restore the
wrong DAIF mask on completion, potentially leaving interrupts
permanently disabled.

This follows the same logic as the existing x86 fixes:

  6381c24cd6d5 ("kprobes/x86: Fix page-fault handling logic")
  6a5022a56ac3 ("kprobes/x86: Allow to handle reentered kprobe on single-stepping")

v2 -> v3:
  - Patch 1: unchanged.
  - Folded Patch 3 into Patch 2 so the saved_irqflag fix lands in the
    same commit as the reentry change, keeping the series bisectable.

v1 -> v2:
  - Patch 1: moved simulated kprobe check to function entry (per
    maintainer review); removed redundant xol_insn NULL check from
    the inner branch.
  - Patch 2: unchanged.
  - Patch 3: new in v2; fixes the IRQ flag save/restore gap that
    Patch 2 exposes.
  - Removed the selftest patch (old Patch 3) from this series.
  - Fixed Signed-off-by to use full name (Pu Hu) instead of username
    (hupu).
  - Updated comments and commit messages across all patches.

Reproducer:
  simpleperf record -p <pid> -f 10000 \
    -e preemptirq:preempt_disable \
    -e preemptirq:preempt_enable \
    --duration 9 --call-graph dwarf \
    -o /data/local/tmp/perf.data

Before this series, the crash reproduced frequently. With both patches
applied, it was no longer reproduced in our testing.

Pu Hu (2):
  arm64: kprobes: Only handle faults originating from XOL slot
  arm64: kprobes: Allow reentering kprobes while single-stepping

 arch/arm64/include/asm/kprobes.h   |  6 ++++
 arch/arm64/kernel/probes/kprobes.c | 45 +++++++++++++++++++++++++++++-
 2 files changed, 50 insertions(+), 1 deletion(-)

-- 
2.43.0

[RFC v3 1/2] arm64: kprobes: Only handle faults originating from XOL slot

[Pu Hu]

From: Pu Hu <hupu@transsion.com>

kprobe_fault_handler() currently treats any page fault taken while in
KPROBE_HIT_SS or KPROBE_REENTER state as a kprobe single-step fault. This
assumption does not hold: perf or tracing code may run from the debug
exception path during the single-step window and take its own page fault.

When the fault is handled as a kprobe fault, the PC is rewritten to the
probe address, corrupting the exception recovery context for the real
fault. A typical reproducer is running perf with preemptirq tracepoints
and dwarf callchains while a kprobe is installed on a frequently
executed function.

Fix this in two layers:

1. At function entry, bail out immediately for simulated kprobes
   (ainsn.xol_insn == NULL), since they have no XOL slot and any fault
   taken during their execution cannot be a single-step fault.

2. For kprobes with an XOL slot, only handle the fault when the
   faulting PC matches the XOL instruction address. Faults from any
   other PC are left to the normal page fault handler.

This follows the same principle as the x86 fix in commit 6381c24cd6d5
("kprobes/x86: Fix page-fault handling logic").

Signed-off-by: Pu Hu <hupu@transsion.com>
Signed-off-by: Hongyan Xia <hongyan.xia@transsion.com>
---
 arch/arm64/kernel/probes/kprobes.c | 22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)

diff --git a/arch/arm64/kernel/probes/kprobes.c b/arch/arm64/kernel/probes/kprobes.c
index 43a0361a8bf0..798e4b091d1a 100644
--- a/arch/arm64/kernel/probes/kprobes.c
+++ b/arch/arm64/kernel/probes/kprobes.c
@@ -282,9 +282,31 @@ int __kprobes kprobe_fault_handler(struct pt_regs *regs, unsigned int fsr)
 	struct kprobe *cur = kprobe_running();
 	struct kprobe_ctlblk *kcb = get_kprobe_ctlblk();
 
+	/*
+	 * Simulated kprobes execute in the debug trap context and have no
+	 * XOL slot. Any page fault taken while a simulated kprobe is in
+	 * progress cannot have been caused by kprobe single-stepping and
+	 * must be left alone for the normal page fault handler, including
+	 * fixup_exception.
+	 */
+	if (cur && !cur->ainsn.xol_insn)
+		return 0;
+
 	switch (kcb->kprobe_status) {
 	case KPROBE_HIT_SS:
 	case KPROBE_REENTER:
+		/*
+		 * A page fault taken while in KPROBE_HIT_SS or
+		 * KPROBE_REENTER state is only attributable to kprobe
+		 * single-stepping if the faulting PC points to the
+		 * current kprobe's XOL instruction. If the fault occurred
+		 * elsewhere (e.g. in perf or tracing code invoked from the
+		 * debug exception path), leave it for the normal page fault
+		 * handler to process.
+		 */
+		if (instruction_pointer(regs) != (unsigned long)cur->ainsn.xol_insn)
+			break;
+
 		/*
 		 * We are here because the instruction being single
 		 * stepped caused a page fault. We reset the current
-- 
2.43.0

[RFC v3 2/2] arm64: kprobes: Allow reentering kprobes while single-stepping

[Pu Hu]

From: Pu Hu <hupu@transsion.com>

A kprobe can be hit while another kprobe is in KPROBE_HIT_SS state. This
can happen when tracing or perf code runs from the debug exception path
while the first kprobe is preparing or executing its out-of-line
single-step instruction.

Currently arm64 treats a kprobe hit in KPROBE_HIT_SS as unrecoverable,
the same as a hit in KPROBE_REENTER. This is too strict. A hit in
KPROBE_HIT_SS is still a one-level reentry and can be handled by saving
the current kprobe state and setting up single-step for the new probe,
just like reentry from KPROBE_HIT_ACTIVE or KPROBE_HIT_SSDONE.

The truly unrecoverable case is hitting another kprobe while already in
KPROBE_REENTER, because the reentry save area has already been consumed.

Move KPROBE_HIT_SS to the recoverable reentry cases and leave
KPROBE_REENTER as the unrecoverable nested reentry case.

This change also requires saving saved_irqflag in struct prev_kprobe.
When a nested kprobe calls kprobes_save_local_irqflag(), it overwrites
kcb->saved_irqflag with the currently masked DAIF value, losing the
outer kprobe's original DAIF state. Without this fix, when the outer
kprobe's single-step finishes, kprobes_restore_local_irqflag() applies
the wrong DAIF mask and leaves interrupts permanently disabled.

Extend struct prev_kprobe with a saved_irqflag field and save/restore it
alongside kp and status. This ensures the outer kprobe's original
interrupt state is preserved across reentry.

This mirrors the x86 fix in commit 6a5022a56ac3
("kprobes/x86: Allow to handle reentered kprobe on single-stepping").

Signed-off-by: Pu Hu <hupu@transsion.com>
Signed-off-by: Hongyan Xia <hongyan.xia@transsion.com>
---
 arch/arm64/include/asm/kprobes.h   |  6 ++++++
 arch/arm64/kernel/probes/kprobes.c | 23 ++++++++++++++++++++++-
 2 files changed, 28 insertions(+), 1 deletion(-)

diff --git a/arch/arm64/include/asm/kprobes.h b/arch/arm64/include/asm/kprobes.h
index f2782560647b..35ce2c94040e 100644
--- a/arch/arm64/include/asm/kprobes.h
+++ b/arch/arm64/include/asm/kprobes.h
@@ -26,6 +26,12 @@
 struct prev_kprobe {
 	struct kprobe *kp;
 	unsigned int status;
+
+	/*
+	 * The original DAIF state of the outer kprobe, saved here before
+	 * a nested kprobe overwrites kcb->saved_irqflag during reentry.
+	 */
+	unsigned long saved_irqflag;
 };
 
 /* per-cpu kprobe control block */
diff --git a/arch/arm64/kernel/probes/kprobes.c b/arch/arm64/kernel/probes/kprobes.c
index 798e4b091d1a..4e0efad5caf2 100644
--- a/arch/arm64/kernel/probes/kprobes.c
+++ b/arch/arm64/kernel/probes/kprobes.c
@@ -174,12 +174,27 @@ static void __kprobes save_previous_kprobe(struct kprobe_ctlblk *kcb)
 {
 	kcb->prev_kprobe.kp = kprobe_running();
 	kcb->prev_kprobe.status = kcb->kprobe_status;
+
+	/*
+	 * Save the outer kprobe's original DAIF flags before the nested
+	 * kprobe calls kprobes_save_local_irqflag() and overwrites
+	 * kcb->saved_irqflag. Without this, the outer kprobe will restore
+	 * the wrong DAIF state and leave interrupts permanently masked.
+	 */
+	kcb->prev_kprobe.saved_irqflag = kcb->saved_irqflag;
 }
 
 static void __kprobes restore_previous_kprobe(struct kprobe_ctlblk *kcb)
 {
 	__this_cpu_write(current_kprobe, kcb->prev_kprobe.kp);
 	kcb->kprobe_status = kcb->prev_kprobe.status;
+
+	/*
+	 * Restore the outer kprobe's saved_irqflag so that when its
+	 * single-step completes, kprobes_restore_local_irqflag() uses
+	 * the correct original DAIF value.
+	 */
+	kcb->saved_irqflag = kcb->prev_kprobe.saved_irqflag;
 }
 
 static void __kprobes set_current_kprobe(struct kprobe *p)
@@ -240,10 +255,16 @@ static int __kprobes reenter_kprobe(struct kprobe *p,
 	switch (kcb->kprobe_status) {
 	case KPROBE_HIT_SSDONE:
 	case KPROBE_HIT_ACTIVE:
+	case KPROBE_HIT_SS:
+		/*
+		 * A probe can be hit while another kprobe is preparing or
+		 * executing its XOL single-step instruction. This is still a
+		 * recoverable one-level reentry, so handle it in the same way as
+		 * reentry from KPROBE_HIT_ACTIVE or KPROBE_HIT_SSDONE.
+		 */
 		kprobes_inc_nmissed_count(p);
 		setup_singlestep(p, regs, kcb, 1);
 		break;
-	case KPROBE_HIT_SS:
 	case KPROBE_REENTER:
 		pr_warn("Failed to recover from reentered kprobes.\n");
 		dump_kprobe(p);
-- 
2.43.0

Re: [RFC v3 2/2] arm64: kprobes: Allow reentering kprobes while single-stepping

[Masami Hiramatsu]

On Fri, 10 Jul 2026 06:32:55 +0000
Pu Hu <hupu@transsion.com> wrote:

> From: Pu Hu <hupu@transsion.com>
> 
> A kprobe can be hit while another kprobe is in KPROBE_HIT_SS state. This
> can happen when tracing or perf code runs from the debug exception path
> while the first kprobe is preparing or executing its out-of-line
> single-step instruction.
> 
> Currently arm64 treats a kprobe hit in KPROBE_HIT_SS as unrecoverable,
> the same as a hit in KPROBE_REENTER. This is too strict. A hit in
> KPROBE_HIT_SS is still a one-level reentry and can be handled by saving
> the current kprobe state and setting up single-step for the new probe,
> just like reentry from KPROBE_HIT_ACTIVE or KPROBE_HIT_SSDONE.
> 
> The truly unrecoverable case is hitting another kprobe while already in
> KPROBE_REENTER, because the reentry save area has already been consumed.
> 
> Move KPROBE_HIT_SS to the recoverable reentry cases and leave
> KPROBE_REENTER as the unrecoverable nested reentry case.
> 
> This change also requires saving saved_irqflag in struct prev_kprobe.
> When a nested kprobe calls kprobes_save_local_irqflag(), it overwrites
> kcb->saved_irqflag with the currently masked DAIF value, losing the
> outer kprobe's original DAIF state. Without this fix, when the outer
> kprobe's single-step finishes, kprobes_restore_local_irqflag() applies
> the wrong DAIF mask and leaves interrupts permanently disabled.
> 
> Extend struct prev_kprobe with a saved_irqflag field and save/restore it
> alongside kp and status. This ensures the outer kprobe's original
> interrupt state is preserved across reentry.
> 
> This mirrors the x86 fix in commit 6a5022a56ac3
> ("kprobes/x86: Allow to handle reentered kprobe on single-stepping").
> 

OK, this looks good to me.

Reviewed-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>

for this series.

Will, Catalin, can you pick this series?

Thanks!

> Signed-off-by: Pu Hu <hupu@transsion.com>
> Signed-off-by: Hongyan Xia <hongyan.xia@transsion.com>
> ---
>  arch/arm64/include/asm/kprobes.h   |  6 ++++++
>  arch/arm64/kernel/probes/kprobes.c | 23 ++++++++++++++++++++++-
>  2 files changed, 28 insertions(+), 1 deletion(-)
> 
> diff --git a/arch/arm64/include/asm/kprobes.h b/arch/arm64/include/asm/kprobes.h
> index f2782560647b..35ce2c94040e 100644
> --- a/arch/arm64/include/asm/kprobes.h
> +++ b/arch/arm64/include/asm/kprobes.h
> @@ -26,6 +26,12 @@
>  struct prev_kprobe {
>  	struct kprobe *kp;
>  	unsigned int status;
> +
> +	/*
> +	 * The original DAIF state of the outer kprobe, saved here before
> +	 * a nested kprobe overwrites kcb->saved_irqflag during reentry.
> +	 */
> +	unsigned long saved_irqflag;
>  };
>  
>  /* per-cpu kprobe control block */
> diff --git a/arch/arm64/kernel/probes/kprobes.c b/arch/arm64/kernel/probes/kprobes.c
> index 798e4b091d1a..4e0efad5caf2 100644
> --- a/arch/arm64/kernel/probes/kprobes.c
> +++ b/arch/arm64/kernel/probes/kprobes.c
> @@ -174,12 +174,27 @@ static void __kprobes save_previous_kprobe(struct kprobe_ctlblk *kcb)
>  {
>  	kcb->prev_kprobe.kp = kprobe_running();
>  	kcb->prev_kprobe.status = kcb->kprobe_status;
> +
> +	/*
> +	 * Save the outer kprobe's original DAIF flags before the nested
> +	 * kprobe calls kprobes_save_local_irqflag() and overwrites
> +	 * kcb->saved_irqflag. Without this, the outer kprobe will restore
> +	 * the wrong DAIF state and leave interrupts permanently masked.
> +	 */
> +	kcb->prev_kprobe.saved_irqflag = kcb->saved_irqflag;
>  }
>  
>  static void __kprobes restore_previous_kprobe(struct kprobe_ctlblk *kcb)
>  {
>  	__this_cpu_write(current_kprobe, kcb->prev_kprobe.kp);
>  	kcb->kprobe_status = kcb->prev_kprobe.status;
> +
> +	/*
> +	 * Restore the outer kprobe's saved_irqflag so that when its
> +	 * single-step completes, kprobes_restore_local_irqflag() uses
> +	 * the correct original DAIF value.
> +	 */
> +	kcb->saved_irqflag = kcb->prev_kprobe.saved_irqflag;
>  }
>  
>  static void __kprobes set_current_kprobe(struct kprobe *p)
> @@ -240,10 +255,16 @@ static int __kprobes reenter_kprobe(struct kprobe *p,
>  	switch (kcb->kprobe_status) {
>  	case KPROBE_HIT_SSDONE:
>  	case KPROBE_HIT_ACTIVE:
> +	case KPROBE_HIT_SS:
> +		/*
> +		 * A probe can be hit while another kprobe is preparing or
> +		 * executing its XOL single-step instruction. This is still a
> +		 * recoverable one-level reentry, so handle it in the same way as
> +		 * reentry from KPROBE_HIT_ACTIVE or KPROBE_HIT_SSDONE.
> +		 */
>  		kprobes_inc_nmissed_count(p);
>  		setup_singlestep(p, regs, kcb, 1);
>  		break;
> -	case KPROBE_HIT_SS:
>  	case KPROBE_REENTER:
>  		pr_warn("Failed to recover from reentered kprobes.\n");
>  		dump_kprobe(p);
> -- 
> 2.43.0
> 


-- 
Masami Hiramatsu (Google) <mhiramat@kernel.org>