[RFC v2 0/3] arm64: kprobes: Fix single-step fault and reentry handling

[RFC v2 0/3] arm64: kprobes: Fix single-step fault and reentry handling

[Pu Hu]

From: Pu Hu <hupu@transsion.com>

This series fixes two arm64 kprobes issues observed when running
simpleperf with preemptirq tracepoints and dwarf callchains while a
kprobe is active on a frequently executed kernel function.

The crash happens in the kprobe debug exception path. While a kprobe is
preparing or executing its XOL single-step instruction, perf/trace code
can run in the same window. That code may either take a fault of its own
or hit another kprobe.

Patch 1 fixes kprobe_fault_handler() so that it only handles a fault
taken in KPROBE_HIT_SS or KPROBE_REENTER state when the faulting PC
points at the current kprobe's XOL instruction. Simulated kprobes,
which have no XOL slot at all, are filtered out at function entry so
that the normal page fault handler (including fixup_exception) can
process any fault without interference.

Patch 2 allows a kprobe hit in KPROBE_HIT_SS to be handled as a
recoverable one-level reentry, instead of treating it as unrecoverable.
This is safe because the reentry save area has not yet been consumed at
that point. Only a hit while already in KPROBE_REENTER remains
unrecoverable.

Patch 3 extends struct prev_kprobe with a saved_irqflag field so that
the outer kprobe's original DAIF state is preserved across reentry.
Without this, a nested kprobe from Patch 2 would overwrite
kcb->saved_irqflag, and the outer kprobe would restore the wrong DAIF
mask on completion, potentially leaving interrupts permanently disabled.

This follows the same logic as the existing x86 fixes:

  6381c24cd6d5 ("kprobes/x86: Fix page-fault handling logic")
  6a5022a56ac3 ("kprobes/x86: Allow to handle reentered kprobe on single-stepping")

v1 -> v2:
  - Patch 1: moved simulated kprobe check to function entry (per
    maintainer review); removed redundant xol_insn NULL check from
    the inner branch.
  - Patch 2: unchanged.
  - Patch 3: new in v2; fixes the IRQ flag save/restore gap that
    Patch 2 exposes.
  - Removed the selftest patch (old Patch 3) from this series.
  - Fixed Signed-off-by to use full name (Pu Hu) instead of username
    (hupu).
  - Updated comments and commit messages across all patches.

Reproducer:
  simpleperf record -p <pid> -f 10000 \
    -e preemptirq:preempt_disable \
    -e preemptirq:preempt_enable \
    --duration 9 --call-graph dwarf \
    -o /data/local/tmp/perf.data

Before this series, the crash reproduced frequently. With all three
patches applied, it was no longer reproduced in our testing.

Pu Hu (3):
  arm64: kprobes: Only handle faults originating from XOL slot
  arm64: kprobes: Allow reentering kprobes while single-stepping
  arm64: kprobes: Save and restore saved_irqflag in prev_kprobe

 arch/arm64/include/asm/kprobes.h   |  6 ++++
 arch/arm64/kernel/probes/kprobes.c | 45 +++++++++++++++++++++++++++++-
 2 files changed, 50 insertions(+), 1 deletion(-)

-- 
2.43.0

[RFC v2 1/3] arm64: kprobes: Only handle faults originating from XOL slot

[Pu Hu]

From: Pu Hu <hupu@transsion.com>

kprobe_fault_handler() currently treats any page fault taken while in
KPROBE_HIT_SS or KPROBE_REENTER state as a kprobe single-step fault. This
assumption does not hold: perf or tracing code may run from the debug
exception path during the single-step window and take its own page fault.

When the fault is handled as a kprobe fault, the PC is rewritten to the
probe address, corrupting the exception recovery context for the real
fault. A typical reproducer is running perf with preemptirq tracepoints
and dwarf callchains while a kprobe is installed on a frequently
executed function.

Fix this in two layers:

1. At function entry, bail out immediately for simulated kprobes
   (ainsn.xol_insn == NULL), since they have no XOL slot and any fault
   taken during their execution cannot be a single-step fault.

2. For kprobes with an XOL slot, only handle the fault when the
   faulting PC matches the XOL instruction address. Faults from any
   other PC are left to the normal page fault handler.

This follows the same principle as the x86 fix in commit 6381c24cd6d5
("kprobes/x86: Fix page-fault handling logic").

Signed-off-by: Pu Hu <hupu@transsion.com>
Signed-off-by: Hongyan Xia <hongyan.xia@transsion.com>
---
 arch/arm64/kernel/probes/kprobes.c | 22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)

diff --git a/arch/arm64/kernel/probes/kprobes.c b/arch/arm64/kernel/probes/kprobes.c
index 43a0361a8bf0..798e4b091d1a 100644
--- a/arch/arm64/kernel/probes/kprobes.c
+++ b/arch/arm64/kernel/probes/kprobes.c
@@ -282,9 +282,31 @@ int __kprobes kprobe_fault_handler(struct pt_regs *regs, unsigned int fsr)
 	struct kprobe *cur = kprobe_running();
 	struct kprobe_ctlblk *kcb = get_kprobe_ctlblk();
 
+	/*
+	 * Simulated kprobes execute in the debug trap context and have no
+	 * XOL slot. Any page fault taken while a simulated kprobe is in
+	 * progress cannot have been caused by kprobe single-stepping and
+	 * must be left alone for the normal page fault handler, including
+	 * fixup_exception.
+	 */
+	if (cur && !cur->ainsn.xol_insn)
+		return 0;
+
 	switch (kcb->kprobe_status) {
 	case KPROBE_HIT_SS:
 	case KPROBE_REENTER:
+		/*
+		 * A page fault taken while in KPROBE_HIT_SS or
+		 * KPROBE_REENTER state is only attributable to kprobe
+		 * single-stepping if the faulting PC points to the
+		 * current kprobe's XOL instruction. If the fault occurred
+		 * elsewhere (e.g. in perf or tracing code invoked from the
+		 * debug exception path), leave it for the normal page fault
+		 * handler to process.
+		 */
+		if (instruction_pointer(regs) != (unsigned long)cur->ainsn.xol_insn)
+			break;
+
 		/*
 		 * We are here because the instruction being single
 		 * stepped caused a page fault. We reset the current
-- 
2.43.0

Re: [RFC v2 1/3] arm64: kprobes: Only handle faults originating from XOL slot

[Masami Hiramatsu]

On Thu, 9 Jul 2026 14:22:23 +0000
Pu Hu <hupu@transsion.com> wrote:

> From: Pu Hu <hupu@transsion.com>
> 
> kprobe_fault_handler() currently treats any page fault taken while in
> KPROBE_HIT_SS or KPROBE_REENTER state as a kprobe single-step fault. This
> assumption does not hold: perf or tracing code may run from the debug
> exception path during the single-step window and take its own page fault.
> 
> When the fault is handled as a kprobe fault, the PC is rewritten to the
> probe address, corrupting the exception recovery context for the real
> fault. A typical reproducer is running perf with preemptirq tracepoints
> and dwarf callchains while a kprobe is installed on a frequently
> executed function.
> 
> Fix this in two layers:
> 
> 1. At function entry, bail out immediately for simulated kprobes
>    (ainsn.xol_insn == NULL), since they have no XOL slot and any fault
>    taken during their execution cannot be a single-step fault.
> 
> 2. For kprobes with an XOL slot, only handle the fault when the
>    faulting PC matches the XOL instruction address. Faults from any
>    other PC are left to the normal page fault handler.
> 
> This follows the same principle as the x86 fix in commit 6381c24cd6d5
> ("kprobes/x86: Fix page-fault handling logic").
> 
> Signed-off-by: Pu Hu <hupu@transsion.com>
> Signed-off-by: Hongyan Xia <hongyan.xia@transsion.com>

This looks good to me.

Reviewed-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>

Thanks,

> ---
>  arch/arm64/kernel/probes/kprobes.c | 22 ++++++++++++++++++++++
>  1 file changed, 22 insertions(+)
> 
> diff --git a/arch/arm64/kernel/probes/kprobes.c b/arch/arm64/kernel/probes/kprobes.c
> index 43a0361a8bf0..798e4b091d1a 100644
> --- a/arch/arm64/kernel/probes/kprobes.c
> +++ b/arch/arm64/kernel/probes/kprobes.c
> @@ -282,9 +282,31 @@ int __kprobes kprobe_fault_handler(struct pt_regs *regs, unsigned int fsr)
>  	struct kprobe *cur = kprobe_running();
>  	struct kprobe_ctlblk *kcb = get_kprobe_ctlblk();
>  
> +	/*
> +	 * Simulated kprobes execute in the debug trap context and have no
> +	 * XOL slot. Any page fault taken while a simulated kprobe is in
> +	 * progress cannot have been caused by kprobe single-stepping and
> +	 * must be left alone for the normal page fault handler, including
> +	 * fixup_exception.
> +	 */
> +	if (cur && !cur->ainsn.xol_insn)
> +		return 0;
> +
>  	switch (kcb->kprobe_status) {
>  	case KPROBE_HIT_SS:
>  	case KPROBE_REENTER:
> +		/*
> +		 * A page fault taken while in KPROBE_HIT_SS or
> +		 * KPROBE_REENTER state is only attributable to kprobe
> +		 * single-stepping if the faulting PC points to the
> +		 * current kprobe's XOL instruction. If the fault occurred
> +		 * elsewhere (e.g. in perf or tracing code invoked from the
> +		 * debug exception path), leave it for the normal page fault
> +		 * handler to process.
> +		 */
> +		if (instruction_pointer(regs) != (unsigned long)cur->ainsn.xol_insn)
> +			break;
> +
>  		/*
>  		 * We are here because the instruction being single
>  		 * stepped caused a page fault. We reset the current
> -- 
> 2.43.0
> 


-- 
Masami Hiramatsu (Google) <mhiramat@kernel.org>

[RFC v2 2/3] arm64: kprobes: Allow reentering kprobes while single-stepping

[Pu Hu]

From: Pu Hu <hupu@transsion.com>

A kprobe can be hit while another kprobe is in KPROBE_HIT_SS state. This
can happen when tracing or perf code runs from the debug exception path
while the first kprobe is preparing or executing its out-of-line
single-step instruction.

Currently arm64 treats a kprobe hit in KPROBE_HIT_SS as unrecoverable,
the same as a hit in KPROBE_REENTER. This is too strict. A hit in
KPROBE_HIT_SS is still a one-level reentry and can be handled by saving
the current kprobe state and setting up single-step for the new probe,
just like reentry from KPROBE_HIT_ACTIVE or KPROBE_HIT_SSDONE.

The truly unrecoverable case is hitting another kprobe while already in
KPROBE_REENTER, because the reentry save area has already been consumed.

Move KPROBE_HIT_SS to the recoverable reentry cases and leave
KPROBE_REENTER as the unrecoverable nested reentry case.

This mirrors the x86 fix in commit 6a5022a56ac3
("kprobes/x86: Allow to handle reentered kprobe on single-stepping").

Signed-off-by: Pu Hu <hupu@transsion.com>
Signed-off-by: Hongyan Xia <hongyan.xia@transsion.com>
---
 arch/arm64/kernel/probes/kprobes.c | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/arch/arm64/kernel/probes/kprobes.c b/arch/arm64/kernel/probes/kprobes.c
index 798e4b091d1a..2ca5916eca2f 100644
--- a/arch/arm64/kernel/probes/kprobes.c
+++ b/arch/arm64/kernel/probes/kprobes.c
@@ -240,10 +240,16 @@ static int __kprobes reenter_kprobe(struct kprobe *p,
 	switch (kcb->kprobe_status) {
 	case KPROBE_HIT_SSDONE:
 	case KPROBE_HIT_ACTIVE:
+	case KPROBE_HIT_SS:
+		/*
+		 * A probe can be hit while another kprobe is preparing or
+		 * executing its XOL single-step instruction. This is still a
+		 * recoverable one-level reentry, so handle it in the same way as
+		 * reentry from KPROBE_HIT_ACTIVE or KPROBE_HIT_SSDONE.
+		 */
 		kprobes_inc_nmissed_count(p);
 		setup_singlestep(p, regs, kcb, 1);
 		break;
-	case KPROBE_HIT_SS:
 	case KPROBE_REENTER:
 		pr_warn("Failed to recover from reentered kprobes.\n");
 		dump_kprobe(p);
-- 
2.43.0

Re: [RFC v2 2/3] arm64: kprobes: Allow reentering kprobes while single-stepping

[Masami Hiramatsu]

On Thu, 9 Jul 2026 14:22:25 +0000
Pu Hu <hupu@transsion.com> wrote:

> From: Pu Hu <hupu@transsion.com>
> 
> A kprobe can be hit while another kprobe is in KPROBE_HIT_SS state. This
> can happen when tracing or perf code runs from the debug exception path
> while the first kprobe is preparing or executing its out-of-line
> single-step instruction.
> 
> Currently arm64 treats a kprobe hit in KPROBE_HIT_SS as unrecoverable,
> the same as a hit in KPROBE_REENTER. This is too strict. A hit in
> KPROBE_HIT_SS is still a one-level reentry and can be handled by saving
> the current kprobe state and setting up single-step for the new probe,
> just like reentry from KPROBE_HIT_ACTIVE or KPROBE_HIT_SSDONE.
> 
> The truly unrecoverable case is hitting another kprobe while already in
> KPROBE_REENTER, because the reentry save area has already been consumed.
> 
> Move KPROBE_HIT_SS to the recoverable reentry cases and leave
> KPROBE_REENTER as the unrecoverable nested reentry case.
> 
> This mirrors the x86 fix in commit 6a5022a56ac3
> ("kprobes/x86: Allow to handle reentered kprobe on single-stepping").
> 

Hi, as Sashiko commented, we have to save the saved_irqflag to
prev_kprobbe.

https://sashiko.dev/#/patchset/20260709142215.226872-1-hupu%40transsion.com?part=2

Thank you,

> Signed-off-by: Pu Hu <hupu@transsion.com>
> Signed-off-by: Hongyan Xia <hongyan.xia@transsion.com>
> ---
>  arch/arm64/kernel/probes/kprobes.c | 8 +++++++-
>  1 file changed, 7 insertions(+), 1 deletion(-)
> 
> diff --git a/arch/arm64/kernel/probes/kprobes.c b/arch/arm64/kernel/probes/kprobes.c
> index 798e4b091d1a..2ca5916eca2f 100644
> --- a/arch/arm64/kernel/probes/kprobes.c
> +++ b/arch/arm64/kernel/probes/kprobes.c
> @@ -240,10 +240,16 @@ static int __kprobes reenter_kprobe(struct kprobe *p,
>  	switch (kcb->kprobe_status) {
>  	case KPROBE_HIT_SSDONE:
>  	case KPROBE_HIT_ACTIVE:
> +	case KPROBE_HIT_SS:
> +		/*
> +		 * A probe can be hit while another kprobe is preparing or
> +		 * executing its XOL single-step instruction. This is still a
> +		 * recoverable one-level reentry, so handle it in the same way as
> +		 * reentry from KPROBE_HIT_ACTIVE or KPROBE_HIT_SSDONE.
> +		 */
>  		kprobes_inc_nmissed_count(p);
>  		setup_singlestep(p, regs, kcb, 1);
>  		break;
> -	case KPROBE_HIT_SS:
>  	case KPROBE_REENTER:
>  		pr_warn("Failed to recover from reentered kprobes.\n");
>  		dump_kprobe(p);
> -- 
> 2.43.0
> 


-- 
Masami Hiramatsu (Google) <mhiramat@kernel.org>

Re: [RFC v2 2/3] arm64: kprobes: Allow reentering kprobes while single-stepping

[Pu Hu]

On 7/10/2026 1:09 PM, Masami Hiramatsu wrote:
> On Thu, 9 Jul 2026 14:22:25 +0000
> Pu Hu <hupu@transsion.com> wrote:
>
>> From: Pu Hu <hupu@transsion.com>
>>
>> A kprobe can be hit while another kprobe is in KPROBE_HIT_SS state. This
>> can happen when tracing or perf code runs from the debug exception path
>> while the first kprobe is preparing or executing its out-of-line
>> single-step instruction.
>>
>> Currently arm64 treats a kprobe hit in KPROBE_HIT_SS as unrecoverable,
>> the same as a hit in KPROBE_REENTER. This is too strict. A hit in
>> KPROBE_HIT_SS is still a one-level reentry and can be handled by saving
>> the current kprobe state and setting up single-step for the new probe,
>> just like reentry from KPROBE_HIT_ACTIVE or KPROBE_HIT_SSDONE.
>>
>> The truly unrecoverable case is hitting another kprobe while already in
>> KPROBE_REENTER, because the reentry save area has already been consumed.
>>
>> Move KPROBE_HIT_SS to the recoverable reentry cases and leave
>> KPROBE_REENTER as the unrecoverable nested reentry case.
>>
>> This mirrors the x86 fix in commit 6a5022a56ac3
>> ("kprobes/x86: Allow to handle reentered kprobe on single-stepping").
>>
>
> Hi, as Sashiko commented, we have to save the saved_irqflag to
> prev_kprobbe.
>
> https://sashiko.dev/#/patchset/20260709142215.226872-1-hupu%40transsion.com?part=2
>
> Thank you,
>

Hi Masami,

We already added the saved_irqflag support in Patch 3 of this series:

   arm64: kprobes: Save and restore saved_irqflag in prev_kprobe

However, I see the issue. Patch 2 and Patch 3 are separate commits,
so Patch 2 alone is not bisectable. I'll prepare a v3 that folds them
together, so the reentry logic and the saved_irqflag preservation land
in a single commit.

Thanks for pointing this out.

Thanks,
Pu Hu

[RFC v2 3/3] arm64: kprobes: Save and restore saved_irqflag in prev_kprobe

[Pu Hu]

From: Pu Hu <hupu@transsion.com>

When a kprobe is reentered from KPROBE_HIT_SS state (allowed by the
previous patch), setup_singlestep() for the nested kprobe calls
kprobes_save_local_irqflag(), overwriting kcb->saved_irqflag with the
currently masked DAIF value. The outer kprobe's original DAIF state is
lost.

When the nested kprobe completes, restore_previous_kprobe() brings back
the outer kprobe's kp and status, but saved_irqflag still contains the
nested kprobe's value. When the outer kprobe's single-step eventually
finishes, kprobes_restore_local_irqflag() applies the wrong DAIF mask,
leaving interrupts permanently disabled.

Fix this by extending struct prev_kprobe with a saved_irqflag field, and
saving/restoring it alongside kp and status. This ensures the outer
kprobe's original interrupt state is preserved across reentry.

Signed-off-by: Pu Hu <hupu@transsion.com>
Signed-off-by: Hongyan Xia <hongyan.xia@transsion.com>
---
 arch/arm64/include/asm/kprobes.h   |  6 ++++++
 arch/arm64/kernel/probes/kprobes.c | 15 +++++++++++++++
 2 files changed, 21 insertions(+)

diff --git a/arch/arm64/include/asm/kprobes.h b/arch/arm64/include/asm/kprobes.h
index f2782560647b..35ce2c94040e 100644
--- a/arch/arm64/include/asm/kprobes.h
+++ b/arch/arm64/include/asm/kprobes.h
@@ -26,6 +26,12 @@
 struct prev_kprobe {
 	struct kprobe *kp;
 	unsigned int status;
+
+	/*
+	 * The original DAIF state of the outer kprobe, saved here before
+	 * a nested kprobe overwrites kcb->saved_irqflag during reentry.
+	 */
+	unsigned long saved_irqflag;
 };
 
 /* per-cpu kprobe control block */
diff --git a/arch/arm64/kernel/probes/kprobes.c b/arch/arm64/kernel/probes/kprobes.c
index 2ca5916eca2f..4e0efad5caf2 100644
--- a/arch/arm64/kernel/probes/kprobes.c
+++ b/arch/arm64/kernel/probes/kprobes.c
@@ -174,12 +174,27 @@ static void __kprobes save_previous_kprobe(struct kprobe_ctlblk *kcb)
 {
 	kcb->prev_kprobe.kp = kprobe_running();
 	kcb->prev_kprobe.status = kcb->kprobe_status;
+
+	/*
+	 * Save the outer kprobe's original DAIF flags before the nested
+	 * kprobe calls kprobes_save_local_irqflag() and overwrites
+	 * kcb->saved_irqflag. Without this, the outer kprobe will restore
+	 * the wrong DAIF state and leave interrupts permanently masked.
+	 */
+	kcb->prev_kprobe.saved_irqflag = kcb->saved_irqflag;
 }
 
 static void __kprobes restore_previous_kprobe(struct kprobe_ctlblk *kcb)
 {
 	__this_cpu_write(current_kprobe, kcb->prev_kprobe.kp);
 	kcb->kprobe_status = kcb->prev_kprobe.status;
+
+	/*
+	 * Restore the outer kprobe's saved_irqflag so that when its
+	 * single-step completes, kprobes_restore_local_irqflag() uses
+	 * the correct original DAIF value.
+	 */
+	kcb->saved_irqflag = kcb->prev_kprobe.saved_irqflag;
 }
 
 static void __kprobes set_current_kprobe(struct kprobe *p)
-- 
2.43.0