From: Masami Hiramatsu (Google) <mhiramat@kernel.org>
To: Hongyan Xia <hongyan.xia@transsion.com>
Cc: Pu Hu <hupu@transsion.com>, Jiazi Li <jiazi.li@transsion.com>,
"catalin.marinas@arm.com" <catalin.marinas@arm.com>,
"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
"naveen@kernel.org" <naveen@kernel.org>,
"mhiramat@kernel.org" <mhiramat@kernel.org>,
"yang@os.amperecomputing.com" <yang@os.amperecomputing.com>,
Will Deacon <will@kernel.org>,
"davem@davemloft.net" <davem@davemloft.net>,
"linux-arm-kernel@lists.infradead.org"
<linux-arm-kernel@lists.infradead.org>,
"linux-trace-kernel@vger.kernel.org"
<linux-trace-kernel@vger.kernel.org>
Subject: Re: [RFC v3 2/2] arm64: kprobes: Allow reentering kprobes while single-stepping
Date: Sun, 19 Jul 2026 14:51:43 +0900 [thread overview]
Message-ID: <20260719145143.78fdbc359174e1180a5646be@kernel.org> (raw)
In-Reply-To: <be182a18-4cea-445d-a319-1ed3a6c17598@transsion.com>
On Fri, 17 Jul 2026 11:31:31 +0000
Hongyan Xia <hongyan.xia@transsion.com> wrote:
> On 7/17/2026 7:01 PM, Will Deacon wrote:
> > On Fri, Jul 17, 2026 at 01:51:12AM +0000, Hongyan Xia wrote:
> >> On 7/16/2026 11:20 PM, Will Deacon wrote:
> >>> On Thu, Jul 16, 2026 at 02:38:58PM +0000, Pu Hu wrote:
> >>>> On 7/16/2026 9:24 PM, Will Deacon wrote:
> >>>>> On Fri, Jul 10, 2026 at 06:32:55AM +0000, Pu Hu wrote:
> >>>>>> From: Pu Hu <hupu@transsion.com>
> >>>>>>
> >>>>>> A kprobe can be hit while another kprobe is in KPROBE_HIT_SS state. This
> >>>>>> can happen when tracing or perf code runs from the debug exception path
> >>>>>> while the first kprobe is preparing or executing its out-of-line
> >>>>>> single-step instruction.
> >>>>>
> >>>>> I don't understand this part. The single-step runs with debug exceptions
> >>>>> disabled (kprobes_save_local_irqflag() sets PSTATE.D) so how do we end
> >>>>> up taking one?
> >>>>
> >>>> You are right that the single-step runs with debug exceptions disabled.
> >>>> However, the case I was referring to is not a hardware breakpoint or a
> >>>> software-step exception, but another Breakpoint Instruction exception
> >>>> generated by executing a BRK instruction. A BRK instruction exception is
> >>>> not masked by PSTATE.D, so it can still be taken while handling a kprobe.
> >>>>
> >>>> As far as I understand the architecture, there are two different cases here:
> >>>>
> >>>> - Breakpoint Instruction exceptions, generated by executing a BRK
> >>>> instruction.
> >>>> - Breakpoint exceptions, generated by the debug logic, for example by
> >>>> programmed breakpoint registers.
> >>>>
> >>>> PSTATE.D masks debug exceptions such as hardware breakpoints,
> >>>> watchpoints and software-step exceptions, but it does not mask
> >>>> Breakpoint Instruction exceptions generated by BRK. This also seems
> >>>> consistent with the pseudocode for BRK,
> >>>> Arch64.SoftwareBreakpoint(imm16), which does not appear to check
> >>>> PSTATE.D before taking the exception.
> >>>>
> >>>> Therefore, even if kprobes_save_local_irqflag() sets PSTATE.D while
> >>>> handling the first kprobe, if the code executed from that path reaches
> >>>> another instruction patched with BRK, it can still take a Breakpoint
> >>>> Instruction exception. In other words, the nested case I mentioned is
> >>>> another kprobe BRK being hit, not a hardware debug exception or a
> >>>> software-step exception.
> >>>
> >>> Yes, that's correct, but if we're doing the out-of-line step, how do we
> >>> end up executing a BRK? Or are you saying that it's the kprobes
> >>> BRK64_OPCODE_KPROBES_SS instruction that we use to implement the
> >>> single-step that is the problem? If so, how does taking that exception
> >>> result in us executing tracing or perf code?
> >>>
> >>> Sorry for all the questions, I just haven't understood what's going on
> >>> here from the commit message.
> >>
> >> The key is that, when you use 'perf --call-graph dwarf' to sample
> >> certain events, kernel perf code will sample a piece of user stack each
> >> time those events are hit, and copy_to/from_user() triggers page faults.
> >> Say you are profiling preempt_enable events:
> >>
> >> 1st BRK -> preempt_disable() -> debug_exception() -> set SS state ->
> >> preempt_enable() -> triggers perf -> perf_sample() -> sample user stack
> >> using copy_to/from_user() -> page fault or 2nd BRK on the page fault path.
> >>
> >> The key is perf sampling the user stack while the 1st BRK is still
> >> running. When a page fault is hit, a can of worms is released, including
> >> a possible 2nd BRK.
> >
> > Thanks. So perf is run synchronously from the debug exception entry path,
>
> Yes, exactly.
>
> > rather than because of a second exception taking place. Got it. But then
> > it sounds like we should really make the debug exception handling path (at
> > least, the part that runs for handling the kprobe step) noinstr to avoid
> > getting into this state to begin with. Is that practical?
>
> Not sure about making the whole path noinstr (@Masami might have a
> better opinion on this than me). Personally I don't mind either
> disallowing it or making it correct.
Yeah I agree with making it noinstr.
>
> But it might be a good idea not to diverge too much between ISAs. This
> patch is pretty much mirroring what the x86 side handles this situation.
I don't mind modifying how it is handled in each architecture, since
this part is too much depending on the ISA.
Thanks,
--
Masami Hiramatsu (Google) <mhiramat@kernel.org>
next prev parent reply other threads:[~2026-07-19 5:52 UTC|newest]
Thread overview: 13+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-07-10 6:32 [RFC v3 0/2] rm64: kprobes: Fix single-step fault and reentry handling Pu Hu
[not found] ` <20260710063242.228714-3-hupu@transsion.com>
2026-07-10 9:38 ` [RFC v3 2/2] arm64: kprobes: Allow reentering kprobes while single-stepping Masami Hiramatsu
2026-07-15 13:56 ` Pu Hu
2026-07-16 13:24 ` Will Deacon
2026-07-16 14:38 ` Pu Hu
2026-07-16 15:20 ` Will Deacon
2026-07-17 1:51 ` Hongyan Xia
2026-07-17 11:01 ` Will Deacon
2026-07-17 11:31 ` Hongyan Xia
2026-07-17 18:02 ` Will Deacon
2026-07-18 3:17 ` Hongyan Xia
2026-07-19 5:51 ` Masami Hiramatsu [this message]
2026-07-17 18:23 ` [RFC v3 0/2] rm64: kprobes: Fix single-step fault and reentry handling Will Deacon
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20260719145143.78fdbc359174e1180a5646be@kernel.org \
--to=mhiramat@kernel.org \
--cc=catalin.marinas@arm.com \
--cc=davem@davemloft.net \
--cc=hongyan.xia@transsion.com \
--cc=hupu@transsion.com \
--cc=jiazi.li@transsion.com \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-trace-kernel@vger.kernel.org \
--cc=naveen@kernel.org \
--cc=will@kernel.org \
--cc=yang@os.amperecomputing.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox