From: Mark Rutland <mark.rutland@arm.com>
To: Ben Niu <BenNiu@meta.com>
Cc: catalin.marinas@arm.com, will@kernel.org, tytso@mit.edu,
Jason@zx2c4.com, linux-arm-kernel@lists.infradead.org,
niuben003@gmail.com, linux-kernel@vger.kernel.org
Subject: Re: Withdraw [PATCH] tracing: Enable kprobe tracing for Arm64 asm functions
Date: Sat, 13 Dec 2025 01:33:06 +0900 [thread overview]
Message-ID: <aTxDwiqIhGG1NkUl@J2N7QTR9R3> (raw)
In-Reply-To: <aTnVEdv2j5ur48aF@meta.com>
On Wed, Dec 10, 2025 at 12:16:17PM -0800, Ben Niu wrote:
> On Mon, Nov 17, 2025 at 10:34:22AM +0000, Mark Rutland wrote:
> > On Thu, Oct 30, 2025 at 11:07:51AM -0700, Ben Niu wrote:
> > > On Thu, Oct 30, 2025 at 12:35:25PM +0000, Mark Rutland wrote:
> > > > Is there something specific you want to trace, but cannot currently
> > > > trace (on arm64)?
> > >
> > > For some reason, we only saw Arm64 Linux asm functions __arch_copy_to_user and
> > > __arch_copy_from_user being hot in our workloads, not those counterpart asm
> > > functions on x86, so we are trying to understand and improve performance of
> > > those Arm64 asm functions.
> >
> > Are you sure that's not an artifact of those being out-of-line on arm64,
> > but inline on x86? On x86, the out-of-line forms are only used when the
> > CPU doesn't have FSRM, and when the CPU *does* have FSRM, the logic gets
> > inlined. See raw_copy_from_user(), raw_copy_to_user(), and
> > copy_user_generic() in arch/x86/include/asm/uaccess_64.h.
>
> On x86, INLINE_COPY_TO_USER is not defined in the latest linux kernel and our
> internal branch, so _copy_to_user is always defined as an extern function
> (no-inline), which ends up inlining copy_user_generic. copy_user_generic
> executes FSRM rep movs if CPU supports it (our case), otherwise, it calls
> rep_movs_alternative, which issues plain movs to copy memory.
> > Have you checked that inlining is not skewing your results, and
> > artificially making those look hotter on am64 by virtue of centralizing
> > samples to the same IP/PC range?
>
> As mentioned above, _copy_to_user is not inlined on x86.
Thanks for confirming!
> > Can you share any information on those workloads? e.g. which callchains
> > were hot?
>
> Please reach out to James Greenhalgh and Chris Goodyer at Arm for more details
> about those workloads, which I can't share in a public channel.
If you can't share this info publicly, that's fair enough.
Please note that upstream it's hard to justify changing things based on
confidential information. Sharing information with me in private isn't
all that helpful as it would not be clear what I could subsequently
share in public.
The reason that I've asked is that it would be very interesting to know
whether there's a specific subsystem, driver, or code path that's
hitting this hard, because a better option might be "don't do that", and
attempt to avoid the uaccesses entirely (e.g. accessing a kernel alias
with get_user_pages()).
If there's anything that you can share about that, it'd be very helpful.
Mark.
prev parent reply other threads:[~2025-12-12 16:33 UTC|newest]
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-10-27 18:17 [PATCH] tracing: Enable kprobe tracing for Arm64 asm functions Ben Niu
2025-10-29 14:33 ` Mark Rutland
2025-10-29 23:53 ` Ben Niu
2025-10-30 12:35 ` Mark Rutland
2025-10-30 18:07 ` Ben Niu
2025-11-17 10:34 ` Mark Rutland
2025-12-10 20:16 ` Withdraw " Ben Niu
2025-12-12 16:33 ` Mark Rutland [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=aTxDwiqIhGG1NkUl@J2N7QTR9R3 \
--to=mark.rutland@arm.com \
--cc=BenNiu@meta.com \
--cc=Jason@zx2c4.com \
--cc=catalin.marinas@arm.com \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=niuben003@gmail.com \
--cc=tytso@mit.edu \
--cc=will@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox