BPF List
 help / color / mirror / Atom feed
From: Ihor Solodrai <ihor.solodrai@linux.dev>
To: Leon Hwang <leon.hwang@linux.dev>,
	Catalin Marinas <catalin.marinas@arm.com>,
	Will Deacon <will@kernel.org>,
	Steven Rostedt <rostedt@goodmis.org>,
	Masami Hiramatsu <mhiramat@kernel.org>,
	Mark Rutland <mark.rutland@arm.com>,
	Alexei Starovoitov <ast@kernel.org>,
	Daniel Borkmann <daniel@iogearbox.net>,
	Andrii Nakryiko <andrii@kernel.org>,
	Eduard Zingerman <eddyz87@gmail.com>,
	Kumar Kartikeya Dwivedi <memxor@gmail.com>,
	Martin KaFai Lau <martin.lau@linux.dev>,
	Song Liu <song@kernel.org>,
	Yonghong Song <yonghong.song@linux.dev>,
	Jiri Olsa <jolsa@kernel.org>,
	Emil Tsalapatis <emil@etsalapatis.com>,
	Shuah Khan <shuah@kernel.org>,
	"Jose Fernandez (Anthropic)" <jose.fernandez@linux.dev>
Cc: bpf@vger.kernel.org, linux-arm-kernel@lists.infradead.org,
	linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org,
	linux-kselftest@vger.kernel.org
Subject: Re: [PATCH 1/2] arm64: ftrace: enable single ftrace_ops for direct calls
Date: Thu, 30 Jul 2026 16:03:33 -0700	[thread overview]
Message-ID: <77edc25d-a521-4f65-84ed-dbb016593902@linux.dev> (raw)
In-Reply-To: <20260727142844.21212-2-leon.hwang@linux.dev>

On 7/27/26 7:28 AM, Leon Hwang wrote:
> The BPF tracing multi link updates several direct-call sites through one
> ftrace_ops. Its implementation is therefore gated by
> HAVE_SINGLE_FTRACE_DIRECT_OPS in addition to
> DYNAMIC_FTRACE_WITH_DIRECT_CALLS.
> 
> Select HAVE_SINGLE_FTRACE_DIRECT_OPS whenever arm64 enables dynamic ftrace
> direct calls. This enables BPF tracing multi links on arm64. Also
> generalize the unreachable-trampoline comment because the single-ops path
> does not use ops->direct_call.

Hi Leon,

I don't think this change can land as is yet. The series doesn't even
apply cleanly to bpf-next, but that's minor.

More importantly, it depends on Jose's series [1], which is not in the
mainline yet. And there Mark has raised performance concerns [2] and
the discussion still seems to be open.

[1] https://lore.kernel.org/all/20260609-arm64-ftrace-direct-calls-v1-0-4a46f266697f@linux.dev/
[2] https://lore.kernel.org/all/amjnf5gz0xP5PTSB@J2N7QTR9R3/

> 
> Assisted-by: Codex:gpt-5.6-sol
> Signed-off-by: Leon Hwang <leon.hwang@linux.dev>
> ---
>  arch/arm64/Kconfig         | 2 ++
>  arch/arm64/kernel/ftrace.c | 3 +--
>  2 files changed, 3 insertions(+), 2 deletions(-)
> 
> diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
> index 0de419ed780f..c98dca76859b 100644
> --- a/arch/arm64/Kconfig
> +++ b/arch/arm64/Kconfig
> @@ -188,6 +188,8 @@ config ARM64
>  		    CLANG_SUPPORTS_DYNAMIC_FTRACE_WITH_ARGS)
>  	select HAVE_DYNAMIC_FTRACE_WITH_DIRECT_CALLS \
>  		if DYNAMIC_FTRACE_WITH_ARGS
> +	select HAVE_SINGLE_FTRACE_DIRECT_OPS \
> +		if DYNAMIC_FTRACE_WITH_DIRECT_CALLS\

The select is only conditional on DYNAMIC_FTRACE_WITH_DIRECT_CALLS, so
it can be set along with HAVE_DYNAMIC_FTRACE_WITH_CALL_OPS. And AFAIU
this would make the fast path effectively dead: every BPF direct call
routed through ftrace_caller now goes through the slow path.

As Mark noted in the other thread, on arm64 trampolines come from
EXECMEM_BPF, so they always land out of BL range (chance of landing in
range is 256M/terabytes).

I vibe-slop-coded a benchmark and ran it on a Neoverse V2 machine, and
toggling your config change seems to be causing a 1.3x regression in
the tracing overhead:

  do-nothing fentry (r0=0; exit) on __arm64_sys_getpid, 20 M calls, min-of-N, several boots. Results:

              untraced (base)   traced        overhead
  Kernel A    ~128 ns/call      ~145.5 ns     ~17.6 ns   (fast path: br x17)
  Kernel B    ~127 ns/call      ~150.4 ns     ~23.4 ns   (slow path: save regs + call_direct_funcs + hash)

This confirms Jiri's suspicion.

However my understanding is the regression should mostly disappear in
case some version of in-range trampoline allocation on arm64 lands.

So, I think the landing sequence should be something like follows:
  * in-range BPF-trampoline allocation that Jose proposed [3]
  * then HAVE_SINGLE_FTRACE_DIRECT_OPS selection

After all of that reaches mainline, then a selftest patch can go
through the bpf-next.

[3] https://lore.kernel.org/all/amn7cBQ-cqs2XlKV@linux.dev/

>  	select HAVE_DYNAMIC_FTRACE_WITH_CALL_OPS \
>  		if (DYNAMIC_FTRACE_WITH_ARGS && !CFI && \
>  		    (CC_IS_CLANG || !CC_OPTIMIZE_FOR_SIZE))
> diff --git a/arch/arm64/kernel/ftrace.c b/arch/arm64/kernel/ftrace.c
> index e1a3c0b3a051..56ba72a87dfa 100644
> --- a/arch/arm64/kernel/ftrace.c
> +++ b/arch/arm64/kernel/ftrace.c
> @@ -301,8 +301,7 @@ static bool ftrace_find_callable_addr(struct dyn_ftrace *rec,
>  
>  	/*
>  	 * If a custom trampoline is unreachable, rely on the ftrace_caller
> -	 * trampoline which knows how to indirectly reach that trampoline
> -	 * through ops->direct_call.
> +	 * trampoline which knows how to indirectly reach that trampoline.
>  	 */
>  	if (*addr != FTRACE_ADDR && !reachable_by_bl(*addr, pc))
>  		*addr = FTRACE_ADDR;


  parent reply	other threads:[~2026-07-30 23:04 UTC|newest]

Thread overview: 13+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-07-27 14:28 [PATCH 0/2] arm64: ftrace: enable single ftrace_ops for direct calls Leon Hwang
2026-07-27 14:28 ` [PATCH 1/2] " Leon Hwang
2026-07-27 14:52   ` sashiko-bot
2026-07-28  2:19     ` Leon Hwang
2026-07-28 15:46       ` Steven Rostedt
2026-07-29  6:27         ` Leon Hwang
2026-07-29 11:56   ` Jiri Olsa
2026-07-30  3:05     ` Leon Hwang
2026-07-30 10:58       ` Jiri Olsa
2026-07-31  2:30         ` Leon Hwang
2026-07-30 23:03   ` Ihor Solodrai [this message]
2026-07-31  2:21     ` Leon Hwang
2026-07-27 14:28 ` [PATCH 2/2] selftests/bpf: Enable tracing_multi tests on arm64 Leon Hwang

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=77edc25d-a521-4f65-84ed-dbb016593902@linux.dev \
    --to=ihor.solodrai@linux.dev \
    --cc=andrii@kernel.org \
    --cc=ast@kernel.org \
    --cc=bpf@vger.kernel.org \
    --cc=catalin.marinas@arm.com \
    --cc=daniel@iogearbox.net \
    --cc=eddyz87@gmail.com \
    --cc=emil@etsalapatis.com \
    --cc=jolsa@kernel.org \
    --cc=jose.fernandez@linux.dev \
    --cc=leon.hwang@linux.dev \
    --cc=linux-arm-kernel@lists.infradead.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-kselftest@vger.kernel.org \
    --cc=linux-trace-kernel@vger.kernel.org \
    --cc=mark.rutland@arm.com \
    --cc=martin.lau@linux.dev \
    --cc=memxor@gmail.com \
    --cc=mhiramat@kernel.org \
    --cc=rostedt@goodmis.org \
    --cc=shuah@kernel.org \
    --cc=song@kernel.org \
    --cc=will@kernel.org \
    --cc=yonghong.song@linux.dev \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox