[PATCH bpf-next 0/3] Fix duplicate caller frame in arm64 stack traces

[PATCH bpf-next 0/3] Fix duplicate caller frame in arm64 stack traces

[Andrii Nakryiko]

On arm64 the caller is reported twice when the kernel builds a callchain for
the current task from regs filled by perf_fetch_caller_regs(), instead of regs
saved by a real exception. There was a similar problem on x86-64, but it was
since fixed. Now it's time for arm64.

Patch 1 has the fix, patches 2-3 add BPF selftests for this.

I'm targeting bpf-next because of BPF selftests and to have BPF CI test
everything. Patch 1 doesn't have to go through bpf-next, but it's simple enough,
so if arm64 folks are ok with that, please ack and then we'll land everything
in one go into bpf-next.

Andrii Nakryiko (3):
  arm64: perf: Don't report the caller twice in caller-regs callchains
  selftests/bpf: Add stacktrace_ips test for tracepoint
  selftests/bpf: Run stacktrace_ips raw_tp, tp and fentry tests on arm64

 arch/arm64/include/asm/perf_event.h           |  2 +-
 .../selftests/bpf/prog_tests/stacktrace_ips.c | 67 ++++++++++++++++++-
 .../selftests/bpf/progs/stacktrace_ips.c      | 18 ++++-
 3 files changed, 81 insertions(+), 6 deletions(-)

-- 
2.53.1-Meta

[PATCH bpf-next 1/3] arm64: perf: Don't report the caller twice in caller-regs callchains

[Andrii Nakryiko]

perf_arch_fetch_caller_regs() sets regs->pc to the return address of
the function it is used in (CALLER_ADDR0), but points regs->regs[29]
at that function's own frame record, whose saved LR is that same
return address. perf_callchain_kernel() reports regs->pc and then
unwinds from the frame record at x29, so the caller shows up twice at
the top of every kernel callchain built from perf_fetch_caller_regs():
tracepoint events, software events emitted by __perf_sw_event_sched()
(context-switches, cgroup-switches, cpu-migrations), and
bpf_get_stack() and bpf_get_stackid() in tracepoint, raw tracepoint,
tp_btf and fentry/fexit programs. E.g., for a tracepoint program
attached to sched_process_fork:

  __traceiter_sched_process_fork+0x44
  __traceiter_sched_process_fork+0x44
  kernel_clone+0x250
  ...

Use __builtin_frame_address(1), the caller's frame record, the same
way kunwind_init_from_caller() does, so that pc and x29 describe the
same frame.

x86 had the same duplicate entry, fixed by commit 6d08340d1e35
("Revert "perf/x86: Always store regs->ip in perf_callchain_kernel()"").

Fixes: 5b09a094f2fb ("arm64: perf: Fix callchain parse error with kernel tracepoint events")
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
---
 arch/arm64/include/asm/perf_event.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/arm64/include/asm/perf_event.h b/arch/arm64/include/asm/perf_event.h
index ee45b4e77347..13efb46f0fe8 100644
--- a/arch/arm64/include/asm/perf_event.h
+++ b/arch/arm64/include/asm/perf_event.h
@@ -15,7 +15,7 @@
 
 #define perf_arch_fetch_caller_regs(regs, __ip) { \
 	(regs)->pc = (__ip);    \
-	(regs)->regs[29] = (unsigned long) __builtin_frame_address(0); \
+	(regs)->regs[29] = (unsigned long) __builtin_frame_address(1); \
 	(regs)->sp = current_stack_pointer; \
 	(regs)->pstate = PSR_MODE_EL1h;	\
 }
-- 
2.53.1-Meta

[PATCH bpf-next 2/3] selftests/bpf: Add stacktrace_ips test for tracepoint

[Andrii Nakryiko]

Add a tp subtest that checks the stack captured by bpf_get_stack()
from a tracepoint program, recorded only for the test's own process.
Depending on static calls and inlining, the stack may or may not have
__traceiter_bpf_testmod_test_read in front of bpf_testmod_test_read,
so accept both. Also check that the entry after it differs: kernels
without commit 6d08340d1e35 ("Revert "perf/x86: Always store regs->ip
in perf_callchain_kernel()"") report the tracepoint's caller twice.

Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
---
 .../selftests/bpf/prog_tests/stacktrace_ips.c | 54 +++++++++++++++++++
 .../selftests/bpf/progs/stacktrace_ips.c      | 13 +++++
 2 files changed, 67 insertions(+)

diff --git a/tools/testing/selftests/bpf/prog_tests/stacktrace_ips.c b/tools/testing/selftests/bpf/prog_tests/stacktrace_ips.c
index da42b00e3d1f..a03d4c10c469 100644
--- a/tools/testing/selftests/bpf/prog_tests/stacktrace_ips.c
+++ b/tools/testing/selftests/bpf/prog_tests/stacktrace_ips.c
@@ -137,6 +137,58 @@ static void test_stacktrace_ips_raw_tp(void)
 	stacktrace_ips__destroy(skel);
 }
 
+static void test_stacktrace_ips_tp(void)
+{
+	__u64 traceiter_addr, read_addr, *ips;
+	struct stacktrace_ips *skel;
+	struct ksyms *ksyms = NULL;
+	struct ksym *ksym;
+	int i;
+
+	skel = stacktrace_ips__open_and_load();
+	if (!ASSERT_OK_PTR(skel, "stacktrace_ips__open_and_load"))
+		return;
+
+	if (!skel->kconfig->CONFIG_UNWINDER_ORC) {
+		test__skip();
+		goto cleanup;
+	}
+
+	skel->bss->my_pid = getpid();
+
+	skel->links.tp_test = bpf_program__attach(skel->progs.tp_test);
+	if (!ASSERT_OK_PTR(skel->links.tp_test, "bpf_program__attach"))
+		goto cleanup;
+
+	trigger_module_test_read(1);
+
+	ksyms = load_kallsyms_local();
+	if (!ASSERT_OK_PTR(ksyms, "load_kallsyms_local"))
+		goto cleanup;
+	traceiter_addr = ksym_get_addr_local(ksyms, "__traceiter_bpf_testmod_test_read");
+	read_addr = ksym_get_addr_local(ksyms, "bpf_testmod_test_read");
+
+	/*
+	 * Depending on static calls and inlining, __traceiter_bpf_testmod_test_read
+	 * may or may not be in front of bpf_testmod_test_read.
+	 */
+	ips = skel->bss->tp_ips;
+	ksym = ksym_search_local(ksyms, ips[0]);
+	if (!ASSERT_OK_PTR(ksym, "ksym_search_local"))
+		goto cleanup;
+	i = ksym->addr == traceiter_addr ? 1 : 0;
+
+	ksym = ksym_search_local(ksyms, ips[i]);
+	if (!ASSERT_OK_PTR(ksym, "ksym_search_local"))
+		goto cleanup;
+	ASSERT_EQ(ksym->addr, read_addr, "stack_cmp");
+	ASSERT_NEQ(ips[i], ips[i + 1], "stack_dup");
+
+cleanup:
+	free_kallsyms_local(ksyms);
+	stacktrace_ips__destroy(skel);
+}
+
 static void test_stacktrace_ips_kprobe(bool retprobe)
 {
 	LIBBPF_OPTS(bpf_kprobe_opts, opts,
@@ -238,6 +290,8 @@ static void __test_stacktrace_ips(void)
 		test_stacktrace_ips_kprobe_multi(true);
 	if (test__start_subtest("raw_tp"))
 		test_stacktrace_ips_raw_tp();
+	if (test__start_subtest("tp"))
+		test_stacktrace_ips_tp();
 	if (test__start_subtest("kprobe"))
 		test_stacktrace_ips_kprobe(false);
 	if (test__start_subtest("kretprobe"))
diff --git a/tools/testing/selftests/bpf/progs/stacktrace_ips.c b/tools/testing/selftests/bpf/progs/stacktrace_ips.c
index 6830f2978613..df10ef3cb14d 100644
--- a/tools/testing/selftests/bpf/progs/stacktrace_ips.c
+++ b/tools/testing/selftests/bpf/progs/stacktrace_ips.c
@@ -53,6 +53,19 @@ int rawtp_test(void *ctx)
 	return 0;
 }
 
+__u64 tp_ips[128];
+int my_pid;
+
+SEC("tp/bpf_testmod/bpf_testmod_test_read")
+int tp_test(void *ctx)
+{
+	if (bpf_get_current_pid_tgid() >> 32 != my_pid)
+		return 0;
+
+	bpf_get_stack(ctx, tp_ips, sizeof(tp_ips), 0);
+	return 0;
+}
+
 SEC("fentry/bpf_testmod_stacktrace_test")
 int fentry_test(struct pt_regs *ctx)
 {
-- 
2.53.1-Meta

[PATCH bpf-next 3/3] selftests/bpf: Run stacktrace_ips raw_tp, tp and fentry tests on arm64

[Andrii Nakryiko]

The raw_tp, tp and fentry subtests check stacks unwound from
perf_fetch_caller_regs() regs, which arm64 reported with the caller
twice until perf_arch_fetch_caller_regs() was fixed. Run them on arm64
as well, where kernel stacks are always unwound through frame records.

Leave the kprobe and fexit subtests x86-only: on arm64, kprobes at
function entry miss the probed function's caller, and fexit stacks
have an extra entry for the traced function (kretprobe subtests pass,
but don't use these regs).

Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
---
 .../selftests/bpf/prog_tests/stacktrace_ips.c     | 15 +++++++++++----
 .../testing/selftests/bpf/progs/stacktrace_ips.c  |  5 +++--
 2 files changed, 14 insertions(+), 6 deletions(-)

diff --git a/tools/testing/selftests/bpf/prog_tests/stacktrace_ips.c b/tools/testing/selftests/bpf/prog_tests/stacktrace_ips.c
index a03d4c10c469..aee46543f390 100644
--- a/tools/testing/selftests/bpf/prog_tests/stacktrace_ips.c
+++ b/tools/testing/selftests/bpf/prog_tests/stacktrace_ips.c
@@ -2,7 +2,7 @@
 #include <test_progs.h>
 #include "stacktrace_ips.skel.h"
 
-#ifdef __x86_64__
+#if defined(__x86_64__) || defined(__aarch64__)
 static int check_stacktrace_ips(int fd, __u32 key, int cnt, ...)
 {
 	__u64 ips[PERF_MAX_STACK_DEPTH];
@@ -47,6 +47,12 @@ static int check_stacktrace_ips(int fd, __u32 key, int cnt, ...)
 	return err;
 }
 
+/* arm64 always unwinds kernel stacks through frame records */
+static bool unwinder_ok(struct stacktrace_ips *skel)
+{
+	return skel->kconfig->CONFIG_UNWINDER_ORC || skel->kconfig->CONFIG_ARM64;
+}
+
 static void test_stacktrace_ips_kprobe_multi(bool retprobe)
 {
 	LIBBPF_OPTS(bpf_kprobe_multi_opts, opts,
@@ -106,7 +112,7 @@ static void test_stacktrace_ips_raw_tp(void)
 	if (!ASSERT_OK_PTR(skel, "stacktrace_ips__open_and_load"))
 		return;
 
-	if (!skel->kconfig->CONFIG_UNWINDER_ORC) {
+	if (!unwinder_ok(skel)) {
 		test__skip();
 		goto cleanup;
 	}
@@ -149,7 +155,7 @@ static void test_stacktrace_ips_tp(void)
 	if (!ASSERT_OK_PTR(skel, "stacktrace_ips__open_and_load"))
 		return;
 
-	if (!skel->kconfig->CONFIG_UNWINDER_ORC) {
+	if (!unwinder_ok(skel)) {
 		test__skip();
 		goto cleanup;
 	}
@@ -244,7 +250,8 @@ static void test_stacktrace_ips_trampoline(bool retprobe)
 	if (!ASSERT_OK_PTR(skel, "stacktrace_ips__open_and_load"))
 		return;
 
-	if (!skel->kconfig->CONFIG_UNWINDER_ORC) {
+	/* on arm64, fexit stacks also have a frame for the traced function */
+	if ((retprobe && !skel->kconfig->CONFIG_UNWINDER_ORC) || !unwinder_ok(skel)) {
 		test__skip();
 		goto cleanup;
 	}
diff --git a/tools/testing/selftests/bpf/progs/stacktrace_ips.c b/tools/testing/selftests/bpf/progs/stacktrace_ips.c
index df10ef3cb14d..11dfebd4ddb2 100644
--- a/tools/testing/selftests/bpf/progs/stacktrace_ips.c
+++ b/tools/testing/selftests/bpf/progs/stacktrace_ips.c
@@ -19,14 +19,15 @@ struct {
 } stackmap SEC(".maps");
 
 extern bool CONFIG_UNWINDER_ORC __kconfig __weak;
+extern bool CONFIG_ARM64 __kconfig __weak;
 
 /*
- * This function is here to have CONFIG_UNWINDER_ORC
+ * This function is here to have CONFIG_UNWINDER_ORC and CONFIG_ARM64
  * used and added to object BTF.
  */
 int unused(void)
 {
-	return CONFIG_UNWINDER_ORC ? 0 : 1;
+	return CONFIG_UNWINDER_ORC || CONFIG_ARM64 ? 0 : 1;
 }
 
 __u32 stack_key;
-- 
2.53.1-Meta

Re: [PATCH bpf-next 2/3] selftests/bpf: Add stacktrace_ips test for tracepoint

[sashiko-bot]

> Add a tp subtest that checks the stack captured by bpf_get_stack()
> from a tracepoint program, recorded only for the test's own process.
> Depending on static calls and inlining, the stack may or may not have
> __traceiter_bpf_testmod_test_read in front of bpf_testmod_test_read,
> so accept both. Also check that the entry after it differs: kernels
> without commit 6d08340d1e35 ("Revert "perf/x86: Always store regs->ip
> in perf_callchain_kernel()"") report the tracepoint's caller twice.
> 
> Signed-off-by: Andrii Nakryiko <andrii@kernel.org>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261114001846.873499-1-andrii@kernel.org?part=2

Re: [PATCH bpf-next 1/3] arm64: perf: Don't report the caller twice in caller-regs callchains

[sashiko-bot]

> perf_arch_fetch_caller_regs() sets regs->pc to the return address of
> the function it is used in (CALLER_ADDR0), but points regs->regs[29]
> at that function's own frame record, whose saved LR is that same
> return address. perf_callchain_kernel() reports regs->pc and then
> unwinds from the frame record at x29, so the caller shows up twice at
> the top of every kernel callchain built from perf_fetch_caller_regs():
> tracepoint events, software events emitted by __perf_sw_event_sched()
> (context-switches, cgroup-switches, cpu-migrations), and
> bpf_get_stack() and bpf_get_stackid() in tracepoint, raw tracepoint,
> tp_btf and fentry/fexit programs. E.g., for a tracepoint program
> attached to sched_process_fork:
> 
>   __traceiter_sched_process_fork+0x44
>   __traceiter_sched_process_fork+0x44
>   kernel_clone+0x250
>   ...
> 
> Use __builtin_frame_address(1), the caller's frame record, the same
> way kunwind_init_from_caller() does, so that pc and x29 describe the
> same frame.
> 
> x86 had the same duplicate entry, fixed by commit 6d08340d1e35
> ("Revert "perf/x86: Always store regs->ip in perf_callchain_kernel()"").
> 
> Fixes: 5b09a094f2fb ("arm64: perf: Fix callchain parse error with kernel tracepoint events")
> Signed-off-by: Andrii Nakryiko <andrii@kernel.org>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261114001846.873499-1-andrii@kernel.org?part=1

Re: [PATCH bpf-next 3/3] selftests/bpf: Run stacktrace_ips raw_tp, tp and fentry tests on arm64

[sashiko-bot]

> The raw_tp, tp and fentry subtests check stacks unwound from
> perf_fetch_caller_regs() regs, which arm64 reported with the caller
> twice until perf_arch_fetch_caller_regs() was fixed. Run them on arm64
> as well, where kernel stacks are always unwound through frame records.
> 
> Leave the kprobe and fexit subtests x86-only: on arm64, kprobes at
> function entry miss the probed function's caller, and fexit stacks
> have an extra entry for the traced function (kretprobe subtests pass,
> but don't use these regs).
> 
> Signed-off-by: Andrii Nakryiko <andrii@kernel.org>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261114001846.873499-1-andrii@kernel.org?part=3

Re: [PATCH bpf-next 1/3] arm64: perf: Don't report the caller twice in caller-regs callchains

[Alexei Starovoitov]

On Fri, Nov 13, 2026 at 04:18 PM Andrii Nakryiko <andrii@kernel.org> wrote:
> perf_arch_fetch_caller_regs() sets regs->pc to the return address of
> the function it is used in (CALLER_ADDR0), but points regs->regs[29]
> at that function's own frame record, whose saved LR is that same
> return address. perf_callchain_kernel() reports regs->pc and then
> unwinds from the frame record at x29, so the caller shows up twice at
> the top of every kernel callchain built from perf_fetch_caller_regs():
> tracepoint events, software events emitted by __perf_sw_event_sched()
> (context-switches, cgroup-switches, cpu-migrations), and
> bpf_get_stack() and bpf_get_stackid() in tracepoint, raw tracepoint,
> tp_btf and fentry/fexit programs. E.g., for a tracepoint program
> attached to sched_process_fork:
>
>   __traceiter_sched_process_fork+0x44
>   __traceiter_sched_process_fork+0x44
>   kernel_clone+0x250
>   ...
>
> Use __builtin_frame_address(1), the caller's frame record, the same
> way kunwind_init_from_caller() does, so that pc and x29 describe the
> same frame.
>
> x86 had the same duplicate entry, fixed by commit 6d08340d1e35
> ("Revert "perf/x86: Always store regs->ip in perf_callchain_kernel()"").

riscv has the same duplicate it seems.
Its perf_arch_fetch_caller_regs() stores __builtin_frame_address(0)
in s0. walk_stackframe() reports epc, then frame->ra of that frame.
Can be a follow up by riscv folks.

> @@ -15,7 +15,7 @@
>
>  #define perf_arch_fetch_caller_regs(regs, __ip) { \
>  	(regs)->pc = (__ip);    \
> -	(regs)->regs[29] = (unsigned long) __builtin_frame_address(0); \
> +	(regs)->regs[29] = (unsigned long) __builtin_frame_address(1); \
>  	(regs)->sp = current_stack_pointer; \
>  	(regs)->pstate = PSR_MODE_EL1h;	\
>  }

Will, Mark,
this looks correct to me.
Mind if I take it via bpf-next together with the selftests?