[PATCH 6.12] x86/fgraph: Fix return_to_handler regs.rsp value

[PATCH 6.12] x86/fgraph: Fix return_to_handler regs.rsp value

[Gyokhan Kochmarla]

From: Jiri Olsa <jolsa@kernel.org>

commit 8bc11700e0d23d4fdb7d8d5a73b2e95de427cabc upstream.

The previous change (Fixes commit) messed up the rsp register value,
which is wrong because it's already adjusted with FRAME_SIZE, we need
the original rsp value.

This change does not affect fprobe current kernel unwind, the !perf_hw_regs
path perf_callchain_kernel:

        if (perf_hw_regs(regs)) {
                if (perf_callchain_store(entry, regs->ip))
                        return;
                unwind_start(&state, current, regs, NULL);
        } else {
                unwind_start(&state, current, NULL, (void *)regs->sp);
        }

which uses pt_regs.sp as first_frame boundary (FRAME_SIZE shift makes
no difference, unwind stil stops at the right frame).

This change fixes the other path when we want to unwind directly from
pt_regs sp/fp/ip state, which is coming in following change.

Fixes: 20a0bc10272f ("x86/fgraph,bpf: Fix stack ORC unwind from kprobe_multi return probe")
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Reviewed-by: Steven Rostedt (Google) <rostedt@goodmis.org>
Link: https://lore.kernel.org/bpf/20260126211837.472802-2-jolsa@kernel.org
Signed-off-by: Gyokhan Kochmarla <gyokhan@amazon.de>
---
 arch/x86/kernel/ftrace_64.S | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/arch/x86/kernel/ftrace_64.S b/arch/x86/kernel/ftrace_64.S
index 8a3cff618692..143fc62bf6f8 100644
--- a/arch/x86/kernel/ftrace_64.S
+++ b/arch/x86/kernel/ftrace_64.S
@@ -349,6 +349,9 @@ SYM_CODE_START(return_to_handler)
 	UNWIND_HINT_UNDEFINED
 	ANNOTATE_NOENDBR
 
+	/* Store original rsp for pt_regs.sp value. */
+	movq %rsp, %rdi
+
 	/* Restore return_to_handler value that got eaten by previous ret instruction. */
 	subq $8, %rsp
 	UNWIND_HINT_FUNC
@@ -359,7 +362,7 @@ SYM_CODE_START(return_to_handler)
 	movq %rax, RAX(%rsp)
 	movq %rdx, RDX(%rsp)
 	movq %rbp, RBP(%rsp)
-	movq %rsp, RSP(%rsp)
+	movq %rdi, RSP(%rsp)
 	movq %rsp, %rdi
 
 	call ftrace_return_to_handler
-- 
2.47.3




Amazon Web Services Development Center Germany GmbH
Tamara-Danz-Str. 13
10243 Berlin
Geschaeftsfuehrung: Christof Hellmis, Andreas Stieger
Eingetragen am Amtsgericht Charlottenburg unter HRB 257764 B
Sitz: Berlin
Ust-ID: DE 365 538 597

Re: [PATCH 6.12] x86/fgraph: Fix return_to_handler regs.rsp value

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:

Pre-existing issues:
- [High] The `return_to_handler` assembly routine creates a `pt_regs` (or `ftrace_regs`) structure on the kernel stack but only initializes four registers (`rax`, `rdx`, `rbp`, `rsp`), leaving the rest of the structure uninitialized. This partially initialized structure is passed to BPF kretprobe programs, leading to an information leak of kernel stack memory.
--

commit ff6f1024b92860971d84c833c6dac53501145715
Author: Jiri Olsa <jolsa@kernel.org>

x86/fgraph: Fix return_to_handler regs.rsp value

This commit fixes the unwinding rsp register value in return_to_handler. It 
saves the original rsp before the FRAME_SIZE adjustment, replacing the 
previous incorrect value. This ensures fprobe and direct unwinders see the 
right stack pointer.

Link: https://lore.kernel.org/bpf/20260126211837.472802-2-jolsa@kernel.org

> diff --git a/arch/x86/kernel/ftrace_64.S b/arch/x86/kernel/ftrace_64.S
> --- a/arch/x86/kernel/ftrace_64.S
> +++ b/arch/x86/kernel/ftrace_64.S
> @@ -364,6 +364,9 @@ SYM_CODE_START(return_to_handler)
>  	UNWIND_HINT_UNDEFINED
>  	ANNOTATE_NOENDBR
>  
> +	/* Store original rsp for pt_regs.sp value. */
> +	movq %rsp, %rdi
> +
>  	/* Restore return_to_handler value that got eaten by previous ret instruction. */
>  	subq $8, %rsp
>  	UNWIND_HINT_FUNC
> @@ -374,7 +377,7 @@ SYM_CODE_START(return_to_handler)
>  	movq %rax, RAX(%rsp)
>  	movq %rdx, RDX(%rsp)
>  	movq %rbp, RBP(%rsp)
> -	movq %rsp, RSP(%rsp)
> +	movq %rdi, RSP(%rsp)

[Severity: High]
This is a pre-existing issue, but could this code leak kernel stack memory
to BPF programs?

This assembly creates a pt_regs structure on the stack by allocating
FRAME_SIZE bytes, but only explicitly saves rax, rdx, rbp, and rsp. Later in
ftrace_return_to_handler and arch_ftrace_partial_regs, only a few additional
fields are populated.

The remaining fields (like rdi, rsi, rbx, and r12-r15) remain uninitialized.

Because this partially initialized pt_regs structure is passed to BPF
kprobe_multi return programs via kprobe_multi_link_exit_handler(), and the
BPF verifier allows reading any field of the pt_regs context, does this
allow BPF programs to extract residual kernel stack data?

>  	movq %rsp, %rdi
>  
>  	call ftrace_return_to_handler

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260526192324.79459-1-gyokhan@amazon.de?part=1

Re: [PATCH 6.12] x86/fgraph: Fix return_to_handler regs.rsp value

[Jiri Olsa]

On Tue, May 26, 2026 at 07:50:07PM +0000, sashiko-bot@kernel.org wrote:
> Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
> 
> Pre-existing issues:
> - [High] The `return_to_handler` assembly routine creates a `pt_regs` (or `ftrace_regs`) structure on the kernel stack but only initializes four registers (`rax`, `rdx`, `rbp`, `rsp`), leaving the rest of the structure uninitialized. This partially initialized structure is passed to BPF kretprobe programs, leading to an information leak of kernel stack memory.
> --
> 
> commit ff6f1024b92860971d84c833c6dac53501145715
> Author: Jiri Olsa <jolsa@kernel.org>
> 
> x86/fgraph: Fix return_to_handler regs.rsp value
> 
> This commit fixes the unwinding rsp register value in return_to_handler. It 
> saves the original rsp before the FRAME_SIZE adjustment, replacing the 
> previous incorrect value. This ensures fprobe and direct unwinders see the 
> right stack pointer.
> 
> Link: https://lore.kernel.org/bpf/20260126211837.472802-2-jolsa@kernel.org
> 
> > diff --git a/arch/x86/kernel/ftrace_64.S b/arch/x86/kernel/ftrace_64.S
> > --- a/arch/x86/kernel/ftrace_64.S
> > +++ b/arch/x86/kernel/ftrace_64.S
> > @@ -364,6 +364,9 @@ SYM_CODE_START(return_to_handler)
> >  	UNWIND_HINT_UNDEFINED
> >  	ANNOTATE_NOENDBR
> >  
> > +	/* Store original rsp for pt_regs.sp value. */
> > +	movq %rsp, %rdi
> > +
> >  	/* Restore return_to_handler value that got eaten by previous ret instruction. */
> >  	subq $8, %rsp
> >  	UNWIND_HINT_FUNC
> > @@ -374,7 +377,7 @@ SYM_CODE_START(return_to_handler)
> >  	movq %rax, RAX(%rsp)
> >  	movq %rdx, RDX(%rsp)
> >  	movq %rbp, RBP(%rsp)
> > -	movq %rsp, RSP(%rsp)
> > +	movq %rdi, RSP(%rsp)
> 
> [Severity: High]
> This is a pre-existing issue, but could this code leak kernel stack memory
> to BPF programs?
> 
> This assembly creates a pt_regs structure on the stack by allocating
> FRAME_SIZE bytes, but only explicitly saves rax, rdx, rbp, and rsp. Later in
> ftrace_return_to_handler and arch_ftrace_partial_regs, only a few additional
> fields are populated.
> 
> The remaining fields (like rdi, rsi, rbx, and r12-r15) remain uninitialized.
> 
> Because this partially initialized pt_regs structure is passed to BPF
> kprobe_multi return programs via kprobe_multi_link_exit_handler(), and the
> BPF verifier allows reading any field of the pt_regs context, does this
> allow BPF programs to extract residual kernel stack data?

I think because kprobe/bpf is root only, it's not really an issue,
cc-ing Steven and Masami

jirka

Re: [PATCH 6.12] x86/fgraph: Fix return_to_handler regs.rsp value

[Steven Rostedt]

On Wed, 27 May 2026 09:36:14 +0200
Jiri Olsa <olsajiri@gmail.com> wrote:

> > Because this partially initialized pt_regs structure is passed to BPF
> > kprobe_multi return programs via kprobe_multi_link_exit_handler(), and the
> > BPF verifier allows reading any field of the pt_regs context, does this
> > allow BPF programs to extract residual kernel stack data?  
> 
> I think because kprobe/bpf is root only, it's not really an issue,

Correct. If you can add kprobes to bpf programs, there's a hell of a lot
more you can "leak" than stack addresses :-p

-- Steve

Re: [PATCH 6.12] x86/fgraph: Fix return_to_handler regs.rsp value

[Sasha Levin]

> commit 8bc11700e0d23d4fdb7d8d5a73b2e95de427cabc upstream.

Queued for 6.12.y, thanks.

--
Thanks,
Sasha