[PATCH] tracing/osnoise: Fix crash in timerlat_dump_stack()

[PATCH] tracing/osnoise: Fix crash in timerlat_dump_stack()

[Tomas Glozar]

We have observed kernel panics when using timerlat with stack saving,
with the following dmesg output:

memcpy: detected buffer overflow: 88 byte write of buffer size 0
WARNING: CPU: 2 PID: 8153 at lib/string_helpers.c:1032 __fortify_report+0x55/0xa0
CPU: 2 UID: 0 PID: 8153 Comm: timerlatu/2 Kdump: loaded Not tainted 6.15.3-200.fc42.x86_64 #1 PREEMPT(lazy)
Call Trace:
 <TASK>
 ? trace_buffer_lock_reserve+0x2a/0x60
 __fortify_panic+0xd/0xf
 __timerlat_dump_stack.cold+0xd/0xd
 timerlat_dump_stack.part.0+0x47/0x80
 timerlat_fd_read+0x36d/0x390
 vfs_read+0xe2/0x390
 ? syscall_exit_to_user_mode+0x1d5/0x210
 ksys_read+0x73/0xe0
 do_syscall_64+0x7b/0x160
 ? exc_page_fault+0x7e/0x1a0
 entry_SYSCALL_64_after_hwframe+0x76/0x7e

__timerlat_dump_stack() constructs the ftrace stack entry like this:

struct stack_entry *entry;
...
memcpy(&entry->caller, fstack->calls, size);
entry->size = fstack->nr_entries;

Since commit e7186af7fb26 ("tracing: Add back FORTIFY_SOURCE logic to
kernel_stack event structure"), struct stack_entry marks its caller
field with __counted_by(size). At the time of the memcpy, entry->size
contains garbage from the ringbuffer, which under some circumstances is
zero, triggering a kernel panic by buffer overflow.

Populate the size field before the memcpy so that the out-of-bounds
check knows the correct size. This is analogous to
__ftrace_trace_stack().

Cc: stable@vger.kernel.org
Fixes: e7186af7fb26 ("tracing: Add back FORTIFY_SOURCE logic to kernel_stack event structure")
Signed-off-by: Tomas Glozar <tglozar@redhat.com>
---
 kernel/trace/trace_osnoise.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

Note: This has been so far only reproduced on laptops (three different
machines). Not sure what the reason for that is, but it is clearly
a bug.

diff --git a/kernel/trace/trace_osnoise.c b/kernel/trace/trace_osnoise.c
index 6819b93309ce..fd259da0aa64 100644
--- a/kernel/trace/trace_osnoise.c
+++ b/kernel/trace/trace_osnoise.c
@@ -637,8 +637,8 @@ __timerlat_dump_stack(struct trace_buffer *buffer, struct trace_stack *fstack, u
 
 	entry = ring_buffer_event_data(event);
 
-	memcpy(&entry->caller, fstack->calls, size);
 	entry->size = fstack->nr_entries;
+	memcpy(&entry->caller, fstack->calls, size);
 
 	trace_buffer_unlock_commit_nostack(buffer, event);
 }
-- 
2.47.1

Re: [PATCH] tracing/osnoise: Fix crash in timerlat_dump_stack()

[Masami Hiramatsu]

On Wed, 16 Jul 2025 16:36:01 +0200
Tomas Glozar <tglozar@redhat.com> wrote:

> We have observed kernel panics when using timerlat with stack saving,
> with the following dmesg output:
> 
> memcpy: detected buffer overflow: 88 byte write of buffer size 0
> WARNING: CPU: 2 PID: 8153 at lib/string_helpers.c:1032 __fortify_report+0x55/0xa0
> CPU: 2 UID: 0 PID: 8153 Comm: timerlatu/2 Kdump: loaded Not tainted 6.15.3-200.fc42.x86_64 #1 PREEMPT(lazy)
> Call Trace:
>  <TASK>
>  ? trace_buffer_lock_reserve+0x2a/0x60
>  __fortify_panic+0xd/0xf
>  __timerlat_dump_stack.cold+0xd/0xd
>  timerlat_dump_stack.part.0+0x47/0x80
>  timerlat_fd_read+0x36d/0x390
>  vfs_read+0xe2/0x390
>  ? syscall_exit_to_user_mode+0x1d5/0x210
>  ksys_read+0x73/0xe0
>  do_syscall_64+0x7b/0x160
>  ? exc_page_fault+0x7e/0x1a0
>  entry_SYSCALL_64_after_hwframe+0x76/0x7e
> 
> __timerlat_dump_stack() constructs the ftrace stack entry like this:
> 
> struct stack_entry *entry;
> ...
> memcpy(&entry->caller, fstack->calls, size);
> entry->size = fstack->nr_entries;
> 
> Since commit e7186af7fb26 ("tracing: Add back FORTIFY_SOURCE logic to
> kernel_stack event structure"), struct stack_entry marks its caller
> field with __counted_by(size). At the time of the memcpy, entry->size
> contains garbage from the ringbuffer, which under some circumstances is
> zero, triggering a kernel panic by buffer overflow.
> 
> Populate the size field before the memcpy so that the out-of-bounds
> check knows the correct size. This is analogous to
> __ftrace_trace_stack().
> 

Looks good to me.

Acked-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>

Thank you!

> Cc: stable@vger.kernel.org
> Fixes: e7186af7fb26 ("tracing: Add back FORTIFY_SOURCE logic to kernel_stack event structure")
> Signed-off-by: Tomas Glozar <tglozar@redhat.com>
> ---
>  kernel/trace/trace_osnoise.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> Note: This has been so far only reproduced on laptops (three different
> machines). Not sure what the reason for that is, but it is clearly
> a bug.
> 
> diff --git a/kernel/trace/trace_osnoise.c b/kernel/trace/trace_osnoise.c
> index 6819b93309ce..fd259da0aa64 100644
> --- a/kernel/trace/trace_osnoise.c
> +++ b/kernel/trace/trace_osnoise.c
> @@ -637,8 +637,8 @@ __timerlat_dump_stack(struct trace_buffer *buffer, struct trace_stack *fstack, u
>  
>  	entry = ring_buffer_event_data(event);
>  
> -	memcpy(&entry->caller, fstack->calls, size);
>  	entry->size = fstack->nr_entries;
> +	memcpy(&entry->caller, fstack->calls, size);
>  
>  	trace_buffer_unlock_commit_nostack(buffer, event);
>  }
> -- 
> 2.47.1
> 
> 


-- 
Masami Hiramatsu (Google) <mhiramat@kernel.org>