* [BUG] tracing: trace_pipe reads out of bounds when it races with the ring buffer mmap reader
@ 2026-11-16 17:56 Kyuwon Cho
0 siblings, 0 replies; only message in thread
From: Kyuwon Cho @ 2026-11-16 17:56 UTC (permalink / raw)
To: Steven Rostedt, Masami Hiramatsu
Cc: Vincent Donnefort, Mathieu Desnoyers, linux-trace-kernel,
linux-kernel
[-- Attachment #1: Type: text/plain, Size: 5785 bytes --]
Hi Steve, Masami, Vincent,
While testing v7.4-rc3 with an LLM-guided test generator that we are
developing as a research prototype, we found that reading a CPU's
trace_pipe while another task consumes the same CPU buffer through the
mmap interface (TRACE_MMAP_IOCTL_GET_READER) makes trace_pipe print
entries that have already been overwritten. With KASAN this ends in an
out-of-bounds read in the event output code:
BUG: KASAN: use-after-free in string (lib/vsprintf.c:650 lib/vsprintf.c:732)
Read of size 1 at addr ff11000011e2d8ed by task test/89
Call Trace:
<TASK>
string (lib/vsprintf.c:650 lib/vsprintf.c:732)
vsnprintf (lib/vsprintf.c:2948)
seq_buf_vprintf (lib/seq_buf.c:74)
trace_seq_vprintf (kernel/trace/trace_seq.c:175)
trace_event_printf (kernel/trace/trace_output.c:360)
trace_raw_output_workqueue_rescued (include/trace/events/workqueue.h:210)
print_trace_line (kernel/trace/trace.c:3082 kernel/trace/trace.c:3248)
tracing_read_pipe (kernel/trace/trace.c:5475)
vfs_read (fs/read_write.c:552)
ksys_read (fs/read_write.c:696)
do_syscall_64 (arch/x86/entry/syscall_64.c:61 arch/x86/entry/syscall_64.c:89)
entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:121)
</TASK>
The buggy address belongs to the physical page:
page: refcount:0 mapcount:0 mapping:0000000000000000 index:0x0 pfn:0x11e2d
page_type: f0(buddy)
and the type check in trace_assign_type() fires as well:
WARNING: kernel/trace/trace_output.c:1798 at trace_print_print+0xeb/0x130, CPU#1: test/89
Call Trace:
<TASK>
print_trace_line (kernel/trace/trace.c:3082 kernel/trace/trace.c:3248)
tracing_read_pipe (kernel/trace/trace.c:5475)
vfs_read (fs/read_write.c:552)
ksys_read (fs/read_write.c:696)
do_syscall_64 (arch/x86/entry/syscall_64.c:61 arch/x86/entry/syscall_64.c:89)
entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:121)
</TASK>
The stale entry carries another event's type, so that event's output
function decodes trace_marker data as its fields, including __data_loc
offsets and lengths that point outside the ring buffer page. Other runs
reported slab-out-of-bounds and global-out-of-bounds reads from the same
path; the ring buffer pages themselves are not freed.
tracing_buffers_ioctl() calls ring_buffer_map_get_reader() without
trace_access_lock():
if (cmd == TRACE_MMAP_IOCTL_GET_READER) {
...
return ring_buffer_map_get_reader(iter->array_buffer->buffer,
iter->cpu_file);
}
The other consuming readers of a CPU buffer, tracing_read_pipe(),
tracing_buffers_read() and tracing_buffers_splice_read(), all take
trace_access_lock(iter->cpu_file). tracing_read_pipe() peeks an event,
prints it with reader_lock dropped and only then consumes it, so a
GET_READER in between can hand the reader page back to the writers. This
has been the case since the ioctl was added in cf9f0f7c4c5b ("tracing:
Allow user-space mapping of the ring-buffer").
Taking trace_access_lock(iter->cpu_file) around
ring_buffer_map_get_reader() in tracing_buffers_ioctl(), as the other
readers do, makes the problem go away here: the reproducer below ran
three times without a report, and the ring-buffer map_test selftest
still passes.
The reproducer needs root, for tracefs. One thread writes to trace_marker
on CPU 0; on CPU 1, one thread loops on TRACE_MMAP_IOCTL_GET_READER on
the mmapped per_cpu/cpu0/trace_pipe_raw and another reads
per_cpu/cpu0/trace_pipe. On an x86_64 QEMU guest with 2 vCPUs, KASAN
and lockdep (gcc 13.3, config attached), the WARNING appears within
seconds in every run and KASAN reports in 2 of 3 runs. Neither reader
alone triggers it.
Checked against v7.4-rc3 (aebb72744764); tracing_buffers_ioctl() is the
same in trace/for-next. We found no earlier report on syzbot or on
linux-trace-kernel.
// Run as root with tracefs available and at least 2 CPUs.
// CPU 0: write to trace_marker. CPU 1: one thread polls
// TRACE_MMAP_IOCTL_GET_READER on the mmapped per_cpu/cpu0/trace_pipe_raw,
// another reads per_cpu/cpu0/trace_pipe.
#define _GNU_SOURCE
#include <fcntl.h>
#include <pthread.h>
#include <sched.h>
#include <stdio.h>
#include <string.h>
#include <sys/ioctl.h>
#include <sys/mman.h>
#include <sys/mount.h>
#include <sys/stat.h>
#include <unistd.h>
#define T "/sys/kernel/tracing"
#define TRACE_MMAP_IOCTL_GET_READER _IO('R', 0x20)
static volatile int stop;
static void pin(int cpu)
{
cpu_set_t set;
CPU_ZERO(&set);
CPU_SET(cpu, &set);
sched_setaffinity(0, sizeof(set), &set);
}
static void put(const char *file, const char *val)
{
int fd = open(file, O_WRONLY | O_TRUNC);
if (fd >= 0) {
write(fd, val, strlen(val));
close(fd);
}
}
static void *writer(void *arg)
{
char buf[1000];
int fd = open(T "/trace_marker", O_WRONLY);
pin(0);
memset(buf, 'a', sizeof(buf));
for (unsigned long i = 0; !stop; i++)
write(fd, buf, 1 + i % sizeof(buf));
return NULL;
}
static void *mapper(void *arg)
{
int fd = open(T "/per_cpu/cpu0/trace_pipe_raw", O_RDONLY | O_NONBLOCK);
pin(1);
if (mmap(NULL, getpagesize(), PROT_READ, MAP_SHARED, fd, 0) == MAP_FAILED) {
perror("mmap");
return NULL;
}
while (!stop)
ioctl(fd, TRACE_MMAP_IOCTL_GET_READER);
return NULL;
}
static void *cooked(void *arg)
{
char buf[4096];
int fd = open(T "/per_cpu/cpu0/trace_pipe", O_RDONLY | O_NONBLOCK);
pin(1);
while (!stop)
read(fd, buf, sizeof(buf));
return NULL;
}
int main(void)
{
pthread_t th[3];
mount("tracefs", T, "tracefs", 0, NULL);
put(T "/buffer_size_kb", "16");
put(T "/tracing_on", "1");
pthread_create(&th[0], NULL, writer, NULL);
pthread_create(&th[1], NULL, mapper, NULL);
pthread_create(&th[2], NULL, cooked, NULL);
sleep(60);
stop = 1;
for (int i = 0; i < 3; i++)
pthread_join(th[i], NULL);
return 0;
}
Thanks,
Kyuwon Cho
[-- Attachment #2: config.gz --]
[-- Type: application/gzip, Size: 36886 bytes --]
^ permalink raw reply [flat|nested] only message in thread
only message in thread, other threads:[~2026-11-16 17:56 UTC | newest]
Thread overview: (only message) (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-11-16 17:56 [BUG] tracing: trace_pipe reads out of bounds when it races with the ring buffer mmap reader Kyuwon Cho
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox