From: Kyuwon Cho <kyuwoncho18@gmail.com>
To: Steven Rostedt <rostedt@goodmis.org>,
Masami Hiramatsu <mhiramat@kernel.org>
Cc: Vincent Donnefort <vdonnefort@google.com>,
Mathieu Desnoyers <mathieu.desnoyers@efficios.com>,
linux-trace-kernel@vger.kernel.org, linux-kernel@vger.kernel.org
Subject: [BUG] tracing: trace_pipe reads out of bounds when it races with the ring buffer mmap reader
Date: Mon, 16 Nov 2026 09:56:31 -0800 (PST) [thread overview]
Message-ID: <179485165154.2547186.6129837870360499372@gmail.com> (raw)
[-- Attachment #1: Type: text/plain, Size: 5785 bytes --]
Hi Steve, Masami, Vincent,
While testing v7.4-rc3 with an LLM-guided test generator that we are
developing as a research prototype, we found that reading a CPU's
trace_pipe while another task consumes the same CPU buffer through the
mmap interface (TRACE_MMAP_IOCTL_GET_READER) makes trace_pipe print
entries that have already been overwritten. With KASAN this ends in an
out-of-bounds read in the event output code:
BUG: KASAN: use-after-free in string (lib/vsprintf.c:650 lib/vsprintf.c:732)
Read of size 1 at addr ff11000011e2d8ed by task test/89
Call Trace:
<TASK>
string (lib/vsprintf.c:650 lib/vsprintf.c:732)
vsnprintf (lib/vsprintf.c:2948)
seq_buf_vprintf (lib/seq_buf.c:74)
trace_seq_vprintf (kernel/trace/trace_seq.c:175)
trace_event_printf (kernel/trace/trace_output.c:360)
trace_raw_output_workqueue_rescued (include/trace/events/workqueue.h:210)
print_trace_line (kernel/trace/trace.c:3082 kernel/trace/trace.c:3248)
tracing_read_pipe (kernel/trace/trace.c:5475)
vfs_read (fs/read_write.c:552)
ksys_read (fs/read_write.c:696)
do_syscall_64 (arch/x86/entry/syscall_64.c:61 arch/x86/entry/syscall_64.c:89)
entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:121)
</TASK>
The buggy address belongs to the physical page:
page: refcount:0 mapcount:0 mapping:0000000000000000 index:0x0 pfn:0x11e2d
page_type: f0(buddy)
and the type check in trace_assign_type() fires as well:
WARNING: kernel/trace/trace_output.c:1798 at trace_print_print+0xeb/0x130, CPU#1: test/89
Call Trace:
<TASK>
print_trace_line (kernel/trace/trace.c:3082 kernel/trace/trace.c:3248)
tracing_read_pipe (kernel/trace/trace.c:5475)
vfs_read (fs/read_write.c:552)
ksys_read (fs/read_write.c:696)
do_syscall_64 (arch/x86/entry/syscall_64.c:61 arch/x86/entry/syscall_64.c:89)
entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:121)
</TASK>
The stale entry carries another event's type, so that event's output
function decodes trace_marker data as its fields, including __data_loc
offsets and lengths that point outside the ring buffer page. Other runs
reported slab-out-of-bounds and global-out-of-bounds reads from the same
path; the ring buffer pages themselves are not freed.
tracing_buffers_ioctl() calls ring_buffer_map_get_reader() without
trace_access_lock():
if (cmd == TRACE_MMAP_IOCTL_GET_READER) {
...
return ring_buffer_map_get_reader(iter->array_buffer->buffer,
iter->cpu_file);
}
The other consuming readers of a CPU buffer, tracing_read_pipe(),
tracing_buffers_read() and tracing_buffers_splice_read(), all take
trace_access_lock(iter->cpu_file). tracing_read_pipe() peeks an event,
prints it with reader_lock dropped and only then consumes it, so a
GET_READER in between can hand the reader page back to the writers. This
has been the case since the ioctl was added in cf9f0f7c4c5b ("tracing:
Allow user-space mapping of the ring-buffer").
Taking trace_access_lock(iter->cpu_file) around
ring_buffer_map_get_reader() in tracing_buffers_ioctl(), as the other
readers do, makes the problem go away here: the reproducer below ran
three times without a report, and the ring-buffer map_test selftest
still passes.
The reproducer needs root, for tracefs. One thread writes to trace_marker
on CPU 0; on CPU 1, one thread loops on TRACE_MMAP_IOCTL_GET_READER on
the mmapped per_cpu/cpu0/trace_pipe_raw and another reads
per_cpu/cpu0/trace_pipe. On an x86_64 QEMU guest with 2 vCPUs, KASAN
and lockdep (gcc 13.3, config attached), the WARNING appears within
seconds in every run and KASAN reports in 2 of 3 runs. Neither reader
alone triggers it.
Checked against v7.4-rc3 (aebb72744764); tracing_buffers_ioctl() is the
same in trace/for-next. We found no earlier report on syzbot or on
linux-trace-kernel.
// Run as root with tracefs available and at least 2 CPUs.
// CPU 0: write to trace_marker. CPU 1: one thread polls
// TRACE_MMAP_IOCTL_GET_READER on the mmapped per_cpu/cpu0/trace_pipe_raw,
// another reads per_cpu/cpu0/trace_pipe.
#define _GNU_SOURCE
#include <fcntl.h>
#include <pthread.h>
#include <sched.h>
#include <stdio.h>
#include <string.h>
#include <sys/ioctl.h>
#include <sys/mman.h>
#include <sys/mount.h>
#include <sys/stat.h>
#include <unistd.h>
#define T "/sys/kernel/tracing"
#define TRACE_MMAP_IOCTL_GET_READER _IO('R', 0x20)
static volatile int stop;
static void pin(int cpu)
{
cpu_set_t set;
CPU_ZERO(&set);
CPU_SET(cpu, &set);
sched_setaffinity(0, sizeof(set), &set);
}
static void put(const char *file, const char *val)
{
int fd = open(file, O_WRONLY | O_TRUNC);
if (fd >= 0) {
write(fd, val, strlen(val));
close(fd);
}
}
static void *writer(void *arg)
{
char buf[1000];
int fd = open(T "/trace_marker", O_WRONLY);
pin(0);
memset(buf, 'a', sizeof(buf));
for (unsigned long i = 0; !stop; i++)
write(fd, buf, 1 + i % sizeof(buf));
return NULL;
}
static void *mapper(void *arg)
{
int fd = open(T "/per_cpu/cpu0/trace_pipe_raw", O_RDONLY | O_NONBLOCK);
pin(1);
if (mmap(NULL, getpagesize(), PROT_READ, MAP_SHARED, fd, 0) == MAP_FAILED) {
perror("mmap");
return NULL;
}
while (!stop)
ioctl(fd, TRACE_MMAP_IOCTL_GET_READER);
return NULL;
}
static void *cooked(void *arg)
{
char buf[4096];
int fd = open(T "/per_cpu/cpu0/trace_pipe", O_RDONLY | O_NONBLOCK);
pin(1);
while (!stop)
read(fd, buf, sizeof(buf));
return NULL;
}
int main(void)
{
pthread_t th[3];
mount("tracefs", T, "tracefs", 0, NULL);
put(T "/buffer_size_kb", "16");
put(T "/tracing_on", "1");
pthread_create(&th[0], NULL, writer, NULL);
pthread_create(&th[1], NULL, mapper, NULL);
pthread_create(&th[2], NULL, cooked, NULL);
sleep(60);
stop = 1;
for (int i = 0; i < 3; i++)
pthread_join(th[i], NULL);
return 0;
}
Thanks,
Kyuwon Cho
[-- Attachment #2: config.gz --]
[-- Type: application/gzip, Size: 36886 bytes --]
reply other threads:[~2026-11-16 17:56 UTC|newest]
Thread overview: [no followups] expand[flat|nested] mbox.gz Atom feed
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=179485165154.2547186.6129837870360499372@gmail.com \
--to=kyuwoncho18@gmail.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-trace-kernel@vger.kernel.org \
--cc=mathieu.desnoyers@efficios.com \
--cc=mhiramat@kernel.org \
--cc=rostedt@goodmis.org \
--cc=vdonnefort@google.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox