Linux Trace Kernel
 help / color / mirror / Atom feed
From: Kyuwon Cho <kyuwoncho18@gmail.com>
To: Steven Rostedt <rostedt@goodmis.org>,
	Masami Hiramatsu <mhiramat@kernel.org>
Cc: Vincent Donnefort <vdonnefort@google.com>,
	Mathieu Desnoyers <mathieu.desnoyers@efficios.com>,
	linux-trace-kernel@vger.kernel.org, linux-kernel@vger.kernel.org
Subject: [BUG] tracing: trace_pipe reads out of bounds when it races with the ring buffer mmap reader
Date: Mon, 16 Nov 2026 09:56:31 -0800 (PST)	[thread overview]
Message-ID: <179485165154.2547186.6129837870360499372@gmail.com> (raw)

[-- Attachment #1: Type: text/plain, Size: 5785 bytes --]

Hi Steve, Masami, Vincent,

While testing v7.4-rc3 with an LLM-guided test generator that we are
developing as a research prototype, we found that reading a CPU's
trace_pipe while another task consumes the same CPU buffer through the
mmap interface (TRACE_MMAP_IOCTL_GET_READER) makes trace_pipe print
entries that have already been overwritten. With KASAN this ends in an
out-of-bounds read in the event output code:

  BUG: KASAN: use-after-free in string (lib/vsprintf.c:650 lib/vsprintf.c:732)
  Read of size 1 at addr ff11000011e2d8ed by task test/89
  Call Trace:
   <TASK>
   string (lib/vsprintf.c:650 lib/vsprintf.c:732)
   vsnprintf (lib/vsprintf.c:2948)
   seq_buf_vprintf (lib/seq_buf.c:74)
   trace_seq_vprintf (kernel/trace/trace_seq.c:175)
   trace_event_printf (kernel/trace/trace_output.c:360)
   trace_raw_output_workqueue_rescued (include/trace/events/workqueue.h:210)
   print_trace_line (kernel/trace/trace.c:3082 kernel/trace/trace.c:3248)
   tracing_read_pipe (kernel/trace/trace.c:5475)
   vfs_read (fs/read_write.c:552)
   ksys_read (fs/read_write.c:696)
   do_syscall_64 (arch/x86/entry/syscall_64.c:61 arch/x86/entry/syscall_64.c:89)
   entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:121)
   </TASK>
  The buggy address belongs to the physical page:
  page: refcount:0 mapcount:0 mapping:0000000000000000 index:0x0 pfn:0x11e2d
  page_type: f0(buddy)

and the type check in trace_assign_type() fires as well:

  WARNING: kernel/trace/trace_output.c:1798 at trace_print_print+0xeb/0x130, CPU#1: test/89
  Call Trace:
   <TASK>
   print_trace_line (kernel/trace/trace.c:3082 kernel/trace/trace.c:3248)
   tracing_read_pipe (kernel/trace/trace.c:5475)
   vfs_read (fs/read_write.c:552)
   ksys_read (fs/read_write.c:696)
   do_syscall_64 (arch/x86/entry/syscall_64.c:61 arch/x86/entry/syscall_64.c:89)
   entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:121)
   </TASK>

The stale entry carries another event's type, so that event's output
function decodes trace_marker data as its fields, including __data_loc
offsets and lengths that point outside the ring buffer page. Other runs
reported slab-out-of-bounds and global-out-of-bounds reads from the same
path; the ring buffer pages themselves are not freed.

tracing_buffers_ioctl() calls ring_buffer_map_get_reader() without
trace_access_lock():

	if (cmd == TRACE_MMAP_IOCTL_GET_READER) {
		...
		return ring_buffer_map_get_reader(iter->array_buffer->buffer,
						  iter->cpu_file);
	}

The other consuming readers of a CPU buffer, tracing_read_pipe(),
tracing_buffers_read() and tracing_buffers_splice_read(), all take
trace_access_lock(iter->cpu_file). tracing_read_pipe() peeks an event,
prints it with reader_lock dropped and only then consumes it, so a
GET_READER in between can hand the reader page back to the writers. This
has been the case since the ioctl was added in cf9f0f7c4c5b ("tracing:
Allow user-space mapping of the ring-buffer").

Taking trace_access_lock(iter->cpu_file) around
ring_buffer_map_get_reader() in tracing_buffers_ioctl(), as the other
readers do, makes the problem go away here: the reproducer below ran
three times without a report, and the ring-buffer map_test selftest
still passes.

The reproducer needs root, for tracefs. One thread writes to trace_marker
on CPU 0; on CPU 1, one thread loops on TRACE_MMAP_IOCTL_GET_READER on
the mmapped per_cpu/cpu0/trace_pipe_raw and another reads
per_cpu/cpu0/trace_pipe. On an x86_64 QEMU guest with 2 vCPUs, KASAN
and lockdep (gcc 13.3, config attached), the WARNING appears within
seconds in every run and KASAN reports in 2 of 3 runs. Neither reader
alone triggers it.

Checked against v7.4-rc3 (aebb72744764); tracing_buffers_ioctl() is the
same in trace/for-next. We found no earlier report on syzbot or on
linux-trace-kernel.

// Run as root with tracefs available and at least 2 CPUs.
// CPU 0: write to trace_marker. CPU 1: one thread polls
// TRACE_MMAP_IOCTL_GET_READER on the mmapped per_cpu/cpu0/trace_pipe_raw,
// another reads per_cpu/cpu0/trace_pipe.
#define _GNU_SOURCE
#include <fcntl.h>
#include <pthread.h>
#include <sched.h>
#include <stdio.h>
#include <string.h>
#include <sys/ioctl.h>
#include <sys/mman.h>
#include <sys/mount.h>
#include <sys/stat.h>
#include <unistd.h>

#define T "/sys/kernel/tracing"
#define TRACE_MMAP_IOCTL_GET_READER _IO('R', 0x20)

static volatile int stop;

static void pin(int cpu)
{
	cpu_set_t set;

	CPU_ZERO(&set);
	CPU_SET(cpu, &set);
	sched_setaffinity(0, sizeof(set), &set);
}

static void put(const char *file, const char *val)
{
	int fd = open(file, O_WRONLY | O_TRUNC);

	if (fd >= 0) {
		write(fd, val, strlen(val));
		close(fd);
	}
}

static void *writer(void *arg)
{
	char buf[1000];
	int fd = open(T "/trace_marker", O_WRONLY);

	pin(0);
	memset(buf, 'a', sizeof(buf));
	for (unsigned long i = 0; !stop; i++)
		write(fd, buf, 1 + i % sizeof(buf));
	return NULL;
}

static void *mapper(void *arg)
{
	int fd = open(T "/per_cpu/cpu0/trace_pipe_raw", O_RDONLY | O_NONBLOCK);

	pin(1);
	if (mmap(NULL, getpagesize(), PROT_READ, MAP_SHARED, fd, 0) == MAP_FAILED) {
		perror("mmap");
		return NULL;
	}
	while (!stop)
		ioctl(fd, TRACE_MMAP_IOCTL_GET_READER);
	return NULL;
}

static void *cooked(void *arg)
{
	char buf[4096];
	int fd = open(T "/per_cpu/cpu0/trace_pipe", O_RDONLY | O_NONBLOCK);

	pin(1);
	while (!stop)
		read(fd, buf, sizeof(buf));
	return NULL;
}

int main(void)
{
	pthread_t th[3];

	mount("tracefs", T, "tracefs", 0, NULL);
	put(T "/buffer_size_kb", "16");
	put(T "/tracing_on", "1");
	pthread_create(&th[0], NULL, writer, NULL);
	pthread_create(&th[1], NULL, mapper, NULL);
	pthread_create(&th[2], NULL, cooked, NULL);
	sleep(60);
	stop = 1;
	for (int i = 0; i < 3; i++)
		pthread_join(th[i], NULL);
	return 0;
}

Thanks,
Kyuwon Cho

[-- Attachment #2: config.gz --]
[-- Type: application/gzip, Size: 36886 bytes --]

                 reply	other threads:[~2026-11-16 17:56 UTC|newest]

Thread overview: [no followups] expand[flat|nested]  mbox.gz  Atom feed

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=179485165154.2547186.6129837870360499372@gmail.com \
    --to=kyuwoncho18@gmail.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-trace-kernel@vger.kernel.org \
    --cc=mathieu.desnoyers@efficios.com \
    --cc=mhiramat@kernel.org \
    --cc=rostedt@goodmis.org \
    --cc=vdonnefort@google.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox