From: Steven Rostedt <rostedt@goodmis.org>
To: Tengda Wu <wutengda@huaweicloud.com>
Cc: Masami Hiramatsu <mhiramat@kernel.org>,
Mathieu Desnoyers <mathieu.desnoyers@efficios.com>,
linux-trace-kernel@vger.kernel.org, linux-kernel@vger.kernel.org
Subject: Re: [PATCH] tracing/snapshot: Avoid CPU buffer swap during reserve/commit
Date: Wed, 29 Jul 2026 22:04:48 -0400 [thread overview]
Message-ID: <20260729220448.13d19821@robin> (raw)
In-Reply-To: <20260730011912.2325121-1-wutengda@huaweicloud.com>
On Thu, 30 Jul 2026 01:19:12 +0000
Tengda Wu <wutengda@huaweicloud.com> wrote:
> Commit 3163f635b20e ("tracing: Fix race issue between cpu buffer write
> and swap") fixed most of the race conditions between snapshot's
> ring_buffer_swap_cpu and ring_buffer_lock_{reserve, commit}. It achieved
> this by replacing the asynchronous swap with smp_call_function_single to
> trigger an interrupt on the target CPU to handle the swap.
I'm curious. How did you discover this race?
>
> However, this interrupt can still break in at any point during
> ring_buffer_lock_{reserve, commit}. Currently, ring_buffer_swap_cpu()
> relies on a cooperative check where cpu_buffer->committing is combined
> with a condition in rb_reserve_next_event() that verifies if
> READ_ONCE(cpu_buffer->buffer) != buffer to ensure state consistency. In
> principle, once the execution passes this conditional check, no issues
> should arise as long as cpu_buffer->committing remains non-zero.
> Unfortunately, there is a window where cpu_buffer->committing can drop
> to zero, which subsequently triggers a warning during rb_commit:
>
> ring_buffer_lock_reserve
> cpu_buffer = buffer->buffers[cpu]; // cpu_buffer_a
> rb_reserve_next_event
> rb_start_commit // inc committing
> if (unlikely(READ_ONCE(cpu_buffer->buffer) != buffer)) {...}
> __rb_reserve_next
> rb_move_tail
> rb_end_commit(cpu_buffer); // dec committing => 0
> /* smp_call interrupt hits here, successfully swaps! */
> local_inc(&cpu_buffer->committing);
>
> ring_buffer_unlock_commit
> cpu_buffer = buffer->buffers[cpu]; // cpu_buffer_b
> rb_commit
> rb_end_commit
> RB_WARN_ON(cpu_buffer, !local_read(&cpu_buffer->committing))
> // triggers warning
>
> Instead of maintaining fragile complexity to allow the swap to intervene at
> arbitrary points, simply replace smp_call_function_single with work_on_cpu.
> This ensures the swap operation runs in a process context on the target
> CPU, guaranteeing it only executes after ring_buffer_lock_{reserve, commit}
> has fully completed.
No this is not the answer. The issue is with the ring buffer swap code
but you band-aid in with a fix to the user of the code. The fix needs
to stay with the ring buffer. Yes that means it needs to handle the
case where an interrupt comes in at a page swap.
It needs to find another way to know that it's in the middle of that
swap, and return an -EBUSY from the interrupt.
-- Steve
next prev parent reply other threads:[~2026-07-30 2:04 UTC|newest]
Thread overview: 3+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-07-30 1:19 [PATCH] tracing/snapshot: Avoid CPU buffer swap during reserve/commit Tengda Wu
2026-07-30 2:04 ` Steven Rostedt [this message]
2026-07-30 4:10 ` Tengda Wu
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20260729220448.13d19821@robin \
--to=rostedt@goodmis.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-trace-kernel@vger.kernel.org \
--cc=mathieu.desnoyers@efficios.com \
--cc=mhiramat@kernel.org \
--cc=wutengda@huaweicloud.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox