* Re: [RFC PATCH 0/7] mm: defer address-space teardown of large exiting processes to a kthread
From: Aditya Sharma @ 2026-07-20 18:38 UTC (permalink / raw)
To: Liam R . Howlett
Cc: Andrew Morton, David Hildenbrand, Lorenzo Stoakes,
Vlastimil Babka, Mike Rapoport, Suren Baghdasaryan, Michal Hocko,
David Rientjes, Shakeel Butt, Jonathan Corbet, Shuah Khan,
Steven Rostedt, Masami Hiramatsu, Mathieu Desnoyers, Kees Cook,
Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
Dietmar Eggemann, Ben Segall, Mel Gorman, Valentin Schneider,
K Prateek Nayak, linux-mm, linux-doc, linux-trace-kernel,
linux-kernel, imbrenda
In-Reply-To: <ljr2jkxomq4ccgzq2ip3c66mjdryomhl2gxkswv4wzry5ye2ax@xaehxzjkkk3m>
Sent using Zoho Mail
From: Liam R . Howlett <liam@infradead.org>
To: "Aditya Sharma"<adi.sharma@zohomail.in>
Cc: "Andrew Morton"<akpm@linux-foundation.org>, "David Hildenbrand"<david@kernel.org>, "Lorenzo Stoakes"<ljs@kernel.org>, "Vlastimil Babka"<vbabka@kernel.org>, "Mike Rapoport"<rppt@kernel.org>, "Suren Baghdasaryan"<surenb@google.com>, "Michal Hocko"<mhocko@suse.com>, "David Rientjes"<rientjes@google.com>, "Shakeel Butt"<shakeel.butt@linux.dev>, "Jonathan Corbet"<corbet@lwn.net>, "Shuah Khan"<skhan@linuxfoundation.org>, "Steven Rostedt"<rostedt@goodmis.org>, "Masami Hiramatsu"<mhiramat@kernel.org>, "Mathieu Desnoyers"<mathieu.desnoyers@efficios.com>, "Kees Cook"<kees@kernel.org>, "Ingo Molnar"<mingo@redhat.com>, "Peter Zijlstra"<peterz@infradead.org>, "Juri Lelli"<juri.lelli@redhat.com>, "Vincent Guittot"<vincent.guittot@linaro.org>, "Dietmar Eggemann"<dietmar.eggemann@arm.com>, "Ben Segall"<bsegall@google.com>, "Mel Gorman"<mgorman@suse.de>, "Valentin Schneider"<vschneid@redhat.com>, "K Prateek Nayak"<kprateek.nayak@amd.com>, <linux-mm@kvack.org>, <linux-doc@vger.kernel.org>, <linux-trace-kernel@vger.kernel.org>, <linux-kernel@vger.kernel.org>, <imbrenda@linux.ibm.com>
Date: Mon, 20 Jul 2026 03:58:47 +0530
Subject: Re: [RFC PATCH 0/7] mm: defer address-space teardown of large exiting processes to a kthread
> This reads like it was written by an AI, but there is no assisted-by
> tags. Which AI did you use to generate these patches and/or cover
> letter?
I used claude opus 4.8 to refine commit messages (the design is mine,
the model was used to convey the intentions concretely). Further, I used
claude fable 5 for drafting the cover letter and reviewing the series, where
it flagged issues (I fixed some of those by hand and some were fixed by fable itself,
which I then reviewed). It also assisted in writing the benchmark harness
behind the numbers and did the litmus test the cover letter mentions. Every numbers
was measured by me, on my hardware. I take full responsibily of the entire
series.
My bad that I did not add an assisted-by tag. Should have added it to v1
v2 will add it to the patches
> I do not see a benefit to running the task cleanup 'off the side'. In
> modern computers, that may mean switching core types that run at
> different speeds.
That's why the reaper's affinity is a preference, not a bind (patch 2).
It uses kthread_affine_preferred() with housekeeping mask, so it can be
re-affined. kthread_bind_mask() would have taken that away. On a hybrid cpu,
it can be affined(by the (super)user) to whichever cores are appropriate.
I do not have numbers for P core and E core teardown costs. Will measure and
send them.
> Regardless of speed, the CPU that will be doing the work is remote to
> the workload by design, (or may be, depending on the scheduling?). That
> is, you are tasking another CPU to do cleanup of local CPU-aware
> information.. The ideal CPU to do a task exit is the one that's doing it
> already.
Yes, it does cost something. Five consecutive 4GB inline teardowns in my testing
costed 372ms in exit path, while 440 ms in the case of kthread drain for the same
(five teardowns on another cpu). This cost became 1.18x. For multi gb mms,
the memory to be reaped would be much larger than the cpu cache can hold.
For small mms (< 64 MB, configurable by sysctl), the teardowns happen inline.
The oom_reaper already reaps the victim's address space from a remote context,
as returning memory matters more than locality of teardown.
> There are hidden costs to your approach and I don't really understand
> how this could possibly speed things up - even parallelized tasks will
> take more CPU time in total, even if locking issues are avoided.
This is not meant to speed up teardown. Rather the latency for someone waiting
on death is reduced. Killing a 16 GB redis cost 292.97ms before returning to
waitpid(). It costs 0.24 with async teardown. The memory itself was returned
in about 300ms.
"parallelized" - This series does not parallelize the teardown. The design is serial.
N procesess exiting together today would tear down concurrently today. The reaper here,
uses one thread (nice 19, affined to housekeeping mask).
The numbers in cover letter support that this work is memory bound rather than CPU bound.
In sync case: teardown is around 56 GB/s on my machine (1GB in 18.01 ms and 16GB in 282.16ms).
Thirty two concurrent 0.5GB exits reap in 157.42ms (around 101 GB/s). So, a concurrency
of 32 gets about 1.8x. The single reaper thread takes 650ms to clear the same (about 25 GB/s).
Thus, this serialization took 4x the drain time, but takes peak concurrency down to
one cpu. Memory actully comes back later. Whether that's a good tradeoff, depends on
what else we wish to run on the box (open questions #1 in cover letter)
> Sure, the task waiting for cleanup to complete may think it's done and
> continue to do the Next Thing - but if it really needs the cleanup to be
> completed
It is off by default and opt in (sysctl), so anything that depends on the
current semantics, continues to work exactly the same by doing nothing.
Also, mmput_async() already defers the teardown to a workqueue. So, in the current
kernel, there already exists a way where continuing to the next thing
does not necessarily mean that the memory is reclaimed.
> completed then you've just made it impossible to know when it has
> happened.
There are three vmstat counters and two tracepoint (patch 6).
Someone who opts in to the mechanism, and does need to make sure that the
cleanup is completed, can use the counters and tracepoints.
> Also, you've just made this extremely hard to debug if something is
> missed.
There are reap tracepoints that carry the pid, so we can get per pid completion status.
However, there is no number indicating the current depth of the queue. The series addds queued,
sync and rejected counters. Will add a reaped counter in v2 so that current queue depth
can be estimated. Please mention what else tracability can be added to this,
and what senairo do you have in mind that these counters and tracepoints might not
cover. I can add those to v2 as well.
> The work that is done in teardown is necessary. Your time (and tokens?)
> would be better spent trying to improve what we have to do instead of
> shifting the work around.
This does not compete with with current teardown path. it goes through the
same exit_mm(). Any future improvements there, also improve the reaper's drain
times.
Thanks
Aditya
^ permalink raw reply
* [PATCH] selftests/tracing: Fix flakiness in trigger-hist-poll.tc by waiting for task exit
From: Masami Hiramatsu (Google) @ 2026-07-21 0:20 UTC (permalink / raw)
To: Steven Rostedt
Cc: oe-lkp, lkp, linux-kernel, Shuah Khan, Masami Hiramatsu,
linux-trace-kernel, oliver.sang
From: Masami Hiramatsu (Google) <mhiramat@kernel.org>
Commit c5c413534d40 ("selftests/tracing: Have trigger-hist-poll.tc use
sched_process_exit") changed the target event from sched_process_free to
sched_process_exit to avoid timeouts under CONFIG_RCU_LAZY.
However, sched_process_exit fires inside do_exit() before the exiting task
completes teardown and before the parent shell reaps it. When poll unblocks
upon sched_process_exit, checking whether /proc/${BGPID} exists immediately
after poll returns causes a race condition between poll exiting and the
parent shell reaping the child process.
Fix this by waiting for the background sleep process (wait ${BGPID}) and
removing the fragile /proc/${BGPID} check. Checking whether the event was
logged in the trace buffer already verifies that poll did not exit before
the event occurred.
Reported-by: kernel test robot <oliver.sang@intel.com>
Closes: https://lore.kernel.org/oe-lkp/202607161651.ab0a5bf6-lkp@intel.com
Fixes: c5c413534d40 ("selftests/tracing: Have trigger-hist-poll.tc use sched_process_exit")
Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
---
.../ftrace/test.d/trigger/trigger-hist-poll.tc | 14 ++++----------
1 file changed, 4 insertions(+), 10 deletions(-)
diff --git a/tools/testing/selftests/ftrace/test.d/trigger/trigger-hist-poll.tc b/tools/testing/selftests/ftrace/test.d/trigger/trigger-hist-poll.tc
index 04eb8546fc07..d54d54f9ec77 100644
--- a/tools/testing/selftests/ftrace/test.d/trigger/trigger-hist-poll.tc
+++ b/tools/testing/selftests/ftrace/test.d/trigger/trigger-hist-poll.tc
@@ -38,11 +38,7 @@ BGPID=$!
${POLL} -I -t 4000 ${EVENT}/hist
echo 0 > tracing_on
-if [ -d /proc/${BGPID} ]; then
- echo "poll exits too soon"
- kill -KILL ${BGPID} ||:
- exit_fail
-fi
+wait ${BGPID} 2>/dev/null ||:
if ! grep -qw "sleep" trace; then
echo "poll exits before event happens"
@@ -60,11 +56,7 @@ BGPID=$!
${POLL} -P -t 4000 ${EVENT}/hist
echo 0 > tracing_on
-if [ -d /proc/${BGPID} ]; then
- echo "poll exits too soon"
- kill -KILL ${BGPID} ||:
- exit_fail
-fi
+wait ${BGPID} 2>/dev/null ||:
if ! grep -qw "sleep" trace; then
echo "poll exits before event happens"
@@ -72,3 +64,5 @@ if ! grep -qw "sleep" trace; then
fi
exit_pass
+
+
^ permalink raw reply related
* Re: Linking Patchwork with Sashiko?
From: Masami Hiramatsu @ 2026-07-21 0:34 UTC (permalink / raw)
To: Steven Rostedt; +Cc: Roman Gushchin, Masami Hiramatsu, Linux trace kernel
In-Reply-To: <20260720093655.4da2224d@robin>
On Mon, 20 Jul 2026 09:36:55 -0400
Steven Rostedt <rostedt@goodmis.org> wrote:
>
> [ Removed all Cc's. Added Masami and linux-trace-kernel mailing list ]
>
> On Thu, 16 Jul 2026 00:09:39 +0000
> Roman Gushchin <roman.gushchin@linux.dev> wrote:
>
> > Steven Rostedt <rostedt@goodmis.org> writes:
> >
> > > On Wed, 15 Jul 2026 21:08:47 +0000
> > > Roman Gushchin <roman.gushchin@linux.dev> wrote:
> > >
> > >> At this moment there are 36 mailing lists who opted in for delivering email
> > >> reviews over email (without counting linux-media@). In every single case
> > >> it was based on maintainers requests in all cases of a disagreement
> > >> between maintainers I took the conservative side.
> > >
> > > Oh, I think I missed this opt-in. Can you add replies to patches sent
> > > to linux-trace-kernel. Currently, I've been manually looking to see if
> > > a patch had a Sashiko review and sending the author a link to the
> > > review and told them to please reply to the comments.
> > >
> > > Note, I have not had anyone tell me the are against doing so.
> >
> > May I ask you to raise this question with other linux-trace-kernel
> > maintainers and cc me on the thread? Just to make sure everyone is
> > onboard.
>
> Masami,
>
> Are you OK with having Sashiko send an email on what it finds to the
> author and the mailing list?
Yeah, I hope that is good to make a discussion on ML about reviewed
result.
Mostly those are reasonable now, so maybe we need to decide a standard
format for "all-Ack".
Current problem is that the Sashiko does not look back the previous
discussion, so the commit or code comment should clearly state
"this code is intensionally do something."
A current issue is that, since Sashiko does not reflect the content
of past discussions, we must explicitly state within commits or code
comments, (where you have determined that a past review comment was
incorrect) that the code is intentionally doing this operation and why.
> >
> > Also, please, let me know if sashiko should reply to the author and
> > also if it should send an email if there are no issues found.
>
> I think if it finds no issues it should still send the email, as it
> adds to the confidence of the patch. It also lets us know that the
> email wasn't just dropped.
>
> Oh, I would think that "Failed to apply" results should *not* be sent.
Yes, the fact that there is no response from Sashiko implies that
the application failed.
Thank you,
--
Masami Hiramatsu (Google) <mhiramat@kernel.org>
^ permalink raw reply
* Re: [linus:master] [selftests/tracing] c5c413534d: kselftests-bpf.ftrace.ftracetest-ktap._instance_event_trigger_-_test_poll_wait_on_histogram.fail
From: Masami Hiramatsu @ 2026-07-21 0:38 UTC (permalink / raw)
To: Masami Hiramatsu
Cc: kernel test robot, Steven Rostedt, oe-lkp, lkp, linux-kernel,
Shuah Khan, linux-trace-kernel
In-Reply-To: <20260720232952.94b33d59eddb69cf6457ded4@kernel.org>
On Mon, 20 Jul 2026 23:29:52 +0900
Masami Hiramatsu (Google) <mhiramat@kernel.org> wrote:
> On Thu, 16 Jul 2026 22:30:11 +0800
> kernel test robot <oliver.sang@intel.com> wrote:
>
> >
> >
> >
> > Hello,
> >
> > in fact, we found below two test cases randomly failed upon c5c413534d, while
> > keep succeed on parent:
> >
> > =========================================================================================
> > tbox_group/testcase/rootfs/kconfig/compiler/group:
> > igk-rpl-d04/kselftests-bpf/debian-13-x86_64-20250902.cgz/x86_64-rhel-9.4-bpf/gcc-14/ftrace
> >
> > 22a78be4123dce81 c5c413534d40eb4c982f1794e62
> > ---------------- ---------------------------
> > fail:runs %reproduction fail:runs
> > | | |
> > :31 52% 16:32 kselftests-bpf.ftrace.ftracetest-ktap._instance_event_trigger_-_test_poll_wait_on_histogram.fail
> > :31 29% 9:32 kselftests-bpf.ftrace.ftracetest-ktap.event_trigger_-_test_poll_wait_on_histogram.fail
> >
> >
> >
> > kernel test robot noticed "kselftests-bpf.ftrace.ftracetest-ktap._instance_event_trigger_-_test_poll_wait_on_histogram.fail" on:
> >
> > commit: c5c413534d40eb4c982f1794e62813f926aba747 ("selftests/tracing: Have trigger-hist-poll.tc use sched_process_exit")
> > https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master
> >
> > [test failed on linux-next/master 49362394dad7df66c274c867a271394c10ca2bb8]
>
> Hmm, this seems like a scheduler (event) behavior is different from what
> the test expected.
>
> Steve, I think we should use another deterministic events.
I've sent a fix.
https://lore.kernel.org/all/178459323652.354837.10711230683477156733.stgit@devnote2/
But this drops 2 checkpoints. I think we need to use another deterministic
event, like trace_marker (but trace_marker does not support histogram trigger)
Thanks,
>
> Thank you,
>
> --
> Masami Hiramatsu (Google) <mhiramat@kernel.org>
--
Masami Hiramatsu (Google) <mhiramat@kernel.org>
^ permalink raw reply
* Re: [PATCH v4 16/17] selftests/verification: Rearrange the wwnr_printk test
From: Gabriele Monaco @ 2026-07-21 6:24 UTC (permalink / raw)
To: linux-kernel, linux-trace-kernel, Steven Rostedt, Shuah Khan,
linux-kselftest
Cc: Wen Yang, Nam Cao, Thomas Weissschuh, Tomas Glozar, John Kacur
In-Reply-To: <20260717154638.220789-17-gmonaco@redhat.com>
On Fri, 2026-07-17 at 17:46 +0200, Gabriele Monaco wrote:
> +# loads may flood the ringbuffer, wait for all pending printks
> +wait_dmesg_flush() {
> + local last_before last_after=$(dmesg | grep "rv:" | tail -n 1 ||
> true)
> + while [ "$last_before" != "$last_after" ]; do
> + last_before=$last_after
> + sleep .3
> + last_after=$(dmesg | grep "rv:" | tail -n 1 || true)
> + done
> }
>
> echo 1 > monitors/wwnr/enable
> echo printk > monitors/wwnr/reactors
>
> load
> +wait_dmesg_flush
On some systems this may trigger wwnr's reactions and run indefinitely, I'm
going to send a V5 moving this after turning monitoring/reacting of and adding a
timeout, just in case those switches don't work.
>
> echo 0 > monitoring_on
> ! load || false
> echo 1 > monitoring_on
>
> load
> +wait_dmesg_flush
>
> echo 0 > reacting_on
> ! load || false
^ permalink raw reply
* Re: [RFC PATCH 0/7] mm: defer address-space teardown of large exiting processes to a kthread
From: David Hildenbrand (Arm) @ 2026-07-21 10:19 UTC (permalink / raw)
To: Aditya Sharma
Cc: Andrew Morton, Lorenzo Stoakes, Liam R . Howlett, Vlastimil Babka,
Mike Rapoport, Suren Baghdasaryan, Michal Hocko, David Rientjes,
Shakeel Butt, Jonathan Corbet, Shuah Khan, Steven Rostedt,
Masami Hiramatsu, Mathieu Desnoyers, Kees Cook, Ingo Molnar,
Peter Zijlstra, Juri Lelli, Vincent Guittot, Dietmar Eggemann,
Ben Segall, Mel Gorman, Valentin Schneider, K Prateek Nayak,
linux-mm, linux-doc, linux-trace-kernel, linux-kernel, imbrenda
In-Reply-To: <19f80cb1ed8.13fd666243425.2308143427066829733@zohomail.in>
[...]
> says that the s390 case was addressed with KVM specific async destruction,
> but what you point to is QEMU's shadow process. I will correct that in v2.
CLONE_VM seems to work for the use cases you had in mind.
There would have to be a *pretty* convincing story why we would want more
complexity in the kernel to handle this.
Some smaller performance improvements are not worth it.
--
Cheers,
David
^ permalink raw reply
* [PATCH v3 00/18] tracing/remotes: Add printk, dump_on_panic and boot parameters
From: Vincent Donnefort @ 2026-07-21 10:35 UTC (permalink / raw)
To: rostedt, mhiramat, linux-trace-kernel
Cc: mathieu.desnoyers, kernel-team, linux-kernel, Vincent Donnefort
This series extends the recently introduced trace remotes
infrastructure, bringing useful features for developers:
* dump_on_panic: Dump the trace remote buffer on system panic.
* dmesg: Redirect remote events to dmesg.
* trace_remote=: Configure a trace_remote from the commandline.
* poll_ms: Modify the polling period.
It also brings a couple of optimisations:
* In-header compressed length support for small events.
* Single work thread for remote polling.
And some misc improvements:
* Use kstrtobool where possible
* Free resources on remote registration failure
v3:
* Make sure dmesg and panic handlers are using null-terminated
trace_seq.
* Make struct trace_remote more compact
* Fix dump_on_panic test unloaded case (Sashiko)
* Rebase on 7.2-rc4
v2 (https://lore.kernel.org/all/20260605163825.1762953-1-vdonnefort@google.com/):
Sashiko had a few spot-on comments on the v1 which forced me to heavily
re-arrange the series, so I thought it'd be alright to send a v2
already including those changes.
* dump_on_oops -> dump_on_panic
* printk -> dmesg
* Allow to configure poll_ms
* Make the ring_buffer_iter functions panic-friendly
* Gate tracefs .open on remote registration (Sashiko)
* Use irqsave for reader lock (Sashiko)
* Fix iter locking for TRI_DMESG (Sashiko)
* Yield in the dmesg work (Sashiko)
v1 (https://lore.kernel.org/all/20260602171146.2238998-1-vdonnefort@google.com/)
Vincent Donnefort (18):
tracing/remotes: Gate tracefs files opening on trace remote
registration
tracing/remotes: Release tracefs,eventfs on registration failure
tracing/remotes: Use kstrtobool for boolean tracefs files
tracing/remotes: Use a single per-remote polling work
tracing/simple_ring_buffer: Add support for compressed length
tracing/remotes: Add dmesg tracefs file
tracing/remotes: selftests: Add a test for the dmesg tracefs file
tracing/remotes: selftests: Prefix hypervisor folder
ring-buffer: Use irqsave for the reader lock in
ring_buffer_poll_remote
ring-buffer: Use panic-friendly locking in ring_buffer_iter interface
ring-buffer: Add ring_buffer_read_remote_meta_page()
ring-buffer: Add kerneldoc for ring_buffer_poll_remote
tracing/remotes: Add dump_on_panic tracefs file
tracing/remotes: selftests: Add a test for the dump_on_panic tracefs
file
tracing/remotes: Add poll_ms tracefs file
tracing/remotes: Add trace_remote cmdline options
Documentation: tracing/remotes: Add detailed tracefs layout
Documentation/kernel-parameters: Add trace_remote
.../admin-guide/kernel-parameters.txt | 19 +
Documentation/trace/remotes.rst | 66 +-
include/linux/ring_buffer.h | 1 +
kernel/trace/ring_buffer.c | 88 +-
kernel/trace/simple_ring_buffer.c | 25 +-
kernel/trace/trace_remote.c | 825 +++++++++++++++---
.../buffer_size.tc | 0
.../test.d/remotes/00hypervisor/dmesg.tc | 11 +
.../remotes/00hypervisor/dump_on_panic.tc | 11 +
.../{hypervisor => 00hypervisor}/hotplug.tc | 0
.../{hypervisor => 00hypervisor}/reset.tc | 0
.../{hypervisor => 00hypervisor}/trace.tc | 0
.../trace_pipe.tc | 0
.../{hypervisor => 00hypervisor}/unloading.tc | 0
.../selftests/ftrace/test.d/remotes/dmesg.tc | 72 ++
.../ftrace/test.d/remotes/dump_on_panic.tc | 51 ++
.../selftests/ftrace/test.d/remotes/functions | 2 +
17 files changed, 1031 insertions(+), 140 deletions(-)
rename tools/testing/selftests/ftrace/test.d/remotes/{hypervisor => 00hypervisor}/buffer_size.tc (100%)
create mode 100644 tools/testing/selftests/ftrace/test.d/remotes/00hypervisor/dmesg.tc
create mode 100644 tools/testing/selftests/ftrace/test.d/remotes/00hypervisor/dump_on_panic.tc
rename tools/testing/selftests/ftrace/test.d/remotes/{hypervisor => 00hypervisor}/hotplug.tc (100%)
rename tools/testing/selftests/ftrace/test.d/remotes/{hypervisor => 00hypervisor}/reset.tc (100%)
rename tools/testing/selftests/ftrace/test.d/remotes/{hypervisor => 00hypervisor}/trace.tc (100%)
rename tools/testing/selftests/ftrace/test.d/remotes/{hypervisor => 00hypervisor}/trace_pipe.tc (100%)
rename tools/testing/selftests/ftrace/test.d/remotes/{hypervisor => 00hypervisor}/unloading.tc (100%)
create mode 100644 tools/testing/selftests/ftrace/test.d/remotes/dmesg.tc
create mode 100644 tools/testing/selftests/ftrace/test.d/remotes/dump_on_panic.tc
base-commit: 1590cf0329716306e948a8fc29f1d3ee87d3989f
--
2.55.0.229.g6434b31f56-goog
^ permalink raw reply
* [PATCH v3 01/18] tracing/remotes: Gate tracefs files opening on trace remote registration
From: Vincent Donnefort @ 2026-07-21 10:35 UTC (permalink / raw)
To: rostedt, mhiramat, linux-trace-kernel
Cc: mathieu.desnoyers, kernel-team, linux-kernel, Vincent Donnefort
In-Reply-To: <20260721103603.342266-1-vdonnefort@google.com>
Currently, if a remote fails to register, its resources will leak and
will not be properly tear-downed.
To prevent a user accessing a remote tracefs that is about to be
destroyed, keep track of the registered remotes in a global list,
similarly to trace instances. Gate the tracefs to open function based on
the presence of the remote in that list.
Signed-off-by: Vincent Donnefort <vdonnefort@google.com>
diff --git a/kernel/trace/trace_remote.c b/kernel/trace/trace_remote.c
index 0f6ef5c36d84..522f3e67f17b 100644
--- a/kernel/trace/trace_remote.c
+++ b/kernel/trace/trace_remote.c
@@ -39,6 +39,7 @@ struct trace_remote_iterator {
};
struct trace_remote {
+ struct list_head node;
struct trace_remote_callbacks *cbs;
void *priv;
struct trace_buffer *trace_buffer;
@@ -57,6 +58,9 @@ struct trace_remote {
bool tracing_on;
};
+static DEFINE_MUTEX(trace_remotes_lock);
+static LIST_HEAD(trace_remotes);
+
static bool trace_remote_loaded(struct trace_remote *remote)
{
return !!remote->trace_buffer;
@@ -170,6 +174,60 @@ static void trace_remote_reset(struct trace_remote *remote, int cpu)
trace_remote_try_unload(remote);
}
+static int trace_remote_tracefs_open(struct inode *inode, struct file *filp)
+{
+ void *i_private = inode->i_private;
+ struct trace_remote *r;
+
+ if (!i_private)
+ return -ENODEV;
+
+ guard(mutex)(&trace_remotes_lock);
+
+ /* i_private is either a struct trace_remote or a struct remote_event */
+ list_for_each_entry(r, &trace_remotes, node) {
+ if (r == i_private)
+ return 0;
+ if (!r->events)
+ continue;
+ if (i_private >= (void *)r->events &&
+ i_private < (void *)(r->events + r->nr_events))
+ return 0;
+ }
+
+ return -ENODEV;
+}
+
+#define DEFINE_TRACE_REMOTE_ATTRIBUTE_FUNCS(__name) \
+static int __name ## _open(struct inode *inode, struct file *file) \
+{ \
+ int ret = trace_remote_tracefs_open(inode, file); \
+ \
+ if (ret) \
+ return ret; \
+ \
+ return single_open(file, __name ## _show, inode->i_private); \
+}
+
+#define DEFINE_TRACE_REMOTE_ATTRIBUTE(__name) \
+DEFINE_TRACE_REMOTE_ATTRIBUTE_FUNCS(__name) \
+static const struct file_operations __name ## _fops = { \
+ .open = __name ## _open, \
+ .read = seq_read, \
+ .write = __name ## _write, \
+ .llseek = seq_lseek, \
+ .release = single_release, \
+}
+
+#define DEFINE_TRACE_REMOTE_SHOW_ATTRIBUTE(__name) \
+DEFINE_TRACE_REMOTE_ATTRIBUTE_FUNCS(__name) \
+static const struct file_operations __name ## _fops = { \
+ .open = __name ## _open, \
+ .read = seq_read, \
+ .llseek = seq_lseek, \
+ .release = single_release, \
+}
+
static ssize_t
tracing_on_write(struct file *filp, const char __user *ubuf, size_t cnt, loff_t *ppos)
{
@@ -198,7 +256,7 @@ static int tracing_on_show(struct seq_file *s, void *unused)
return 0;
}
-DEFINE_SHOW_STORE_ATTRIBUTE(tracing_on);
+DEFINE_TRACE_REMOTE_ATTRIBUTE(tracing_on);
static ssize_t buffer_size_kb_write(struct file *filp, const char __user *ubuf, size_t cnt,
loff_t *ppos)
@@ -235,7 +293,7 @@ static int buffer_size_kb_show(struct seq_file *s, void *unused)
return 0;
}
-DEFINE_SHOW_STORE_ATTRIBUTE(buffer_size_kb);
+DEFINE_TRACE_REMOTE_ATTRIBUTE(buffer_size_kb);
static int trace_remote_get(struct trace_remote *remote, int cpu)
{
@@ -593,6 +651,11 @@ static int trace_pipe_open(struct inode *inode, struct file *filp)
struct trace_remote *remote = inode->i_private;
struct trace_remote_iterator *iter;
int cpu = tracing_get_cpu(inode);
+ int ret;
+
+ ret = trace_remote_tracefs_open(inode, filp);
+ if (ret)
+ return ret;
guard(mutex)(&remote->lock);
@@ -734,22 +797,26 @@ static int trace_open(struct inode *inode, struct file *filp)
int cpu = tracing_get_cpu(inode);
int ret;
+ ret = trace_remote_tracefs_open(inode, filp);
+ if (ret)
+ return ret;
+
if (!(filp->f_mode & FMODE_READ))
return 0;
+ ret = seq_open(filp, &trace_sops);
+ if (ret)
+ return ret;
+
guard(mutex)(&remote->lock);
iter = trace_remote_iter(remote, cpu, TRI_NONCONSUMING);
- if (IS_ERR(iter))
+ if (IS_ERR(iter)) {
+ seq_release(inode, filp);
return PTR_ERR(iter);
-
- ret = seq_open(filp, &trace_sops);
- if (ret) {
- trace_remote_iter_free(iter);
- return ret;
}
- ((struct seq_file *)filp->private_data)->private = (void *)iter;
+ ((struct seq_file *)filp->private_data)->private = iter;
return 0;
}
@@ -932,8 +999,12 @@ int trace_remote_register(const char *name, struct trace_remote_callbacks *cbs,
}
ret = cbs->init ? cbs->init(remote->dentry, priv) : 0;
- if (ret)
+ if (ret) {
pr_err("Init failed for trace remote '%s' (%d)\n", name, ret);
+ } else {
+ guard(mutex)(&trace_remotes_lock);
+ list_add(&remote->node, &trace_remotes);
+ }
return ret;
}
@@ -1070,7 +1141,7 @@ static ssize_t remote_event_enable_write(struct file *filp, const char __user *u
return count;
}
-DEFINE_SHOW_STORE_ATTRIBUTE(remote_event_enable);
+DEFINE_TRACE_REMOTE_ATTRIBUTE(remote_event_enable);
static int remote_event_id_show(struct seq_file *s, void *unused)
{
@@ -1080,7 +1151,7 @@ static int remote_event_id_show(struct seq_file *s, void *unused)
return 0;
}
-DEFINE_SHOW_ATTRIBUTE(remote_event_id);
+DEFINE_TRACE_REMOTE_SHOW_ATTRIBUTE(remote_event_id);
static int remote_event_format_show(struct seq_file *s, void *unused)
{
@@ -1109,7 +1180,7 @@ static int remote_event_format_show(struct seq_file *s, void *unused)
return 0;
}
-DEFINE_SHOW_ATTRIBUTE(remote_event_format);
+DEFINE_TRACE_REMOTE_SHOW_ATTRIBUTE(remote_event_format);
static int remote_event_callback(const char *name, umode_t *mode, void **data,
const struct file_operations **fops)
@@ -1184,6 +1255,7 @@ static ssize_t remote_events_dir_enable_read(struct file *filp, char __user *ubu
}
static const struct file_operations remote_events_dir_enable_fops = {
+ .open = trace_remote_tracefs_open,
.write = remote_events_dir_enable_write,
.read = remote_events_dir_enable_read,
};
@@ -1208,6 +1280,7 @@ remote_events_dir_header_page_read(struct file *filp, char __user *ubuf, size_t
}
static const struct file_operations remote_events_dir_header_page_fops = {
+ .open = trace_remote_tracefs_open,
.read = remote_events_dir_header_page_read,
};
@@ -1231,6 +1304,7 @@ remote_events_dir_header_event_read(struct file *filp, char __user *ubuf, size_t
}
static const struct file_operations remote_events_dir_header_event_fops = {
+ .open = trace_remote_tracefs_open,
.read = remote_events_dir_header_event_read,
};
--
2.55.0.229.g6434b31f56-goog
^ permalink raw reply related
* [PATCH v3 02/18] tracing/remotes: Release tracefs,eventfs on registration failure
From: Vincent Donnefort @ 2026-07-21 10:35 UTC (permalink / raw)
To: rostedt, mhiramat, linux-trace-kernel
Cc: mathieu.desnoyers, kernel-team, linux-kernel, Vincent Donnefort
In-Reply-To: <20260721103603.342266-1-vdonnefort@google.com>
In trace_remote_register(), if registration of events or the init
callback fails, the created tracefs and eventfs directories are leaked.
Release the entire eventfs and tracefs hierarchy on trace_remote
registration failure.
Signed-off-by: Vincent Donnefort <vdonnefort@google.com>
diff --git a/kernel/trace/trace_remote.c b/kernel/trace/trace_remote.c
index 522f3e67f17b..2bc3afe1204c 100644
--- a/kernel/trace/trace_remote.c
+++ b/kernel/trace/trace_remote.c
@@ -45,7 +45,8 @@ struct trace_remote {
struct trace_buffer *trace_buffer;
struct trace_buffer_desc *trace_buffer_desc;
struct dentry *dentry;
- struct eventfs_inode *eventfs;
+ struct eventfs_inode *eventfs_root;
+ struct eventfs_inode *eventfs_subdir;
struct remote_event *events;
unsigned long nr_events;
unsigned long trace_buffer_size;
@@ -60,6 +61,7 @@ struct trace_remote {
static DEFINE_MUTEX(trace_remotes_lock);
static LIST_HEAD(trace_remotes);
+static struct dentry *trace_remotes_root;
static bool trace_remote_loaded(struct trace_remote *remote)
{
@@ -865,23 +867,21 @@ static const struct file_operations trace_fops = {
static int trace_remote_init_tracefs(const char *name, struct trace_remote *remote)
{
struct dentry *remote_d, *percpu_d, *d;
- static struct dentry *root;
- static DEFINE_MUTEX(lock);
bool root_inited = false;
int cpu;
- guard(mutex)(&lock);
+ lockdep_assert_held(&trace_remotes_lock);
- if (!root) {
- root = tracefs_create_dir(TRACEFS_DIR, NULL);
- if (!root) {
+ if (!trace_remotes_root) {
+ trace_remotes_root = tracefs_create_dir(TRACEFS_DIR, NULL);
+ if (!trace_remotes_root) {
pr_err("Failed to create tracefs dir "TRACEFS_DIR"\n");
return -ENOMEM;
}
root_inited = true;
}
- remote_d = tracefs_create_dir(name, root);
+ remote_d = tracefs_create_dir(name, trace_remotes_root);
if (!remote_d) {
pr_err("Failed to create tracefs dir "TRACEFS_DIR"%s/\n", name);
goto err;
@@ -939,8 +939,8 @@ static int trace_remote_init_tracefs(const char *name, struct trace_remote *remo
err:
if (root_inited) {
- tracefs_remove(root);
- root = NULL;
+ tracefs_remove(trace_remotes_root);
+ trace_remotes_root = NULL;
} else {
tracefs_remove(remote_d);
}
@@ -948,8 +948,26 @@ static int trace_remote_init_tracefs(const char *name, struct trace_remote *remo
return -ENOMEM;
}
+static void trace_remote_remove_tracefs(struct trace_remote *remote)
+{
+ lockdep_assert_held(&trace_remotes_lock);
+
+ if (!remote->dentry)
+ return;
+
+ tracefs_remove(remote->dentry);
+ remote->dentry = NULL;
+
+ if (!list_empty(&trace_remotes))
+ return;
+
+ tracefs_remove(trace_remotes_root);
+ trace_remotes_root = NULL;
+}
+
static int trace_remote_register_events(const char *remote_name, struct trace_remote *remote,
struct remote_event *events, size_t nr_events);
+static void trace_remote_unregister_events(struct trace_remote *remote);
/**
* trace_remote_register() - Register a Tracefs remote
@@ -972,10 +990,9 @@ static int trace_remote_register_events(const char *remote_name, struct trace_re
int trace_remote_register(const char *name, struct trace_remote_callbacks *cbs, void *priv,
struct remote_event *events, size_t nr_events)
{
- struct trace_remote *remote;
+ struct trace_remote *remote __free(kfree) = kzalloc_obj(*remote);
int ret;
- remote = kzalloc_obj(*remote);
if (!remote)
return -ENOMEM;
@@ -986,13 +1003,15 @@ int trace_remote_register(const char *name, struct trace_remote_callbacks *cbs,
mutex_init(&remote->lock);
init_rwsem(&remote->reader_lock);
- if (trace_remote_init_tracefs(name, remote)) {
- kfree(remote);
- return -ENOMEM;
- }
+ guard(mutex)(&trace_remotes_lock);
+
+ ret = trace_remote_init_tracefs(name, remote);
+ if (ret)
+ return ret;
ret = trace_remote_register_events(name, remote, events, nr_events);
if (ret) {
+ trace_remote_remove_tracefs(remote);
pr_err("Failed to register events for trace remote '%s' (%d)\n",
name, ret);
return ret;
@@ -1000,13 +1019,16 @@ int trace_remote_register(const char *name, struct trace_remote_callbacks *cbs,
ret = cbs->init ? cbs->init(remote->dentry, priv) : 0;
if (ret) {
+ trace_remote_unregister_events(remote);
+ trace_remote_remove_tracefs(remote);
pr_err("Init failed for trace remote '%s' (%d)\n", name, ret);
- } else {
- guard(mutex)(&trace_remotes_lock);
- list_add(&remote->node, &trace_remotes);
+ return ret;
}
- return ret;
+ list_add(&remote->node, &trace_remotes);
+ retain_and_null_ptr(remote);
+
+ return 0;
}
EXPORT_SYMBOL_GPL(trace_remote_register);
@@ -1335,7 +1357,6 @@ static int remote_events_dir_callback(const char *name, umode_t *mode, void **da
static int trace_remote_init_eventfs(const char *remote_name, struct trace_remote *remote,
struct remote_event *evt)
{
- struct eventfs_inode *eventfs = remote->eventfs;
static struct eventfs_entry dir_entries[] = {
{
.name = "enable",
@@ -1360,35 +1381,37 @@ static int trace_remote_init_eventfs(const char *remote_name, struct trace_remot
.callback = remote_event_callback,
}
};
- bool eventfs_create = false;
+ struct eventfs_inode *eventfs_root, *eventfs_subdir, *e;
- if (!eventfs) {
- eventfs = eventfs_create_events_dir("events", remote->dentry, dir_entries,
- ARRAY_SIZE(dir_entries), remote);
- if (IS_ERR(eventfs))
- return PTR_ERR(eventfs);
+ eventfs_root = remote->eventfs_root;
+ eventfs_subdir = remote->eventfs_subdir;
+ if (!eventfs_root) {
+ eventfs_root = eventfs_create_events_dir("events", remote->dentry, dir_entries,
+ ARRAY_SIZE(dir_entries), remote);
+ if (IS_ERR(eventfs_root))
+ return PTR_ERR(eventfs_root);
/*
* Create similar hierarchy as local events even if a single system is supported at
* the moment
*/
- eventfs = eventfs_create_dir(remote_name, eventfs, NULL, 0, NULL);
- if (IS_ERR(eventfs))
- return PTR_ERR(eventfs);
-
- remote->eventfs = eventfs;
- eventfs_create = true;
- }
-
- eventfs = eventfs_create_dir(evt->name, eventfs, entries, ARRAY_SIZE(entries), evt);
- if (IS_ERR(eventfs)) {
- if (eventfs_create) {
- eventfs_remove_events_dir(remote->eventfs);
- remote->eventfs = NULL;
+ eventfs_subdir = eventfs_create_dir(remote_name, eventfs_root, NULL, 0, NULL);
+ if (IS_ERR(eventfs_subdir)) {
+ eventfs_remove_events_dir(eventfs_root);
+ return PTR_ERR(eventfs_subdir);
}
- return PTR_ERR(eventfs);
}
+ e = eventfs_create_dir(evt->name, eventfs_subdir, entries, ARRAY_SIZE(entries), evt);
+ if (IS_ERR(e)) {
+ if (!remote->eventfs_root)
+ eventfs_remove_events_dir(eventfs_root);
+ return PTR_ERR(e);
+ }
+
+ remote->eventfs_root = eventfs_root;
+ remote->eventfs_subdir = eventfs_subdir;
+
return 0;
}
@@ -1403,11 +1426,11 @@ static int trace_remote_attach_events(struct trace_remote *remote, struct remote
if (evt->remote)
return -EEXIST;
- evt->remote = remote;
-
/* We need events to be sorted for efficient lookup */
if (i && evt->id <= events[i - 1].id)
return -EINVAL;
+
+ evt->remote = remote;
}
remote->events = events;
@@ -1416,14 +1439,33 @@ static int trace_remote_attach_events(struct trace_remote *remote, struct remote
return 0;
}
+static void trace_remote_detach_events(struct trace_remote *remote, struct remote_event *events,
+ size_t nr_events)
+{
+ int i;
+
+ for (i = 0; i < nr_events; i++) {
+ struct remote_event *evt = &events[i];
+
+ if (evt->remote == remote)
+ evt->remote = NULL;
+ }
+
+ remote->events = NULL;
+ remote->nr_events = 0;
+}
+
static int trace_remote_register_events(const char *remote_name, struct trace_remote *remote,
struct remote_event *events, size_t nr_events)
{
int i, ret;
ret = trace_remote_attach_events(remote, events, nr_events);
- if (ret)
+ if (ret) {
+ /* It is safe to call detach on a half-registered array */
+ trace_remote_detach_events(remote, events, nr_events);
return ret;
+ }
for (i = 0; i < nr_events; i++) {
struct remote_event *evt = &events[i];
@@ -1437,6 +1479,13 @@ static int trace_remote_register_events(const char *remote_name, struct trace_re
return 0;
}
+static void trace_remote_unregister_events(struct trace_remote *remote)
+{
+ trace_remote_detach_events(remote, remote->events, remote->nr_events);
+ if (remote->eventfs_root)
+ eventfs_remove_events_dir(remote->eventfs_root);
+}
+
static int __cmp_events(const void *key, const void *data)
{
const struct remote_event *evt = data;
--
2.55.0.229.g6434b31f56-goog
^ permalink raw reply related
* [PATCH v3 03/18] tracing/remotes: Use kstrtobool for boolean tracefs files
From: Vincent Donnefort @ 2026-07-21 10:35 UTC (permalink / raw)
To: rostedt, mhiramat, linux-trace-kernel
Cc: mathieu.desnoyers, kernel-team, linux-kernel, Vincent Donnefort
In-Reply-To: <20260721103603.342266-1-vdonnefort@google.com>
Use kstrtobool in trace_remote.c where possible. This is more user-friendly
as it allows a better variety of input strings.
Signed-off-by: Vincent Donnefort <vdonnefort@google.com>
diff --git a/kernel/trace/trace_remote.c b/kernel/trace/trace_remote.c
index 2bc3afe1204c..94a52445cff9 100644
--- a/kernel/trace/trace_remote.c
+++ b/kernel/trace/trace_remote.c
@@ -235,10 +235,10 @@ tracing_on_write(struct file *filp, const char __user *ubuf, size_t cnt, loff_t
{
struct seq_file *seq = filp->private_data;
struct trace_remote *remote = seq->private;
- unsigned long val;
+ bool val;
int ret;
- ret = kstrtoul_from_user(ubuf, cnt, 10, &val);
+ ret = kstrtobool_from_user(ubuf, cnt, &val);
if (ret)
return ret;
@@ -1148,10 +1148,10 @@ static ssize_t remote_event_enable_write(struct file *filp, const char __user *u
struct seq_file *seq = filp->private_data;
struct remote_event *evt = seq->private;
struct trace_remote *remote = evt->remote;
- u8 enable;
+ bool enable;
int ret;
- ret = kstrtou8_from_user(ubuf, count, 10, &enable);
+ ret = kstrtobool_from_user(ubuf, count, &enable);
if (ret)
return ret;
@@ -1232,10 +1232,10 @@ static ssize_t remote_events_dir_enable_write(struct file *filp, const char __us
size_t count, loff_t *ppos)
{
struct trace_remote *remote = file_inode(filp)->i_private;
+ bool enable;
int i, ret;
- u8 enable;
- ret = kstrtou8_from_user(ubuf, count, 10, &enable);
+ ret = kstrtobool_from_user(ubuf, count, &enable);
if (ret)
return ret;
--
2.55.0.229.g6434b31f56-goog
^ permalink raw reply related
* [PATCH v3 04/18] tracing/remotes: Use a single per-remote polling work
From: Vincent Donnefort @ 2026-07-21 10:35 UTC (permalink / raw)
To: rostedt, mhiramat, linux-trace-kernel
Cc: mathieu.desnoyers, kernel-team, linux-kernel, Vincent Donnefort
In-Reply-To: <20260721103603.342266-1-vdonnefort@google.com>
Having a per-iterator polling work is wasteful when logging several
trace_remote per_cpu/trace_pipe files in parallel. This result in one
work running per-CPU, where only one would suffice.
Transition to a single per-remote polling work, scheduled on the first
consumer creation and stopped when the last consuming iterator is freed.
This blanket polls all CPUs, regardless of which ones are actually being
read. This is acceptable because the poll consists of reading the
meta-page, which is a fast operation. Also, it is more common to log all
CPUs in the system than only one, so this use-case should be favoured.
Signed-off-by: Vincent Donnefort <vdonnefort@google.com>
diff --git a/kernel/trace/trace_remote.c b/kernel/trace/trace_remote.c
index 94a52445cff9..3af622c94784 100644
--- a/kernel/trace/trace_remote.c
+++ b/kernel/trace/trace_remote.c
@@ -26,7 +26,6 @@ enum tri_type {
struct trace_remote_iterator {
struct trace_remote *remote;
struct trace_seq seq;
- struct delayed_work poll_work;
unsigned long lost_events;
u64 ts;
struct ring_buffer_iter *rb_iter;
@@ -55,8 +54,10 @@ struct trace_remote {
struct rw_semaphore reader_lock;
struct rw_semaphore *pcpu_reader_locks;
unsigned int nr_readers;
+ unsigned int poll_cnt;
unsigned int poll_ms;
bool tracing_on;
+ struct delayed_work poll_work;
};
static DEFINE_MUTEX(trace_remotes_lock);
@@ -350,17 +351,6 @@ static bool trace_remote_has_cpu(struct trace_remote *remote, int cpu)
return ring_buffer_poll_remote(remote->trace_buffer, cpu) == 0;
}
-static void __poll_remote(struct work_struct *work)
-{
- struct delayed_work *dwork = to_delayed_work(work);
- struct trace_remote_iterator *iter;
-
- iter = container_of(dwork, struct trace_remote_iterator, poll_work);
- ring_buffer_poll_remote(iter->remote->trace_buffer, iter->cpu);
- schedule_delayed_work((struct delayed_work *)work,
- msecs_to_jiffies(iter->remote->poll_ms));
-}
-
static void __free_ring_buffer_iter(struct trace_remote_iterator *iter, int cpu)
{
if (cpu != RING_BUFFER_ALL_CPUS) {
@@ -404,6 +394,36 @@ static int __alloc_ring_buffer_iter(struct trace_remote_iterator *iter, int cpu)
return 0;
}
+static void trace_remote_do_poll(struct trace_remote *remote)
+{
+ ring_buffer_poll_remote(remote->trace_buffer, RING_BUFFER_ALL_CPUS);
+ schedule_delayed_work(&remote->poll_work, msecs_to_jiffies(remote->poll_ms));
+}
+
+static void __poll_remote(struct work_struct *work)
+{
+ struct delayed_work *dwork = to_delayed_work(work);
+
+ trace_remote_do_poll(container_of(dwork, struct trace_remote, poll_work));
+}
+
+static void trace_remote_inc_poll(struct trace_remote *remote)
+{
+ /* poll_cnt <= nr_readers, inherits its overflow protection */
+ if (!remote->poll_cnt++)
+ trace_remote_do_poll(remote);
+}
+
+static void trace_remote_dec_poll(struct trace_remote *remote)
+{
+ if (WARN_ON_ONCE(!remote->poll_cnt))
+ return;
+
+ remote->poll_cnt--;
+ if (!remote->poll_cnt)
+ cancel_delayed_work_sync(&remote->poll_work);
+}
+
static struct trace_remote_iterator
*trace_remote_iter(struct trace_remote *remote, int cpu, enum tri_type type)
{
@@ -433,9 +453,7 @@ static struct trace_remote_iterator
switch (type) {
case TRI_CONSUMING:
- ring_buffer_poll_remote(remote->trace_buffer, cpu);
- INIT_DELAYED_WORK(&iter->poll_work, __poll_remote);
- schedule_delayed_work(&iter->poll_work, msecs_to_jiffies(remote->poll_ms));
+ trace_remote_inc_poll(remote);
break;
case TRI_NONCONSUMING:
ret = __alloc_ring_buffer_iter(iter, cpu);
@@ -469,7 +487,7 @@ static void trace_remote_iter_free(struct trace_remote_iterator *iter)
switch (iter->type) {
case TRI_CONSUMING:
- cancel_delayed_work_sync(&iter->poll_work);
+ trace_remote_dec_poll(remote);
break;
case TRI_NONCONSUMING:
__free_ring_buffer_iter(iter, iter->cpu);
@@ -1002,6 +1020,7 @@ int trace_remote_register(const char *name, struct trace_remote_callbacks *cbs,
remote->poll_ms = 100;
mutex_init(&remote->lock);
init_rwsem(&remote->reader_lock);
+ INIT_DELAYED_WORK(&remote->poll_work, __poll_remote);
guard(mutex)(&trace_remotes_lock);
--
2.55.0.229.g6434b31f56-goog
^ permalink raw reply related
* [PATCH v3 05/18] tracing/simple_ring_buffer: Add support for compressed length
From: Vincent Donnefort @ 2026-07-21 10:35 UTC (permalink / raw)
To: rostedt, mhiramat, linux-trace-kernel
Cc: mathieu.desnoyers, kernel-team, linux-kernel, Vincent Donnefort
In-Reply-To: <20260721103603.342266-1-vdonnefort@google.com>
The array length is the total size in bytes of the data for the current
event. It is possible to compress this value into the event header type,
which has 28 unused types, saving 32 bits for sufficiently small events.
The compressed length is expressed as a multiple of the ring-buffer
alignment, 4-bytes by default. Enforces this alignment.
Signed-off-by: Vincent Donnefort <vdonnefort@google.com>
diff --git a/kernel/trace/simple_ring_buffer.c b/kernel/trace/simple_ring_buffer.c
index f4642f5adda3..1a97d17cca24 100644
--- a/kernel/trace/simple_ring_buffer.c
+++ b/kernel/trace/simple_ring_buffer.c
@@ -207,7 +207,15 @@ static unsigned long rb_event_size(unsigned long length)
{
struct ring_buffer_event *event;
- return length + RB_EVNT_HDR_SIZE + sizeof(event->array[0]);
+ if (!length)
+ length++;
+
+ length = ALIGN(length, RB_ALIGNMENT);
+
+ if (length > RB_MAX_SMALL_DATA)
+ length += sizeof(event->array[0]);
+
+ return length + RB_EVNT_HDR_SIZE;
}
static struct ring_buffer_event *
@@ -223,12 +231,15 @@ rb_event_add_ts_extend(struct ring_buffer_event *event, u64 delta)
static struct ring_buffer_event *
simple_rb_reserve_next(struct simple_rb_per_cpu *cpu_buffer, unsigned long length, u64 timestamp)
{
- unsigned long ts_ext_size = 0, event_size = rb_event_size(length);
struct simple_buffer_page *tail = cpu_buffer->tail_page;
+ unsigned long event_size, array_size, ts_ext_size = 0;
struct ring_buffer_event *event;
u32 write, prev_write;
u64 time_delta;
+ event_size = rb_event_size(length);
+ array_size = event_size - RB_EVNT_HDR_SIZE;
+
time_delta = timestamp - cpu_buffer->write_stamp;
if (test_time_stamp(time_delta))
@@ -259,9 +270,13 @@ simple_rb_reserve_next(struct simple_rb_per_cpu *cpu_buffer, unsigned long lengt
time_delta = 0;
}
- event->type_len = 0;
+ if (length > RB_MAX_SMALL_DATA) {
+ event->type_len = 0;
+ event->array[0] = array_size;
+ } else {
+ event->type_len = DIV_ROUND_UP(array_size, RB_ALIGNMENT);
+ }
event->time_delta = time_delta;
- event->array[0] = event_size - RB_EVNT_HDR_SIZE;
return event;
}
@@ -284,7 +299,7 @@ void *simple_ring_buffer_reserve(struct simple_rb_per_cpu *cpu_buffer, unsigned
rb_event = simple_rb_reserve_next(cpu_buffer, length, timestamp);
- return &rb_event->array[1];
+ return rb_event->type_len ? &rb_event->array[0] : &rb_event->array[1];
}
EXPORT_SYMBOL_GPL(simple_ring_buffer_reserve);
--
2.55.0.229.g6434b31f56-goog
^ permalink raw reply related
* [PATCH v3 06/18] tracing/remotes: Add dmesg tracefs file
From: Vincent Donnefort @ 2026-07-21 10:35 UTC (permalink / raw)
To: rostedt, mhiramat, linux-trace-kernel
Cc: mathieu.desnoyers, kernel-team, linux-kernel, Vincent Donnefort
In-Reply-To: <20260721103603.342266-1-vdonnefort@google.com>
When enabled, the dmesg tracefs file enables the redirection of all
events to dmesg. This is similar to tp_printk.
Signed-off-by: Vincent Donnefort <vdonnefort@google.com>
diff --git a/kernel/trace/trace_remote.c b/kernel/trace/trace_remote.c
index 3af622c94784..c54e1c65d163 100644
--- a/kernel/trace/trace_remote.c
+++ b/kernel/trace/trace_remote.c
@@ -21,6 +21,7 @@
enum tri_type {
TRI_CONSUMING,
TRI_NONCONSUMING,
+ TRI_DMESG,
};
struct trace_remote_iterator {
@@ -43,6 +44,7 @@ struct trace_remote {
void *priv;
struct trace_buffer *trace_buffer;
struct trace_buffer_desc *trace_buffer_desc;
+ struct trace_remote_iterator *dmesg;
struct dentry *dentry;
struct eventfs_inode *eventfs_root;
struct eventfs_inode *eventfs_subdir;
@@ -394,10 +396,15 @@ static int __alloc_ring_buffer_iter(struct trace_remote_iterator *iter, int cpu)
return 0;
}
+static bool trace_remote_do_dmesg(struct trace_remote *remote);
+
static void trace_remote_do_poll(struct trace_remote *remote)
{
+ bool yield;
+
ring_buffer_poll_remote(remote->trace_buffer, RING_BUFFER_ALL_CPUS);
- schedule_delayed_work(&remote->poll_work, msecs_to_jiffies(remote->poll_ms));
+ yield = trace_remote_do_dmesg(remote);
+ schedule_delayed_work(&remote->poll_work, yield ? 0 : msecs_to_jiffies(remote->poll_ms));
}
static void __poll_remote(struct work_struct *work)
@@ -452,6 +459,14 @@ static struct trace_remote_iterator
trace_seq_init(&iter->seq);
switch (type) {
+ case TRI_DMESG:
+ /* only one printk iter allowed */
+ if (WARN_ON_ONCE(remote->dmesg)) {
+ ret = -EBUSY;
+ break;
+ }
+ smp_store_release(&remote->dmesg, iter);
+ fallthrough;
case TRI_CONSUMING:
trace_remote_inc_poll(remote);
break;
@@ -486,6 +501,11 @@ static void trace_remote_iter_free(struct trace_remote_iterator *iter)
lockdep_assert_held(&remote->lock);
switch (iter->type) {
+ case TRI_DMESG:
+ WARN_ON_ONCE(remote->dmesg != iter);
+ smp_store_release(&remote->dmesg, NULL);
+ flush_delayed_work(&remote->poll_work);
+ fallthrough;
case TRI_CONSUMING:
trace_remote_dec_poll(remote);
break;
@@ -498,13 +518,24 @@ static void trace_remote_iter_free(struct trace_remote_iterator *iter)
trace_remote_put(remote);
}
+static bool trace_remote_iter_is_consuming(struct trace_remote_iterator *iter)
+{
+ switch (iter->type) {
+ case TRI_CONSUMING:
+ case TRI_DMESG:
+ return true;
+ default:
+ return false;
+ }
+}
+
static void trace_remote_iter_read_start(struct trace_remote_iterator *iter)
{
struct trace_remote *remote = iter->remote;
int cpu = iter->cpu;
/* Acquire global reader lock */
- if (cpu == RING_BUFFER_ALL_CPUS && iter->type == TRI_CONSUMING)
+ if (cpu == RING_BUFFER_ALL_CPUS && trace_remote_iter_is_consuming(iter))
down_write(&remote->reader_lock);
else
down_read(&remote->reader_lock);
@@ -521,7 +552,7 @@ static void trace_remote_iter_read_start(struct trace_remote_iterator *iter)
if (WARN_ON_ONCE(!remote->pcpu_reader_locks))
return;
- if (iter->type == TRI_CONSUMING)
+ if (trace_remote_iter_is_consuming(iter))
down_write(&remote->pcpu_reader_locks[cpu]);
else
down_read(&remote->pcpu_reader_locks[cpu]);
@@ -538,14 +569,14 @@ static void trace_remote_iter_read_finished(struct trace_remote_iterator *iter)
* No need for the remote lock here, iter holds a reference on
* remote->nr_readers
*/
- if (iter->type == TRI_CONSUMING)
+ if (trace_remote_iter_is_consuming(iter))
up_write(&remote->pcpu_reader_locks[cpu]);
else
up_read(&remote->pcpu_reader_locks[cpu]);
}
/* Release global reader lock */
- if (cpu == RING_BUFFER_ALL_CPUS && iter->type == TRI_CONSUMING)
+ if (cpu == RING_BUFFER_ALL_CPUS && trace_remote_iter_is_consuming(iter))
up_write(&remote->reader_lock);
else
up_read(&remote->reader_lock);
@@ -562,10 +593,9 @@ __peek_event(struct trace_remote_iterator *iter, int cpu, u64 *ts, unsigned long
struct ring_buffer_event *rb_evt;
struct ring_buffer_iter *rb_iter;
- switch (iter->type) {
- case TRI_CONSUMING:
+ if (trace_remote_iter_is_consuming(iter)) {
return ring_buffer_peek(iter->remote->trace_buffer, cpu, ts, lost_events);
- case TRI_NONCONSUMING:
+ } else {
rb_iter = __get_rb_iter(iter, cpu);
if (!rb_iter)
return NULL;
@@ -629,14 +659,10 @@ static void trace_remote_iter_move(struct trace_remote_iterator *iter)
{
struct trace_buffer *trace_buffer = iter->remote->trace_buffer;
- switch (iter->type) {
- case TRI_CONSUMING:
+ if (trace_remote_iter_is_consuming(iter))
ring_buffer_consume(trace_buffer, iter->evt_cpu, NULL, NULL);
- break;
- case TRI_NONCONSUMING:
+ else
ring_buffer_iter_advance(__get_rb_iter(iter, iter->evt_cpu));
- break;
- }
}
static struct remote_event *trace_remote_find_event(struct trace_remote *remote, unsigned short id);
@@ -882,6 +908,87 @@ static const struct file_operations trace_fops = {
.release = trace_release,
};
+static bool trace_remote_do_dmesg(struct trace_remote *remote)
+{
+ struct trace_remote_iterator *iter = smp_load_acquire(&remote->dmesg);
+ unsigned int max_events = 1000;
+
+ if (!iter)
+ return false;
+
+ trace_remote_iter_read_start(iter);
+
+ while (trace_remote_iter_read_event(iter)) {
+ trace_seq_init(&iter->seq);
+
+ trace_remote_iter_print_event(iter);
+ trace_seq_putc(&iter->seq, 0);
+ if (!pr_info("%s", iter->seq.buffer))
+ break;
+
+ trace_remote_iter_move(iter);
+
+ if (!(--max_events))
+ break;
+ }
+
+ trace_remote_iter_read_finished(iter);
+
+ return !max_events;
+}
+
+static int trace_remote_enable_dmesg(struct trace_remote *remote, bool enable)
+{
+ struct trace_remote_iterator *iter = remote->dmesg;
+
+ lockdep_assert_held(&remote->lock);
+
+ if (enable == !!iter)
+ return 0;
+
+ if (enable) {
+ iter = trace_remote_iter(remote, RING_BUFFER_ALL_CPUS, TRI_DMESG);
+ if (IS_ERR(iter))
+ return PTR_ERR(iter);
+ } else {
+ trace_remote_iter_free(remote->dmesg);
+ /* trace_remote_iter_free has reset remote->dmesg */
+ }
+
+ return 0;
+}
+
+static ssize_t
+dmesg_write(struct file *filp, const char __user *ubuf, size_t cnt, loff_t *ppos)
+{
+ struct seq_file *seq = filp->private_data;
+ struct trace_remote *remote = seq->private;
+ bool val;
+ int ret;
+
+ ret = kstrtobool_from_user(ubuf, cnt, &val);
+ if (ret)
+ return ret;
+
+ guard(mutex)(&remote->lock);
+
+ ret = trace_remote_enable_dmesg(remote, val);
+ if (ret)
+ return ret;
+
+ return cnt;
+}
+
+static int dmesg_show(struct seq_file *s, void *unused)
+{
+ struct trace_remote *remote = s->private;
+
+ seq_printf(s, "%d\n", !!remote->dmesg);
+
+ return 0;
+}
+DEFINE_TRACE_REMOTE_ATTRIBUTE(dmesg);
+
static int trace_remote_init_tracefs(const char *name, struct trace_remote *remote)
{
struct dentry *remote_d, *percpu_d, *d;
@@ -922,6 +1029,10 @@ static int trace_remote_init_tracefs(const char *name, struct trace_remote *remo
if (!d)
goto err;
+ d = trace_create_file("dmesg", TRACEFS_MODE_WRITE, remote_d, remote, &dmesg_fops);
+ if (!d)
+ goto err;
+
percpu_d = tracefs_create_dir("per_cpu", remote_d);
if (!percpu_d) {
pr_err("Failed to create tracefs dir "TRACEFS_DIR"%s/per_cpu/\n", name);
--
2.55.0.229.g6434b31f56-goog
^ permalink raw reply related
* [PATCH v3 07/18] tracing/remotes: selftests: Add a test for the dmesg tracefs file
From: Vincent Donnefort @ 2026-07-21 10:35 UTC (permalink / raw)
To: rostedt, mhiramat, linux-trace-kernel
Cc: mathieu.desnoyers, kernel-team, linux-kernel, Vincent Donnefort
In-Reply-To: <20260721103603.342266-1-vdonnefort@google.com>
Exercise the newly introduced dmesg tracefs file that turns on and off
the dmesg redirection.
Signed-off-by: Vincent Donnefort <vdonnefort@google.com>
diff --git a/tools/testing/selftests/ftrace/test.d/remotes/dmesg.tc b/tools/testing/selftests/ftrace/test.d/remotes/dmesg.tc
new file mode 100644
index 000000000000..aebeea9dbab6
--- /dev/null
+++ b/tools/testing/selftests/ftrace/test.d/remotes/dmesg.tc
@@ -0,0 +1,72 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0
+# description: Test trace remote dmesg redirection
+# requires: remotes/test
+
+. $TEST_DIR/remotes/functions
+
+test_dmesg()
+{
+ echo 0 > tracing_on
+ assert_unloaded
+
+ #
+ # Test dmesg on/off when tracing is disabled
+ #
+ echo 1 > dmesg
+ test $(cat dmesg) -eq 1
+ assert_loaded
+
+ echo 0 > dmesg
+ test $(cat dmesg) -eq 0
+ assert_unloaded
+
+ #
+ # Test events are logged to dmesg
+ #
+ dmesg -c > /dev/null
+
+ echo 1 > tracing_on
+ assert_loaded
+ echo 1 > dmesg
+ test $(cat dmesg) -eq 1
+
+ nr_events=128
+ for i in $(seq 1 $nr_events); do
+ echo $i > write_event
+ done
+
+ sleep 1
+ output=$(mktemp $TMPDIR/remote_test.XXXXXX)
+ dmesg | grep "selftest id=" | sed 's/^[^]]*] //'> $output
+
+ check_trace 1 $nr_events $output
+
+ rm $output
+
+ #
+ # Disable dmesg and Test events were not consumed by dmesg
+ #
+ echo 0 > dmesg
+ test $(cat dmesg) -eq 0
+
+ start_id=$(($nr_events + 1))
+ end_id=$(($start_id + $nr_events))
+
+ for i in $(seq $start_id $end_id); do
+ echo $i > write_event
+ done
+
+ sleep 1
+
+ output=$(dump_trace_pipe)
+ check_trace $start_id $end_id $output
+ rm $output
+}
+
+if [ -z "$SOURCE_REMOTE_TEST" ]; then
+ set -e
+
+ setup_remote_test
+ test_dmesg
+fi
diff --git a/tools/testing/selftests/ftrace/test.d/remotes/functions b/tools/testing/selftests/ftrace/test.d/remotes/functions
index 05224fac3653..4a14aa72fdf0 100644
--- a/tools/testing/selftests/ftrace/test.d/remotes/functions
+++ b/tools/testing/selftests/ftrace/test.d/remotes/functions
@@ -8,6 +8,7 @@ setup_remote()
cd remotes/$name/
echo 0 > tracing_on
+ echo 0 > dmesg
clear_trace
echo 7 > buffer_size_kb
echo 0 > events/enable
diff --git a/tools/testing/selftests/ftrace/test.d/remotes/hypervisor/dmesg.tc b/tools/testing/selftests/ftrace/test.d/remotes/hypervisor/dmesg.tc
new file mode 100644
index 000000000000..bf4a3c145e7a
--- /dev/null
+++ b/tools/testing/selftests/ftrace/test.d/remotes/hypervisor/dmesg.tc
@@ -0,0 +1,11 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0
+# description: Test the hypervisor trace dmesg redirection
+# requires: remotes/hypervisor/write_event
+
+SOURCE_REMOTE_TEST=1
+. $TEST_DIR/remotes/dmesg.tc
+
+set -e
+setup_remote "hypervisor"
+test_dmesg
--
2.55.0.229.g6434b31f56-goog
^ permalink raw reply related
* [PATCH v3 08/18] tracing/remotes: selftests: Prefix hypervisor folder
From: Vincent Donnefort @ 2026-07-21 10:35 UTC (permalink / raw)
To: rostedt, mhiramat, linux-trace-kernel
Cc: mathieu.desnoyers, kernel-team, linux-kernel, Vincent Donnefort
In-Reply-To: <20260721103603.342266-1-vdonnefort@google.com>
Avoid interleaving run tests by prefixing the hypervisor folder.
Signed-off-by: Vincent Donnefort <vdonnefort@google.com>
diff --git a/tools/testing/selftests/ftrace/test.d/remotes/hypervisor/buffer_size.tc b/tools/testing/selftests/ftrace/test.d/remotes/00hypervisor/buffer_size.tc
similarity index 100%
rename from tools/testing/selftests/ftrace/test.d/remotes/hypervisor/buffer_size.tc
rename to tools/testing/selftests/ftrace/test.d/remotes/00hypervisor/buffer_size.tc
diff --git a/tools/testing/selftests/ftrace/test.d/remotes/hypervisor/dmesg.tc b/tools/testing/selftests/ftrace/test.d/remotes/00hypervisor/dmesg.tc
similarity index 100%
rename from tools/testing/selftests/ftrace/test.d/remotes/hypervisor/dmesg.tc
rename to tools/testing/selftests/ftrace/test.d/remotes/00hypervisor/dmesg.tc
diff --git a/tools/testing/selftests/ftrace/test.d/remotes/hypervisor/hotplug.tc b/tools/testing/selftests/ftrace/test.d/remotes/00hypervisor/hotplug.tc
similarity index 100%
rename from tools/testing/selftests/ftrace/test.d/remotes/hypervisor/hotplug.tc
rename to tools/testing/selftests/ftrace/test.d/remotes/00hypervisor/hotplug.tc
diff --git a/tools/testing/selftests/ftrace/test.d/remotes/hypervisor/reset.tc b/tools/testing/selftests/ftrace/test.d/remotes/00hypervisor/reset.tc
similarity index 100%
rename from tools/testing/selftests/ftrace/test.d/remotes/hypervisor/reset.tc
rename to tools/testing/selftests/ftrace/test.d/remotes/00hypervisor/reset.tc
diff --git a/tools/testing/selftests/ftrace/test.d/remotes/hypervisor/trace.tc b/tools/testing/selftests/ftrace/test.d/remotes/00hypervisor/trace.tc
similarity index 100%
rename from tools/testing/selftests/ftrace/test.d/remotes/hypervisor/trace.tc
rename to tools/testing/selftests/ftrace/test.d/remotes/00hypervisor/trace.tc
diff --git a/tools/testing/selftests/ftrace/test.d/remotes/hypervisor/trace_pipe.tc b/tools/testing/selftests/ftrace/test.d/remotes/00hypervisor/trace_pipe.tc
similarity index 100%
rename from tools/testing/selftests/ftrace/test.d/remotes/hypervisor/trace_pipe.tc
rename to tools/testing/selftests/ftrace/test.d/remotes/00hypervisor/trace_pipe.tc
diff --git a/tools/testing/selftests/ftrace/test.d/remotes/hypervisor/unloading.tc b/tools/testing/selftests/ftrace/test.d/remotes/00hypervisor/unloading.tc
similarity index 100%
rename from tools/testing/selftests/ftrace/test.d/remotes/hypervisor/unloading.tc
rename to tools/testing/selftests/ftrace/test.d/remotes/00hypervisor/unloading.tc
--
2.55.0.229.g6434b31f56-goog
^ permalink raw reply
* [PATCH v3 09/18] ring-buffer: Use irqsave for the reader lock in ring_buffer_poll_remote
From: Vincent Donnefort @ 2026-07-21 10:35 UTC (permalink / raw)
To: rostedt, mhiramat, linux-trace-kernel
Cc: mathieu.desnoyers, kernel-team, linux-kernel, Vincent Donnefort,
Sashiko
In-Reply-To: <20260721103603.342266-1-vdonnefort@google.com>
Calling rb_wakeups with the reader lock but interrupts enabled can lead
to a deadlock: the irq_work might run on the same CPU, but will
block when acquiring that very same reader spinlock.
First, rb_wakeups doesn't even need to be called under the reader lock.
Move the function outside of the reader lock scope.
Second, the reader lock must be called with IRQs disabled anyway. Use
the irqsave variant of the spinlock.
Fixes: 2e67fabd8b77 ("ring-buffer: Introduce ring-buffer remotes")
Reported-by: Sashiko <sashiko-bot@kernel.org>
Signed-off-by: Vincent Donnefort <vdonnefort@google.com>
diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c
index 804ccae694d2..f970d461b2ba 100644
--- a/kernel/trace/ring_buffer.c
+++ b/kernel/trace/ring_buffer.c
@@ -6798,13 +6798,17 @@ int ring_buffer_poll_remote(struct trace_buffer *buffer, int cpu)
struct ring_buffer_per_cpu *cpu_buffer;
if (cpu != RING_BUFFER_ALL_CPUS) {
+ bool wakeup;
+
if (!cpumask_test_cpu(cpu, buffer->cpumask))
return -EINVAL;
cpu_buffer = buffer->buffers[cpu];
- guard(raw_spinlock)(&cpu_buffer->reader_lock);
- if (rb_read_remote_meta_page(cpu_buffer))
+ scoped_guard(raw_spinlock_irqsave, &cpu_buffer->reader_lock)
+ wakeup = rb_read_remote_meta_page(cpu_buffer);
+
+ if (wakeup)
rb_wakeups(buffer, cpu_buffer);
return 0;
@@ -6819,7 +6823,7 @@ int ring_buffer_poll_remote(struct trace_buffer *buffer, int cpu)
for_each_buffer_cpu(buffer, cpu) {
cpu_buffer = buffer->buffers[cpu];
- guard(raw_spinlock)(&cpu_buffer->reader_lock);
+ guard(raw_spinlock_irqsave)(&cpu_buffer->reader_lock);
rb_read_remote_meta_page(cpu_buffer);
}
--
2.55.0.229.g6434b31f56-goog
^ permalink raw reply related
* [PATCH v3 10/18] ring-buffer: Use panic-friendly locking in ring_buffer_iter interface
From: Vincent Donnefort @ 2026-07-21 10:35 UTC (permalink / raw)
To: rostedt, mhiramat, linux-trace-kernel
Cc: mathieu.desnoyers, kernel-team, linux-kernel, Vincent Donnefort
In-Reply-To: <20260721103603.342266-1-vdonnefort@google.com>
In preparation for allowing trace_remote to dump the buffer on panic,
make the non-consuming iterator functions panic-friendly.
Signed-off-by: Vincent Donnefort <vdonnefort@google.com>
diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c
index f970d461b2ba..517bdb061083 100644
--- a/kernel/trace/ring_buffer.c
+++ b/kernel/trace/ring_buffer.c
@@ -5601,6 +5601,9 @@ static void rb_iter_reset(struct ring_buffer_iter *iter)
}
}
+static inline bool rb_reader_lock(struct ring_buffer_per_cpu *cpu_buffer);
+static inline void rb_reader_unlock(struct ring_buffer_per_cpu *cpu_buffer, bool locked);
+
/**
* ring_buffer_iter_reset - reset an iterator
* @iter: The iterator to reset
@@ -5612,15 +5615,18 @@ void ring_buffer_iter_reset(struct ring_buffer_iter *iter)
{
struct ring_buffer_per_cpu *cpu_buffer;
unsigned long flags;
+ bool dolock;
if (!iter)
return;
cpu_buffer = iter->cpu_buffer;
- raw_spin_lock_irqsave(&cpu_buffer->reader_lock, flags);
+ local_irq_save(flags);
+ dolock = rb_reader_lock(cpu_buffer);
rb_iter_reset(iter);
- raw_spin_unlock_irqrestore(&cpu_buffer->reader_lock, flags);
+ rb_reader_unlock(cpu_buffer, dolock);
+ local_irq_restore(flags);
}
EXPORT_SYMBOL_GPL(ring_buffer_iter_reset);
@@ -6297,11 +6303,14 @@ ring_buffer_iter_peek(struct ring_buffer_iter *iter, u64 *ts)
struct ring_buffer_per_cpu *cpu_buffer = iter->cpu_buffer;
struct ring_buffer_event *event;
unsigned long flags;
+ bool dolock;
again:
- raw_spin_lock_irqsave(&cpu_buffer->reader_lock, flags);
+ local_irq_save(flags);
+ dolock = rb_reader_lock(cpu_buffer);
event = rb_iter_peek(iter, ts);
- raw_spin_unlock_irqrestore(&cpu_buffer->reader_lock, flags);
+ rb_reader_unlock(cpu_buffer, dolock);
+ local_irq_restore(flags);
if (event && event->type_len == RINGBUF_TYPE_PADDING)
goto again;
@@ -6439,12 +6448,15 @@ void ring_buffer_iter_advance(struct ring_buffer_iter *iter)
{
struct ring_buffer_per_cpu *cpu_buffer = iter->cpu_buffer;
unsigned long flags;
+ bool dolock;
- raw_spin_lock_irqsave(&cpu_buffer->reader_lock, flags);
+ local_irq_save(flags);
+ dolock = rb_reader_lock(cpu_buffer);
iter->missed_events = 0;
rb_advance_iter(iter);
- raw_spin_unlock_irqrestore(&cpu_buffer->reader_lock, flags);
+ rb_reader_unlock(cpu_buffer, dolock);
+ local_irq_restore(flags);
}
EXPORT_SYMBOL_GPL(ring_buffer_iter_advance);
--
2.55.0.229.g6434b31f56-goog
^ permalink raw reply related
* [PATCH v3 11/18] ring-buffer: Add ring_buffer_read_remote_meta_page()
From: Vincent Donnefort @ 2026-07-21 10:35 UTC (permalink / raw)
To: rostedt, mhiramat, linux-trace-kernel
Cc: mathieu.desnoyers, kernel-team, linux-kernel, Vincent Donnefort
In-Reply-To: <20260721103603.342266-1-vdonnefort@google.com>
In preparation for the introduction of a panic handler for trace
remotes, add a ring_buffer_read_remote_meta_page(). This is basically
similar to ring_buffer_poll_remote, but it doesn't try to wake-up
readers and, in the !RING_BUFFER_ALL_CPUS case, uses panic-friendly
locks.
While at it, update trace_remote_has_cpu() to use this new function
instead of ring_buffer_poll_remote(), avoiding unnecessary wakeups when
verifying if a CPU buffer is active.
Signed-off-by: Vincent Donnefort <vdonnefort@google.com>
diff --git a/include/linux/ring_buffer.h b/include/linux/ring_buffer.h
index 0670742b2d60..3c0de6593c81 100644
--- a/include/linux/ring_buffer.h
+++ b/include/linux/ring_buffer.h
@@ -299,6 +299,7 @@ struct ring_buffer_remote {
void *priv;
};
+int ring_buffer_read_remote_meta_page(struct trace_buffer *buffer, int cpu);
int ring_buffer_poll_remote(struct trace_buffer *buffer, int cpu);
struct trace_buffer *
diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c
index 517bdb061083..c35e36c553aa 100644
--- a/kernel/trace/ring_buffer.c
+++ b/kernel/trace/ring_buffer.c
@@ -6805,6 +6805,47 @@ bool ring_buffer_empty_cpu(struct trace_buffer *buffer, int cpu)
}
EXPORT_SYMBOL_GPL(ring_buffer_empty_cpu);
+/**
+ * ring_buffer_read_remote_meta_page - read the meta page of a remote ring buffer
+ * @buffer: The ring buffer
+ * @cpu: The CPU buffer to read (or RING_BUFFER_ALL_CPUS)
+ *
+ * Returns:
+ * 0 on success, or -EINVAL if the CPU is not in the buffer's cpumask.
+ */
+int ring_buffer_read_remote_meta_page(struct trace_buffer *buffer, int cpu)
+{
+ struct ring_buffer_per_cpu *cpu_buffer;
+
+ if (cpu != RING_BUFFER_ALL_CPUS) {
+ unsigned long flags;
+ bool dolock;
+
+ if (!cpumask_test_cpu(cpu, buffer->cpumask))
+ return -EINVAL;
+
+ cpu_buffer = buffer->buffers[cpu];
+
+ local_irq_save(flags);
+ dolock = rb_reader_lock(cpu_buffer);
+ rb_read_remote_meta_page(cpu_buffer);
+ rb_reader_unlock(cpu_buffer, dolock);
+ local_irq_restore(flags);
+ return 0;
+ }
+
+ guard(cpus_read_lock)();
+
+ for_each_buffer_cpu(buffer, cpu) {
+ cpu_buffer = buffer->buffers[cpu];
+
+ guard(raw_spinlock_irqsave)(&cpu_buffer->reader_lock);
+ rb_read_remote_meta_page(cpu_buffer);
+ }
+
+ return 0;
+}
+
int ring_buffer_poll_remote(struct trace_buffer *buffer, int cpu)
{
struct ring_buffer_per_cpu *cpu_buffer;
diff --git a/kernel/trace/trace_remote.c b/kernel/trace/trace_remote.c
index c54e1c65d163..fcd2dd754f6e 100644
--- a/kernel/trace/trace_remote.c
+++ b/kernel/trace/trace_remote.c
@@ -350,7 +350,7 @@ static bool trace_remote_has_cpu(struct trace_remote *remote, int cpu)
if (cpu == RING_BUFFER_ALL_CPUS)
return true;
- return ring_buffer_poll_remote(remote->trace_buffer, cpu) == 0;
+ return ring_buffer_read_remote_meta_page(remote->trace_buffer, cpu) == 0;
}
static void __free_ring_buffer_iter(struct trace_remote_iterator *iter, int cpu)
--
2.55.0.229.g6434b31f56-goog
^ permalink raw reply related
* [PATCH v3 12/18] ring-buffer: Add kerneldoc for ring_buffer_poll_remote
From: Vincent Donnefort @ 2026-07-21 10:35 UTC (permalink / raw)
To: rostedt, mhiramat, linux-trace-kernel
Cc: mathieu.desnoyers, kernel-team, linux-kernel, Vincent Donnefort
In-Reply-To: <20260721103603.342266-1-vdonnefort@google.com>
Document ring_buffer_poll_remote().
Signed-off-by: Vincent Donnefort <vdonnefort@google.com>
diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c
index c35e36c553aa..0e3a53226bfc 100644
--- a/kernel/trace/ring_buffer.c
+++ b/kernel/trace/ring_buffer.c
@@ -6846,6 +6846,17 @@ int ring_buffer_read_remote_meta_page(struct trace_buffer *buffer, int cpu)
return 0;
}
+/**
+ * ring_buffer_poll_remote - poll a remote ring buffer for new data
+ * @buffer: The ring buffer
+ * @cpu: The CPU buffer to poll (or RING_BUFFER_ALL_CPUS)
+ *
+ * This function polls the specified remote CPU buffer (or all of them)
+ * by reading its meta page to update the local reader's view. If new
+ * entries are detected, it triggers wakeups for any waiting readers.
+ * Returns:
+ * 0 on success, or -EINVAL if the CPU is not in the buffer's cpumask.
+ */
int ring_buffer_poll_remote(struct trace_buffer *buffer, int cpu)
{
struct ring_buffer_per_cpu *cpu_buffer;
--
2.55.0.229.g6434b31f56-goog
^ permalink raw reply related
* [PATCH v3 13/18] tracing/remotes: Add dump_on_panic tracefs file
From: Vincent Donnefort @ 2026-07-21 10:35 UTC (permalink / raw)
To: rostedt, mhiramat, linux-trace-kernel
Cc: mathieu.desnoyers, kernel-team, linux-kernel, Vincent Donnefort
In-Reply-To: <20260721103603.342266-1-vdonnefort@google.com>
When enabled, dump_on_panic will dump the content of the trace remote
buffer if the system panics.
Signed-off-by: Vincent Donnefort <vdonnefort@google.com>
diff --git a/kernel/trace/trace_remote.c b/kernel/trace/trace_remote.c
index fcd2dd754f6e..d109b2503db9 100644
--- a/kernel/trace/trace_remote.c
+++ b/kernel/trace/trace_remote.c
@@ -7,6 +7,7 @@
#include <linux/kstrtox.h>
#include <linux/lockdep.h>
#include <linux/mutex.h>
+#include <linux/panic_notifier.h>
#include <linux/tracefs.h>
#include <linux/trace_remote.h>
#include <linux/trace_seq.h>
@@ -22,6 +23,7 @@ enum tri_type {
TRI_CONSUMING,
TRI_NONCONSUMING,
TRI_DMESG,
+ TRI_PANIC,
};
struct trace_remote_iterator {
@@ -59,6 +61,9 @@ struct trace_remote {
unsigned int poll_cnt;
unsigned int poll_ms;
bool tracing_on;
+ bool panic_on;
+ struct notifier_block panic_notifier;
+ struct trace_remote_iterator *panic_iter;
struct delayed_work poll_work;
};
@@ -71,10 +76,15 @@ static bool trace_remote_loaded(struct trace_remote *remote)
return !!remote->trace_buffer;
}
+static void trace_remote_unload(struct trace_remote *remote);
+static int trace_remote_panic_load(struct trace_remote *remote);
+static void trace_remote_panic_unload(struct trace_remote *remote);
+
static int trace_remote_load(struct trace_remote *remote)
{
struct ring_buffer_remote *rb_remote = &remote->rb_remote;
struct trace_buffer_desc *desc;
+ int ret = 0;
lockdep_assert_held(&remote->lock);
@@ -89,15 +99,28 @@ static int trace_remote_load(struct trace_remote *remote)
rb_remote->swap_reader_page = remote->cbs->swap_reader_page;
rb_remote->priv = remote->priv;
rb_remote->reset = remote->cbs->reset;
+ remote->trace_buffer_desc = desc;
remote->trace_buffer = ring_buffer_alloc_remote(rb_remote);
if (!remote->trace_buffer) {
remote->cbs->unload_trace_buffer(desc, remote->priv);
return -ENOMEM;
}
- remote->trace_buffer_desc = desc;
+ if (remote->panic_on) {
+ ret = trace_remote_panic_load(remote);
+ if (ret)
+ trace_remote_unload(remote);
+ }
- return 0;
+ return ret;
+}
+
+static void trace_remote_unload(struct trace_remote *remote)
+{
+ trace_remote_panic_unload(remote);
+ ring_buffer_free(remote->trace_buffer);
+ remote->trace_buffer = NULL;
+ remote->cbs->unload_trace_buffer(remote->trace_buffer_desc, remote->priv);
}
static void trace_remote_try_unload(struct trace_remote *remote)
@@ -115,9 +138,7 @@ static void trace_remote_try_unload(struct trace_remote *remote)
if (!ring_buffer_empty(remote->trace_buffer))
return;
- ring_buffer_free(remote->trace_buffer);
- remote->trace_buffer = NULL;
- remote->cbs->unload_trace_buffer(remote->trace_buffer_desc, remote->priv);
+ trace_remote_unload(remote);
}
static int trace_remote_enable_tracing(struct trace_remote *remote)
@@ -434,58 +455,68 @@ static void trace_remote_dec_poll(struct trace_remote *remote)
static struct trace_remote_iterator
*trace_remote_iter(struct trace_remote *remote, int cpu, enum tri_type type)
{
- struct trace_remote_iterator *iter = NULL;
+ struct trace_remote_iterator *iter __free(kfree) = kzalloc_obj(*iter);
int ret;
lockdep_assert_held(&remote->lock);
- if (type == TRI_NONCONSUMING && !trace_remote_loaded(remote))
- return NULL;
+ if (!iter)
+ return ERR_PTR(-ENOMEM);
- ret = trace_remote_get(remote, cpu);
- if (ret)
- return ERR_PTR(ret);
+ switch (type) {
+ case TRI_NONCONSUMING:
+ if (!trace_remote_loaded(remote))
+ return NULL;
+ fallthrough;
+ case TRI_CONSUMING:
+ case TRI_DMESG:
+ ret = trace_remote_get(remote, cpu);
+ if (ret)
+ return ERR_PTR(ret);
+ break;
+ case TRI_PANIC:
+ break;
+ }
if (!trace_remote_has_cpu(remote, cpu)) {
ret = -ENODEV;
goto err;
}
- iter = kzalloc_obj(*iter);
- if (iter) {
- iter->remote = remote;
- iter->cpu = cpu;
- iter->type = type;
- trace_seq_init(&iter->seq);
+ iter->remote = remote;
+ iter->cpu = cpu;
+ iter->type = type;
+ trace_seq_init(&iter->seq);
- switch (type) {
- case TRI_DMESG:
- /* only one printk iter allowed */
- if (WARN_ON_ONCE(remote->dmesg)) {
- ret = -EBUSY;
- break;
- }
- smp_store_release(&remote->dmesg, iter);
- fallthrough;
- case TRI_CONSUMING:
- trace_remote_inc_poll(remote);
- break;
- case TRI_NONCONSUMING:
- ret = __alloc_ring_buffer_iter(iter, cpu);
- break;
+ switch (type) {
+ case TRI_DMESG:
+ /* only one dmesg iter allowed */
+ if (WARN_ON_ONCE(remote->dmesg)) {
+ ret = -EBUSY;
+ goto err;
}
-
+ smp_store_release(&remote->dmesg, iter);
+ fallthrough;
+ case TRI_CONSUMING:
+ trace_remote_inc_poll(remote);
+ break;
+ case TRI_PANIC:
+ case TRI_NONCONSUMING:
+ ret = __alloc_ring_buffer_iter(iter, cpu);
if (ret)
goto err;
-
- return iter;
+ break;
}
- ret = -ENOMEM;
+
+ return no_free_ptr(iter);
err:
- kfree(iter);
- trace_remote_put(remote);
-
+ switch (type) {
+ case TRI_PANIC:
+ break;
+ default:
+ trace_remote_put(remote);
+ }
return ERR_PTR(ret);
}
@@ -508,14 +539,18 @@ static void trace_remote_iter_free(struct trace_remote_iterator *iter)
fallthrough;
case TRI_CONSUMING:
trace_remote_dec_poll(remote);
+ trace_remote_put(remote);
break;
case TRI_NONCONSUMING:
+ trace_remote_put(remote);
+ __free_ring_buffer_iter(iter, iter->cpu);
+ break;
+ case TRI_PANIC:
__free_ring_buffer_iter(iter, iter->cpu);
break;
}
kfree(iter);
- trace_remote_put(remote);
}
static bool trace_remote_iter_is_consuming(struct trace_remote_iterator *iter)
@@ -989,6 +1024,117 @@ static int dmesg_show(struct seq_file *s, void *unused)
}
DEFINE_TRACE_REMOTE_ATTRIBUTE(dmesg);
+static int trace_remote_panic_handler(struct notifier_block *self, unsigned long ev, void *v)
+{
+ struct trace_remote *remote = container_of(self, struct trace_remote, panic_notifier);
+ struct trace_remote_iterator *iter = smp_load_acquire(&remote->panic_iter);
+ int cpu;
+
+ if (!iter) {
+ pr_warn("Unexpected error: no panic iterator for the trace remote\n");
+ return NOTIFY_DONE;
+ }
+
+ for_each_possible_cpu(cpu) {
+ if (iter->rb_iters[cpu]) {
+ /* No RING_BUFFER_ALL_CPUS to avoid taking cpu_read_lock() */
+ ring_buffer_read_remote_meta_page(remote->trace_buffer, cpu);
+ ring_buffer_iter_reset(iter->rb_iters[cpu]);
+ }
+ }
+
+ while (trace_remote_iter_read_event(iter)) {
+ trace_seq_init(&iter->seq);
+
+ trace_remote_iter_print_event(iter);
+ trace_seq_putc(&iter->seq, 0);
+ pr_emerg("%s", iter->seq.buffer);
+
+ trace_remote_iter_move(iter);
+ }
+
+ return NOTIFY_DONE;
+}
+
+static int trace_remote_panic_load(struct trace_remote *remote)
+{
+ struct notifier_block *notifier = &remote->panic_notifier;
+ struct trace_remote_iterator *iter;
+
+ lockdep_assert_held(&remote->lock);
+
+ if (remote->panic_iter)
+ return 0;
+
+ iter = trace_remote_iter(remote, RING_BUFFER_ALL_CPUS, TRI_PANIC);
+ if (IS_ERR(iter))
+ return PTR_ERR(iter);
+
+ smp_store_release(&remote->panic_iter, iter);
+
+ notifier->notifier_call = trace_remote_panic_handler;
+ notifier->priority = INT_MAX - 1;
+ atomic_notifier_chain_register(&panic_notifier_list, notifier);
+
+ return 0;
+}
+
+static void trace_remote_panic_unload(struct trace_remote *remote)
+{
+ struct trace_remote_iterator *iter = remote->panic_iter;
+
+ lockdep_assert_held(&remote->lock);
+
+ if (!iter)
+ return;
+
+ atomic_notifier_chain_unregister(&panic_notifier_list, &remote->panic_notifier);
+ smp_store_release(&remote->panic_iter, NULL);
+ trace_remote_iter_free(iter);
+}
+
+static ssize_t dump_on_panic_write(struct file *filp, const char __user *ubuf,
+ size_t cnt, loff_t *ppos)
+{
+ struct seq_file *seq = filp->private_data;
+ struct trace_remote *remote = seq->private;
+ bool enable;
+ int ret;
+
+ ret = kstrtobool_from_user(ubuf, cnt, &enable);
+ if (ret)
+ return ret;
+
+ guard(mutex)(&remote->lock);
+
+ if (enable == remote->panic_on)
+ return cnt;
+
+ if (trace_remote_loaded(remote)) {
+ if (enable) {
+ ret = trace_remote_panic_load(remote);
+ if (ret)
+ return ret;
+ } else {
+ trace_remote_panic_unload(remote);
+ }
+ }
+
+ remote->panic_on = enable;
+
+ return cnt;
+}
+
+static int dump_on_panic_show(struct seq_file *s, void *unused)
+{
+ struct trace_remote *remote = s->private;
+
+ seq_printf(s, "%d\n", remote->panic_on);
+
+ return 0;
+}
+DEFINE_TRACE_REMOTE_ATTRIBUTE(dump_on_panic);
+
static int trace_remote_init_tracefs(const char *name, struct trace_remote *remote)
{
struct dentry *remote_d, *percpu_d, *d;
@@ -1016,6 +1162,11 @@ static int trace_remote_init_tracefs(const char *name, struct trace_remote *remo
if (!d)
goto err;
+ d = trace_create_file("dump_on_panic", TRACEFS_MODE_WRITE, remote_d, remote,
+ &dump_on_panic_fops);
+ if (!d)
+ goto err;
+
d = trace_create_file("buffer_size_kb", TRACEFS_MODE_WRITE, remote_d, remote,
&buffer_size_kb_fops);
if (!d)
--
2.55.0.229.g6434b31f56-goog
^ permalink raw reply related
* [PATCH v3 14/18] tracing/remotes: selftests: Add a test for the dump_on_panic tracefs file
From: Vincent Donnefort @ 2026-07-21 10:35 UTC (permalink / raw)
To: rostedt, mhiramat, linux-trace-kernel
Cc: mathieu.desnoyers, kernel-team, linux-kernel, Vincent Donnefort
In-Reply-To: <20260721103603.342266-1-vdonnefort@google.com>
Exercise the newly introduced dump_on_panic tracefs file that turns on
or off the trace remote buffer dump on system panic.
Signed-off-by: Vincent Donnefort <vdonnefort@google.com>
diff --git a/tools/testing/selftests/ftrace/test.d/remotes/00hypervisor/dump_on_panic.tc b/tools/testing/selftests/ftrace/test.d/remotes/00hypervisor/dump_on_panic.tc
new file mode 100644
index 000000000000..5e3d3c412ecd
--- /dev/null
+++ b/tools/testing/selftests/ftrace/test.d/remotes/00hypervisor/dump_on_panic.tc
@@ -0,0 +1,11 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0
+# description: Test hypervisor trace dump_on_panic
+# requires: remotes/hypervisor/write_event
+
+SOURCE_REMOTE_TEST=1
+. $TEST_DIR/remotes/dump_on_panic.tc
+
+set -e
+setup_remote "hypervisor"
+test_dump_on_panic
diff --git a/tools/testing/selftests/ftrace/test.d/remotes/dump_on_panic.tc b/tools/testing/selftests/ftrace/test.d/remotes/dump_on_panic.tc
new file mode 100644
index 000000000000..cb9b4c6dc86c
--- /dev/null
+++ b/tools/testing/selftests/ftrace/test.d/remotes/dump_on_panic.tc
@@ -0,0 +1,51 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0
+# description: Test trace remote dump_on_panic
+# requires: remotes/test
+
+. $TEST_DIR/remotes/functions
+
+test_dump_on_panic()
+{
+ #
+ # Toggle when the buffer is loaded
+ #
+ assert_loaded
+ echo 1 > dump_on_panic
+ echo 0 > dump_on_panic
+
+ #
+ # Toggle when the buffer is unloaded
+ #
+ echo 0 > tracing_on
+ assert_unloaded
+ echo 1 > dump_on_panic
+ echo 0 > dump_on_panic
+
+ #
+ # Load and unload buffer while dump_on_panic is enabled
+ #
+ echo 0 > tracing_on
+ assert_unloaded
+
+ echo 1 > dump_on_panic
+ echo 1 > tracing_on
+ echo 0 > tracing_on
+
+ # REMOVE ME FOR A PROPER OOPS TEST
+ return
+
+ echo 1 > tracing_on
+
+ for i in $(seq 1 32); do
+ echo $i > write_event
+ done
+
+ echo c > /proc/sysrq-trigger
+}
+
+if [ -z "$SOURCE_REMOTE_TEST" ]; then
+ set -e
+ setup_remote_test
+ test_dump_on_panic
+fi
diff --git a/tools/testing/selftests/ftrace/test.d/remotes/functions b/tools/testing/selftests/ftrace/test.d/remotes/functions
index 4a14aa72fdf0..bdd28b5b8596 100644
--- a/tools/testing/selftests/ftrace/test.d/remotes/functions
+++ b/tools/testing/selftests/ftrace/test.d/remotes/functions
@@ -9,6 +9,7 @@ setup_remote()
cd remotes/$name/
echo 0 > tracing_on
echo 0 > dmesg
+ echo 0 > dump_on_panic
clear_trace
echo 7 > buffer_size_kb
echo 0 > events/enable
--
2.55.0.229.g6434b31f56-goog
^ permalink raw reply related
* [PATCH v3 15/18] tracing/remotes: Add poll_ms tracefs file
From: Vincent Donnefort @ 2026-07-21 10:36 UTC (permalink / raw)
To: rostedt, mhiramat, linux-trace-kernel
Cc: mathieu.desnoyers, kernel-team, linux-kernel, Vincent Donnefort
In-Reply-To: <20260721103603.342266-1-vdonnefort@google.com>
Add a tracefs file to configure the trace remote polling period. Keep
the default value to 100ms.
Signed-off-by: Vincent Donnefort <vdonnefort@google.com>
diff --git a/kernel/trace/trace_remote.c b/kernel/trace/trace_remote.c
index d109b2503db9..33dd3333b90a 100644
--- a/kernel/trace/trace_remote.c
+++ b/kernel/trace/trace_remote.c
@@ -1135,6 +1135,40 @@ static int dump_on_panic_show(struct seq_file *s, void *unused)
}
DEFINE_TRACE_REMOTE_ATTRIBUTE(dump_on_panic);
+static ssize_t poll_ms_write(struct file *filp, const char __user *ubuf, size_t cnt, loff_t *ppos)
+{
+ struct seq_file *seq = filp->private_data;
+ struct trace_remote *remote = seq->private;
+ unsigned int val;
+ int ret;
+
+ ret = kstrtouint_from_user(ubuf, cnt, 10, &val);
+ if (ret)
+ return ret;
+
+ if (!val)
+ return -EINVAL;
+
+ guard(mutex)(&remote->lock);
+
+ if (val < remote->poll_ms && remote->poll_cnt)
+ mod_delayed_work(system_percpu_wq, &remote->poll_work, msecs_to_jiffies(val));
+
+ remote->poll_ms = val;
+
+ return cnt;
+}
+
+static int poll_ms_show(struct seq_file *s, void *unused)
+{
+ struct trace_remote *remote = s->private;
+
+ seq_printf(s, "%u\n", remote->poll_ms);
+
+ return 0;
+}
+DEFINE_TRACE_REMOTE_ATTRIBUTE(poll_ms);
+
static int trace_remote_init_tracefs(const char *name, struct trace_remote *remote)
{
struct dentry *remote_d, *percpu_d, *d;
@@ -1167,6 +1201,10 @@ static int trace_remote_init_tracefs(const char *name, struct trace_remote *remo
if (!d)
goto err;
+ d = trace_create_file("poll_ms", TRACEFS_MODE_WRITE, remote_d, remote, &poll_ms_fops);
+ if (!d)
+ goto err;
+
d = trace_create_file("buffer_size_kb", TRACEFS_MODE_WRITE, remote_d, remote,
&buffer_size_kb_fops);
if (!d)
--
2.55.0.229.g6434b31f56-goog
^ permalink raw reply related
* [PATCH v3 16/18] tracing/remotes: Add trace_remote cmdline options
From: Vincent Donnefort @ 2026-07-21 10:36 UTC (permalink / raw)
To: rostedt, mhiramat, linux-trace-kernel
Cc: mathieu.desnoyers, kernel-team, linux-kernel, Vincent Donnefort
In-Reply-To: <20260721103603.342266-1-vdonnefort@google.com>
Following the same format as trace_instance, add a cmdline to configure
a trace remote on registration:
trace_remote=<remote>^<opt1>^<opt2>,<evt1>,<evt2>
Enabling events automatically turns on tracing.
Signed-off-by: Vincent Donnefort <vdonnefort@google.com>
diff --git a/kernel/trace/trace_remote.c b/kernel/trace/trace_remote.c
index 33dd3333b90a..11de5cea876e 100644
--- a/kernel/trace/trace_remote.c
+++ b/kernel/trace/trace_remote.c
@@ -13,6 +13,8 @@
#include <linux/trace_seq.h>
#include <linux/types.h>
+#include <asm/setup.h>
+
#include "trace.h"
#define TRACEFS_DIR "remotes"
@@ -1169,6 +1171,132 @@ static int poll_ms_show(struct seq_file *s, void *unused)
}
DEFINE_TRACE_REMOTE_ATTRIBUTE(poll_ms);
+static char trace_remote_cmdline[COMMAND_LINE_SIZE];
+
+static int __init set_trace_remote_cmdline(char *str)
+{
+ static int idx;
+
+ if (!str)
+ return 0;
+
+ strscpy(trace_remote_cmdline + idx, str, COMMAND_LINE_SIZE - idx);
+ idx += strlen(str);
+ trace_remote_cmdline[idx++] = '\t';
+ return 1;
+}
+__setup("trace_remote=", set_trace_remote_cmdline);
+
+static void trace_remote_apply_cmdline_opts(struct trace_remote *remote, char *cmdline)
+{
+ bool dmesg_on = false;
+ char *opt;
+ int ret;
+
+ while ((opt = strsep(&cmdline, "^"))) {
+ if (!*opt)
+ continue;
+
+ if (!strcmp(opt, "dump_on_panic")) {
+ remote->panic_on = true;
+ } else if (!strcmp(opt, "dmesg")) {
+ dmesg_on = true;
+ } else if (!strncmp(opt, "buf_size=", 9)) {
+ /* buf_size can only be applied if the buffer is unloaded */
+ if (!WARN_ON(trace_remote_loaded(remote)))
+ remote->trace_buffer_size = memparse(opt + 9, NULL);
+ } else if (!strncmp(opt, "poll=", 5)) {
+ unsigned int poll_ms;
+
+ if (!kstrtouint(opt + 5, 10, &poll_ms) && poll_ms > 0)
+ remote->poll_ms = poll_ms;
+ else
+ pr_warn("Invalid trace remote poll '%s'\n", opt);
+ } else {
+ pr_warn("Unknown trace remote option '%s'\n", opt);
+ }
+ }
+
+ if (dmesg_on) {
+ ret = trace_remote_enable_dmesg(remote, true);
+ if (ret)
+ pr_warn("Failed to enable trace remote dmesg (%d)\n", ret);
+ }
+}
+
+static struct remote_event *
+trace_remote_find_event_by_name(struct trace_remote *remote, const char *name);
+
+static int
+trace_remote_enable_event(struct trace_remote *remote, struct remote_event *evt, bool enable);
+
+static void trace_remote_apply_cmdline_events(struct trace_remote *remote, char *cmdline)
+{
+ bool tracing_on = false;
+ char *token;
+ int ret;
+
+ while ((token = strsep(&cmdline, ","))) {
+ struct remote_event *evt;
+
+ if (!*token)
+ continue;
+
+ evt = trace_remote_find_event_by_name(remote, token);
+ if (!evt) {
+ pr_warn("trace remote event '%s' not found\n", token);
+ continue;
+ }
+
+ ret = trace_remote_enable_event(remote, evt, true);
+ if (ret)
+ pr_warn("Failed to enable trace remote event '%s' (%d)\n", token, ret);
+ else
+ tracing_on = true;
+ }
+
+ if (tracing_on) {
+ ret = trace_remote_enable_tracing(remote);
+ if (ret)
+ pr_warn("Failed to enable trace remote tracing (%d)\n", ret);
+ }
+}
+
+static void trace_remote_apply_cmdline(const char *name, struct trace_remote *remote)
+{
+ char *cmdline __free(kfree) = NULL;
+ char *events_cmdline = NULL;
+ char *opts_cmdline = NULL;
+ char *curr, *next;
+
+ if (!trace_remote_cmdline[0])
+ return;
+
+ cmdline = kstrdup(trace_remote_cmdline, GFP_KERNEL);
+ if (!cmdline)
+ return;
+
+ next = cmdline;
+ while ((curr = strsep(&next, "\t"))) {
+ char *token = strsep(&curr, ",");
+ char *rname = strsep(&token, "^");
+
+ if (strcmp(rname, name) == 0) {
+ opts_cmdline = token;
+ events_cmdline = curr;
+ break;
+ }
+ }
+
+ guard(mutex)(&remote->lock);
+
+ if (opts_cmdline)
+ trace_remote_apply_cmdline_opts(remote, opts_cmdline);
+ if (events_cmdline)
+ trace_remote_apply_cmdline_events(remote, events_cmdline);
+}
+
+
static int trace_remote_init_tracefs(const char *name, struct trace_remote *remote)
{
struct dentry *remote_d, *percpu_d, *d;
@@ -1345,6 +1473,7 @@ int trace_remote_register(const char *name, struct trace_remote_callbacks *cbs,
}
list_add(&remote->node, &trace_remotes);
+ trace_remote_apply_cmdline(name, remote);
retain_and_null_ptr(remote);
return 0;
@@ -1818,3 +1947,15 @@ static struct remote_event *trace_remote_find_event(struct trace_remote *remote,
return bsearch((const void *)(unsigned long)id, remote->events, remote->nr_events,
sizeof(*remote->events), __cmp_events);
}
+
+static struct remote_event *
+trace_remote_find_event_by_name(struct trace_remote *remote, const char *name)
+{
+ int i;
+
+ for (i = 0; i < remote->nr_events; i++) {
+ if (!strcmp(remote->events[i].name, name))
+ return &remote->events[i];
+ }
+ return NULL;
+}
--
2.55.0.229.g6434b31f56-goog
^ permalink raw reply related
* [PATCH v3 17/18] Documentation: tracing/remotes: Add detailed tracefs layout
From: Vincent Donnefort @ 2026-07-21 10:36 UTC (permalink / raw)
To: rostedt, mhiramat, linux-trace-kernel
Cc: mathieu.desnoyers, kernel-team, linux-kernel, Vincent Donnefort
In-Reply-To: <20260721103603.342266-1-vdonnefort@google.com>
Add a description for each tracefs file available in a trace remote
instance.
Signed-off-by: Vincent Donnefort <vdonnefort@google.com>
diff --git a/Documentation/trace/remotes.rst b/Documentation/trace/remotes.rst
index 1f9d764f69aa..b02ebed4a03f 100644
--- a/Documentation/trace/remotes.rst
+++ b/Documentation/trace/remotes.rst
@@ -19,8 +19,8 @@ for which the host kernel can see and expose to user space.
Register a remote
=================
-A remote must provide a set of callbacks `struct trace_remote_callbacks` whom
-description can be found below. Those callbacks allows Tracefs to enable and
+A remote must provide a set of callbacks `struct trace_remote_callbacks` whose
+description can be found below. Those callbacks allow Tracefs to enable and
disable tracing and events, to load and unload a tracing buffer (a set of
ring-buffers) and to swap a reader page with the head page, which enables
consuming reading.
@@ -28,8 +28,66 @@ consuming reading.
.. kernel-doc:: include/linux/trace_remote.h
Once registered, an instance will appear for this remote in the Tracefs
-directory **remotes/**. Buffers can then be read using the usual Tracefs files
-**trace_pipe** and **trace**.
+directory **remotes/**. The files within this directory allow configuring
+and reading the remote buffer (see `The File System` below).
+
+The File System
+===============
+A remote tracing instance is represented by a directory in Tracefs under
+**remotes/**. The layout and files within it are very similar to standard ftrace
+instances. Inside the remote directory, the following files and directories are
+available:
+
+ tracing_on
+ This file allows enabling or disabling the remote tracing.
+
+ buffer_size_kb
+ This file displays and allows changing the size of the per-CPU ring
+ buffers used by the remote. It also shows if the buffer is **loaded** or
+ **unloaded**. To change the size, the remote buffers must be unloaded
+ first. Remote buffers are automatically unloaded when **tracing_on** is
+ off, no one is reading the buffer (either by accessing **trace_pipe** or
+ when **dmesg** is on) and no events remain in the buffer.
+
+ trace
+ Display the human-readable content of the remote buffers. Reading this
+ file is non-consuming. Writing to this file clears the ring buffers.
+
+ trace_pipe
+ Similar to **trace** but reading it consumes the events from the ring
+ buffers (consuming read). It blocks if there are no new events.
+
+ dmesg
+ When enabled, all events from the remote are redirected to the kernel
+ dmesg. This is similar to the **tp_printk** option for in-kernel events.
+ It counts as a reader of the remote buffers and prevents unloading.
+
+ dump_on_panic
+ When enabled, the remote tracing buffer is dumped to the console when a
+ kernel panic occurs.
+
+ poll_ms
+ Modifies the polling interval for the trace_remote.
+
+ per_cpu/
+ This directory contains subdirectories for each possible CPU (e.g.,
+ **cpu0/**, **cpu1/** ...)
+
+ per_cpu/cpuX/trace
+ This is similar to the **trace** file, but it will only display the data
+ specific for the CPU. If written to, it only clears the specific CPU
+ buffer.
+
+ per_cpu/cpuX/trace_pipe
+ This is similar to the **trace_pipe** file, and is a consuming read, but
+ it will only display (and consume) the data specific to the CPU.
+
+ events/
+ This directory contains remote events that can be enabled or disabled.
+
+ events/enable
+ Allows enabling or disabling all the remote events.
+
Declare a remote event
======================
--
2.55.0.229.g6434b31f56-goog
^ permalink raw reply related
* [PATCH v3 18/18] Documentation/kernel-parameters: Add trace_remote
From: Vincent Donnefort @ 2026-07-21 10:36 UTC (permalink / raw)
To: rostedt, mhiramat, linux-trace-kernel
Cc: mathieu.desnoyers, kernel-team, linux-kernel, Vincent Donnefort
In-Reply-To: <20260721103603.342266-1-vdonnefort@google.com>
The trace_remote parameter allows configuring e a trace remote on
registration. The syntax is similar to trace_instance.
Signed-off-by: Vincent Donnefort <vdonnefort@google.com>
diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index b5493a7f8f22..3c1cf3025925 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -7787,6 +7787,25 @@ Kernel parameters
See also Documentation/trace/ftrace.rst "trace options"
section.
+ trace_remote=[remote-info]
+ [FTRACE] Configure a trace remote instance at boot.
+ Format: <name>[^option1[^option2...]][,event1[,event2...]]
+
+ Supported options:
+
+ dump_on_panic - Enable dumping the trace buffer on
+ panic.
+ dmesg - Redirect tracing output to dmesg.
+ buf_size=<size> - Set the trace buffer size (e.g. 2M).
+ poll=<ms> - Set the trace remote polling interval
+ in milliseconds.
+
+ Events are a comma-separated list of events to enable.
+ If events are specified, tracing is automatically enabled.
+
+ Multiple remotes can be configured by specifying this
+ parameter multiple times.
+
trace_trigger=[trigger-list]
[FTRACE] Add an event trigger on specific events.
Set a trigger on top of a specific event, with an optional
--
2.55.0.229.g6434b31f56-goog
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox