[PATCH v2 0/2] Fix trace remotes read with an offline CPU

[PATCH v2 0/2] Fix trace remotes read with an offline CPU

[Vincent Donnefort]

This small series is fixing non-consuming read of a trace remote when the
trace_buffer is created after a CPU is offline.

It also extends hotplug testing coverage to include this test case.

I have based this series on top of kvmarm/next which contains the hypervisor
tracing patches.

Changes in v2:

 * Add a trap to restore offlined CPU on test failure in hotplug.tc.
 * Fix assert_loaded/assert_unloaded error propagation.
 * Ensure we probe for existing events when setting up consuming read
   iterator.

v1: https://lore.kernel.org/all/20260401025003.3258729-1-vdonnefort@google.com/

Vincent Donnefort (2):
  tracing: Non-consuming read for trace remotes with an offline CPU
  tracing: selftests: Extend hotplug testing for trace remotes

 kernel/trace/trace_remote.c                   | 22 ++++-
 .../selftests/ftrace/test.d/remotes/functions | 19 +++-
 .../ftrace/test.d/remotes/hotplug.tc          | 88 +++++++++++++++++++
 .../test.d/remotes/hypervisor/hotplug.tc      | 11 +++
 .../selftests/ftrace/test.d/remotes/trace.tc  | 27 +-----
 .../ftrace/test.d/remotes/trace_pipe.tc       | 25 ------
 6 files changed, 134 insertions(+), 58 deletions(-)
 create mode 100644 tools/testing/selftests/ftrace/test.d/remotes/hotplug.tc
 create mode 100644 tools/testing/selftests/ftrace/test.d/remotes/hypervisor/hotplug.tc


base-commit: 5ad2ff071b5980f072a85c8114649218971c586e
-- 
2.53.0.1118.gaef5881109-goog

[PATCH v2 1/2] tracing: Non-consuming read for trace remotes with an offline CPU

[Vincent Donnefort]

When a trace_buffer is created while a CPU is offline, this CPU is
cleared from the trace_buffer CPU mask, preventing the creation of a
non-consuming iterator (ring_buffer_iter). For trace remotes, it means
the iterator fails to be allocated (-ENOMEM) even though there are
available ring buffers in the trace_buffer.

For non-consuming reads of trace remotes, skip missing ring_buffer_iter
to allow reading the available ring buffers.

Signed-off-by: Vincent Donnefort <vdonnefort@google.com>

diff --git a/kernel/trace/trace_remote.c b/kernel/trace/trace_remote.c
index 0d78e5f5fe98..d6c3f94d67cd 100644
--- a/kernel/trace/trace_remote.c
+++ b/kernel/trace/trace_remote.c
@@ -282,6 +282,14 @@ static void trace_remote_put(struct trace_remote *remote)
 	trace_remote_try_unload(remote);
 }
 
+static bool trace_remote_has_cpu(struct trace_remote *remote, int cpu)
+{
+	if (cpu == RING_BUFFER_ALL_CPUS)
+		return true;
+
+	return ring_buffer_poll_remote(remote->trace_buffer, cpu) == 0;
+}
+
 static void __poll_remote(struct work_struct *work)
 {
 	struct delayed_work *dwork = to_delayed_work(work);
@@ -324,6 +332,10 @@ static int __alloc_ring_buffer_iter(struct trace_remote_iterator *iter, int cpu)
 		iter->rb_iters[cpu] = ring_buffer_read_start(iter->remote->trace_buffer, cpu,
 							     GFP_KERNEL);
 		if (!iter->rb_iters[cpu]) {
+			/* This CPU isn't part of trace_buffer. Skip it */
+			if (!trace_remote_has_cpu(iter->remote, cpu))
+				continue;
+
 			__free_ring_buffer_iter(iter, RING_BUFFER_ALL_CPUS);
 			return -ENOMEM;
 		}
@@ -347,10 +359,10 @@ static struct trace_remote_iterator
 	if (ret)
 		return ERR_PTR(ret);
 
-	/* Test the CPU */
-	ret = ring_buffer_poll_remote(remote->trace_buffer, cpu);
-	if (ret)
+	if (!trace_remote_has_cpu(remote, cpu)) {
+		ret = -ENODEV;
 		goto err;
+	}
 
 	iter = kzalloc_obj(*iter);
 	if (iter) {
@@ -361,6 +373,7 @@ static struct trace_remote_iterator
 
 		switch (type) {
 		case TRI_CONSUMING:
+			ring_buffer_poll_remote(remote->trace_buffer, cpu);
 			INIT_DELAYED_WORK(&iter->poll_work, __poll_remote);
 			schedule_delayed_work(&iter->poll_work, msecs_to_jiffies(remote->poll_ms));
 			break;
@@ -476,6 +489,9 @@ __peek_event(struct trace_remote_iterator *iter, int cpu, u64 *ts, unsigned long
 		return ring_buffer_peek(iter->remote->trace_buffer, cpu, ts, lost_events);
 	case TRI_NONCONSUMING:
 		rb_iter = __get_rb_iter(iter, cpu);
+		if (!rb_iter)
+			return NULL;
+
 		rb_evt = ring_buffer_iter_peek(rb_iter, ts);
 		if (!rb_evt)
 			return NULL;
-- 
2.53.0.1118.gaef5881109-goog

[PATCH v2 2/2] tracing: selftests: Extend hotplug testing for trace remotes

[Vincent Donnefort]

The hotplug testing only tries reading a trace remote buffer, loaded
before a CPU is offline. Extend this testing to cover:

  * A trace remote buffer loaded after a CPU is offline.
  * A trace remote buffer loaded before a CPU is online.

Because of these added test cases, move the hotplug testing into a
separate hotplug.tc file.

Signed-off-by: Vincent Donnefort <vdonnefort@google.com>

diff --git a/tools/testing/selftests/ftrace/test.d/remotes/functions b/tools/testing/selftests/ftrace/test.d/remotes/functions
index 97a09d564a34..05224fac3653 100644
--- a/tools/testing/selftests/ftrace/test.d/remotes/functions
+++ b/tools/testing/selftests/ftrace/test.d/remotes/functions
@@ -24,12 +24,21 @@ setup_remote_test()
 
 assert_loaded()
 {
-	grep -q "(loaded)" buffer_size_kb
+	grep -q "(loaded)" buffer_size_kb || return 1
 }
 
 assert_unloaded()
 {
-	grep -q "(unloaded)" buffer_size_kb
+	grep -q "(unloaded)" buffer_size_kb || return 1
+}
+
+reload_remote()
+{
+	echo 0 > tracing_on
+	clear_trace
+	assert_unloaded
+	echo 1 > tracing_on
+	assert_loaded
 }
 
 dump_trace_pipe()
@@ -79,10 +88,12 @@ get_cpu_ids()
     sed -n 's/^processor\s*:\s*\([0-9]\+\).*/\1/p' /proc/cpuinfo
 }
 
-get_page_size() {
+get_page_size()
+{
     sed -ne 's/^.*data.*size:\([0-9][0-9]*\).*/\1/p' events/header_page
 }
 
-get_selftest_event_size() {
+get_selftest_event_size()
+{
     sed -ne 's/^.*field:.*;.*size:\([0-9][0-9]*\);.*/\1/p' events/*/selftest/format | awk '{s+=$1} END {print s}'
 }
diff --git a/tools/testing/selftests/ftrace/test.d/remotes/hotplug.tc b/tools/testing/selftests/ftrace/test.d/remotes/hotplug.tc
new file mode 100644
index 000000000000..145617eb8061
--- /dev/null
+++ b/tools/testing/selftests/ftrace/test.d/remotes/hotplug.tc
@@ -0,0 +1,88 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0
+# description: Test trace remote read with an offline CPU
+# requires: remotes/test
+
+. $TEST_DIR/remotes/functions
+
+hotunplug_one_cpu()
+{
+	[ "$(get_cpu_ids | wc -l)" -ge 2 ] || return 1
+
+	for cpu in $(get_cpu_ids); do
+		echo 0 > /sys/devices/system/cpu/cpu$cpu/online || return 1
+		break
+	done
+
+	echo $cpu
+}
+
+# Check non-consuming and consuming read
+check_read()
+{
+    for i in $(seq 1 8); do
+        echo $i > write_event
+    done
+
+    check_trace 1 8 trace
+
+    output=$(dump_trace_pipe)
+    check_trace 1 8 $output
+    rm $output
+}
+
+test_hotplug()
+{
+    echo 0 > trace
+    assert_loaded
+
+    #
+    # Test a trace buffer containing an offline CPU
+    #
+
+    cpu=$(hotunplug_one_cpu) || exit_unsupported
+    trap "echo 1 > /sys/devices/system/cpu/cpu$cpu/online" EXIT
+
+    check_read
+
+    #
+    # Test a trace buffer with a missing CPU
+    #
+
+    reload_remote
+
+    check_read
+
+    #
+    # Test a trace buffer with a CPU added later
+    #
+
+    echo 1 > /sys/devices/system/cpu/cpu$cpu/online
+    trap "" EXIT
+    assert_loaded
+
+    check_read
+
+    # Test if the ring-buffer for the newly added CPU is both writable and
+    # readable
+    for i in $(seq 1 8); do
+        taskset -c $cpu echo $i > write_event
+    done
+
+    cd per_cpu/cpu$cpu/
+
+    check_trace 1 8 trace
+
+    output=$(dump_trace_pipe)
+    check_trace 1 8 $output
+    rm $output
+
+    cd -
+}
+
+if [ -z "$SOURCE_REMOTE_TEST" ]; then
+    set -e
+
+    setup_remote_test
+    test_hotplug
+fi
diff --git a/tools/testing/selftests/ftrace/test.d/remotes/hypervisor/hotplug.tc b/tools/testing/selftests/ftrace/test.d/remotes/hypervisor/hotplug.tc
new file mode 100644
index 000000000000..580ec32c8f81
--- /dev/null
+++ b/tools/testing/selftests/ftrace/test.d/remotes/hypervisor/hotplug.tc
@@ -0,0 +1,11 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0
+# description: Test hypervisor trace read with an offline CPU
+# requires: remotes/hypervisor/write_event
+
+SOURCE_REMOTE_TEST=1
+. $TEST_DIR/remotes/hotplug.tc
+
+set -e
+setup_remote "hypervisor"
+test_hotplug
diff --git a/tools/testing/selftests/ftrace/test.d/remotes/trace.tc b/tools/testing/selftests/ftrace/test.d/remotes/trace.tc
index 170f7648732a..bc9377a70e8d 100644
--- a/tools/testing/selftests/ftrace/test.d/remotes/trace.tc
+++ b/tools/testing/selftests/ftrace/test.d/remotes/trace.tc
@@ -58,11 +58,7 @@ test_trace()
     #
 
     # Ensure the writer is not on the reader page by reloading the buffer
-    echo 0 > tracing_on
-    echo 0 > trace
-    assert_unloaded
-    echo 1 > tracing_on
-    assert_loaded
+    reload_remote
 
     # Ensure ring-buffer overflow by emitting events from the same CPU
     for cpu in $(get_cpu_ids); do
@@ -96,27 +92,6 @@ test_trace()
 
         cd - > /dev/null
     done
-
-    #
-    # Test with hotplug
-    #
-
-    [ "$(get_cpu_ids | wc -l)" -ge 2 ] || return 0
-
-    echo 0 > trace
-
-    for cpu in $(get_cpu_ids); do
-        echo 0 > /sys/devices/system/cpu/cpu$cpu/online || return 0
-        break
-    done
-
-    for i in $(seq 1 8); do
-        echo $i > write_event
-    done
-
-    check_trace 1 8 trace
-
-    echo 1 > /sys/devices/system/cpu/cpu$cpu/online
 }
 
 if [ -z "$SOURCE_REMOTE_TEST" ]; then
diff --git a/tools/testing/selftests/ftrace/test.d/remotes/trace_pipe.tc b/tools/testing/selftests/ftrace/test.d/remotes/trace_pipe.tc
index 669a7288ed7c..7f7b7b79c490 100644
--- a/tools/testing/selftests/ftrace/test.d/remotes/trace_pipe.tc
+++ b/tools/testing/selftests/ftrace/test.d/remotes/trace_pipe.tc
@@ -92,31 +92,6 @@ test_trace_pipe()
         rm $output
         cd - > /dev/null
     done
-
-    #
-    # Test interaction with hotplug
-    #
-
-    [ "$(get_cpu_ids | wc -l)" -ge 2 ] || return 0
-
-    echo 0 > trace
-
-    for cpu in $(get_cpu_ids); do
-        echo 0 > /sys/devices/system/cpu/cpu$cpu/online || return 0
-        break
-    done
-
-    for i in $(seq 1 8); do
-        echo $i > write_event
-    done
-
-    output=$(dump_trace_pipe)
-
-    check_trace 1 8 $output
-
-    rm $output
-
-    echo 1 > /sys/devices/system/cpu/cpu$cpu/online
 }
 
 if [ -z "$SOURCE_REMOTE_TEST" ]; then
-- 
2.53.0.1118.gaef5881109-goog

Re: [PATCH v2 0/2] Fix trace remotes read with an offline CPU

[Marc Zyngier]

On Wed, 01 Apr 2026 05:50:58 +0100, Vincent Donnefort wrote:
> This small series is fixing non-consuming read of a trace remote when the
> trace_buffer is created after a CPU is offline.
> 
> It also extends hotplug testing coverage to include this test case.
> 
> I have based this series on top of kvmarm/next which contains the hypervisor
> tracing patches.
> 
> [...]

Applied to next, thanks!

[1/2] tracing: Non-consuming read for trace remotes with an offline CPU
      commit: ce47b798ed1e44a6ae2c2966cdf7cba6b428083e
[2/2] tracing: selftests: Extend hotplug testing for trace remotes
      commit: ec07906bdc52848bd7dc93d1d44e642dcdc7a15a

Cheers,

	M.
-- 
Without deviation from the norm, progress is not possible.