* [PATCH v11 1/5] ring-buffer: Fix to update per-subbuf entries of persistent ring buffer
2026-03-19 9:12 [PATCH v11 0/5] ring-buffer: Making persistent ring buffers robust Masami Hiramatsu (Google)
@ 2026-03-19 9:12 ` Masami Hiramatsu (Google)
2026-03-19 9:12 ` [PATCH v11 2/5] ring-buffer: Flush and stop persistent ring buffer on panic Masami Hiramatsu (Google)
` (3 subsequent siblings)
4 siblings, 0 replies; 9+ messages in thread
From: Masami Hiramatsu (Google) @ 2026-03-19 9:12 UTC (permalink / raw)
To: Steven Rostedt
Cc: Masami Hiramatsu, Mathieu Desnoyers, linux-kernel,
linux-trace-kernel, Ian Rogers
From: Masami Hiramatsu (Google) <mhiramat@kernel.org>
Since the validation loop in rb_meta_validate_events() updates
the same cpu_buffer->head_page->entries, the other subbuf entries
are not updated.
Fix to use head_page to update the entries field, since it is the
cursor in this loop.
Fixes: 5f3b6e839f3c ("ring-buffer: Validate boot range memory events")
Cc: stable@vger.kernel.org
Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
---
0 files changed
diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c
index 96e0d80d492b..d6bebb782efc 100644
--- a/kernel/trace/ring_buffer.c
+++ b/kernel/trace/ring_buffer.c
@@ -2024,7 +2024,7 @@ static void rb_meta_validate_events(struct ring_buffer_per_cpu *cpu_buffer)
entries += ret;
entry_bytes += local_read(&head_page->page->commit);
- local_set(&cpu_buffer->head_page->entries, ret);
+ local_set(&head_page->entries, ret);
if (head_page == cpu_buffer->commit_page)
break;
^ permalink raw reply related [flat|nested] 9+ messages in thread* [PATCH v11 2/5] ring-buffer: Flush and stop persistent ring buffer on panic
2026-03-19 9:12 [PATCH v11 0/5] ring-buffer: Making persistent ring buffers robust Masami Hiramatsu (Google)
2026-03-19 9:12 ` [PATCH v11 1/5] ring-buffer: Fix to update per-subbuf entries of persistent ring buffer Masami Hiramatsu (Google)
@ 2026-03-19 9:12 ` Masami Hiramatsu (Google)
2026-03-19 9:12 ` [PATCH v11 3/5] ring-buffer: Skip invalid sub-buffers when validating persistent ring buffer Masami Hiramatsu (Google)
` (2 subsequent siblings)
4 siblings, 0 replies; 9+ messages in thread
From: Masami Hiramatsu (Google) @ 2026-03-19 9:12 UTC (permalink / raw)
To: Steven Rostedt
Cc: Masami Hiramatsu, Mathieu Desnoyers, linux-kernel,
linux-trace-kernel, Ian Rogers
From: Masami Hiramatsu (Google) <mhiramat@kernel.org>
On real hardware, panic and machine reboot may not flush hardware cache
to memory. This means the persistent ring buffer, which relies on a
coherent state of memory, may not have its events written to the buffer
and they may be lost. Moreover, there may be inconsistency with the
counters which are used for validation of the integrity of the
persistent ring buffer which may cause all data to be discarded.
To avoid this issue, stop recording of the ring buffer on panic and
flush the cache of the ring buffer's memory.
Fixes: e645535a954a ("tracing: Add option to use memmapped memory for trace boot instance")
Cc: stable@vger.kernel.org
Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
---
Changes in v11:
- Do nothing by default since flush_cache_vmap() does nothing on x86
but it can cause deadlock on some architectures via on_each_cpu()
because other CPUs will be stoppped when panic notifier is called.
Changes in v9:
- Fix typo of & to &&.
- Fix typo of "Generic"
Changes in v6:
- Introduce asm/ring_buffer.h for arch_ring_buffer_flush_range().
- Use flush_cache_vmap() instead of flush_cache_all().
Changes in v5:
- Use ring_buffer_record_off() instead of ring_buffer_record_disable().
- Use flush_cache_all() to ensure flush all cache.
Changes in v3:
- update patch description.
---
0 files changed
diff --git a/arch/alpha/include/asm/Kbuild b/arch/alpha/include/asm/Kbuild
index 483965c5a4de..b154b4e3dfa8 100644
--- a/arch/alpha/include/asm/Kbuild
+++ b/arch/alpha/include/asm/Kbuild
@@ -5,4 +5,5 @@ generic-y += agp.h
generic-y += asm-offsets.h
generic-y += kvm_para.h
generic-y += mcs_spinlock.h
+generic-y += ring_buffer.h
generic-y += text-patching.h
diff --git a/arch/arc/include/asm/Kbuild b/arch/arc/include/asm/Kbuild
index 4c69522e0328..483caacc6988 100644
--- a/arch/arc/include/asm/Kbuild
+++ b/arch/arc/include/asm/Kbuild
@@ -5,5 +5,6 @@ generic-y += extable.h
generic-y += kvm_para.h
generic-y += mcs_spinlock.h
generic-y += parport.h
+generic-y += ring_buffer.h
generic-y += user.h
generic-y += text-patching.h
diff --git a/arch/arm/include/asm/Kbuild b/arch/arm/include/asm/Kbuild
index 03657ff8fbe3..decad5f2c826 100644
--- a/arch/arm/include/asm/Kbuild
+++ b/arch/arm/include/asm/Kbuild
@@ -3,6 +3,7 @@ generic-y += early_ioremap.h
generic-y += extable.h
generic-y += flat.h
generic-y += parport.h
+generic-y += ring_buffer.h
generated-y += mach-types.h
generated-y += unistd-nr.h
diff --git a/arch/arm64/include/asm/ring_buffer.h b/arch/arm64/include/asm/ring_buffer.h
new file mode 100644
index 000000000000..62316c406888
--- /dev/null
+++ b/arch/arm64/include/asm/ring_buffer.h
@@ -0,0 +1,10 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef _ASM_ARM64_RING_BUFFER_H
+#define _ASM_ARM64_RING_BUFFER_H
+
+#include <asm/cacheflush.h>
+
+/* Flush D-cache on persistent ring buffer */
+#define arch_ring_buffer_flush_range(start, end) dcache_clean_pop(start, end)
+
+#endif /* _ASM_ARM64_RING_BUFFER_H */
diff --git a/arch/csky/include/asm/Kbuild b/arch/csky/include/asm/Kbuild
index 3a5c7f6e5aac..7dca0c6cdc84 100644
--- a/arch/csky/include/asm/Kbuild
+++ b/arch/csky/include/asm/Kbuild
@@ -9,6 +9,7 @@ generic-y += qrwlock.h
generic-y += qrwlock_types.h
generic-y += qspinlock.h
generic-y += parport.h
+generic-y += ring_buffer.h
generic-y += user.h
generic-y += vmlinux.lds.h
generic-y += text-patching.h
diff --git a/arch/hexagon/include/asm/Kbuild b/arch/hexagon/include/asm/Kbuild
index 1efa1e993d4b..0f887d4238ed 100644
--- a/arch/hexagon/include/asm/Kbuild
+++ b/arch/hexagon/include/asm/Kbuild
@@ -5,4 +5,5 @@ generic-y += extable.h
generic-y += iomap.h
generic-y += kvm_para.h
generic-y += mcs_spinlock.h
+generic-y += ring_buffer.h
generic-y += text-patching.h
diff --git a/arch/loongarch/include/asm/Kbuild b/arch/loongarch/include/asm/Kbuild
index 9034b583a88a..7e92957baf6a 100644
--- a/arch/loongarch/include/asm/Kbuild
+++ b/arch/loongarch/include/asm/Kbuild
@@ -10,5 +10,6 @@ generic-y += qrwlock.h
generic-y += user.h
generic-y += ioctl.h
generic-y += mmzone.h
+generic-y += ring_buffer.h
generic-y += statfs.h
generic-y += text-patching.h
diff --git a/arch/m68k/include/asm/Kbuild b/arch/m68k/include/asm/Kbuild
index b282e0dd8dc1..62543bf305ff 100644
--- a/arch/m68k/include/asm/Kbuild
+++ b/arch/m68k/include/asm/Kbuild
@@ -3,5 +3,6 @@ generated-y += syscall_table.h
generic-y += extable.h
generic-y += kvm_para.h
generic-y += mcs_spinlock.h
+generic-y += ring_buffer.h
generic-y += spinlock.h
generic-y += text-patching.h
diff --git a/arch/microblaze/include/asm/Kbuild b/arch/microblaze/include/asm/Kbuild
index 7178f990e8b3..0030309b47ad 100644
--- a/arch/microblaze/include/asm/Kbuild
+++ b/arch/microblaze/include/asm/Kbuild
@@ -5,6 +5,7 @@ generic-y += extable.h
generic-y += kvm_para.h
generic-y += mcs_spinlock.h
generic-y += parport.h
+generic-y += ring_buffer.h
generic-y += syscalls.h
generic-y += tlb.h
generic-y += user.h
diff --git a/arch/mips/include/asm/Kbuild b/arch/mips/include/asm/Kbuild
index 684569b2ecd6..9771c3d85074 100644
--- a/arch/mips/include/asm/Kbuild
+++ b/arch/mips/include/asm/Kbuild
@@ -12,5 +12,6 @@ generic-y += mcs_spinlock.h
generic-y += parport.h
generic-y += qrwlock.h
generic-y += qspinlock.h
+generic-y += ring_buffer.h
generic-y += user.h
generic-y += text-patching.h
diff --git a/arch/nios2/include/asm/Kbuild b/arch/nios2/include/asm/Kbuild
index 28004301c236..0a2530964413 100644
--- a/arch/nios2/include/asm/Kbuild
+++ b/arch/nios2/include/asm/Kbuild
@@ -5,6 +5,7 @@ generic-y += cmpxchg.h
generic-y += extable.h
generic-y += kvm_para.h
generic-y += mcs_spinlock.h
+generic-y += ring_buffer.h
generic-y += spinlock.h
generic-y += user.h
generic-y += text-patching.h
diff --git a/arch/openrisc/include/asm/Kbuild b/arch/openrisc/include/asm/Kbuild
index cef49d60d74c..8aa34621702d 100644
--- a/arch/openrisc/include/asm/Kbuild
+++ b/arch/openrisc/include/asm/Kbuild
@@ -8,4 +8,5 @@ generic-y += spinlock_types.h
generic-y += spinlock.h
generic-y += qrwlock_types.h
generic-y += qrwlock.h
+generic-y += ring_buffer.h
generic-y += user.h
diff --git a/arch/parisc/include/asm/Kbuild b/arch/parisc/include/asm/Kbuild
index 4fb596d94c89..d48d158f7241 100644
--- a/arch/parisc/include/asm/Kbuild
+++ b/arch/parisc/include/asm/Kbuild
@@ -4,4 +4,5 @@ generated-y += syscall_table_64.h
generic-y += agp.h
generic-y += kvm_para.h
generic-y += mcs_spinlock.h
+generic-y += ring_buffer.h
generic-y += user.h
diff --git a/arch/powerpc/include/asm/Kbuild b/arch/powerpc/include/asm/Kbuild
index 2e23533b67e3..805b5aeebb6f 100644
--- a/arch/powerpc/include/asm/Kbuild
+++ b/arch/powerpc/include/asm/Kbuild
@@ -5,4 +5,5 @@ generated-y += syscall_table_spu.h
generic-y += agp.h
generic-y += mcs_spinlock.h
generic-y += qrwlock.h
+generic-y += ring_buffer.h
generic-y += early_ioremap.h
diff --git a/arch/riscv/include/asm/Kbuild b/arch/riscv/include/asm/Kbuild
index bd5fc9403295..7721b63642f4 100644
--- a/arch/riscv/include/asm/Kbuild
+++ b/arch/riscv/include/asm/Kbuild
@@ -14,5 +14,6 @@ generic-y += ticket_spinlock.h
generic-y += qrwlock.h
generic-y += qrwlock_types.h
generic-y += qspinlock.h
+generic-y += ring_buffer.h
generic-y += user.h
generic-y += vmlinux.lds.h
diff --git a/arch/s390/include/asm/Kbuild b/arch/s390/include/asm/Kbuild
index 80bad7de7a04..0c1fc47c3ba0 100644
--- a/arch/s390/include/asm/Kbuild
+++ b/arch/s390/include/asm/Kbuild
@@ -7,3 +7,4 @@ generated-y += unistd_nr.h
generic-y += asm-offsets.h
generic-y += mcs_spinlock.h
generic-y += mmzone.h
+generic-y += ring_buffer.h
diff --git a/arch/sh/include/asm/Kbuild b/arch/sh/include/asm/Kbuild
index 4d3f10ed8275..f0403d3ee8ab 100644
--- a/arch/sh/include/asm/Kbuild
+++ b/arch/sh/include/asm/Kbuild
@@ -3,4 +3,5 @@ generated-y += syscall_table.h
generic-y += kvm_para.h
generic-y += mcs_spinlock.h
generic-y += parport.h
+generic-y += ring_buffer.h
generic-y += text-patching.h
diff --git a/arch/sparc/include/asm/Kbuild b/arch/sparc/include/asm/Kbuild
index 17ee8a273aa6..49c6bb326b75 100644
--- a/arch/sparc/include/asm/Kbuild
+++ b/arch/sparc/include/asm/Kbuild
@@ -4,4 +4,5 @@ generated-y += syscall_table_64.h
generic-y += agp.h
generic-y += kvm_para.h
generic-y += mcs_spinlock.h
+generic-y += ring_buffer.h
generic-y += text-patching.h
diff --git a/arch/um/include/asm/Kbuild b/arch/um/include/asm/Kbuild
index 1b9b82bbe322..2a1629ba8140 100644
--- a/arch/um/include/asm/Kbuild
+++ b/arch/um/include/asm/Kbuild
@@ -17,6 +17,7 @@ generic-y += module.lds.h
generic-y += parport.h
generic-y += percpu.h
generic-y += preempt.h
+generic-y += ring_buffer.h
generic-y += runtime-const.h
generic-y += softirq_stack.h
generic-y += switch_to.h
diff --git a/arch/x86/include/asm/Kbuild b/arch/x86/include/asm/Kbuild
index 4566000e15c4..078fd2c0d69d 100644
--- a/arch/x86/include/asm/Kbuild
+++ b/arch/x86/include/asm/Kbuild
@@ -14,3 +14,4 @@ generic-y += early_ioremap.h
generic-y += fprobe.h
generic-y += mcs_spinlock.h
generic-y += mmzone.h
+generic-y += ring_buffer.h
diff --git a/arch/xtensa/include/asm/Kbuild b/arch/xtensa/include/asm/Kbuild
index 13fe45dea296..e57af619263a 100644
--- a/arch/xtensa/include/asm/Kbuild
+++ b/arch/xtensa/include/asm/Kbuild
@@ -6,5 +6,6 @@ generic-y += mcs_spinlock.h
generic-y += parport.h
generic-y += qrwlock.h
generic-y += qspinlock.h
+generic-y += ring_buffer.h
generic-y += user.h
generic-y += text-patching.h
diff --git a/include/asm-generic/ring_buffer.h b/include/asm-generic/ring_buffer.h
new file mode 100644
index 000000000000..201d2aee1005
--- /dev/null
+++ b/include/asm-generic/ring_buffer.h
@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Generic arch dependent ring_buffer macros.
+ */
+#ifndef __ASM_GENERIC_RING_BUFFER_H__
+#define __ASM_GENERIC_RING_BUFFER_H__
+
+#include <linux/cacheflush.h>
+
+/* Flush cache on ring buffer range if needed. Do nothing by default. */
+#define arch_ring_buffer_flush_range(start, end) do { } while (0)
+
+#endif /* __ASM_GENERIC_RING_BUFFER_H__ */
diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c
index d6bebb782efc..3d2acaf75e79 100644
--- a/kernel/trace/ring_buffer.c
+++ b/kernel/trace/ring_buffer.c
@@ -7,6 +7,7 @@
#include <linux/ring_buffer_types.h>
#include <linux/sched/isolation.h>
#include <linux/trace_recursion.h>
+#include <linux/panic_notifier.h>
#include <linux/trace_events.h>
#include <linux/ring_buffer.h>
#include <linux/trace_clock.h>
@@ -31,6 +32,7 @@
#include <linux/oom.h>
#include <linux/mm.h>
+#include <asm/ring_buffer.h>
#include <asm/local64.h>
#include <asm/local.h>
#include <asm/setup.h>
@@ -559,6 +561,7 @@ struct trace_buffer {
unsigned long range_addr_start;
unsigned long range_addr_end;
+ struct notifier_block flush_nb;
struct ring_buffer_meta *meta;
@@ -2520,6 +2523,16 @@ static void rb_free_cpu_buffer(struct ring_buffer_per_cpu *cpu_buffer)
kfree(cpu_buffer);
}
+/* Stop recording on a persistent buffer and flush cache if needed. */
+static int rb_flush_buffer_cb(struct notifier_block *nb, unsigned long event, void *data)
+{
+ struct trace_buffer *buffer = container_of(nb, struct trace_buffer, flush_nb);
+
+ ring_buffer_record_off(buffer);
+ arch_ring_buffer_flush_range(buffer->range_addr_start, buffer->range_addr_end);
+ return NOTIFY_DONE;
+}
+
static struct trace_buffer *alloc_buffer(unsigned long size, unsigned flags,
int order, unsigned long start,
unsigned long end,
@@ -2650,6 +2663,12 @@ static struct trace_buffer *alloc_buffer(unsigned long size, unsigned flags,
mutex_init(&buffer->mutex);
+ /* Persistent ring buffer needs to flush cache before reboot. */
+ if (start && end) {
+ buffer->flush_nb.notifier_call = rb_flush_buffer_cb;
+ atomic_notifier_chain_register(&panic_notifier_list, &buffer->flush_nb);
+ }
+
return_ptr(buffer);
fail_free_buffers:
@@ -2748,6 +2767,9 @@ ring_buffer_free(struct trace_buffer *buffer)
{
int cpu;
+ if (buffer->range_addr_start && buffer->range_addr_end)
+ atomic_notifier_chain_unregister(&panic_notifier_list, &buffer->flush_nb);
+
cpuhp_state_remove_instance(CPUHP_TRACE_RB_PREPARE, &buffer->node);
irq_work_sync(&buffer->irq_work.work);
^ permalink raw reply related [flat|nested] 9+ messages in thread* [PATCH v11 3/5] ring-buffer: Skip invalid sub-buffers when validating persistent ring buffer
2026-03-19 9:12 [PATCH v11 0/5] ring-buffer: Making persistent ring buffers robust Masami Hiramatsu (Google)
2026-03-19 9:12 ` [PATCH v11 1/5] ring-buffer: Fix to update per-subbuf entries of persistent ring buffer Masami Hiramatsu (Google)
2026-03-19 9:12 ` [PATCH v11 2/5] ring-buffer: Flush and stop persistent ring buffer on panic Masami Hiramatsu (Google)
@ 2026-03-19 9:12 ` Masami Hiramatsu (Google)
2026-03-19 9:12 ` [PATCH v11 4/5] ring-buffer: Skip invalid sub-buffers when rewinding " Masami Hiramatsu (Google)
2026-03-19 9:12 ` [PATCH v11 5/5] ring-buffer: Add persistent ring buffer selftest Masami Hiramatsu (Google)
4 siblings, 0 replies; 9+ messages in thread
From: Masami Hiramatsu (Google) @ 2026-03-19 9:12 UTC (permalink / raw)
To: Steven Rostedt
Cc: Masami Hiramatsu, Mathieu Desnoyers, linux-kernel,
linux-trace-kernel, Ian Rogers
From: Masami Hiramatsu (Google) <mhiramat@kernel.org>
Skip invalid sub-buffers when validating the persistent ring buffer
instead of discarding the entire ring buffer. Only skipped buffers
are invalidated (cleared).
If the cache data in memory fails to be synchronized during a reboot,
the persistent ring buffer may become partially corrupted, but other
sub-buffers may still contain readable event data. Only discard the
subbuffers that are found to be corrupted.
Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
---
Changes in v11:
- Fix a typo.
Changes in v9:
- Add meta->subbuf_size check.
- Fix a typo.
- Handle invalid reader_page case.
Changes in v8:
- Add comment in rb_valudate_buffer()
- Clear the RB_MISSED_* flags in rb_valudate_buffer() instead of
skipping subbuf.
- Remove unused subbuf local variable from rb_cpu_meta_valid().
Changes in v7:
- Combined with Handling RB_MISSED_* flags patch, focus on validation at boot.
- Remove checking subbuffer data when validating metadata, because it should be done
later.
- Do not mark the discarded sub buffer page but just reset it.
Changes in v6:
- Show invalid page detection message once per CPU.
Changes in v5:
- Instead of showing errors for each page, just show the number
of discarded pages at last.
Changes in v3:
- Record missed data event on commit.
---
0 files changed
diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c
index 3d2acaf75e79..67826021867b 100644
--- a/kernel/trace/ring_buffer.c
+++ b/kernel/trace/ring_buffer.c
@@ -370,6 +370,12 @@ static __always_inline unsigned int rb_page_commit(struct buffer_page *bpage)
return local_read(&bpage->page->commit);
}
+/* Size is determined by what has been committed */
+static __always_inline unsigned int rb_page_size(struct buffer_page *bpage)
+{
+ return rb_page_commit(bpage) & ~RB_MISSED_MASK;
+}
+
static void free_buffer_page(struct buffer_page *bpage)
{
/* Range pages are not to be freed */
@@ -1762,7 +1768,6 @@ static bool rb_cpu_meta_valid(struct ring_buffer_cpu_meta *meta, int cpu,
unsigned long *subbuf_mask)
{
int subbuf_size = PAGE_SIZE;
- struct buffer_data_page *subbuf;
unsigned long buffers_start;
unsigned long buffers_end;
int i;
@@ -1770,6 +1775,11 @@ static bool rb_cpu_meta_valid(struct ring_buffer_cpu_meta *meta, int cpu,
if (!subbuf_mask)
return false;
+ if (meta->subbuf_size != PAGE_SIZE) {
+ pr_info("Ring buffer boot meta [%d] invalid subbuf_size\n", cpu);
+ return false;
+ }
+
buffers_start = meta->first_buffer;
buffers_end = meta->first_buffer + (subbuf_size * meta->nr_subbufs);
@@ -1786,11 +1796,12 @@ static bool rb_cpu_meta_valid(struct ring_buffer_cpu_meta *meta, int cpu,
return false;
}
- subbuf = rb_subbufs_from_meta(meta);
-
bitmap_clear(subbuf_mask, 0, meta->nr_subbufs);
- /* Is the meta buffers and the subbufs themselves have correct data? */
+ /*
+ * Ensure the meta::buffers array has correct data. The data in each subbufs
+ * are checked later in rb_meta_validate_events().
+ */
for (i = 0; i < meta->nr_subbufs; i++) {
if (meta->buffers[i] < 0 ||
meta->buffers[i] >= meta->nr_subbufs) {
@@ -1798,18 +1809,12 @@ static bool rb_cpu_meta_valid(struct ring_buffer_cpu_meta *meta, int cpu,
return false;
}
- if ((unsigned)local_read(&subbuf->commit) > subbuf_size) {
- pr_info("Ring buffer boot meta [%d] buffer invalid commit\n", cpu);
- return false;
- }
-
if (test_bit(meta->buffers[i], subbuf_mask)) {
pr_info("Ring buffer boot meta [%d] array has duplicates\n", cpu);
return false;
}
set_bit(meta->buffers[i], subbuf_mask);
- subbuf = (void *)subbuf + subbuf_size;
}
return true;
@@ -1873,13 +1878,22 @@ static int rb_read_data_buffer(struct buffer_data_page *dpage, int tail, int cpu
return events;
}
-static int rb_validate_buffer(struct buffer_data_page *dpage, int cpu)
+static int rb_validate_buffer(struct buffer_data_page *dpage, int cpu,
+ struct ring_buffer_cpu_meta *meta)
{
unsigned long long ts;
+ unsigned long tail;
u64 delta;
- int tail;
- tail = local_read(&dpage->commit);
+ /*
+ * When a sub-buffer is recovered from a read, the commit value may
+ * have RB_MISSED_* bits set, as these bits are reset on reuse.
+ * Even after clearing these bits, a commit value greater than the
+ * subbuf_size is considered invalid.
+ */
+ tail = local_read(&dpage->commit) & ~RB_MISSED_MASK;
+ if (tail > meta->subbuf_size)
+ return -1;
return rb_read_data_buffer(dpage, tail, cpu, &ts, &delta);
}
@@ -1890,6 +1904,7 @@ static void rb_meta_validate_events(struct ring_buffer_per_cpu *cpu_buffer)
struct buffer_page *head_page, *orig_head;
unsigned long entry_bytes = 0;
unsigned long entries = 0;
+ int discarded = 0;
int ret;
u64 ts;
int i;
@@ -1900,14 +1915,19 @@ static void rb_meta_validate_events(struct ring_buffer_per_cpu *cpu_buffer)
orig_head = head_page = cpu_buffer->head_page;
/* Do the reader page first */
- ret = rb_validate_buffer(cpu_buffer->reader_page->page, cpu_buffer->cpu);
+ ret = rb_validate_buffer(cpu_buffer->reader_page->page, cpu_buffer->cpu, meta);
if (ret < 0) {
- pr_info("Ring buffer reader page is invalid\n");
- goto invalid;
+ pr_info("Ring buffer meta [%d] invalid reader page detected\n",
+ cpu_buffer->cpu);
+ discarded++;
+ /* Instead of discard whole ring buffer, discard only this sub-buffer. */
+ local_set(&cpu_buffer->reader_page->entries, 0);
+ local_set(&cpu_buffer->reader_page->page->commit, 0);
+ } else {
+ entries += ret;
+ entry_bytes += rb_page_size(cpu_buffer->reader_page);
+ local_set(&cpu_buffer->reader_page->entries, ret);
}
- entries += ret;
- entry_bytes += local_read(&cpu_buffer->reader_page->page->commit);
- local_set(&cpu_buffer->reader_page->entries, ret);
ts = head_page->page->time_stamp;
@@ -1935,7 +1955,7 @@ static void rb_meta_validate_events(struct ring_buffer_per_cpu *cpu_buffer)
break;
/* Stop rewind if the page is invalid. */
- ret = rb_validate_buffer(head_page->page, cpu_buffer->cpu);
+ ret = rb_validate_buffer(head_page->page, cpu_buffer->cpu, meta);
if (ret < 0)
break;
@@ -2014,21 +2034,24 @@ static void rb_meta_validate_events(struct ring_buffer_per_cpu *cpu_buffer)
if (head_page == cpu_buffer->reader_page)
continue;
- ret = rb_validate_buffer(head_page->page, cpu_buffer->cpu);
+ ret = rb_validate_buffer(head_page->page, cpu_buffer->cpu, meta);
if (ret < 0) {
- pr_info("Ring buffer meta [%d] invalid buffer page\n",
- cpu_buffer->cpu);
- goto invalid;
- }
-
- /* If the buffer has content, update pages_touched */
- if (ret)
- local_inc(&cpu_buffer->pages_touched);
-
- entries += ret;
- entry_bytes += local_read(&head_page->page->commit);
- local_set(&head_page->entries, ret);
+ if (!discarded)
+ pr_info("Ring buffer meta [%d] invalid buffer page detected\n",
+ cpu_buffer->cpu);
+ discarded++;
+ /* Instead of discard whole ring buffer, discard only this sub-buffer. */
+ local_set(&head_page->entries, 0);
+ local_set(&head_page->page->commit, 0);
+ } else {
+ /* If the buffer has content, update pages_touched */
+ if (ret)
+ local_inc(&cpu_buffer->pages_touched);
+ entries += ret;
+ entry_bytes += rb_page_size(head_page);
+ local_set(&head_page->entries, ret);
+ }
if (head_page == cpu_buffer->commit_page)
break;
}
@@ -2042,7 +2065,8 @@ static void rb_meta_validate_events(struct ring_buffer_per_cpu *cpu_buffer)
local_set(&cpu_buffer->entries, entries);
local_set(&cpu_buffer->entries_bytes, entry_bytes);
- pr_info("Ring buffer meta [%d] is from previous boot!\n", cpu_buffer->cpu);
+ pr_info("Ring buffer meta [%d] is from previous boot! (%d pages discarded)\n",
+ cpu_buffer->cpu, discarded);
return;
invalid:
@@ -3329,12 +3353,6 @@ rb_iter_head_event(struct ring_buffer_iter *iter)
return NULL;
}
-/* Size is determined by what has been committed */
-static __always_inline unsigned rb_page_size(struct buffer_page *bpage)
-{
- return rb_page_commit(bpage) & ~RB_MISSED_MASK;
-}
-
static __always_inline unsigned
rb_commit_index(struct ring_buffer_per_cpu *cpu_buffer)
{
^ permalink raw reply related [flat|nested] 9+ messages in thread* [PATCH v11 4/5] ring-buffer: Skip invalid sub-buffers when rewinding persistent ring buffer
2026-03-19 9:12 [PATCH v11 0/5] ring-buffer: Making persistent ring buffers robust Masami Hiramatsu (Google)
` (2 preceding siblings ...)
2026-03-19 9:12 ` [PATCH v11 3/5] ring-buffer: Skip invalid sub-buffers when validating persistent ring buffer Masami Hiramatsu (Google)
@ 2026-03-19 9:12 ` Masami Hiramatsu (Google)
2026-03-22 10:13 ` kernel test robot
2026-03-22 23:18 ` kernel test robot
2026-03-19 9:12 ` [PATCH v11 5/5] ring-buffer: Add persistent ring buffer selftest Masami Hiramatsu (Google)
4 siblings, 2 replies; 9+ messages in thread
From: Masami Hiramatsu (Google) @ 2026-03-19 9:12 UTC (permalink / raw)
To: Steven Rostedt
Cc: Masami Hiramatsu, Mathieu Desnoyers, linux-kernel,
linux-trace-kernel, Ian Rogers
From: Masami Hiramatsu (Google) <mhiramat@kernel.org>
Skip invalid sub-buffers when rewinding the persistent ring buffer
instead of stopping the rewinding the ring buffer. The skipped
buffers are cleared.
To ensure the rewinding stops at the unused page, this also clears
buffer_data_page::time_stamp when tracing resets the buffer. This
allows us to identify unused pages and empty pages.
Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
---
Changes in v11:
- Reset timestamp when the buffer is invalid.
- When rewinding, skip subbuf page if timestamp is wrong and
check timestamp after validating buffer data page.
Changes in v10:
- Newly added.
---
0 files changed
diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c
index 67826021867b..b436d2982c5e 100644
--- a/kernel/trace/ring_buffer.c
+++ b/kernel/trace/ring_buffer.c
@@ -363,6 +363,7 @@ struct buffer_page {
static void rb_init_page(struct buffer_data_page *bpage)
{
local_set(&bpage->commit, 0);
+ bpage->time_stamp = 0;
}
static __always_inline unsigned int rb_page_commit(struct buffer_page *bpage)
@@ -1878,12 +1879,14 @@ static int rb_read_data_buffer(struct buffer_data_page *dpage, int tail, int cpu
return events;
}
-static int rb_validate_buffer(struct buffer_data_page *dpage, int cpu,
+static int rb_validate_buffer(struct buffer_page *bpage, int cpu,
struct ring_buffer_cpu_meta *meta)
{
+ struct buffer_data_page *dpage = bpage->page;
unsigned long long ts;
unsigned long tail;
u64 delta;
+ int ret = -1;
/*
* When a sub-buffer is recovered from a read, the commit value may
@@ -1892,9 +1895,17 @@ static int rb_validate_buffer(struct buffer_data_page *dpage, int cpu,
* subbuf_size is considered invalid.
*/
tail = local_read(&dpage->commit) & ~RB_MISSED_MASK;
- if (tail > meta->subbuf_size)
- return -1;
- return rb_read_data_buffer(dpage, tail, cpu, &ts, &delta);
+ if (tail <= meta->subbuf_size)
+ ret = rb_read_data_buffer(dpage, tail, cpu, &ts, &delta);
+
+ if (ret < 0) {
+ local_set(&bpage->entries, 0);
+ local_set(&bpage->page->commit, 0);
+ } else {
+ local_set(&bpage->entries, ret);
+ }
+
+ return ret;
}
/* If the meta data has been validated, now validate the events */
@@ -1915,18 +1926,14 @@ static void rb_meta_validate_events(struct ring_buffer_per_cpu *cpu_buffer)
orig_head = head_page = cpu_buffer->head_page;
/* Do the reader page first */
- ret = rb_validate_buffer(cpu_buffer->reader_page->page, cpu_buffer->cpu, meta);
+ ret = rb_validate_buffer(cpu_buffer->reader_page, cpu_buffer->cpu, meta);
if (ret < 0) {
pr_info("Ring buffer meta [%d] invalid reader page detected\n",
cpu_buffer->cpu);
discarded++;
- /* Instead of discard whole ring buffer, discard only this sub-buffer. */
- local_set(&cpu_buffer->reader_page->entries, 0);
- local_set(&cpu_buffer->reader_page->page->commit, 0);
} else {
entries += ret;
entry_bytes += rb_page_size(cpu_buffer->reader_page);
- local_set(&cpu_buffer->reader_page->entries, ret);
}
ts = head_page->page->time_stamp;
@@ -1945,26 +1952,33 @@ static void rb_meta_validate_events(struct ring_buffer_per_cpu *cpu_buffer)
if (head_page == cpu_buffer->tail_page)
break;
- /* Ensure the page has older data than head. */
- if (ts < head_page->page->time_stamp)
+ /* Rewind until unused page (no timestamp, no commit). */
+ if (!head_page->page->time_stamp && rb_page_commit(head_page) == 0)
break;
- ts = head_page->page->time_stamp;
- /* Ensure the page has correct timestamp and some data. */
- if (!ts || rb_page_commit(head_page) == 0)
- break;
-
- /* Stop rewind if the page is invalid. */
- ret = rb_validate_buffer(head_page->page, cpu_buffer->cpu, meta);
- if (ret < 0)
- break;
-
- /* Recover the number of entries and update stats. */
- local_set(&head_page->entries, ret);
- if (ret)
- local_inc(&cpu_buffer->pages_touched);
- entries += ret;
- entry_bytes += rb_page_commit(head_page);
+ /*
+ * Skip if the page is invalid, or its timestamp is newer than the
+ * previous valid page.
+ */
+ ret = rb_validate_buffer(head_page, cpu_buffer->cpu, meta);
+ if (ret >= 0 && ts < head_page->page->time_stamp) {
+ local_set(&bpage->entries, 0);
+ local_set(&bpage->page->commit, 0);
+ head_page->page->time_stamp = ts;
+ ret = -1;
+ }
+ if (ret < 0) {
+ if (!discarded)
+ pr_info("Ring buffer meta [%d] invalid buffer page detected\n",
+ cpu_buffer->cpu);
+ discarded++;
+ } else {
+ entries += ret;
+ entry_bytes += rb_page_size(head_page);
+ if (ret > 0)
+ local_inc(&cpu_buffer->pages_touched);
+ ts = head_page->page->time_stamp;
+ }
}
if (i)
pr_info("Ring buffer [%d] rewound %d pages\n", cpu_buffer->cpu, i);
@@ -2034,15 +2048,12 @@ static void rb_meta_validate_events(struct ring_buffer_per_cpu *cpu_buffer)
if (head_page == cpu_buffer->reader_page)
continue;
- ret = rb_validate_buffer(head_page->page, cpu_buffer->cpu, meta);
+ ret = rb_validate_buffer(head_page, cpu_buffer->cpu, meta);
if (ret < 0) {
if (!discarded)
pr_info("Ring buffer meta [%d] invalid buffer page detected\n",
cpu_buffer->cpu);
discarded++;
- /* Instead of discard whole ring buffer, discard only this sub-buffer. */
- local_set(&head_page->entries, 0);
- local_set(&head_page->page->commit, 0);
} else {
/* If the buffer has content, update pages_touched */
if (ret)
@@ -2050,7 +2061,6 @@ static void rb_meta_validate_events(struct ring_buffer_per_cpu *cpu_buffer)
entries += ret;
entry_bytes += rb_page_size(head_page);
- local_set(&head_page->entries, ret);
}
if (head_page == cpu_buffer->commit_page)
break;
@@ -2081,7 +2091,7 @@ static void rb_meta_validate_events(struct ring_buffer_per_cpu *cpu_buffer)
/* Reset all the subbuffers */
for (i = 0; i < meta->nr_subbufs - 1; i++, rb_inc_page(&head_page)) {
local_set(&head_page->entries, 0);
- local_set(&head_page->page->commit, 0);
+ rb_init_page(head_page->page);
}
}
^ permalink raw reply related [flat|nested] 9+ messages in thread* Re: [PATCH v11 4/5] ring-buffer: Skip invalid sub-buffers when rewinding persistent ring buffer
2026-03-19 9:12 ` [PATCH v11 4/5] ring-buffer: Skip invalid sub-buffers when rewinding " Masami Hiramatsu (Google)
@ 2026-03-22 10:13 ` kernel test robot
2026-03-22 23:18 ` kernel test robot
1 sibling, 0 replies; 9+ messages in thread
From: kernel test robot @ 2026-03-22 10:13 UTC (permalink / raw)
To: Masami Hiramatsu (Google), Steven Rostedt
Cc: oe-kbuild-all, Masami Hiramatsu, Mathieu Desnoyers, linux-kernel,
linux-trace-kernel, Ian Rogers
Hi Masami,
kernel test robot noticed the following build errors:
[auto build test ERROR on trace/for-next]
[also build test ERROR on geert-m68k/for-next geert-m68k/for-linus openrisc/for-next deller-parisc/for-next powerpc/next powerpc/fixes s390/features uml/next tip/x86/core linus/master uml/fixes v7.0-rc4 next-20260320]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Masami-Hiramatsu-Google/ring-buffer-Fix-to-update-per-subbuf-entries-of-persistent-ring-buffer/20260322-122412
base: https://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace for-next
patch link: https://lore.kernel.org/r/177391156211.193994.7531495945584650297.stgit%40mhiramat.tok.corp.google.com
patch subject: [PATCH v11 4/5] ring-buffer: Skip invalid sub-buffers when rewinding persistent ring buffer
config: arc-defconfig (https://download.01.org/0day-ci/archive/20260322/202603221806.j3AZggGX-lkp@intel.com/config)
compiler: arc-linux-gcc (GCC) 15.2.0
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20260322/202603221806.j3AZggGX-lkp@intel.com/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp@intel.com>
| Closes: https://lore.kernel.org/oe-kbuild-all/202603221806.j3AZggGX-lkp@intel.com/
All errors (new ones prefixed by >>):
In file included from ./arch/arc/include/generated/asm/local.h:1,
from include/linux/ring_buffer_types.h:5,
from kernel/trace/ring_buffer.c:7:
kernel/trace/ring_buffer.c: In function 'rb_meta_validate_events':
>> kernel/trace/ring_buffer.c:1965:36: error: 'bpage' undeclared (first use in this function); did you mean 'page'?
1965 | local_set(&bpage->entries, 0);
| ^~~~~
include/asm-generic/local.h:30:44: note: in definition of macro 'local_set'
30 | #define local_set(l,i) atomic_long_set((&(l)->a),(i))
| ^
kernel/trace/ring_buffer.c:1965:36: note: each undeclared identifier is reported only once for each function it appears in
1965 | local_set(&bpage->entries, 0);
| ^~~~~
include/asm-generic/local.h:30:44: note: in definition of macro 'local_set'
30 | #define local_set(l,i) atomic_long_set((&(l)->a),(i))
| ^
vim +1965 kernel/trace/ring_buffer.c
1910
1911 /* If the meta data has been validated, now validate the events */
1912 static void rb_meta_validate_events(struct ring_buffer_per_cpu *cpu_buffer)
1913 {
1914 struct ring_buffer_cpu_meta *meta = cpu_buffer->ring_meta;
1915 struct buffer_page *head_page, *orig_head;
1916 unsigned long entry_bytes = 0;
1917 unsigned long entries = 0;
1918 int discarded = 0;
1919 int ret;
1920 u64 ts;
1921 int i;
1922
1923 if (!meta || !meta->head_buffer)
1924 return;
1925
1926 orig_head = head_page = cpu_buffer->head_page;
1927
1928 /* Do the reader page first */
1929 ret = rb_validate_buffer(cpu_buffer->reader_page, cpu_buffer->cpu, meta);
1930 if (ret < 0) {
1931 pr_info("Ring buffer meta [%d] invalid reader page detected\n",
1932 cpu_buffer->cpu);
1933 discarded++;
1934 } else {
1935 entries += ret;
1936 entry_bytes += rb_page_size(cpu_buffer->reader_page);
1937 }
1938
1939 ts = head_page->page->time_stamp;
1940
1941 /*
1942 * Try to rewind the head so that we can read the pages which already
1943 * read in the previous boot.
1944 */
1945 if (head_page == cpu_buffer->tail_page)
1946 goto skip_rewind;
1947
1948 rb_dec_page(&head_page);
1949 for (i = 0; i < meta->nr_subbufs + 1; i++, rb_dec_page(&head_page)) {
1950
1951 /* Rewind until tail (writer) page. */
1952 if (head_page == cpu_buffer->tail_page)
1953 break;
1954
1955 /* Rewind until unused page (no timestamp, no commit). */
1956 if (!head_page->page->time_stamp && rb_page_commit(head_page) == 0)
1957 break;
1958
1959 /*
1960 * Skip if the page is invalid, or its timestamp is newer than the
1961 * previous valid page.
1962 */
1963 ret = rb_validate_buffer(head_page, cpu_buffer->cpu, meta);
1964 if (ret >= 0 && ts < head_page->page->time_stamp) {
> 1965 local_set(&bpage->entries, 0);
1966 local_set(&bpage->page->commit, 0);
1967 head_page->page->time_stamp = ts;
1968 ret = -1;
1969 }
1970 if (ret < 0) {
1971 if (!discarded)
1972 pr_info("Ring buffer meta [%d] invalid buffer page detected\n",
1973 cpu_buffer->cpu);
1974 discarded++;
1975 } else {
1976 entries += ret;
1977 entry_bytes += rb_page_size(head_page);
1978 if (ret > 0)
1979 local_inc(&cpu_buffer->pages_touched);
1980 ts = head_page->page->time_stamp;
1981 }
1982 }
1983 if (i)
1984 pr_info("Ring buffer [%d] rewound %d pages\n", cpu_buffer->cpu, i);
1985
1986 /* The last rewound page must be skipped. */
1987 if (head_page != orig_head)
1988 rb_inc_page(&head_page);
1989
1990 /*
1991 * If the ring buffer was rewound, then inject the reader page
1992 * into the location just before the original head page.
1993 */
1994 if (head_page != orig_head) {
1995 struct buffer_page *bpage = orig_head;
1996
1997 rb_dec_page(&bpage);
1998 /*
1999 * Insert the reader_page before the original head page.
2000 * Since the list encode RB_PAGE flags, general list
2001 * operations should be avoided.
2002 */
2003 cpu_buffer->reader_page->list.next = &orig_head->list;
2004 cpu_buffer->reader_page->list.prev = orig_head->list.prev;
2005 orig_head->list.prev = &cpu_buffer->reader_page->list;
2006 bpage->list.next = &cpu_buffer->reader_page->list;
2007
2008 /* Make the head_page the reader page */
2009 cpu_buffer->reader_page = head_page;
2010 bpage = head_page;
2011 rb_inc_page(&head_page);
2012 head_page->list.prev = bpage->list.prev;
2013 rb_dec_page(&bpage);
2014 bpage->list.next = &head_page->list;
2015 rb_set_list_to_head(&bpage->list);
2016 cpu_buffer->pages = &head_page->list;
2017
2018 cpu_buffer->head_page = head_page;
2019 meta->head_buffer = (unsigned long)head_page->page;
2020
2021 /* Reset all the indexes */
2022 bpage = cpu_buffer->reader_page;
2023 meta->buffers[0] = rb_meta_subbuf_idx(meta, bpage->page);
2024 bpage->id = 0;
2025
2026 for (i = 1, bpage = head_page; i < meta->nr_subbufs;
2027 i++, rb_inc_page(&bpage)) {
2028 meta->buffers[i] = rb_meta_subbuf_idx(meta, bpage->page);
2029 bpage->id = i;
2030 }
2031
2032 /* We'll restart verifying from orig_head */
2033 head_page = orig_head;
2034 }
2035
2036 skip_rewind:
2037 /* If the commit_buffer is the reader page, update the commit page */
2038 if (meta->commit_buffer == (unsigned long)cpu_buffer->reader_page->page) {
2039 cpu_buffer->commit_page = cpu_buffer->reader_page;
2040 /* Nothing more to do, the only page is the reader page */
2041 goto done;
2042 }
2043
2044 /* Iterate until finding the commit page */
2045 for (i = 0; i < meta->nr_subbufs + 1; i++, rb_inc_page(&head_page)) {
2046
2047 /* Reader page has already been done */
2048 if (head_page == cpu_buffer->reader_page)
2049 continue;
2050
2051 ret = rb_validate_buffer(head_page, cpu_buffer->cpu, meta);
2052 if (ret < 0) {
2053 if (!discarded)
2054 pr_info("Ring buffer meta [%d] invalid buffer page detected\n",
2055 cpu_buffer->cpu);
2056 discarded++;
2057 } else {
2058 /* If the buffer has content, update pages_touched */
2059 if (ret)
2060 local_inc(&cpu_buffer->pages_touched);
2061
2062 entries += ret;
2063 entry_bytes += rb_page_size(head_page);
2064 }
2065 if (head_page == cpu_buffer->commit_page)
2066 break;
2067 }
2068
2069 if (head_page != cpu_buffer->commit_page) {
2070 pr_info("Ring buffer meta [%d] commit page not found\n",
2071 cpu_buffer->cpu);
2072 goto invalid;
2073 }
2074 done:
2075 local_set(&cpu_buffer->entries, entries);
2076 local_set(&cpu_buffer->entries_bytes, entry_bytes);
2077
2078 pr_info("Ring buffer meta [%d] is from previous boot! (%d pages discarded)\n",
2079 cpu_buffer->cpu, discarded);
2080 return;
2081
2082 invalid:
2083 /* The content of the buffers are invalid, reset the meta data */
2084 meta->head_buffer = 0;
2085 meta->commit_buffer = 0;
2086
2087 /* Reset the reader page */
2088 local_set(&cpu_buffer->reader_page->entries, 0);
2089 local_set(&cpu_buffer->reader_page->page->commit, 0);
2090
2091 /* Reset all the subbuffers */
2092 for (i = 0; i < meta->nr_subbufs - 1; i++, rb_inc_page(&head_page)) {
2093 local_set(&head_page->entries, 0);
2094 rb_init_page(head_page->page);
2095 }
2096 }
2097
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
^ permalink raw reply [flat|nested] 9+ messages in thread* Re: [PATCH v11 4/5] ring-buffer: Skip invalid sub-buffers when rewinding persistent ring buffer
2026-03-19 9:12 ` [PATCH v11 4/5] ring-buffer: Skip invalid sub-buffers when rewinding " Masami Hiramatsu (Google)
2026-03-22 10:13 ` kernel test robot
@ 2026-03-22 23:18 ` kernel test robot
2026-03-23 11:50 ` Masami Hiramatsu
1 sibling, 1 reply; 9+ messages in thread
From: kernel test robot @ 2026-03-22 23:18 UTC (permalink / raw)
To: Masami Hiramatsu (Google), Steven Rostedt
Cc: llvm, oe-kbuild-all, Masami Hiramatsu, Mathieu Desnoyers,
linux-kernel, linux-trace-kernel, Ian Rogers
Hi Masami,
kernel test robot noticed the following build errors:
[auto build test ERROR on trace/for-next]
[also build test ERROR on geert-m68k/for-next geert-m68k/for-linus openrisc/for-next deller-parisc/for-next powerpc/next powerpc/fixes s390/features uml/next tip/x86/core uml/fixes v7.0-rc4 next-20260320]
[cannot apply to linus/master]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Masami-Hiramatsu-Google/ring-buffer-Fix-to-update-per-subbuf-entries-of-persistent-ring-buffer/20260322-122412
base: https://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace for-next
patch link: https://lore.kernel.org/r/177391156211.193994.7531495945584650297.stgit%40mhiramat.tok.corp.google.com
patch subject: [PATCH v11 4/5] ring-buffer: Skip invalid sub-buffers when rewinding persistent ring buffer
config: x86_64-kexec (https://download.01.org/0day-ci/archive/20260323/202603230725.uMAZiKJx-lkp@intel.com/config)
compiler: clang version 20.1.8 (https://github.com/llvm/llvm-project 87f0227cb60147a26a1eeb4fb06e3b505e9c7261)
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20260323/202603230725.uMAZiKJx-lkp@intel.com/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp@intel.com>
| Closes: https://lore.kernel.org/oe-kbuild-all/202603230725.uMAZiKJx-lkp@intel.com/
All errors (new ones prefixed by >>):
>> kernel/trace/ring_buffer.c:1965:15: error: use of undeclared identifier 'bpage'
1965 | local_set(&bpage->entries, 0);
| ^
kernel/trace/ring_buffer.c:1966:15: error: use of undeclared identifier 'bpage'
1966 | local_set(&bpage->page->commit, 0);
| ^
2 errors generated.
vim +/bpage +1965 kernel/trace/ring_buffer.c
1910
1911 /* If the meta data has been validated, now validate the events */
1912 static void rb_meta_validate_events(struct ring_buffer_per_cpu *cpu_buffer)
1913 {
1914 struct ring_buffer_cpu_meta *meta = cpu_buffer->ring_meta;
1915 struct buffer_page *head_page, *orig_head;
1916 unsigned long entry_bytes = 0;
1917 unsigned long entries = 0;
1918 int discarded = 0;
1919 int ret;
1920 u64 ts;
1921 int i;
1922
1923 if (!meta || !meta->head_buffer)
1924 return;
1925
1926 orig_head = head_page = cpu_buffer->head_page;
1927
1928 /* Do the reader page first */
1929 ret = rb_validate_buffer(cpu_buffer->reader_page, cpu_buffer->cpu, meta);
1930 if (ret < 0) {
1931 pr_info("Ring buffer meta [%d] invalid reader page detected\n",
1932 cpu_buffer->cpu);
1933 discarded++;
1934 } else {
1935 entries += ret;
1936 entry_bytes += rb_page_size(cpu_buffer->reader_page);
1937 }
1938
1939 ts = head_page->page->time_stamp;
1940
1941 /*
1942 * Try to rewind the head so that we can read the pages which already
1943 * read in the previous boot.
1944 */
1945 if (head_page == cpu_buffer->tail_page)
1946 goto skip_rewind;
1947
1948 rb_dec_page(&head_page);
1949 for (i = 0; i < meta->nr_subbufs + 1; i++, rb_dec_page(&head_page)) {
1950
1951 /* Rewind until tail (writer) page. */
1952 if (head_page == cpu_buffer->tail_page)
1953 break;
1954
1955 /* Rewind until unused page (no timestamp, no commit). */
1956 if (!head_page->page->time_stamp && rb_page_commit(head_page) == 0)
1957 break;
1958
1959 /*
1960 * Skip if the page is invalid, or its timestamp is newer than the
1961 * previous valid page.
1962 */
1963 ret = rb_validate_buffer(head_page, cpu_buffer->cpu, meta);
1964 if (ret >= 0 && ts < head_page->page->time_stamp) {
> 1965 local_set(&bpage->entries, 0);
1966 local_set(&bpage->page->commit, 0);
1967 head_page->page->time_stamp = ts;
1968 ret = -1;
1969 }
1970 if (ret < 0) {
1971 if (!discarded)
1972 pr_info("Ring buffer meta [%d] invalid buffer page detected\n",
1973 cpu_buffer->cpu);
1974 discarded++;
1975 } else {
1976 entries += ret;
1977 entry_bytes += rb_page_size(head_page);
1978 if (ret > 0)
1979 local_inc(&cpu_buffer->pages_touched);
1980 ts = head_page->page->time_stamp;
1981 }
1982 }
1983 if (i)
1984 pr_info("Ring buffer [%d] rewound %d pages\n", cpu_buffer->cpu, i);
1985
1986 /* The last rewound page must be skipped. */
1987 if (head_page != orig_head)
1988 rb_inc_page(&head_page);
1989
1990 /*
1991 * If the ring buffer was rewound, then inject the reader page
1992 * into the location just before the original head page.
1993 */
1994 if (head_page != orig_head) {
1995 struct buffer_page *bpage = orig_head;
1996
1997 rb_dec_page(&bpage);
1998 /*
1999 * Insert the reader_page before the original head page.
2000 * Since the list encode RB_PAGE flags, general list
2001 * operations should be avoided.
2002 */
2003 cpu_buffer->reader_page->list.next = &orig_head->list;
2004 cpu_buffer->reader_page->list.prev = orig_head->list.prev;
2005 orig_head->list.prev = &cpu_buffer->reader_page->list;
2006 bpage->list.next = &cpu_buffer->reader_page->list;
2007
2008 /* Make the head_page the reader page */
2009 cpu_buffer->reader_page = head_page;
2010 bpage = head_page;
2011 rb_inc_page(&head_page);
2012 head_page->list.prev = bpage->list.prev;
2013 rb_dec_page(&bpage);
2014 bpage->list.next = &head_page->list;
2015 rb_set_list_to_head(&bpage->list);
2016 cpu_buffer->pages = &head_page->list;
2017
2018 cpu_buffer->head_page = head_page;
2019 meta->head_buffer = (unsigned long)head_page->page;
2020
2021 /* Reset all the indexes */
2022 bpage = cpu_buffer->reader_page;
2023 meta->buffers[0] = rb_meta_subbuf_idx(meta, bpage->page);
2024 bpage->id = 0;
2025
2026 for (i = 1, bpage = head_page; i < meta->nr_subbufs;
2027 i++, rb_inc_page(&bpage)) {
2028 meta->buffers[i] = rb_meta_subbuf_idx(meta, bpage->page);
2029 bpage->id = i;
2030 }
2031
2032 /* We'll restart verifying from orig_head */
2033 head_page = orig_head;
2034 }
2035
2036 skip_rewind:
2037 /* If the commit_buffer is the reader page, update the commit page */
2038 if (meta->commit_buffer == (unsigned long)cpu_buffer->reader_page->page) {
2039 cpu_buffer->commit_page = cpu_buffer->reader_page;
2040 /* Nothing more to do, the only page is the reader page */
2041 goto done;
2042 }
2043
2044 /* Iterate until finding the commit page */
2045 for (i = 0; i < meta->nr_subbufs + 1; i++, rb_inc_page(&head_page)) {
2046
2047 /* Reader page has already been done */
2048 if (head_page == cpu_buffer->reader_page)
2049 continue;
2050
2051 ret = rb_validate_buffer(head_page, cpu_buffer->cpu, meta);
2052 if (ret < 0) {
2053 if (!discarded)
2054 pr_info("Ring buffer meta [%d] invalid buffer page detected\n",
2055 cpu_buffer->cpu);
2056 discarded++;
2057 } else {
2058 /* If the buffer has content, update pages_touched */
2059 if (ret)
2060 local_inc(&cpu_buffer->pages_touched);
2061
2062 entries += ret;
2063 entry_bytes += rb_page_size(head_page);
2064 }
2065 if (head_page == cpu_buffer->commit_page)
2066 break;
2067 }
2068
2069 if (head_page != cpu_buffer->commit_page) {
2070 pr_info("Ring buffer meta [%d] commit page not found\n",
2071 cpu_buffer->cpu);
2072 goto invalid;
2073 }
2074 done:
2075 local_set(&cpu_buffer->entries, entries);
2076 local_set(&cpu_buffer->entries_bytes, entry_bytes);
2077
2078 pr_info("Ring buffer meta [%d] is from previous boot! (%d pages discarded)\n",
2079 cpu_buffer->cpu, discarded);
2080 return;
2081
2082 invalid:
2083 /* The content of the buffers are invalid, reset the meta data */
2084 meta->head_buffer = 0;
2085 meta->commit_buffer = 0;
2086
2087 /* Reset the reader page */
2088 local_set(&cpu_buffer->reader_page->entries, 0);
2089 local_set(&cpu_buffer->reader_page->page->commit, 0);
2090
2091 /* Reset all the subbuffers */
2092 for (i = 0; i < meta->nr_subbufs - 1; i++, rb_inc_page(&head_page)) {
2093 local_set(&head_page->entries, 0);
2094 rb_init_page(head_page->page);
2095 }
2096 }
2097
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
^ permalink raw reply [flat|nested] 9+ messages in thread* Re: [PATCH v11 4/5] ring-buffer: Skip invalid sub-buffers when rewinding persistent ring buffer
2026-03-22 23:18 ` kernel test robot
@ 2026-03-23 11:50 ` Masami Hiramatsu
0 siblings, 0 replies; 9+ messages in thread
From: Masami Hiramatsu @ 2026-03-23 11:50 UTC (permalink / raw)
To: kernel test robot
Cc: Steven Rostedt, llvm, oe-kbuild-all, Mathieu Desnoyers,
linux-kernel, linux-trace-kernel, Ian Rogers
On Mon, 23 Mar 2026 07:18:07 +0800
kernel test robot <lkp@intel.com> wrote:
> Hi Masami,
>
> kernel test robot noticed the following build errors:
>
> [auto build test ERROR on trace/for-next]
> [also build test ERROR on geert-m68k/for-next geert-m68k/for-linus openrisc/for-next deller-parisc/for-next powerpc/next powerpc/fixes s390/features uml/next tip/x86/core uml/fixes v7.0-rc4 next-20260320]
> [cannot apply to linus/master]
> [If your patch is applied to the wrong git tree, kindly drop us a note.
> And when submitting patch, we suggest to use '--base' as documented in
> https://git-scm.com/docs/git-format-patch#_base_tree_information]
>
> url: https://github.com/intel-lab-lkp/linux/commits/Masami-Hiramatsu-Google/ring-buffer-Fix-to-update-per-subbuf-entries-of-persistent-ring-buffer/20260322-122412
> base: https://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace for-next
> patch link: https://lore.kernel.org/r/177391156211.193994.7531495945584650297.stgit%40mhiramat.tok.corp.google.com
> patch subject: [PATCH v11 4/5] ring-buffer: Skip invalid sub-buffers when rewinding persistent ring buffer
> config: x86_64-kexec (https://download.01.org/0day-ci/archive/20260323/202603230725.uMAZiKJx-lkp@intel.com/config)
> compiler: clang version 20.1.8 (https://github.com/llvm/llvm-project 87f0227cb60147a26a1eeb4fb06e3b505e9c7261)
> reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20260323/202603230725.uMAZiKJx-lkp@intel.com/reproduce)
>
> If you fix the issue in a separate patch/commit (i.e. not just a new version of
> the same patch/commit), kindly add following tags
> | Reported-by: kernel test robot <lkp@intel.com>
> | Closes: https://lore.kernel.org/oe-kbuild-all/202603230725.uMAZiKJx-lkp@intel.com/
>
> All errors (new ones prefixed by >>):
>
> >> kernel/trace/ring_buffer.c:1965:15: error: use of undeclared identifier 'bpage'
> 1965 | local_set(&bpage->entries, 0);
> | ^
> kernel/trace/ring_buffer.c:1966:15: error: use of undeclared identifier 'bpage'
> 1966 | local_set(&bpage->page->commit, 0);
> | ^
> 2 errors generated.
>
>
> vim +/bpage +1965 kernel/trace/ring_buffer.c
>
> 1910
> 1911 /* If the meta data has been validated, now validate the events */
> 1912 static void rb_meta_validate_events(struct ring_buffer_per_cpu *cpu_buffer)
> 1913 {
> 1914 struct ring_buffer_cpu_meta *meta = cpu_buffer->ring_meta;
> 1915 struct buffer_page *head_page, *orig_head;
> 1916 unsigned long entry_bytes = 0;
> 1917 unsigned long entries = 0;
> 1918 int discarded = 0;
> 1919 int ret;
> 1920 u64 ts;
> 1921 int i;
> 1922
> 1923 if (!meta || !meta->head_buffer)
> 1924 return;
> 1925
> 1926 orig_head = head_page = cpu_buffer->head_page;
> 1927
> 1928 /* Do the reader page first */
> 1929 ret = rb_validate_buffer(cpu_buffer->reader_page, cpu_buffer->cpu, meta);
> 1930 if (ret < 0) {
> 1931 pr_info("Ring buffer meta [%d] invalid reader page detected\n",
> 1932 cpu_buffer->cpu);
> 1933 discarded++;
> 1934 } else {
> 1935 entries += ret;
> 1936 entry_bytes += rb_page_size(cpu_buffer->reader_page);
> 1937 }
> 1938
> 1939 ts = head_page->page->time_stamp;
> 1940
> 1941 /*
> 1942 * Try to rewind the head so that we can read the pages which already
> 1943 * read in the previous boot.
> 1944 */
> 1945 if (head_page == cpu_buffer->tail_page)
> 1946 goto skip_rewind;
> 1947
> 1948 rb_dec_page(&head_page);
> 1949 for (i = 0; i < meta->nr_subbufs + 1; i++, rb_dec_page(&head_page)) {
> 1950
> 1951 /* Rewind until tail (writer) page. */
> 1952 if (head_page == cpu_buffer->tail_page)
> 1953 break;
> 1954
> 1955 /* Rewind until unused page (no timestamp, no commit). */
> 1956 if (!head_page->page->time_stamp && rb_page_commit(head_page) == 0)
> 1957 break;
> 1958
> 1959 /*
> 1960 * Skip if the page is invalid, or its timestamp is newer than the
> 1961 * previous valid page.
> 1962 */
> 1963 ret = rb_validate_buffer(head_page, cpu_buffer->cpu, meta);
> 1964 if (ret >= 0 && ts < head_page->page->time_stamp) {
> > 1965 local_set(&bpage->entries, 0);
> 1966 local_set(&bpage->page->commit, 0);
Ooops, sorry, I made a copy & paste mistake. this should be head_page->entries and head_page->page_commit.
Let me send v12 on the latest tracing/fixes.
Thanks,
> 1967 head_page->page->time_stamp = ts;
> 1968 ret = -1;
> 1969 }
> 1970 if (ret < 0) {
> 1971 if (!discarded)
> 1972 pr_info("Ring buffer meta [%d] invalid buffer page detected\n",
> 1973 cpu_buffer->cpu);
> 1974 discarded++;
> 1975 } else {
> 1976 entries += ret;
> 1977 entry_bytes += rb_page_size(head_page);
> 1978 if (ret > 0)
> 1979 local_inc(&cpu_buffer->pages_touched);
> 1980 ts = head_page->page->time_stamp;
> 1981 }
> 1982 }
> 1983 if (i)
> 1984 pr_info("Ring buffer [%d] rewound %d pages\n", cpu_buffer->cpu, i);
> 1985
> 1986 /* The last rewound page must be skipped. */
> 1987 if (head_page != orig_head)
> 1988 rb_inc_page(&head_page);
> 1989
> 1990 /*
> 1991 * If the ring buffer was rewound, then inject the reader page
> 1992 * into the location just before the original head page.
> 1993 */
> 1994 if (head_page != orig_head) {
> 1995 struct buffer_page *bpage = orig_head;
> 1996
> 1997 rb_dec_page(&bpage);
> 1998 /*
> 1999 * Insert the reader_page before the original head page.
> 2000 * Since the list encode RB_PAGE flags, general list
> 2001 * operations should be avoided.
> 2002 */
> 2003 cpu_buffer->reader_page->list.next = &orig_head->list;
> 2004 cpu_buffer->reader_page->list.prev = orig_head->list.prev;
> 2005 orig_head->list.prev = &cpu_buffer->reader_page->list;
> 2006 bpage->list.next = &cpu_buffer->reader_page->list;
> 2007
> 2008 /* Make the head_page the reader page */
> 2009 cpu_buffer->reader_page = head_page;
> 2010 bpage = head_page;
> 2011 rb_inc_page(&head_page);
> 2012 head_page->list.prev = bpage->list.prev;
> 2013 rb_dec_page(&bpage);
> 2014 bpage->list.next = &head_page->list;
> 2015 rb_set_list_to_head(&bpage->list);
> 2016 cpu_buffer->pages = &head_page->list;
> 2017
> 2018 cpu_buffer->head_page = head_page;
> 2019 meta->head_buffer = (unsigned long)head_page->page;
> 2020
> 2021 /* Reset all the indexes */
> 2022 bpage = cpu_buffer->reader_page;
> 2023 meta->buffers[0] = rb_meta_subbuf_idx(meta, bpage->page);
> 2024 bpage->id = 0;
> 2025
> 2026 for (i = 1, bpage = head_page; i < meta->nr_subbufs;
> 2027 i++, rb_inc_page(&bpage)) {
> 2028 meta->buffers[i] = rb_meta_subbuf_idx(meta, bpage->page);
> 2029 bpage->id = i;
> 2030 }
> 2031
> 2032 /* We'll restart verifying from orig_head */
> 2033 head_page = orig_head;
> 2034 }
> 2035
> 2036 skip_rewind:
> 2037 /* If the commit_buffer is the reader page, update the commit page */
> 2038 if (meta->commit_buffer == (unsigned long)cpu_buffer->reader_page->page) {
> 2039 cpu_buffer->commit_page = cpu_buffer->reader_page;
> 2040 /* Nothing more to do, the only page is the reader page */
> 2041 goto done;
> 2042 }
> 2043
> 2044 /* Iterate until finding the commit page */
> 2045 for (i = 0; i < meta->nr_subbufs + 1; i++, rb_inc_page(&head_page)) {
> 2046
> 2047 /* Reader page has already been done */
> 2048 if (head_page == cpu_buffer->reader_page)
> 2049 continue;
> 2050
> 2051 ret = rb_validate_buffer(head_page, cpu_buffer->cpu, meta);
> 2052 if (ret < 0) {
> 2053 if (!discarded)
> 2054 pr_info("Ring buffer meta [%d] invalid buffer page detected\n",
> 2055 cpu_buffer->cpu);
> 2056 discarded++;
> 2057 } else {
> 2058 /* If the buffer has content, update pages_touched */
> 2059 if (ret)
> 2060 local_inc(&cpu_buffer->pages_touched);
> 2061
> 2062 entries += ret;
> 2063 entry_bytes += rb_page_size(head_page);
> 2064 }
> 2065 if (head_page == cpu_buffer->commit_page)
> 2066 break;
> 2067 }
> 2068
> 2069 if (head_page != cpu_buffer->commit_page) {
> 2070 pr_info("Ring buffer meta [%d] commit page not found\n",
> 2071 cpu_buffer->cpu);
> 2072 goto invalid;
> 2073 }
> 2074 done:
> 2075 local_set(&cpu_buffer->entries, entries);
> 2076 local_set(&cpu_buffer->entries_bytes, entry_bytes);
> 2077
> 2078 pr_info("Ring buffer meta [%d] is from previous boot! (%d pages discarded)\n",
> 2079 cpu_buffer->cpu, discarded);
> 2080 return;
> 2081
> 2082 invalid:
> 2083 /* The content of the buffers are invalid, reset the meta data */
> 2084 meta->head_buffer = 0;
> 2085 meta->commit_buffer = 0;
> 2086
> 2087 /* Reset the reader page */
> 2088 local_set(&cpu_buffer->reader_page->entries, 0);
> 2089 local_set(&cpu_buffer->reader_page->page->commit, 0);
> 2090
> 2091 /* Reset all the subbuffers */
> 2092 for (i = 0; i < meta->nr_subbufs - 1; i++, rb_inc_page(&head_page)) {
> 2093 local_set(&head_page->entries, 0);
> 2094 rb_init_page(head_page->page);
> 2095 }
> 2096 }
> 2097
>
> --
> 0-DAY CI Kernel Test Service
> https://github.com/intel/lkp-tests/wiki
>
--
Masami Hiramatsu (Google) <mhiramat@kernel.org>
^ permalink raw reply [flat|nested] 9+ messages in thread
* [PATCH v11 5/5] ring-buffer: Add persistent ring buffer selftest
2026-03-19 9:12 [PATCH v11 0/5] ring-buffer: Making persistent ring buffers robust Masami Hiramatsu (Google)
` (3 preceding siblings ...)
2026-03-19 9:12 ` [PATCH v11 4/5] ring-buffer: Skip invalid sub-buffers when rewinding " Masami Hiramatsu (Google)
@ 2026-03-19 9:12 ` Masami Hiramatsu (Google)
4 siblings, 0 replies; 9+ messages in thread
From: Masami Hiramatsu (Google) @ 2026-03-19 9:12 UTC (permalink / raw)
To: Steven Rostedt
Cc: Masami Hiramatsu, Mathieu Desnoyers, linux-kernel,
linux-trace-kernel, Ian Rogers
From: Masami Hiramatsu (Google) <mhiramat@kernel.org>
Add a self-destractive test for the persistent ring buffer. This
will invalidate some sub-buffer pages in the persistent ring buffer
when kernel gets panic, and check whether the number of detected
invalid pages and the total entry_bytes are the same as record
after reboot.
This can ensure the kernel correctly recover partially corrupted
persistent ring buffer when boot.
The test only runs on the persistent ring buffer whose name is
"ptracingtest". And user has to fill it up with events before
kernel panics.
To run the test, enable CONFIG_RING_BUFFER_PERSISTENT_SELFTEST
and you have to setup the kernel cmdline;
reserve_mem=20M:2M:trace trace_instance=ptracingtest^traceoff@trace
panic=1
And run following commands after the 1st boot;
cd /sys/kernel/tracing/instances/ptracingtest
echo 1 > tracing_on
echo 1 > events/enable
sleep 3
echo c > /proc/sysrq-trigger
After panic message, the kernel will reboot and run the verification
on the persistent ring buffer, e.g.
Ring buffer meta [2] invalid buffer page detected
Ring buffer meta [2] is from previous boot! (318 pages discarded)
Ring buffer testing [2] invalid pages: PASSED (318/318)
Ring buffer testing [2] entry_bytes: PASSED (1300476/1300476)
Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
---
Changes in v10:
- Add entry_bytes test.
- Do not compile test code if CONFIG_RING_BUFFER_PERSISTENT_SELFTEST=n.
Changes in v9:
- Test also reader pages.
---
0 files changed
diff --git a/include/linux/ring_buffer.h b/include/linux/ring_buffer.h
index 994f52b34344..0670742b2d60 100644
--- a/include/linux/ring_buffer.h
+++ b/include/linux/ring_buffer.h
@@ -238,6 +238,7 @@ int ring_buffer_subbuf_size_get(struct trace_buffer *buffer);
enum ring_buffer_flags {
RB_FL_OVERWRITE = 1 << 0,
+ RB_FL_TESTING = 1 << 1,
};
#ifdef CONFIG_RING_BUFFER
diff --git a/kernel/trace/Kconfig b/kernel/trace/Kconfig
index e130da35808f..094d5511bb17 100644
--- a/kernel/trace/Kconfig
+++ b/kernel/trace/Kconfig
@@ -1202,6 +1202,21 @@ config RING_BUFFER_VALIDATE_TIME_DELTAS
Only say Y if you understand what this does, and you
still want it enabled. Otherwise say N
+config RING_BUFFER_PERSISTENT_SELFTEST
+ bool "Enable persistent ring buffer selftest"
+ depends on RING_BUFFER
+ help
+ Run a selftest on the persistent ring buffer which names
+ "ptracingtest" (and its backup) when panic_on_reboot by
+ invalidating ring buffer pages.
+ Note that user has to enable events on the persistent ring
+ buffer manually to fill up ring buffers before rebooting.
+ Since this invalidates the data on test target ring buffer,
+ "ptracingtest" persistent ring buffer must not be used for
+ actual tracing, but only for testing.
+
+ If unsure, say N
+
config MMIOTRACE_TEST
tristate "Test module for mmiotrace"
depends on MMIOTRACE && m
diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c
index b436d2982c5e..cfd895d6b56e 100644
--- a/kernel/trace/ring_buffer.c
+++ b/kernel/trace/ring_buffer.c
@@ -64,6 +64,10 @@ struct ring_buffer_cpu_meta {
unsigned long commit_buffer;
__u32 subbuf_size;
__u32 nr_subbufs;
+#ifdef CONFIG_RING_BUFFER_PERSISTENT_SELFTEST
+ __u32 nr_invalid;
+ __u32 entry_bytes;
+#endif
int buffers[];
};
@@ -2077,6 +2081,19 @@ static void rb_meta_validate_events(struct ring_buffer_per_cpu *cpu_buffer)
pr_info("Ring buffer meta [%d] is from previous boot! (%d pages discarded)\n",
cpu_buffer->cpu, discarded);
+
+#ifdef CONFIG_RING_BUFFER_PERSISTENT_SELFTEST
+ if (meta->nr_invalid)
+ pr_info("Ring buffer testing [%d] invalid pages: %s (%d/%d)\n",
+ cpu_buffer->cpu,
+ (discarded == meta->nr_invalid) ? "PASSED" : "FAILED",
+ discarded, meta->nr_invalid);
+ if (meta->entry_bytes)
+ pr_info("Ring buffer testing [%d] entry_bytes: %s (%ld/%ld)\n",
+ cpu_buffer->cpu,
+ (entry_bytes == meta->entry_bytes) ? "PASSED" : "FAILED",
+ (long)entry_bytes, (long)meta->entry_bytes);
+#endif
return;
invalid:
@@ -2557,12 +2574,64 @@ static void rb_free_cpu_buffer(struct ring_buffer_per_cpu *cpu_buffer)
kfree(cpu_buffer);
}
+#ifdef CONFIG_RING_BUFFER_PERSISTENT_SELFTEST
+static void rb_test_inject_invalid_pages(struct trace_buffer *buffer)
+{
+ struct ring_buffer_per_cpu *cpu_buffer;
+ struct ring_buffer_cpu_meta *meta;
+ struct buffer_data_page *dpage;
+ u32 entry_bytes = 0;
+ unsigned long ptr;
+ int subbuf_size;
+ int invalid = 0;
+ int cpu;
+ int i;
+
+ if (!(buffer->flags & RB_FL_TESTING))
+ return;
+
+ guard(preempt)();
+ cpu = smp_processor_id();
+
+ cpu_buffer = buffer->buffers[cpu];
+ meta = cpu_buffer->ring_meta;
+ ptr = (unsigned long)rb_subbufs_from_meta(meta);
+ subbuf_size = meta->subbuf_size;
+
+ for (i = 0; i < meta->nr_subbufs; i++) {
+ int idx = meta->buffers[i];
+
+ dpage = (void *)(ptr + idx * subbuf_size);
+ /* Skip unused pages */
+ if (!local_read(&dpage->commit))
+ continue;
+
+ /* Invalidate even pages. */
+ if (!(i & 0x1)) {
+ local_add(subbuf_size + 1, &dpage->commit);
+ invalid++;
+ } else {
+ /* Count total commit bytes. */
+ entry_bytes += local_read(&dpage->commit);
+ }
+ }
+
+ pr_info("Inject invalidated %d pages on CPU%d, total size: %ld\n",
+ invalid, cpu, (long)entry_bytes);
+ meta->nr_invalid = invalid;
+ meta->entry_bytes = entry_bytes;
+}
+#else /* !CONFIG_RING_BUFFER_PERSISTENT_SELFTEST */
+#define rb_test_inject_invalid_pages(buffer) do { } while (0)
+#endif
+
/* Stop recording on a persistent buffer and flush cache if needed. */
static int rb_flush_buffer_cb(struct notifier_block *nb, unsigned long event, void *data)
{
struct trace_buffer *buffer = container_of(nb, struct trace_buffer, flush_nb);
ring_buffer_record_off(buffer);
+ rb_test_inject_invalid_pages(buffer);
arch_ring_buffer_flush_range(buffer->range_addr_start, buffer->range_addr_end);
return NOTIFY_DONE;
}
diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c
index 5e1129b011cb..dc23fa63c789 100644
--- a/kernel/trace/trace.c
+++ b/kernel/trace/trace.c
@@ -9349,6 +9349,8 @@ static void setup_trace_scratch(struct trace_array *tr,
memset(tscratch, 0, size);
}
+#define TRACE_TEST_PTRACING_NAME "ptracingtest"
+
static int
allocate_trace_buffer(struct trace_array *tr, struct array_buffer *buf, unsigned long size)
{
@@ -9361,6 +9363,8 @@ allocate_trace_buffer(struct trace_array *tr, struct array_buffer *buf, unsigned
buf->tr = tr;
if (tr->range_addr_start && tr->range_addr_size) {
+ if (!strcmp(tr->name, TRACE_TEST_PTRACING_NAME))
+ rb_flags |= RB_FL_TESTING;
/* Add scratch buffer to handle 128 modules */
buf->buffer = ring_buffer_alloc_range(size, rb_flags, 0,
tr->range_addr_start,
^ permalink raw reply related [flat|nested] 9+ messages in thread