* [RFC 1/3] rv: add per-edge dwell-time statistics primitive
2026-08-27 7:23 [RFC 0/3] rv: per-edge dwell-time statistics for per-cpu monitors Tobias Schaffner
@ 2026-08-27 7:23 ` Tobias Schaffner
2026-08-27 8:22 ` Gabriele Monaco
2026-08-27 7:23 ` [RFC 2/3] rv: add per-monitor edge-stat facility and stats file Tobias Schaffner
` (2 subsequent siblings)
3 siblings, 1 reply; 8+ messages in thread
From: Tobias Schaffner @ 2026-08-27 7:23 UTC (permalink / raw)
To: Steven Rostedt, Gabriele Monaco
Cc: Jonathan Corbet, Shuah Khan, Masami Hiramatsu, Mathieu Desnoyers,
Jan Kiszka, Philippe Gerum, linux-trace-kernel, linux-doc,
linux-kselftest, linux-kernel, Tobias Schaffner
Add a small primitive that records, per automaton edge, how long the
monitor dwelled before taking it with a count, a sum and a maximum.
The counters are per-CPU and lock-free, so a monitor's hot path can update
them without disabling interrupts and without perturbing the very latency
being measured.
Signed-off-by: Tobias Schaffner <tobias.schaffner@siemens.com>
---
MAINTAINERS | 1 +
include/linux/rv_edge_stat.h | 45 ++++++++++++++++++++++++++++++++++++
2 files changed, 46 insertions(+)
create mode 100644 include/linux/rv_edge_stat.h
diff --git a/MAINTAINERS b/MAINTAINERS
index 8014b9f8253e..04589dda0873 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -23654,6 +23654,7 @@ L: linux-trace-kernel@vger.kernel.org
S: Maintained
F: Documentation/trace/rv/
F: include/linux/rv.h
+F: include/linux/rv_edge_stat.h
F: include/rv/
F: kernel/trace/rv/
F: tools/testing/selftests/verification/
diff --git a/include/linux/rv_edge_stat.h b/include/linux/rv_edge_stat.h
new file mode 100644
index 000000000000..751de8074dcc
--- /dev/null
+++ b/include/linux/rv_edge_stat.h
@@ -0,0 +1,45 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Per-edge dwell-time statistics for RV monitors.
+ *
+ * Copyright (C) 2026 Siemens AG
+ * Author: Tobias Schaffner <tobias.schaffner@siemens.com>
+ */
+#ifndef _LINUX_RV_EDGE_STAT_H
+#define _LINUX_RV_EDGE_STAT_H
+
+#include <linux/compiler.h>
+#include <linux/types.h>
+#include <asm/local64.h>
+
+/*
+ * Per-CPU counters kept in local64_t so accounting is safe against interrupt
+ * and NMI nesting on the owning CPU without disabling interrupts -- the same
+ * approach the trace ring buffer uses. Only the owning CPU writes.
+ */
+struct rv_edge_stat {
+ local64_t count;
+ local64_t sum_ns;
+ local64_t max_ns;
+};
+
+static __always_inline
+void rv_edge_stat_account(struct rv_edge_stat *s, u64 dwell_ns)
+{
+ s64 max;
+
+ local64_inc(&s->count);
+ local64_add(dwell_ns, &s->sum_ns);
+
+ /* Keep the largest dwell; retry only if a nested update raced us. */
+ max = local64_read(&s->max_ns);
+ while (dwell_ns > (u64)max) {
+ s64 prev = local64_cmpxchg(&s->max_ns, max, dwell_ns);
+
+ if (prev == max)
+ break;
+ max = prev;
+ }
+}
+
+#endif /* _LINUX_RV_EDGE_STAT_H */
--
2.43.0
^ permalink raw reply related [flat|nested] 8+ messages in thread* Re: [RFC 1/3] rv: add per-edge dwell-time statistics primitive
2026-08-27 7:23 ` [RFC 1/3] rv: add per-edge dwell-time statistics primitive Tobias Schaffner
@ 2026-08-27 8:22 ` Gabriele Monaco
2026-08-27 10:10 ` Tobias Schaffner
0 siblings, 1 reply; 8+ messages in thread
From: Gabriele Monaco @ 2026-08-27 8:22 UTC (permalink / raw)
To: Tobias Schaffner, Steven Rostedt
Cc: Jonathan Corbet, Shuah Khan, Masami Hiramatsu, Mathieu Desnoyers,
Jan Kiszka, Philippe Gerum, linux-trace-kernel, linux-doc,
linux-kselftest, linux-kernel
On Thu, 2026-08-27 at 09:23 +0200, Tobias Schaffner wrote:
> Add a small primitive that records, per automaton edge, how long the
> monitor dwelled before taking it with a count, a sum and a maximum.
>
> The counters are per-CPU and lock-free, so a monitor's hot path can update
> them without disabling interrupts and without perturbing the very latency
> being measured.
>
> Signed-off-by: Tobias Schaffner <tobias.schaffner@siemens.com>
> ---
> MAINTAINERS | 1 +
> include/linux/rv_edge_stat.h | 45 ++++++++++++++++++++++++++++++++++++
> 2 files changed, 46 insertions(+)
> create mode 100644 include/linux/rv_edge_stat.h
>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 8014b9f8253e..04589dda0873 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -23654,6 +23654,7 @@ L: linux-trace-kernel@vger.kernel.org
> S: Maintained
> F: Documentation/trace/rv/
> F: include/linux/rv.h
> +F: include/linux/rv_edge_stat.h
> F: include/rv/
Any reason why you don't use include/rv/edge_stat.h ?
You wouldn't need to touch the maintainers file and could just
#include <rv/edge_stat.h>
Thanks,
Gabriele
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [RFC 1/3] rv: add per-edge dwell-time statistics primitive
2026-08-27 8:22 ` Gabriele Monaco
@ 2026-08-27 10:10 ` Tobias Schaffner
0 siblings, 0 replies; 8+ messages in thread
From: Tobias Schaffner @ 2026-08-27 10:10 UTC (permalink / raw)
To: Gabriele Monaco, Steven Rostedt
Cc: Jonathan Corbet, Shuah Khan, Masami Hiramatsu, Mathieu Desnoyers,
Jan Kiszka, Philippe Gerum, linux-trace-kernel, linux-doc,
linux-kselftest, linux-kernel
On 8/27/26 10:22, Gabriele Monaco wrote:
> On Thu, 2026-08-27 at 09:23 +0200, Tobias Schaffner wrote:
>> Add a small primitive that records, per automaton edge, how long the
>> monitor dwelled before taking it with a count, a sum and a maximum.
>>
>> The counters are per-CPU and lock-free, so a monitor's hot path can update
>> them without disabling interrupts and without perturbing the very latency
>> being measured.
>>
>> Signed-off-by: Tobias Schaffner <tobias.schaffner@siemens.com>
>> ---
>> MAINTAINERS | 1 +
>> include/linux/rv_edge_stat.h | 45 ++++++++++++++++++++++++++++++++++++
>> 2 files changed, 46 insertions(+)
>> create mode 100644 include/linux/rv_edge_stat.h
>>
>> diff --git a/MAINTAINERS b/MAINTAINERS
>> index 8014b9f8253e..04589dda0873 100644
>> --- a/MAINTAINERS
>> +++ b/MAINTAINERS
>> @@ -23654,6 +23654,7 @@ L: linux-trace-kernel@vger.kernel.org
>> S: Maintained
>> F: Documentation/trace/rv/
>> F: include/linux/rv.h
>> +F: include/linux/rv_edge_stat.h
>> F: include/rv/
>
> Any reason why you don't use include/rv/edge_stat.h ?
No good reason. You're right. Happy to move it in a V2.
Best,
Tobias
> You wouldn't need to touch the maintainers file and could just
>
> #include <rv/edge_stat.h>
>
> Thanks,
> Gabriele
>
^ permalink raw reply [flat|nested] 8+ messages in thread
* [RFC 2/3] rv: add per-monitor edge-stat facility and stats file
2026-08-27 7:23 [RFC 0/3] rv: per-edge dwell-time statistics for per-cpu monitors Tobias Schaffner
2026-08-27 7:23 ` [RFC 1/3] rv: add per-edge dwell-time statistics primitive Tobias Schaffner
@ 2026-08-27 7:23 ` Tobias Schaffner
2026-08-27 7:24 ` [RFC 3/3] rv: collect per-edge dwell time for per-cpu DA/HA monitors Tobias Schaffner
2026-08-27 8:18 ` [RFC 0/3] rv: per-edge dwell-time statistics for per-cpu monitors Gabriele Monaco
3 siblings, 0 replies; 8+ messages in thread
From: Tobias Schaffner @ 2026-08-27 7:23 UTC (permalink / raw)
To: Steven Rostedt, Gabriele Monaco
Cc: Jonathan Corbet, Shuah Khan, Masami Hiramatsu, Mathieu Desnoyers,
Jan Kiszka, Philippe Gerum, linux-trace-kernel, linux-doc,
linux-kselftest, linux-kernel, Tobias Schaffner
Add CONFIG_RV_EDGE_STAT, an optional feature that records how long a
monitor's automaton dwells in a state and exposes it per edge through a
per-monitor "stats" tracefs file.
The core allocates a per-CPU buffer on first enable and reports, per edge
and per CPU, the count, maximum and summed dwell time. Only the owning CPU
writes the counters, so a reader snapshots them with local64_read() with no
IPI and no locking on the accounting path.
Signed-off-by: Tobias Schaffner <tobias.schaffner@siemens.com>
---
.../trace/rv/runtime-verification.rst | 24 ++++
include/linux/rv.h | 14 +++
include/linux/rv_edge_stat.h | 25 +++-
kernel/trace/rv/Kconfig | 11 ++
kernel/trace/rv/rv.c | 116 +++++++++++++++++-
5 files changed, 184 insertions(+), 6 deletions(-)
diff --git a/Documentation/trace/rv/runtime-verification.rst b/Documentation/trace/rv/runtime-verification.rst
index c700dde9259c..194b2f9db461 100644
--- a/Documentation/trace/rv/runtime-verification.rst
+++ b/Documentation/trace/rv/runtime-verification.rst
@@ -229,3 +229,27 @@ For example::
nop
[panic]
printk
+
+**monitors/MONITOR/stats**
+
+Present only when the kernel is built with CONFIG_RV_EDGE_STAT=y and *MONITOR*
+is a per-cpu DA/HA (automaton) monitor. It reports how long the automaton
+dwells in each state before leaving it, timed with local_clock() and accounted
+per outgoing edge and per CPU.
+
+- The first line is a header naming the columns.
+- Each following line describes one edge on one CPU::
+
+ cpu edge label count max_ns sum_ns
+
+ *count* is the number of times the edge was taken, *max_ns* and *sum_ns* are
+ the worst and total dwell in nanoseconds, and *label* is "state:event".
+
+The counters are reset each time the monitor is enabled.
+
+For example::
+
+ # cat monitors/wip/stats
+ # cpu edge label count max_ns sum_ns
+ 0 0 preemptive:preempt_disable 4210 183200 95501200
+ 0 4 non_preemptive:preempt_enable 4208 42600 3812900
diff --git a/include/linux/rv.h b/include/linux/rv.h
index 541ba404926a..7eeecce17e50 100644
--- a/include/linux/rv.h
+++ b/include/linux/rv.h
@@ -136,6 +136,16 @@ struct rv_reactor {
};
#endif
+/**
+ * struct rv_edge_cfg - per-edge dwell-time statistics for a monitor
+ * @n_edges: number of automaton edges (STATE_MAX * EVENT_MAX)
+ * @edge_name: optional, write a human name for @edge into @buf (may be NULL)
+ */
+struct rv_edge_cfg {
+ unsigned int n_edges;
+ void (*edge_name)(unsigned int edge, char *buf, size_t len);
+};
+
struct rv_monitor {
const char *name;
const char *description;
@@ -146,6 +156,10 @@ struct rv_monitor {
#ifdef CONFIG_RV_REACTORS
struct rv_reactor *reactor;
__printf(1, 0) void (*react)(const char *msg, va_list args);
+#endif
+#ifdef CONFIG_RV_EDGE_STAT
+ const struct rv_edge_cfg *edge_cfg;
+ void __percpu *edge_pcpu;
#endif
struct list_head list;
struct rv_monitor *parent;
diff --git a/include/linux/rv_edge_stat.h b/include/linux/rv_edge_stat.h
index 751de8074dcc..fda30ff728a1 100644
--- a/include/linux/rv_edge_stat.h
+++ b/include/linux/rv_edge_stat.h
@@ -9,14 +9,11 @@
#define _LINUX_RV_EDGE_STAT_H
#include <linux/compiler.h>
+#include <linux/percpu.h>
+#include <linux/rv.h>
#include <linux/types.h>
#include <asm/local64.h>
-/*
- * Per-CPU counters kept in local64_t so accounting is safe against interrupt
- * and NMI nesting on the owning CPU without disabling interrupts -- the same
- * approach the trace ring buffer uses. Only the owning CPU writes.
- */
struct rv_edge_stat {
local64_t count;
local64_t sum_ns;
@@ -42,4 +39,22 @@ void rv_edge_stat_account(struct rv_edge_stat *s, u64 dwell_ns)
}
}
+#ifdef CONFIG_RV_EDGE_STAT
+/**
+ * rv_edge_account - record a dwell of @dwell_ns on @edge of monitor @mon
+ *
+ * Cheap and lock-free: the local64_t counters make this safe against interrupt
+ * and NMI nesting on the current CPU without disabling interrupts, so it does
+ * not perturb the latency being measured. The caller only needs to stay on its
+ * CPU for the call (as tracepoint probes already do).
+ */
+static __always_inline void
+rv_edge_account(struct rv_monitor *mon, unsigned int edge, u64 dwell_ns)
+{
+ struct rv_edge_stat *e = this_cpu_ptr(mon->edge_pcpu);
+
+ rv_edge_stat_account(&e[edge], dwell_ns);
+}
+#endif /* CONFIG_RV_EDGE_STAT */
+
#endif /* _LINUX_RV_EDGE_STAT_H */
diff --git a/kernel/trace/rv/Kconfig b/kernel/trace/rv/Kconfig
index 3884b14df375..9d76dff394ca 100644
--- a/kernel/trace/rv/Kconfig
+++ b/kernel/trace/rv/Kconfig
@@ -59,6 +59,17 @@ config RV_PER_TASK_MONITORS
This option configures the maximum number of per-task RV monitors that can run
simultaneously.
+config RV_EDGE_STAT
+ bool "Per-edge dwell-time statistics"
+ depends on RV
+ help
+ Record per-edge dwell-time statistics for per-cpu DA/HA monitors and
+ expose them through a per-monitor "stats" tracefs file. This times
+ each monitored automaton transition with local_clock(), so leave it
+ off if you do not need the statistics.
+
+ If unsure, say N.
+
source "kernel/trace/rv/monitors/wip/Kconfig"
source "kernel/trace/rv/monitors/wwnr/Kconfig"
diff --git a/kernel/trace/rv/rv.c b/kernel/trace/rv/rv.c
index ee4e68102f17..88a0bbaec4d0 100644
--- a/kernel/trace/rv/rv.c
+++ b/kernel/trace/rv/rv.c
@@ -142,6 +142,12 @@
#include <linux/module.h>
#include <linux/init.h>
#include <linux/slab.h>
+#include <linux/seq_file.h>
+#ifdef CONFIG_RV_EDGE_STAT
+#include <linux/percpu.h>
+#include <linux/rv_edge_stat.h>
+#include <linux/smp.h>
+#endif
#ifdef CONFIG_RV_MON_EVENTS
#define CREATE_TRACE_POINTS
@@ -278,6 +284,9 @@ static void rv_disable_single(struct rv_monitor *mon)
__rv_disable_monitor(mon, true);
}
+static int rv_edge_setup(struct rv_monitor *mon);
+static void rv_edge_reset(struct rv_monitor *mon);
+
static int rv_enable_single(struct rv_monitor *mon)
{
int retval;
@@ -289,9 +298,15 @@ static int rv_enable_single(struct rv_monitor *mon)
retval = mon->enable();
- if (!retval)
+ if (!retval) {
mon->enabled = 1;
+ if (rv_edge_setup(mon))
+ pr_warn("rv: %s: edge statistics unavailable (out of memory)\n",
+ mon->name);
+ rv_edge_reset(mon);
+ }
+
return retval;
}
@@ -412,6 +427,101 @@ static const struct file_operations interface_desc_fops = {
.read = monitor_desc_read_data,
};
+#ifdef CONFIG_RV_EDGE_STAT
+static size_t rv_edge_blob_size(const struct rv_monitor *mon)
+{
+ return mon->edge_cfg->n_edges * sizeof(struct rv_edge_stat);
+}
+
+static void rv_edge_reset_ipi(void *info)
+{
+ struct rv_monitor *mon = info;
+
+ memset(this_cpu_ptr(mon->edge_pcpu), 0, rv_edge_blob_size(mon));
+}
+
+/* rv_edge_reset - zero the statistics; call from a monitor reset/enable. */
+static void rv_edge_reset(struct rv_monitor *mon)
+{
+ if (mon->edge_pcpu)
+ on_each_cpu(rv_edge_reset_ipi, mon, 1);
+}
+
+/*
+ * The counters are per-CPU and only the owning CPU writes them, so a reader on
+ * any CPU can snapshot them with local64_read().
+ */
+static int rv_edge_stats_show(struct seq_file *seq, void *v)
+{
+ struct rv_monitor *mon = seq->private;
+ const struct rv_edge_cfg *cfg = mon->edge_cfg;
+ unsigned int e;
+ int cpu;
+
+ seq_puts(seq, "# cpu edge label count max_ns sum_ns\n");
+
+ if (!mon->edge_pcpu)
+ return 0;
+
+ for_each_online_cpu(cpu) {
+ struct rv_edge_stat *s = per_cpu_ptr(mon->edge_pcpu, cpu);
+
+ for (e = 0; e < cfg->n_edges; e++) {
+ char lbl[48] = "";
+
+ if (cfg->edge_name)
+ cfg->edge_name(e, lbl, sizeof(lbl));
+ seq_printf(seq, "%d %u %s %llu %llu %llu\n",
+ cpu, e, lbl,
+ (u64)local64_read(&s[e].count),
+ (u64)local64_read(&s[e].max_ns),
+ (u64)local64_read(&s[e].sum_ns));
+ }
+ }
+ return 0;
+}
+
+static int rv_edge_stats_open(struct inode *inode, struct file *file)
+{
+ return single_open(file, rv_edge_stats_show, inode->i_private);
+}
+
+static const struct file_operations rv_edge_stats_fops = {
+ .open = rv_edge_stats_open,
+ .read = seq_read,
+ .llseek = seq_lseek,
+ .release = single_release,
+};
+
+/*
+ * Allocate the per-CPU buffer and expose stats. Done on first enable
+ * rather than at registration because a DA/HA monitor's edge_cfg is bound by
+ * da_monitor_init(), which runs from the monitor's enable path.
+ */
+static int rv_edge_setup(struct rv_monitor *mon)
+{
+ if (!mon->edge_cfg || !mon->edge_cfg->n_edges || mon->edge_pcpu)
+ return 0;
+
+ mon->edge_pcpu = __alloc_percpu(rv_edge_blob_size(mon),
+ __alignof__(struct rv_edge_stat));
+ if (!mon->edge_pcpu)
+ return -ENOMEM;
+
+ if (!rv_create_file("stats", RV_MODE_READ, mon->root_d, mon,
+ &rv_edge_stats_fops)) {
+ free_percpu(mon->edge_pcpu);
+ mon->edge_pcpu = NULL;
+ return -ENOMEM;
+ }
+
+ return 0;
+}
+#else
+static int rv_edge_setup(struct rv_monitor *mon) { return 0; }
+static void rv_edge_reset(struct rv_monitor *mon) { }
+#endif /* CONFIG_RV_EDGE_STAT */
+
/*
* During the registration of a monitor, this function creates
* the monitor dir, where the specific options of the monitor
@@ -747,6 +857,10 @@ static const struct file_operations monitoring_on_fops = {
static void destroy_monitor_dir(struct rv_monitor *mon)
{
+#ifdef CONFIG_RV_EDGE_STAT
+ free_percpu(mon->edge_pcpu);
+ mon->edge_pcpu = NULL;
+#endif
rv_remove(mon->root_d);
}
--
2.43.0
^ permalink raw reply related [flat|nested] 8+ messages in thread* [RFC 3/3] rv: collect per-edge dwell time for per-cpu DA/HA monitors
2026-08-27 7:23 [RFC 0/3] rv: per-edge dwell-time statistics for per-cpu monitors Tobias Schaffner
2026-08-27 7:23 ` [RFC 1/3] rv: add per-edge dwell-time statistics primitive Tobias Schaffner
2026-08-27 7:23 ` [RFC 2/3] rv: add per-monitor edge-stat facility and stats file Tobias Schaffner
@ 2026-08-27 7:24 ` Tobias Schaffner
2026-08-27 8:18 ` [RFC 0/3] rv: per-edge dwell-time statistics for per-cpu monitors Gabriele Monaco
3 siblings, 0 replies; 8+ messages in thread
From: Tobias Schaffner @ 2026-08-27 7:24 UTC (permalink / raw)
To: Steven Rostedt, Gabriele Monaco
Cc: Jonathan Corbet, Shuah Khan, Masami Hiramatsu, Mathieu Desnoyers,
Jan Kiszka, Philippe Gerum, linux-trace-kernel, linux-doc,
linux-kselftest, linux-kernel, Tobias Schaffner
With the core facility in place, hook it into the DA/HA layer so that any
per-cpu automaton monitor also reports how long it lingers in each state.
Add a selftest that enables a per-cpu monitor, checks the stats file
appears and is populated under load, and skips cleanly otherwise.
Signed-off-by: Tobias Schaffner <tobias.schaffner@siemens.com>
---
include/linux/rv.h | 4 ++
include/rv/da_monitor.h | 53 +++++++++++++++++++
tools/testing/selftests/verification/config | 1 +
.../verification/test.d/rv_edge_stats.tc | 32 +++++++++++
4 files changed, 90 insertions(+)
create mode 100644 tools/testing/selftests/verification/test.d/rv_edge_stats.tc
diff --git a/include/linux/rv.h b/include/linux/rv.h
index 7eeecce17e50..09363f79ca90 100644
--- a/include/linux/rv.h
+++ b/include/linux/rv.h
@@ -27,6 +27,10 @@
struct da_monitor {
bool monitoring;
unsigned int curr_state;
+#ifdef CONFIG_RV_EDGE_STAT
+ /* local_clock() ns when curr_state was entered; 0 = not yet stamped. */
+ u64 state_ns;
+#endif
};
#ifdef CONFIG_RV_LTL_MONITOR
diff --git a/include/rv/da_monitor.h b/include/rv/da_monitor.h
index 34b8fba9ecd4..59ca9a286c10 100644
--- a/include/rv/da_monitor.h
+++ b/include/rv/da_monitor.h
@@ -16,6 +16,8 @@
#include <rv/automata.h>
#include <linux/rv.h>
+#include <linux/rv_edge_stat.h>
+#include <linux/sched/clock.h>
#include <linux/stringify.h>
#include <linux/bug.h>
#include <linux/sched.h>
@@ -30,6 +32,54 @@
static struct rv_monitor rv_this;
+/* per-edge dwell statistics, wired up for per-cpu monitors. */
+#if defined(CONFIG_RV_EDGE_STAT) && RV_MON_TYPE == RV_MON_PER_CPU
+static void
+rv_this_edge_name(unsigned int edge, char *buf, size_t len)
+{
+ snprintf(buf, len, "%s:%s", model_get_state_name(edge / EVENT_MAX),
+ model_get_event_name(edge % EVENT_MAX));
+}
+
+static const struct rv_edge_cfg rv_this_edge_cfg = {
+ .n_edges = STATE_MAX * EVENT_MAX,
+ .edge_name = rv_this_edge_name,
+};
+
+/* Hand the model's edge descriptor to the core; called from da_monitor_init(). */
+static inline void rv_edge_bind(void)
+{
+ rv_this.edge_cfg = &rv_this_edge_cfg;
+}
+
+/* Stamp the moment a state is entered, so its dwell can be timed on exit. */
+static __always_inline void rv_da_edge_enter(struct da_monitor *da_mon)
+{
+ da_mon->state_ns = local_clock();
+}
+
+/* Account the dwell in @curr, then stamp entry into the next state. */
+static __always_inline void
+rv_da_edge_account(struct da_monitor *da_mon, enum states curr, enum events ev)
+{
+ u64 now = local_clock();
+ u64 prev = da_mon->state_ns;
+
+ da_mon->state_ns = now;
+ /*
+ * local_clock() is not guaranteed monotonic; drop the sample if it did
+ * not advance so a backward step cannot underflow into a bogus dwell.
+ */
+ if (rv_this.edge_pcpu && prev && now > prev)
+ rv_edge_account(&rv_this, curr * EVENT_MAX + ev, now - prev);
+}
+#else
+static inline void rv_edge_bind(void) { }
+static inline void rv_da_edge_enter(struct da_monitor *da_mon) { }
+static inline void
+rv_da_edge_account(struct da_monitor *da_mon, enum states curr, enum events ev) { }
+#endif /* CONFIG_RV_EDGE_STAT && RV_MON_PER_CPU */
+
/*
* Hook to allow the implementation of hybrid automata: define it with a
* function that takes curr_state, event and next_state and returns true if the
@@ -113,6 +163,7 @@ static inline void da_monitor_reset(struct da_monitor *da_mon)
static inline void da_monitor_start(struct da_monitor *da_mon)
{
da_mon->curr_state = model_get_initial_state();
+ rv_da_edge_enter(da_mon);
da_monitor_init_hook(da_mon);
/* Pairs with smp_load_acquire in da_monitoring(). */
smp_store_release(&da_mon->monitoring, 1);
@@ -275,6 +326,7 @@ static inline void da_monitor_reset_state_all(void)
*/
static inline int da_monitor_init(void)
{
+ rv_edge_bind();
da_monitor_reset_state_all();
return 0;
}
@@ -696,6 +748,7 @@ static inline bool da_event(struct da_monitor *da_mon, enum events event, da_id_
if (likely(try_cmpxchg(&da_mon->curr_state, &curr_state, next_state))) {
if (!da_monitor_event_hook(da_mon, curr_state, event, next_state, id))
return false;
+ rv_da_edge_account(da_mon, curr_state, event);
da_trace_event(da_mon, model_get_state_name(curr_state),
model_get_event_name(event),
model_get_state_name(next_state),
diff --git a/tools/testing/selftests/verification/config b/tools/testing/selftests/verification/config
index 43072c1c38f4..de229777d5c8 100644
--- a/tools/testing/selftests/verification/config
+++ b/tools/testing/selftests/verification/config
@@ -1 +1,2 @@
CONFIG_RV=y
+CONFIG_RV_EDGE_STAT=y
diff --git a/tools/testing/selftests/verification/test.d/rv_edge_stats.tc b/tools/testing/selftests/verification/test.d/rv_edge_stats.tc
new file mode 100644
index 000000000000..5fb2384d938c
--- /dev/null
+++ b/tools/testing/selftests/verification/test.d/rv_edge_stats.tc
@@ -0,0 +1,32 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0-or-later
+# description: Test per-edge dwell-time statistics (stats)
+
+MON=wip
+
+check_requires "$MON:monitor"
+
+# The "stats" file is created on first enable; if it is still missing the
+# kernel was built without CONFIG_RV_EDGE_STAT, so skip.
+echo 1 > "monitors/$MON/enable"
+if [ ! -e "monitors/$MON/stats" ]; then
+ echo 0 > "monitors/$MON/enable"
+ echo "CONFIG_RV_EDGE_STAT is not enabled."
+ exit_unsupported
+fi
+
+# The first line is the column header; the body has one line per (cpu, edge).
+head -n1 "monitors/$MON/stats" | grep -q "^# cpu edge label count max_ns sum_ns"
+[ "$(grep -cvE '^#' "monitors/$MON/stats")" -gt 0 ]
+
+# Drive some scheduler activity so the automaton records transitions.
+for _ in 1 2 3 4 5 6 7 8 9 10; do
+ (true) &
+ wait
+done
+
+# At least one edge must now show a non-zero count, with sum_ns >= max_ns > 0.
+grep -vE '^#' "monitors/$MON/stats" | \
+ awk '$4 > 0 && $5 > 0 && $6 >= $5 { hit = 1 } END { exit !hit }'
+
+echo 0 > "monitors/$MON/enable"
--
2.43.0
^ permalink raw reply related [flat|nested] 8+ messages in thread* Re: [RFC 0/3] rv: per-edge dwell-time statistics for per-cpu monitors
2026-08-27 7:23 [RFC 0/3] rv: per-edge dwell-time statistics for per-cpu monitors Tobias Schaffner
` (2 preceding siblings ...)
2026-08-27 7:24 ` [RFC 3/3] rv: collect per-edge dwell time for per-cpu DA/HA monitors Tobias Schaffner
@ 2026-08-27 8:18 ` Gabriele Monaco
2026-08-27 18:21 ` Tobias Schaffner
3 siblings, 1 reply; 8+ messages in thread
From: Gabriele Monaco @ 2026-08-27 8:18 UTC (permalink / raw)
To: Tobias Schaffner, Steven Rostedt
Cc: Jonathan Corbet, Shuah Khan, Masami Hiramatsu, Mathieu Desnoyers,
Jan Kiszka, Philippe Gerum, linux-trace-kernel, linux-doc,
linux-kselftest, linux-kernel
On Thu, 2026-08-27 at 09:23 +0200, Tobias Schaffner wrote:
> Hi Steven, Gabriele,
>
> I am using the RV system for a downstream project called EVL (aka Xenomai4).
> It is a small co-kernel system, small enough that I can model the whole
> co-kernel with RV instead of just a fragment of it.
Hi Tobias,
Haven't heard about Xenomai in a while.. That sounds very interesting.
Since it's something out-of-tree you likely don't want to submit monitors
upstream, but if you can trigger BPF events from your co-kernel you may want to
have a look at RV BPF monitors, which I'm likely going to submit in a couple of
days.
Otherwise we can always get out-of-tree monitors as modules, currently not
supported just out of a missing use-case.
> The patches in this series add a per-monitor "stats" file that exposes how
> often an edge has been visited and the dwell time in the prior state for
> per-cpu monitors.
The idea crossed my mind a few times, good you came up with an implementation!
> This allows me to not only check if the model is correct but also to see if
> all paths are exercised by my stress tooling and to decompose the latency of
> the co-kernel's wakeup path.
Your first need (see if all paths are exercised) doesn't really require the stat
to be per-monitor-instance (i.e. per-cpu) right?
You could have a global accumulator matrix, at least for count and sum where you
can be atomic. This could extend the feature to other types of monitors.
Just brainstorming here though.
> I think that some of the upstream monitors like e.g. the sts monitor could
> also
> profit from this.
>
> As an example, here is the sts monitor on a StarFive VisionFive2 running an rt
> kernel (isolcpus=2-3), after enabling it and generating some scheduler load.
> Only the edges the automaton actually takes are non-zero:
>
> # cat monitors/sched/sts/stats
> # cpu edge label count max_ns sum_ns
> 0 0 can_sched:irq_disable 367732 3998500 14773091250
> 0 4 can_sched:schedule_entry 40282 398750 45510750
> 0 7 cant_sched:irq_enable 367732 32500 532791750
> 0 8 cant_sched:irq_entry 26188 6250 38357250
> 0 13 disable_to_switch:irq_enable 4933 12500 12275000
> 0 14 disable_to_switch:irq_entry 31 2250 35250
> 0 15 disable_to_switch:sched_switch 35373 26750 111615750
> 0 18 enable_to_exit:irq_disable 373 3250 649750
> 0 19 enable_to_exit:irq_enable 373 20000 1676250
> 0 20 enable_to_exit:irq_entry 309 4750 420000
> 0 23 enable_to_exit:schedule_exit 40298 3000 37335000
> 0 25 in_irq:irq_enable 31 12250 140750
> 0 30 scheduling:irq_disable 40337 18000 45449500
> 0 37 switching:irq_enable 35373 9250 72471000
> [ cpus 1-2 omitted ]
> 3 0 can_sched:irq_disable 567 3983750 300860250
> 3 4 can_sched:schedule_entry 150 1500 171000
> 3 7 cant_sched:irq_enable 567 21500 1777500
> 3 8 cant_sched:irq_entry 76 6500 164750
> 3 15 disable_to_switch:sched_switch 150 4250 413750
> 3 23 enable_to_exit:schedule_exit 150 1500 147750
> 3 30 scheduling:irq_disable 150 1500 166000
> 3 37 switching:irq_enable 150 3750 277000
>
> The sts model splits the schedule->switch path into separate states, so the
> per-edge dwell decomposes the scheduler's interrupts-off window: on cpu0 the
> prep phase (disable_to_switch, irqs off until the switch) tops out at 26.8us
> and the switch itself (switching:irq_enable) at 9.3us, over 35k switches. The
> counts also line up with the model, e.g. every irq_disable in can_sched has a
> matching irq_enable in cant_sched (367732 == 367732). The isolated cpus 2-3
> take almost no switches (150 on cpu3, versus 35k on cpu0) and their worst case
> is tighter still (~4us), so the same file also makes the effect of cpu
> isolation visible per cpu.
>
> The changes are gated by CONFIG_RV_EDGE_STAT and dormant until a monitor is
> enabled, so existing setups are unaffected. When enabled, each accepted
> transition adds one local_clock() and a few lock-free local64_t updates. With
> the config off there is no code on the hot path at all.
>
> I focused on per-cpu monitors as a first step. Per-task and per-object
> monitors would have to aggregate entities that share a cpu, which is harder
> to get right, so I left them out for now.
As said before, I haven't really played with an implementation but I think
atomic types may help here.
> What is your opinion on this? Do you think this is worth getting upstreamed?
I will have a look at your patches but it's definitely something I'd want.
Thanks,
Gabriele
>
> Thanks for taking a look,
> Tobias
>
> Tobias Schaffner (3):
> rv: add per-edge dwell-time statistics primitive
> rv: add per-monitor edge-stat facility and stats file
> rv: collect per-edge dwell time for per-cpu DA/HA monitors
>
> .../trace/rv/runtime-verification.rst | 24 ++++
> MAINTAINERS | 1 +
> include/linux/rv.h | 18 +++
> include/linux/rv_edge_stat.h | 60 +++++++++
> include/rv/da_monitor.h | 53 ++++++++
> kernel/trace/rv/Kconfig | 11 ++
> kernel/trace/rv/rv.c | 116 +++++++++++++++++-
> tools/testing/selftests/verification/config | 1 +
> .../verification/test.d/rv_edge_stats.tc | 32 +++++
> 9 files changed, 315 insertions(+), 1 deletion(-)
> create mode 100644 include/linux/rv_edge_stat.h
> create mode 100644
> tools/testing/selftests/verification/test.d/rv_edge_stats.tc
^ permalink raw reply [flat|nested] 8+ messages in thread* Re: [RFC 0/3] rv: per-edge dwell-time statistics for per-cpu monitors
2026-08-27 8:18 ` [RFC 0/3] rv: per-edge dwell-time statistics for per-cpu monitors Gabriele Monaco
@ 2026-08-27 18:21 ` Tobias Schaffner
0 siblings, 0 replies; 8+ messages in thread
From: Tobias Schaffner @ 2026-08-27 18:21 UTC (permalink / raw)
To: Gabriele Monaco, Steven Rostedt
Cc: Jonathan Corbet, Shuah Khan, Masami Hiramatsu, Mathieu Desnoyers,
Jan Kiszka, Philippe Gerum, linux-trace-kernel, linux-doc,
linux-kselftest, linux-kernel
Hi Gabriele,
thanks for the quick reply and the motivating words.
On 8/27/26 10:18, Gabriele Monaco wrote:
> On Thu, 2026-08-27 at 09:23 +0200, Tobias Schaffner wrote:
>> Hi Steven, Gabriele,
>>
>> I am using the RV system for a downstream project called EVL (aka Xenomai4).
>> It is a small co-kernel system, small enough that I can model the whole
>> co-kernel with RV instead of just a fragment of it.
>
> Hi Tobias,
>
> Haven't heard about Xenomai in a while.. That sounds very interesting.
>
> Since it's something out-of-tree you likely don't want to submit monitors
> upstream, but if you can trigger BPF events from your co-kernel you may want to
> have a look at RV BPF monitors, which I'm likely going to submit in a couple of
> days.
Exactly. The monitors themselves will live downstream in the Dovetail
and EVL projects.
I will definitely have a look at the RV BPF monitors!
> Otherwise we can always get out-of-tree monitors as modules, currently not
> supported just out of a missing use-case.
>
>> The patches in this series add a per-monitor "stats" file that exposes how
>> often an edge has been visited and the dwell time in the prior state for
>> per-cpu monitors.
>
> The idea crossed my mind a few times, good you came up with an implementation!
>
>> This allows me to not only check if the model is correct but also to see if
>> all paths are exercised by my stress tooling and to decompose the latency of
>> the co-kernel's wakeup path.
>
> Your first need (see if all paths are exercised) doesn't really require the stat
> to be per-monitor-instance (i.e. per-cpu) right?
> You could have a global accumulator matrix, at least for count and sum where you
> can be atomic. This could extend the feature to other types of monitors.
Yes that's correct.
For the dwell timings, though, keeping it per-cpu matters to make
isolation effects visible. I use sum and count there to derive
the average.
> Just brainstorming here though.
>
>> I think that some of the upstream monitors like e.g. the sts monitor could
>> also
>> profit from this.
>>
>> As an example, here is the sts monitor on a StarFive VisionFive2 running an rt
>> kernel (isolcpus=2-3), after enabling it and generating some scheduler load.
>> Only the edges the automaton actually takes are non-zero:
>>
>> # cat monitors/sched/sts/stats
>> # cpu edge label count max_ns sum_ns
>> 0 0 can_sched:irq_disable 367732 3998500 14773091250
>> 0 4 can_sched:schedule_entry 40282 398750 45510750
>> 0 7 cant_sched:irq_enable 367732 32500 532791750
>> 0 8 cant_sched:irq_entry 26188 6250 38357250
>> 0 13 disable_to_switch:irq_enable 4933 12500 12275000
>> 0 14 disable_to_switch:irq_entry 31 2250 35250
>> 0 15 disable_to_switch:sched_switch 35373 26750 111615750
>> 0 18 enable_to_exit:irq_disable 373 3250 649750
>> 0 19 enable_to_exit:irq_enable 373 20000 1676250
>> 0 20 enable_to_exit:irq_entry 309 4750 420000
>> 0 23 enable_to_exit:schedule_exit 40298 3000 37335000
>> 0 25 in_irq:irq_enable 31 12250 140750
>> 0 30 scheduling:irq_disable 40337 18000 45449500
>> 0 37 switching:irq_enable 35373 9250 72471000
>> [ cpus 1-2 omitted ]
>> 3 0 can_sched:irq_disable 567 3983750 300860250
>> 3 4 can_sched:schedule_entry 150 1500 171000
>> 3 7 cant_sched:irq_enable 567 21500 1777500
>> 3 8 cant_sched:irq_entry 76 6500 164750
>> 3 15 disable_to_switch:sched_switch 150 4250 413750
>> 3 23 enable_to_exit:schedule_exit 150 1500 147750
>> 3 30 scheduling:irq_disable 150 1500 166000
>> 3 37 switching:irq_enable 150 3750 277000
>>
>> The sts model splits the schedule->switch path into separate states, so the
>> per-edge dwell decomposes the scheduler's interrupts-off window: on cpu0 the
>> prep phase (disable_to_switch, irqs off until the switch) tops out at 26.8us
>> and the switch itself (switching:irq_enable) at 9.3us, over 35k switches. The
>> counts also line up with the model, e.g. every irq_disable in can_sched has a
>> matching irq_enable in cant_sched (367732 == 367732). The isolated cpus 2-3
>> take almost no switches (150 on cpu3, versus 35k on cpu0) and their worst case
>> is tighter still (~4us), so the same file also makes the effect of cpu
>> isolation visible per cpu.
>>
>> The changes are gated by CONFIG_RV_EDGE_STAT and dormant until a monitor is
>> enabled, so existing setups are unaffected. When enabled, each accepted
>> transition adds one local_clock() and a few lock-free local64_t updates. With
>> the config off there is no code on the hot path at all.
>>
>> I focused on per-cpu monitors as a first step. Per-task and per-object
>> monitors would have to aggregate entities that share a cpu, which is harder
>> to get right, so I left them out for now.
>
> As said before, I haven't really played with an implementation but I think
> atomic types may help here.
>
>> What is your opinion on this? Do you think this is worth getting upstreamed?
>
> I will have a look at your patches but it's definitely something I'd want.
Looking forward to your input! I am very open to guidance on how to get
this right.
Best,
Tobias
> Thanks,
> Gabriele
>
>>
>> Thanks for taking a look,
>> Tobias
>>
>> Tobias Schaffner (3):
>> rv: add per-edge dwell-time statistics primitive
>> rv: add per-monitor edge-stat facility and stats file
>> rv: collect per-edge dwell time for per-cpu DA/HA monitors
>>
>> .../trace/rv/runtime-verification.rst | 24 ++++
>> MAINTAINERS | 1 +
>> include/linux/rv.h | 18 +++
>> include/linux/rv_edge_stat.h | 60 +++++++++
>> include/rv/da_monitor.h | 53 ++++++++
>> kernel/trace/rv/Kconfig | 11 ++
>> kernel/trace/rv/rv.c | 116 +++++++++++++++++-
>> tools/testing/selftests/verification/config | 1 +
>> .../verification/test.d/rv_edge_stats.tc | 32 +++++
>> 9 files changed, 315 insertions(+), 1 deletion(-)
>> create mode 100644 include/linux/rv_edge_stat.h
>> create mode 100644
>> tools/testing/selftests/verification/test.d/rv_edge_stats.tc
>
^ permalink raw reply [flat|nested] 8+ messages in thread