From: wen.yang@linux.dev
To: Gabriele Monaco <gmonaco@redhat.com>
Cc: Nam Cao <namcao@linutronix.de>,
linux-trace-kernel@vger.kernel.org, linux-kernel@vger.kernel.org,
Wen Yang <wen.yang@linux.dev>
Subject: [PATCH v5 6/9] rv: Add tlob hybrid automaton monitor
Date: Thu, 20 Aug 2026 02:15:23 +0800 [thread overview]
Message-ID: <392de71fe1497f2b9d8ce06c387ec83c4b116c47.1787161646.git.wen.yang@linux.dev> (raw)
In-Reply-To: <cover.1787161646.git.wen.yang@linux.dev>
From: Wen Yang <wen.yang@linux.dev>
tlob (task latency over budget) is a per-task hybrid automaton RV
monitor that tracks wall-clock time across a user-delimited code
section and emits an error when elapsed time exceeds a configurable
budget.
Four-state automaton (running, waiting, sleeping, stopped) driven by
sched_switch/sched_wakeup tracepoints and a user-visible start/stop
pair: "stop" only fires from running and parks the window in stopped,
where a later "start" restarts it in place (same pool slot, no
re-registration); both callers of stop run while the task is on CPU.
A single clk_elapsed < BUDGET_NS() invariant is enforced by a
per-task HRTIMER_MODE_REL_HARD timer; on expiry the monitor records a
per-state breakdown (running_ns, waiting_ns, sleeping_ns) before
emitting error_env_tlob.
Uprobe pairs are registered through a tracefs monitor file as
"p PATH:OFFSET_START OFFSET_STOP threshold=NS". A pre-allocated
mempool hard-caps concurrently monitored tasks at TLOB_MAX_MONITORED
(past it, fresh starts return -ENOSPC) with allocation-free start/stop
on the uprobe hot path.
Signed-off-by: Wen Yang <wen.yang@linux.dev>
---
Documentation/trace/rv/index.rst | 1 +
Documentation/trace/rv/monitor_tlob.rst | 194 ++++
kernel/trace/rv/Kconfig | 5 +
kernel/trace/rv/Makefile | 2 +
kernel/trace/rv/monitors/tlob/Kconfig | 12 +
kernel/trace/rv/monitors/tlob/tlob.c | 1132 ++++++++++++++++++++
kernel/trace/rv/monitors/tlob/tlob.h | 149 +++
kernel/trace/rv/monitors/tlob/tlob_trace.h | 48 +
kernel/trace/rv/rv_trace.h | 1 +
9 files changed, 1544 insertions(+)
create mode 100644 Documentation/trace/rv/monitor_tlob.rst
create mode 100644 kernel/trace/rv/monitors/tlob/Kconfig
create mode 100644 kernel/trace/rv/monitors/tlob/tlob.c
create mode 100644 kernel/trace/rv/monitors/tlob/tlob.h
create mode 100644 kernel/trace/rv/monitors/tlob/tlob_trace.h
diff --git a/Documentation/trace/rv/index.rst b/Documentation/trace/rv/index.rst
index 29769f06bb0f..1501545b5f08 100644
--- a/Documentation/trace/rv/index.rst
+++ b/Documentation/trace/rv/index.rst
@@ -16,5 +16,6 @@ Runtime Verification
monitor_wwnr.rst
monitor_sched.rst
monitor_rtapp.rst
+ monitor_tlob.rst
monitor_stall.rst
monitor_deadline.rst
diff --git a/Documentation/trace/rv/monitor_tlob.rst b/Documentation/trace/rv/monitor_tlob.rst
new file mode 100644
index 000000000000..2e606b0e67a4
--- /dev/null
+++ b/Documentation/trace/rv/monitor_tlob.rst
@@ -0,0 +1,194 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+Monitor tlob
+============
+
+- Name: tlob - task latency over budget
+- Type: per-object hybrid automaton (RV_MON_PER_OBJ)
+- Author: Wen Yang <wen.yang@linux.dev>
+
+Description
+-----------
+
+The tlob monitor tracks per-task elapsed wall-clock time (CLOCK_MONOTONIC,
+spanning running, waiting, and sleeping states) and reports a violation when
+the monitored task exceeds a configurable per-invocation budget threshold.
+
+The monitor implements a four-state hybrid automaton with a single clock
+environment variable ``clk_elapsed``. The clock invariant
+``clk_elapsed < BUDGET_NS()`` is active in the ``running``, ``waiting``, and
+``sleeping`` states (``stopped`` has no invariant, hence no timer); when it
+is violated the HA timer fires and the framework emits ``error_env_tlob``
+then calls ``da_monitor_reset()`` automatically::
+
+ | (initial)
+ v
+ +--------------+ +----------+
+ | running | --------> | stopped |
+ |->+--------------+ <-------- +----------+
+ switch_in preempt sleep
+ | | |
+ | | |
+ | v v
+ +---------+ +---------+
+ | waiting | | sleeping|
+ +---------+ +---------+
+ ^ v
+ | wakeup |
+ | |
+ +------------+
+
+ A fourth state, ``stopped``, has no clock invariant (hence no timer).
+ ``running`` reaches it on ``stop`` (``tlob_stop_task()``, window ended,
+ per-task state parked rather than freed) and returns to ``running`` on
+ ``start`` (``tlob_start_task()`` restarting the same task's parked
+ window).
+
+ Key transitions:
+ running --(sleep)------> sleeping (task blocks waiting for a resource)
+ running --(preempt)----> waiting (task preempted, back in runqueue)
+ sleeping --(wakeup)-----> waiting (resource available, enters runqueue)
+ waiting --(switch_in)--> running (scheduler picks task, back on CPU)
+ running --(stop)-------> stopped (tlob_stop_task(): window ended, parked)
+ stopped --(start)------> running (tlob_start_task(): window restarted)
+
+ ``tlob_start_task()`` calls ``da_handle_start_run_event(task->pid, ws, start_tlob)``.
+ The ``start_tlob`` edge goes ``stopped`` -> ``running`` for both a fresh
+ allocation (the initial state is ``stopped``) and a parked window's restart;
+ there is no ``start`` self-loop on ``running`` (a running task's START is
+ rejected with ``-EALREADY``). The transition triggers ``ha_setup_invariants()``,
+ which anchors ``clk_elapsed`` and arms the budget timer automatically.
+ ``tlob_stop_task()`` cancels the HA timer synchronously
+ via ``ha_cancel_timer_sync()``, then dispatches the ``stop_tlob`` event
+ (running -> stopped) instead of resetting the monitor: the per-task state
+ is parked, not freed, so a later ``tlob_start_task()`` call for the same
+ task can restart it without reallocating. Final teardown (task exit,
+ uprobe unbind, or monitor disable) is what actually calls
+ ``da_monitor_reset()`` and frees the state.
+
+The non-running condition (monitor not yet started, or reset after a budget
+violation or monitor disable) is handled implicitly by the RV framework
+(``da_mon->monitoring == 0``) - it is not an explicit DA state. A
+``tlob_stop_task()`` does not reset the monitor: the window parks in the
+explicit ``stopped`` state.
+
+Per-task state lives in ``struct tlob_task_state`` which is stored as
+``monitor_target`` in the framework's ``da_monitor_storage``, indexed by
+pid. The per-invocation ``threshold_ns`` is read via
+``ha_get_target(ha_mon)->threshold_ns`` inside the HA constraint functions,
+following the same pattern as the ``nomiss`` monitor.
+
+Usage
+-----
+
+tracefs interface (uprobe-based external monitoring)
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The ``monitor`` tracefs file instruments an unmodified binary via uprobes.
+The format follows the ftrace ``uprobe_events`` convention (``PATH:OFFSET``
+for the probe location, ``key=value`` for configuration parameters)::
+
+ p PATH:OFFSET_START OFFSET_STOP threshold=NS
+
+The uprobe at ``OFFSET_START`` fires ``tlob_start_task()``; the uprobe at
+``OFFSET_STOP`` fires ``tlob_stop_task()``. Both offsets are ELF file
+offsets of entry points in ``PATH``. ``PATH`` may contain ``:``; the last
+``:`` in the ``PATH:OFFSET_START`` token is the separator.
+
+A given task may hit the START/STOP pair any number of times: each pair
+of hits is one independent measurement window, and the underlying
+per-task state is reused across windows rather than reallocated each
+time. Removing a binding while one of its tasks is between windows
+(parked, having already hit STOP) frees that task's state immediately;
+a task still inside a window when its binding is removed keeps running
+unaffected and is cleaned up normally when it next exits.
+
+To remove a binding, use ``-PATH:OFFSET_START``::
+
+ echo 1 > /sys/kernel/tracing/rv/monitors/tlob/enable
+
+ echo "p /usr/bin/myapp:0x12a0 0x12f0 threshold=5000000" \
+ > /sys/kernel/tracing/rv/monitors/tlob/monitor
+
+ # Remove a binding
+ echo "-/usr/bin/myapp:0x12a0" > /sys/kernel/tracing/rv/monitors/tlob/monitor
+
+ # List registered bindings
+ cat /sys/kernel/tracing/rv/monitors/tlob/monitor
+
+ # Read violations from the trace buffer
+ cat /sys/kernel/tracing/trace
+
+Violation tracepoints
+~~~~~~~~~~~~~~~~~~~~~
+
+Two tracepoints are emitted together on a budget violation:
+
+``error_env_tlob``
+ Standard HA clock-invariant tracepoint (emitted by the RV framework).
+ Fields: ``id`` (task pid), ``state``, ``event`` (``"budget_exceeded"``),
+ ``env`` (``"clk_elapsed"``).
+
+``detail_env_tlob``
+ Tlob-specific breakdown of elapsed time per DA state.
+ Fields: ``id`` (task pid), ``threshold_ns``, ``running_ns``,
+ ``waiting_ns``, ``sleeping_ns``.
+
+ Use ``detail_env_tlob`` to diagnose *which phase* consumed the budget:
+ high ``sleeping_ns`` indicates I/O latency; high ``waiting_ns`` indicates
+ scheduler pressure; high ``running_ns`` indicates a compute overrun.
+
+Example: correlate the two tracepoints to see the breakdown::
+
+ trace-cmd record -e error_env_tlob -e detail_env_tlob &
+ # ... run workload ...
+ trace-cmd report
+
+tracefs files
+~~~~~~~~~~~~~
+
+The following files are specific to tlob under
+``/sys/kernel/tracing/rv/monitors/tlob/``:
+
+``monitor`` (rw)
+ Write ``p PATH:OFFSET_START OFFSET_STOP threshold=NS``
+ to bind two entry uprobes. Write ``-PATH:OFFSET_START`` to remove a
+ binding. Read to list registered bindings in the same format.
+ See the `tracefs interface (uprobe-based external monitoring)`_ section above.
+
+Kernel API
+----------
+
+``tlob_start_task`` and ``tlob_stop_task`` are the implementation-level
+functions called by the uprobe entry/exit handlers; the interface is
+driven from userspace.
+
+.. kernel-doc:: kernel/trace/rv/monitors/tlob/tlob.c
+ :functions: tlob_start_task tlob_stop_task
+
+Design notes
+------------
+
+Limitations:
+
+- A fresh window dispatches ``start_tlob`` (initial ``stopped`` ->
+ ``running``) via ``da_handle_start_run_event(task->pid, ws, start_tlob)``,
+ so monitoring always begins in ``running``. Monitoring a non-current
+ task that is already in waiting or sleeping state at call time
+ misclassifies the first interval as ``running_ns``.
+- ``TASK_STOPPED`` and ``TASK_TRACED`` carry ``prev_state != 0`` and are
+ therefore counted as ``sleeping_ns``, indistinguishable from
+ I/O-blocked time.
+- ``sched_wakeup_new`` is not hooked. In practice this is not an issue
+ because ``tlob_start_task`` is always called from a running context.
+
+Specification
+-------------
+
+Graphviz DOT file in tools/verification/models/tlob.dot.
+
+KUnit tests under ``kernel/trace/rv/monitors/tlob/tlob_kunit.c``
+(CONFIG_TLOB_KUNIT_TEST).
+
+User-space integration tests under ``tools/testing/selftests/verification/``
+(requires CONFIG_RV_MON_TLOB=y and root).
diff --git a/kernel/trace/rv/Kconfig b/kernel/trace/rv/Kconfig
index efa930f94ea4..222b3bea8079 100644
--- a/kernel/trace/rv/Kconfig
+++ b/kernel/trace/rv/Kconfig
@@ -84,8 +84,13 @@ source "kernel/trace/rv/monitors/deadline/Kconfig"
source "kernel/trace/rv/monitors/nomiss/Kconfig"
# Add new deadline monitors here
+source "kernel/trace/rv/monitors/tlob/Kconfig"
# Add new monitors here
+config RV_UPROBE
+ bool
+ depends on RV && UPROBES
+
config RV_REACTORS
bool "Runtime verification reactors"
default y
diff --git a/kernel/trace/rv/Makefile b/kernel/trace/rv/Makefile
index cdbf68c84f5a..cd0ec11f0e05 100644
--- a/kernel/trace/rv/Makefile
+++ b/kernel/trace/rv/Makefile
@@ -21,7 +21,9 @@ obj-$(CONFIG_RV_MON_STALL) += monitors/stall/stall.o
obj-$(CONFIG_RV_MON_DEADLINE) += monitors/deadline/deadline.o
obj-$(CONFIG_RV_MON_NOMISS) += monitors/nomiss/nomiss.o
obj-$(CONFIG_RV_MON_WAKEUP) += monitors/wakeup/wakeup.o
+obj-$(CONFIG_RV_MON_TLOB) += monitors/tlob/tlob.o
# Add new monitors here
+obj-$(CONFIG_RV_UPROBE) += rv_uprobe.o
obj-$(CONFIG_RV_REACTORS) += rv_reactors.o
obj-$(CONFIG_RV_REACT_PRINTK) += reactor_printk.o
obj-$(CONFIG_RV_REACT_PANIC) += reactor_panic.o
diff --git a/kernel/trace/rv/monitors/tlob/Kconfig b/kernel/trace/rv/monitors/tlob/Kconfig
new file mode 100644
index 000000000000..aa43382073d2
--- /dev/null
+++ b/kernel/trace/rv/monitors/tlob/Kconfig
@@ -0,0 +1,12 @@
+# SPDX-License-Identifier: GPL-2.0-only
+#
+config RV_MON_TLOB
+ bool "tlob monitor"
+ depends on RV && UPROBES && HIGH_RES_TIMERS
+ select HA_MON_EVENTS_ID
+ select RV_UPROBE
+ help
+ Enable the tlob (task latency over budget) hybrid-automaton RV
+ monitor. tlob tracks per-task elapsed wall-clock time across a
+ user-delimited code section and emits error_env_tlob when the
+ elapsed time exceeds a configurable per-invocation budget.
diff --git a/kernel/trace/rv/monitors/tlob/tlob.c b/kernel/trace/rv/monitors/tlob/tlob.c
new file mode 100644
index 000000000000..99acd34726f1
--- /dev/null
+++ b/kernel/trace/rv/monitors/tlob/tlob.c
@@ -0,0 +1,1132 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * tlob: task latency over budget monitor
+ *
+ * Tracks the elapsed wall-clock time (CLOCK_MONOTONIC) of a marked code
+ * path and flags per-task latency-budget overruns. The hrtimer callback
+ * emits error_env_tlob on violation plus detail_env_tlob, a per-state
+ * (running/waiting/sleeping) time breakdown.
+ *
+ * RV_MON_PER_OBJ: per-task state (struct tlob_task_state) lives as
+ * monitor_target in the framework's hash table. One HA clock invariant:
+ * clk_elapsed < BUDGET_NS() in running/waiting/sleeping (stopped parks).
+ *
+ * Copyright (C) 2026 Wen Yang <wen.yang@linux.dev>
+ */
+#include <linux/kernel.h>
+#include <linux/mempool.h>
+#include <linux/module.h>
+#include <linux/init.h>
+#include <linux/namei.h>
+#include <linux/rv.h>
+#include <linux/slab.h>
+#include <kunit/visibility.h>
+#include <rv/instrumentation.h>
+#include <rv/rv_uprobe.h>
+#include <rv.h>
+
+#define MODULE_NAME "tlob"
+
+#include <trace/events/sched.h>
+#include <rv_trace.h>
+
+/*
+ * Per-task latency monitoring state. One instance per monitoring window.
+ * Stored as monitor_target in da_monitor_storage; freed via call_rcu.
+ */
+enum tlob_acc_idx {
+ TLOB_ACC_RUNNING,
+ TLOB_ACC_WAITING,
+ TLOB_ACC_SLEEPING,
+ TLOB_ACC_MAX,
+};
+
+struct tlob_task_state {
+ struct task_struct *task; /* via get_task_struct */
+ u64 threshold_ns; /* budget in nanoseconds */
+
+ /*
+ * Per-window: 1 = this window ended (stop or timer expiry). Blocks
+ * timer re-arm in ha_setup_invariants(); cleared on restart.
+ */
+ atomic_t stopping;
+
+ /*
+ * Per-task, one-shot: final teardown has claimed this slot; never
+ * reset (a window can end and restart, the task cannot). atomic_t
+ * so cmpxchg is well-defined on every arch.
+ */
+ atomic_t destroying;
+
+ bool budget_exceeded;
+
+ /*
+ * Opaque owner: the binding that started this task. Set once on
+ * fresh allocation (NULL for callers with no binding), cleared by
+ * tlob_unbind_reap() for an active task whose binding is removed.
+ * Immutable elsewhere. Protected by tlob_ws_lock.
+ */
+ void *binding;
+ /*
+ * Linked into binding->started_list for the whole lifetime (not just
+ * while parked) so unbind reaping finds parked and active tasks.
+ * Protected by tlob_ws_lock.
+ */
+ struct list_head started_node;
+
+ /* Serialises accs_ns[]; held briefly (hardirq-safe). */
+ raw_spinlock_t entry_lock;
+ u64 accs_ns[TLOB_ACC_MAX]; /* per-state elapsed ns */
+ ktime_t last_ts;
+
+ struct rcu_head rcu;
+};
+
+#define RV_MON_TYPE RV_MON_PER_OBJ
+#define HA_TIMER_TYPE HA_TIMER_HRTIMER
+
+typedef struct tlob_task_state *monitor_target;
+
+static inline void tlob_reset_notify(struct da_monitor *da_mon);
+#define da_monitor_reset_hook tlob_reset_notify
+
+static inline void tlob_extra_cleanup(struct da_monitor *da_mon);
+#define da_extra_cleanup tlob_extra_cleanup
+
+#define EVENT_NONE_LBL "budget_exceeded"
+
+#include "tlob.h"
+
+#define DA_MON_POOL_SIZE TLOB_MAX_MONITORED
+
+#include <rv/ha_monitor.h>
+
+/*
+ * da_monitor_reset_hook: runs on hrtimer expiry, final teardown, and
+ * monitor disable. A normal stop never resets: tlob_stop_task() dispatches
+ * "stop" (running -> stopped, tlob.dot) instead. Only timer expiry is a
+ * genuine budget violation.
+ */
+static inline void tlob_reset_notify(struct da_monitor *da_mon)
+{
+ struct ha_monitor *ha_mon = to_ha_monitor(da_mon);
+ struct tlob_task_state *ws;
+
+ ha_monitor_reset_env(da_mon);
+
+ ws = ha_get_target(ha_mon);
+ if (!ws)
+ return;
+
+ /*
+ * stopping==1 means tlob_stop_task() ended this window already.
+ * acquire pairs with the _release clear in ha_setup_invariants().
+ */
+ if (atomic_read_acquire(&ws->stopping))
+ return;
+
+ /*
+ * Monitor disable (ha_mon_destroying set) is not a violation: the
+ * teardown paths free ws regardless. Couples to an HA-layer flag
+ * with no public contract; a framework-level equivalent would be
+ * cleaner.
+ */
+ if (unlikely(READ_ONCE(ha_mon_destroying)))
+ return;
+
+ /* Genuine expiry: end the window so a later start takes the restart path. */
+ atomic_set(&ws->stopping, 1);
+
+ /* Stamped regardless of the tracepoint; tlob_stop_task() reads it. */
+ WRITE_ONCE(ws->budget_exceeded, true);
+
+ if (!trace_detail_env_tlob_enabled())
+ return;
+
+ unsigned int curr_state = READ_ONCE(da_mon->curr_state);
+ u64 accs[TLOB_ACC_MAX], partial_ns;
+ unsigned long flags;
+
+ /* Snapshot accumulators; partial_ns covers curr_state time not yet folded in. */
+ raw_spin_lock_irqsave(&ws->entry_lock, flags);
+ partial_ns = ktime_get_ns() - ktime_to_ns(ws->last_ts);
+ accs[TLOB_ACC_RUNNING] = ws->accs_ns[TLOB_ACC_RUNNING] +
+ (curr_state == running_tlob ? partial_ns : 0);
+ accs[TLOB_ACC_WAITING] = ws->accs_ns[TLOB_ACC_WAITING] +
+ (curr_state == waiting_tlob ? partial_ns : 0);
+ accs[TLOB_ACC_SLEEPING] = ws->accs_ns[TLOB_ACC_SLEEPING] +
+ (curr_state == sleeping_tlob ? partial_ns : 0);
+ raw_spin_unlock_irqrestore(&ws->entry_lock, flags);
+
+ trace_detail_env_tlob(da_get_id(da_mon), ws->threshold_ns,
+ accs[TLOB_ACC_RUNNING],
+ accs[TLOB_ACC_WAITING],
+ accs[TLOB_ACC_SLEEPING]);
+}
+
+#define BUDGET_NS(ha_mon) (ha_get_target(ha_mon)->threshold_ns)
+
+/* HA constraint functions (called by ha_monitor_handle_constraint) */
+
+static u64 ha_get_env(struct ha_monitor *ha_mon, enum envs_tlob env,
+ u64 time_ns)
+{
+ if (env == clk_elapsed_tlob)
+ return ha_get_clk_ns(ha_mon, env, time_ns);
+ return ENV_INVALID_VALUE;
+}
+
+/*
+ * Invariant: clk_elapsed < BUDGET_NS in running/waiting/sleeping. "stopped"
+ * is exempt: the parked period must not be measured against the old window's
+ * clock anchor (restart from "stopped" would otherwise spuriously overrun).
+ */
+static inline bool ha_verify_invariants(struct ha_monitor *ha_mon,
+ enum states curr_state, enum events event,
+ enum states next_state, u64 time_ns)
+{
+ if (curr_state == stopped_tlob)
+ return true;
+ return ha_check_invariant_ns(ha_mon, clk_elapsed_tlob, time_ns, BUDGET_NS(ha_mon));
+}
+
+/*
+ * The clock stays in guard (anchor) representation all window: env_store
+ * holds the window-start timestamp, re-anchored on start/restart.
+ * ha_invariant_passed_ns() never stores the deadline representation (the
+ * framework dropped ha_set_invariant_ns(), commit ab2900ae252b), so calling
+ * ha_inv_to_guard() here would subtract BUDGET_NS from the anchor and skew
+ * every check by one budget. nomiss likewise never converts.
+ */
+
+/* No per-event guard conditions for tlob; invariants suffice. */
+static inline bool ha_verify_guards(struct ha_monitor *ha_mon,
+ enum states curr_state, enum events event,
+ enum states next_state, u64 time_ns)
+{
+ return true;
+}
+
+/*
+ * Guard on stopping: a sched_switch after ha_cancel_timer_sync() would
+ * re-arm the timer (ODEBUG splat). _acquire pairs with cmpxchg_release in
+ * tlob_stop_task.
+ *
+ * Entering stopped_tlob also resets env_store to the invalid sentinel, so a
+ * restart re-anchors the clock; a stale anchor would wrap the restart's
+ * timer delay to ~U64_MAX.
+ */
+static inline void ha_setup_invariants(struct ha_monitor *ha_mon,
+ enum states curr_state, enum events event,
+ enum states next_state, u64 time_ns)
+{
+ if (next_state == stopped_tlob) {
+ /*
+ * Window ending: reset env_store to the invalid sentinel so
+ * the next window gets a fresh clock anchor. Keep stopping==1
+ * so __tlob_acc() continues to block sched events while parked.
+ */
+ ha_monitor_reset_all_stored(ha_mon);
+ return;
+ }
+
+ if (atomic_read_acquire(&ha_get_target(ha_mon)->stopping)) {
+ /*
+ * Restart (stopped -> running): arm the timer, then clear
+ * stopping so __tlob_acc() admits sched events only once the
+ * state is already running_tlob. _release pairs with the
+ * acquires in __tlob_acc/tlob_reset_notify.
+ */
+ if (next_state < state_max_tlob)
+ ha_start_timer_ns(ha_mon, clk_elapsed_tlob, BUDGET_NS(ha_mon), time_ns);
+ atomic_set_release(&ha_get_target(ha_mon)->stopping, 0);
+ return;
+ }
+
+ if (next_state < state_max_tlob)
+ ha_start_timer_ns(ha_mon, clk_elapsed_tlob, BUDGET_NS(ha_mon), time_ns);
+ else
+ ha_cancel_timer(ha_mon);
+}
+
+static bool ha_verify_constraint(struct ha_monitor *ha_mon,
+ enum states curr_state, enum events event,
+ enum states next_state, u64 time_ns)
+{
+ if (!ha_verify_invariants(ha_mon, curr_state, event, next_state, time_ns))
+ return false;
+
+ if (!ha_verify_guards(ha_mon, curr_state, event, next_state, time_ns))
+ return false;
+
+ ha_setup_invariants(ha_mon, curr_state, event, next_state, time_ns);
+
+ return true;
+}
+
+/*
+ * Pre-allocated pool of TLOB_MAX_MONITORED slots. mempool_alloc_preallocated()
+ * pops a reserve slot without touching the allocator (bounded start latency;
+ * -ENOSPC past the cap). Slots return via destroy/cleanup; mempool_free() is
+ * safe from RCU-callback context.
+ */
+static mempool_t tlob_ws_pool;
+
+static void tlob_ws_return_cb(struct rcu_head *head)
+{
+ struct tlob_task_state *ws =
+ container_of(head, struct tlob_task_state, rcu);
+
+ mempool_free(ws, &tlob_ws_pool);
+}
+
+/* Direct return without RCU delay (ws was never published to the hash). */
+static void tlob_ws_direct_return(struct tlob_task_state *ws)
+{
+ mempool_free(ws, &tlob_ws_pool);
+}
+
+static struct tlob_task_state *tlob_ws_alloc(void)
+{
+ struct tlob_task_state *ws =
+ mempool_alloc_preallocated(&tlob_ws_pool);
+
+ if (!ws)
+ return NULL;
+
+ memset(ws, 0, sizeof(*ws));
+ INIT_LIST_HEAD(&ws->started_node);
+ return ws;
+}
+
+/*
+ * Uprobe binding list; protected by tlob_uprobe_mutex. When both are
+ * taken, tlob_uprobe_mutex is always acquired before tlob_ws_lock:
+ * inverting the order would be a silent lock-order inversion.
+ */
+static LIST_HEAD(tlob_uprobe_list);
+static DEFINE_MUTEX(tlob_uprobe_mutex);
+
+/* Serialises tlob_task_state ownership: restart, detach, unbind reap. */
+static DEFINE_SPINLOCK(tlob_ws_lock);
+
+/* Per-uprobe-binding state: a start + stop probe pair for one binary region. */
+struct tlob_uprobe_binding {
+ struct list_head list;
+ u64 threshold_ns;
+ char binpath[TLOB_MAX_PATH];
+ loff_t offset_start;
+ loff_t offset_stop;
+ /*
+ * All tlob_task_states this binding ever started, for each task's
+ * lifetime. Protected by tlob_ws_lock.
+ */
+ struct list_head started_list;
+ DECLARE_RV_UPROBE(start_probe);
+ DECLARE_RV_UPROBE(stop_probe);
+};
+
+/*
+ * Unlink ws from its binding's started_list before returning it to the pool.
+ * ws->binding is left stale: the next tlob_ws_alloc() memsets it, and the
+ * restart path checks destroying first. Idempotent (list_del_init no-op).
+ */
+static inline void tlob_detach_from_binding(struct tlob_task_state *ws)
+{
+ if (!ws->binding)
+ return;
+ guard(spinlock)(&tlob_ws_lock);
+ list_del_init(&ws->started_node);
+}
+
+/*
+ * Per-entry teardown during monitor disable. cmpxchg on destroying
+ * (0->1) claims ownership -- not stopping, which can be long-lived (a
+ * parked task).
+ *
+ * No timer cancel or locking needed: disable_tlob() already synced every
+ * uprobe/tracepoint, da_monitor_destroy() ran da_monitor_reset_all() +
+ * synchronize_rcu(), and ha_mon_destroying blocks new timer callbacks.
+ */
+static inline void tlob_extra_cleanup(struct da_monitor *da_mon)
+{
+ struct ha_monitor *ha_mon = to_ha_monitor(da_mon);
+ struct tlob_task_state *ws = ha_get_target(ha_mon);
+
+ if (!ws)
+ return;
+
+ if (atomic_cmpxchg_release(&ws->destroying, 0, 1) != 0)
+ return;
+
+ tlob_detach_from_binding(ws);
+ put_task_struct(ws->task);
+ /*
+ * da_monitor_destroy() has already called synchronize_rcu(); no
+ * reader holds ws. Return the slot directly without call_rcu.
+ */
+ mempool_free(ws, &tlob_ws_pool);
+}
+
+/*
+ * Accumulate elapsed ns into accs_ns[idx] since last_ts and advance it.
+ * Returns true if monitored with an active window. The stopping gate is
+ * what keeps scheduler events from reaching a parked task (no "stopped"
+ * self-loops, see tlob.h) and keeps accs_ns[] from growing while parked.
+ */
+static inline bool __tlob_acc(struct task_struct *task, ktime_t now,
+ enum tlob_acc_idx idx)
+{
+ struct tlob_task_state *ws;
+ unsigned long flags;
+
+ guard(rcu)();
+ ws = da_get_target_by_id(task->pid);
+ /* acquire pairs with the _release clear in ha_setup_invariants(). */
+ if (!ws || atomic_read_acquire(&ws->stopping))
+ return false;
+ raw_spin_lock_irqsave(&ws->entry_lock, flags);
+ ws->accs_ns[idx] += ktime_to_ns(ktime_sub(now, ws->last_ts));
+ ws->last_ts = now;
+ raw_spin_unlock_irqrestore(&ws->entry_lock, flags);
+ return true;
+}
+
+static inline bool tlob_acc_running(struct task_struct *task, ktime_t now)
+{
+ return __tlob_acc(task, now, TLOB_ACC_RUNNING);
+}
+
+static inline bool tlob_acc_waiting(struct task_struct *task, ktime_t now)
+{
+ return __tlob_acc(task, now, TLOB_ACC_WAITING);
+}
+
+/*
+ * handle_sched_switch - advance the DA on every context switch.
+ *
+ * Emits sleep (running -> sleeping), preempt (running -> waiting) for prev,
+ * and switch_in (waiting -> running) for next. One ktime_get() shared by
+ * both acc calls keeps prev/next on the same context-switch timestamp.
+ *
+ * No waiting->sleeping edge: a task blocks (calls schedule()) only on CPU
+ * (running); waiting means TASK_RUNNING on the runqueue.
+ */
+static void handle_sched_switch(void *data, bool preempt_unused,
+ struct task_struct *prev,
+ struct task_struct *next,
+ unsigned int prev_state)
+{
+ ktime_t now = ktime_get();
+ bool prev_preempted = (prev_state == 0);
+
+ if (tlob_acc_running(prev, now))
+ da_handle_event(prev->pid, NULL,
+ prev_preempted ? preempt_tlob : sleep_tlob);
+ if (tlob_acc_waiting(next, now))
+ da_handle_event(next->pid, NULL, switch_in_tlob);
+}
+
+static inline bool tlob_acc_sleeping(struct task_struct *task, ktime_t now)
+{
+ return __tlob_acc(task, now, TLOB_ACC_SLEEPING);
+}
+
+/*
+ * handle_sched_wakeup - sleeping -> waiting transition. try_to_wake_up()
+ * skips TASK_RUNNING tasks, so this never fires for running/waiting.
+ */
+static void handle_sched_wakeup(void *data, struct task_struct *p)
+{
+ ktime_t now = ktime_get();
+
+ if (tlob_acc_sleeping(p, now))
+ da_handle_event(p->pid, NULL, wakeup_tlob);
+}
+
+/* Forward decl: used by handle_sched_process_exit() and tlob_unbind_reap(). */
+static int tlob_stop_task(struct task_struct *task, void *binding);
+static void tlob_destroy_task(struct task_struct *task);
+
+/*
+ * handle_sched_process_exit - clean up a task that exits without hitting its
+ * STOP uprobe (killed, unmapped mid-region, ...). The task is always in
+ * running_tlob here: do_exit() runs in the task's own context, which
+ * required passing through switch_in_tlob (running). tlob_stop_task() ends
+ * the window (or is a harmless -EAGAIN/-ESRCH); tlob_destroy_task() then
+ * frees the slot, as no restart can follow exit.
+ */
+static void handle_sched_process_exit(void *data, struct task_struct *p,
+ bool group_dead)
+{
+ tlob_stop_task(p, NULL);
+ tlob_destroy_task(p);
+}
+
+/**
+ * tlob_start_task - begin monitoring @task with budget @threshold_ns ns.
+ * @task: Task to monitor; may be current or another task.
+ * @threshold_ns: Budget in ns, in [1000, TLOB_MAX_THRESHOLD_NS].
+ * @binding: Opaque owner, recorded on fresh allocation and checked for
+ * an exact match on restart; NULL for callers that never
+ * restart a parked window.
+ *
+ * Allocates a fresh entry if @task has none, or restarts a parked entry in
+ * place when @binding matches (see tlob.dot: "start" fires from both
+ * running and stopped).
+ *
+ * Returns 0, -ENODEV, -ERANGE, -EALREADY, -ESRCH, or -ENOSPC (fresh start
+ * past pool capacity).
+ */
+static int tlob_start_task(struct task_struct *task, u64 threshold_ns, void *binding)
+{
+ struct tlob_task_state *ws;
+
+ if (!da_monitor_enabled())
+ return -ENODEV;
+
+ if (threshold_ns < TLOB_MIN_THRESHOLD_NS ||
+ threshold_ns > TLOB_MAX_THRESHOLD_NS)
+ return -ERANGE;
+
+ /* Serialise duplicate-check + pool-slot claim; see tlob_ws_lock. */
+ guard(spinlock)(&tlob_ws_lock);
+
+ /*
+ * da_get_target_by_id() uses hash_for_each_possible_rcu(), which
+ * requires an RCU read-side critical section.
+ */
+ scoped_guard(rcu) {
+ ws = da_get_target_by_id(task->pid);
+ if (ws) {
+ if (!atomic_read(&ws->stopping))
+ return -EALREADY;
+ if (atomic_read(&ws->destroying))
+ return -ESRCH;
+ /*
+ * Exact match only. An orphaned parked ws (binding
+ * cleared while active, then parked) is not adopted:
+ * that would need re-linking into the new binding's
+ * started_list. Accepted gap; the slot is reclaimed
+ * at task exit.
+ */
+ if (ws->binding != binding)
+ return -EALREADY;
+
+ /* Restart in place: same slot, hash entry, task ref, list node. */
+ ws->threshold_ns = threshold_ns;
+ WRITE_ONCE(ws->budget_exceeded, false);
+ memset(ws->accs_ns, 0, sizeof(ws->accs_ns));
+ ws->last_ts = ktime_get();
+
+ /*
+ * Keep stopping set: __tlob_acc() gates out sched
+ * events until ha_setup_invariants() clears it after
+ * the state is running_tlob. Clearing here would let
+ * events hit stopped_tlob (INVALID transitions).
+ */
+
+ /* Only failure here: monitor disabled since the check above. */
+ if (!da_handle_start_run_event(task->pid, ws, start_tlob))
+ return -ENODEV;
+ return 0;
+ }
+ }
+
+ ws = tlob_ws_alloc();
+ if (!ws)
+ return -ENOSPC;
+
+ ws->task = task;
+ get_task_struct(task);
+ ws->threshold_ns = threshold_ns;
+ ws->last_ts = ktime_get();
+ raw_spin_lock_init(&ws->entry_lock);
+ ws->binding = binding;
+ if (binding)
+ list_add_tail(&ws->started_node,
+ &((struct tlob_uprobe_binding *)binding)->started_list);
+
+ /* Dispatch failed (pool exhausted or monitor disabled): unwind the slot. */
+ if (!da_handle_start_run_event(task->pid, ws, start_tlob)) {
+ if (binding)
+ list_del_init(&ws->started_node);
+ put_task_struct(task);
+ tlob_ws_direct_return(ws);
+ return -ENOSPC;
+ }
+
+ return 0;
+}
+
+/**
+ * tlob_stop_task - end the current monitoring window for @task.
+ * @task: Task to stop.
+ * @binding: Opaque owner; must match ws->binding to end a normal (uprobe)
+ * window. NULL (task exit) skips the check.
+ *
+ * Ends the window (dispatches "stop") but does NOT free the entry: it stays
+ * parked so a later tlob_start_task() can restart it. Call
+ * tlob_destroy_task() once @task will never restart.
+ *
+ * cmpxchg on stopping (0->1) under RCU claims ownership; the winner cancels
+ * the timer synchronously.
+ *
+ * Returns 0, -EOVERFLOW (budget exceeded), -ESRCH (not monitored),
+ * -EAGAIN (window already ended), or -EALREADY (owned by another binding).
+ */
+static int tlob_stop_task(struct task_struct *task, void *binding)
+{
+ struct ha_monitor *ha_mon;
+ struct tlob_task_state *ws;
+ bool budget_exceeded;
+
+ scoped_guard(rcu) {
+ ha_mon = ha_get_monitor(task->pid, NULL);
+ if (!ha_mon)
+ return -ESRCH;
+
+ ws = ha_get_target(ha_mon);
+ if (WARN_ON_ONCE(!ws))
+ return -ESRCH;
+
+ /* Only the binding that opened the window may end it; NULL
+ * (task exit) skips the check. Symmetric with the restart
+ * check in tlob_start_task(). */
+ if (binding && ws->binding != binding)
+ return -EALREADY;
+
+ /* cmpxchg (0->1) claims the window under RCU; _release pairs
+ * with the acquire in ha_setup_invariants(). */
+ if (atomic_cmpxchg_release(&ws->stopping, 0, 1) != 0)
+ return -EAGAIN;
+
+ /*
+ * ws may be destroyed concurrently (unbind -> call_rcu), so
+ * keep its access under RCU; dispatch re-looks-up under RCU.
+ */
+ ha_cancel_timer_sync(ha_mon);
+ budget_exceeded = READ_ONCE(ws->budget_exceeded);
+ }
+
+ /* running -> stopped: no reset or destroy, the entry stays parked. */
+ da_handle_event(task->pid, NULL, stop_tlob);
+
+ return budget_exceeded ? -EOVERFLOW : 0;
+}
+
+/*
+ * tlob_destroy_task - final teardown for @task's entry: frees the pool slot,
+ * drops the task_struct ref, removes the hash entry, whether active or parked.
+ * Idempotent via the destroying cmpxchg (same pattern as tlob_extra_cleanup()).
+ * Callers must end the window first (see handle_sched_process_exit()).
+ */
+static void tlob_destroy_task(struct task_struct *task)
+{
+ struct ha_monitor *ha_mon;
+ struct tlob_task_state *ws;
+
+ scoped_guard(rcu) {
+ ha_mon = ha_get_monitor(task->pid, NULL);
+ if (!ha_mon)
+ return;
+ ws = ha_get_target(ha_mon);
+ if (WARN_ON_ONCE(!ws))
+ return;
+ if (atomic_cmpxchg_release(&ws->destroying, 0, 1) != 0)
+ return;
+ }
+
+ tlob_detach_from_binding(ws);
+
+ /* Force the window ended: @task may never have reached STOP or a timer. */
+ atomic_set(&ws->stopping, 1);
+ ha_cancel_timer_sync(ha_mon);
+
+ scoped_guard(rcu) {
+ da_monitor_reset(&ha_mon->da_mon);
+ }
+ da_destroy_storage(task->pid);
+
+ put_task_struct(ws->task);
+ call_rcu(&ws->rcu, tlob_ws_return_cb);
+}
+
+static int tlob_uprobe_entry_handler(struct uprobe_consumer *self,
+ struct pt_regs *regs, __u64 *data)
+{
+ struct tlob_uprobe_binding *b =
+ container_of(self, struct tlob_uprobe_binding, start_probe.uc);
+
+ tlob_start_task(current, b->threshold_ns, b);
+ return 0;
+}
+
+static int tlob_uprobe_stop_handler(struct uprobe_consumer *self,
+ struct pt_regs *regs, __u64 *data)
+{
+ struct tlob_uprobe_binding *b =
+ container_of(self, struct tlob_uprobe_binding, stop_probe.uc);
+
+ tlob_stop_task(current, b);
+ return 0;
+}
+
+/*
+ * Register start + stop entry uprobes for a binding.
+ * Called with tlob_uprobe_mutex held.
+ */
+static int tlob_add_uprobe(u64 threshold_ns, const char *binpath,
+ loff_t offset_start, loff_t offset_stop)
+{
+ struct tlob_uprobe_binding *tmp_b;
+ char pathbuf[TLOB_MAX_PATH];
+ struct inode *inode;
+ struct path path __free(path_put) = {};
+ char *canon;
+ int ret;
+
+ if (binpath[0] != '/')
+ return -EINVAL;
+
+ struct tlob_uprobe_binding *b __free(kfree) = kzalloc_obj(*b, GFP_KERNEL);
+ if (!b)
+ return -ENOMEM;
+
+ b->threshold_ns = threshold_ns;
+ b->offset_start = offset_start;
+ b->offset_stop = offset_stop;
+ INIT_LIST_HEAD(&b->started_list);
+
+ ret = kern_path(binpath, LOOKUP_FOLLOW, &path);
+ if (ret)
+ return ret;
+
+ if (!d_is_reg(path.dentry))
+ return -EINVAL;
+
+ inode = d_real_inode(path.dentry);
+
+ /* Reject duplicate start offset for the same binary inode. */
+ list_for_each_entry(tmp_b, &tlob_uprobe_list, list) {
+ if (tmp_b->offset_start == offset_start &&
+ rv_uprobe_is_registered(&tmp_b->start_probe) &&
+ d_real_inode(tmp_b->start_probe.path.dentry) == inode)
+ return -EEXIST;
+ }
+
+ canon = d_path(&path, pathbuf, sizeof(pathbuf));
+ if (IS_ERR(canon))
+ return PTR_ERR(canon);
+ strscpy(b->binpath, canon, sizeof(b->binpath));
+
+ b->start_probe.uc.handler = tlob_uprobe_entry_handler;
+ ret = rv_uprobe_register(b->binpath, offset_start, &b->start_probe);
+ if (ret)
+ return ret;
+
+ b->stop_probe.uc.handler = tlob_uprobe_stop_handler;
+ ret = rv_uprobe_register(b->binpath, offset_stop, &b->stop_probe);
+ if (ret) {
+ rv_uprobe_unregister(&b->start_probe);
+ return ret;
+ }
+
+ /* NOT "b = no_free_ptr(b)": the re-assignment would free the live node. */
+ list_add_tail(&no_free_ptr(b)->list, &tlob_uprobe_list);
+ return 0;
+}
+
+/*
+ * tlob_unbind_reap - detach every task @b started, destroy the parked ones.
+ *
+ * Caller must have unregistered @b's uprobes and called rv_uprobe_sync():
+ * no start/stop can then be in flight for @b, so started_list is safe to
+ * walk. Active tasks are detached (binding cleared) and left running,
+ * matching unbind behaviour today; parked tasks are destroyed, or their
+ * pool slot leaks until the task next exits.
+ */
+static void tlob_unbind_reap(struct tlob_uprobe_binding *b)
+{
+ struct tlob_task_state *ws, *tmp;
+ LIST_HEAD(to_destroy);
+
+ scoped_guard(spinlock, &tlob_ws_lock) {
+ list_for_each_entry_safe(ws, tmp, &b->started_list, started_node) {
+ list_del_init(&ws->started_node);
+ ws->binding = NULL;
+ if (atomic_read(&ws->stopping))
+ list_add_tail(&ws->started_node, &to_destroy);
+ }
+ }
+
+ list_for_each_entry_safe(ws, tmp, &to_destroy, started_node) {
+ list_del_init(&ws->started_node);
+ tlob_destroy_task(ws->task);
+ }
+}
+
+static int tlob_remove_uprobe_by_key(loff_t offset_start, const char *binpath)
+{
+ struct tlob_uprobe_binding *b, *tmp;
+ struct path remove_path;
+ struct inode *inode;
+ int ret;
+
+ ret = kern_path(binpath, LOOKUP_FOLLOW, &remove_path);
+ if (ret)
+ return ret;
+
+ inode = d_real_inode(remove_path.dentry);
+
+ ret = -ENOENT;
+ list_for_each_entry_safe(b, tmp, &tlob_uprobe_list, list) {
+ if (b->offset_start != offset_start)
+ continue;
+ if (d_real_inode(b->start_probe.path.dentry) != inode)
+ continue;
+ list_del(&b->list);
+ /*
+ * rv_uprobe_sync() may sleep; list_del() already made the
+ * binding invisible to new readers.
+ */
+ rv_uprobe_unregister_nosync(&b->start_probe);
+ rv_uprobe_unregister_nosync(&b->stop_probe);
+ rv_uprobe_sync();
+ tlob_unbind_reap(b);
+ path_put(&b->start_probe.path);
+ path_put(&b->stop_probe.path);
+ kfree(b);
+ ret = 0;
+ break;
+ }
+
+ path_put(&remove_path);
+ return ret;
+}
+
+static void tlob_remove_all_uprobes(void)
+{
+ struct tlob_uprobe_binding *b, *tmp;
+ LIST_HEAD(pending);
+
+ mutex_lock(&tlob_uprobe_mutex);
+ list_for_each_entry_safe(b, tmp, &tlob_uprobe_list, list) {
+ list_move(&b->list, &pending);
+ rv_uprobe_unregister_nosync(&b->start_probe);
+ rv_uprobe_unregister_nosync(&b->stop_probe);
+ }
+ mutex_unlock(&tlob_uprobe_mutex);
+
+ if (list_empty(&pending))
+ return;
+
+ /* One sync covers all dequeued probes: consumers are then safe to free. */
+ rv_uprobe_sync();
+
+ list_for_each_entry_safe(b, tmp, &pending, list) {
+ list_del(&b->list);
+ tlob_unbind_reap(b);
+ path_put(&b->start_probe.path);
+ path_put(&b->stop_probe.path);
+ kfree(b);
+ }
+}
+
+static ssize_t tlob_monitor_read(struct file *file,
+ char __user *ubuf,
+ size_t count, loff_t *ppos)
+{
+ const int line_sz = TLOB_MAX_PATH + 128;
+ struct tlob_uprobe_binding *b;
+ char *buf;
+ int n = 0, buf_sz, pos = 0;
+ ssize_t ret;
+
+ mutex_lock(&tlob_uprobe_mutex);
+ list_for_each_entry(b, &tlob_uprobe_list, list)
+ n++;
+
+ buf_sz = (n ? n : 1) * line_sz + 1;
+ buf = kmalloc(buf_sz, GFP_KERNEL);
+ if (!buf) {
+ mutex_unlock(&tlob_uprobe_mutex);
+ return -ENOMEM;
+ }
+
+ list_for_each_entry(b, &tlob_uprobe_list, list) {
+ pos += scnprintf(buf + pos, buf_sz - pos,
+ "p %s:0x%llx 0x%llx threshold=%llu\n",
+ b->binpath,
+ (unsigned long long)b->offset_start,
+ (unsigned long long)b->offset_stop,
+ b->threshold_ns);
+ }
+ mutex_unlock(&tlob_uprobe_mutex);
+
+ ret = simple_read_from_buffer(ubuf, count, ppos, buf, pos);
+ kfree(buf);
+ return ret;
+}
+
+/*
+ * Parse "p PATH:OFFSET_START OFFSET_STOP threshold=NS".
+ * PATH may contain ':'; the last ':' separates path from offset.
+ * Returns 0, -EINVAL, or -ERANGE.
+ */
+static int tlob_parse_uprobe_line(char *buf, u64 *thr_out,
+ char **path_out,
+ loff_t *start_out, loff_t *stop_out)
+{
+ unsigned long long thr = 0, stop_val = 0;
+ long long start_val;
+ char *p, *path_token, *token, *colon;
+ bool got_stop = false, got_thr = false;
+ int n;
+
+ /* Must start with "p " */
+ if (buf[0] != 'p' || buf[1] != ' ')
+ return -EINVAL;
+
+ p = buf + 2;
+ while (*p == ' ')
+ p++;
+
+ /* First space-delimited token is PATH:OFFSET_START */
+ path_token = strsep(&p, " \t");
+ if (!path_token || !*path_token)
+ return -EINVAL;
+
+ /* Split at last ':' to handle paths that contain ':'. */
+ colon = strrchr(path_token, ':');
+ if (!colon || colon - path_token < 2)
+ return -EINVAL;
+ *colon = '\0';
+
+ if (path_token[0] != '/')
+ return -EINVAL;
+
+ n = 0;
+ if (sscanf(colon + 1, "%lli%n", &start_val, &n) != 1 || n == 0)
+ return -EINVAL;
+ if (start_val < 0)
+ return -EINVAL;
+
+ /* Remaining tokens: OFFSET_STOP threshold=NS */
+ while (p && (token = strsep(&p, " \t")) != NULL) {
+ if (!*token)
+ continue;
+ if (strncmp(token, "threshold=", 10) == 0) {
+ if (kstrtoull(token + 10, 0, &thr))
+ return -EINVAL;
+ if (thr < TLOB_MIN_THRESHOLD_NS || thr > TLOB_MAX_THRESHOLD_NS)
+ return -ERANGE;
+ got_thr = true;
+ } else if (!got_stop) {
+ long long sv;
+
+ n = 0;
+ if (sscanf(token, "%lli%n", &sv, &n) != 1 || n == 0)
+ return -EINVAL;
+ if (sv < 0)
+ return -EINVAL;
+ stop_val = (unsigned long long)sv;
+ got_stop = true;
+ } else {
+ return -EINVAL;
+ }
+ }
+
+ if (!got_stop || !got_thr)
+ return -EINVAL;
+ if (start_val == (long long)stop_val)
+ return -EINVAL;
+
+ *thr_out = thr;
+ *path_out = path_token;
+ *start_out = (loff_t)start_val;
+ *stop_out = (loff_t)stop_val;
+ return 0;
+}
+
+/*
+ * Parse "-PATH:OFFSET_START" (ftrace uprobe_events removal convention).
+ */
+static int tlob_parse_remove_line(char *buf, char **path_out,
+ loff_t *start_out)
+{
+ char *binpath, *colon;
+ long long off;
+ int n = 0;
+
+ if (buf[0] != '-')
+ return -EINVAL;
+ binpath = buf + 1;
+ if (binpath[0] != '/')
+ return -EINVAL;
+ colon = strrchr(binpath, ':');
+ if (!colon || colon - binpath < 2)
+ return -EINVAL;
+ *colon = '\0';
+ if (sscanf(colon + 1, "%lli%n", &off, &n) != 1 || n == 0)
+ return -EINVAL;
+ if (off < 0)
+ return -EINVAL;
+ *path_out = binpath;
+ *start_out = (loff_t)off;
+ return 0;
+}
+
+static int tlob_create_or_delete_uprobe(char *buf)
+{
+ loff_t offset_start, offset_stop;
+ u64 threshold_ns;
+ char *binpath;
+ int ret;
+
+ if (buf[0] == '-') {
+ ret = tlob_parse_remove_line(buf, &binpath, &offset_start);
+ if (ret)
+ return ret;
+ mutex_lock(&tlob_uprobe_mutex);
+ ret = tlob_remove_uprobe_by_key(offset_start, binpath);
+ mutex_unlock(&tlob_uprobe_mutex);
+ return ret;
+ }
+ ret = tlob_parse_uprobe_line(buf, &threshold_ns, &binpath,
+ &offset_start, &offset_stop);
+ if (ret)
+ return ret;
+ mutex_lock(&tlob_uprobe_mutex);
+ ret = tlob_add_uprobe(threshold_ns, binpath, offset_start, offset_stop);
+ mutex_unlock(&tlob_uprobe_mutex);
+ return ret;
+}
+
+static ssize_t tlob_monitor_write(struct file *file,
+ const char __user *ubuf,
+ size_t count, loff_t *ppos)
+{
+ char buf[TLOB_MAX_PATH + 128];
+
+ if (count >= sizeof(buf))
+ return -EINVAL;
+ if (copy_from_user(buf, ubuf, count))
+ return -EFAULT;
+ buf[count] = '\0';
+ if (count > 0 && buf[count - 1] == '\n')
+ buf[count - 1] = '\0';
+ return tlob_create_or_delete_uprobe(buf) ?: (ssize_t)count;
+}
+
+static const struct file_operations tlob_monitor_fops = {
+ .open = simple_open,
+ .read = tlob_monitor_read,
+ .write = tlob_monitor_write,
+ .llseek = noop_llseek,
+};
+
+static int __tlob_init_monitor(void)
+{
+ int retval;
+
+ retval = mempool_init_kmalloc_pool(&tlob_ws_pool, TLOB_MAX_MONITORED,
+ sizeof(struct tlob_task_state));
+ if (retval)
+ return retval;
+
+ retval = ha_monitor_init();
+ if (retval) {
+ mempool_exit(&tlob_ws_pool);
+ return retval;
+ }
+
+ rv_this.enabled = 1;
+ return 0;
+}
+
+static void __tlob_destroy_monitor(void)
+{
+ rv_this.enabled = 0;
+ tlob_remove_all_uprobes();
+ /*
+ * A grace period only makes the call_rcu()'d tlob_ws_return_cb()
+ * callbacks eligible to run; rcu_barrier() waits until they have all
+ * returned their slots before the pool is destroyed.
+ */
+ ha_monitor_destroy();
+ rcu_barrier();
+ mempool_exit(&tlob_ws_pool);
+}
+
+static int tlob_enable_hooks(void)
+{
+ rv_attach_trace_probe("tlob", sched_switch, handle_sched_switch);
+ rv_attach_trace_probe("tlob", sched_wakeup, handle_sched_wakeup);
+ rv_attach_trace_probe("tlob", sched_process_exit, handle_sched_process_exit);
+ return 0;
+}
+
+static void tlob_disable_hooks(void)
+{
+ rv_detach_trace_probe("tlob", sched_switch, handle_sched_switch);
+ rv_detach_trace_probe("tlob", sched_wakeup, handle_sched_wakeup);
+ rv_detach_trace_probe("tlob", sched_process_exit, handle_sched_process_exit);
+}
+
+static int enable_tlob(void)
+{
+ int retval;
+
+ retval = __tlob_init_monitor();
+ if (retval)
+ return retval;
+
+ return tlob_enable_hooks();
+}
+
+static void disable_tlob(void)
+{
+ tlob_disable_hooks();
+ __tlob_destroy_monitor();
+}
+
+static struct rv_monitor rv_this = {
+ .name = "tlob",
+ .description = "Per-task latency-over-budget monitor.",
+ .enable = enable_tlob,
+ .disable = disable_tlob,
+ .reset = da_monitor_reset_all,
+ .enabled = 0,
+};
+
+static int __init register_tlob(void)
+{
+ int ret;
+
+ ret = rv_register_monitor(&rv_this, NULL);
+ if (ret)
+ return ret;
+
+ if (rv_this.root_d) {
+ if (!rv_create_file("monitor", RV_MODE_WRITE, rv_this.root_d, NULL,
+ &tlob_monitor_fops)) {
+ rv_unregister_monitor(&rv_this);
+ return -ENOMEM;
+ }
+ }
+
+ return 0;
+}
+
+static void __exit unregister_tlob(void)
+{
+ rv_unregister_monitor(&rv_this);
+}
+
+module_init(register_tlob);
+module_exit(unregister_tlob);
+
+MODULE_LICENSE("GPL");
+MODULE_AUTHOR("Wen Yang <wen.yang@linux.dev>");
+MODULE_DESCRIPTION("tlob: task latency over budget per-task monitor.");
diff --git a/kernel/trace/rv/monitors/tlob/tlob.h b/kernel/trace/rv/monitors/tlob/tlob.h
new file mode 100644
index 000000000000..94e7382c2130
--- /dev/null
+++ b/kernel/trace/rv/monitors/tlob/tlob.h
@@ -0,0 +1,149 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _RV_TLOB_H
+#define _RV_TLOB_H
+
+/*
+ * C representation of the tlob hybrid automaton (see tlob.dot).
+ *
+ * States: stopped (initial; parked), running (on CPU), waiting (runqueue),
+ * sleeping (blocked). Events: start/stop (tlob_start_task/tlob_stop_task),
+ * sleep/preempt/wakeup/switch_in (sched tracepoints).
+ *
+ * "stop" fires only from running (both callers run on CPU); "stopped"
+ * leaves only via "start" (fresh start or in-place restart). running[start]
+ * is INVALID: a stray re-start must not silently reset the budget clock.
+ *
+ * Invariant: clk_elapsed < BUDGET_NS() in running/waiting/sleeping; stopped
+ * parks the window, no clock while parked. start re-inits the monitor
+ * (da_handle_start_run_event()); stop dispatches after ha_cancel_timer_sync();
+ * final teardown uses ha_cancel_timer_sync() + da_monitor_reset() +
+ * da_destroy_storage().
+ *
+ * Format: Documentation/trace/rv/deterministic_automata.rst
+ */
+
+#include <linux/rv.h>
+#include <linux/sched.h>
+
+#define MONITOR_NAME tlob
+
+enum states_tlob {
+ stopped_tlob,
+ running_tlob,
+ sleeping_tlob,
+ waiting_tlob,
+ state_max_tlob,
+};
+
+#define INVALID_STATE state_max_tlob
+
+enum events_tlob {
+ preempt_tlob,
+ sleep_tlob,
+ start_tlob,
+ stop_tlob,
+ switch_in_tlob,
+ wakeup_tlob,
+ event_max_tlob,
+};
+
+/*
+ * HA clock env: clk_elapsed, wall-clock since the window start; anchored in
+ * running/waiting/sleeping, cleared on stop.
+ */
+enum envs_tlob {
+ clk_elapsed_tlob,
+ env_max_tlob,
+ env_max_stored_tlob = env_max_tlob,
+};
+
+_Static_assert(env_max_stored_tlob <= MAX_HA_ENV_LEN, "Not enough slots");
+#define HA_CLK_NS
+
+struct automaton_tlob {
+ char *state_names[state_max_tlob];
+ char *event_names[event_max_tlob];
+ char *env_names[env_max_tlob];
+ unsigned char function[state_max_tlob][event_max_tlob];
+ unsigned char initial_state;
+ bool final_states[state_max_tlob];
+};
+
+static const struct automaton_tlob automaton_tlob = {
+ .state_names = {
+ "stopped",
+ "running",
+ "sleeping",
+ "waiting",
+ },
+ .event_names = {
+ "preempt",
+ "sleep",
+ "start",
+ "stop",
+ "switch_in",
+ "wakeup",
+ },
+ .env_names = {
+ "clk_elapsed",
+ },
+ .function = {
+ /* stopped (initial; window parked, sched events not routed) */
+ {
+ INVALID_STATE, /* preempt (not on CPU) */
+ INVALID_STATE, /* sleep (not on CPU) */
+ running_tlob, /* start (tlob_start_task, fresh or restart) */
+ INVALID_STATE, /* stop (already stopped) */
+ INVALID_STATE, /* switch_in (not on CPU) */
+ INVALID_STATE, /* wakeup (not on CPU) */
+ },
+ /* running */
+ {
+ waiting_tlob, /* preempt (sched_switch, prev_state == 0) */
+ sleeping_tlob, /* sleep (sched_switch, prev_state != 0) */
+ INVALID_STATE, /* start (running task's START is -EALREADY) */
+ stopped_tlob, /* stop (tlob_stop_task) */
+ INVALID_STATE, /* switch_in (already on CPU) */
+ INVALID_STATE, /* wakeup (TASK_RUNNING can't be woken) */
+ },
+ /* sleeping */
+ {
+ INVALID_STATE, /* preempt (not on CPU) */
+ INVALID_STATE, /* sleep (already sleeping) */
+ INVALID_STATE, /* start (not in running state) */
+ INVALID_STATE, /* stop (not in running state) */
+ INVALID_STATE, /* switch_in (must go through waiting first) */
+ waiting_tlob, /* wakeup */
+ },
+ /* waiting */
+ {
+ INVALID_STATE, /* preempt (not on CPU) */
+ INVALID_STATE, /* sleep (not on CPU) */
+ INVALID_STATE, /* start (not in running state) */
+ INVALID_STATE, /* stop (not in running state) */
+ running_tlob, /* switch_in */
+ INVALID_STATE, /* wakeup (already TASK_RUNNING) */
+ },
+ },
+ .initial_state = stopped_tlob,
+ .final_states = { 0, 1, 0, 0 },
+};
+
+/*
+ * Hard cap on concurrently monitored tasks. tlob_ws_pool pre-allocates
+ * this many slots; a fresh start past the cap returns -ENOSPC with bounded
+ * latency (mempool_alloc_preallocated() never touches the allocator).
+ * Restarts reuse the same slot.
+ */
+#define TLOB_MAX_MONITORED 64U
+
+/* Maximum binary path length for uprobe binding. */
+#define TLOB_MAX_PATH 256
+
+/* Minimum monitoring budget (1 us). */
+#define TLOB_MIN_THRESHOLD_NS 1000ULL
+
+/* Upper budget bound (1 hour): keeps the u64 ns accumulators far from overflow. */
+#define TLOB_MAX_THRESHOLD_NS 3600000000000ULL
+
+#endif /* _RV_TLOB_H */
diff --git a/kernel/trace/rv/monitors/tlob/tlob_trace.h b/kernel/trace/rv/monitors/tlob/tlob_trace.h
new file mode 100644
index 000000000000..b3a7cf4ad3ea
--- /dev/null
+++ b/kernel/trace/rv/monitors/tlob/tlob_trace.h
@@ -0,0 +1,48 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+/*
+ * Snippet to be included in rv_trace.h
+ */
+
+#ifdef CONFIG_RV_MON_TLOB
+DEFINE_EVENT(event_da_monitor_id, event_tlob,
+ TP_PROTO(int id, char *state, char *event,
+ char *next_state, bool final_state),
+ TP_ARGS(id, state, event, next_state, final_state));
+
+DEFINE_EVENT(error_da_monitor_id, error_tlob,
+ TP_PROTO(int id, char *state, char *event),
+ TP_ARGS(id, state, event));
+
+DEFINE_EVENT(error_env_da_monitor_id, error_env_tlob,
+ TP_PROTO(int id, char *state, char *event, char *env),
+ TP_ARGS(id, state, event, env));
+
+/*
+ * detail_env_tlob - per-state latency breakdown on budget violation.
+ * Emitted right after error_env_tlob from the hrtimer callback.
+ */
+TRACE_EVENT(detail_env_tlob,
+ TP_PROTO(int id, u64 threshold_ns,
+ u64 running_ns, u64 waiting_ns, u64 sleeping_ns),
+ TP_ARGS(id, threshold_ns, running_ns, waiting_ns, sleeping_ns),
+ TP_STRUCT__entry(
+ __field(int, id)
+ __field(u64, threshold_ns)
+ __field(u64, running_ns)
+ __field(u64, waiting_ns)
+ __field(u64, sleeping_ns)
+ ),
+ TP_fast_assign(
+ __entry->id = id;
+ __entry->threshold_ns = threshold_ns;
+ __entry->running_ns = running_ns;
+ __entry->waiting_ns = waiting_ns;
+ __entry->sleeping_ns = sleeping_ns;
+ ),
+ TP_printk("pid=%d threshold_ns=%llu"
+ " running_ns=%llu waiting_ns=%llu sleeping_ns=%llu",
+ __entry->id, __entry->threshold_ns,
+ __entry->running_ns, __entry->waiting_ns, __entry->sleeping_ns)
+);
+#endif /* CONFIG_RV_MON_TLOB */
diff --git a/kernel/trace/rv/rv_trace.h b/kernel/trace/rv/rv_trace.h
index 2f8a932432c9..4bfa39717cef 100644
--- a/kernel/trace/rv/rv_trace.h
+++ b/kernel/trace/rv/rv_trace.h
@@ -189,6 +189,7 @@ DECLARE_EVENT_CLASS(error_env_da_monitor_id,
#include <monitors/stall/stall_trace.h>
#include <monitors/nomiss/nomiss_trace.h>
+#include <monitors/tlob/tlob_trace.h>
// Add new monitors based on CONFIG_HA_MON_EVENTS_ID here
#endif
--
2.25.1
next prev parent reply other threads:[~2026-08-19 18:16 UTC|newest]
Thread overview: 19+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-08-19 18:15 [PATCH v5 0/9] rv: Add task latency over budget RV monitor wen.yang
2026-08-19 18:15 ` [PATCH v5 1/9] rv: Introduce DA_MON_ALLOCATION_STRATEGY wen.yang
2026-08-19 18:30 ` sashiko-bot
2026-08-19 18:15 ` [PATCH v5 2/9] rv: Add generic uprobe infrastructure for RV monitors wen.yang
2026-08-19 18:27 ` sashiko-bot
2026-08-19 18:15 ` [PATCH v5 3/9] rv: Add tlob model DOT file wen.yang
2026-08-19 18:25 ` sashiko-bot
2026-08-19 18:15 ` [PATCH v5 4/9] rv: Fix ha_invariant_passed_ns silent bypass of invariant check wen.yang
2026-08-19 18:32 ` sashiko-bot
2026-08-19 18:15 ` [PATCH v5 5/9] rv: Make da_monitor_reset_hook and EVENT_NONE_LBL overridable wen.yang
2026-08-19 18:30 ` sashiko-bot
2026-08-19 18:15 ` wen.yang [this message]
2026-08-19 18:34 ` [PATCH v5 6/9] rv: Add tlob hybrid automaton monitor sashiko-bot
2026-08-19 18:15 ` [PATCH v5 7/9] rv: Add KUnit tests for the tlob monitor wen.yang
2026-08-19 18:24 ` sashiko-bot
2026-08-19 18:15 ` [PATCH v5 8/9] selftests/verification: Add tlob selftests wen.yang
2026-08-19 18:27 ` sashiko-bot
2026-08-19 18:15 ` [PATCH v5 9/9] selftests/ftrace: Walk up to find test.d/functions when a subdirectory is passed wen.yang
2026-08-19 18:31 ` sashiko-bot
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=392de71fe1497f2b9d8ce06c387ec83c4b116c47.1787161646.git.wen.yang@linux.dev \
--to=wen.yang@linux.dev \
--cc=gmonaco@redhat.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-trace-kernel@vger.kernel.org \
--cc=namcao@linutronix.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox