[PATCH v3 0/9] rv/tlob: Add task latency over budget RV monitor

[wen.yang@linux.dev]

From: Wen Yang <wen.yang@linux.dev>

This series introduces tlob (task latency over budget), a per-task
hybrid automaton RV monitor that measures elapsed wall-clock time across
a user-delimited code section and fires when the time exceeds a
configurable budget.

The series applies cleanly on top of:
  [1] git://git.kernel.org/pub/scm/linux/kernel/git/gmonaco/linux.git rv-fixes-7.1
      "rv fixes for v7.1"

Background
----------
The existing wwnr monitor uses a two-state DA to detect tasks that are
woken but never run.  tlob extends the RV framework to a three-state
hybrid automaton:

  running  (initial) -- on CPU
  waiting             -- in the scheduler runqueue, not yet on CPU
  sleeping            -- blocked on a lock, I/O, or similar resource

A single HA clock invariant, clk_elapsed < BUDGET_NS(), is active in
all states.  The framework enforces it via a per-task hrtimer.  On
expiry, error_env_tlob is emitted, followed by detail_env_tlob which
carries a per-state time breakdown (running_ns, waiting_ns, sleeping_ns)
that pinpoints whether the overrun occurred in the running, waiting, or
sleeping state.

Userspace interface
-------------------
Tasks are registered for monitoring by writing to the tracefs monitor
file:

  # echo "p /path/to/binary:START_OFFSET STOP_OFFSET threshold=NS" \
        > /sys/kernel/tracing/rv/monitors/tlob/monitor

Two uprobes are registered at START_OFFSET (entry) and STOP_OFFSET
(exit) of the delimited section.  When a task executes the entry uprobe,
the monitor starts; when the task reaches the exit uprobe or the budget
expires, monitoring stops and the slot is returned to the pool.

Multiple uprobe pairs can be registered for the same binary or different
binaries.  Each task can have at most one active monitoring session; if
a task hits a start uprobe while already monitored, the prior session is
cancelled and a new one begins.

Series structure
----------------
Patch 1: rv/da: introduce DA_MON_ALLOCATION_STRATEGY
  Consolidates per-object DA storage allocation under a compile-time
  selector with three strategies:
    DA_ALLOC_AUTO   (default) - lock-free kmalloc_nolock; unbounded
    DA_ALLOC_POOL             - pre-allocated fixed-size pool
    DA_ALLOC_MANUAL           - caller pre-inserts storage

  da_handle_start_event() and da_handle_start_run_event() call
  da_prepare_storage() which resolves at compile time to the correct
  allocation function.

  This patch also includes critical correctness fixes for the pool
  implementation:
  - Add tracepoint_synchronize_unregister() in da_monitor_destroy_pool()
    to fix UAF where in-flight handlers access freed pool storage
  - Fix duplicate hash entry race in da_create_or_get_pool() via
    concurrent-insert detection under RCU
  - Add capacity field to fix build error (DA_MON_POOL_SIZE undeclared
    in da_pool_return_cb)

Patch 2: rv: add generic uprobe infrastructure for RV monitors
  Introduces rv_uprobe, a thin wrapper around uprobe_consumer for RV
  monitors.  Provides rv_uprobe_register(), rv_uprobe_unregister(),
  and rv_uprobe_sync() for safe teardown.

Patch 3: rv/tlob: add tlob model DOT file
  The formal model used to generate tlob.h.

Patch 4: rv/ha: fix ha_invariant_passed_ns silent bypass of invariant check
  Fixes a bug where ha_invariant_passed_ns() returned 0 early when
  env_store was invalid (U64_MAX), leaving it at U64_MAX and causing
  ha_check_invariant_ns() to always pass.  The fix calls ha_reset_clk_ns()
  then ha_set_invariant_ns() on first use.

Patch 5: rv/ha: make da_monitor_reset_hook and EVENT_NONE_LBL overridable
  Allows tlob to override EVENT_NONE_LBL for its start_tlob self-loop.

Patch 6: rv/tlob: add tlob hybrid automaton monitor
  The main tlob implementation, including:
  - Three-state HA (running/waiting/sleeping)
  - Per-task hrtimer enforcement (HRTIMER_MODE_REL_HARD)
  - DA_ALLOC_POOL for allocation-free hot path
  - Uprobe registration via tracefs monitor file
  - Per-state time accumulation (running_ns, waiting_ns, sleeping_ns)
  - HIGH_RES_TIMERS dependency in Kconfig

Patches 7-9: Tests
  - KUnit tests for tlob monitor
  - Selftest infrastructure fixes
  - tlob selftests (uprobe binding, state tracking, violation detection)

Changes since v2
----------------
All feedback from Gabriele Monaco has been addressed:

-- Patch 02 (per-task slot ordering / ha_monitor_reset_env):
   Dropped from v3; rebased on top of Gabriele's series [1].

-- Patch 03 (verificationtest-ktap):
   Changed to use realpath for robustness as suggested.

-- Patch 04 (pre-allocated storage pool):
   Complete redesign as DA_MON_ALLOCATION_STRATEGY:
   - Three strategies (AUTO/POOL/MANUAL) via compile-time macro
   - da_monitor_init_prealloc() removed; da_monitor_init() selects
     internally
   - da_create_or_get_kmalloc() removed (no viable use case)
   - nomiss updated to use DA_ALLOC_MANUAL
   - da_extra_cleanup() hook added for per-entry teardown

   Critical bug fixes included in this patch:
   - tracepoint_synchronize_unregister() added to da_monitor_destroy_pool()
     to prevent UAF from in-flight handlers accessing freed pool storage
   - Duplicate hash entry race fixed in da_create_or_get_pool() via
     concurrent-insert detection and slot return under RCU
   - capacity field added to fix DA_MON_POOL_SIZE undeclared build error

-- Patch 05 (generic uprobe infrastructure):
   Carried unchanged into v3.

-- Patch 06 (rvgen __init arrow reset):
   Carried unchanged into v3.

-- Patch 08 (tlob monitor):
   Split and refactored:
   - ioctl interface deferred to follow-up series (tracefs-only in v3)
   - Handler simplification: three inline helpers (tlob_acc_running/
     waiting/sleeping) with scoped_guard(rcu)
   - do_prev/do_next flags removed (da_handle_event skips unmonitored)
   - scoped_guard(rcu) and guard(mutex) applied throughout
   - tlob_stop_all() removed; da_extra_cleanup() hook used instead
   - start_tlob self-loop added to DOT model as suggested
   - ha_setup_invariants() guards against redundant timer restart
   - HIGH_RES_TIMERS dependency added to Kconfig

Additional improvements in v3
------------------------------
Beyond the v2 feedback, this version includes:

1. Simplified tlob monitor implementation:
   - Removed redundant tlob_num_monitored atomic counter
     (da_handle_event already handles unmonitored tasks via hash lookup)
   - Eliminated extra cacheline touch on every sched_switch/sched_wakeup
   - Several repeated pattern simplifications.

2. Extracted common accumulation logic:
   - __tlob_acc() using offsetof() replaces three nearly-identical functions
   - Reduces code duplication while maintaining type safety

3. Complete test coverage:
   - KUnit tests for core functionality
   - Comprehensive selftests for uprobe integration, state tracking,
     and violation detection

Testing
-------
All patches have been tested on:
- x86_64 with CONFIG_PREEMPT_RT
- All KUnit tests pass
- All selftests pass with verificationtest-ktap

  
[1] git://git.kernel.org/pub/scm/linux/kernel/git/gmonaco/linux.git rv-fixes-7.1
    "rv fixes for v7.1"


Wen Yang (9):
  rv/da: introduce DA_MON_ALLOCATION_STRATEGY
  rv: add generic uprobe infrastructure for RV monitors
  rv/tlob: add tlob model DOT file
  rv/ha: fix ha_invariant_passed_ns silent bypass of invariant check
  rv/ha: make da_monitor_reset_hook and EVENT_NONE_LBL overridable
  rv/tlob: add tlob hybrid automaton monitor
  rv/tlob: add KUnit tests for the tlob monitor
  selftests/verification: fix verificationtest-ktap for out-of-tree
    execution
  selftests/verification: add tlob selftests

 Documentation/trace/rv/index.rst              |   1 +
 Documentation/trace/rv/monitor_tlob.rst       | 177 ++++
 include/rv/da_monitor.h                       | 276 ++++-
 include/rv/ha_monitor.h                       |  22 +-
 include/rv/rv_uprobe.h                        | 119 +++
 kernel/trace/rv/Kconfig                       |   5 +
 kernel/trace/rv/Makefile                      |   3 +
 kernel/trace/rv/monitors/nomiss/nomiss.c      |   6 +-
 kernel/trace/rv/monitors/tlob/.kunitconfig    |   6 +
 kernel/trace/rv/monitors/tlob/Kconfig         |  19 +
 kernel/trace/rv/monitors/tlob/tlob.c          | 968 ++++++++++++++++++
 kernel/trace/rv/monitors/tlob/tlob.h          | 148 +++
 kernel/trace/rv/monitors/tlob/tlob_kunit.c    |  92 ++
 kernel/trace/rv/monitors/tlob/tlob_trace.h    |  49 +
 kernel/trace/rv/rv_trace.h                    |   1 +
 kernel/trace/rv/rv_uprobe.c                   | 182 ++++
 .../testing/selftests/verification/.gitignore |   2 +
 tools/testing/selftests/verification/Makefile |  19 +-
 .../verification/test.d/tlob/Makefile         |  20 +
 .../verification/test.d/tlob/test.d/functions |   1 +
 .../verification/test.d/tlob/tlob_sym.c       | 189 ++++
 .../verification/test.d/tlob/tlob_target.c    | 138 +++
 .../verification/test.d/tlob/uprobe_bind.tc   |  37 +
 .../test.d/tlob/uprobe_detail_running.tc      |  51 +
 .../test.d/tlob/uprobe_detail_sleeping.tc     |  50 +
 .../test.d/tlob/uprobe_detail_waiting.tc      |  66 ++
 .../verification/test.d/tlob/uprobe_multi.tc  |  64 ++
 .../test.d/tlob/uprobe_no_event.tc            |  19 +
 .../test.d/tlob/uprobe_violation.tc           |  67 ++
 .../verification/verificationtest-ktap        |   4 +-
 tools/verification/models/tlob.dot            |  22 +
 31 files changed, 2789 insertions(+), 34 deletions(-)
 create mode 100644 Documentation/trace/rv/monitor_tlob.rst
 create mode 100644 include/rv/rv_uprobe.h
 create mode 100644 kernel/trace/rv/monitors/tlob/.kunitconfig
 create mode 100644 kernel/trace/rv/monitors/tlob/Kconfig
 create mode 100644 kernel/trace/rv/monitors/tlob/tlob.c
 create mode 100644 kernel/trace/rv/monitors/tlob/tlob.h
 create mode 100644 kernel/trace/rv/monitors/tlob/tlob_kunit.c
 create mode 100644 kernel/trace/rv/monitors/tlob/tlob_trace.h
 create mode 100644 kernel/trace/rv/rv_uprobe.c
 create mode 100644 tools/testing/selftests/verification/test.d/tlob/Makefile
 create mode 100644 tools/testing/selftests/verification/test.d/tlob/test.d/functions
 create mode 100644 tools/testing/selftests/verification/test.d/tlob/tlob_sym.c
 create mode 100644 tools/testing/selftests/verification/test.d/tlob/tlob_target.c
 create mode 100644 tools/testing/selftests/verification/test.d/tlob/uprobe_bind.tc
 create mode 100644 tools/testing/selftests/verification/test.d/tlob/uprobe_detail_running.tc
 create mode 100644 tools/testing/selftests/verification/test.d/tlob/uprobe_detail_sleeping.tc
 create mode 100644 tools/testing/selftests/verification/test.d/tlob/uprobe_detail_waiting.tc
 create mode 100644 tools/testing/selftests/verification/test.d/tlob/uprobe_multi.tc
 create mode 100644 tools/testing/selftests/verification/test.d/tlob/uprobe_no_event.tc
 create mode 100644 tools/testing/selftests/verification/test.d/tlob/uprobe_violation.tc
 create mode 100644 tools/verification/models/tlob.dot

-- 
2.43.0