[RFC PATCH 0/7] mm/damon/perf: observability framework for hardware-sampled access reports

[RFC PATCH 0/7] mm/damon/perf: observability framework for hardware-sampled access reports

[Kunwu Chan]

From: Kunwu Chan <kunwu.chan@gmail.com>

This series adds a unified observability framework to DAMON's
hardware-sampled access report pipeline.  When a PMU backend delivers
zero callbacks, the framework pinpoints the failing stage —
was the event not created?  not enabled?  did the hardware produce
no samples?  did the ring drop them?  did the drain fail to match? —
without a single printk.  Every PMU backend (ARM SPE, AMD IBS,
Intel PEBS, …) funnels through the same damon_perf_observe_*() API,
which fans out to per-CPU pipeline counters, debugfs statistics,
and tracepoints.

The framework does not contain any PMU driver.  It is pure
observability infrastructure that sits alongside the existing
DAMON software statistics (scheme stats, kdamond stats) and
complements them for the hardware-sampling path.

Why this framework exists
-------------------------
The framework was built to answer a concrete question during ARM SPE
bring-up: what prevents SPE from delivering samples to DAMON?  The
answer could be anywhere in the pipeline — the event wasn't created,
the PMU wasn't enabled, the hardware produced no samples, the ring
dropped them, or the drain failed to match — and without per-stage
visibility, the only way to find out was to insert printks across the
entire pipeline.

The framework instruments every stage with zero-overhead counters
(when CONFIG_DAMON_PERF_OBSERVE=n) and optional tracepoints, so the
failing stage is revealed at a glance.  It immediately diagnosed the
ARM SPE gap: Layers 1-3 (event create/bind/enable) passed, but the
PMU was not producing data — a PMU-level gap, not a DAMON pipeline
bug.  This confirmed that SPE required an AUX buffer backend, which
does not fit the perf_event_create_kernel_counter() API.

During the subsequent ARM SPE AUX backend development [2] (which builds
on the perf AUX kernel-consumer API [3]), the framework was used to
verify each pipeline fix.  When the AUX backend was complete, the same
framework confirmed a fully functional pipeline: Layers 1-7 all pass,
real SPE hardware samples flow through the ring, match, and update
stages.  The before/after results are shown in the Testing section.

We are posting the framework now so that other PMU backends can reuse
the same infrastructure instead of re-deriving it.

How to use
----------
Compile with CONFIG_DAMON_PERF_OBSERVE=y.  After starting a kdamond
with a perf event, the pipeline is visible at a glance:

  # cat /sys/kernel/debug/damon/perf_stats
      --------------  ----------
      Counter          Value
      --------------  ----------
      callback          233
      valid             233
      ...
      enqueue           233
      dequeue           233
      match              12
      update              9

For automated diagnosis, run the selftest:

  # sudo ./damon_perf_obs_test.sh --pmu software --freq 1 --sample-freq 100

For structured diagnostics, enable the tracepoints under
/sys/kernel/debug/tracing/events/damon/.

What the series adds
--------------------
Patch 1 defines the three tracepoints (damon_perf_sample with 8 fields
including sample_flags, sample_type, and execution context, plus
damon_perf_report_missed and damon_perf_drain) with the CONFIG switch,
the observe API declarations, no-op stubs, and the Kconfig / Makefile
wiring.  (Reviewers: tracepoint folks — Steven, Masami, Mathieu.)

Patch 2 implements the observe_*() core: per-CPU monotonic counters,
event-lifecycle hooks with lazily-allocated per-event per-CPU state,
NMI-safe sample classification with execution-context detection,
ring enqueue/dequeue/peak, match/miss/update, and framework init.

Patch 3 adds the debugfs perf_stats interface (debug-only, format
unstable).

Patch 4 wires the observe calls into the vaddr overflow handlers and
the kdamond drain, integrating the framework into the hot path.
Reads ring->tail once with READ_ONCE and reuses the cached value
for the peak-occupancy estimate, avoiding a compiler reload and a
torn read on the NMI producer side.

Patch 5 adds the automated selftest with snapshot/delta accounting,
per-CPU state-column checks, clean-session guard, and a PMU support
verdict (FULLY INTEGRATED / PLUMBING-ONLY / UNUSABLE).
(Reviewers: kselftest folks — Shuah.)

Patch 6 documents the framework.

Patch 7 adds CONFIG_DAMON_PERF_DEBUG as an optional Kconfig option for
verbose pr_debug() output via damon_perf_dbg(), wired into every
event-lifecycle hook, the drain summary, and the init paths.  Adds a
pipeline health check to the selftest that diagnoses which stage is
broken when callbacks are zero.

Testing
=======
Tested on Kunpeng 920 (256 CPUs, ARM SPE, kernel 7.1.0-rc5-mm-new-damon+).

KUnit: 35/35 PASSED (damon 29, damon-operations 5, damon-sysfs 1).

The framework is PMU-agnostic.  Three configurations demonstrate this,
with the ARM SPE case being the framework's core diagnostic value:

**Software PMU** (page-fault sampling, exercises the full pipeline,
no hardware required):

  $ sudo ./damon_perf_obs_test.sh --pmu software --freq 1 --sample-freq 100
  SUMMARY: 27 passed, 0 failed, 0 skipped (FULLY INTEGRATED)
  callback=170, valid=170, enqueue=170, dequeue=170, match=6, update=3

**CPU-clock** (address-less PMU, plumbing-only):

  $ sudo ./damon_perf_obs_test.sh --pmu software --config 0 --freq 1 --sample-freq 100
  SUMMARY: 22 passed, 0 failed, 3 skipped (PLUMBING-ONLY)
  callback=1913, valid=0, addr_zero=1913, enqueue=0
  Layers 1-4 pass; Ring/Match/Drain skipped as expected.

**ARM SPE** (hardware trace).  This is the scenario that motivated the
  framework: without an AUX buffer backend, SPE cannot deliver samples
  through perf_event_create_kernel_counter(), and the framework
  pinpoints the failure without a single printk:

  $ sudo ./damon_perf_obs_test.sh --pmu arm_spe_0 --freq 0 --period 256

  --- Layer 2-3: Enable & Run (via per-CPU state) ---
    [PASS] Event Created (max per-CPU state >= CREATED)
    [PASS] Event Bound (max per-CPU state >= BOUND)
    [PASS] Event Enabled (max per-CPU state >= ENABLED)
    [PASS] Perf event creation (no errors in dmesg delta)

  --- Analysis ---
    Pipeline diagnosis: PMU not producing data or AUX pipeline broken
    [FAIL] Sampling — 0 callbacks — PMU is not delivering samples to DAMON
    [SKIP] Event Running — no callbacks — state cannot advance past ENABLED
    Ring: enqueue=0 dequeue=0 overflow=0

  --- PMU support verdict ---
    UNUSABLE: the PMU never delivered a sample to DAMON

  SUMMARY: 19 passed, 1 failed, 5 skipped

  Layers 1-3 pass — the per-CPU state advances to ENABLED — so the
  kernel-side plumbing is intact.  The health check confirms
  state >= ENABLED but enqueue=0, ruling out event-creation and enable
  bugs.  The failure is isolated to Layer 4 (Sampling): the PMU itself
  is not producing data.  This is the expected result for ARM SPE
  without an AUX buffer backend, and the framework diagnoses it as a
  PMU-level gap rather than a DAMON pipeline bug.

  After the AUX backend is added [2] (which itself depends on the
  perf AUX kernel-consumer API [3]), the same test script confirms a
  fully functional pipeline, including real SPE hardware samples:

  $ sudo ./damon_perf_obs_test.sh --pmu arm_spe_0 --freq 0 --period 256
  SUMMARY: 27 passed, 0 failed, 0 skipped (FULLY INTEGRATED)
  callback=47248, valid=47248, enqueue=15161, dequeue=15161,
  overflow=32087, match=255, update=4
  All 7 pipeline stages pass.  The AUX drain path delivers samples in
  process context (context=0), and the ring overflow counter (32087)
  shows the ring is sized for software PMU rates — a real-world
  observation the framework provides without any additional
  instrumentation.

Known limitations
=================
- The debugfs perf_stats format is explicitly unstable and must not
  be used by scripts.  Tracepoints and counters are the stable
  diagnostic interface.
- The global per-CPU lifecycle state is monotonic and never reset,
  so a previous RUNNING event can mask a current event's failure.
  Per-event per-CPU state is available for accurate tracking.
- This series depends on Ravi's hardware-sampled access reports
  branch [1] for the following facilities:
    * Per-CPU SPSC report ring (damon_report_rings,
      DAMON_REPORT_RING_MASK)
    * damon_report_access() and damon_report_page_fault() as the
      ring producer
    * kdamond_check_reported_accesses() as the drain path
    * damon_perf_event lifecycle (create/bind/enable/disable/free)
  Without [1] this series cannot compile, so [1] must be merged
  first.

References
==========
Link [1]: https://github.com/damonitor/linux.git
          branch: ravi_hw_sampled_access_reports_rfc_v1
          commit: fd968106e5bf
Link [2]: https://lore.kernel.org/all/20260816142222.689624-1-kunwu.chan@linux.dev/
Link [3]: https://lore.kernel.org/all/20260814144927.489172-1-kunwu.chan@linux.dev/

Base: fd968106e5bf ("mm/damon: add damos_node_eligible_mem_bp tracepoint")

Kunwu Chan (4):
  mm/damon/perf: add observability framework with tracepoints and CONFIG
    switch
  mm/damon/perf: implement observe API and per-CPU statistics engine
  mm/damon: integrate observe API into vaddr overflow handlers and core
    drain
  selftests/damon: add automated layer-by-layer observability test

Lian Wang (3):
  mm/damon/perf: add debugfs statistics interface
  Docs/mm/damon: document the perf observability framework
  mm/damon/perf: add CONFIG_DAMON_PERF_DEBUG and pipeline health check

 Documentation/admin-guide/mm/damon/index.rst  |   1 +
 .../mm/damon/perf-observability.rst           | 210 +++++++
 include/linux/damon.h                         |  44 ++
 include/trace/events/damon.h                  | 120 ++++
 mm/damon/Kconfig                              |  33 +
 mm/damon/Makefile                             |   1 +
 mm/damon/core.c                               | 129 +++-
 mm/damon/perf/Makefile                        |   5 +
 mm/damon/perf/debugfs.c                       | 145 +++++
 mm/damon/perf/perf.h                          | 235 +++++++
 mm/damon/perf/stats.c                         | 308 ++++++++++
 mm/damon/vaddr.c                              | 104 +++-
 tools/testing/selftests/damon/Makefile        |   1 +
 .../selftests/damon/damon_perf_obs_test.sh    | 578 ++++++++++++++++++
 14 files changed, 1880 insertions(+), 34 deletions(-)
 create mode 100644 Documentation/admin-guide/mm/damon/perf-observability.rst
 create mode 100644 mm/damon/perf/Makefile
 create mode 100644 mm/damon/perf/debugfs.c
 create mode 100644 mm/damon/perf/perf.h
 create mode 100644 mm/damon/perf/stats.c
 create mode 100755 tools/testing/selftests/damon/damon_perf_obs_test.sh

-- 
2.43.0

[RFC PATCH 1/7] mm/damon/perf: add observability framework with tracepoints and CONFIG switch

[Kunwu Chan]

From: Kunwu Chan <kunwu.chan@gmail.com>

Add the observe framework headers together with the compile-time switch
that gates the whole feature: the three DAMON perf tracepoints this
series adds (guarded with CONFIG_DAMON_PERF_OBSERVE so they are not
registered when the switch is off; damon_perf_ring_overflow is provided
by the base series), the observe API declarations with static-inline
no-ops for disabled builds, the access-report contract (miss-reason
enum, report-source enum, report source field, per-event cpu_state
member), and the Kconfig and Makefile wiring.

The sample tracepoint carries what the PMU actually populated
(data->sample_flags), what was requested (perf_event->attr.
sample_type), and the execution context (process/softirq/hardirq/NMI)
in a single line, so PMU support gaps and context expectations (e.g.
IBS overflow in NMI, SPE AUX drain in process context) are verifiable
at a glance.

Every following commit in this series builds with CONFIG_DAMON_PERF_OBSERVE
both enabled and disabled.

Co-developed-by: Lian Wang <lianux.mm@gmail.com>
Signed-off-by: Lian Wang <lianux.mm@gmail.com>
Signed-off-by: Kunwu Chan <kunwu.chan@gmail.com>
---
 include/linux/damon.h        |  44 +++++++
 include/trace/events/damon.h | 120 ++++++++++++++++++
 mm/damon/Kconfig             |  17 +++
 mm/damon/Makefile            |   1 +
 mm/damon/perf/Makefile       |   3 +
 mm/damon/perf/perf.h         | 228 +++++++++++++++++++++++++++++++++++
 6 files changed, 413 insertions(+)
 create mode 100644 mm/damon/perf/Makefile
 create mode 100644 mm/damon/perf/perf.h

diff --git a/include/linux/damon.h b/include/linux/damon.h
index 11f1c1071b9b..c191c065b0e4 100644
--- a/include/linux/damon.h
+++ b/include/linux/damon.h
@@ -116,6 +116,19 @@ struct damon_target {
 	bool obsolete;
 };
 
+/**
+ * enum damon_report_source - Tells which subsystem produced an access report.
+ *
+ * Ring and matching counters aggregate all sources; this enum lets callers
+ * tag reports so that tracepoints and future per-source breakdowns can
+ * distinguish NMI overflow-handler samples from
+ * page-fault hints.
+ */
+enum damon_report_source {
+	DAMON_REPORT_SRC_PERF_OVERFLOW = 0,  /* overflow_handler (IBS, PEBS) */
+	DAMON_REPORT_SRC_PAGE_FAULT,         /* damon_report_page_fault() */
+};
+
 /**
  * struct damon_access_report - Represent single access report information.
  * @paddr:		Start physical address of the accessed address range.
@@ -125,6 +138,8 @@ struct damon_target {
  * @tid:		The task id of the task that made the access.
  * @tgid:		Thread group id of the task that made the access.
  * @is_write:		Whether the access is write.
+ * @source:		Which subsystem produced this report
+ *			(enum damon_report_source).
  *
  * Any DAMON API callers that notified access events can report the information
  * to DAMON using damon_report_access().  This struct contains the reporting
@@ -138,10 +153,28 @@ struct damon_access_report {
 	pid_t tid;
 	pid_t tgid;
 	bool is_write;
+#ifdef CONFIG_DAMON_PERF_OBSERVE
+	int source;
+#endif /* CONFIG_DAMON_PERF_OBSERVE */
 /* private: */
 	unsigned long report_jiffies;	/* when this report is made */
 };
 
+/*
+ * Reason codes for trace_damon_perf_report_missed.
+ *
+ * DAMON_REPORT_MISS_TGID:       tgid mismatch (pid-based monitoring,
+ *                               missed at drain-loop level before the
+ *                               per-target iteration).
+ * DAMON_REPORT_MISS_NOREGION:   binary search found no containing region.
+ * DAMON_REPORT_MISS_BOUNDARY:   address + size straddles region boundary.
+ */
+enum damon_report_miss_reason {
+	DAMON_REPORT_MISS_TGID = 1,
+	DAMON_REPORT_MISS_NOREGION = 2,
+	DAMON_REPORT_MISS_BOUNDARY = 3,
+};
+
 /**
  * enum damos_action - Represents an action of a Data Access Monitoring-based
  * Operation Scheme.
@@ -1027,6 +1060,17 @@ struct damon_perf_event {
 	struct hlist_node hlist_node;
 	bool init_complete;
 	bool any_cpu_failed;
+#ifdef CONFIG_DAMON_PERF_OBSERVE
+	/*
+	 * Per-CPU lifecycle state (enum damon_perf_event_state).
+	 * Allocated lazily on the first observe_event_created(),
+	 * freed on observe_event_destroyed().  Each event tracks
+	 * its own progression through CREATED->BOUND->ENABLED,
+	 * so destroying one event does not overwrite another"s
+	 * state on the same CPU.
+	 */
+	int __percpu *cpu_state;
+#endif /* CONFIG_DAMON_PERF_OBSERVE */
 	struct damon_ctx *ctx;
 };
 
diff --git a/include/trace/events/damon.h b/include/trace/events/damon.h
index 877627c9a1a1..c87fbeefb85a 100644
--- a/include/trace/events/damon.h
+++ b/include/trace/events/damon.h
@@ -91,6 +91,126 @@ TRACE_EVENT(damon_perf_ring_overflow,
 	TP_printk("cpu=%d", __entry->cpu)
 );
 
+#ifdef CONFIG_DAMON_PERF_OBSERVE
+/*
+ * Fires from NMI overflow handlers on every hardware sample received,
+ * before any DAMON-side filtering.  Records the raw address, full
+ * data_src (mem_op, mem_lvl, mem_snoop, mem_remote), period, and a
+ * reason code so userspace can distinguish:
+ *
+ *   0 = valid sample, queued to per-CPU ring
+ *   1 = data == NULL
+ *   2 = addr == 0 (PMU did not populate data->addr)
+ *   3 = kernel address (vaddr handler: addr >= TASK_SIZE)
+ *   4 = phys_addr not valid (paddr handler: !PERF_SAMPLE_PHYS_ADDR)
+ *
+ * data_src carries the raw union perf_mem_data_src value; use
+ * perf_mem__xxx macros to decode.
+ *
+ * sample_flags is what the PMU *actually* populated (from
+ * data->sample_flags); sample_type is what was *requested* (from
+ * perf_event->attr.sample_type).  Comparing them immediately
+ * reveals whether the PMU is providing the fields DAMON asked for
+ * — e.g. sample_type has PERF_SAMPLE_PHYS_ADDR but sample_flags
+ * does not → the PMU does not support physical-address sampling.
+ */
+TRACE_EVENT(damon_perf_sample,
+
+	TP_PROTO(unsigned long addr, u64 data_src, u64 period, int cpu,
+		u8 reason, u64 sample_flags, u64 sample_type,
+		u8 context),
+
+	TP_ARGS(addr, data_src, period, cpu, reason, sample_flags,
+		sample_type, context),
+
+	TP_STRUCT__entry(
+		__field(unsigned long, addr)
+		__field(u64, data_src)
+		__field(u64, period)
+		__field(int, cpu)
+		__field(u8, reason)
+		__field(u64, sample_flags)
+		__field(u64, sample_type)
+		__field(u8, context)
+	),
+
+	TP_fast_assign(
+		__entry->addr = addr;
+		__entry->data_src = data_src;
+		__entry->period = period;
+		__entry->cpu = cpu;
+		__entry->reason = reason;
+		__entry->sample_flags = sample_flags;
+		__entry->sample_type = sample_type;
+		__entry->context = context;
+	),
+
+	TP_printk("addr=0x%lx data_src=0x%llx period=%llu cpu=%d reason=%u context=%u sample_flags=0x%llx sample_type=0x%llx",
+		__entry->addr, __entry->data_src, __entry->period,
+		__entry->cpu, __entry->reason, __entry->context,
+		__entry->sample_flags, __entry->sample_type)
+);
+
+/*
+ * Fires when a report survived all ring/drain checks but could not be
+ * applied to any DAMON region.  Reasons correspond to
+ * enum damon_report_miss_reason:
+ *
+ *   DAMON_REPORT_MISS_TGID     (1): no target matched the report's tgid
+ *   DAMON_REPORT_MISS_NOREGION (2): binary search found no containing region
+ *   DAMON_REPORT_MISS_BOUNDARY (3): address + size straddles region boundary
+ *
+ * Note: tgid mismatches are now resolved in the drain loop *before*
+ * iterating targets, so there is at most one trace hit per report
+ * (rather than one per non-matching target as in earlier revisions).
+ */
+TRACE_EVENT(damon_perf_report_missed,
+
+	TP_PROTO(unsigned long addr, int cpu, int reason),
+
+	TP_ARGS(addr, cpu, reason),
+
+	TP_STRUCT__entry(
+		__field(unsigned long, addr)
+		__field(int, cpu)
+		__field(int, reason)
+	),
+
+	TP_fast_assign(
+		__entry->addr = addr;
+		__entry->cpu = cpu;
+		__entry->reason = reason;
+	),
+
+	TP_printk("addr=0x%lx cpu=%d reason=%d", __entry->addr,
+		__entry->cpu, __entry->reason)
+);
+
+/*
+ * Per-tick drain summary.  Fires from kdamond after draining the per-CPU
+ * SPSC ring, so users can observe total vs matched without polling dmesg
+ * or correlating individual miss tracepoints.
+ */
+TRACE_EVENT(damon_perf_drain,
+
+	TP_PROTO(unsigned int total, unsigned int matched),
+
+	TP_ARGS(total, matched),
+
+	TP_STRUCT__entry(
+		__field(unsigned int, total)
+		__field(unsigned int, matched)
+	),
+
+	TP_fast_assign(
+		__entry->total = total;
+		__entry->matched = matched;
+	),
+
+	TP_printk("total=%u matched=%u", __entry->total, __entry->matched)
+);
+#endif /* CONFIG_DAMON_PERF_OBSERVE */
+
 /* Per-tick DAMOS_QUOTA_NODE_ELIGIBLE_MEM_BP goal evaluation. */
 TRACE_EVENT(damos_node_eligible_mem_bp,
 
diff --git a/mm/damon/Kconfig b/mm/damon/Kconfig
index ad629f0f31d8..9f811510760f 100644
--- a/mm/damon/Kconfig
+++ b/mm/damon/Kconfig
@@ -131,4 +131,21 @@ config DAMON_ACMA
 	  min/max memory for the system and maximum memory pressure stall time
 	  ratio.
 
+config DAMON_PERF_OBSERVE
+	bool "DAMON perf event observability framework"
+	depends on DAMON
+	depends on PERF_EVENTS
+	depends on DEBUG_FS
+	default n
+	help
+	  Enable per-CPU pipeline counters, tracepoints, and a
+	  debug-only debugfs perf_stats file for DAMON
+	  hardware-sampled access reports.  The debugfs format is
+	  unstable and must not be used by scripts; counters and
+	  tracepoints are the diagnostic interface.
+
+	  When disabled, all observe functions are compiled to
+	  static-inline no-ops with zero runtime overhead.
+
+	  If unsure, say N.
 endmenu
diff --git a/mm/damon/Makefile b/mm/damon/Makefile
index 22494754f41e..04da39a9f56c 100644
--- a/mm/damon/Makefile
+++ b/mm/damon/Makefile
@@ -9,3 +9,4 @@ obj-$(CONFIG_DAMON_RECLAIM)	+= modules-common.o reclaim.o
 obj-$(CONFIG_DAMON_LRU_SORT)	+= modules-common.o lru_sort.o
 obj-$(CONFIG_DAMON_STAT)	+= modules-common.o stat.o
 obj-$(CONFIG_DAMON_ACMA)	+= modules-common.o acma.o
+obj-$(CONFIG_DAMON)		+= perf/
diff --git a/mm/damon/perf/Makefile b/mm/damon/perf/Makefile
new file mode 100644
index 000000000000..cc0d4f1d1d28
--- /dev/null
+++ b/mm/damon/perf/Makefile
@@ -0,0 +1,3 @@
+# SPDX-License-Identifier: GPL-2.0
+
+# Observability: per-CPU counters, tracepoints, debugfs perf_stats
diff --git a/mm/damon/perf/perf.h b/mm/damon/perf/perf.h
new file mode 100644
index 000000000000..78e23d436336
--- /dev/null
+++ b/mm/damon/perf/perf.h
@@ -0,0 +1,228 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * DAMON Hardware-sampled Access Report Observability Framework
+ *
+ * Single entry-point for all hardware sampling backends (ARM SPE,
+ * AMD IBS, Intel PEBS, …).  Every event, sample, ring operation,
+ * match decision, and region update flows through the
+ * damon_perf_observe_*() API, which fans out to per-CPU counters
+ * and tracepoints.  When CONFIG_DAMON_PERF_OBSERVE=n, everything
+ * compiles to static-inline no-ops.
+ *
+ * Author: Kunwu Chan <kunwu.chan@gmail.com>
+ */
+
+#ifndef _DAMON_PERF_H
+#define _DAMON_PERF_H
+
+struct perf_event;
+#include <linux/types.h>
+
+struct damon_perf_event;
+
+/*
+ * Per-event state machine
+ *
+ * Each per-CPU damon_perf_event transitions through these states.
+ * State is tracked in a per-CPU integer (damon_perf_cpu_state).
+ */
+enum damon_perf_event_state {
+	DAMON_PERF_STATE_UNINIT = 0,
+	DAMON_PERF_STATE_CREATED,	/* struct allocated, cpuhp registered */
+	DAMON_PERF_STATE_BOUND,		/* perf_event_create_kernel_counter() ok */
+	DAMON_PERF_STATE_ENABLED,	/* perf_event_enable() called */
+	DAMON_PERF_STATE_RUNNING,	/* first overflow callback received */
+	DAMON_PERF_STATE_ERROR,		/* unrecoverable failure */
+};
+
+/*
+ * Per-CPU statistics
+ *
+ * All counters are monotonic, best-effort reads.  Userspace computes
+ * deltas between snapshots.  Stored per-CPU so the NMI fast path uses
+ * this_cpu_inc() with no locking.  Counter values are raw facts:
+ * interpretation (thresholds, verdicts) belongs in userspace.
+ *
+ * The kernel provides tracepoints under events/damon/ for structured,
+ * stable diagnostics.  The debugfs perf_stats file is DEBUG ONLY and
+ * its format may change without notice.
+ */
+struct damon_perf_stats {
+	/* Per-CPU event state (enum damon_perf_event_state) */
+	int cpu_state;
+
+	/* Sampling pipeline */
+	u64 callback;
+	u64 sample_valid;
+	u64 sample_null;
+	u64 sample_addr_zero;
+	u64 sample_kernel;
+	u64 sample_invalid_phys;
+
+	/* Ring */
+	u64 enqueue;
+	u64 dequeue;
+	u64 overflow;
+	u64 ring_peak;
+
+	/* Matching */
+	u64 match;
+	u64 miss_tgid;
+	u64 miss_region;
+	u64 miss_boundary;
+	u64 update;
+};
+
+#ifdef CONFIG_DAMON_PERF_OBSERVE
+
+/*
+ * damon_perf_observe_*() — Unified Observability API
+ *
+ * These are the ONLY hooks that hardware-sampling backends should
+ * call.  They are split into NMI-safe (sampling, ring-enqueue) and
+ * process-context (event lifecycle, drain, matching, update) groups.
+ *
+ * Counters always increment when CONFIG_DAMON_PERF_OBSERVE=y.
+ * Tracepoints are guarded by trace_*_enabled() and incur zero
+ * overhead when ftrace is not attached.
+ */
+
+/* Event lifecycle — process context (kdamond / cpuhp callbacks) */
+void damon_perf_observe_event_created(struct damon_perf_event *event,
+		int cpu);
+void damon_perf_observe_event_bound(struct damon_perf_event *event,
+		int cpu, struct perf_event *perf_event);
+void damon_perf_observe_event_enabled(struct damon_perf_event *event,
+		int cpu, int state, int oncpu);
+void damon_perf_observe_event_disabled(struct damon_perf_event *event,
+		int cpu, int state);
+void damon_perf_observe_event_destroyed(struct damon_perf_event *event,
+		int cpu);
+void damon_perf_observe_event_free(struct damon_perf_event *event);
+
+/*
+ * Sample observed — NMI-safe.
+ *
+ * @reason:  0 = valid (queued to ring)
+ *           1 = data NULL
+ *           2 = addr == 0 (PMU did not populate)
+ *           3 = kernel address (vaddr only)
+ *           4 = phys_addr not valid (paddr only)
+ */
+void damon_perf_observe_sample(unsigned long addr, u64 data_src,
+		u64 period, int cpu, u8 reason,
+		u64 sample_flags, u64 sample_type);
+
+/* Ring operations — enqueue/overflow are NMI-safe */
+void damon_perf_observe_ring_enqueue(void);
+void damon_perf_observe_ring_overflow(int cpu);
+void damon_perf_observe_ring_dequeue(int cpu);
+void damon_perf_observe_ring_peak(unsigned int occupancy);
+
+/* Matching — process context (kdamond drain loop) */
+void damon_perf_observe_match(unsigned long addr, int cpu);
+void damon_perf_observe_miss(unsigned long addr, int cpu, int reason);
+void damon_perf_observe_update(int cpu);
+void damon_perf_observe_drain(unsigned int total, unsigned int matched);
+
+/* Debugfs (debug-only, format unstable) */
+int damon_perf_debugfs_init(void);
+
+/* Per-CPU stats accessors (for debugfs) */
+void damon_perf_stats_snapshot(int cpu, struct damon_perf_stats *dst);
+void damon_perf_stats_aggregate(struct damon_perf_stats *dst);
+
+/* Subsystem init */
+int damon_perf_framework_init(void);
+
+#else /* !CONFIG_DAMON_PERF_OBSERVE */
+
+static inline void damon_perf_observe_event_created(struct damon_perf_event *e,
+						    int c)
+{
+}
+
+static inline void damon_perf_observe_event_bound(struct damon_perf_event *e,
+						  int c, struct perf_event *p)
+{
+}
+
+static inline void damon_perf_observe_event_enabled(struct damon_perf_event *e,
+						    int c, int s, int o)
+{
+}
+
+static inline void damon_perf_observe_event_disabled(struct damon_perf_event *e,
+						     int c, int s)
+{
+}
+
+static inline void damon_perf_observe_event_destroyed(struct damon_perf_event *e,
+						      int c)
+{
+}
+
+static inline void damon_perf_observe_event_free(struct damon_perf_event *e)
+{
+}
+
+static inline void damon_perf_observe_sample(unsigned long a, u64 d, u64 p,
+					     int c, u8 r, u64 f, u64 t)
+{
+}
+
+static inline void damon_perf_observe_ring_enqueue(void)
+{
+}
+
+static inline void damon_perf_observe_ring_overflow(int c)
+{
+}
+
+static inline void damon_perf_observe_ring_dequeue(int c)
+{
+}
+
+static inline void damon_perf_observe_ring_peak(unsigned int o)
+{
+}
+
+static inline void damon_perf_observe_match(unsigned long a, int c)
+{
+}
+
+static inline void damon_perf_observe_miss(unsigned long a, int c, int r)
+{
+}
+
+static inline void damon_perf_observe_update(int c)
+{
+}
+
+static inline void damon_perf_observe_drain(unsigned int t, unsigned int m)
+{
+}
+
+static inline int damon_perf_debugfs_init(void)
+{
+	return 0;
+}
+
+static inline void damon_perf_stats_snapshot(int c, struct damon_perf_stats *d)
+{
+	memset(d, 0, sizeof(*d));
+}
+
+static inline void damon_perf_stats_aggregate(struct damon_perf_stats *d)
+{
+	memset(d, 0, sizeof(*d));
+}
+
+static inline int damon_perf_framework_init(void)
+{
+	return 0;
+}
+
+#endif /* CONFIG_DAMON_PERF_OBSERVE */
+
+#endif /* _DAMON_PERF_H */
-- 
2.43.0

[RFC PATCH 2/7] mm/damon/perf: implement observe API and per-CPU statistics engine

[Kunwu Chan]

From: Kunwu Chan <kunwu.chan@gmail.com>

Implement the observe_*() core: per-CPU monotonic counters, the
event-lifecycle hooks with lazily-allocated per-event per-CPU state
alongside the global per-CPU stage, NMI-safe sample classification,
ring enqueue/dequeue/peak, match/miss/update, and framework init.  The
global per-CPU stage advances monotonically so a late create/bind/
enable cannot regress a CPU that is already RUNNING.

damon_perf_observe_sample() records the execution context
(process/softirq/hardirq/NMI) so overflow callbacks can be verified to
fire in the expected context.

Co-developed-by: Lian Wang <lianux.mm@gmail.com>
Signed-off-by: Lian Wang <lianux.mm@gmail.com>
Signed-off-by: Kunwu Chan <kunwu.chan@gmail.com>
---
 mm/damon/perf/Makefile |   2 +
 mm/damon/perf/stats.c  | 295 +++++++++++++++++++++++++++++++++++++++++
 2 files changed, 297 insertions(+)
 create mode 100644 mm/damon/perf/stats.c

diff --git a/mm/damon/perf/Makefile b/mm/damon/perf/Makefile
index cc0d4f1d1d28..5c46d3da7ef8 100644
--- a/mm/damon/perf/Makefile
+++ b/mm/damon/perf/Makefile
@@ -1,3 +1,5 @@
 # SPDX-License-Identifier: GPL-2.0
 
 # Observability: per-CPU counters, tracepoints, debugfs perf_stats
+obj-$(CONFIG_DAMON_PERF_OBSERVE)	+= damon-perf.o
+damon-perf-objs			:= stats.o
diff --git a/mm/damon/perf/stats.c b/mm/damon/perf/stats.c
new file mode 100644
index 000000000000..a869f115bb26
--- /dev/null
+++ b/mm/damon/perf/stats.c
@@ -0,0 +1,295 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * DAMON Perf Observability — Per-CPU Statistics & Observe API
+ *
+ * All hardware-sampling backends funnel through the observe functions
+ * defined here.  Counters always increment when
+ * CONFIG_DAMON_PERF_OBSERVE=y; tracepoints are guarded by
+ * trace_*_enabled() for zero overhead when ftrace is not attached.
+ *
+ * Per-CPU counters are best-effort: individual u64 writes are atomic
+ * on 64-bit platforms, but no cross-field consistency is guaranteed.
+ * Snapshot reads may race with concurrent writers.  This is debug data
+ * — do not build policy on it.
+ */
+
+#include <linux/cpu.h>
+#include <linux/percpu.h>
+
+#include <trace/events/damon.h>
+
+#include "perf.h"
+
+/*
+ * Per-CPU statistics
+ */
+static DEFINE_PER_CPU(struct damon_perf_stats, damon_perf_stats);
+
+/*
+ * Per-CPU event state for the state machine — the maximum lifecycle
+ * stage ever reached on each CPU (any event).  Set by the NMI
+ * observe_sample() (ENABLED→RUNNING) and by lifecycle functions as a
+ * monotonic "best CPU so far".  Never reset to UNINIT — destroying one
+ * event must not hide that another event is still running on the same
+ * CPU.  Per-event per-CPU state (event->cpu_state) tracks individual
+ * event lifecycle for correctness when multiple events exist.
+ */
+static DEFINE_PER_CPU(int, damon_perf_cpu_state);
+
+/*
+ * Event lifecycle — process context (kdamond / cpuhp callbacks).
+ *
+ * Each event tracks its own per-CPU lifecycle state via
+ * event->cpu_state (allocated on first CREATED, freed on DESTROYED).
+ * The global damon_perf_cpu_state is also advanced (monotonic) so that
+ * debugfs perf_stats shows the "best" stage any event reached on each
+ * CPU.  Diagnostics are emitted via tracepoints, not dmesg.
+ */
+
+/*
+ * Advance the global per-CPU lifecycle state monotonically.  The global
+ * reflects the best stage any event ever reached on this CPU; a late
+ * create/bind/enable must not regress a CPU that is already RUNNING.
+ */
+static void damon_perf_cpu_state_advance(int cpu, int state)
+{
+	int cur = READ_ONCE(per_cpu(damon_perf_cpu_state, cpu));
+
+	if (state > cur)
+		WRITE_ONCE(per_cpu(damon_perf_cpu_state, cpu), state);
+}
+
+void damon_perf_observe_event_created(struct damon_perf_event *event, int cpu)
+{
+	if (!event->cpu_state) {
+		event->cpu_state = alloc_percpu(int);
+		if (!event->cpu_state)
+			return;
+	}
+	*per_cpu_ptr(event->cpu_state, cpu) = DAMON_PERF_STATE_CREATED;
+	damon_perf_cpu_state_advance(cpu, DAMON_PERF_STATE_CREATED);
+}
+
+void damon_perf_observe_event_bound(struct damon_perf_event *event,
+		int cpu, struct perf_event *perf_event)
+{
+	if (event->cpu_state)
+		*per_cpu_ptr(event->cpu_state, cpu) = DAMON_PERF_STATE_BOUND;
+	damon_perf_cpu_state_advance(cpu, DAMON_PERF_STATE_BOUND);
+}
+
+void damon_perf_observe_event_enabled(struct damon_perf_event *event,
+		int cpu, int state, int oncpu)
+{
+	if (event->cpu_state)
+		*per_cpu_ptr(event->cpu_state, cpu) = DAMON_PERF_STATE_ENABLED;
+	damon_perf_cpu_state_advance(cpu, DAMON_PERF_STATE_ENABLED);
+}
+
+void damon_perf_observe_event_disabled(struct damon_perf_event *event,
+		int cpu, int state)
+{
+	/* State unchanged: the event may be re-enabled later. */
+}
+
+void damon_perf_observe_event_destroyed(struct damon_perf_event *event, int cpu)
+{
+	/*
+	 * Reset only this CPU's slot.  The per-CPU array is owned by the
+	 * event as a whole and must NOT be freed here: destroyed() is
+	 * invoked from the per-CPU CPU-offline callback, so freeing the
+	 * whole array on the first offline CPU would leave every other
+	 * still-online CPU with a dangling pointer.  The array is freed
+	 * once by damon_perf_observe_event_free() at event teardown.
+	 */
+	if (event->cpu_state)
+		*per_cpu_ptr(event->cpu_state, cpu) = DAMON_PERF_STATE_UNINIT;
+}
+
+void damon_perf_observe_event_free(struct damon_perf_event *event)
+{
+	if (event->cpu_state) {
+		free_percpu(event->cpu_state);
+		event->cpu_state = NULL;
+	}
+}
+
+/*
+ * Sample observed — NMI-safe.
+ *
+ * The @cpu argument is the CPU the sample fired on; it is passed
+ * through to the tracepoint but stats are always written on the
+ * current CPU via this_cpu_ptr().
+ */
+
+void damon_perf_observe_sample(unsigned long addr, u64 data_src,
+		u64 period, int cpu, u8 reason,
+		u64 sample_flags, u64 sample_type)
+{
+	this_cpu_inc(damon_perf_stats.callback);
+
+	switch (reason) {
+	case 0:
+		this_cpu_inc(damon_perf_stats.sample_valid);
+		break;
+	case 1:
+		this_cpu_inc(damon_perf_stats.sample_null);
+		break;
+	case 2:
+		this_cpu_inc(damon_perf_stats.sample_addr_zero);
+		break;
+	case 3:
+		this_cpu_inc(damon_perf_stats.sample_kernel);
+		break;
+	case 4:
+		this_cpu_inc(damon_perf_stats.sample_invalid_phys);
+		break;
+	}
+
+	/* First callback advances state to RUNNING */
+	if (this_cpu_read(damon_perf_cpu_state) == DAMON_PERF_STATE_ENABLED)
+		this_cpu_write(damon_perf_cpu_state, DAMON_PERF_STATE_RUNNING);
+
+	/*
+	 * Record the execution context so callers can verify whether
+	 * overflow callbacks actually fire in the expected context
+	 * (e.g. NMI for IBS, process for SPE AUX drain).
+	 *
+	 * 0 = process, 1 = softirq, 2 = hardirq, 3 = NMI
+	 */
+	if (trace_damon_perf_sample_enabled())
+		trace_damon_perf_sample(addr, data_src, period, cpu, reason,
+				sample_flags, sample_type,
+				in_nmi() ? 3 : in_hardirq() ? 2 :
+				in_serving_softirq() ? 1 : 0);
+}
+
+/*
+ * Ring operations — enqueue / overflow are NMI-safe
+ */
+
+void damon_perf_observe_ring_enqueue(void)
+{
+	this_cpu_inc(damon_perf_stats.enqueue);
+}
+
+void damon_perf_observe_ring_overflow(int cpu)
+{
+	this_cpu_inc(damon_perf_stats.overflow);
+	if (trace_damon_perf_ring_overflow_enabled())
+		trace_damon_perf_ring_overflow(cpu);
+}
+
+void damon_perf_observe_ring_dequeue(int cpu)
+{
+	per_cpu_ptr(&damon_perf_stats, cpu)->dequeue++;
+}
+
+void damon_perf_observe_ring_peak(unsigned int occupancy)
+{
+	struct damon_perf_stats *st = this_cpu_ptr(&damon_perf_stats);
+
+	if (occupancy > READ_ONCE(st->ring_peak))
+		WRITE_ONCE(st->ring_peak, occupancy);
+}
+
+/*
+ * Matching — process context (kdamond drain loop)
+ */
+
+void damon_perf_observe_match(unsigned long addr, int cpu)
+{
+	per_cpu_ptr(&damon_perf_stats, cpu)->match++;
+}
+
+void damon_perf_observe_miss(unsigned long addr, int cpu, int reason)
+{
+	struct damon_perf_stats *st = per_cpu_ptr(&damon_perf_stats, cpu);
+
+	switch (reason) {
+	case DAMON_REPORT_MISS_TGID:
+		st->miss_tgid++;
+		break;
+	case DAMON_REPORT_MISS_NOREGION:
+		st->miss_region++;
+		break;
+	case DAMON_REPORT_MISS_BOUNDARY:
+		st->miss_boundary++;
+		break;
+	}
+
+	if (trace_damon_perf_report_missed_enabled())
+		trace_damon_perf_report_missed(addr, cpu, reason);
+}
+
+void damon_perf_observe_update(int cpu)
+{
+	per_cpu_ptr(&damon_perf_stats, cpu)->update++;
+}
+
+void damon_perf_observe_drain(unsigned int total, unsigned int matched)
+{
+	if (trace_damon_perf_drain_enabled())
+		trace_damon_perf_drain(total, matched);
+}
+
+/*
+ * Stats accessors (for debugfs)
+ */
+
+/*
+ * Best-effort per-CPU snapshot.  Individual u64 writes are atomic on
+ * 64-bit platforms; no cross-field consistency is guaranteed.  This is
+ * debug data only — do not build policy on it.
+ */
+void damon_perf_stats_snapshot(int cpu, struct damon_perf_stats *dst)
+{
+	struct damon_perf_stats *st = per_cpu_ptr(&damon_perf_stats, cpu);
+
+	*dst = *st;
+	dst->cpu_state = per_cpu(damon_perf_cpu_state, cpu);
+}
+
+void damon_perf_stats_aggregate(struct damon_perf_stats *dst)
+{
+	int cpu;
+
+	memset(dst, 0, sizeof(*dst));
+	dst->cpu_state = DAMON_PERF_STATE_RUNNING; /* start optimistic, take min */
+	cpus_read_lock();
+	for_each_online_cpu(cpu) {
+		struct damon_perf_stats *st = per_cpu_ptr(&damon_perf_stats, cpu);
+
+		dst->callback		+= st->callback;
+		dst->sample_valid	+= st->sample_valid;
+		dst->sample_null	+= st->sample_null;
+		dst->sample_addr_zero	+= st->sample_addr_zero;
+		dst->sample_kernel	+= st->sample_kernel;
+		dst->sample_invalid_phys += st->sample_invalid_phys;
+		dst->enqueue		+= st->enqueue;
+		dst->dequeue		+= st->dequeue;
+		dst->overflow		+= st->overflow;
+		dst->ring_peak		= max(dst->ring_peak, st->ring_peak);
+		dst->match		+= st->match;
+		dst->miss_tgid		+= st->miss_tgid;
+		dst->miss_region	+= st->miss_region;
+		dst->miss_boundary	+= st->miss_boundary;
+		dst->update		+= st->update;
+		/*
+		 * Aggregate per-CPU state: the pipeline hasn't passed a
+		 * stage until ALL CPUs have passed it, so take the min.
+		 */
+		dst->cpu_state = min_t(int, dst->cpu_state,
+				per_cpu(damon_perf_cpu_state, cpu));
+	}
+	cpus_read_unlock();
+}
+
+/*
+ * Subsystem init
+ */
+
+int damon_perf_framework_init(void)
+{
+	return 0;
+}
-- 
2.43.0

[RFC PATCH 3/7] mm/damon/perf: add debugfs statistics interface

[Kunwu Chan]

From: Lian Wang <lianux.mm@gmail.com>

Expose the aggregated per-CPU counters and the global event-stage in a
debug-only debugfs perf_stats file.  The format is explicitly unstable;
tracepoints are the stable diagnostic interface.

Co-developed-by: Kunwu Chan <kunwu.chan@gmail.com>
Signed-off-by: Kunwu Chan <kunwu.chan@gmail.com>
Signed-off-by: Lian Wang <lianux.mm@gmail.com>
---
 mm/damon/perf/Makefile  |   2 +-
 mm/damon/perf/debugfs.c | 142 ++++++++++++++++++++++++++++++++++++++++
 mm/damon/perf/stats.c   |   2 +-
 3 files changed, 144 insertions(+), 2 deletions(-)
 create mode 100644 mm/damon/perf/debugfs.c

diff --git a/mm/damon/perf/Makefile b/mm/damon/perf/Makefile
index 5c46d3da7ef8..150cbaa875fa 100644
--- a/mm/damon/perf/Makefile
+++ b/mm/damon/perf/Makefile
@@ -2,4 +2,4 @@
 
 # Observability: per-CPU counters, tracepoints, debugfs perf_stats
 obj-$(CONFIG_DAMON_PERF_OBSERVE)	+= damon-perf.o
-damon-perf-objs			:= stats.o
+damon-perf-objs			:= stats.o debugfs.o
diff --git a/mm/damon/perf/debugfs.c b/mm/damon/perf/debugfs.c
new file mode 100644
index 000000000000..c54dd7644ac3
--- /dev/null
+++ b/mm/damon/perf/debugfs.c
@@ -0,0 +1,142 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * DAMON Perf Observability — debugfs Interface
+ *
+ * Exposes one file under /sys/kernel/debug/damon/:
+ *
+ *   perf_stats — per-CPU pipeline statistics in tabular form
+ *
+ * DEBUG ONLY — format may change without notice; do not parse in
+ * scripts.  For stable diagnostics, use the tracepoints under
+ *   /sys/kernel/debug/tracing/events/damon/
+ */
+
+#include <linux/cpu.h>
+#include <linux/cpumask.h>
+#include <linux/debugfs.h>
+#include <linux/seq_file.h>
+
+#include "perf.h"
+
+static struct dentry *damon_debugfs_dir;
+
+/*
+ * perf_stats
+ */
+
+static const char *state_name(int s)
+{
+	switch (s) {
+	case DAMON_PERF_STATE_UNINIT:	return "UNINIT";
+	case DAMON_PERF_STATE_CREATED:	return "CREATED";
+	case DAMON_PERF_STATE_BOUND:	return "BOUND";
+	case DAMON_PERF_STATE_ENABLED:	return "ENABLED";
+	case DAMON_PERF_STATE_RUNNING:	return "RUNNING";
+	case DAMON_PERF_STATE_ERROR:	return "ERROR";
+	default:			return "?";
+	}
+}
+
+static int perf_stats_show(struct seq_file *m, void *v)
+{
+	struct damon_perf_stats agg, st;
+	int cpu;
+	bool first = true;
+
+	damon_perf_stats_aggregate(&agg);
+
+	seq_puts(m, "    --------------  ----------\n");
+	seq_puts(m, "    Counter          Value\n");
+	seq_puts(m, "    --------------  ----------\n");
+
+#define STAT_ROW(label, field) \
+	seq_printf(m, "    %-12s   %8llu\n", label, agg.field)
+
+	STAT_ROW("callback",    callback);
+	STAT_ROW("valid",       sample_valid);
+	STAT_ROW("null",        sample_null);
+	STAT_ROW("addr_zero",   sample_addr_zero);
+	STAT_ROW("kernel",      sample_kernel);
+	STAT_ROW("inv_phys",    sample_invalid_phys);
+	STAT_ROW("enqueue",     enqueue);
+	STAT_ROW("dequeue",     dequeue);
+	STAT_ROW("overflow",    overflow);
+	STAT_ROW("ring_peak",   ring_peak);
+	STAT_ROW("match",       match);
+	STAT_ROW("miss_tgid",   miss_tgid);
+	STAT_ROW("miss_region", miss_region);
+	STAT_ROW("miss_bound",  miss_boundary);
+	STAT_ROW("update",      update);
+
+#undef STAT_ROW
+
+	seq_puts(m, "    --------------  ----------\n\n");
+
+	/* Per-CPU breakdown */
+	cpus_read_lock();
+	for_each_online_cpu(cpu) {
+		damon_perf_stats_snapshot(cpu, &st);
+
+		/* Skip truly idle CPUs */
+		if (st.cpu_state == DAMON_PERF_STATE_UNINIT &&
+		    !st.callback && !st.enqueue && !st.dequeue)
+			continue;
+
+		if (first) {
+			seq_puts(m, "  Per-CPU (non-zero / non-UNINIT):\n");
+			first = false;
+		}
+
+		seq_printf(m, "  CPU%02d: st=%-7s cb=%llu enq=%llu deq=%llu ovf=%llu match=%llu tgid=%llu noreg=%llu bound=%llu upd=%llu\n",
+			cpu, state_name(st.cpu_state),
+			st.callback, st.enqueue, st.dequeue,
+			st.overflow, st.match,
+			st.miss_tgid, st.miss_region, st.miss_boundary,
+			st.update);
+	}
+	cpus_read_unlock();
+
+	return 0;
+}
+
+static int perf_stats_open(struct inode *inode, struct file *file)
+{
+	return single_open(file, perf_stats_show, NULL);
+}
+
+static const struct file_operations perf_stats_fops = {
+	.open		= perf_stats_open,
+	.read		= seq_read,
+	.llseek		= seq_lseek,
+	.release	= single_release,
+};
+
+/*
+ * Init / teardown
+ */
+
+int damon_perf_debugfs_init(void)
+{
+	if (!debugfs_initialized())
+		return -ENODEV;
+
+	/*
+	 * Create the "damon" directory first.  debugfs_create_dir() mounts
+	 * debugfs via simple_pin_fs() before touching debugfs_mount, so it
+	 * is safe to call during initcall time -- unlike debugfs_lookup(),
+	 * which dereferences debugfs_mount unconditionally and crashes with
+	 * a NULL mount.  If the directory already exists (created by another
+	 * DAMON interface) debugfs_create_dir() returns -EEXIST; fall back
+	 * to debugfs_lookup(), which is now safe because the mount exists.
+	 */
+	damon_debugfs_dir = debugfs_create_dir("damon", NULL);
+	if (damon_debugfs_dir == ERR_PTR(-EEXIST))
+		damon_debugfs_dir = debugfs_lookup("damon", NULL);
+	if (IS_ERR(damon_debugfs_dir))
+		return PTR_ERR(damon_debugfs_dir);
+
+	debugfs_create_file("perf_stats", 0400, damon_debugfs_dir,
+			NULL, &perf_stats_fops);
+
+	return 0;
+}
diff --git a/mm/damon/perf/stats.c b/mm/damon/perf/stats.c
index a869f115bb26..ae5b0037a31d 100644
--- a/mm/damon/perf/stats.c
+++ b/mm/damon/perf/stats.c
@@ -291,5 +291,5 @@ void damon_perf_stats_aggregate(struct damon_perf_stats *dst)
 
 int damon_perf_framework_init(void)
 {
-	return 0;
+	return damon_perf_debugfs_init();
 }
-- 
2.43.0

[RFC PATCH 4/7] mm/damon: integrate observe API into vaddr overflow handlers and core drain

[Kunwu Chan]

From: Kunwu Chan <kunwu.chan@gmail.com>

Wire the observe_*() calls into the DAMON hot paths: vaddr access-check
overflow handlers report into the per-CPU ring, and the kdamond drain
matches each report against the target whose tgid owns it.  The observe
calls are pure side-effect statistics (no-ops under
CONFIG_DAMON_PERF_OBSERVE=n), so the switch never changes DAMON matching
semantics.  The vaddr teardown frees the per-event cpu_state array.

Read ring->tail once with READ_ONCE in damon_report_access() and reuse
the cached value for the peak-occupancy estimate, avoiding a torn read
and a compiler reload on the producer side.

Co-developed-by: Lian Wang <lianux.mm@gmail.com>
Signed-off-by: Lian Wang <lianux.mm@gmail.com>
Signed-off-by: Kunwu Chan <kunwu.chan@gmail.com>
---
 mm/damon/core.c  | 129 +++++++++++++++++++++++++++++++++++++----------
 mm/damon/vaddr.c | 104 +++++++++++++++++++++++++++++++++++---
 2 files changed, 199 insertions(+), 34 deletions(-)

diff --git a/mm/damon/core.c b/mm/damon/core.c
index 609d627e2b33..377f07122fb0 100644
--- a/mm/damon/core.c
+++ b/mm/damon/core.c
@@ -21,6 +21,7 @@
 
 /* for damon_get_folio() used by node eligible memory metrics */
 #include "ops-common.h"
+#include "perf/perf.h"
 
 #define CREATE_TRACE_POINTS
 #include <trace/events/damon.h>
@@ -2243,24 +2244,41 @@ void damon_report_access(struct damon_access_report *report)
 	preempt_disable();
 	if (local_inc_return(this_cpu_ptr(&damon_report_ring_busy)) != 1) {
 		/* NMI nested on a process-context producer; drop. */
-		trace_damon_perf_ring_overflow(smp_processor_id());
+#ifdef CONFIG_DAMON_PERF_OBSERVE
+		damon_perf_observe_ring_overflow(smp_processor_id());
+#endif /* CONFIG_DAMON_PERF_OBSERVE */
 		goto out;
 	}
 
 	ring = this_cpu_ptr(&damon_report_rings);
 	head = ring->head;
 	next = (head + 1) & DAMON_REPORT_RING_MASK;
+	{
+		unsigned int tail = READ_ONCE(ring->tail);
 
-	if (next == READ_ONCE(ring->tail)) {
-		trace_damon_perf_ring_overflow(smp_processor_id());
-		goto out;
-	}
+		if (next == tail) {
+#ifdef CONFIG_DAMON_PERF_OBSERVE
+			damon_perf_observe_ring_overflow(smp_processor_id());
+#endif /* CONFIG_DAMON_PERF_OBSERVE */
+			goto out;
+		}
 
-	ring->entries[head] = *report;
-	ring->entries[head].report_jiffies = jiffies;
-	smp_wmb(); /* publish entry before head advance */
-	WRITE_ONCE(ring->head, next);
-	WRITE_ONCE(*this_cpu_ptr(&damon_ring_pending), 1);
+		ring->entries[head] = *report;
+		ring->entries[head].report_jiffies = jiffies;
+		smp_wmb(); /* publish entry before head advance */
+		WRITE_ONCE(ring->head, next);
+		WRITE_ONCE(*this_cpu_ptr(&damon_ring_pending), 1);
+#ifdef CONFIG_DAMON_PERF_OBSERVE
+		damon_perf_observe_ring_enqueue();
+		/*
+		 * Track peak occupancy for health evaluation.
+		 * next is the new head; tail was read before enqueue
+		 * (may be slightly stale — acceptable for a peak estimate).
+		 */
+		damon_perf_observe_ring_peak(
+				(next - tail) & DAMON_REPORT_RING_MASK);
+#endif /* CONFIG_DAMON_PERF_OBSERVE */
+	}
 out:
 	local_dec(this_cpu_ptr(&damon_report_ring_busy));
 	preempt_enable();
@@ -2276,6 +2294,9 @@ void damon_report_page_fault(struct vm_fault *vmf, bool huge_pmd)
 		.tid = current->pid,
 		.tgid = task_tgid_nr(current),
 		.is_write = vmf->flags & FAULT_FLAG_WRITE,
+#ifdef CONFIG_DAMON_PERF_OBSERVE
+		.source = DAMON_REPORT_SRC_PAGE_FAULT,
+#endif /* CONFIG_DAMON_PERF_OBSERVE */
 	};
 
 	if (huge_pmd)
@@ -3917,8 +3938,16 @@ static bool damon_sample_filter_out(struct damon_access_report *report,
 	return !filter->allow;
 }
 
-static void kdamond_apply_access_report(struct damon_access_report *report,
-		struct damon_target *t,
+/*
+ * Try to apply one access report to a target's region snapshot.
+ *
+ * Caller has already resolved tgid (for pid-based monitoring), so this
+ * function only does address-to-region matching.  Miss reasons for
+ * trace_damon_perf_report_missed use enum damon_report_miss_reason.
+ *
+ * Return: true if the report fell inside a known region, false otherwise.
+ */
+static bool kdamond_apply_access_report(struct damon_access_report *report,
 		struct damon_region **regions, unsigned int nr_regions,
 		struct damon_ctx *ctx)
 {
@@ -3926,13 +3955,7 @@ static void kdamond_apply_access_report(struct damon_access_report *report,
 	unsigned long addr;
 	int left, right, mid;
 
-	if (damon_target_has_pid(ctx)) {
-		if (pid_nr(t->pid) != report->tgid)
-			return;
-		addr = report->vaddr;
-	} else {
-		addr = report->paddr;
-	}
+	addr = damon_target_has_pid(ctx) ? report->vaddr : report->paddr;
 
 	/* Binary search the snapshot for the region containing addr. */
 	left = 0;
@@ -3951,17 +3974,27 @@ static void kdamond_apply_access_report(struct damon_access_report *report,
 		}
 	}
 
-	if (!r)
-		return;
+	if (!r) {
+		damon_perf_observe_miss(addr, report->cpu,
+				DAMON_REPORT_MISS_NOREGION);
+		return false;
+	}
 	/* Reject reports straddling a region boundary. */
-	if (addr + report->size > r->ar.end)
-		return;
+	if (addr + report->size > r->ar.end) {
+		damon_perf_observe_miss(addr, report->cpu,
+				DAMON_REPORT_MISS_BOUNDARY);
+		return false;
+	}
 	if (!r->access_reported) {
 		damon_update_region_access_rate(r, true, &ctx->attrs);
 		r->access_reported = true;
+		damon_perf_observe_update(report->cpu);
 	}
+	damon_perf_observe_match(addr, report->cpu);
+	return true;
 }
 
+
 static unsigned int kdamond_apply_zero_access_report(struct damon_ctx *ctx)
 {
 	struct damon_target *t;
@@ -4045,6 +4078,7 @@ static unsigned int kdamond_check_reported_accesses(struct damon_ctx *ctx)
 	struct damon_target_lookup *tbl;
 	unsigned int nr_targets = 0;
 	unsigned int i;
+	unsigned int total_reports = 0, matched_reports = 0;
 
 	tbl = damon_build_target_lookup(ctx, &nr_targets);
 	if (!tbl) {
@@ -4077,6 +4111,10 @@ static unsigned int kdamond_check_reported_accesses(struct damon_ctx *ctx)
 		while (tail != head) {
 			struct damon_access_report *report =
 				&ring->entries[tail];
+			bool applied = false;
+
+			/* Count every entry removed from the ring */
+			damon_perf_observe_ring_dequeue(report->cpu);
 
 			if (time_before(report->report_jiffies,
 					jiffies - usecs_to_jiffies(
@@ -4085,16 +4123,52 @@ static unsigned int kdamond_check_reported_accesses(struct damon_ctx *ctx)
 			if (damon_sample_filter_out(report,
 					&ctx->sample_control))
 				goto next;
-			for (i = 0; i < nr_targets; i++)
-				kdamond_apply_access_report(report,
-						tbl[i].t,
+			/*
+			 * For pid-based monitoring, resolve tgid to the
+			 * single matching target before calling
+			 * kdamond_apply_access_report(), avoiding a
+			 * spurious miss tracepoint for every non-matching
+			 * target.
+			 */
+			if (damon_target_has_pid(ctx)) {
+				for (i = 0; i < nr_targets; i++) {
+					if (pid_nr(tbl[i].t->pid) ==
+							report->tgid) {
+						applied =
+					    kdamond_apply_access_report(
+							report,
+							tbl[i].regions,
+							tbl[i].nr_regions,
+							ctx);
+						break;
+					}
+				}
+				if (!applied && i == nr_targets)
+					damon_perf_observe_miss(
+						report->vaddr,
+						report->cpu,
+						DAMON_REPORT_MISS_TGID);
+			} else {
+				for (i = 0; i < nr_targets; i++)
+					applied |=
+					    kdamond_apply_access_report(
+						report,
 						tbl[i].regions,
 						tbl[i].nr_regions, ctx);
+			}
+			total_reports++;
+			if (applied)
+				matched_reports++;
+
 next:
 			tail = (tail + 1) & DAMON_REPORT_RING_MASK;
 		}
 		WRITE_ONCE(ring->tail, tail);
 	}
+
+	if (total_reports)
+		damon_perf_observe_drain(total_reports, matched_reports);
+
 	/* For nr_accesses_bp, absence of access should also be reported. */
 	return kdamond_apply_zero_access_report(ctx);
 }
@@ -4158,8 +4232,9 @@ static int kdamond_fn(void *data)
 		ctx->passed_sample_intervals++;
 
 		if (!list_empty(&ctx->perf_events) ||
-		    ctx->sample_control.primitives_enabled.page_fault)
+		    ctx->sample_control.primitives_enabled.page_fault) {
 			max_nr_accesses = kdamond_check_reported_accesses(ctx);
+		}
 		else if (ctx->ops.check_accesses)
 			max_nr_accesses = ctx->ops.check_accesses(ctx);
 		if (ctx->ops.apply_probes)
diff --git a/mm/damon/vaddr.c b/mm/damon/vaddr.c
index 73fcea91afa0..a68c7262d533 100644
--- a/mm/damon/vaddr.c
+++ b/mm/damon/vaddr.c
@@ -17,6 +17,8 @@
 #include <linux/pagewalk.h>
 #include <linux/sched/mm.h>
 
+#include "perf/perf.h"
+
 #include "../internal.h"
 #include "ops-common.h"
 
@@ -975,13 +977,49 @@ static void damon_perf_overflow_vaddr(struct perf_event *perf_event,
 		struct perf_sample_data *data, struct pt_regs *regs)
 {
 	struct damon_access_report report;
+	u64 data_src_val;
+	u64 period_val;
+
+	/*
+	 * Observe every hardware sample through the unified API.
+	 *
+	 * reason encodes why a sample was dropped at the handler level:
+	 *   0 = valid, queued to ring
+	 *   1 = data == NULL
+	 *   2 = addr == 0 (PMU did not populate data->addr)
+	 *   3 = kernel address (addr >= TASK_SIZE)
+	 */
+	if (!data) {
+		damon_perf_observe_sample(0, 0, 0,
+				smp_processor_id(), 1, 0,
+				perf_event->attr.sample_type);
+		return;
+	}
 
-	if (!data || !data->addr)
+	data_src_val = data->data_src.val;
+	period_val = data->period;
+
+	if (!data->addr) {
+		damon_perf_observe_sample(0, data_src_val, period_val,
+				smp_processor_id(), 2,
+				data->sample_flags,
+				perf_event->attr.sample_type);
 		return;
+	}
 
 	/* Drop kernel-VA hits -- only user-space VAs land in damon vaddr regions. */
-	if (data->addr >= TASK_SIZE)
+	if (data->addr >= TASK_SIZE) {
+		damon_perf_observe_sample(data->addr, data_src_val, period_val,
+				smp_processor_id(), 3,
+				data->sample_flags,
+				perf_event->attr.sample_type);
 		return;
+	}
+
+	damon_perf_observe_sample(data->addr, data_src_val, period_val,
+			smp_processor_id(), 0,
+			data->sample_flags,
+			perf_event->attr.sample_type);
 
 	report = (struct damon_access_report){
 		.vaddr = data->addr & PAGE_MASK,
@@ -990,6 +1028,9 @@ static void damon_perf_overflow_vaddr(struct perf_event *perf_event,
 		.tid = current->pid,
 		.tgid = current->tgid,
 		.is_write = !!(data->data_src.mem_op & PERF_MEM_OP_STORE),
+#ifdef CONFIG_DAMON_PERF_OBSERVE
+		.source = DAMON_REPORT_SRC_PERF_OVERFLOW,
+#endif /* CONFIG_DAMON_PERF_OBSERVE */
 	};
 	damon_report_access(&report);
 }
@@ -998,9 +1039,18 @@ static void damon_perf_overflow_paddr(struct perf_event *perf_event,
 		struct perf_sample_data *data, struct pt_regs *regs)
 {
 	struct damon_access_report report;
+	u64 data_src_val;
+	u64 period_val;
 
-	if (!data)
+	if (!data) {
+		damon_perf_observe_sample(0, 0, 0,
+				smp_processor_id(), 1, 0,
+				perf_event->attr.sample_type);
 		return;
+	}
+
+	data_src_val = data->data_src.val;
+	period_val = data->period;
 
 	/*
 	 * AMD IBS Op only populates data->phys_addr when
@@ -1008,14 +1058,27 @@ static void damon_perf_overflow_paddr(struct perf_event *perf_event,
 	 * carries a stale value.  Gate on sample_flags rather than testing
 	 * phys_addr for zero (which would also drop legitimate page 0).
 	 */
-	if (!(data->sample_flags & PERF_SAMPLE_PHYS_ADDR))
+	if (!(data->sample_flags & PERF_SAMPLE_PHYS_ADDR)) {
+		damon_perf_observe_sample(0, data_src_val,
+				period_val, smp_processor_id(), 4,
+				data->sample_flags,
+				perf_event->attr.sample_type);
 		return;
+	}
+
+	damon_perf_observe_sample(data->phys_addr, data_src_val, period_val,
+			smp_processor_id(), 0,
+			data->sample_flags,
+			perf_event->attr.sample_type);
 
 	report = (struct damon_access_report){
 		.paddr = data->phys_addr & PAGE_MASK,
 		.size = PAGE_SIZE,
 		.cpu = smp_processor_id(),
 		.is_write = !!(data->data_src.mem_op & PERF_MEM_OP_STORE),
+#ifdef CONFIG_DAMON_PERF_OBSERVE
+		.source = DAMON_REPORT_SRC_PERF_OVERFLOW,
+#endif /* CONFIG_DAMON_PERF_OBSERVE */
 	};
 	damon_report_access(&report);
 }
@@ -1070,6 +1133,8 @@ static int damon_perf_cpu_online(unsigned int cpu, struct hlist_node *node)
 	if (!perf)
 		return 0;
 
+	damon_perf_observe_event_created(event, cpu);
+
 	damon_perf_event_init_attr(event, &attr);
 
 	/*
@@ -1092,14 +1157,20 @@ static int damon_perf_cpu_online(unsigned int cpu, struct hlist_node *node)
 		return 0;	/* never block CPU online */
 	}
 	*per_cpu_ptr(perf->event, cpu) = perf_event;
+
+	damon_perf_observe_event_bound(event, cpu, perf_event);
+
 	/*
 	 * Late-online CPU after the substrate is armed: events are created
 	 * with attr.disabled = 1 and would otherwise stay quiescent on this
 	 * CPU until the next arm walk.  Enable here so coverage matches the
 	 * already-online CPUs.
 	 */
-	if (event->ctx && READ_ONCE(event->ctx->perf_events_active))
+	if (event->ctx && READ_ONCE(event->ctx->perf_events_active)) {
 		perf_event_enable(perf_event);
+		damon_perf_observe_event_enabled(event, cpu,
+				perf_event->state, perf_event->oncpu);
+	}
 	return 0;
 }
 
@@ -1115,6 +1186,7 @@ static int damon_perf_cpu_offline(unsigned int cpu, struct hlist_node *node)
 
 	perf_event = per_cpu(*perf->event, cpu);
 	if (perf_event) {
+		damon_perf_observe_event_destroyed(event, cpu);
 		perf_event_disable(perf_event);
 		perf_event_release_kernel(perf_event);
 		*per_cpu_ptr(perf->event, cpu) = NULL;
@@ -1133,8 +1205,12 @@ void damon_perf_event_arm(struct damon_perf_event *event)
 
 	for_each_online_cpu(cpu) {
 		perf_event = *per_cpu_ptr(perf->event, cpu);
-		if (perf_event)
+		if (perf_event) {
 			perf_event_enable(perf_event);
+			damon_perf_observe_event_enabled(event, cpu,
+					perf_event->state,
+					perf_event->oncpu);
+		}
 	}
 }
 
@@ -1149,8 +1225,11 @@ void damon_perf_event_disarm(struct damon_perf_event *event)
 
 	for_each_online_cpu(cpu) {
 		perf_event = *per_cpu_ptr(perf->event, cpu);
-		if (perf_event)
+		if (perf_event) {
 			perf_event_disable(perf_event);
+			damon_perf_observe_event_disabled(event, cpu,
+					perf_event->state);
+		}
 	}
 }
 
@@ -1192,6 +1271,7 @@ int damon_perf_init(struct damon_ctx *ctx, struct damon_perf_event *event)
 	return 0;
 
 free_event:
+	damon_perf_observe_event_free(event);
 	free_percpu(perf->event);
 free_perf:
 	kfree(perf);
@@ -1203,6 +1283,8 @@ void damon_perf_cleanup(struct damon_ctx *ctx, struct damon_perf_event *event)
 {
 	struct damon_perf *perf = event->priv;
 
+	damon_perf_observe_event_free(event);
+
 	if (!perf)
 		return;
 
@@ -1244,6 +1326,14 @@ static int __init damon_va_initcall(void)
 	if (err < 0)
 		return err;
 	damon_perf_cpuhp_state = err;
+
+#ifdef CONFIG_DAMON_PERF_OBSERVE
+	err = damon_perf_framework_init();
+	if (err < 0)
+		pr_warn("damon-perf: framework init failed, observability unavailable: %d\n",
+			err);
+	/* Non-fatal: vaddr/fvaddr ops still register. */
+#endif /* CONFIG_DAMON_PERF_OBSERVE */
 #endif
 
 	err = damon_register_ops(&ops);
-- 
2.43.0

[RFC PATCH 5/7] selftests/damon: add automated layer-by-layer observability test

[Kunwu Chan]

From: Kunwu Chan <kunwu.chan@gmail.com>

Drive the observe framework end-to-end with a software page-fault PMU
positive control (no hardware PMU required): create a kdamond with a
perf event via the DAMON sysfs interface, run a memory-pressure
workload, capture a bounded trace window, and assert each pipeline
layer from per-run snapshot/delta counter deltas (counters are
cumulative since boot, so the snapshot is taken before the workload
window): callbacks, valid data addresses, ring enqueue/dequeue/
overflow, drain match/update, the four DAMON perf tracepoints, and the
per-CPU state column advancing CREATED/BOUND/ENABLED to RUNNING after
the first callback.

A clean-session guard refuses to run while a kdamond already exists.
Ends with a PMU support verdict (FULLY INTEGRATED / PLUMBING-ONLY /
UNUSABLE).  Cleanup is ownership safe: it tears down only the kdamond,
workload and debugfs mount created by this invocation, and retains the
raw evidence directory by default.

Co-developed-by: Lian Wang <lianux.mm@gmail.com>
Signed-off-by: Lian Wang <lianux.mm@gmail.com>
Signed-off-by: Kunwu Chan <kunwu.chan@gmail.com>
---
 tools/testing/selftests/damon/Makefile        |   1 +
 .../selftests/damon/damon_perf_obs_test.sh    | 562 ++++++++++++++++++
 2 files changed, 563 insertions(+)
 create mode 100755 tools/testing/selftests/damon/damon_perf_obs_test.sh

diff --git a/tools/testing/selftests/damon/Makefile b/tools/testing/selftests/damon/Makefile
index 2180c328a825..1db8fa95ba2d 100644
--- a/tools/testing/selftests/damon/Makefile
+++ b/tools/testing/selftests/damon/Makefile
@@ -23,4 +23,5 @@ TEST_PROGS += sysfs_no_op_commit_break.py
 
 EXTRA_CLEAN = __pycache__
 
+TEST_PROGS += damon_perf_obs_test.sh
 include ../lib.mk
diff --git a/tools/testing/selftests/damon/damon_perf_obs_test.sh b/tools/testing/selftests/damon/damon_perf_obs_test.sh
new file mode 100755
index 000000000000..4c4074cdd191
--- /dev/null
+++ b/tools/testing/selftests/damon/damon_perf_obs_test.sh
@@ -0,0 +1,562 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+#
+# DAMON Perf Observability Framework — Automated Layer-by-Layer Test
+#
+# Validates all 7 pipeline stages:
+#   Layer 1: Event Create   Layer 2: Event Bind
+#   Layer 3: Event Enable   Layer 4: Sampling (callback)
+#   Layer 5: Ring           Layer 6: Drain
+#   Layer 7: Match & Update
+#
+# The framework counters are cumulative since boot, so this script
+# snapshots them before the workload and reports per-run deltas.  It
+# also clears the trace buffer before the sampling window so trace.txt
+# carries only records produced by this run.
+#
+# Usage:
+#   # Software page-fault positive control (exercises the FULL pipeline,
+#   # works on any machine, no HW PMU required).  Defaults to sampling
+#   # every page fault (period 1); override with --freq/--period:
+#   sudo ./damon_perf_obs_test.sh --pmu software
+#   sudo ./damon_perf_obs_test.sh --pmu software --freq 1 --sample-freq 100
+#
+#   # With ARM SPE:
+#   sudo ./damon_perf_obs_test.sh --pmu arm_spe_0 --freq 0 --period 256
+#
+#   # With any PMU type number:
+#   sudo ./damon_perf_obs_test.sh --pmu-type 38 --freq 0 --period 256
+#
+# Notes:
+#   - `--config N` sets perf_event_attr.config.  For PERF_TYPE_SOFTWARE,
+#     config 2 (PERF_COUNT_SW_PAGE_FAULTS) is the only software event
+#     that populates data->addr; cpu-clock (config 0) delivers callbacks
+#     with addr always 0, so it can only validate Layers 1-4.
+#   - ARM SPE cannot sample through perf_event_create_kernel_counter()
+#     (it requires an AUX ring buffer, see arm_spe_pmu.c), so an SPE run
+#     is expected to report zero callbacks until an AUX backend exists.
+#     A zero-callback delta is the correct "PMU not usable" verdict.
+#
+# Requirements:
+#   - CONFIG_DAMON_PERF_OBSERVE=y (fatal if missing)
+#   - Root privileges
+#   - debugfs mounted
+
+set -e
+
+# ---- defaults ----
+PMU_NAME=""
+PMU_TYPE=""
+PMU_CONFIG=0
+CONFIG_EXPLICIT=0
+FREQ=0
+PERIOD=256
+FREQ_EXPLICIT=0
+PERIOD_EXPLICIT=0
+SAMPLE_FREQ=100
+TIMEOUT=10
+TRACE_WINDOW=3
+TARGET_PID=""
+RESULTS_DIR="/tmp/damon_perf_test_$$"
+PASSED=0
+FAILED=0
+SKIPPED=0
+STRESS_PID=""
+CREATED_KDAMOND=0
+MOUNTED_DEBUGFS=0
+KEEP_RESULTS=${KEEP_RESULTS:-1}
+
+# ---- helpers ----
+pass() { echo "  [PASS] $1"; PASSED=$((PASSED + 1)); }
+fail() { echo "  [FAIL] $1 — $2"; FAILED=$((FAILED + 1)); }
+skip() { echo "  [SKIP] $1 — $2"; SKIPPED=$((SKIPPED + 1)); }
+die()  { echo "FATAL: $1"; exit 1; }
+
+# ---- sysfs roots (kept under 100 columns) ----
+KD=/sys/kernel/mm/damon/admin/kdamonds
+ADMIN=$KD/0
+TRACE=/sys/kernel/debug/tracing
+TPD=$TRACE/events/damon
+PE=$ADMIN/contexts/0/monitoring_attrs/sample/perf_events
+
+# ---- saved pre-test state (restored in cleanup so the test is
+# ---- side-effect free: tracepoints, tracing_on)
+ORIG_TRACING_ON=$(cat $TRACE/tracing_on 2>/dev/null || echo 0)
+ORIG_TP_SAMPLE=$(cat $TRACE/events/damon/damon_perf_sample/enable 2>/dev/null || echo 0)
+ORIG_TP_OVERFLOW=$(cat $TRACE/events/damon/damon_perf_ring_overflow/enable 2>/dev/null || echo 0)
+ORIG_TP_MISSED=$(cat $TRACE/events/damon/damon_perf_report_missed/enable 2>/dev/null || echo 0)
+ORIG_TP_DRAIN=$(cat $TRACE/events/damon/damon_perf_drain/enable 2>/dev/null || echo 0)
+
+cleanup() {
+	echo ""
+	echo "=== Cleaning up ==="
+	if [[ -n "$STRESS_PID" ]]; then
+		kill "$STRESS_PID" 2>/dev/null || true
+		wait "$STRESS_PID" 2>/dev/null || true
+		STRESS_PID=""
+	fi
+	# Tear down only the kdamond instance created by this test.  In
+	# particular, the early "existing kdamonds" guard must be read-only.
+	if [[ "$CREATED_KDAMOND" == "1" ]]; then
+		echo off > $ADMIN/state 2>/dev/null || true
+		echo 0 > $KD/nr_kdamonds 2>/dev/null || true
+		CREATED_KDAMOND=0
+	fi
+	# Restore tracepoint and tracing state
+	echo 0 > $TRACE/tracing_on 2>/dev/null || true
+	echo "$ORIG_TP_SAMPLE" > $TRACE/events/damon/damon_perf_sample/enable 2>/dev/null || true
+	echo "$ORIG_TP_OVERFLOW" > $TPD/damon_perf_ring_overflow/enable 2>/dev/null || true
+	echo "$ORIG_TP_MISSED" > $TPD/damon_perf_report_missed/enable 2>/dev/null || true
+	echo "$ORIG_TP_DRAIN" > $TRACE/events/damon/damon_perf_drain/enable 2>/dev/null || true
+	echo "$ORIG_TRACING_ON" > $TRACE/tracing_on 2>/dev/null || true
+	if [[ "$MOUNTED_DEBUGFS" == "1" ]]; then
+		umount /sys/kernel/debug 2>/dev/null || true
+		MOUNTED_DEBUGFS=0
+	fi
+	[ "$KEEP_RESULTS" != "1" ] && rm -rf "$RESULTS_DIR"
+}
+trap cleanup EXIT
+
+# ---- argument parsing ----
+while [[ $# -gt 0 ]]; do
+	case "$1" in
+		--pmu)       PMU_NAME="$2"; shift 2 ;;
+		--pmu-type)  PMU_TYPE="$2"; shift 2 ;;
+		--config)    PMU_CONFIG="$2"; CONFIG_EXPLICIT=1; shift 2 ;;
+		--freq)      FREQ="$2"; FREQ_EXPLICIT=1; shift 2 ;;
+		--period)    PERIOD="$2"; PERIOD_EXPLICIT=1; shift 2 ;;
+		--sample-freq) SAMPLE_FREQ="$2"; shift 2 ;;
+		--trace-window) TRACE_WINDOW="$2"; shift 2 ;;
+		--timeout)   TIMEOUT="$2"; shift 2 ;;
+		--pid)       TARGET_PID="$2"; shift 2 ;;
+		*) echo "Unknown: $1"; exit 1 ;;
+	esac
+done
+
+# Resolve PMU type
+if [[ -n "$PMU_NAME" && -z "$PMU_TYPE" ]]; then
+	if [[ "$PMU_NAME" == "software" ]]; then
+		PMU_TYPE=1
+	else
+		PMU_TYPE=$(cat /sys/bus/event_source/devices/$PMU_NAME/type 2>/dev/null) ||
+			die "Cannot find PMU: $PMU_NAME"
+	fi
+fi
+[[ -z "$PMU_TYPE" ]] && die "Specify --pmu <name> or --pmu-type <number>"
+
+# For PERF_TYPE_SOFTWARE default to PERF_COUNT_SW_PAGE_FAULTS (config 2):
+# the only software event that carries a data address, i.e. the only one
+# that can exercise Layers 5-7.  Override with --config 0 for a pure
+# plumbing (cpu-clock) smoke test.
+if [[ "$PMU_TYPE" == "1" && "$CONFIG_EXPLICIT" == "0" ]]; then
+	PMU_CONFIG=2
+fi
+
+# For the page-fault positive control, sample every fault (period 1) by
+# default so enough reports flow for the ring/drain/match checks to be
+# meaningful on a short run.  Explicit --freq/--period override this.
+if [[ "$PMU_TYPE" == "1" && "$PMU_CONFIG" == "2" &&
+      "$FREQ_EXPLICIT" == "0" && "$PERIOD_EXPLICIT" == "0" ]]; then
+	FREQ=0
+	PERIOD=1
+fi
+
+# For cpu-clock (config 0), sample_period is a TIME in ns, so the
+# default period 256 would mean 4 MHz of callbacks per CPU.  Never
+# let an unguarded default hit that: fall back to a gentle 100 Hz.
+if [[ "$PMU_TYPE" == "1" && "$PMU_CONFIG" == "0" &&
+      "$FREQ_EXPLICIT" == "0" && "$PERIOD_EXPLICIT" == "0" ]]; then
+	FREQ=1
+	SAMPLE_FREQ=100
+fi
+
+# ---- Layer 0: Environment ----
+echo "=========================================="
+echo " DAMON Perf Observability — Layer-by-Layer Test"
+echo "=========================================="
+echo "PMU type: $PMU_TYPE  config: $PMU_CONFIG  freq: $FREQ  period: $PERIOD  timeout: ${TIMEOUT}s"
+if [[ -n "$TARGET_PID" ]]; then
+	echo "Target PID: $TARGET_PID (explicit)"
+else
+	echo "Target PID: workload process (started below)"
+fi
+echo ""
+
+mkdir -p "$RESULTS_DIR"
+
+echo "--- Layer 0: Environment ---"
+
+# Check kernel config
+CONFIG=""
+if [[ -f /proc/config.gz ]]; then
+	CONFIG=$(zcat /proc/config.gz)
+elif [[ -f /boot/config-$(uname -r) ]]; then
+	CONFIG=$(cat /boot/config-$(uname -r))
+else
+	die "Cannot read /proc/config.gz or /boot/config-$(uname -r)"
+fi
+
+for opt in DAMON DAMON_SYSFS DAMON_VADDR PERF_EVENTS DEBUG_FS TRACING \
+		TRACEPOINTS; do
+	if echo "$CONFIG" | grep -q "CONFIG_${opt}=y"; then
+		pass "CONFIG_${opt}=y"
+	else
+		fail "CONFIG_${opt}" "not enabled"
+	fi
+done
+
+# DAMON_PERF_OBSERVE is fatal — the test cannot run without it
+if echo "$CONFIG" | grep -q "CONFIG_DAMON_PERF_OBSERVE=y"; then
+	pass "CONFIG_DAMON_PERF_OBSERVE=y"
+else
+	die "kernel not built with CONFIG_DAMON_PERF_OBSERVE=y"
+fi
+
+# Check root
+[[ $(id -u) -eq 0 ]] || die "Must run as root"
+
+# Mount debugfs only when this test owns the mount, and undo it on exit.
+if ! mountpoint -q /sys/kernel/debug; then
+	mount -t debugfs none /sys/kernel/debug || die "Cannot mount debugfs"
+	MOUNTED_DEBUGFS=1
+fi
+[[ -d /sys/kernel/debug/damon ]] || die "debugfs damon/ not found"
+pass "debugfs mounted"
+
+# Check tracepoints
+for tp in damon_perf_sample damon_perf_ring_overflow damon_perf_report_missed damon_perf_drain; do
+	if [[ -d /sys/kernel/debug/tracing/events/damon/$tp ]]; then
+		pass "tracepoint $tp exists"
+	else
+		fail "tracepoint $tp" "not found"
+	fi
+done
+
+# Check debugfs file (perf_stats only; format is debug-only, not an ABI)
+if [[ -f /sys/kernel/debug/damon/perf_stats ]]; then
+	pass "debugfs perf_stats exists"
+else
+	fail "debugfs perf_stats" "not found"
+fi
+
+# ---- Guard: refuse to run if kdamonds already exist ----
+NR_KDAMONDS=$(cat $KD/nr_kdamonds 2>/dev/null || echo 0)
+if [[ "$NR_KDAMONDS" -gt 0 ]]; then
+	skip "runtime" "kdamonds exist (nr_kdamonds=$NR_KDAMONDS) — refusing"
+	KEEP_RESULTS=1
+	exit 0
+fi
+
+# ---- Generate memory pressure workload first: the default DAMON
+# ---- target must be the workload process itself, so the workload
+# ---- must be running before the target PID is written.
+echo ""
+echo "Starting memory workload for ${TIMEOUT}s..."
+if command -v stress-ng &>/dev/null; then
+	stress-ng --vm 2 --vm-bytes 256M --timeout "${TIMEOUT}s" &
+	STRESS_PID=$!
+elif command -v stress &>/dev/null; then
+	stress --vm 2 --vm-bytes 256M --timeout "${TIMEOUT}s" &
+	STRESS_PID=$!
+else
+	# Fallback: dd-based memory pressure
+	dd if=/dev/zero of=/dev/null bs=1M count=1024 &
+	STRESS_PID=$!
+fi
+
+# Resolve target PID: explicit --pid wins, otherwise the workload
+if [[ -z "$TARGET_PID" ]]; then
+	TARGET_PID=$STRESS_PID
+fi
+
+echo ""
+echo "--- Layer 1: Event Create ---"
+
+# Configure kdamond
+echo 1 > /sys/kernel/mm/damon/admin/kdamonds/nr_kdamonds
+CREATED_KDAMOND=1
+echo 1 > /sys/kernel/mm/damon/admin/kdamonds/0/contexts/nr_contexts
+echo 1 > /sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/nr_targets
+echo "$TARGET_PID" > /sys/kernel/mm/damon/admin/kdamonds/0/contexts/0/targets/0/pid_target
+
+# Disable page-fault-based access check
+echo 0 > $PE/nr_perf_events 2>/dev/null || true
+
+# Configure perf event
+echo 1 > $PE/nr_perf_events
+echo "$PMU_TYPE" > $PE/0/type
+echo "$PMU_CONFIG" > $PE/0/config
+
+if [[ "$FREQ" -eq 1 ]]; then
+	echo "$SAMPLE_FREQ" > $PE/0/sample_freq
+else
+	echo "$PERIOD" > $PE/0/sample_period
+fi
+echo "$FREQ" > $PE/0/freq
+
+# Take a dmesg snapshot before enabling the kdamond, so we can
+# detect error messages that appear during the test.
+DMESG_BEFORE="$RESULTS_DIR/dmesg_before.txt"
+dmesg > "$DMESG_BEFORE" 2>/dev/null || true
+# Snapshot counters BEFORE the workload window.  All counters are
+# cumulative since boot; every Analysis number below is a delta against
+# this snapshot.  Read before echo on so the window starts at zero.
+STATS_BASE="$RESULTS_DIR/perf_stats_base.txt"
+cat /sys/kernel/debug/damon/perf_stats > "$STATS_BASE" 2>/dev/null || true
+
+echo on > /sys/kernel/mm/damon/admin/kdamonds/0/state
+sleep 2
+
+DMESG_OUT="$RESULTS_DIR/dmesg_create.txt"
+DMESG_DELTA="$RESULTS_DIR/dmesg_delta.txt"
+dmesg > "$DMESG_OUT" 2>/dev/null || true
+# Delta: lines in $DMESG_OUT not already present in $DMESG_BEFORE
+awk 'NR==FNR { seen[$0]++ }
+     NR>FNR { if (seen[$0] > 0) seen[$0]--; else print }' \
+	"$DMESG_BEFORE" "$DMESG_OUT" > "$DMESG_DELTA" || true
+
+# Check state is on
+STATE_VAL=$(cat /sys/kernel/mm/damon/admin/kdamonds/0/state 2>/dev/null)
+if [[ "$STATE_VAL" == "on" ]]; then
+	pass "Kdamond state is on"
+else
+	fail "Kdamond state" "expected 'on', got '$STATE_VAL'"
+fi
+
+echo ""
+echo "--- Layer 2-3: Enable & Run (via per-CPU state) ---"
+
+# Parse the maximum per-CPU state from the debugfs output.
+# The per-CPU line format is:
+#   CPU%02d: st=<state> cb=... enq=... ...
+max_cpu_state() {
+	awk -F'st=' '/^  CPU/ {
+		split($2, a, " ")
+		s = a[1]
+		# Map state name to numeric rank
+		if (s == "ERROR")   v = 5
+		else if (s == "RUNNING") v = 4
+		else if (s == "ENABLED") v = 3
+		else if (s == "BOUND")   v = 2
+		else if (s == "CREATED") v = 1
+		else v = 0
+		if (v > max) max = v
+	} END { print max+0 }' "$1" 2>/dev/null
+}
+
+CPU_ST_BASE=$(max_cpu_state "$STATS_BASE")
+
+if [[ "$CPU_ST_BASE" -ge 1 ]]; then
+	pass "Event Created (max per-CPU state >= CREATED)"
+else
+	fail "Event Created" "max per-CPU state is $CPU_ST_BASE"
+fi
+
+if [[ "$CPU_ST_BASE" -ge 2 ]]; then
+	pass "Event Bound (max per-CPU state >= BOUND)"
+else
+	fail "Event Bound" "max per-CPU state is $CPU_ST_BASE"
+fi
+
+if [[ "$CPU_ST_BASE" -ge 3 ]]; then
+	pass "Event Enabled (max per-CPU state >= ENABLED)"
+else
+	fail "Event Enabled" "max per-CPU state is $CPU_ST_BASE"
+fi
+
+# Check dmesg delta for errors (vaddr.c pr_warn_ratelimited paths)
+if grep -q 'damon-perf.*failed\|event create failed' "$DMESG_DELTA" 2>/dev/null; then
+	fail "Perf event" "perf event creation failed (see dmesg delta)"
+else
+	pass "Perf event creation (no errors in dmesg delta)"
+fi
+
+echo ""
+echo "--- Layer 4: Sampling (Callback) ---"
+
+# Clear the trace buffer so trace.txt only contains records produced by
+# this run, then enable tracepoints for a bounded sampling window.
+echo 0 > $TRACE/tracing_on 2>/dev/null || true
+echo > $TRACE/trace 2>/dev/null || true
+echo 1 > $TRACE/events/damon/damon_perf_sample/enable
+echo 1 > $TRACE/events/damon/damon_perf_ring_overflow/enable
+echo 1 > $TRACE/events/damon/damon_perf_report_missed/enable
+echo 1 > $TRACE/events/damon/damon_perf_drain/enable
+echo 1 > $TRACE/tracing_on
+
+# Sampling window: keeps trace.txt bounded even on PMUs that sample at
+# tens of kHz.  Counters keep accumulating for the full TIMEOUT.
+sleep "$TRACE_WINDOW"
+echo 0 > $TRACE/tracing_on 2>/dev/null || true
+[[ "$TIMEOUT" -gt "$TRACE_WINDOW" ]] && sleep $((TIMEOUT - TRACE_WINDOW))
+
+kill $STRESS_PID 2>/dev/null || true
+wait $STRESS_PID 2>/dev/null || true
+STRESS_PID=""
+echo "Workload done."
+
+sleep 2  # let kdamond drain
+
+# Collect trace
+TRACE_OUT="$RESULTS_DIR/trace.txt"
+cat /sys/kernel/debug/tracing/trace > "$TRACE_OUT" 2>/dev/null || true
+TRACE_LINES=$(wc -l < "$TRACE_OUT" 2>/dev/null || echo 0)
+
+# Collect stats (re-read after workload)
+STATS_OUT="$RESULTS_DIR/perf_stats.txt"
+cat /sys/kernel/debug/damon/perf_stats > "$STATS_OUT" 2>/dev/null || true
+
+# ---- Analysis ----
+echo ""
+echo "--- Analysis ---"
+
+
+# Aggregate counter reads
+stat_of() {
+	awk -v k="$1" '$1==k {print $2}' "$2" 2>/dev/null
+}
+
+# Per-run delta between baseline and post-workload snapshots
+delta() {
+	local b a
+	b=$(stat_of "$1" "$STATS_BASE")
+	a=$(stat_of "$1" "$STATS_OUT")
+	[[ -z "$b" ]] && b=0
+	[[ -z "$a" ]] && a=0
+	echo $((a-b))
+}
+
+CALLBACK=$(delta callback)
+VALID=$(delta valid)
+ADDR_ZERO=$(delta addr_zero)
+KERNEL=$(delta kernel)
+ENQUEUE=$(delta enqueue)
+DEQUEUE=$(delta dequeue)
+OVERFLOW=$(delta overflow)
+MATCH=$(delta match)
+UPDATE=$(delta update)
+
+echo "  Callback delta this run: ${CALLBACK} (cumulative totals in perf_stats.txt)"
+
+if [[ "$CALLBACK" -gt 0 ]]; then
+	pass "Sampling: ${CALLBACK} callbacks received"
+else
+	fail "Sampling" "0 callbacks — PMU is not delivering samples to DAMON"
+fi
+
+echo "  Callback breakdown (delta): valid=${VALID} addr_zero=${ADDR_ZERO} kernel=${KERNEL}"
+
+# Verify RUNNING state: first callback advances per-CPU state to
+# RUNNING, which persists until the kdamond is stopped.
+CPU_ST_FINAL=$(max_cpu_state "$STATS_OUT")
+if [[ "$CALLBACK" -gt 0 && "$CPU_ST_FINAL" -ge 4 ]]; then
+	pass "Event Running (max per-CPU state >= RUNNING)"
+elif [[ "$CALLBACK" -eq 0 ]]; then
+	skip "Event Running" "no callbacks — state cannot advance past ENABLED"
+else
+	fail "Event Running" "callbacks > 0 but max state is $CPU_ST_FINAL"
+fi
+
+echo "  Ring: enqueue=${ENQUEUE} dequeue=${DEQUEUE} overflow=${OVERFLOW}"
+
+if [[ "$ENQUEUE" -gt 0 ]]; then
+	pass "Ring: enqueue > 0"
+	if [[ "$DEQUEUE" -gt 0 ]]; then
+		pass "Ring: dequeue > 0"
+	else
+		fail "Ring: dequeue" "enqueued but never dequeued"
+	fi
+else
+	skip "Ring" "no enqueues (no valid samples: addr=0 or no callbacks)"
+fi
+
+echo "  Match: match=${MATCH} update=${UPDATE}"
+
+if [[ "$MATCH" -gt 0 ]]; then
+	pass "Drain & Match: ${MATCH} matched"
+	if [[ "$UPDATE" -gt 0 ]]; then
+		pass "Update: ${UPDATE} region updates"
+	else
+		fail "Update" "matched but never updated"
+	fi
+else
+	skip "Match/Update" "no matches (no valid samples reached region matching)"
+fi
+
+# Drain tracepoint: kdamond fires damon_perf_drain whenever it drained
+# at least one report.  The drain tracepoint is only expected when the
+# ring actually produced entries.
+DRAIN_COUNT=$(grep -c "damon_perf_drain" "$TRACE_OUT" 2>/dev/null || true)
+DRAIN_COUNT=${DRAIN_COUNT:-0}
+echo "  Drain tracepoint: ${DRAIN_COUNT} records"
+if [[ "$MATCH" -gt 0 ]]; then
+	if [[ "$DRAIN_COUNT" -gt 0 ]]; then
+		pass "Drain tracepoint fired (${DRAIN_COUNT} records)"
+	else
+		fail "Drain tracepoint" "matches occurred but damon_perf_drain never fired"
+	fi
+else
+	skip "Drain tracepoint" "no drained reports to summarize"
+fi
+
+# Context verification
+if grep -q 'context=' "$TRACE_OUT" 2>/dev/null; then
+	NMI_COUNT=$(grep -c 'context=3' "$TRACE_OUT" 2>/dev/null || true)
+	PROC_COUNT=$(grep -c 'context=0' "$TRACE_OUT" 2>/dev/null || true)
+	NMI_COUNT=${NMI_COUNT:-0}
+	PROC_COUNT=${PROC_COUNT:-0}
+	echo "  Context: NMI=${NMI_COUNT} process=${PROC_COUNT}"
+	pass "Context field in trace output"
+else
+	skip "Context" "no trace output to analyze"
+fi
+
+# Show a few sample records as raw evidence (they are the per-sample
+# view of the pipeline; useful for PMU support evaluation).
+echo ""
+echo "  First damon_perf_sample record(s) this run:"
+if grep -q "damon_perf_sample" "$TRACE_OUT" 2>/dev/null; then
+	grep -m 3 "damon_perf_sample" "$TRACE_OUT" | sed 's/^/    /'
+else
+	echo "    (none — no samples captured in the ${TRACE_WINDOW}s window)"
+fi
+echo "  First damon_perf_drain record(s) this run:"
+if grep -q "damon_perf_drain" "$TRACE_OUT" 2>/dev/null; then
+	grep -m 3 "damon_perf_drain" "$TRACE_OUT" | sed 's/^/    /'
+else
+	echo "    (none)"
+fi
+
+# ---- PMU support verdict ----
+echo ""
+echo "--- PMU support verdict ---"
+if [[ "$CALLBACK" -eq 0 ]]; then
+	echo "  UNUSABLE: the PMU never delivered a sample to DAMON"
+	echo "  (e.g. ARM SPE requires an AUX ring buffer that kernel"
+	echo "  counters do not provide; see arm_spe_pmu.c)"
+elif [[ "$VALID" -eq 0 ]]; then
+	echo "  PLUMBING-ONLY: callbacks flow but no data addresses"
+	echo "  (address-less PMU, e.g. cpu-clock / task-clock)"
+elif [[ "$ENQUEUE" -gt 0 && "$MATCH" -gt 0 ]]; then
+	echo "  FULLY INTEGRATED: samples carry addresses and reach"
+	echo "  DAMON region matching/update"
+else
+	echo "  PARTIAL: callbacks with addresses, but the drain/match"
+	echo "  pipeline did not complete (see counters above)"
+fi
+
+# ---- Summary ----
+echo ""
+echo "=========================================="
+echo " SUMMARY: $PASSED passed, $FAILED failed, $SKIPPED skipped"
+echo "=========================================="
+echo ""
+echo "Results retained in: $RESULTS_DIR (set KEEP_RESULTS=0 to remove)"
+
+if [[ "$FAILED" -gt 0 ]]; then
+	echo "Overall: FAIL ($FAILED checks failed)"
+	exit 1
+else
+	echo "Overall: PASS"
+	exit 0
+fi
-- 
2.43.0

[RFC PATCH 6/7] Docs/mm/damon: document the perf observability framework

[Kunwu Chan]

From: Lian Wang <lianux.mm@gmail.com>

Add documentation for the DAMON perf observability framework,
covering the CONFIG_DAMON_PERF_OBSERVE Kconfig option, the
debugfs perf_stats interface, the tracepoints, and the per-CPU
pipeline counter model.  The debugfs format is explicitly marked
as unstable and must not be used by scripts.

Co-developed-by: Kunwu Chan <kunwu.chan@gmail.com>
Signed-off-by: Kunwu Chan <kunwu.chan@gmail.com>
Signed-off-by: Lian Wang <lianux.mm@gmail.com>
---
 Documentation/admin-guide/mm/damon/index.rst  |   1 +
 .../mm/damon/perf-observability.rst           | 210 ++++++++++++++++++
 2 files changed, 211 insertions(+)
 create mode 100644 Documentation/admin-guide/mm/damon/perf-observability.rst

diff --git a/Documentation/admin-guide/mm/damon/index.rst b/Documentation/admin-guide/mm/damon/index.rst
index 3ce3164480c7..623a5c312b69 100644
--- a/Documentation/admin-guide/mm/damon/index.rst
+++ b/Documentation/admin-guide/mm/damon/index.rst
@@ -15,3 +15,4 @@ access monitoring and access-aware system operations.
    reclaim
    lru_sort
    stat
+   perf-observability
diff --git a/Documentation/admin-guide/mm/damon/perf-observability.rst b/Documentation/admin-guide/mm/damon/perf-observability.rst
new file mode 100644
index 000000000000..3aa8185de314
--- /dev/null
+++ b/Documentation/admin-guide/mm/damon/perf-observability.rst
@@ -0,0 +1,210 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+=============================================
+DAMON Perf Event Observability Framework
+=============================================
+
+The DAMON perf event observability framework provides per-CPU counters and
+tracepoints for hardware-sampled access reports.  When DAMON is configured to
+use a hardware PMU (e.g. AMD IBS, Intel PEBS, or ARM SPE) instead of page-table
+walks, this framework exposes raw pipeline diagnostics so that every stage of
+the PMU-to-DAMON pipeline can be inspected.
+
+Counters are best-effort: individual ``u64`` writes are atomic on 64-bit
+platforms, but no cross-field consistency is guaranteed.  Do not build
+policy on snapshot reads.  For stable, structured diagnostics, use the
+tracepoints under ``/sys/kernel/debug/tracing/events/damon/``.
+
+Pipeline Stages
+===============
+
+::
+
+  PMU hardware → overflow_handler / AUX drain
+    → damon_report_access() → per-CPU SPSC ring
+    → kdamond drain → target match → region update
+
+  Layer 1:  Event Create   perf_event_create_kernel_counter()
+  Layer 2:  Event Bind     per-CPU PMU attachment
+  Layer 3:  Event Enable   perf_event_enable()
+  Layer 4:  Sampling       callback / AUX record received
+  Layer 5:  Ring           SPSC enqueue / dequeue / overflow
+  Layer 6:  Drain          kdamond consumes entries from ring
+  Layer 7:  Match & Update region access-rate update
+
+Each layer has a dedicated counter, and most layers have corresponding
+tracepoints.  Per-CPU event state (UNINIT → CREATED → BOUND → ENABLED →
+RUNNING) is recorded unconditionally and exposed via the debugfs
+perf_stats file.
+
+Overhead Control
+================
+
+Two levels of overhead control are provided:
+
+1. **Compile-time** — ``CONFIG_DAMON_PERF_OBSERVE``
+   When set to ``n``, all observe functions are compiled to static-inline
+   no-ops.  No code is generated and no runtime overhead exists.
+
+2. **Per-tracepoint on/off** — standard ftrace ``enable`` files
+   Individual tracepoints (``damon_perf_sample``, etc.) can be
+   enabled or disabled independently via
+   ``/sys/kernel/debug/tracing/events/damon/``.  Counter increments are
+   unconditional (cheap per-CPU ``inc``); tracepoint decisions are
+   guarded by the ftrace static key and are zero-overhead when disabled.
+
+When ``CONFIG_DAMON_PERF_OBSERVE=y``, per-CPU counters always increment.
+There is no runtime toggle for counters; compile-time is the sole gate.
+
+Debugfs Interface
+=================
+
+Mount debugfs::
+
+    # mount -t debugfs none /sys/kernel/debug
+
+One file is created under ``/sys/kernel/debug/damon/``:
+
+perf_stats
+----------
+
+**DEBUG ONLY — format may change without notice.**  Do not parse in
+scripts or tools.  For stable diagnostics, use the tracepoints.
+
+Read-only.  Aggregated counter table with all pipeline counters plus
+per-CPU breakdown::
+
+    # cat /sys/kernel/debug/damon/perf_stats
+      ┌──────────────┬──────────┐
+      │ Counter      │   Value  │
+      ├──────────────┼──────────┤
+      │ callback     │    233   │
+      │ valid        │      0   │
+      │ null         │      0   │
+      │ addr_zero    │    233   │
+      │ kernel       │      0   │
+      │ inv_phys     │      0   │
+      │ enqueue      │      0   │
+      │ dequeue      │      0   │
+      │ overflow     │      0   │
+      │ ring_peak    │      0   │
+      │ match        │      0   │
+      │ miss_tgid    │      0   │
+      │ miss_region  │      0   │
+      │ miss_bound   │      0   │
+      │ update       │      0   │
+      └──────────────┴──────────┘
+
+      Per-CPU (non-zero / non-UNINIT):
+      CPU00: st=BOUND   cb=44 enq=0 deq=0 ovf=0 match=0 miss=0 upd=0
+      ...
+
+The ``st=<state>`` column shows the per-CPU event state machine position
+(UNINIT, CREATED, BOUND, ENABLED, RUNNING, ERROR), derived from the
+lifecycle observe calls.  This allows verifying lifecycle progression
+without parsing dmesg.
+
+All counters are monotonic (cumulative since boot); userspace computes
+deltas between snapshots.
+
+Tracepoints
+===========
+
+Four tracepoints are defined::
+
+    damon_perf_sample
+    damon_perf_ring_overflow
+    damon_perf_report_missed
+    damon_perf_drain
+
+Enable via ftrace::
+
+    # echo 1 > /sys/kernel/debug/tracing/events/damon/damon_perf_sample/enable
+    # cat /sys/kernel/debug/tracing/trace_pipe
+
+Each ``damon_perf_sample`` record includes:
+
+  - ``addr``: the accessed virtual address (0 if the PMU did not populate)
+  - ``data_src``: PERF_MEM_* encoding (PMU-dependent)
+  - ``period``: sample period or frequency count
+  - ``cpu``: CPU that generated the sample
+  - ``reason``: 0=valid, 1=null-data, 2=addr-zero, 3=kernel-addr, 4=invalid-phys
+  - ``sample_flags``: what the PMU actually populated
+  - ``sample_type``: what DAMON requested
+  - ``context``: 0=process, 1=softirq, 2=hardirq, 3=NMI
+
+The ``context`` field is particularly useful for cross-PMU validation.
+For example, AMD IBS samples arrive in NMI context (context=3), while
+ARM SPE data from an AUX backend would arrive in process context (context=0).
+A mismatch between the expected and actual context is immediately visible.
+
+Selftest
+========
+
+A comprehensive automated test script is provided::
+
+    # cd tools/testing/selftests/damon
+    # sudo ./damon_perf_obs_test.sh --pmu arm_spe_0 --freq 0 --period 256
+
+The script performs a layer-by-layer validation:
+
+1. Checks kernel configuration (CONFIG_DAMON, CONFIG_DAMON_PERF_OBSERVE, etc.)
+2. Verifies PMU availability, tracepoints, and the debugfs perf_stats file
+3. Refuses to run if existing kdamonds are present (side-effect guard)
+4. Configures DAMON with the specified PMU via sysfs
+5. Runs a memory workload (stress-ng, stress, or dd fallback)
+6. Collects dmesg delta, trace output, and perf_stats
+7. Verifies per-CPU state progression and counter values
+
+Example output::
+
+    --- Layer 0: Environment ---
+    [PASS] CONFIG_DAMON_PERF_OBSERVE=y
+    [PASS] debugfs perf_stats exists
+
+    --- Layer 2-3: Enable & Run (via per-CPU state) ---
+    [PASS] Event Created (max per-CPU state >= CREATED)
+    [PASS] Event Bound (max per-CPU state >= BOUND)
+    [PASS] Event Enabled (max per-CPU state >= ENABLED)
+
+    --- Layer 4: Sampling (Callback) ---
+    [PASS] Sampling: 84532 callbacks received
+    Callback breakdown: valid=82103 addr_zero=0 kernel=2429
+
+    --- Layer 5: Ring ---
+    [PASS] Ring: enqueue > 0
+    [PASS] Ring: dequeue > 0
+    Ring: enqueue=82100 dequeue=81987 overflow=0
+
+    --- Layer 6: Drain & Match ---
+    [PASS] Drain & Match: 81987 matched
+    [PASS] Update: 81987 region updates
+
+Additional PMU examples::
+
+    # Software page-fault event (positive control):
+    sudo ./damon_perf_obs_test.sh --pmu software --freq 1 --sample-freq 100
+
+    # Any PMU by type number:
+    sudo ./damon_perf_obs_test.sh --pmu-type 38 --freq 0 --period 256
+
+Kernel Configuration
+====================
+
+Required for observability::
+
+    CONFIG_DAMON=y
+    CONFIG_DAMON_SYSFS=y
+    CONFIG_DAMON_VADDR=y
+    CONFIG_PERF_EVENTS=y
+    CONFIG_DEBUG_FS=y
+    CONFIG_TRACING=y
+    CONFIG_TRACEPOINTS=y
+
+Optional (enables observability framework)::
+
+    CONFIG_DAMON_PERF_OBSERVE=y
+
+When ``CONFIG_DAMON_PERF_OBSERVE=n``, ``/sys/kernel/debug/damon/perf_stats``
+is not created, tracepoints are not registered, and all observe functions
+are compiled to empty static inlines with zero overhead.
-- 
2.43.0

[RFC PATCH 7/7] mm/damon/perf: add CONFIG_DAMON_PERF_DEBUG and pipeline health check

[Kunwu Chan]

From: Lian Wang <lianux.mm@gmail.com>

Add CONFIG_DAMON_PERF_DEBUG as an optional Kconfig option that
enables pr_debug() output for the observability pipeline via the
damon_perf_dbg() macro.  Default off, zero overhead when disabled.

Add a pipeline health check to the observability selftest that
diagnoses which stage is broken when callbacks are zero, using the
existing per-CPU state and counter deltas.

Co-developed-by: Kunwu Chan <kunwu.chan@gmail.com>
Signed-off-by: Kunwu Chan <kunwu.chan@gmail.com>
Signed-off-by: Lian Wang <lianux.mm@gmail.com>
---
 mm/damon/Kconfig                               | 16 ++++++++++++++++
 mm/damon/perf/debugfs.c                        |  5 ++++-
 mm/damon/perf/perf.h                           |  7 +++++++
 mm/damon/perf/stats.c                          | 15 ++++++++++++++-
 .../selftests/damon/damon_perf_obs_test.sh     | 18 +++++++++++++++++-
 5 files changed, 58 insertions(+), 3 deletions(-)

diff --git a/mm/damon/Kconfig b/mm/damon/Kconfig
index 9f811510760f..35ec4d12c5b8 100644
--- a/mm/damon/Kconfig
+++ b/mm/damon/Kconfig
@@ -148,4 +148,20 @@ config DAMON_PERF_OBSERVE
 	  static-inline no-ops with zero runtime overhead.
 
 	  If unsure, say N.
+
+config DAMON_PERF_DEBUG
+	bool "DAMON Perf verbose debugging output"
+	depends on DAMON_PERF_OBSERVE
+	help
+	  Enable verbose per-event and per-drain pr_debug() output
+	  for the DAMON perf observability pipeline.  When enabled,
+	  key lifecycle transitions and sampling events are logged
+	  at KERN_DEBUG level, visible via dynamic_debug or when
+	  DEBUG is defined at compile time.
+
+	  This adds dmesg noise and should only be enabled for
+	  development or troubleshooting.
+
+	  If unsure, say N.
+
 endmenu
diff --git a/mm/damon/perf/debugfs.c b/mm/damon/perf/debugfs.c
index c54dd7644ac3..48f3d23c2dbc 100644
--- a/mm/damon/perf/debugfs.c
+++ b/mm/damon/perf/debugfs.c
@@ -132,11 +132,14 @@ int damon_perf_debugfs_init(void)
 	damon_debugfs_dir = debugfs_create_dir("damon", NULL);
 	if (damon_debugfs_dir == ERR_PTR(-EEXIST))
 		damon_debugfs_dir = debugfs_lookup("damon", NULL);
-	if (IS_ERR(damon_debugfs_dir))
+	if (IS_ERR(damon_debugfs_dir)) {
+		damon_perf_dbg("debugfs init failed: %ld\n", PTR_ERR(damon_debugfs_dir));
 		return PTR_ERR(damon_debugfs_dir);
+	}
 
 	debugfs_create_file("perf_stats", 0400, damon_debugfs_dir,
 			NULL, &perf_stats_fops);
 
+	damon_perf_dbg("debugfs init ok\n");
 	return 0;
 }
diff --git a/mm/damon/perf/perf.h b/mm/damon/perf/perf.h
index 78e23d436336..908c06f2e3db 100644
--- a/mm/damon/perf/perf.h
+++ b/mm/damon/perf/perf.h
@@ -18,6 +18,13 @@
 struct perf_event;
 #include <linux/types.h>
 
+#ifdef CONFIG_DAMON_PERF_DEBUG
+#define damon_perf_dbg(fmt, ...) \
+	pr_debug("damon-perf: " fmt, ##__VA_ARGS__)
+#else
+#define damon_perf_dbg(fmt, ...) no_printk(fmt, ##__VA_ARGS__)
+#endif
+
 struct damon_perf_event;
 
 /*
diff --git a/mm/damon/perf/stats.c b/mm/damon/perf/stats.c
index ae5b0037a31d..e2c1e5764d41 100644
--- a/mm/damon/perf/stats.c
+++ b/mm/damon/perf/stats.c
@@ -67,6 +67,7 @@ void damon_perf_observe_event_created(struct damon_perf_event *event, int cpu)
 			return;
 	}
 	*per_cpu_ptr(event->cpu_state, cpu) = DAMON_PERF_STATE_CREATED;
+	damon_perf_dbg("cpu %d: event created\n", cpu);
 	damon_perf_cpu_state_advance(cpu, DAMON_PERF_STATE_CREATED);
 }
 
@@ -76,6 +77,7 @@ void damon_perf_observe_event_bound(struct damon_perf_event *event,
 	if (event->cpu_state)
 		*per_cpu_ptr(event->cpu_state, cpu) = DAMON_PERF_STATE_BOUND;
 	damon_perf_cpu_state_advance(cpu, DAMON_PERF_STATE_BOUND);
+	damon_perf_dbg("cpu %d: event bound\n", cpu);
 }
 
 void damon_perf_observe_event_enabled(struct damon_perf_event *event,
@@ -84,12 +86,14 @@ void damon_perf_observe_event_enabled(struct damon_perf_event *event,
 	if (event->cpu_state)
 		*per_cpu_ptr(event->cpu_state, cpu) = DAMON_PERF_STATE_ENABLED;
 	damon_perf_cpu_state_advance(cpu, DAMON_PERF_STATE_ENABLED);
+	damon_perf_dbg("cpu %d: event enabled\n", cpu);
 }
 
 void damon_perf_observe_event_disabled(struct damon_perf_event *event,
 		int cpu, int state)
 {
 	/* State unchanged: the event may be re-enabled later. */
+	damon_perf_dbg("cpu %d: event disabled\n", cpu);
 }
 
 void damon_perf_observe_event_destroyed(struct damon_perf_event *event, int cpu)
@@ -104,6 +108,7 @@ void damon_perf_observe_event_destroyed(struct damon_perf_event *event, int cpu)
 	 */
 	if (event->cpu_state)
 		*per_cpu_ptr(event->cpu_state, cpu) = DAMON_PERF_STATE_UNINIT;
+	damon_perf_dbg("cpu %d: event destroyed\n", cpu);
 }
 
 void damon_perf_observe_event_free(struct damon_perf_event *event)
@@ -111,6 +116,7 @@ void damon_perf_observe_event_free(struct damon_perf_event *event)
 	if (event->cpu_state) {
 		free_percpu(event->cpu_state);
 		event->cpu_state = NULL;
+		damon_perf_dbg("event freed\n");
 	}
 }
 
@@ -231,6 +237,7 @@ void damon_perf_observe_drain(unsigned int total, unsigned int matched)
 {
 	if (trace_damon_perf_drain_enabled())
 		trace_damon_perf_drain(total, matched);
+	damon_perf_dbg("drain: total=%u matched=%u\n", total, matched);
 }
 
 /*
@@ -291,5 +298,11 @@ void damon_perf_stats_aggregate(struct damon_perf_stats *dst)
 
 int damon_perf_framework_init(void)
 {
-	return damon_perf_debugfs_init();
+	int ret = damon_perf_debugfs_init();
+
+	if (ret)
+		damon_perf_dbg("framework init failed: %d\n", ret);
+	else
+		damon_perf_dbg("framework init ok\n");
+	return ret;
 }
diff --git a/tools/testing/selftests/damon/damon_perf_obs_test.sh b/tools/testing/selftests/damon/damon_perf_obs_test.sh
index 4c4074cdd191..cd567c151ae6 100755
--- a/tools/testing/selftests/damon/damon_perf_obs_test.sh
+++ b/tools/testing/selftests/damon/damon_perf_obs_test.sh
@@ -344,7 +344,7 @@ max_cpu_state() {
 	} END { print max+0 }' "$1" 2>/dev/null
 }
 
-CPU_ST_BASE=$(max_cpu_state "$STATS_BASE")
+CPU_ST_BASE=$(max_cpu_state /sys/kernel/debug/damon/perf_stats)
 
 if [[ "$CPU_ST_BASE" -ge 1 ]]; then
 	pass "Event Created (max per-CPU state >= CREATED)"
@@ -431,6 +431,22 @@ VALID=$(delta valid)
 ADDR_ZERO=$(delta addr_zero)
 KERNEL=$(delta kernel)
 ENQUEUE=$(delta enqueue)
+# Pipeline health check: diagnose which stage is broken when
+# callbacks are zero, using the existing per-CPU state and
+# counter deltas.  This is a best-effort diagnostic, not a
+# substitute for detailed per-backend debugging.
+if [[ "$CALLBACK" -eq 0 ]]; then
+	CPU_ST_BASE_VAL=$(max_cpu_state /sys/kernel/debug/damon/perf_stats)
+	if [[ "$CPU_ST_BASE_VAL" -le 1 ]]; then
+		echo "  Pipeline diagnosis: event not created or bound (state=$CPU_ST_BASE_VAL)"
+	elif [[ "$CPU_ST_BASE_VAL" -eq 2 ]]; then
+		echo "  Pipeline diagnosis: event bound but not enabled (state=BOUND)"
+	elif [[ "$ENQUEUE" -eq 0 ]]; then
+		echo "  Pipeline diagnosis: PMU not producing data or AUX pipeline broken"
+	else
+		echo "  Pipeline diagnosis: samples enqueued but none valid"
+	fi
+fi
 DEQUEUE=$(delta dequeue)
 OVERFLOW=$(delta overflow)
 MATCH=$(delta match)
-- 
2.43.0