[RFC PATCH 0/4] mm/damon/perf: add ARM SPE AUX backend

[RFC PATCH 0/4] mm/damon/perf: add ARM SPE AUX backend

[Kunwu Chan]

From: Kunwu Chan <kunwu.chan@gmail.com>

This series adds an AUX trace-buffer backend to the DAMON perf
observability framework, enabling ARM SPE (Statistical Profiling
Extension) to deliver hardware-sampled access reports into DAMON's
existing SPSC report ring.

Patch 2 touches drivers/perf/arm_spe_pmu.c to expose the PMU matcher.
This change is tightly coupled with the DAMON AUX backend.  ARM SPE
PMU driver maintainers only need to review patch 2.

This series is based on the Ravi's hardware-sampled access reports 
branch [1], and depends on the perf AUX kernel-consumer RFC series [2].
The dependencies is not upstream yet, so this series remains RFC.

Why a dedicated AUX backend
---------------------------
ARM SPE and similar PMUs do not deliver samples through the standard
perf overflow callback.  Instead, they write trace data into an AUX
buffer managed by the perf core.  The AUX buffer is consumed from
process context (kdamond) rather than from NMI, so the existing
overflow-callback path in DAMON cannot be reused.

The backend model adds three operations to the damon_perf_event
lifecycle: init (allocate the AUX buffer), arm (position the consumer
cursor), and drain (parse the SPE packet stream and publish reports).
The backend is selected at event creation time by matching the PMU
object via its event_init callback.

What the series adds
--------------------
Patch 1 introduces the backend operations table, the backend state
fields in damon_perf_event, and the Kconfig options.  ARM SPE must be
built into the kernel; the KUnit test option is separate.

Patch 2 is the ARM SPE backend.  It owns a per-CPU AUX buffer,
decodes the SPE packet stream (the same encoding that perf's
userspace arm-spe-decoder handles), resolves sampled
CONTEXTIDR_EL1 values to tgids under RCU, and publishes synthesized
access reports.  The drain runs before the SPSC ring consumer on
each monitoring interval, and a final drain after event disable
ensures no records are lost.  The backend also modifies the ARM SPE
PMU driver to expose arm_spe_pmu_match() for PMU object matching.

Patch 3 adds byte-exact KUnit tests for the SPE record parser.  The
tests cover load and store records, timestamp terminators, multiple
records in one window, PAD and ALIGNMENT packets at both odd and
aligned positions, extended addresses, bad-packet resynchronization,
truncated records, and records without a virtual address.  A split-
record test verifies that a record spanning two AUX snapshots leaves
the tail unchanged until the terminating packet arrives.

Patch 4 adds a DAMON selftest for the AUX backend.  It checks the
backend integration, the required AUX-before-ring lifecycle ordering,
runs the parser KUnit suite through debugfs, and on an ARM SPE system
creates a live DAMON session with a controlled userspace target,
verifying positive end-to-end pipeline counters and final
enqueue/dequeue closure.

Testing
=======
Tested on Kunpeng 920 (256 CPUs, ARM SPE, kernel 7.1.0-rc5-mm-new-damon+):

  ARM SPE KUnit (spe_parse_one_record):
    16 passed, 0 failed, 0 skipped
    (store record, load record with timestamp, two records, pad-wrapped,
     alignment at odd/aligned positions, bad packet resync, record without
     address, events/source/counter, truncated record retained, truncated
     packet retained, extended address, invalid extended header, pad-only,
     empty window, split record across two snapshots)

  AUX selftest (damon_perf_aux_test.sh arm_spe_0):
    SUMMARY: 52 passed, 0 failed, 0 skipped (Overall PASS)
    callback=58318, valid=58318, enqueue=43390, dequeue=43390,
    match=34049, update=564
    final ring closure: enqueue=43390 dequeue=43390

Known limitations
=================
- AUX_BACKEND_MAX=4: the AUX backend registration table holds at most 4
  backend types.  This is a framework limit, not a per-PMU-instance cap.

- ARM SPE must be built into the kernel (ARM_SPE_PMU=y), not as a
  module, because the backend calls arm_spe_pmu_match() at initcall
  time.

- The AUX buffer is 2 pages (8 KiB).  The SPE hardware pauses when the
  non-overwrite ring is full; the drain frees space and re-enables
  the event.

- CONTEXTIDR_EL1 carries the sampled task's pid, resolved to tgid
  under RCU.  Records without a valid tgid are dropped.

- This series requires the perf AUX kernel API patches [2] to be applied
  first.  These patches are not yet upstream and must be reviewed by the
  perf subsystem maintainers.

References
==========
Link [1]: https://github.com/damonitor/linux.git
          branch: ravi_hw_sampled_access_reports_rfc_v1
          commit: fd968106e5bf

Link [2]: https://lore.kernel.org/all/20260814144927.489172-1-kunwu.chan@linux.dev/
          perf AUX kernel API PATCH Series:
          - perf/core: add AUX buffer ownership for kernel events
          - perf/core: add AUX ring accessors for kernel consumers
          - perf/core: add KUnit tests for AUX kernel-consumer API
          - selftests/perf_events: add userspace AUX regression test
          - selftests/perf_events: add AUX kernel API selftest script


Kunwu Chan (2):
  mm/damon/perf: introduce AUX backend interface and Kconfig
  mm/damon/perf: add AUX trace-buffer PMU backend for ARM SPE

Lian Wang (ProcessMission) (2):
  mm/damon/perf: add KUnit tests for the SPE record parser
  selftests/damon: add DAMON perf AUX backend test

 drivers/perf/arm_spe_pmu.c                    |   8 +
 include/linux/damon.h                         |   5 +
 include/linux/perf/arm_spe_pmu.h              |  11 +
 mm/damon/Kconfig                              |  28 +
 mm/damon/core.c                               |  22 +-
 mm/damon/ops-common.h                         |   6 +-
 mm/damon/perf/Makefile                        |   5 +
 mm/damon/perf/aux_backend.c                   | 121 +++++
 mm/damon/perf/aux_backend.h                   |  72 +++
 mm/damon/perf/spe_backend.c                   | 479 ++++++++++++++++++
 mm/damon/perf/spe_parser.h                    | 109 ++++
 mm/damon/perf/spe_parser_test.c               | 365 +++++++++++++
 mm/damon/vaddr.c                              |  68 ++-
 tools/testing/selftests/damon/Makefile        |   1 +
 .../selftests/damon/damon_perf_aux_test.sh    | 414 +++++++++++++++
 15 files changed, 1704 insertions(+), 10 deletions(-)
 create mode 100644 include/linux/perf/arm_spe_pmu.h
 create mode 100644 mm/damon/perf/aux_backend.c
 create mode 100644 mm/damon/perf/aux_backend.h
 create mode 100644 mm/damon/perf/spe_backend.c
 create mode 100644 mm/damon/perf/spe_parser.h
 create mode 100644 mm/damon/perf/spe_parser_test.c
 create mode 100755 tools/testing/selftests/damon/damon_perf_aux_test.sh

-- 
2.43.0

[RFC PATCH 1/4] mm/damon/perf: introduce AUX backend interface and Kconfig

[Kunwu Chan]

From: Kunwu Chan <kunwu.chan@gmail.com>

Add the backend operations used by AUX trace-buffer PMUs, backend state
to damon_perf_event, and stubs for configurations without AUX support.

Add CONFIG_DAMON_PERF_AUX for the ARM SPE transport and a separate
CONFIG_DAMON_PERF_SPE_KUNIT_TEST option.  The current transport requires
ARM SPE to be built into the kernel; it remains independent of the
optional observability counters and tracepoints.

Co-developed-by: Lian Wang (ProcessMission) <lianux.mm@gmail.com>
Signed-off-by: Lian Wang (ProcessMission) <lianux.mm@gmail.com>
Signed-off-by: Kunwu Chan <kunwu.chan@gmail.com>
---
 include/linux/damon.h       |  5 +++
 mm/damon/Kconfig            | 28 +++++++++++++++
 mm/damon/perf/aux_backend.h | 72 +++++++++++++++++++++++++++++++++++++
 3 files changed, 105 insertions(+)
 create mode 100644 mm/damon/perf/aux_backend.h

diff --git a/include/linux/damon.h b/include/linux/damon.h
index c191c065b0e4..a27c5f6c459b 100644
--- a/include/linux/damon.h
+++ b/include/linux/damon.h
@@ -8,6 +8,7 @@
 #ifndef _DAMON_H_
 #define _DAMON_H_
 
+#include <linux/cpumask.h>
 #include <linux/math64.h>
 #include <linux/memcontrol.h>
 #include <linux/mutex.h>
@@ -127,6 +128,7 @@ struct damon_target {
 enum damon_report_source {
 	DAMON_REPORT_SRC_PERF_OVERFLOW = 0,  /* overflow_handler (IBS, PEBS) */
 	DAMON_REPORT_SRC_PAGE_FAULT,         /* damon_report_page_fault() */
+	DAMON_REPORT_SRC_PERF_AUX,           /* AUX trace-buffer backend */
 };
 
 /**
@@ -538,6 +540,7 @@ struct damos_filter {
 struct damon_ctx;
 struct damon_target_lookup;
 struct damos;
+struct damon_perf_backend_ops;
 
 /**
  * struct damos_walk_control - Control damos_walk().
@@ -1056,6 +1059,8 @@ struct damon_perf_event_attr {
 struct damon_perf_event {
 	struct damon_perf_event_attr attr;
 	void *priv;
+	const struct damon_perf_backend_ops *ops;
+	cpumask_t aux_cpumask;
 	struct list_head list;
 	struct hlist_node hlist_node;
 	bool init_complete;
diff --git a/mm/damon/Kconfig b/mm/damon/Kconfig
index 9fac286df124..e9fb62ec186f 100644
--- a/mm/damon/Kconfig
+++ b/mm/damon/Kconfig
@@ -149,4 +149,32 @@ config DAMON_PERF_OBSERVE
 
 	  If unsure, say N.
 
+
+config DAMON_PERF_AUX
+	bool "DAMON AUX trace-buffer backend support"
+	depends on DAMON
+	depends on PERF_EVENTS
+	depends on ARM_SPE_PMU=y
+	default n
+	help
+	  Enable the functional AUX trace-buffer transport for DAMON perf
+	  events.  This provides backend selection and drain scheduling for
+	  ARM SPE, which does not deliver samples through an overflow
+	  callback.  ARM SPE must be built into the kernel.
+
+	  This transport is independent of CONFIG_DAMON_PERF_OBSERVE.
+
+	  If unsure, say N.
+
+config DAMON_PERF_SPE_KUNIT_TEST
+	bool "Test the DAMON perf SPE parser" if !KUNIT_ALL_TESTS
+	depends on DAMON_PERF_AUX && KUNIT=y
+	default KUNIT_ALL_TESTS
+	help
+	  Test the ARM SPE parser with byte-exact packet streams.  The cases
+	  cover valid records, truncation, alignment, error resynchronization
+	  and consumed-length accounting.  Results can be exposed through
+	  debugfs when CONFIG_KUNIT_DEBUGFS is enabled.
+
+	  If unsure, say N.
 endmenu
diff --git a/mm/damon/perf/aux_backend.h b/mm/damon/perf/aux_backend.h
new file mode 100644
index 000000000000..70e6ac6771bc
--- /dev/null
+++ b/mm/damon/perf/aux_backend.h
@@ -0,0 +1,72 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _DAMON_PERF_AUX_BACKEND_H
+#define _DAMON_PERF_AUX_BACKEND_H
+
+#include <linux/bits.h>
+#include <linux/errno.h>
+#include <linux/perf_event.h>
+#include <linux/types.h>
+
+struct damon_ctx;
+struct damon_perf_event;
+
+/**
+ * struct damon_perf_backend_ops - PMU-specific AUX backend operations
+ * @name: Human-readable backend name.
+ * @flags: Bitmask of DAMON_PERF_BACKEND_* flags.
+ * @match_pmu: Return true when this backend claims @perf_event.
+ * @init: Allocate per-event, per-CPU resources.
+ * @cleanup: Release resources initialized for one CPU.
+ * @arm: Prepare the AUX producer before perf_event_enable().
+ * @disarm: Quiesce backend state after perf_event_disable().
+ * @drain: Parse pending AUX data into DAMON access reports.
+ *
+ * All callbacks run in process context.  The caller serializes resource
+ * lifetime against CPU hotplug and invokes @drain only for CPUs recorded
+ * in damon_perf_event::aux_cpumask.
+ */
+struct damon_perf_backend_ops {
+	const char *name;
+	u32 flags;
+	bool (*match_pmu)(struct perf_event *perf_event);
+	int (*init)(struct damon_perf_event *event, int cpu,
+		    struct perf_event *perf_event);
+	void (*cleanup)(struct damon_perf_event *event, int cpu);
+	int (*arm)(struct damon_perf_event *event, int cpu);
+	void (*disarm)(struct damon_perf_event *event, int cpu);
+	unsigned int (*drain)(struct damon_perf_event *event, int cpu);
+};
+
+#define DAMON_PERF_BACKEND_AUX	BIT(0)
+
+#ifdef CONFIG_DAMON_PERF_AUX
+void damon_perf_aux_drain(struct damon_ctx *ctx);
+int damon_perf_aux_register_backend(const struct damon_perf_backend_ops *ops);
+const struct damon_perf_backend_ops *
+damon_perf_aux_find_backend(struct perf_event *perf_event);
+void damon_perf_aux_select(struct damon_perf_event *event,
+			   struct perf_event *perf_event);
+#else
+static inline void damon_perf_aux_drain(struct damon_ctx *ctx)
+{
+}
+
+static inline int
+damon_perf_aux_register_backend(const struct damon_perf_backend_ops *ops)
+{
+	return -EOPNOTSUPP;
+}
+
+static inline const struct damon_perf_backend_ops *
+damon_perf_aux_find_backend(struct perf_event *perf_event)
+{
+	return NULL;
+}
+
+static inline void damon_perf_aux_select(struct damon_perf_event *event,
+					 struct perf_event *perf_event)
+{
+}
+#endif
+
+#endif /* _DAMON_PERF_AUX_BACKEND_H */
-- 
2.43.0

[RFC PATCH 2/4] mm/damon/perf: add AUX trace-buffer PMU backend for ARM SPE

[Kunwu Chan]

From: Kunwu Chan <kunwu.chan@gmail.com>

Add an ARM SPE backend that owns a per-CPU AUX buffer, drains its packet
stream from kdamond process context, and publishes synthesized access
reports through DAMON's existing SPSC report ring.

Drain AUX before consuming the SPSC ring on each monitoring interval.
After disabling the events, run the same AUX-then-ring sequence once more
so records finalized by perf_event_disable() are not lost.

Decode CONTEXTIDR_EL1 as a sampled tid and resolve its tgid under RCU.
Drop records whose context is missing or stale instead of assigning them
to an arbitrary DAMON target.  Retain incomplete trailing records across
drains, guarantee progress for aligned ALIGNMENT packets, and reject
unsupported extended packet classes.

Match ARM SPE events by the PMU event_init callback, avoiding a fixed-size
PMU registry and its device-lifetime problems.  Roll back partially armed
CPU events on failure and release per-CPU AUX state on all error paths.

This depends on the perf AUX kernel-consumer API series.

Co-developed-by: Lian Wang (ProcessMission) <lianux.mm@gmail.com>
Signed-off-by: Lian Wang (ProcessMission) <lianux.mm@gmail.com>
Signed-off-by: Kunwu Chan <kunwu.chan@gmail.com>
---
 drivers/perf/arm_spe_pmu.c       |   8 +
 include/linux/perf/arm_spe_pmu.h |  11 +
 mm/damon/core.c                  |  22 +-
 mm/damon/ops-common.h            |   6 +-
 mm/damon/perf/Makefile           |   5 +
 mm/damon/perf/aux_backend.c      | 121 ++++++++
 mm/damon/perf/spe_backend.c      | 479 +++++++++++++++++++++++++++++++
 mm/damon/perf/spe_parser.h       | 109 +++++++
 mm/damon/vaddr.c                 |  68 ++++-
 9 files changed, 819 insertions(+), 10 deletions(-)
 create mode 100644 include/linux/perf/arm_spe_pmu.h
 create mode 100644 mm/damon/perf/aux_backend.c
 create mode 100644 mm/damon/perf/spe_backend.c
 create mode 100644 mm/damon/perf/spe_parser.h

diff --git a/drivers/perf/arm_spe_pmu.c b/drivers/perf/arm_spe_pmu.c
index dbd0da111639..50430342475c 100644
--- a/drivers/perf/arm_spe_pmu.c
+++ b/drivers/perf/arm_spe_pmu.c
@@ -27,6 +27,8 @@
 #include <linux/module.h>
 #include <linux/of.h>
 #include <linux/perf_event.h>
+#include <linux/perf/arm_spe_pmu.h>
+
 #include <linux/perf/arm_pmu.h>
 #include <linux/platform_device.h>
 #include <linux/printk.h>
@@ -1097,6 +1099,12 @@ static int arm_spe_pmu_perf_init(struct arm_spe_pmu *spe_pmu)
 	return perf_pmu_register(&spe_pmu->pmu, name, -1);
 }
 
+bool arm_spe_pmu_match(struct perf_event *perf_event)
+{
+	return perf_event->pmu->event_init == arm_spe_pmu_event_init;
+}
+EXPORT_SYMBOL_GPL(arm_spe_pmu_match);
+
 static void arm_spe_pmu_perf_destroy(struct arm_spe_pmu *spe_pmu)
 {
 	perf_pmu_unregister(&spe_pmu->pmu);
diff --git a/include/linux/perf/arm_spe_pmu.h b/include/linux/perf/arm_spe_pmu.h
new file mode 100644
index 000000000000..da5fb3d7b080
--- /dev/null
+++ b/include/linux/perf/arm_spe_pmu.h
@@ -0,0 +1,11 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LINUX_PERF_ARM_SPE_PMU_H
+#define _LINUX_PERF_ARM_SPE_PMU_H
+
+#include <linux/types.h>
+
+struct perf_event;
+
+bool arm_spe_pmu_match(struct perf_event *perf_event);
+
+#endif /* _LINUX_PERF_ARM_SPE_PMU_H */
diff --git a/mm/damon/core.c b/mm/damon/core.c
index 92b21f9484c9..aded25624e2d 100644
--- a/mm/damon/core.c
+++ b/mm/damon/core.c
@@ -22,6 +22,7 @@
 /* for damon_get_folio() used by node eligible memory metrics */
 #include "ops-common.h"
 #include "perf/perf.h"
+#include "perf/aux_backend.h"
 
 #define CREATE_TRACE_POINTS
 #include <trace/events/damon.h>
@@ -1763,8 +1764,14 @@ static int damon_commit_perf_events(struct damon_ctx *dst,
 			 * the kdamond runs.  Arm now if we are committing into a
 			 * running ctx whose substrate is already armed.
 			 */
-			if (dst->perf_events_active)
-				damon_perf_event_arm(new_event);
+			if (dst->perf_events_active) {
+				err = damon_perf_event_arm(new_event);
+				if (err) {
+					damon_perf_cleanup(dst, new_event);
+					kfree(new_event);
+					goto out;
+				}
+			}
 		}
 		list_add_tail(&new_event->list, &dst->perf_events);
 	}
@@ -4077,6 +4084,9 @@ static unsigned int kdamond_check_reported_accesses(struct damon_ctx *ctx)
 	unsigned int i;
 	unsigned int total_reports = 0, matched_reports = 0;
 
+	/* AUX backends publish into the same ring consumed below. */
+	damon_perf_aux_drain(ctx);
+
 	tbl = damon_build_target_lookup(ctx, &nr_targets);
 	if (!tbl) {
 		pr_warn_ratelimited(
@@ -4194,8 +4204,10 @@ static int kdamond_fn(void *data)
 		struct damon_perf_event *event;
 
 		WRITE_ONCE(ctx->perf_events_active, true);
-		list_for_each_entry(event, &ctx->perf_events, list)
-			damon_perf_event_arm(event);
+		list_for_each_entry(event, &ctx->perf_events, list) {
+			if (damon_perf_event_arm(event))
+				goto done;
+		}
 	}
 
 	if (ctx->ops.init)
@@ -4323,7 +4335,7 @@ static int kdamond_fn(void *data)
 		WRITE_ONCE(ctx->perf_events_active, false);
 		list_for_each_entry(event, &ctx->perf_events, list)
 			damon_perf_event_disarm(event);
-		/* Drain any in-flight reports queued before disarm took effect. */
+		/* Final AUX drain and ring drain after perf_event_disable(). */
 		kdamond_check_reported_accesses(ctx);
 	}
 	damon_destroy_targets(ctx);
diff --git a/mm/damon/ops-common.h b/mm/damon/ops-common.h
index 35da400a67ec..7142ffa8878a 100644
--- a/mm/damon/ops-common.h
+++ b/mm/damon/ops-common.h
@@ -33,11 +33,12 @@ bool damos_ops_has_filter(struct damos *s);
  */
 struct damon_perf {
 	struct perf_event * __percpu *event;
+	void			*aux_priv;
 };
 
 int damon_perf_init(struct damon_ctx *ctx, struct damon_perf_event *event);
 void damon_perf_cleanup(struct damon_ctx *ctx, struct damon_perf_event *event);
-void damon_perf_event_arm(struct damon_perf_event *event);
+int damon_perf_event_arm(struct damon_perf_event *event);
 void damon_perf_event_disarm(struct damon_perf_event *event);
 
 #else /* !CONFIG_PERF_EVENTS */
@@ -53,8 +54,9 @@ static inline void damon_perf_cleanup(struct damon_ctx *ctx,
 {
 }
 
-static inline void damon_perf_event_arm(struct damon_perf_event *event)
+static inline int damon_perf_event_arm(struct damon_perf_event *event)
 {
+	return 0;
 }
 
 static inline void damon_perf_event_disarm(struct damon_perf_event *event)
diff --git a/mm/damon/perf/Makefile b/mm/damon/perf/Makefile
index 150cbaa875fa..76857e443234 100644
--- a/mm/damon/perf/Makefile
+++ b/mm/damon/perf/Makefile
@@ -3,3 +3,8 @@
 # Observability: per-CPU counters, tracepoints, debugfs perf_stats
 obj-$(CONFIG_DAMON_PERF_OBSERVE)	+= damon-perf.o
 damon-perf-objs			:= stats.o debugfs.o
+
+obj-$(CONFIG_DAMON_PERF_AUX)		+= damon-perf-aux.o
+damon-perf-aux-objs			:= aux_backend.o spe_backend.o
+damon-perf-aux-$(CONFIG_DAMON_PERF_SPE_KUNIT_TEST) += \
+					spe_parser_test.o
diff --git a/mm/damon/perf/aux_backend.c b/mm/damon/perf/aux_backend.c
new file mode 100644
index 000000000000..c6c55c7845a5
--- /dev/null
+++ b/mm/damon/perf/aux_backend.c
@@ -0,0 +1,121 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * DAMON Perf AUX Backend — Generic Drain Scheduler
+ *
+ * Provides damon_perf_aux_drain() which is called by kdamond_fn() each
+ * tick before the SPSC ring drain.  It iterates all perf events on the
+ * context and for any event whose backend ops carry DAMON_PERF_BACKEND_AUX,
+ * invokes the per-CPU drain callback to parse PMU records and feed them
+ * into damon_report_access().
+ *
+ * Also provides the auto-selection helper that assigns a backend to an
+ * event based on the PMU object of the created perf_event.
+ */
+
+#include <linux/cpu.h>
+#include <linux/cpumask.h>
+#include <linux/damon.h>
+#include <linux/list.h>
+
+#include "aux_backend.h"
+
+/* Registered backends (populated at initcall time). */
+#define AUX_BACKEND_MAX 4
+
+static const struct damon_perf_backend_ops *aux_backends[AUX_BACKEND_MAX];
+static int nr_aux_backends;
+
+/**
+ * damon_perf_aux_register_backend - Register an AUX backend ops table
+ * @ops: Backend callbacks to register.
+ *
+ * Called at initcall time by each backend.  Returns 0 on success,
+ * -ENOSPC if the static table is full.
+ */
+int damon_perf_aux_register_backend(const struct damon_perf_backend_ops *ops)
+{
+	if (nr_aux_backends >= AUX_BACKEND_MAX)
+		return -ENOSPC;
+
+	aux_backends[nr_aux_backends++] = ops;
+	return 0;
+}
+
+/**
+ * damon_perf_aux_find_backend - Find the backend that claims the PMU
+ * @perf_event: Created perf event whose PMU should be matched.
+ *
+ * Returns the ops table whose match_pmu() claims @perf_event->pmu
+ * (exact PMU object comparison), or NULL if no backend matches.
+ */
+const struct damon_perf_backend_ops *
+damon_perf_aux_find_backend(struct perf_event *perf_event)
+{
+	int i;
+
+	for (i = 0; i < nr_aux_backends; i++) {
+		if (aux_backends[i]->match_pmu &&
+		    aux_backends[i]->match_pmu(perf_event))
+			return aux_backends[i];
+	}
+	return NULL;
+}
+
+/**
+ * damon_perf_aux_select - Auto-select a backend for @event
+ * @event: DAMON perf event that will own the selected backend.
+ * @perf_event: Created perf event used for capability and PMU matching.
+ *
+ * Called from damon_perf_cpu_online() after the first perf_event is
+ * created.  Checks the PMU capabilities of the created event; if it has
+ * PERF_PMU_CAP_ITRACE, looks up a registered AUX backend that claims
+ * the PMU object of the created event.
+ * Overflow-handler PMUs (IBS, PEBS, generic counters) keep ops == NULL.
+ */
+void damon_perf_aux_select(struct damon_perf_event *event,
+			   struct perf_event *perf_event)
+{
+	if (event->ops)
+		return;		/* already assigned */
+
+	if (!(perf_event->pmu->capabilities & PERF_PMU_CAP_ITRACE))
+		return;		/* not an ITRACE / AUX PMU */
+
+	event->ops = damon_perf_aux_find_backend(perf_event);
+}
+
+/**
+ * damon_perf_aux_drain - Drain all AUX backends into the SPSC ring
+ * @ctx: DAMON context whose AUX events should be drained.
+ *
+ * Must be called BEFORE kdamond_check_reported_accesses() each tick
+ * so that freshly-parsed records are available for the ring drain.
+ * Also called at kdamond stop for a final flush.
+ */
+void damon_perf_aux_drain(struct damon_ctx *ctx)
+{
+	struct damon_perf_event *event;
+	int cpu;
+
+	/*
+	 * Hold the CPU hotplug read lock so that a concurrent CPU offline
+	 * callback cannot free the per-CPU backend resources (st->win,
+	 * AUX buffer) while drain is accessing them.  The offline path
+	 * runs under the write-side hotplug lock and clears aux_cpumask
+	 * before freeing, so once cpus_read_lock() is held any CPU still
+	 * in the mask has live resources.
+	 */
+	cpus_read_lock();
+	list_for_each_entry(event, &ctx->perf_events, list) {
+		if (!event->ops ||
+		    !(event->ops->flags & DAMON_PERF_BACKEND_AUX))
+			continue;
+		if (!event->ops->drain)
+			continue;
+
+		/* Only CPUs with initialized AUX resources. */
+		for_each_cpu(cpu, &event->aux_cpumask)
+			event->ops->drain(event, cpu);
+	}
+	cpus_read_unlock();
+}
diff --git a/mm/damon/perf/spe_backend.c b/mm/damon/perf/spe_backend.c
new file mode 100644
index 000000000000..42dd62cd0fcf
--- /dev/null
+++ b/mm/damon/perf/spe_backend.c
@@ -0,0 +1,479 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * DAMON Perf - ARM SPE Backend
+ *
+ * Implements the damon_perf_backend_ops for ARM SPE (Statistical
+ * Profiling Extension).  SPE delivers samples into an AUX trace buffer
+ * owned by the perf core: perf_event_setup_aux() allocates it at init,
+ * the arm_spe_pmu driver writes it through perf_aux_output_begin()/
+ * perf_aux_output_end() (updating rb->aux_head), and each kdamond tick
+ * this backend copies the newly written window [aux_tail, aux_head)
+ * into a linear scratch buffer, parses the SPE record stream (same
+ * packet encoding as tools/perf/util/arm-spe-decoder), and feeds
+ * synthesized damon_access_report entries into the per-CPU SPSC ring
+ * via damon_report_access().
+ *
+ * The AUX ring runs in non-overwrite streaming mode: when the buffer
+ * is full the PMU pauses itself, and this backend's drain both frees
+ * the space (advancing aux_tail) and re-enables the paused event.
+ *
+ * Parser and record layout live in spe_parser.h; the parsing function
+ * itself is kept in this module so the KUnit test (spe_parser_test.c)
+ * can exercise it without a perf PMU.
+ */
+
+#include <linux/bits.h>
+#include <linux/damon.h>
+#include <linux/mm.h>
+#include <linux/module.h>
+#include <linux/percpu.h>
+#include <linux/perf_event.h>
+#include <linux/perf/arm_spe_pmu.h>
+#include <linux/rcupdate.h>
+#include <linux/sched.h>
+#include <linux/slab.h>
+#include <linux/string.h>
+
+#include "../ops-common.h"
+#include "aux_backend.h"
+#include "perf.h"
+#include "spe_parser.h"
+
+/*
+ * Parse one SPE record from the linear drain window.
+ *
+ * st->win holds a window of st->win_size bytes starting at the stream
+ * position st->aux_tail.  On success the cursor is advanced past the
+ * record.  An incomplete trailing record is retained in the AUX ring so
+ * the producer can finish it after the consumer releases any complete
+ * prefix records.
+ * SPE records are padded with PAD packets by the driver
+ * (arm_spe_pmu_pad_buf), so a complete record followed by padding
+ * reaches the window end only after the last record was committed.
+ */
+int spe_parse_one_record(struct spe_parser_state *st, struct spe_record *rec)
+{
+	unsigned int pos = 0;
+	unsigned int record_start = 0;
+	u32 local_tid = 0;
+	bool have_tid = false;
+	bool record_started = false;
+
+	memset(rec, 0, sizeof(*rec));
+
+	while (pos < st->win_size) {
+		u8 hdr = st->win[pos];
+		u8 hdr1;
+		u64 payload;
+		int width, index;
+
+		/* PAD - 1 byte, skip. */
+		if (hdr == SPE_HDR_PAD) {
+			pos++;
+			if (!record_started)
+				record_start = pos;
+			continue;
+		}
+		/* END - record terminator, no payload. */
+		if (hdr == SPE_HDR_END) {
+			record_started = true;
+			pos++;
+			goto record_done;
+		}
+		/* TIMESTAMP - record terminator, always 8-byte payload. */
+		if (hdr == SPE_HDR_TIMESTAMP) {
+			record_started = true;
+			if (pos + 1 + 8 > st->win_size)
+				goto truncated;
+			pos += 1 + 8;
+			goto record_done;
+		}
+		/* EVENTS / DATA-SOURCE - payload is not needed by DAMON. */
+		if ((hdr & SPE_HDR_MASK1) == SPE_HDR_EVENTS ||
+		    (hdr & SPE_HDR_MASK1) == SPE_HDR_SOURCE) {
+			record_started = true;
+			width = 1 << ((hdr >> 4) & 0x3);
+			if (pos + 1 + width > st->win_size)
+				goto truncated;
+			pos += 1 + width;
+			continue;
+		}
+		/* CONTEXT / OP-TYPE / EXTENDED share MASK2. */
+		if ((hdr & SPE_HDR_MASK2) == SPE_HDR_CONTEXT ||
+		    (hdr & SPE_HDR_MASK2) == SPE_HDR_OP_TYPE) {
+			record_started = true;
+			index = hdr & 0x3;
+			width = 1 << ((hdr >> 4) & 0x3);
+			pos += 1;
+		} else if ((hdr & SPE_HDR_MASK2) == SPE_HDR_EXTENDED) {
+			/*
+			 * Extended header: a second byte carries the
+			 * width and (for data packets) the upper index
+			 * bits.  hdr1 == 0 is the ALIGNMENT pseudo-packet:
+			 * consume bytes up to the next 2^(hdr[3:0]+1)
+			 * aligned position, exactly like
+			 * arm_spe_get_alignment().  hdr is the extended
+			 * marker (0x20), so this alignment is 2 bytes.
+			 */
+			if (pos + 2 > st->win_size)
+				goto truncated;
+			hdr1 = st->win[pos + 1];
+			if (hdr1 == SPE_HDR1_ALIGNMENT) {
+				/*
+				 * Alignment is relative to the original
+				 * AUX stream position, not the scratch
+				 * window.  st->aux_tail is the absolute
+				 * offset of the window start.
+				 */
+				unsigned long stream = st->aux_tail + pos;
+				unsigned int align = 1U << ((hdr & 0xf) + 1);
+				unsigned int skip = align -
+					(stream & (align - 1));
+
+				if (pos + skip > st->win_size)
+					goto truncated;
+				pos += skip;
+				if (!record_started)
+					record_start = pos;
+				continue;
+			}
+			if ((hdr1 & SPE_HDR_MASK3) != SPE_HDR_ADDRESS &&
+			    (hdr1 & SPE_HDR_MASK3) != SPE_HDR_COUNTER)
+				goto bad_packet;
+			record_started = true;
+			index = ((hdr & 0x3) << 3) | (hdr1 & 0x7);
+			width = 1 << ((hdr1 >> 4) & 0x3);
+			hdr = hdr1;
+			pos += 2;
+		} else if ((hdr & SPE_HDR_MASK3) == SPE_HDR_ADDRESS ||
+			   (hdr & SPE_HDR_MASK3) == SPE_HDR_COUNTER) {
+			record_started = true;
+			index = hdr & 0x7;
+			width = 1 << ((hdr >> 4) & 0x3);
+			pos += 1;
+		} else {
+			goto bad_packet;
+		}
+
+		if (pos + width > st->win_size)
+			goto truncated;
+
+		/* Little-endian payload. */
+		{
+			const u8 *src = st->win + pos;
+			int j;
+
+			payload = 0;
+			for (j = width - 1; j >= 0; j--)
+				payload = (payload << 8) | src[j];
+		}
+		pos += width;
+
+		if ((hdr & SPE_HDR_MASK2) == SPE_HDR_CONTEXT) {
+			/*
+			 * Bits 3-2 encode the context format:
+			 * 0 = 32-bit CONTEXTIDR_EL1,
+			 * 1 = 64-bit CONTEXTIDR_EL1 (FEAT_CONTEXTIDR_EL1_64).
+			 * Both carry the task pid in the lower 32 bits.
+			 */
+			if (((hdr >> 2) & 0x3) <= 1) {
+				local_tid = (u32)payload;
+				have_tid = true;
+			}
+		} else if ((hdr & SPE_HDR_MASK2) == SPE_HDR_OP_TYPE) {
+			/* LD/ST/ATOMIC class: payload bit 0 = store. */
+			if ((index & SPE_OP_CLASS_MASK) == SPE_OP_CLASS_LDST)
+				rec->is_write = !!(payload & SPE_OP_PKT_ST);
+		} else if ((hdr & SPE_HDR_MASK3) == SPE_HDR_ADDRESS) {
+			if (index == SPE_ADDR_DATA_VIRT) {
+				rec->va = payload & GENMASK_ULL(55, 0);
+				rec->have_addr = true;
+			}
+		}
+		/* EVENTS/COUNTER/TIMESTAMP payloads are ignored. */
+	}
+
+truncated:
+	/*
+	 * PAD and ALIGNMENT packets before the record are independently
+	 * consumable.  Keep the record itself in the AUX ring; the next
+	 * drain copies it again together with newly produced bytes.
+	 */
+	st->aux_tail += record_start;
+	st->bytes += record_start;
+	return SPE_PARSE_NEED_MORE;
+
+record_done:
+	st->aux_tail += pos;
+	st->bytes += pos;
+	if (!rec->have_addr)
+		return SPE_PARSE_SKIP;
+	rec->tid = have_tid ? local_tid : 0;
+	st->records++;
+	return SPE_PARSE_REPORT;
+
+bad_packet:
+	/* Resync one byte forward: drop the offending byte. */
+	pos++;
+	st->aux_tail += pos;
+	st->bytes += pos;
+	return SPE_PARSE_ERROR;
+}
+
+/* ---- Report synthesis ---------------------------------------------- */
+
+static void spe_submit(struct spe_record *rec, int cpu,
+		       struct perf_event *perf_event)
+{
+	struct damon_access_report report = {
+		.vaddr = rec->va & PAGE_MASK,
+		.size = PAGE_SIZE,
+		.cpu = cpu,
+		.is_write = rec->is_write,
+#ifdef CONFIG_DAMON_PERF_OBSERVE
+		.source = DAMON_REPORT_SRC_PERF_AUX,
+#endif
+	};
+
+	/*
+	 * CONTEXTIDR_EL1 carries the sampled task's pid, not its tgid.
+	 * Resolve pid -> tgid here under RCU (no lifetime pin) so
+	 * kdamond_check_reported_accesses() can match the report against
+	 * DAMON's pid targets.  A task that exits between sampling and
+	 * this lookup yields no match (tgid == 0), which mirrors perf's
+	 * own CONTEXTIDR semantics; pid-reuse can misattribute a sample
+	 * the same way it would in any hardware-context-based tool.
+	 */
+	if (rec->tid) {
+		struct task_struct *task;
+
+		rcu_read_lock();
+		task = find_task_by_vpid(rec->tid);
+		if (task) {
+			report.tid = rec->tid;
+			report.tgid = task_tgid_nr(task);
+		}
+		rcu_read_unlock();
+	}
+
+	/*
+	 * A missing or stale CONTEXTID cannot be attributed safely.  Do not
+	 * turn it into an access by assigning an arbitrary DAMON target.
+	 */
+	if (!report.tgid) {
+		damon_perf_observe_miss(rec->va, cpu,
+					DAMON_REPORT_MISS_TGID);
+		return;
+	}
+
+	/* reason 0 denotes a valid sample that is queued to the ring. */
+	damon_perf_observe_sample(rec->va, 0,
+				  0, cpu, 0, 0, perf_event->attr.sample_type);
+	damon_report_access(&report);
+}
+
+/* ---- Backend ops ---------------------------------------------------- */
+
+static bool spe_match_pmu(struct perf_event *perf_event)
+{
+	return arm_spe_pmu_match(perf_event);
+}
+
+static struct spe_parser_state *spe_state(struct damon_perf_event *event,
+					  int cpu)
+{
+	struct damon_perf *perf = event->priv;
+	unsigned long addr = (unsigned long)perf->aux_priv +
+			     per_cpu_offset(cpu);
+
+	return (struct spe_parser_state *)addr;
+}
+
+static int spe_backend_init(struct damon_perf_event *event, int cpu,
+			    struct perf_event *perf_event)
+{
+	struct damon_perf *perf = event->priv;
+	struct spe_parser_state *st;
+
+	int ret;
+
+	/*
+	 * Allocate the AUX buffer the PMU writes into.  Must happen
+	 * before the event is enabled (arm() ordering, see
+	 * damon_perf_event_arm()).  The buffer is non-overwrite
+	 * streaming: the PMU pauses itself when it fills up and this
+	 * backend re-enables it once the drain has freed space.
+	 */
+	ret = perf_event_setup_aux(perf_event, SPE_BUFFER_PAGES, 0);
+	if (ret) {
+		pr_warn_ratelimited("damon-perf: cpu %u aux setup failed: %d\n",
+				    cpu, ret);
+		return ret;
+	}
+
+	if (!perf->aux_priv) {
+		perf->aux_priv = alloc_percpu(struct spe_parser_state);
+		if (!perf->aux_priv)
+			return -ENOMEM;
+	}
+
+	st = spe_state(event, cpu);
+	memset(st, 0, sizeof(*st));
+	st->win = kzalloc(SPE_BUFFER_PAGES * PAGE_SIZE, GFP_KERNEL);
+	if (!st->win)
+		return -ENOMEM;
+
+	return 0;
+}
+
+static void spe_backend_cleanup(struct damon_perf_event *event, int cpu)
+{
+	struct spe_parser_state *st = spe_state(event, cpu);
+
+	kfree(st->win);
+	st->win = NULL;
+}
+
+static int spe_backend_arm(struct damon_perf_event *event, int cpu)
+{
+	struct damon_perf *perf = event->priv;
+	struct spe_parser_state *st = spe_state(event, cpu);
+	struct perf_event *perf_event;
+	unsigned long head;
+
+	perf_event = *per_cpu_ptr(perf->event, cpu);
+	if (!perf_event)
+		return -ENODEV;
+
+	/*
+	 * Start consuming from the current head so data written before
+	 * this session (e.g. a previous arm/disarm cycle) is discarded.
+	 * The consumer cursor must be moved too: perf_aux_output_begin()
+	 * computes free space from user_page->aux_tail.
+	 */
+	head = perf_event_aux_head(perf_event);
+	st->aux_tail = head;
+	if (perf_event_aux_tail_set(perf_event, head) < 0)
+		return -EIO;
+	st->records = 0;
+	st->bytes = 0;
+	st->armed = true;
+	return 0;
+}
+
+static void spe_backend_disarm(struct damon_perf_event *event, int cpu)
+{
+	struct spe_parser_state *st = spe_state(event, cpu);
+
+	/*
+	 * Prevent spe_backend_drain() from calling perf_event_enable()
+	 * during the final drain that follows perf_event_disable().
+	 * The resume-after-full logic is correct only while armed.
+	 */
+	st->armed = false;
+}
+
+static unsigned int spe_backend_drain(struct damon_perf_event *event, int cpu)
+{
+	struct damon_perf *perf = event->priv;
+	struct spe_parser_state *st = spe_state(event, cpu);
+	struct perf_event *perf_event;
+	unsigned long head, size, consumed;
+	long copied;
+	unsigned int drained = 0;
+
+	perf_event = *per_cpu_ptr(perf->event, cpu);
+	if (!perf_event)
+		return 0;
+
+	head = perf_event_aux_head(perf_event);
+	size = head - st->aux_tail;
+	/*
+	 * The AUX head/tail are absolute cursors that can span many
+	 * buffer sizes while the hardware runs.  If the window exceeds
+	 * the buffer size, skip the overwritten prefix (discarded data)
+	 * and clamp to one buffer worth of data.
+	 */
+	if (size > SPE_BUFFER_PAGES * PAGE_SIZE) {
+		st->aux_tail = head - SPE_BUFFER_PAGES * PAGE_SIZE;
+		size = SPE_BUFFER_PAGES * PAGE_SIZE;
+	}
+	if (!size)
+		return 0;
+
+	/*
+	 * Linearize the (possibly wrapped) window into the scratch buffer.
+	 * Do not parse unless the accessor copied the complete snapshot.
+	 */
+	copied = perf_event_aux_copy(perf_event, st->aux_tail,
+				     st->aux_tail + size, st->win);
+	if (copied < 0 || (unsigned long)copied != size)
+		return 0;
+	st->win_size = size;
+
+	while (drained < SPE_BUFFER_MAX_RECORDS) {
+		struct spe_record rec;
+		unsigned long tail0 = st->aux_tail;
+		int ret;
+
+		ret = spe_parse_one_record(st, &rec);
+		if (ret == SPE_PARSE_NEED_MORE)
+			break;
+
+		/* Drop the consumed prefix from the window. */
+		consumed = st->aux_tail - tail0;
+		st->win_size -= consumed;
+		memmove(st->win, st->win + consumed, st->win_size);
+
+		drained++;
+		if (ret == SPE_PARSE_REPORT)
+			spe_submit(&rec, cpu, perf_event);
+	}
+
+	/*
+	 * Release the consumed space back to the AUX ring.  If the tail
+	 * advance fails, do not resume a paused producer.
+	 */
+	if (perf_event_aux_tail_set(perf_event, st->aux_tail) < 0)
+		return drained;
+
+	/*
+	 * ARM SPE stops its hardware when a non-overwrite ring is full,
+	 * but leaves event->state ACTIVE.  perf_event_enable() therefore
+	 * cannot restart it by itself.  Move the event through OFF after
+	 * releasing space, then enable it again.  Do this only for a ring
+	 * snapshot that was exactly full and while the backend is armed;
+	 * the final drain after disarm must not restart the producer.
+	 */
+	if (st->armed && size == SPE_BUFFER_PAGES * PAGE_SIZE) {
+		perf_event_pause(perf_event, false);
+		perf_event_enable(perf_event);
+	}
+
+	return drained;
+}
+
+/* ---- Backend registration ------------------------------------------ */
+
+static const struct damon_perf_backend_ops spe_backend_ops = {
+	.name		= "arm_spe",
+	.flags		= DAMON_PERF_BACKEND_AUX,
+	.match_pmu	= spe_match_pmu,
+	.init		= spe_backend_init,
+	.cleanup	= spe_backend_cleanup,
+	.arm		= spe_backend_arm,
+	.disarm		= spe_backend_disarm,
+	.drain		= spe_backend_drain,
+};
+
+static int __init spe_backend_initcall(void)
+{
+	int ret;
+
+	ret = damon_perf_aux_register_backend(&spe_backend_ops);
+	if (ret)
+		pr_warn("damon-perf: SPE backend registration failed: %d\n",
+			ret);
+	return ret;
+}
+late_initcall(spe_backend_initcall);
diff --git a/mm/damon/perf/spe_parser.h b/mm/damon/perf/spe_parser.h
new file mode 100644
index 000000000000..ef9035ee7784
--- /dev/null
+++ b/mm/damon/perf/spe_parser.h
@@ -0,0 +1,109 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * ARM SPE Record Format Definitions
+ *
+ * Constants and parser interface for the byte stream produced by the
+ * Statistical Profiling Extension (ARM ARM DDI 0487, Chapter D8).
+ * Packet encodings and decode order match
+ * tools/perf/util/arm-spe-decoder/arm-spe-pkt-decoder.c.
+ */
+
+#ifndef _DAMON_PERF_SPE_PARSER_H
+#define _DAMON_PERF_SPE_PARSER_H
+
+#include <linux/types.h>
+
+/* AUX buffer geometry. */
+#define SPE_BUFFER_MIN_PAGES	2	/* arm_spe_pmu_setup_aux() minimum */
+#define SPE_BUFFER_PAGES	16	/* 64 KiB per-CPU trace buffer */
+#define SPE_BUFFER_MAX_RECORDS	8192	/* per-drain record budget */
+
+/* Packet header masks/values (arm-spe-pkt-decoder.h). */
+#define SPE_HDR_MASK1		0xcf
+#define SPE_HDR_MASK2		0xfc
+#define SPE_HDR_MASK3		0xf8
+#define SPE_HDR_PAD		0x00
+#define SPE_HDR_END		0x01
+#define SPE_HDR_TIMESTAMP	0x71
+#define SPE_HDR_EVENTS		0x42
+#define SPE_HDR_SOURCE		0x43
+#define SPE_HDR_CONTEXT		0x64
+#define SPE_HDR_OP_TYPE		0x48
+#define SPE_HDR_EXTENDED	0x20
+#define SPE_HDR_ADDRESS		0xb0
+#define SPE_HDR_COUNTER		0x98
+#define SPE_HDR1_ALIGNMENT	0x00
+
+/* Address packet index for the virtual data address. */
+#define SPE_ADDR_DATA_VIRT	2
+
+/* OP-TYPE index bits[1:0] == 1 selects the LD/ST/ATOMIC class. */
+#define SPE_OP_CLASS_MASK	0x3
+#define SPE_OP_CLASS_LDST	0x1
+#define SPE_OP_PKT_ST		0x1
+
+/*
+ * Parser return values.  The distinction between NEED_MORE and SKIP is
+ * essential: NEED_MORE means the window ran out mid-record and the record
+ * remains unconsumed, while SKIP means a complete record was consumed that
+ * has no virtual address.
+ */
+enum spe_parse_ret {
+	SPE_PARSE_NEED_MORE = 0,	/* incomplete record retained */
+	SPE_PARSE_REPORT = 1,		/* complete record with a VA */
+	SPE_PARSE_SKIP = 2,		/* complete record without a VA */
+	SPE_PARSE_ERROR = -EINVAL,	/* bad packet; parser resynced */
+};
+
+/**
+ * struct spe_record - One parsed SPE record.
+ * @va:		DATA_VIRT address (bits[55:0]).
+ * @is_write:	Store/load class from OP-TYPE payload bit 0.
+ * @tid:	CONTEXTIDR_EL1 payload (the sampled task's pid).
+ * @have_addr:	An ADDRESS/DATA_VIRT packet was seen.
+ */
+struct spe_record {
+	unsigned long	va;
+	bool		is_write;
+	u32		tid;
+	bool		have_addr;
+};
+
+/**
+ * struct spe_parser_state - Per-(event,cpu) parser state.
+ * @win:	Linear copy of the drained AUX window (read-only for
+ *		the parser; the caller owns the buffer).
+ * @win_size:	Current window length (bytes).
+ * @aux_tail:	Absolute consumption cursor (rb->aux_head domain).
+ * @records:	Records parsed with a VA (session total).
+ * @bytes:	Bytes consumed (session total).
+ *
+ * spe_parse_one_record() never modifies @win.  It advances @aux_tail
+ * and @bytes by the bytes it consumes; the caller is responsible for
+ * dropping the consumed prefix from @win (memmove + win_size shrink)
+ * before calling again.
+ */
+struct spe_parser_state {
+	u8			*win;
+	unsigned int		win_size;
+	unsigned long		aux_tail;
+	unsigned int		records;
+	unsigned long		bytes;
+	bool			armed;
+};
+
+/**
+ * spe_parse_one_record() - Parse one record from the linear window.
+ *
+ * Reads the next record from @st->win (a window of @st->win_size bytes
+ * starting at the stream position @st->aux_tail) and advances
+ * @st->aux_tail past it.  Returns one of enum spe_parse_ret.
+ *
+ * An incomplete trailing record is retained.  The next drain reparses it
+ * from its first packet after the producer appends more bytes.  Leading PAD
+ * and ALIGNMENT packets can be consumed independently.
+ */
+int spe_parse_one_record(struct spe_parser_state *st,
+			 struct spe_record *rec);
+
+#endif /* _DAMON_PERF_SPE_PARSER_H */
diff --git a/mm/damon/vaddr.c b/mm/damon/vaddr.c
index a68c7262d533..814571c4db78 100644
--- a/mm/damon/vaddr.c
+++ b/mm/damon/vaddr.c
@@ -18,6 +18,7 @@
 #include <linux/sched/mm.h>
 
 #include "perf/perf.h"
+#include "perf/aux_backend.h"
 
 #include "../internal.h"
 #include "ops-common.h"
@@ -1159,6 +1160,21 @@ static int damon_perf_cpu_online(unsigned int cpu, struct hlist_node *node)
 	*per_cpu_ptr(perf->event, cpu) = perf_event;
 
 	damon_perf_observe_event_bound(event, cpu, perf_event);
+	if (!event->ops)
+		cpumask_clear(&event->aux_cpumask);
+	damon_perf_aux_select(event, perf_event);
+	if (event->ops && event->ops->init) {
+		int ret = event->ops->init(event, cpu, perf_event);
+
+		if (ret) {
+			pr_warn_ratelimited("damon-perf: cpu %u AUX init failed: %d\n", cpu, ret);
+			perf_event_release_kernel(perf_event);
+			*per_cpu_ptr(perf->event, cpu) = NULL;
+			event->any_cpu_failed = true;
+			return 0;
+		}
+		cpumask_set_cpu(cpu, &event->aux_cpumask);
+	}
 
 	/*
 	 * Late-online CPU after the substrate is armed: events are created
@@ -1167,6 +1183,16 @@ static int damon_perf_cpu_online(unsigned int cpu, struct hlist_node *node)
 	 * already-online CPUs.
 	 */
 	if (event->ctx && READ_ONCE(event->ctx->perf_events_active)) {
+		if (event->ops && event->ops->arm &&
+		    event->ops->arm(event, cpu)) {
+			if (cpumask_test_and_clear_cpu(cpu, &event->aux_cpumask) &&
+			    event->ops->cleanup)
+				event->ops->cleanup(event, cpu);
+			perf_event_release_kernel(perf_event);
+			*per_cpu_ptr(perf->event, cpu) = NULL;
+			event->any_cpu_failed = true;
+			return 0;
+		}
 		perf_event_enable(perf_event);
 		damon_perf_observe_event_enabled(event, cpu,
 				perf_event->state, perf_event->oncpu);
@@ -1188,30 +1214,58 @@ static int damon_perf_cpu_offline(unsigned int cpu, struct hlist_node *node)
 	if (perf_event) {
 		damon_perf_observe_event_destroyed(event, cpu);
 		perf_event_disable(perf_event);
+		if (event->ops && event->ops->disarm)
+			event->ops->disarm(event, cpu);
+		if (cpumask_test_and_clear_cpu(cpu, &event->aux_cpumask) &&
+		    event->ops && event->ops->cleanup)
+			event->ops->cleanup(event, cpu);
 		perf_event_release_kernel(perf_event);
 		*per_cpu_ptr(perf->event, cpu) = NULL;
 	}
 	return 0;
 }
 
-void damon_perf_event_arm(struct damon_perf_event *event)
+int damon_perf_event_arm(struct damon_perf_event *event)
 {
 	struct damon_perf *perf = event->priv;
 	struct perf_event *perf_event;
-	int cpu;
+	int cpu, failed_cpu = nr_cpu_ids;
 
 	if (!perf)
-		return;
+		return -EINVAL;
 
 	for_each_online_cpu(cpu) {
 		perf_event = *per_cpu_ptr(perf->event, cpu);
 		if (perf_event) {
+			if (event->ops && event->ops->arm &&
+			    event->ops->arm(event, cpu)) {
+				event->any_cpu_failed = true;
+				failed_cpu = cpu;
+				break;
+			}
 			perf_event_enable(perf_event);
 			damon_perf_observe_event_enabled(event, cpu,
 					perf_event->state,
 					perf_event->oncpu);
 		}
 	}
+	if (failed_cpu == nr_cpu_ids)
+		return 0;
+
+	/* Roll back CPUs enabled by this arm attempt. */
+	for_each_online_cpu(cpu) {
+		if (cpu >= failed_cpu)
+			break;
+		perf_event = *per_cpu_ptr(perf->event, cpu);
+		if (!perf_event)
+			continue;
+		perf_event_disable(perf_event);
+		if (event->ops && event->ops->disarm)
+			event->ops->disarm(event, cpu);
+		damon_perf_observe_event_disabled(event, cpu,
+						  perf_event->state);
+	}
+	return -EIO;
 }
 
 void damon_perf_event_disarm(struct damon_perf_event *event)
@@ -1227,6 +1281,8 @@ void damon_perf_event_disarm(struct damon_perf_event *event)
 		perf_event = *per_cpu_ptr(perf->event, cpu);
 		if (perf_event) {
 			perf_event_disable(perf_event);
+			if (event->ops && event->ops->disarm)
+				event->ops->disarm(event, cpu);
 			damon_perf_observe_event_disabled(event, cpu,
 					perf_event->state);
 		}
@@ -1272,6 +1328,8 @@ int damon_perf_init(struct damon_ctx *ctx, struct damon_perf_event *event)
 
 free_event:
 	damon_perf_observe_event_free(event);
+	if (perf->aux_priv)
+		free_percpu((void __percpu *)perf->aux_priv);
 	free_percpu(perf->event);
 free_perf:
 	kfree(perf);
@@ -1291,6 +1349,10 @@ void damon_perf_cleanup(struct damon_ctx *ctx, struct damon_perf_event *event)
 	cpuhp_state_remove_instance(damon_perf_cpuhp_state,
 			&event->hlist_node);
 
+	if (perf->aux_priv) {
+		free_percpu((void __percpu *)perf->aux_priv);
+		perf->aux_priv = NULL;
+	}
 	free_percpu(perf->event);
 	kfree(perf);
 	event->priv = NULL;
-- 
2.43.0

[RFC PATCH 3/4] mm/damon/perf: add KUnit tests for the SPE record parser

[Kunwu Chan]

From: "Lian Wang (ProcessMission)" <lianux.mm@gmail.com>

Add byte-exact tests for load and store records, timestamp terminators,
multiple records, PAD and ALIGNMENT packets, extended addresses, invalid
extended headers, error resynchronization, and records without a virtual
address.

Cover ALIGNMENT packets at both odd and already aligned stream offsets.
Also verify that a record split across two AUX snapshots leaves the tail
unchanged until the terminating packet becomes available.

Co-developed-by: Kunwu Chan <kunwu.chan@gmail.com>
Signed-off-by: Kunwu Chan <kunwu.chan@gmail.com>
Signed-off-by: Lian Wang (ProcessMission) <lianux.mm@gmail.com>
---
 mm/damon/perf/spe_parser_test.c | 365 ++++++++++++++++++++++++++++++++
 1 file changed, 365 insertions(+)
 create mode 100644 mm/damon/perf/spe_parser_test.c

diff --git a/mm/damon/perf/spe_parser_test.c b/mm/damon/perf/spe_parser_test.c
new file mode 100644
index 000000000000..598d9fd7cdc7
--- /dev/null
+++ b/mm/damon/perf/spe_parser_test.c
@@ -0,0 +1,365 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * KUnit tests for the DAMON perf ARM SPE record parser.
+ *
+ * Each parameterized case feeds a byte-exact SPE packet stream (same
+ * encodings and decode order as tools/perf/util/arm-spe-decoder) into
+ * spe_parse_one_record() and verifies the synthesized records, the
+ * return values, and the aux_tail accounting.  The loop mirrors
+ * spe_backend_drain() including the caller-side window shrink.
+ */
+
+#include <kunit/test.h>
+#include <linux/slab.h>
+
+#include "spe_parser.h"
+
+/**
+ * struct spe_parse_case - One parser test case.
+ * @name:	Parameter description (shown on failure).
+ * @stream:	Byte-exact SPE packet stream.
+ * @len:	@stream length.
+ * @exp_reports:	Expected SPE_PARSE_REPORT count.
+ * @exp_skips:		Expected SPE_PARSE_SKIP count.
+ * @exp_errors:		Expected SPE_PARSE_ERROR count.
+ * @exp_tail:		Expected st->aux_tail after the stream.
+ * @exp_va: Expected virtual address of the first REPORT record.
+ * @exp_tid: Expected tid of the first REPORT record.
+ * @exp_is_write: Expected access type of the first REPORT record.
+ */
+struct spe_parse_case {
+	const char *name;
+	const u8 *stream;
+	size_t len;
+	unsigned int exp_reports;
+	unsigned int exp_skips;
+	unsigned int exp_errors;
+	unsigned long exp_tail;
+	unsigned long exp_va;
+	u32 exp_tid;
+	bool exp_is_write;
+};
+
+static const u8 stream_store[] = {
+	0x66, 0x2a, 0x00, 0x00, 0x00,			/* CONTEXT: 64-bit EL1 tid=42 */
+	0xb2, 0x00, 0x10, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,	/* VA 0x1000 */
+	0x49, 0x01,					/* OP-TYPE: ST */
+	0x01,						/* END */
+};
+
+static const u8 stream_ts_load[] = {
+	0xb2, 0x00, 0x20, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,	/* VA 0x2000 */
+	0x49, 0x00,					/* OP-TYPE: load */
+	0x71, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,	/* TIMESTAMP end */
+};
+
+static const u8 stream_two[] = {
+	0x66, 0x2a, 0x00, 0x00, 0x00,			/* CONTEXT: 64-bit EL1 tid=42 */
+	0xb2, 0x00, 0x10, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,	/* VA 0x1000 */
+	0x49, 0x01,					/* OP-TYPE: ST */
+	0x01,						/* END */
+	0xb2, 0x00, 0x20, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,	/* VA 0x2000 */
+	0x49, 0x00,					/* OP-TYPE: load */
+	0x01,						/* END */
+};
+
+static const u8 stream_pad[] = {
+	0x00, 0x00,					/* PAD prefix */
+	0x66, 0x2a, 0x00, 0x00, 0x00,			/* CONTEXT: 64-bit EL1 tid=42 */
+	0xb2, 0x00, 0x10, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,	/* VA 0x1000 */
+	0x49, 0x01,					/* OP-TYPE: ST */
+	0x01,						/* END */
+	0x00, 0x00, 0x00,				/* PAD padding */
+};
+
+static const u8 stream_alignment[] = {
+	0x66, 0x2a, 0x00, 0x00, 0x00,			/* CONTEXT: tid 42, pos 0-4 */
+	0x20, 0x00,					/* ALIGNMENT at odd pos 5 */
+	0xb2, 0x00, 0x10, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,	/* VA 0x1000 */
+	0x49, 0x00,					/* OP-TYPE: load */
+	0x01,						/* END */
+};
+
+static const u8 stream_alignment_aligned[] = {
+	0x20, 0x00,					/* ALIGNMENT at even pos 0 */
+	0x66, 0x2a, 0x00, 0x00, 0x00,			/* CONTEXT: 64-bit EL1 tid=42 */
+	0xb2, 0x00, 0x10, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,	/* VA 0x1000 */
+	0x49, 0x00,					/* OP-TYPE: load */
+	0x01,						/* END */
+};
+
+static const u8 stream_bad[] = {
+	0xff,						/* unknown header */
+	0x66, 0x2a, 0x00, 0x00, 0x00,			/* CONTEXT: 64-bit EL1 tid=42 */
+	0xb2, 0x00, 0x10, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,	/* VA 0x1000 */
+	0x49, 0x01,					/* OP-TYPE: ST */
+	0x01,						/* END */
+};
+
+static const u8 stream_skip[] = {
+	0x49, 0x00,					/* OP-TYPE: load */
+	0x01,						/* END, no address */
+};
+
+static const u8 stream_other_pkts[] = {
+	0x42, 0x05,					/* EVENTS (width 1) */
+	0x43, 0x06,					/* DATA-SOURCE (width 1) */
+	0x98, 0x00, 0x00,				/* COUNTER (width 2) */
+	0xb2, 0x00, 0x30, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,	/* VA 0x3000 */
+	0x49, 0x00,					/* OP-TYPE: load */
+	0x01,						/* END */
+};
+
+static const u8 stream_truncated[] = {
+	0x66, 0x2a, 0x00, 0x00, 0x00,			/* CONTEXT: 64-bit EL1 tid=42 */
+	0xb2, 0x00, 0x10, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,	/* VA 0x1000 */
+	0x49, 0x01,					/* OP-TYPE: ST, no END */
+};
+
+static const u8 stream_truncated_packet[] = {
+	0xb2, 0x00, 0x10,				/* short 8-byte address */
+};
+
+static const u8 stream_ext_addr[] = {
+	0x20, 0xb2,					/* EXTENDED ADDRESS, DATA_VIRT */
+	0x00, 0x40, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,	/* VA 0x4000 */
+	0x49, 0x00,					/* OP-TYPE: load */
+	0x01,						/* END */
+};
+
+static const u8 stream_invalid_extended[] = {
+	0x20, 0x42, 0x00,				/* invalid extended EVENTS */
+	0x01,						/* END after resync */
+};
+
+static const u8 stream_pad_only[] = {
+	0x00, 0x00, 0x00,
+};
+
+static const u8 stream_empty[] = { 0x00 };
+
+static const struct spe_parse_case spe_parse_cases[] = {
+	{
+		.name = "store record",
+		.stream = stream_store,
+		.len = sizeof(stream_store),
+		.exp_reports = 1,
+		.exp_tail = 17,
+		.exp_va = 0x1000,
+		.exp_tid = 42,
+		.exp_is_write = true,
+	},
+	{
+		.name = "load record with timestamp terminator",
+		.stream = stream_ts_load,
+		.len = sizeof(stream_ts_load),
+		.exp_reports = 1,
+		.exp_tail = 20,
+		.exp_va = 0x2000,
+		.exp_tid = 0,
+		.exp_is_write = false,
+	},
+	{
+		.name = "two records in one window",
+		.stream = stream_two,
+		.len = sizeof(stream_two),
+		.exp_reports = 2,
+		.exp_tail = 29,
+		.exp_va = 0x1000,
+		.exp_tid = 42,
+		.exp_is_write = true,
+	},
+	{
+		.name = "pad-wrapped record",
+		.stream = stream_pad,
+		.len = sizeof(stream_pad),
+		.exp_reports = 1,
+		.exp_tail = 22,
+		.exp_va = 0x1000,
+		.exp_tid = 42,
+		.exp_is_write = true,
+	},
+	{
+		.name = "alignment packet at odd position",
+		.stream = stream_alignment,
+		.len = sizeof(stream_alignment),
+		.exp_reports = 1,
+		.exp_tail = 19,
+		.exp_va = 0x1000,
+		.exp_tid = 42,
+		.exp_is_write = false,
+	},
+	{
+		.name = "alignment packet at aligned position",
+		.stream = stream_alignment_aligned,
+		.len = sizeof(stream_alignment_aligned),
+		.exp_reports = 1,
+		.exp_tail = sizeof(stream_alignment_aligned),
+		.exp_va = 0x1000,
+		.exp_tid = 42,
+		.exp_is_write = false,
+	},
+	{
+		.name = "bad packet resync",
+		.stream = stream_bad,
+		.len = sizeof(stream_bad),
+		.exp_reports = 1,
+		.exp_errors = 1,
+		.exp_tail = 18,
+		.exp_va = 0x1000,
+		.exp_tid = 42,
+		.exp_is_write = true,
+	},
+	{
+		.name = "record without address",
+		.stream = stream_skip,
+		.len = sizeof(stream_skip),
+		.exp_skips = 1,
+		.exp_tail = 3,
+	},
+	{
+		.name = "events/source/counter packets ignored",
+		.stream = stream_other_pkts,
+		.len = sizeof(stream_other_pkts),
+		.exp_reports = 1,
+		.exp_tail = 19,
+		.exp_va = 0x3000,
+		.exp_is_write = false,
+	},
+	{
+		.name = "truncated trailing record retained",
+		.stream = stream_truncated,
+		.len = sizeof(stream_truncated),
+		.exp_tail = 0,
+	},
+	{
+		.name = "truncated packet retained",
+		.stream = stream_truncated_packet,
+		.len = sizeof(stream_truncated_packet),
+		.exp_tail = 0,
+	},
+	{
+		.name = "extended address packet",
+		.stream = stream_ext_addr,
+		.len = sizeof(stream_ext_addr),
+		.exp_reports = 1,
+		.exp_tail = 13,
+		.exp_va = 0x4000,
+		.exp_is_write = false,
+	},
+	{
+		.name = "invalid extended header resync",
+		.stream = stream_invalid_extended,
+		.len = sizeof(stream_invalid_extended),
+		.exp_skips = 1,
+		.exp_errors = 1,
+		.exp_tail = sizeof(stream_invalid_extended),
+	},
+	{
+		.name = "pad-only window",
+		.stream = stream_pad_only,
+		.len = sizeof(stream_pad_only),
+		.exp_tail = 3,
+	},
+	{
+		.name = "empty window",
+		.stream = stream_empty,
+		.len = 0,
+		.exp_tail = 0,
+	},
+};
+
+KUNIT_ARRAY_PARAM_DESC(spe_parse, spe_parse_cases, name);
+
+static void spe_parse_case_test(struct kunit *test)
+{
+	const struct spe_parse_case *tc = test->param_value;
+	struct spe_parser_state st = { 0 };
+	struct spe_record rec;
+	u8 *buf;
+	unsigned int reports = 0, skips = 0, errors = 0, guard = 0;
+	bool first_checked = false;
+
+	buf = kunit_kmalloc(test, tc->len ?: 1, GFP_KERNEL);
+	KUNIT_ASSERT_NOT_ERR_OR_NULL(test, buf);
+	memcpy(buf, tc->stream, tc->len);
+	st.win = buf;
+	st.win_size = tc->len;
+
+	while (guard++ < SPE_BUFFER_MAX_RECORDS) {
+		unsigned long tail0 = st.aux_tail;
+		unsigned long consumed;
+		int ret = spe_parse_one_record(&st, &rec);
+
+		if (ret == SPE_PARSE_NEED_MORE)
+			break;
+
+		/* caller-side window shrink, mirrors spe_backend_drain() */
+		consumed = st.aux_tail - tail0;
+		st.win_size -= consumed;
+		memmove(st.win, st.win + consumed, st.win_size);
+
+		switch (ret) {
+		case SPE_PARSE_REPORT:
+			reports++;
+			if (!first_checked) {
+				KUNIT_EXPECT_EQ(test, tc->exp_va, rec.va);
+				KUNIT_EXPECT_EQ(test, tc->exp_tid, rec.tid);
+				KUNIT_EXPECT_EQ(test, tc->exp_is_write,
+						rec.is_write);
+				first_checked = true;
+			}
+			break;
+		case SPE_PARSE_SKIP:
+			skips++;
+			break;
+		case SPE_PARSE_ERROR:
+			errors++;
+			break;
+		}
+	}
+
+	KUNIT_EXPECT_EQ(test, tc->exp_reports, reports);
+	KUNIT_EXPECT_EQ(test, tc->exp_skips, skips);
+	KUNIT_EXPECT_EQ(test, tc->exp_errors, errors);
+	KUNIT_EXPECT_EQ(test, tc->exp_tail, st.aux_tail);
+	KUNIT_EXPECT_EQ(test, tc->exp_tail, st.bytes);
+	KUNIT_EXPECT_EQ(test, tc->exp_reports, st.records);
+}
+
+static void spe_split_record_test(struct kunit *test)
+{
+	struct spe_parser_state st = {
+		.win = (u8 *)stream_store,
+		.win_size = sizeof(stream_store) - 1,
+	};
+	struct spe_record rec;
+	int ret;
+
+	ret = spe_parse_one_record(&st, &rec);
+	KUNIT_ASSERT_EQ(test, SPE_PARSE_NEED_MORE, ret);
+	KUNIT_EXPECT_EQ(test, 0UL, st.aux_tail);
+	KUNIT_EXPECT_EQ(test, 0UL, st.bytes);
+
+	/* The next AUX copy starts at the unchanged tail and includes END. */
+	st.win_size = sizeof(stream_store);
+	ret = spe_parse_one_record(&st, &rec);
+	KUNIT_ASSERT_EQ(test, SPE_PARSE_REPORT, ret);
+	KUNIT_EXPECT_EQ(test, (unsigned long)sizeof(stream_store),
+			st.aux_tail);
+	KUNIT_EXPECT_EQ(test, 0x1000UL, rec.va);
+	KUNIT_EXPECT_EQ(test, 42U, rec.tid);
+}
+
+static struct kunit_case spe_parser_test_cases[] = {
+	KUNIT_CASE_PARAM(spe_parse_case_test, spe_parse_gen_params),
+	KUNIT_CASE(spe_split_record_test),
+	{},
+};
+
+static struct kunit_suite spe_parser_test_suite = {
+	.name = "damon_perf_spe_parser",
+	.test_cases = spe_parser_test_cases,
+};
+
+kunit_test_suite(spe_parser_test_suite);
-- 
2.43.0

[RFC PATCH 4/4] selftests/damon: add DAMON perf AUX backend test

[Kunwu Chan]

From: "Lian Wang (ProcessMission)" <lianux.mm@gmail.com>

Register a DAMON kselftest for the ARM SPE AUX backend.  When kernel
sources are available, check the backend integration and the required
AUX-before-ring lifecycle ordering.  Independently run the SPE parser
KUnit suite through debugfs when it is built.

On an ARM SPE system with no pre-existing kdamond, create one controlled
userspace target and collect counter deltas from a five-second session.
Stop the session explicitly, then verify positive end-to-end pipeline
counters and final enqueue/dequeue closure.  Use a private mktemp result
directory and restore only the DAMON state created by this test.

This is a smoke and lifecycle test; full-ring pause/resume validation
remains a separate hardware stress test.

Co-developed-by: Kunwu Chan <kunwu.chan@gmail.com>
Signed-off-by: Kunwu Chan <kunwu.chan@gmail.com>
Signed-off-by: Lian Wang (ProcessMission) <lianux.mm@gmail.com>
---
 tools/testing/selftests/damon/Makefile        |   1 +
 .../selftests/damon/damon_perf_aux_test.sh    | 414 ++++++++++++++++++
 2 files changed, 415 insertions(+)
 create mode 100755 tools/testing/selftests/damon/damon_perf_aux_test.sh

diff --git a/tools/testing/selftests/damon/Makefile b/tools/testing/selftests/damon/Makefile
index 1db8fa95ba2d..204a0b60d84a 100644
--- a/tools/testing/selftests/damon/Makefile
+++ b/tools/testing/selftests/damon/Makefile
@@ -24,4 +24,5 @@ TEST_PROGS += sysfs_no_op_commit_break.py
 EXTRA_CLEAN = __pycache__
 
 TEST_PROGS += damon_perf_obs_test.sh
+TEST_PROGS += damon_perf_aux_test.sh
 include ../lib.mk
diff --git a/tools/testing/selftests/damon/damon_perf_aux_test.sh b/tools/testing/selftests/damon/damon_perf_aux_test.sh
new file mode 100755
index 000000000000..86ccaca6310c
--- /dev/null
+++ b/tools/testing/selftests/damon/damon_perf_aux_test.sh
@@ -0,0 +1,414 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+#
+# DAMON Perf AUX Backend - Automated Test
+#
+# Validates the AUX trace-buffer backend framework and the ARM SPE
+# backend:
+#   1. Source-level structure checks (files, Makefile, ops interface)
+#   2. SPE parser: runs the KUnit suite against byte-exact binary
+#      fixtures (mm/damon/perf/spe_parser_test.c) through debugfs
+#   3. Architecture checks (exact PMU match, SPSC ring contract,
+#      lifecycle ordering)
+#   4. Runtime smoke test: live DAMON session configured with an ARM
+#      SPE event.  Only runs when no kdamonds exist; the pre-test
+#      sysfs state is restored on exit.
+#
+# Usage:  sudo ./damon_perf_aux_test.sh [pmu-name]
+#
+# Requirements:
+#   - CONFIG_DAMON_PERF_OBSERVE=y
+#   - CONFIG_DAMON_PERF_SPE_KUNIT_TEST=y and CONFIG_KUNIT_DEBUGFS=y
+#     for section 2 (skipped otherwise)
+#   - CONFIG_ARM_SPE_PMU=y for section 4 (skipped otherwise)
+#   - Root privileges
+
+set -e
+PASSED=0; FAILED=0; SKIPPED=0
+pass() { echo "  [PASS] $1"; PASSED=$((PASSED + 1)); }
+fail() { echo "  [FAIL] $1"; FAILED=$((FAILED + 1)); }
+skip() { echo "  [SKIP] $*"; SKIPPED=$((SKIPPED + 1)); }
+
+PMU_NAME="${1:-arm_spe_0}"
+RESULTS_DIR=$(mktemp -d "${TMPDIR:-/tmp}/damon_aux_test.XXXXXX") || exit 1
+exec > >(tee "$RESULTS_DIR/output.log") 2>&1
+
+ROOT="$(git rev-parse --show-toplevel 2>/dev/null || true)"
+if [[ -z "$ROOT" ]]; then
+    ROOT="/usr/src/linux"
+fi
+SRC_DIR="$ROOT/mm/damon/perf"
+
+ADMIN="/sys/kernel/mm/damon/admin"
+
+echo "=========================================="
+echo " DAMON Perf AUX Backend Test"
+echo "=========================================="
+
+# ---- Section 1: Source structure ----
+if [[ -d "$SRC_DIR" && -f "$ROOT/mm/damon/core.c" ]]; then
+echo ""
+echo "--- 1. Backend Source Files ---"
+for f in aux_backend.h aux_backend.c spe_parser.h spe_backend.c spe_parser_test.c; do
+    if [[ -f "$SRC_DIR/$f" ]]; then
+        pass "Source file: $f"
+    else
+        fail "Source file: $f"
+    fi
+done
+
+echo ""
+echo "--- 2. Makefile Registration ---"
+for obj in aux_backend.o spe_backend.o spe_parser_test.o; do
+    if grep -q "$obj" "$SRC_DIR/Makefile" 2>/dev/null; then
+        pass "Makefile registers: $obj"
+    else
+        fail "Makefile registers: $obj"
+    fi
+done
+
+echo ""
+echo "--- 3. Backend Ops Interface ---"
+OPS_COUNT=$(grep -c '(\*match_pmu)\|(\*init)\|(\*cleanup)\|(\*arm)\|(\*disarm)\|(\*drain)' \
+    "$SRC_DIR/aux_backend.h" 2>/dev/null || echo 0)
+if [[ "$OPS_COUNT" -ge 6 ]]; then
+    pass "Backend ops interface: 6 ops defined ($OPS_COUNT found)"
+else
+    fail "Backend ops interface: expected 6 ops, found $OPS_COUNT"
+fi
+if grep -q 'DAMON_PERF_BACKEND_AUX' "$SRC_DIR/aux_backend.h" 2>/dev/null; then
+    pass "DAMON_PERF_BACKEND_AUX flag defined"
+else
+    fail "DAMON_PERF_BACKEND_AUX flag"
+fi
+
+echo ""
+echo "--- 4. SPE Parser Constants ---"
+for c in SPE_BUFFER_MIN_PAGES SPE_BUFFER_MAX_RECORDS SPE_HDR_MASK1 SPE_HDR_PAD \
+    SPE_HDR_END SPE_HDR_TIMESTAMP SPE_HDR_EXTENDED SPE_ADDR_DATA_VIRT \
+    SPE_OP_CLASS_LDST; do
+    if grep -q "$c" "$SRC_DIR/spe_parser.h" 2>/dev/null; then
+        pass "Parser constant: $c"
+    else
+        fail "Parser constant: $c"
+    fi
+done
+else
+    skip "source structure" "kernel source tree not available"
+fi
+
+# ---- Section 2: Parser correctness (KUnit, binary fixtures) ----
+echo ""
+echo "--- 5. SPE Parser: KUnit suite (binary fixtures) ---"
+KUNIT_DIR="/sys/kernel/debug/kunit/damon_perf_spe_parser"
+if [[ -d "$KUNIT_DIR" ]]; then
+    # Writing to "run" triggers a fresh run; "results" then holds the
+    # TAP output of the byte-exact fixture cases.
+    if [[ -f "$KUNIT_DIR/run" ]]; then
+        echo run > "$KUNIT_DIR/run" 2>/dev/null || true
+    fi
+    RES="$KUNIT_DIR/results"
+    if [[ -f "$RES" ]]; then
+        if grep -Eq "^[[:space:]]*not ok" "$RES"; then
+            fail "KUnit parser suite has failing cases"
+            grep -E "^[[:space:]]*not ok" "$RES" | sed 's/^/    /'
+        elif grep -Eq "^[[:space:]]*ok " "$RES"; then
+            pass "KUnit parser suite: all cases pass"
+        else
+            fail "KUnit parser suite produced no completed cases"
+        fi
+        echo "    $(grep '# Totals:' "$RES" | tail -1)"
+        # Spot-check the fixtures that cover the review-critical paths:
+        # record content, consumed-length accounting, truncation,
+        # alignment, extended packets, and error resync.
+        for c in "store record" "bad packet resync" \
+            "alignment packet at odd position" \
+            "alignment packet at aligned position" \
+            "extended address packet" \
+            "invalid extended header resync" \
+            "truncated trailing record retained" \
+            "truncated packet retained" \
+            "pad-only window"; do
+            if grep -Eq "^[[:space:]]*ok .*$c" "$RES"; then
+                pass "fixture: $c"
+            else
+                fail "fixture: $c"
+            fi
+        done
+    else
+        skip "KUnit results" "no results file (suite did not run)"
+    fi
+else
+    skip "KUnit parser suite" "CONFIG_KUNIT_DEBUGFS missing or suite not built"
+fi
+
+# ---- Section 3: Architecture checks ----
+echo ""
+echo "--- 6. Architecture Correctness ---"
+
+if [[ ! -d "$SRC_DIR" || ! -f "$ROOT/mm/damon/core.c" ]]; then
+    skip "architecture source checks" "kernel source tree not available"
+else
+
+# Exact PMU match by pmu object, not by name-prefix matching.
+if grep -q 'arm_spe_pmu_match' "$SRC_DIR/spe_backend.c" 2>/dev/null; then
+    pass "PMU match: exact match via arm_spe_pmu_match()"
+else
+    fail "PMU match: arm_spe_pmu_match() missing"
+fi
+if grep -q 'strncmp.*arm_spe' "$SRC_DIR/spe_backend.c" 2>/dev/null; then
+    fail "PMU match: strncmp name-prefix matching must not be used"
+else
+    pass "PMU match: no strncmp name-prefix matching"
+fi
+
+# Backend selection: gated on the ITRACE capability, then matched by PMU.
+if grep -q 'PERF_PMU_CAP_ITRACE' "$SRC_DIR/aux_backend.c" 2>/dev/null; then
+    pass "PMU selection: gated by PERF_PMU_CAP_ITRACE"
+else
+    fail "PMU selection: ITRACE gate missing"
+fi
+
+# Lifecycle ordering: arm() before perf_event_enable(), disarm() after
+# perf_event_disable().
+ARM_LINE=$(awk '/^int damon_perf_event_arm\(/ { in_fn=1 } \
+    in_fn && /ops->arm/ { print NR; exit }' "$ROOT/mm/damon/vaddr.c")
+ENABLE_LINE=$(awk '/^int damon_perf_event_arm\(/ { in_fn=1 } \
+    in_fn && /perf_event_enable/ { print NR; exit }' "$ROOT/mm/damon/vaddr.c")
+if [[ -n "$ARM_LINE" && -n "$ENABLE_LINE" && "$ARM_LINE" -lt "$ENABLE_LINE" ]]; then
+    pass "Lifecycle: ops->arm() before perf_event_enable()"
+else
+    fail "Lifecycle: arm() must precede perf_event_enable()"
+fi
+
+DISABLE_LINE=$(awk '/^void damon_perf_event_disarm\(/ { in_fn=1 } \
+    in_fn && /perf_event_disable/ { print NR; exit }' "$ROOT/mm/damon/vaddr.c")
+DISARM_LINE=$(awk '/^void damon_perf_event_disarm\(/ { in_fn=1 } \
+    in_fn && /ops->disarm/ { print NR; exit }' "$ROOT/mm/damon/vaddr.c")
+if [[ -n "$DISABLE_LINE" && -n "$DISARM_LINE" && "$DISABLE_LINE" -lt "$DISARM_LINE" ]]; then
+    pass "Lifecycle: ops->disarm() after perf_event_disable()"
+else
+    fail "Lifecycle: disarm() must follow perf_event_disable()"
+fi
+
+# The stop path calls the common check after disarming.  That common check
+# must drain AUX before it starts consuming the SPSC rings.
+DISARM_LOOP=$(grep -n 'damon_perf_event_disarm' "$ROOT/mm/damon/core.c" 2>/dev/null | \
+	tail -1 | cut -d: -f1)
+FINAL_CHECK=$(awk -v start="$DISARM_LOOP" 'NR > start && \
+    /kdamond_check_reported_accesses\(ctx\)/ { print NR; exit }' \
+    "$ROOT/mm/damon/core.c")
+if [[ -n "$DISARM_LOOP" && -n "$FINAL_CHECK" && \
+      "$DISARM_LOOP" -lt "$FINAL_CHECK" ]]; then
+    pass "Lifecycle: final common drain after disarm"
+else
+    fail "Lifecycle: final common drain must follow disarm"
+fi
+
+# Per-tick and final: AUX must publish before the SPSC loop reads rings.
+CHECK_FN=$(grep -n '^static unsigned int kdamond_check_reported_accesses' \
+    "$ROOT/mm/damon/core.c" | cut -d: -f1)
+DRAIN_TICK=$(awk -v start="$CHECK_FN" 'NR > start && \
+    /damon_perf_aux_drain\(ctx\)/ { print NR; exit }' "$ROOT/mm/damon/core.c")
+RING_DRAIN=$(awk -v start="$CHECK_FN" 'NR > start && \
+    /for_each_online_cpu\(cpu\)/ { print NR; exit }' "$ROOT/mm/damon/core.c")
+if [[ -n "$DRAIN_TICK" && -n "$RING_DRAIN" && \
+      "$DRAIN_TICK" -lt "$RING_DRAIN" ]]; then
+    pass "Tick: AUX drain before SPSC ring drain"
+else
+    fail "Tick: AUX drain must precede the ring drain"
+fi
+
+# Per-(event,cpu) parser state scoped via aux_priv.
+if grep -q 'aux_priv' "$ROOT/mm/damon/ops-common.h" 2>/dev/null; then
+    pass "State scoping: per-(event,cpu) via aux_priv"
+else
+    fail "State scoping: aux_priv in ops-common.h"
+fi
+
+# SPSC ring contract: kdamond is the only process-context producer and
+# writes only to its own CPU's ring.  A remote-write helper must not
+# exist.
+if grep -q 'damon_report_access_on_cpu' "$ROOT/mm/damon/core.c" 2>/dev/null; then
+    fail "SPSC: damon_report_access_on_cpu() must not exist"
+else
+    pass "SPSC: no remote-writer helper (current-CPU NMI-safe enqueue only)"
+fi
+if grep -q '^void damon_report_access' "$ROOT/mm/damon/core.c" 2>/dev/null; then
+    pass "SPSC: damon_report_access() present"
+else
+    fail "SPSC: damon_report_access() missing"
+fi
+fi
+
+# ---- Section 4: Runtime smoke test ----
+echo ""
+echo "--- 7. Runtime Smoke Test (live DAMON + SPE) ---"
+
+CREATED=0
+WORK_PID=""
+NR_SAVED=""
+if [[ -f "$ADMIN/kdamonds/nr_kdamonds" ]]; then
+    NR_SAVED=$(cat "$ADMIN/kdamonds/nr_kdamonds")
+fi
+
+restore_damon_state() {
+    # Tear down only what this test created, then restore the saved
+    # number of kdamonds.  A kdamond that was running before the test
+    # is never touched (section 7 skips in that case).
+    if [[ "$CREATED" == "1" ]]; then
+        echo off > "$ADMIN/kdamonds/0/state" 2>/dev/null || true
+        echo 0 > "$ADMIN/kdamonds/nr_kdamonds" 2>/dev/null || true
+        if [[ -n "$NR_SAVED" && "$NR_SAVED" -gt 0 ]]; then
+            echo "$NR_SAVED" > "$ADMIN/kdamonds/nr_kdamonds" 2>/dev/null || true
+        fi
+    fi
+    if [[ -n "$WORK_PID" ]]; then
+        kill "$WORK_PID" 2>/dev/null || true
+        wait "$WORK_PID" 2>/dev/null || true
+        WORK_PID=""
+    fi
+}
+trap restore_damon_state EXIT
+
+if [[ -z "$NR_SAVED" ]]; then
+    skip "runtime smoke" "DAMON admin interface not available"
+elif [[ "$NR_SAVED" -gt 0 ]]; then
+    skip "runtime smoke" "existing kdamonds present (nr_kdamonds=$NR_SAVED);" \
+        "refusing to disturb them"
+elif [[ ! -d "/sys/bus/event_source/devices/$PMU_NAME" ]]; then
+    skip "runtime smoke" "no $PMU_NAME PMU (kernel without ARM SPE or not booted with it)"
+else
+    SPE_TYPE=$(cat "/sys/bus/event_source/devices/$PMU_NAME/type")
+    pass "SPE PMU $PMU_NAME present (type=$SPE_TYPE)"
+    DMESG_LINES_BEFORE=$(dmesg 2>/dev/null | wc -l)
+
+    # Keep a userspace memory workload alive as both the DAMON target and
+    # an SPE data source.  The EXIT trap owns and terminates only this PID.
+    dd if=/dev/zero of=/dev/null bs=1M 2>/dev/null &
+    WORK_PID=$!
+
+    # arm_spe_pmu_event_init() rejects freq mode, so the event must be
+    # configured in period mode.
+    echo 1 > "$ADMIN/kdamonds/nr_kdamonds"
+    CREATED=1
+    echo 1 > "$ADMIN/kdamonds/0/contexts/nr_contexts"
+    echo 1 > "$ADMIN/kdamonds/0/contexts/0/targets/nr_targets"
+    echo "$WORK_PID" > "$ADMIN/kdamonds/0/contexts/0/targets/0/pid_target"
+    echo 1 > "$ADMIN/kdamonds/0/contexts/0/monitoring_attrs/sample/perf_events/nr_perf_events"
+    echo "$SPE_TYPE" > "$ADMIN/kdamonds/0/contexts/0/monitoring_attrs/sample/perf_events/0/type"
+    echo 0 > "$ADMIN/kdamonds/0/contexts/0/monitoring_attrs/sample/perf_events/0/freq"
+    echo 256 > "$ADMIN/kdamonds/0/contexts/0/monitoring_attrs/sample/perf_events/0/sample_period"
+    echo 3 > "$ADMIN/kdamonds/0/contexts/0/monitoring_attrs/sample/perf_events/0/config"
+    PE="$ADMIN/kdamonds/0/contexts/0/monitoring_attrs/sample/perf_events"
+    FREQ_VAL=$(cat "$PE/0/freq" 2>/dev/null || true)
+    if [[ "$FREQ_VAL" == "0" ]]; then
+        pass "SPE event in period mode (freq=0, period=256)"
+    else
+        fail "SPE event must use period mode (freq=0), read back $FREQ_VAL"
+    fi
+
+    STATS="/sys/kernel/debug/damon/perf_stats"
+    if [[ -r "$STATS" ]]; then
+        cp "$STATS" "$RESULTS_DIR/perf_stats-before.log"
+    fi
+
+    echo on > "$ADMIN/kdamonds/0/state"
+    sleep 5
+
+    STATE_NOW=$(cat "$ADMIN/kdamonds/0/state" 2>/dev/null || true)
+    if [[ "$STATE_NOW" == "on" ]]; then
+        pass "kdamond with SPE event is running"
+    else
+        fail "kdamond with SPE event failed to start (state=$STATE_NOW)"
+    fi
+
+    if [[ -r "$STATS" && -f "$RESULTS_DIR/perf_stats-before.log" ]]; then
+        cp "$STATS" "$RESULTS_DIR/perf_stats-running.log"
+    fi
+
+    # Stop explicitly so the final snapshot covers disable -> AUX drain ->
+    # SPSC ring drain, rather than leaving that path to the EXIT trap.
+    echo off > "$ADMIN/kdamonds/0/state"
+    for _ in $(seq 1 50); do
+        [[ "$(cat "$ADMIN/kdamonds/0/state" 2>/dev/null || true)" == "off" ]] && break
+        sleep 0.1
+    done
+    STATE_NOW=$(cat "$ADMIN/kdamonds/0/state" 2>/dev/null || true)
+    if [[ "$STATE_NOW" == "off" ]]; then
+        pass "kdamond stopped after final drain"
+    else
+        fail "kdamond did not stop (state=$STATE_NOW)"
+    fi
+
+    # Check only messages added during this run, not stale boot history.
+    DMESG_AFTER="$RESULTS_DIR/dmesg-after.log"
+    dmesg 2>/dev/null > "$DMESG_AFTER" || true
+    DMESG_LINES_AFTER=$(wc -l < "$DMESG_AFTER")
+    if [[ "$DMESG_LINES_AFTER" -ge "$DMESG_LINES_BEFORE" ]]; then
+        NEW_DMESG=$(tail -n "+$((DMESG_LINES_BEFORE + 1))" "$DMESG_AFTER")
+    else
+        NEW_DMESG=$(cat "$DMESG_AFTER")
+    fi
+    FAIL_LINES=$(printf '%s\n' "$NEW_DMESG" | \
+        grep -Ei "damon-perf.*(fail|warn|error)|WARNING:|BUG:|Oops:|lockdep" || true)
+    if [[ -n "$FAIL_LINES" ]]; then
+        fail "no damon-perf failures in dmesg"
+        echo "$FAIL_LINES" | sed 's/^/    /'
+    else
+        pass "no damon-perf failures in dmesg"
+    fi
+
+    if [[ -r "$STATS" && -f "$RESULTS_DIR/perf_stats-before.log" ]]; then
+        cp "$STATS" "$RESULTS_DIR/perf_stats-final.log"
+        pass "debugfs perf_stats readable"
+
+        stat_value() {
+            awk -v name="$2" '$1 == name { print $2; exit }' "$1"
+        }
+        for counter in callback valid enqueue dequeue match update; do
+            before=$(stat_value "$RESULTS_DIR/perf_stats-before.log" "$counter")
+            after=$(stat_value "$RESULTS_DIR/perf_stats-final.log" "$counter")
+            if [[ "$before" =~ ^[0-9]+$ && "$after" =~ ^[0-9]+$ ]]; then
+                delta=$((after - before))
+            else
+                delta=""
+            fi
+            if [[ "$delta" =~ ^[0-9]+$ && "$delta" -gt 0 ]]; then
+                pass "AUX pipeline: $counter delta=$delta"
+            else
+                fail "AUX pipeline: expected positive $counter delta, got ${delta:-missing}"
+            fi
+        done
+
+        enqueue_before=$(stat_value "$RESULTS_DIR/perf_stats-before.log" enqueue)
+        enqueue_final=$(stat_value "$RESULTS_DIR/perf_stats-final.log" enqueue)
+        dequeue_before=$(stat_value "$RESULTS_DIR/perf_stats-before.log" dequeue)
+        dequeue_final=$(stat_value "$RESULTS_DIR/perf_stats-final.log" dequeue)
+        enqueue_delta=$((enqueue_final - enqueue_before))
+        dequeue_delta=$((dequeue_final - dequeue_before))
+        if [[ "$enqueue_delta" -eq "$dequeue_delta" ]]; then
+            pass "final ring closure: enqueue=$enqueue_delta dequeue=$dequeue_delta"
+        else
+            fail "final ring closure: enqueue=$enqueue_delta dequeue=$dequeue_delta"
+        fi
+    else
+        fail "debugfs perf_stats unavailable; cannot validate AUX data path"
+    fi
+    # Cleanup happens in the EXIT trap (restore_damon_state).
+fi
+
+# ---- Summary ----
+echo ""
+echo "=========================================="
+echo " SUMMARY: $PASSED passed, $FAILED failed, $SKIPPED skipped"
+echo "=========================================="
+echo "Results saved to: $RESULTS_DIR"
+
+if [[ "$FAILED" -gt 0 ]]; then
+    echo "Overall: FAIL"
+    exit 1
+else
+    echo "Overall: PASS"
+    exit 0
+fi
-- 
2.43.0

Re: [RFC PATCH 0/4] mm/damon/perf: add ARM SPE AUX backend

[SJ Park]

Hello Kunwu,

On Sun, 16 Aug 2026 22:22:17 +0800 Kunwu Chan <kunwu.chan@gmail.com> wrote:

> From: Kunwu Chan <kunwu.chan@gmail.com>
> 
> This series adds an AUX trace-buffer backend to the DAMON perf
> observability framework, enabling ARM SPE (Statistical Profiling
> Extension) to deliver hardware-sampled access reports into DAMON's
> existing SPSC report ring.

Awesome.  Thank you for making this.  This will make DAMON be more useful on
ARM machines.

> 
> Patch 2 touches drivers/perf/arm_spe_pmu.c to expose the PMU matcher.
> This change is tightly coupled with the DAMON AUX backend.  ARM SPE
> PMU driver maintainers only need to review patch 2.
> 
> This series is based on the Ravi's hardware-sampled access reports 
> branch [1], and depends on the perf AUX kernel-consumer RFC series [2].
> The dependencies is not upstream yet, so this series remains RFC.

Unfortunately I haven't had a time to thoroughly read the dependent patches.  I
believe the basic idea of this patch series is aligned with the ongoing DAMON
extension roadmap [1], though.  That is, this series is supposed to be a part
of the milestone 3.  We are currently at the near end of milestone 1, and aim
to complete milestone 2 by the time around next year's LSFMMBPF.

I will hold review of this series for now, and take more time on making
milestones 1 and 2 for thier planned delivery timeline.  Sorry for being a
bottleneck of your work, and thanks in advance for your patience.

Please feel free to let me know if you have specific parts that need my review
right now, though.  Otherwise, please make sure this is aligned with the
roadmap until we finish the milestone 2.  I do want ARM SPE support of DAMON.

[1] https://lore.kernel.org/all/20260525225208.1179-1-sj@kernel.org/


Thanks,
SJ

[...]