[RFC PATCH 0/6] mm/damon/core: support multi-source reports-based access monitoring

[RFC PATCH 0/6] mm/damon/core: support multi-source reports-based access monitoring

[SeongJae Park]

TL; DR: Extend DAMON core-operations set layers interface for operations
set driven report-based monitoring such as per-CPU and write-only access
monitoring.

Background
----------

Existing DAMON operations set implementations, namely paddr, vaddr, and
fvaddr, use Accessed bits of page tables as the main source of the
access information.  Accessed bits has some restrictions, though.  For
example, it cannot tell which CPU or CPU made the access, whether the
access was read or write, and which part of the mapped entity was really
accessed.  Also, it cannot capture accesses done without page table
walks, e.g.,  TLB hit case, or unmapped pages.  Exisiting DAMON
operations set can capture the accesses to unmapped pages utilizing
PG_Idle, though.

Depending on the use case, the limitations can be problematic.  Because
the issue stems from the nature of page table Accessed bit, utilizing
access information from different sources can mitigate the issue.  Page
faults, memory access instructions sampling interrupts, system calls, or
any information from other kernel space friends such as subsystems or
device drivers of CXL or GPUs could be examples of the different
sources.

DAMON separates its core and operations set layer.  Operations set layer
handles the low level access information handling, and the core layer
handles more high level works such as the region-based overhead/accuracy
control and access-aware system operations.  Hence we can extend DAMON
to use the different sources by implementing and using another DAMON
operations set.  The core layer features will still be available with
the new sources, without additional changes.

Nevertheless, the current interface between the core and the operations
set layers is optimized for the Accessed bits case.  Specifically, the
interface asks the operations set if a give part of memory has accessed
or not in a given time period (last sampling interval).  It is easy for
Accessed bit use case, since operations set can simply read the current
value of the Accessed bit, which is already well manageed for each
mapping entities by the memory management subsystems.  For some sources
other than Accessed bits, the operations set may need to keep the access
information on its own, and answer to the core layer's question by
reading its own access information collection.  Similar implementations
of such operations set internal access information may required for
multiple operations set implementations.

Core Layer Changes for Reporting-based Monitoring
-------------------------------------------------

Optimize such possible duplicated efforts, by updating DAMON core layer
to support real time access reporting.  The updated interface allows
operations set implementations to report (or, push) their information to
the core layer, on their preferred schedule.  DAMON core layer will
handle the reports by managing meta data and updating the final
monitoring results (DAMON regions) accordingly.

Also add another operations set callback to determine if a given access
report is eligible to be used for given operations set.  For example, if
the operations set implementation is for monitoring only specific CPU or
writes, the operations set could ask core layer to ignore reported
accesses that made by other CPUs, or was for reads.

How Per-CPU or Write-only Monitoring Can Be Implemented
-------------------------------------------------------

With these core layer changes, we can implement new DAMON operations set
that utilizes different information source for page table Accessed bits
restricted use cases.  For example, to utilize page faults information,
we could implement an operations set that installs the fake page fault
protections to access sampling pages per DAMON region for every sampling
interval.  For that, damon_operations->preapre_access_checks() callback
coudl be a good place to implement.  Then the developer would need to
further update PROT_NONE page fault handlers to report the access events
using the new report API, namely damon_report_access().  The report
could contain information about which CPU, process, or thread made the
access, and whether the access is for read or write.  And finally
implementing the report-ignore logic to ignore accesses made by CPUs of
no interest or reads, we can do per-CPU or write-only access monitoring.

Patches Sequence
----------------

The first patch introduces damon_report_access() that any kernel code
that can sleep can use, to report their access information on their
schedule.  The second patch adds DAMON core-operations set interface for
ignoring specific types of data access reports for the given operations
set configuration.  The third patch further update core layer to really
use the reported access information for making the monitoring results
(DAMON regions) for API callers and ABI users.  The fourth patch updates
virtual address operations set to ignore access reports for none-target
virtual spaces.

The fifth and sixth patches are for only showing possible future
extensions of the report struct for per-CPU and write-only monitoring.

Plan for Dropping RFC
---------------------

This patch series is an RFC for early sharing of the idea that also
shared on the last LSFMMBPF[1], as 'damon_report_access()' API plan.
I actually wanted to post this later, but recently I received a few
questiosn about this, so sharing this very early version.  The
implementation is pretty simple and unoptimized.  No real use case of
the changed interface (new operations set implementation) exists.  We
will further optimize the core layer implementation and add one or more
real operations set implementations that utilizing the report-based
interface, by the final version of this patch series.

[1] https://lwn.net/Articles/1016525/

SeongJae Park (6):
  mm/damon/core: introduce damon_report_access()
  mm/damon/core: add eliglble_report() ops callback
  mm/damon/core: check received access reports
  mm/damon/vaddr: impleement eligible_report() callback
  mm/damon: add node_id to damon_access_report
  mm/damon: add write field to damon_access_report

 include/linux/damon.h | 30 +++++++++++++++++
 mm/damon/core.c       | 76 +++++++++++++++++++++++++++++++++++++++++++
 mm/damon/vaddr.c      |  7 ++++
 3 files changed, 113 insertions(+)


base-commit: 78af14fb3cf19eea2f204650a0090aa2c7022257
-- 
2.39.5

[RFC PATCH 1/6] mm/damon/core: introduce damon_report_access()

[SeongJae Park]

DAMON core layer asks operations set layer about past access
information, on core layer's schedule.  In other words, core layer
"pulls" the information from the operations set layer.  This is
problematic for a case that the operations set layer have no time and
space to save the information until the core layer queries.

Add a new DAMON API function for reporting identified data accesses to
DAMON, on the identifiers' schedule.  In other words, it lets the
operations set layer to "push" the information to the core layer.  The
function internally uses mutex, so reporting kernel code should be safe
to sleep.

Signed-off-by: SeongJae Park <sj@kernel.org>
---
 include/linux/damon.h | 21 +++++++++++++++++++++
 mm/damon/core.c       | 35 +++++++++++++++++++++++++++++++++++
 2 files changed, 56 insertions(+)

diff --git a/include/linux/damon.h b/include/linux/damon.h
index 6370cf44486f..a2198909c903 100644
--- a/include/linux/damon.h
+++ b/include/linux/damon.h
@@ -104,6 +104,25 @@ struct damon_target {
 	struct list_head list;
 };
 
+/**
+ * struct damon_access_report - Represent single acces report information.
+ * @pid:		The PID of the virtual address space of the address.
+ *			NULL if it is of the physical address.
+ * @addr:		The start address of the reporting region.
+ * @size:		The size of the reporting region.
+ * @nr_accesses:	Number of detected accesses to the region.
+ *
+ * @pid could be stale, and hence shouldn't be de-referenced.
+ */
+struct damon_access_report {
+	struct pid *pid;
+	unsigned long addr;
+	unsigned long size;
+	int nr_accesses;
+/* private: */
+	unsigned long report_jiffies;	/* when this report is made */
+};
+
 /**
  * enum damos_action - Represents an action of a Data Access Monitoring-based
  * Operation Scheme.
@@ -961,6 +980,8 @@ int damon_stop(struct damon_ctx **ctxs, int nr_ctxs);
 int damon_call(struct damon_ctx *ctx, struct damon_call_control *control);
 int damos_walk(struct damon_ctx *ctx, struct damos_walk_control *control);
 
+void damon_report_access(struct damon_access_report *report);
+
 int damon_set_region_biggest_system_ram_default(struct damon_target *t,
 				unsigned long *start, unsigned long *end);
 
diff --git a/mm/damon/core.c b/mm/damon/core.c
index ea2a17b2dee7..b54ed91f2dce 100644
--- a/mm/damon/core.c
+++ b/mm/damon/core.c
@@ -24,6 +24,8 @@
 #define DAMON_MIN_REGION 1
 #endif
 
+#define DAMON_ACCESS_REPORTS_CAP 1000
+
 static DEFINE_MUTEX(damon_lock);
 static int nr_running_ctxs;
 static bool running_exclusive_ctxs;
@@ -33,6 +35,11 @@ static struct damon_operations damon_registered_ops[NR_DAMON_OPS];
 
 static struct kmem_cache *damon_region_cache __ro_after_init;
 
+static DEFINE_MUTEX(damon_access_reports_lock);
+static struct damon_access_report damon_access_reports[
+	DAMON_ACCESS_REPORTS_CAP];
+static int damon_access_reports_len;
+
 /* Should be called under damon_ops_lock with id smaller than NR_DAMON_OPS */
 static bool __damon_is_registered_ops(enum damon_ops_id id)
 {
@@ -1403,6 +1410,34 @@ int damos_walk(struct damon_ctx *ctx, struct damos_walk_control *control)
 	return 0;
 }
 
+/**
+ * damon_report_access() - Report identified access events to DAMON.
+ * @report:	The reporting access information.
+ *
+ * Report access events to DAMON.
+ *
+ * Context: May sleep.
+ *
+ * NOTE: we may be able to implement this as a lockless queue, and allow any
+ * context.  As the overhead is unknown, and region-based DAMON logics would
+ * guarantee the reports would be not made that frequently, let's start with
+ * this simple implementation, though.
+ */
+void damon_report_access(struct damon_access_report *report)
+{
+	struct damon_access_report *dst;
+
+	/* silently fail for races */
+	if (!mutex_trylock(&damon_access_reports_lock))
+		return;
+	dst = &damon_access_reports[damon_access_reports_len++];
+	if (damon_access_reports_len == DAMON_ACCESS_REPORTS_CAP)
+		damon_access_reports_len = 0;
+	*dst = *report;
+	dst->report_jiffies = jiffies;
+	mutex_unlock(&damon_access_reports_lock);
+}
+
 /*
  * Warn and fix corrupted ->nr_accesses[_bp] for investigations and preventing
  * the problem being propagated.
-- 
2.39.5

[RFC PATCH 2/6] mm/damon/core: add eliglble_report() ops callback

[SeongJae Park]

Not every reported access information will be eligible for all DAMON ops
and target.  For example, virtual address access report will be not
eligible for 'padr' ops, or monitoring targets for a process that
different from the process for the repor.  If it is for monitoring
accesses from specific CPU or write, reports from other CPUs or reads
should be ignored.  Add operations set callback for this report
eligibility checking.

Signed-off-by: SeongJae Park <sj@kernel.org>
---
 include/linux/damon.h | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/include/linux/damon.h b/include/linux/damon.h
index a2198909c903..9ec40ce7dde0 100644
--- a/include/linux/damon.h
+++ b/include/linux/damon.h
@@ -592,6 +592,7 @@ enum damon_ops_id {
  * @update:			Update operations-related data structures.
  * @prepare_access_checks:	Prepare next access check of target regions.
  * @check_accesses:		Check the accesses to target regions.
+ * @eligible_report:		Verify an access report for a target.
  * @get_scheme_score:		Get the score of a region for a scheme.
  * @apply_scheme:		Apply a DAMON-based operation scheme.
  * @target_valid:		Determine if the target is valid.
@@ -618,6 +619,8 @@ enum damon_ops_id {
  * last preparation and update the number of observed accesses of each region.
  * It should also return max number of observed accesses that made as a result
  * of its update.  The value will be used for regions adjustment threshold.
+ * @eligible_report should check if the given access report is eligible to be
+ * used by this operations set for the given target.
  * @get_scheme_score should return the priority score of a region for a scheme
  * as an integer in [0, &DAMOS_MAX_SCORE].
  * @apply_scheme is called from @kdamond when a region for user provided
@@ -635,6 +638,8 @@ struct damon_operations {
 	void (*update)(struct damon_ctx *context);
 	void (*prepare_access_checks)(struct damon_ctx *context);
 	unsigned int (*check_accesses)(struct damon_ctx *context);
+	bool (*eligible_report)(struct damon_access_report *report,
+			struct damon_target *t);
 	int (*get_scheme_score)(struct damon_ctx *context,
 			struct damon_target *t, struct damon_region *r,
 			struct damos *scheme);
-- 
2.39.5

[RFC PATCH 3/6] mm/damon/core: check received access reports

[SeongJae Park]

Reported access information is only saved in the core layer's internal
data structure.  Those are not really being used for final monitoring
results.  Update core layer access information (DAMON regions) using the
reported access information.

Signed-off-by: SeongJae Park <sj@kernel.org>
---
 mm/damon/core.c | 41 +++++++++++++++++++++++++++++++++++++++++
 1 file changed, 41 insertions(+)

diff --git a/mm/damon/core.c b/mm/damon/core.c
index b54ed91f2dce..59567d79af99 100644
--- a/mm/damon/core.c
+++ b/mm/damon/core.c
@@ -2472,6 +2472,46 @@ static void kdamond_init_ctx(struct damon_ctx *ctx)
 	}
 }
 
+static void kdamond_apply_access_report(struct damon_access_report *report,
+		struct damon_target *t, struct damon_ctx *ctx)
+{
+	struct damon_region *r;
+
+	if (ctx->ops.eligible_report && !ctx->ops.eligible_report(report, t))
+		return;
+
+	/* todo: make search faster, e.g., binary search? */
+	damon_for_each_region(r, t) {
+		if (report->addr < r->ar.start)
+			continue;
+		if (r->ar.end < report->addr + report->size)
+			continue;
+		r->nr_accesses += report->nr_accesses;
+	}
+}
+
+static void kdamond_check_reported_accesses(struct damon_ctx *ctx)
+{
+	int i;
+	struct damon_access_report *report;
+	struct damon_target *t;
+
+	mutex_lock(&damon_access_reports_lock);
+	for (i = 0; i < damon_access_reports_len; i++) {
+		report = &damon_access_reports[i];
+		if (time_before(report->report_jiffies,
+					jiffies -
+					usecs_to_jiffies(
+						ctx->attrs.sample_interval)))
+			continue;
+		if (report->pid && !damon_target_has_pid(ctx))
+			continue;
+		damon_for_each_target(t, ctx)
+			kdamond_apply_access_report(report, t, ctx);
+	}
+	mutex_unlock(&damon_access_reports_lock);
+}
+
 /*
  * The monitoring daemon that runs as a kernel thread
  */
@@ -2518,6 +2558,7 @@ static int kdamond_fn(void *data)
 		kdamond_usleep(sample_interval);
 		ctx->passed_sample_intervals++;
 
+		kdamond_check_reported_accesses(ctx);
 		if (ctx->ops.check_accesses)
 			max_nr_accesses = ctx->ops.check_accesses(ctx);
 
-- 
2.39.5

[RFC PATCH 4/6] mm/damon/vaddr: impleement eligible_report() callback

[SeongJae Park]

For [f]vaddr DAMON ops, access reports for virtual address space of a
process that different from that for a given DAMON target shouldn't be
used.  Implement an eligible_report() callback for [f]vaddr that does
the check.

Signed-off-by: SeongJae Park <sj@kernel.org>
---
 mm/damon/vaddr.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/mm/damon/vaddr.c b/mm/damon/vaddr.c
index 46554e49a478..9970f95585ed 100644
--- a/mm/damon/vaddr.c
+++ b/mm/damon/vaddr.c
@@ -610,6 +610,12 @@ static unsigned int damon_va_check_accesses(struct damon_ctx *ctx)
 	return max_nr_accesses;
 }
 
+static bool damon_va_eligible_report(struct damon_access_report *report,
+		struct damon_target *t)
+{
+	return report->pid == t->pid;
+}
+
 /*
  * Functions for the target validity check and cleanup
  */
@@ -710,6 +716,7 @@ static int __init damon_va_initcall(void)
 		.update = damon_va_update,
 		.prepare_access_checks = damon_va_prepare_access_checks,
 		.check_accesses = damon_va_check_accesses,
+		.eligible_report = damon_va_eligible_report,
 		.target_valid = damon_va_target_valid,
 		.cleanup = NULL,
 		.apply_scheme = damon_va_apply_scheme,
-- 
2.39.5

[RFC PATCH 5/6] mm/damon: add node_id to damon_access_report

[SeongJae Park]

In future, per-CPU or per-NUMA node monitoring might be required.  We
can add the information in damon_access_report and implement the
damon_report_access() callers filling the information.  And the new
operations set implementation for the per-CPU or per-NUMA node access
monitoring can show and ignore unnecessary reports via
damon_operations->eligible_report() callback.

This commit is for showing the first part of the idea.

Signed-off-by: SeongJae Park <sj@kernel.org>
---
 include/linux/damon.h | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/include/linux/damon.h b/include/linux/damon.h
index 9ec40ce7dde0..29223fea710f 100644
--- a/include/linux/damon.h
+++ b/include/linux/damon.h
@@ -111,6 +111,7 @@ struct damon_target {
  * @addr:		The start address of the reporting region.
  * @size:		The size of the reporting region.
  * @nr_accesses:	Number of detected accesses to the region.
+ * @node_id:		NUMA node that made the access.
  *
  * @pid could be stale, and hence shouldn't be de-referenced.
  */
@@ -119,6 +120,7 @@ struct damon_access_report {
 	unsigned long addr;
 	unsigned long size;
 	int nr_accesses;
+	int node_id;
 /* private: */
 	unsigned long report_jiffies;	/* when this report is made */
 };
-- 
2.39.5

[RFC PATCH 6/6] mm/damon: add write field to damon_access_report

[SeongJae Park]

This commit is for showing the first part of the idea for implementing
write-only access monitoring.

Signed-off-by: SeongJae Park <sj@kernel.org>
---
 include/linux/damon.h | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/include/linux/damon.h b/include/linux/damon.h
index 29223fea710f..bd5e3287f6f2 100644
--- a/include/linux/damon.h
+++ b/include/linux/damon.h
@@ -112,6 +112,7 @@ struct damon_target {
  * @size:		The size of the reporting region.
  * @nr_accesses:	Number of detected accesses to the region.
  * @node_id:		NUMA node that made the access.
+ * @write:		Whether the access is write.
  *
  * @pid could be stale, and hence shouldn't be de-referenced.
  */
@@ -121,6 +122,7 @@ struct damon_access_report {
 	unsigned long size;
 	int nr_accesses;
 	int node_id;
+	bool write;
 /* private: */
 	unsigned long report_jiffies;	/* when this report is made */
 };
-- 
2.39.5