* [PATCH 1/3] mm/damon/core: export symbols for supporting loadable modules
2023-12-05 2:28 [PATCH 0/3] mm/damon: export DAMON symbols and add sample loadable modules SeongJae Park
@ 2023-12-05 2:28 ` SeongJae Park
2023-12-05 2:28 ` [PATCH 2/3] samples: add working set size estimation DAMON sample module SeongJae Park
` (2 subsequent siblings)
3 siblings, 0 replies; 6+ messages in thread
From: SeongJae Park @ 2023-12-05 2:28 UTC (permalink / raw)
To: Andrew Morton; +Cc: SeongJae Park, damon, linux-mm, linux-kernel
Expose minimum DAMON core symbols for supporting use of basic
functionality from loadable modules. Followup commits will add sample
loadable modules that use the exported symbols.
Signed-off-by: SeongJae Park <sj@kernel.org>
---
mm/damon/core.c | 10 ++++++++++
1 file changed, 10 insertions(+)
diff --git a/mm/damon/core.c b/mm/damon/core.c
index f91715a58dc7..8a5c3e905738 100644
--- a/mm/damon/core.c
+++ b/mm/damon/core.c
@@ -111,6 +111,7 @@ int damon_select_ops(struct damon_ctx *ctx, enum damon_ops_id id)
mutex_unlock(&damon_ops_lock);
return err;
}
+EXPORT_SYMBOL(damon_select_ops);
/*
* Construct a damon_region struct
@@ -343,6 +344,7 @@ struct damos *damon_new_scheme(struct damos_access_pattern *pattern,
return scheme;
}
+EXPORT_SYMBOL(damon_new_scheme);
static void damos_set_next_apply_sis(struct damos *s, struct damon_ctx *ctx)
{
@@ -380,6 +382,7 @@ void damon_destroy_scheme(struct damos *s)
damon_del_scheme(s);
damon_free_scheme(s);
}
+EXPORT_SYMBOL(damon_destroy_scheme);
/*
* Construct a damon_target struct
@@ -401,11 +404,13 @@ struct damon_target *damon_new_target(void)
return t;
}
+EXPORT_SYMBOL(damon_new_target);
void damon_add_target(struct damon_ctx *ctx, struct damon_target *t)
{
list_add_tail(&t->list, &ctx->adaptive_targets);
}
+EXPORT_SYMBOL(damon_add_target);
bool damon_targets_empty(struct damon_ctx *ctx)
{
@@ -464,6 +469,7 @@ struct damon_ctx *damon_new_ctx(void)
return ctx;
}
+EXPORT_SYMBOL(damon_new_ctx);
static void damon_destroy_targets(struct damon_ctx *ctx)
{
@@ -489,6 +495,7 @@ void damon_destroy_ctx(struct damon_ctx *ctx)
kfree(ctx);
}
+EXPORT_SYMBOL(damon_destroy_ctx);
static unsigned int damon_age_for_new_attrs(unsigned int age,
struct damon_attrs *old_attrs, struct damon_attrs *new_attrs)
@@ -616,6 +623,7 @@ void damon_set_schemes(struct damon_ctx *ctx, struct damos **schemes,
for (i = 0; i < nr_schemes; i++)
damon_add_scheme(ctx, schemes[i]);
}
+EXPORT_SYMBOL(damon_set_schemes);
/**
* damon_nr_running_ctxs() - Return number of currently running contexts.
@@ -719,6 +727,7 @@ int damon_start(struct damon_ctx **ctxs, int nr_ctxs, bool exclusive)
return err;
}
+EXPORT_SYMBOL(damon_start);
/*
* __damon_stop() - Stops monitoring of a given context.
@@ -762,6 +771,7 @@ int damon_stop(struct damon_ctx **ctxs, int nr_ctxs)
}
return err;
}
+EXPORT_SYMBOL(damon_stop);
/*
* Reset the aggregated monitoring results ('nr_accesses' of each region).
--
2.34.1
^ permalink raw reply related [flat|nested] 6+ messages in thread* [PATCH 2/3] samples: add working set size estimation DAMON sample module
2023-12-05 2:28 [PATCH 0/3] mm/damon: export DAMON symbols and add sample loadable modules SeongJae Park
2023-12-05 2:28 ` [PATCH 1/3] mm/damon/core: export symbols for supporting " SeongJae Park
@ 2023-12-05 2:28 ` SeongJae Park
2023-12-05 2:28 ` [PATCH 3/3] samples: add proactive reclamation " SeongJae Park
2023-12-05 5:12 ` [PATCH 0/3] mm/damon: export DAMON symbols and add sample loadable modules Christoph Hellwig
3 siblings, 0 replies; 6+ messages in thread
From: SeongJae Park @ 2023-12-05 2:28 UTC (permalink / raw)
To: Andrew Morton; +Cc: SeongJae Park, damon, linux-mm, linux-kernel
Add a sample DAMON application kernel module, namely damon_sample_wsse.
It is same to the one of the example kernel modules that presented at
kernel summit 2021 with 10 minutes live coding[1] but updated for the
latest DAMON interface.
The sample module receives a pid of a process to monitor the access,
monitors the access to the virtual address space of the process,
calculate estimated working set size, and repeatedly print it on the
kernel log.
[1] https://linuxplumbersconf.org/event/11/contributions/984/
Signed-off-by: SeongJae Park <sj@kernel.org>
---
MAINTAINERS | 1 +
samples/Kconfig | 2 +
samples/Makefile | 2 +
samples/damon/Kconfig | 17 +++++++
samples/damon/Makefile | 2 +
samples/damon/damon_sample_wsse.c | 85 +++++++++++++++++++++++++++++++
6 files changed, 109 insertions(+)
create mode 100644 samples/damon/Kconfig
create mode 100644 samples/damon/Makefile
create mode 100644 samples/damon/damon_sample_wsse.c
diff --git a/MAINTAINERS b/MAINTAINERS
index d1fcae09c910..5a44258810ef 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -5764,6 +5764,7 @@ F: Documentation/mm/damon/
F: include/linux/damon.h
F: include/trace/events/damon.h
F: mm/damon/
+F: samples/damon/
F: tools/testing/selftests/damon/
DAVICOM FAST ETHERNET (DMFE) NETWORK DRIVER
diff --git a/samples/Kconfig b/samples/Kconfig
index b288d9991d27..8d5a36f0e5d6 100644
--- a/samples/Kconfig
+++ b/samples/Kconfig
@@ -293,6 +293,8 @@ config SAMPLE_CGROUP
source "samples/rust/Kconfig"
+source "samples/damon/Kconfig"
+
endif # SAMPLES
config HAVE_SAMPLE_FTRACE_DIRECT
diff --git a/samples/Makefile b/samples/Makefile
index b85fa64390c5..5af6bb8afb07 100644
--- a/samples/Makefile
+++ b/samples/Makefile
@@ -39,3 +39,5 @@ obj-$(CONFIG_SAMPLE_KMEMLEAK) += kmemleak/
obj-$(CONFIG_SAMPLE_CORESIGHT_SYSCFG) += coresight/
obj-$(CONFIG_SAMPLE_FPROBE) += fprobe/
obj-$(CONFIG_SAMPLES_RUST) += rust/
+obj-$(CONFIG_SAMPLE_DAMON_WSSE) += damon/
+obj-$(CONFIG_SAMPLE_DAMON_PRCL) += damon/
diff --git a/samples/damon/Kconfig b/samples/damon/Kconfig
new file mode 100644
index 000000000000..f8b9a73717f9
--- /dev/null
+++ b/samples/damon/Kconfig
@@ -0,0 +1,17 @@
+# SPDX-License-Identifier: GPL-2.0
+
+menu "DAMON Samples"
+
+config SAMPLE_DAMON_WSSE
+ tristate "DAMON sameple module for working set size estimation"
+ depends on DAMON && DAMON_VADDR
+ help
+ This builds DAMON sample module for working set size estimation.
+
+ The module receives pid of a process, monitor access to the virtual
+ address space of the process, calculate the estimated working set
+ size of the process, and repeatedly print the size on the kernel log.
+
+ If unsure, say N.
+
+endmenu
diff --git a/samples/damon/Makefile b/samples/damon/Makefile
new file mode 100644
index 000000000000..1ace3ad4aff0
--- /dev/null
+++ b/samples/damon/Makefile
@@ -0,0 +1,2 @@
+# SPDX-License-Identifier: GPL-2.0
+obj-$(CONFIG_SAMPLE_DAMON_WSSE) += damon_sample_wsse.o
diff --git a/samples/damon/damon_sample_wsse.c b/samples/damon/damon_sample_wsse.c
new file mode 100644
index 000000000000..897e6b136a1a
--- /dev/null
+++ b/samples/damon/damon_sample_wsse.c
@@ -0,0 +1,85 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * working set size estimation: monitor access pattern of given process and
+ * print estimated working set size (total size of regions that showing some
+ * access).
+ */
+
+#define pr_fmt(fmt) "damon_sample_wsse: " fmt
+
+#include <linux/damon.h>
+#include <linux/init.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+
+static int target_pid __read_mostly;
+module_param(target_pid, int, 0600);
+
+static struct damon_ctx *ctx;
+static struct pid *target_pidp;
+
+static int damon_sample_wsse_after_aggregate(struct damon_ctx *c)
+{
+ struct damon_target *t;
+
+ damon_for_each_target(t, c) {
+ struct damon_region *r;
+ unsigned long wss = 0;
+
+ damon_for_each_region(r, t) {
+ if (r->nr_accesses > 0)
+ wss += r->ar.end - r->ar.start;
+ }
+ pr_info("wss: %lu\n", wss);
+ }
+ return 0;
+}
+
+static int __init damon_sample_wsse_init(void)
+{
+ struct damon_target *target;
+
+ pr_info("Hello\n");
+
+ ctx = damon_new_ctx();
+ if (!ctx)
+ return -ENOMEM;
+ if (damon_select_ops(ctx, DAMON_OPS_VADDR)) {
+ damon_destroy_ctx(ctx);
+ return -EINVAL;
+ }
+
+ target = damon_new_target();
+ if (!target) {
+ damon_destroy_ctx(ctx);
+ return -ENOMEM;
+ }
+ damon_add_target(ctx, target);
+ target_pidp = find_get_pid(target_pid);
+ if (!target_pidp) {
+ damon_destroy_ctx(ctx);
+ return -EINVAL;
+ }
+ target->pid = target_pidp;
+
+ ctx->callback.after_aggregation = damon_sample_wsse_after_aggregate;
+ return damon_start(&ctx, 1, true);
+}
+
+static void __exit damon_sample_wsse_exit(void)
+{
+ pr_info("Goodbye\n");
+ if (ctx) {
+ damon_stop(&ctx, 1);
+ damon_destroy_ctx(ctx);
+ }
+ if (target_pidp)
+ put_pid(target_pidp);
+}
+
+module_init(damon_sample_wsse_init);
+module_exit(damon_sample_wsse_exit);
+
+MODULE_LICENSE("GPL");
+MODULE_AUTHOR("SeongJae Park");
+MODULE_DESCRIPTION("DAMON sample module for working set size estimation");
--
2.34.1
^ permalink raw reply related [flat|nested] 6+ messages in thread* [PATCH 3/3] samples: add proactive reclamation DAMON sample module
2023-12-05 2:28 [PATCH 0/3] mm/damon: export DAMON symbols and add sample loadable modules SeongJae Park
2023-12-05 2:28 ` [PATCH 1/3] mm/damon/core: export symbols for supporting " SeongJae Park
2023-12-05 2:28 ` [PATCH 2/3] samples: add working set size estimation DAMON sample module SeongJae Park
@ 2023-12-05 2:28 ` SeongJae Park
2023-12-05 5:12 ` [PATCH 0/3] mm/damon: export DAMON symbols and add sample loadable modules Christoph Hellwig
3 siblings, 0 replies; 6+ messages in thread
From: SeongJae Park @ 2023-12-05 2:28 UTC (permalink / raw)
To: Andrew Morton; +Cc: SeongJae Park, damon, linux-mm, linux-kernel
Add a sample DAMON application kernel module, namely damon_sample_prcl.
It is same to the one of the example kernel modules that presented at
kernel summit 2021 with 10 minutes live coding[1] but updated for the
latest DAMON interface.
The sample module receives a pid of a process to monitor the access,
monitors the access to the virtual address space of the process, find
regions that not accessed and proactively reclaim those using DAMOS.
[1] https://linuxplumbersconf.org/event/11/contributions/984/
Signed-off-by: SeongJae Park <sj@kernel.org>
---
samples/damon/Kconfig | 13 ++++
samples/damon/Makefile | 1 +
samples/damon/damon_sample_prcl.c | 102 ++++++++++++++++++++++++++++++
3 files changed, 116 insertions(+)
create mode 100644 samples/damon/damon_sample_prcl.c
diff --git a/samples/damon/Kconfig b/samples/damon/Kconfig
index f8b9a73717f9..459042ce5958 100644
--- a/samples/damon/Kconfig
+++ b/samples/damon/Kconfig
@@ -14,4 +14,17 @@ config SAMPLE_DAMON_WSSE
If unsure, say N.
+config SAMPLE_DAMON_PRCL
+ tristate "DAMON sameple module for access-aware proactive reclamation"
+ depends on DAMON && DAMON_VADDR
+ help
+ This builds DAMON sample module for access-aware proactive
+ reclamation.
+
+ The module receives pid of a process, monitor access to the virtual
+ address space of the process, find memory regions that not accessed,
+ and proactively reclaim the regions.
+
+ If unsure, say N.
+
endmenu
diff --git a/samples/damon/Makefile b/samples/damon/Makefile
index 1ace3ad4aff0..509f6a619dc4 100644
--- a/samples/damon/Makefile
+++ b/samples/damon/Makefile
@@ -1,2 +1,3 @@
# SPDX-License-Identifier: GPL-2.0
obj-$(CONFIG_SAMPLE_DAMON_WSSE) += damon_sample_wsse.o
+obj-$(CONFIG_SAMPLE_DAMON_PRCL) += damon_sample_prcl.o
diff --git a/samples/damon/damon_sample_prcl.c b/samples/damon/damon_sample_prcl.c
new file mode 100644
index 000000000000..487196f18864
--- /dev/null
+++ b/samples/damon/damon_sample_prcl.c
@@ -0,0 +1,102 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * proactive reclamation: monitor access pattern of a given process, find
+ * regiosn that seems not accessed, and proactively page out the regions.
+ */
+
+#define pr_fmt(fmt) "damon_sample_prcl: " fmt
+
+#include <linux/damon.h>
+#include <linux/init.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+
+static int target_pid __read_mostly;
+module_param(target_pid, int, 0600);
+
+static struct damon_ctx *ctx;
+static struct pid *target_pidp;
+
+static int damon_sample_prcl_after_aggregate(struct damon_ctx *c)
+{
+ struct damon_target *t;
+
+ damon_for_each_target(t, c) {
+ struct damon_region *r;
+ unsigned long wss = 0;
+
+ damon_for_each_region(r, t) {
+ if (r->nr_accesses > 0)
+ wss += r->ar.end - r->ar.start;
+ }
+ pr_info("wss: %lu\n", wss);
+ }
+ return 0;
+}
+
+static int __init damon_sample_prcl_init(void)
+{
+ struct damon_target *target;
+ struct damos *scheme;
+
+ pr_info("Hello\n");
+
+ ctx = damon_new_ctx();
+ if (!ctx)
+ return -ENOMEM;
+ if (damon_select_ops(ctx, DAMON_OPS_VADDR)) {
+ damon_destroy_ctx(ctx);
+ return -EINVAL;
+ }
+
+ target = damon_new_target();
+ if (!target) {
+ damon_destroy_ctx(ctx);
+ return -ENOMEM;
+ }
+ damon_add_target(ctx, target);
+ target_pidp = find_get_pid(target_pid);
+ if (!target_pidp) {
+ damon_destroy_ctx(ctx);
+ return -EINVAL;
+ }
+ target->pid = target_pidp;
+
+ ctx->callback.after_aggregation = damon_sample_prcl_after_aggregate;
+
+ scheme = damon_new_scheme(
+ &(struct damos_access_pattern) {
+ .min_sz_region = PAGE_SIZE,
+ .max_sz_region = ULONG_MAX,
+ .min_nr_accesses = 0,
+ .max_nr_accesses = 0},
+ DAMOS_PAGEOUT,
+ 0,
+ &(struct damos_quota){},
+ &(struct damos_watermarks){});
+ if (!scheme) {
+ damon_destroy_ctx(ctx);
+ return -ENOMEM;
+ }
+ damon_set_schemes(ctx, &scheme, 1);
+
+ return damon_start(&ctx, 1, true);
+}
+
+static void __exit damon_sample_prcl_exit(void)
+{
+ pr_info("Goodbye\n");
+ if (ctx) {
+ damon_stop(&ctx, 1);
+ damon_destroy_ctx(ctx);
+ }
+ if (target_pidp)
+ put_pid(target_pidp);
+}
+
+module_init(damon_sample_prcl_init);
+module_exit(damon_sample_prcl_exit);
+
+MODULE_LICENSE("GPL");
+MODULE_AUTHOR("SeongJae Park");
+MODULE_DESCRIPTION("DAMON sample module for proactive reclamation");
--
2.34.1
^ permalink raw reply related [flat|nested] 6+ messages in thread* Re: [PATCH 0/3] mm/damon: export DAMON symbols and add sample loadable modules
2023-12-05 2:28 [PATCH 0/3] mm/damon: export DAMON symbols and add sample loadable modules SeongJae Park
` (2 preceding siblings ...)
2023-12-05 2:28 ` [PATCH 3/3] samples: add proactive reclamation " SeongJae Park
@ 2023-12-05 5:12 ` Christoph Hellwig
2023-12-05 17:23 ` SeongJae Park
3 siblings, 1 reply; 6+ messages in thread
From: Christoph Hellwig @ 2023-12-05 5:12 UTC (permalink / raw)
To: SeongJae Park; +Cc: Andrew Morton, damon, linux-mm, linux-kernel
On Tue, Dec 05, 2023 at 02:28:55AM +0000, SeongJae Park wrote:
> DAMON cannot be used from loadable modules since it is not exporting its
> symbols.
And that is a good thing. We should absolutely not allow random modules
probing MM internals. What you posted here is basically just adding
hooks without even real in-kernel users.
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: [PATCH 0/3] mm/damon: export DAMON symbols and add sample loadable modules
2023-12-05 5:12 ` [PATCH 0/3] mm/damon: export DAMON symbols and add sample loadable modules Christoph Hellwig
@ 2023-12-05 17:23 ` SeongJae Park
0 siblings, 0 replies; 6+ messages in thread
From: SeongJae Park @ 2023-12-05 17:23 UTC (permalink / raw)
To: Christoph Hellwig
Cc: SeongJae Park, Andrew Morton, damon, linux-mm, linux-kernel
Hi Christoph,
On 2023-12-04T21:12:47-08:00 Christoph Hellwig <hch@infradead.org> wrote:
> On Tue, Dec 05, 2023 at 02:28:55AM +0000, SeongJae Park wrote:
> > DAMON cannot be used from loadable modules since it is not exporting its
> > symbols.
>
> And that is a good thing. We should absolutely not allow random modules
> probing MM internals.
I agree.
> What you posted here is basically just adding hooks without even real
> in-kernel users.
I was thinking someone might be able to think even the sample module as real
usage since there was actually some questions about it from real users. That
said, those were more like questions than strong requests, so I still think
this change is somewhat better to be made for at least some folks, but I also
agree that this wouldn't be not really essential for everyone, and could be
only long term maintenance threat.
I personally don't have strong opinion, and thank you for raising your concern.
I will hold this patchset unless someone request this change again with good
rationale.
Btw, I know there were many concerns about unnecessarily exporting symbols, but
do we have a formal guideline or documentation about the requirements for
exporting symbols in sepcific subsystem? I was hoping to have such one so that
I could provide better answer to DAMON's loadable module support questions, but
I was unable to find such one with my poor searching skill. This reply could
be used for the purpose meanwhile, though, so appreciate again. :)
Thanks,
SJ
^ permalink raw reply [flat|nested] 6+ messages in thread