[RFC PATCH 00/18] mm/damon: enable page level properties based access pattern monitoring

[RFC PATCH 00/18] mm/damon: enable page level properties based access pattern monitoring

[SeongJae Park]

TL; DR
======

This patch series eanble access monitoring based on page level
properties including their anonymousness, belonging cgroups and
pg-young-ness, by extending DAMOS stats and regions walk features with
region-internal DAMOS filters.

Background
==========

DAMON fundametally provides only access pattern information in region
granularity.  For some use cases, fixed and fine granularity information
based on non access pattern properties can be useful, though.  For
example, on systems having fast swap devices and slow storage devices,
DAMOS-based proactive reclaim need to be applied differently for
anonymous pages and file-backed pages.

DAMOS makes it possible via DAMOS filters that work in page level
properties including anonymousness, belonging cgroups, and
pg-young-ness.  Having only the fine-grained inforamtion without making
system operation behavior changes can be useful for tuning or monitoring
purposes.  But currently DAMON provides no way for that.

Idea
====

DAMOS has initially developed for only access-aware system operations.
But, efficient acces monitoring results querying is yet another major
usage of today's DAMOS.  DAMOS stats and regions walk, which exposes
accumulated counts and per-region monitoring results that filtered by
DAMOS parameters including target access pattern and quotas, are the key
features for that usage.  Special DAMOS action, DAMOS_STAT, was also
introduced to support doing only monitoring without making any system
operation behavioral changes.

Hence, the fine-grained information that already available for
access-aware system operation can be passed to users by connecting the
region-internal DAMOS filters with stats and regions walk features.

Design
======

Update the interface of DAMON operations set layer, which contains
region-internal DAMOS filters implementation, to report back the amount
of memory that passed the region-internal DAMOS filters to the core
layer.  On the core layer, account the operations set layer reported
stat with a per-scheme accumulated stat.  Also, pass the stat to regions
walk.  In this way, DAMON API users can efficiently get the fine-grained
information.

For the user-space, make DAMON sysfs interface collects the information
using DAMON core API, and expose those to new per-scheme stats file and
per-DAMOS-tried region properties file.

Practical Usages
================

With this patch series, DAMON users can query how many bytes of regions
of specific access temperature is backed by pages of specific type.  The
type can be any of DAMOS filter-supporting one, including anonymousness,
belonging cgroups, and pg-young-ness.  For example, users can visualize
access hotness-based page granulairty histogram for different cgroups,
backing content type, or youngness.  In future, it could be extended to
more types such as whether it is THP, position on LRU lists, etc.  This
can be useful for estimating benefits of a new or an existing
access-aware system optimizations without really committing the changes.

Patches Sequence
================

The patches are separated in four sequences.

First four patches add adding documentation of background knowledge.

Following three patches change the operations set layer interface to
report back the region-internal filter passed memory size, and make the
operations set implementations support the changed symantic.

Following five patches implement per-scheme accumulated stat for
region-internal filter-passed memory size on core API, DAMON sysfs
interface.  First two patches of those are for code change, and
following three patches are for documentation.

Finally, six patches implementing per-region region-internal
filter-passed memory size follows.  First three patches of these are for
code change of preparation, API, and DAMON sysfs interface parts.  The
final three patches are for documentation.

Misc Notes
==========

This patch series depend on another RFC patch series[1] that introduced
damos_call().  Some of patches here, particularly the one that adds
Documentation for damos_call() and changing
damos_call_control->walk_fn() invocation sequence, may be moved and
folded into the patch series when it drops RFC tag.

[1] https://lore.kernel.org/20241213215306.54778-1-sj@kernel.org


SeongJae Park (18):
  mm/damon: clarify trying vs applying on damos_stat kernel-doc comment
  Docs/mm/damon/design: document DAMOS regions walking
  Docs/mm/damon/design: add 'statistics' section
  Docs/admin-guide/mm/damon/usage: link damos stat design doc
  mm/damon: ask apply_scheme() to report filter-passed region-internal
    bytes
  mm/damon/paddr: report filter-passed bytes back for normal actions
  mm/damon/paddr: report filter-passed bytes back for DAMOS_STAT action
  mm/damon/core: implement per-scheme filter-passed bytes stat
  mm/damon/syfs-schemes: implement per-scheme filter-passed bytes stat
  Docs/mm/damon/design: document sz_ops_filter_passed
  Docs/admin-guide/mm/damon/usage: document sz_ops_filter_passed
  Docs/ABI/damon: document per-scheme filter-passed bytes stat file
  mm/damon/core: invoke damos_walk_control->walk_fn() after applying
    action
  mm/damon/core: pass per-region filter-passed bytes to
    damos_walk_control->walk_fn()
  mm/damon/sysfs-schemes: expose per-region filter-passed bytes
  Docs/mm/damon/design: document per-region sz_filter_passed stat
  Docs/admin-guide/mm/damon/usage: document sz_filtered_out of scheme
    tried region directories
  Docs/ABI/damon: document per-region DAMOS filter-passed bytes stat
    file

 .../ABI/testing/sysfs-kernel-mm-damon         | 13 ++++
 Documentation/admin-guide/mm/damon/usage.rst  | 29 ++++----
 Documentation/mm/damon/design.rst             | 53 ++++++++++++++
 include/linux/damon.h                         | 27 ++++++-
 mm/damon/core.c                               | 31 ++++----
 mm/damon/paddr.c                              | 70 +++++++++++++++----
 mm/damon/sysfs-common.h                       |  2 +-
 mm/damon/sysfs-schemes.c                      | 34 ++++++++-
 mm/damon/sysfs.c                              |  5 +-
 mm/damon/vaddr.c                              |  2 +-
 10 files changed, 216 insertions(+), 50 deletions(-)


base-commit: af113b0c56e321562fb85ef5da57f323c34b616b
-- 
2.39.5

[RFC PATCH 02/18] Docs/mm/damon/design: document DAMOS regions walking

[SeongJae Park]

DAMOS' regions walking feature is an important feature for efficiently
retrieving monitoring results or DAMOS-internal behavior.  Document it
on the design document.

Signed-off-by: SeongJae Park <sj@kernel.org>
---
 Documentation/mm/damon/design.rst | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/Documentation/mm/damon/design.rst b/Documentation/mm/damon/design.rst
index f9c50525bdbf..a577ae40e71c 100644
--- a/Documentation/mm/damon/design.rst
+++ b/Documentation/mm/damon/design.rst
@@ -540,6 +540,17 @@ To know how user-space can set the watermarks via :ref:`DAMON sysfs interface
 documentation.
 
 
+Regions Walking
+~~~~~~~~~~~~~~~
+
+DAMOS feature allowing users access each region that a DAMOS action has just
+applied.  Using this feature, DAMON :ref:`API <damon_design_api>` allows users
+access full properties of the regions including the access monitoring results.
+:ref:`DAMON sysfs interface <sysfs_interface>` also allows users read the data
+via special :ref:`files <sysfs_schemes_tried_regions>`.
+
+.. _damon_design_api:
+
 Application Programming Interface
 ---------------------------------
 
-- 
2.39.5

[RFC PATCH 03/18] Docs/mm/damon/design: add 'statistics' section

[SeongJae Park]

DAMOS stats are important feature for tuning of DAMOS-based access-aware
system operation, and efficient access pattern monitoring.  But not well
documented on the design document.  Add a section on the document.

Signed-off-by: SeongJae Park <sj@kernel.org>
---
 Documentation/mm/damon/design.rst | 37 +++++++++++++++++++++++++++++++
 1 file changed, 37 insertions(+)

diff --git a/Documentation/mm/damon/design.rst b/Documentation/mm/damon/design.rst
index a577ae40e71c..8710fa71ad71 100644
--- a/Documentation/mm/damon/design.rst
+++ b/Documentation/mm/damon/design.rst
@@ -539,6 +539,43 @@ To know how user-space can set the watermarks via :ref:`DAMON sysfs interface
 <sysfs_interface>`, refer to :ref:`filters <sysfs_filters>` part of the
 documentation.
 
+Statistics
+~~~~~~~~~~
+
+The statistics of DAMOS behaviors that designed to help monitoring, tuning and
+debugging of DAMOS.
+
+DAMOS accounts below statistics for each scheme, from the beginning of the
+scheme's execution.
+
+- ``nr_tried``: Total number of regions that the scheme is tried to be applied.
+- ``sz_trtied``: Total size of regions that the scheme is tried to be applied.
+- ``nr_applied``: Total number of regions that the scheme is applied.
+- ``sz_applied``: Total size of regions that the scheme is applied.
+- ``qt_exceeds``: Total number of times the quota of the scheme has exceeded.
+
+"A scheme is tried to be applied to a region" means DAMOS core logic determined
+the region is eligible to apply the scheme's :ref:`action
+<damon_design_damos_action>`.  The :ref:`access pattern
+<damon_design_damos_access_patter>`, :ref:`quotas <damon_design_damos_quotas>`,
+:ref:`watermarks <damon_design_damos_watermarks>`, and :ref:`filters
+<damon_design_damos_filters>` that handled on core logic could affect this.
+The core logic will only ask the underlying :ref:`operation set
+<damon_operations_set>` to do apply the action to the region, so whether the
+action is really applied or not is unclear.  That's why it is called "tried".
+
+"A scheme is applied to a region" means the :ref:`operation set
+<damon_operations_set>` has applied the action to at least a part of the
+region.  The :ref:`filters <damon_design_damos_filters>` that handled by the
+operation set, and the types of the :ref:`action <damon_design_damos_action>`
+and the pages of the region can affect this.  For example, if a filter is set
+to exclude anonymous pages and the region has only anonymous pages, or if the
+action is ``pageout`` while all pages of the region are unreclaimable, applying
+the action to the region will fail.
+
+To know how user-space can read the stats via :ref:`DAMON sysfs interface
+<sysfs_interface>`, refer to :ref:s`stats <sysfs_stats>` part of the
+documentation.
 
 Regions Walking
 ~~~~~~~~~~~~~~~
-- 
2.39.5

[RFC PATCH 04/18] Docs/admin-guide/mm/damon/usage: link damos stat design doc

[SeongJae Park]

DAMON sysfs usage document should focus on usage, rather than the detail
of the stat metric itself.  Add a link to the design document on usage
section for DAMOS stat.

Signed-off-by: SeongJae Park <sj@kernel.org>
---
 Documentation/admin-guide/mm/damon/usage.rst | 3 ++-
 Documentation/mm/damon/design.rst            | 2 ++
 2 files changed, 4 insertions(+), 1 deletion(-)

diff --git a/Documentation/admin-guide/mm/damon/usage.rst b/Documentation/admin-guide/mm/damon/usage.rst
index d9be9f7caa7d..af70f7244700 100644
--- a/Documentation/admin-guide/mm/damon/usage.rst
+++ b/Documentation/admin-guide/mm/damon/usage.rst
@@ -457,7 +457,8 @@ schemes/<N>/stats/
 DAMON counts the total number and bytes of regions that each scheme is tried to
 be applied, the two numbers for the regions that each scheme is successfully
 applied, and the total number of the quota limit exceeds.  This statistics can
-be used for online analysis or tuning of the schemes.
+be used for online analysis or tuning of the schemes.  Refer to :ref:`design
+doc <damon_design_damos_stat>` for more details about the stats.
 
 The statistics can be retrieved by reading the files under ``stats`` directory
 (``nr_tried``, ``sz_tried``, ``nr_applied``, ``sz_applied``, and
diff --git a/Documentation/mm/damon/design.rst b/Documentation/mm/damon/design.rst
index 8710fa71ad71..e2666124e382 100644
--- a/Documentation/mm/damon/design.rst
+++ b/Documentation/mm/damon/design.rst
@@ -539,6 +539,8 @@ To know how user-space can set the watermarks via :ref:`DAMON sysfs interface
 <sysfs_interface>`, refer to :ref:`filters <sysfs_filters>` part of the
 documentation.
 
+.. _damon_design_damos_stat:
+
 Statistics
 ~~~~~~~~~~
 
-- 
2.39.5

[RFC PATCH 10/18] Docs/mm/damon/design: document sz_ops_filter_passed

[SeongJae Park]

Document the new per-scheme accumulated stat for total bytes that passed
the operations set layer-handled DAMOS filters on the design document.

Signed-off-by: SeongJae Park <sj@kernel.org>
---
 Documentation/mm/damon/design.rst | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/Documentation/mm/damon/design.rst b/Documentation/mm/damon/design.rst
index e2666124e382..0620675a1e8d 100644
--- a/Documentation/mm/damon/design.rst
+++ b/Documentation/mm/damon/design.rst
@@ -552,6 +552,8 @@ scheme's execution.
 
 - ``nr_tried``: Total number of regions that the scheme is tried to be applied.
 - ``sz_trtied``: Total size of regions that the scheme is tried to be applied.
+- ``sz_ops_filter_passed``: Total bytes that passed operations set
+  layer-handled DAMOS filters.
 - ``nr_applied``: Total number of regions that the scheme is applied.
 - ``sz_applied``: Total size of regions that the scheme is applied.
 - ``qt_exceeds``: Total number of times the quota of the scheme has exceeded.
-- 
2.39.5

[RFC PATCH 11/18] Docs/admin-guide/mm/damon/usage: document sz_ops_filter_passed

[SeongJae Park]

Document the new per-scheme operations set layer-handled DAMOS filters
passed bytes statistic file on DAMON sysfs interface usage document.

Signed-off-by: SeongJae Park <sj@kernel.org>
---
 Documentation/admin-guide/mm/damon/usage.rst | 20 +++++++++-----------
 1 file changed, 9 insertions(+), 11 deletions(-)

diff --git a/Documentation/admin-guide/mm/damon/usage.rst b/Documentation/admin-guide/mm/damon/usage.rst
index af70f7244700..179a9060a32e 100644
--- a/Documentation/admin-guide/mm/damon/usage.rst
+++ b/Documentation/admin-guide/mm/damon/usage.rst
@@ -90,7 +90,7 @@ comma (",").
     │ │ │ │ │ │ │ :ref:`watermarks <sysfs_watermarks>`/metric,interval_us,high,mid,low
     │ │ │ │ │ │ │ :ref:`filters <sysfs_filters>`/nr_filters
     │ │ │ │ │ │ │ │ 0/type,matching,memcg_id
-    │ │ │ │ │ │ │ :ref:`stats <sysfs_schemes_stats>`/nr_tried,sz_tried,nr_applied,sz_applied,qt_exceeds
+    │ │ │ │ │ │ │ :ref:`stats <sysfs_schemes_stats>`/nr_tried,sz_tried,nr_applied,sz_applied,sz_ops_filter_passed,qt_exceeds
     │ │ │ │ │ │ │ :ref:`tried_regions <sysfs_schemes_tried_regions>`/total_bytes
     │ │ │ │ │ │ │ │ 0/start,end,nr_accesses,age
     │ │ │ │ │ │ │ │ ...
@@ -454,18 +454,16 @@ difference is applied to :ref:`stats <damos_stats>` and
 schemes/<N>/stats/
 ------------------
 
-DAMON counts the total number and bytes of regions that each scheme is tried to
-be applied, the two numbers for the regions that each scheme is successfully
-applied, and the total number of the quota limit exceeds.  This statistics can
-be used for online analysis or tuning of the schemes.  Refer to :ref:`design
-doc <damon_design_damos_stat>` for more details about the stats.
+DAMON counts statistics for each scheme.  This statistics can be used for
+online analysis or tuning of the schemes.  Refer to :ref:`design doc
+<damon_design_damos_stat>` for more details about the stats.
 
 The statistics can be retrieved by reading the files under ``stats`` directory
-(``nr_tried``, ``sz_tried``, ``nr_applied``, ``sz_applied``, and
-``qt_exceeds``), respectively.  The files are not updated in real time, so you
-should ask DAMON sysfs interface to update the content of the files for the
-stats by writing a special keyword, ``update_schemes_stats`` to the relevant
-``kdamonds/<N>/state`` file.
+(``nr_tried``, ``sz_tried``, ``nr_applied``, ``sz_applied``,
+``sz_ops_filter_passed``, and ``qt_exceeds``), respectively.  The files are not
+updated in real time, so you should ask DAMON sysfs interface to update the
+content of the files for the stats by writing a special keyword,
+``update_schemes_stats`` to the relevant ``kdamonds/<N>/state`` file.
 
 .. _sysfs_schemes_tried_regions:
 
-- 
2.39.5

[RFC PATCH 16/18] Docs/mm/damon/design: document per-region sz_filter_passed stat

[SeongJae Park]

Update 'Regions Walking' sectioin of design document for the newly added
per-region operations set handling DAMOS filters-passed bytes.

Signed-off-by: SeongJae Park <sj@kernel.org>
---
 Documentation/mm/damon/design.rst | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/Documentation/mm/damon/design.rst b/Documentation/mm/damon/design.rst
index 0620675a1e8d..ed3072dc8686 100644
--- a/Documentation/mm/damon/design.rst
+++ b/Documentation/mm/damon/design.rst
@@ -586,7 +586,8 @@ Regions Walking
 
 DAMOS feature allowing users access each region that a DAMOS action has just
 applied.  Using this feature, DAMON :ref:`API <damon_design_api>` allows users
-access full properties of the regions including the access monitoring results.
+access full properties of the regions including the access monitoring results
+and amount of the region's internal memory that passed the DAMOS filters.
 :ref:`DAMON sysfs interface <sysfs_interface>` also allows users read the data
 via special :ref:`files <sysfs_schemes_tried_regions>`.
 
-- 
2.39.5

[RFC PATCH 17/18] Docs/admin-guide/mm/damon/usage: document sz_filtered_out of scheme tried region directories

[SeongJae Park]

Document the newly added DAMON sysfs interface file for per-scheme-tried
region's bytes that passed the operations set handling DAMOS filters.

Signed-off-by: SeongJae Park <sj@kernel.org>
---
 Documentation/admin-guide/mm/damon/usage.rst | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/Documentation/admin-guide/mm/damon/usage.rst b/Documentation/admin-guide/mm/damon/usage.rst
index 179a9060a32e..a891e830c2a1 100644
--- a/Documentation/admin-guide/mm/damon/usage.rst
+++ b/Documentation/admin-guide/mm/damon/usage.rst
@@ -92,7 +92,7 @@ comma (",").
     │ │ │ │ │ │ │ │ 0/type,matching,memcg_id
     │ │ │ │ │ │ │ :ref:`stats <sysfs_schemes_stats>`/nr_tried,sz_tried,nr_applied,sz_applied,sz_ops_filter_passed,qt_exceeds
     │ │ │ │ │ │ │ :ref:`tried_regions <sysfs_schemes_tried_regions>`/total_bytes
-    │ │ │ │ │ │ │ │ 0/start,end,nr_accesses,age
+    │ │ │ │ │ │ │ │ 0/start,end,nr_accesses,age,sz_filter_passed
     │ │ │ │ │ │ │ │ ...
     │ │ │ │ │ │ ...
     │ │ │ │ ...
@@ -500,10 +500,10 @@ set the ``access pattern`` as their interested pattern that they want to query.
 tried_regions/<N>/
 ------------------
 
-In each region directory, you will find four files (``start``, ``end``,
-``nr_accesses``, and ``age``).  Reading the files will show the start and end
-addresses, ``nr_accesses``, and ``age`` of the region that corresponding
-DAMON-based operation scheme ``action`` has tried to be applied.
+In each region directory, you will find five files (``start``, ``end``,
+``nr_accesses``, ``age``, and ``sz_filter_passed``).  Reading the files will
+show the properties of the region that corresponding DAMON-based operation
+scheme ``action`` has tried to be applied.
 
 Example
 ~~~~~~~
-- 
2.39.5

Re: [RFC PATCH 00/18] mm/damon: enable page level properties based access pattern monitoring

[SeongJae Park]

On Wed, 18 Dec 2024 20:03:09 -0800 SeongJae Park <sj@kernel.org> wrote:

> TL; DR
> ======
> 
> This patch series eanble access monitoring based on page level
> properties including their anonymousness, belonging cgroups and
> pg-young-ness, by extending DAMOS stats and regions walk features with
> region-internal DAMOS filters.

I just released DAMON user-space tool, damo, of v2.6.1.  It includes support of
this feature.  Using the user-space tool, I took and shared a simple
screenshot[1] that might help you better understand what users can do with
this.

[1] https://social.kernel.org/notice/ApLdd6IaUEHk3WhgTA


Thanks,
SJ

[...]