[PATCH 00/10] mm/damon: extend DAMOS filters for inclusion

[PATCH 00/10] mm/damon: extend DAMOS filters for inclusion

[SeongJae Park]

DAMOS fitlers are exclusive filters.  It only excludes memory of given
criterias from the DAMOS action targets.  This has below limitations.

First, the name is not explicitly explaining the behavior.  This
actually resulted in users' confusions[1].  Secondly, combined uses of
multiple filters provide restriced coverages.  For example, building a
DAMOS scheme that applies the action to memory that belongs to cgroup A
"or" cgroup B is impossible.  A workaround would be using two schemes
that fitlers out memory that not belong to cgroup A and cgroup B,
respectively.  It is cumbersome, and difficult to control quota-like
per-scheme features in orchastration.  Monitoring of filter-passed
memory statistic will also be complicated.

Extend DAMOS filters to support not only exclusion (blocking), but also
inclusion (pass-through) behavior.  For this, add a new damos_filter
struct field called 'pass' for DAMON kernel API users.  The filter works
as an inclusion filter or an exclusion filter when it is set or unset,
respectively.  For DAMON user-space ABI users, add a DAMON sysfs file of
same name under DAMOS filter sysfs directory.  To prevent exposing a
behavioral change to old users, set the blocking as the default
behavior.

Note that pass filters work for only inclusion, not exclusion of memory
that not satisfying the criteria.  And the default behavior of DAMOS for
memory that no filter has involved is that the action can be applied to
those.  This means that installing pass filters without block filters
after them is not useful for some usecases.  Read the design document
update part of this patch series for more details.

[1] https://lore.kernel.org/20240320165619.71478-1-sj@kernel.org

Changes from RFC v2
(https://lore.kernel.org/20241227210819.63776-1-sj@kernel.org)
- Wordsmith messages
- Wordsmith design documentation about monitoring-purpose usage
- Rebase on latest mm-unstable

Changes from RFC v1
(https://lore.kernel.org/20241226221445.78433-1-sj@kernel.org)
- Fix encoding issue on the last patch

SeongJae Park (10):
  mm/damon: fixup damos_filter kernel-doc
  mm/damon/core: add damos_filter->pass field
  mm/damon/core: support damos_filter->pass
  mm/damon/paddr: support damos_filter->pass
  mm/damon: add pass argument to damos_new_filter()
  mm/damon/sysfs-schemes: add a file for setting damos_filter->pass
  Docs/mm/damon/design: document pass/block filters behaviors
  Docs/ABI/damon: document DAMOS filter pass sysfs file
  Docs/admin-guide/mm/damon/usage: omit DAMOS filter details in favor of
    design doc
  Docs/admin-guide/mm/damon/usage: document DAMOS filter 'pass' sysfs
    file

 .../ABI/testing/sysfs-kernel-mm-damon         | 14 +++--
 Documentation/admin-guide/mm/damon/usage.rst  | 55 ++++++++++---------
 Documentation/mm/damon/design.rst             | 34 ++++++++++--
 include/linux/damon.h                         | 15 +++--
 mm/damon/core.c                               | 12 ++--
 mm/damon/paddr.c                              |  9 +--
 mm/damon/reclaim.c                            |  2 +-
 mm/damon/sysfs-schemes.c                      | 32 ++++++++++-
 mm/damon/tests/core-kunit.h                   | 14 ++---
 9 files changed, 130 insertions(+), 57 deletions(-)

-- 
2.39.5

[PATCH 07/10] Docs/mm/damon/design: document pass/block filters behaviors

[SeongJae Park]

Update DAMOS filters design document to describe the pass/block
behavior of filters.

Signed-off-by: SeongJae Park <sj@kernel.org>
---
 Documentation/mm/damon/design.rst | 34 +++++++++++++++++++++++++++----
 1 file changed, 30 insertions(+), 4 deletions(-)

diff --git a/Documentation/mm/damon/design.rst b/Documentation/mm/damon/design.rst
index 449eb33688c2..3682e719b8a6 100644
--- a/Documentation/mm/damon/design.rst
+++ b/Documentation/mm/damon/design.rst
@@ -504,9 +504,35 @@ have a list of latency-critical processes.
 
 To let users optimize DAMOS schemes with such special knowledge, DAMOS provides
 a feature called DAMOS filters.  The feature allows users to set an arbitrary
-number of filters for each scheme.  Each filter specifies the type of target
-memory, and whether it should exclude the memory of the type (filter-out), or
-all except the memory of the type (filter-in).
+number of filters for each scheme.  Each filter specifies
+
+- a type of memory (``type``),
+- whether it is for the memory of the type or all except the type
+  (``matching``), and
+- whether it is to allow (pass through the filter) or reject (block) applying
+  the scheme's action to the memory (``pass``).
+
+When multiple filters are installed, each filter is evaluated in the installed
+order.  If a part of memory is matched to one of the filter, next filters are
+ignored.  If a part of memory is not matched to any of the filters, the default
+behavior is applied.  That is, as long as it fulfills other conditions of the
+scheme including the access pattern and quotas, the action will be applied to
+the part of the memory.
+
+For example, let's assume 1) a filter for passing anonymous pages and 2)
+another filter for blocking young pages are installed in the order.  If a page
+of a region that eligible to apply the scheme's action is an anonymous page,
+the scheme's action will be applied to the page regardless of whether it is
+young or not, since it matches with the first filter.  If the page is not
+anonymous but young, the scheme's action will not be applied, since the second
+filter blocks it.  If the page is neither anonymous nor young, no filter is
+involved, so the action will be applied to the page.
+
+The fact that the action can be applied to any memory as long as no block
+filter explicitly excluded it means that installing pass filters without any
+block filter after those is same to not installing the pass filters, in terms
+of the ``action`` applying.  Statistics for DAMOS filters will still be
+accounted, though.  It is therefore still useful for monitoring purpose.
 
 For efficient handling of filters, some types of filters are handled by the
 core layer, while others are handled by operations set.  In the latter case,
@@ -516,7 +542,7 @@ filter are not counted as the scheme has tried to the region.  In contrast, if
 a memory regions is filtered by an operations set layer-handled filter, it is
 counted as the scheme has tried.  This difference affects the statistics.
 
-Below types of filters are currently supported.
+Below ``type`` of filters are currently supported.
 
 - anonymous page
     - Applied to pages that containing data that not stored in files.
-- 
2.39.5

[PATCH 09/10] Docs/admin-guide/mm/damon/usage: omit DAMOS filter details in favor of design doc

[SeongJae Park]

DAMON usage document is describing some details about DAMOS filters,
which are also documented on the design doc.  Deduplicate the details in
favor of the design doc.

Signed-off-by: SeongJae Park <sj@kernel.org>
---
 Documentation/admin-guide/mm/damon/usage.rst | 29 ++++++++++----------
 1 file changed, 14 insertions(+), 15 deletions(-)

diff --git a/Documentation/admin-guide/mm/damon/usage.rst b/Documentation/admin-guide/mm/damon/usage.rst
index 71cf29ae8502..8df3357dcfa3 100644
--- a/Documentation/admin-guide/mm/damon/usage.rst
+++ b/Documentation/admin-guide/mm/damon/usage.rst
@@ -411,13 +411,17 @@ Each filter directory contains six files, namely ``type``, ``matcing``,
 file, you can write one of five special keywords: ``anon`` for anonymous pages,
 ``memcg`` for specific memory cgroup, ``young`` for young pages, ``addr`` for
 specific address range (an open-ended interval), or ``target`` for specific
-DAMON monitoring target filtering.  In case of the memory cgroup filtering, you
-can specify the memory cgroup of the interest by writing the path of the memory
-cgroup from the cgroups mount point to ``memcg_path`` file.  In case of the
-address range filtering, you can specify the start and end address of the range
-to ``addr_start`` and ``addr_end`` files, respectively.  For the DAMON
-monitoring target filtering, you can specify the index of the target between
-the list of the DAMON context's monitoring targets list to ``target_idx`` file.
+DAMON monitoring target filtering.  Meaning of the types are same to the
+description on the :ref:`design doc <damon_design_damos_filters>`.
+
+In case of the memory cgroup filtering, you can specify the memory cgroup of
+the interest by writing the path of the memory cgroup from the cgroups mount
+point to ``memcg_path`` file.  In case of the address range filtering, you can
+specify the start and end address of the range to ``addr_start`` and
+``addr_end`` files, respectively.  For the DAMON monitoring target filtering,
+you can specify the index of the target between the list of the DAMON context's
+monitoring targets list to ``target_idx`` file.
+
 You can write ``Y`` or ``N`` to ``matching`` file to filter out pages that does
 or does not match to the type, respectively.  Then, the scheme's action will
 not be applied to the pages that specified to be filtered out.
@@ -434,14 +438,9 @@ pages of all memory cgroups except ``/having_care_already``.::
     echo /having_care_already > 1/memcg_path
     echo Y > 1/matching
 
-Note that ``anon`` and ``memcg`` filters are currently supported only when
-``paddr`` :ref:`implementation <sysfs_context>` is being used.
-
-Also, memory regions that are filtered out by ``addr`` or ``target`` filters
-are not counted as the scheme has tried to those, while regions that filtered
-out by other type filters are counted as the scheme has tried to.  The
-difference is applied to :ref:`stats <damos_stats>` and
-:ref:`tried regions <sysfs_schemes_tried_regions>`.
+Refer to the :ref:`DAMOS filters design documentation
+<damon_design_damos_filters>` for more details including when each of the
+filters are supported and differences on stats.
 
 .. _sysfs_schemes_stats:
 
-- 
2.39.5

[PATCH 10/10] Docs/admin-guide/mm/damon/usage: document DAMOS filter 'pass' sysfs file

[SeongJae Park]

Update DAMON usage document for the newly added 'pass' sysfs file for
DAMOS filters.

Signed-off-by: SeongJae Park <sj@kernel.org>
---
 Documentation/admin-guide/mm/damon/usage.rst | 34 ++++++++++++--------
 1 file changed, 20 insertions(+), 14 deletions(-)

diff --git a/Documentation/admin-guide/mm/damon/usage.rst b/Documentation/admin-guide/mm/damon/usage.rst
index 8df3357dcfa3..77064c704bf2 100644
--- a/Documentation/admin-guide/mm/damon/usage.rst
+++ b/Documentation/admin-guide/mm/damon/usage.rst
@@ -83,7 +83,7 @@ comma (",").
     │ │ │ │ │ │ │ │ │ 0/target_metric,target_value,current_value
     │ │ │ │ │ │ │ :ref:`watermarks <sysfs_watermarks>`/metric,interval_us,high,mid,low
     │ │ │ │ │ │ │ :ref:`filters <sysfs_filters>`/nr_filters
-    │ │ │ │ │ │ │ │ 0/type,matching,memcg_id
+    │ │ │ │ │ │ │ │ 0/type,matching,memcg_id,pass
     │ │ │ │ │ │ │ :ref:`stats <sysfs_schemes_stats>`/nr_tried,sz_tried,nr_applied,sz_applied,sz_ops_filter_passed,qt_exceeds
     │ │ │ │ │ │ │ :ref:`tried_regions <sysfs_schemes_tried_regions>`/total_bytes
     │ │ │ │ │ │ │ │ 0/start,end,nr_accesses,age,sz_filter_passed
@@ -406,13 +406,14 @@ number (``N``) to the file creates the number of child directories named ``0``
 to ``N-1``.  Each directory represents each filter.  The filters are evaluated
 in the numeric order.
 
-Each filter directory contains six files, namely ``type``, ``matcing``,
-``memcg_path``, ``addr_start``, ``addr_end``, and ``target_idx``.  To ``type``
-file, you can write one of five special keywords: ``anon`` for anonymous pages,
-``memcg`` for specific memory cgroup, ``young`` for young pages, ``addr`` for
-specific address range (an open-ended interval), or ``target`` for specific
-DAMON monitoring target filtering.  Meaning of the types are same to the
-description on the :ref:`design doc <damon_design_damos_filters>`.
+Each filter directory contains seven files, namely ``type``, ``matcing``,
+``pass``, ``memcg_path``, ``addr_start``, ``addr_end``, and ``target_idx``.  To
+``type`` file, you can write one of five special keywords: ``anon`` for
+anonymous pages, ``memcg`` for specific memory cgroup, ``young`` for young
+pages, ``addr`` for specific address range (an open-ended interval), or
+``target`` for specific DAMON monitoring target filtering.  Meaning of the
+types are same to the description on the :ref:`design doc
+<damon_design_damos_filters>`.
 
 In case of the memory cgroup filtering, you can specify the memory cgroup of
 the interest by writing the path of the memory cgroup from the cgroups mount
@@ -422,25 +423,30 @@ specify the start and end address of the range to ``addr_start`` and
 you can specify the index of the target between the list of the DAMON context's
 monitoring targets list to ``target_idx`` file.
 
-You can write ``Y`` or ``N`` to ``matching`` file to filter out pages that does
-or does not match to the type, respectively.  Then, the scheme's action will
-not be applied to the pages that specified to be filtered out.
+You can write ``Y`` or ``N`` to ``matching`` file to specify whether the filter
+is for memory that matches the ``type``.  You can write ``Y`` or ``N`` to
+``pass`` file to specify should this filter let the memory that satisfies the
+``type`` and ``matching`` pass though (allow) or be blocked by (reject) the
+filter.  Allowing means the scheme's action will be applied to the memory.
 
 For example, below restricts a DAMOS action to be applied to only non-anonymous
 pages of all memory cgroups except ``/having_care_already``.::
 
     # echo 2 > nr_filters
-    # # filter out anonymous pages
+    # # block anonymous pages
     echo anon > 0/type
     echo Y > 0/matching
+    echo N > 0/pass
     # # further filter out all cgroups except one at '/having_care_already'
     echo memcg > 1/type
     echo /having_care_already > 1/memcg_path
     echo Y > 1/matching
+    echo N > 1/pass
 
 Refer to the :ref:`DAMOS filters design documentation
-<damon_design_damos_filters>` for more details including when each of the
-filters are supported and differences on stats.
+<damon_design_damos_filters>` for more details including how multiple filters
+of different ``pass`` works, when each of the filters are supported, and
+differences on stats.
 
 .. _sysfs_schemes_stats:
 
-- 
2.39.5

Re: [PATCH 07/10] Docs/mm/damon/design: document pass/block filters behaviors

[SeongJae Park]

On Tue, 7 Jan 2025 12:17:36 -0800 SeongJae Park <sj@kernel.org> wrote:

> Update DAMOS filters design document to describe the pass/block
> behavior of filters.
> 
> Signed-off-by: SeongJae Park <sj@kernel.org>
> ---
>  Documentation/mm/damon/design.rst | 34 +++++++++++++++++++++++++++----
>  1 file changed, 30 insertions(+), 4 deletions(-)
> 
> diff --git a/Documentation/mm/damon/design.rst b/Documentation/mm/damon/design.rst
> index 449eb33688c2..3682e719b8a6 100644
> --- a/Documentation/mm/damon/design.rst
> +++ b/Documentation/mm/damon/design.rst
> @@ -504,9 +504,35 @@ have a list of latency-critical processes.
>  
>  To let users optimize DAMOS schemes with such special knowledge, DAMOS provides
>  a feature called DAMOS filters.  The feature allows users to set an arbitrary
> -number of filters for each scheme.  Each filter specifies the type of target
> -memory, and whether it should exclude the memory of the type (filter-out), or
> -all except the memory of the type (filter-in).
> +number of filters for each scheme.  Each filter specifies
> +
> +- a type of memory (``type``),
> +- whether it is for the memory of the type or all except the type
> +  (``matching``), and
> +- whether it is to allow (pass through the filter) or reject (block) applying
> +  the scheme's action to the memory (``pass``).
> +
> +When multiple filters are installed, each filter is evaluated in the installed
> +order.  If a part of memory is matched to one of the filter, next filters are
> +ignored.  If a part of memory is not matched to any of the filters, the default
> +behavior is applied.  That is, as long as it fulfills other conditions of the
> +scheme including the access pattern and quotas, the action will be applied to
> +the part of the memory.
> +
> +For example, let's assume 1) a filter for passing anonymous pages and 2)
> +another filter for blocking young pages are installed in the order.  If a page
> +of a region that eligible to apply the scheme's action is an anonymous page,
> +the scheme's action will be applied to the page regardless of whether it is
> +young or not, since it matches with the first filter.  If the page is not
> +anonymous but young, the scheme's action will not be applied, since the second
> +filter blocks it.  If the page is neither anonymous nor young, no filter is
> +involved, so the action will be applied to the page.
> +
> +The fact that the action can be applied to any memory as long as no block
> +filter explicitly excluded it means that installing pass filters without any
> +block filter after those is same to not installing the pass filters, in terms
> +of the ``action`` applying.  Statistics for DAMOS filters will still be
> +accounted, though.

The above last sentence is right in a sense, but not useful and could only
confuse readers.  The statistics for DAMOS filters are filters passed size
stat, which is provided as per-scheme accumulated stat and per-region instant
information.  The stat is for any memory that be able to apply the DAMOS action
after the filters check stage.  Hence, whether it has passed the stage due to
existence of a pass filter that matches the memory, or the absence of any
matching filter is distinguishable.

> It is therefore still useful for monitoring purpose.

Hence, the above sentence is completely wrong.  The case (installing pass
filtrs without any block filter after those) is not useful even for monitoring
purpose.

The RFC version of this patch was mentioning it correctly, but was not clearly
describing why it is not also useless for even monitoring purpose.  I was also
confused due to the absence of the context.  I will rewrite this part and send
the whole series again as v2.


Thanks,
SJ

[...]

Re: [PATCH 00/10] mm/damon: extend DAMOS filters for inclusion

[SeongJae Park]

On Tue, 7 Jan 2025 12:17:29 -0800 SeongJae Park <sj@kernel.org> wrote:

[...]
> Extend DAMOS filters to support not only exclusion (blocking), but also
> inclusion (pass-through) behavior.
[...]
> This means that installing pass filters without block filters
> after them is not useful for some usecases.

The above comment is wrong.  The case is just useless.

> Read the design document
> update part of this patch series for more details.

I added why it is wrong in detail as a reply to the design doc patch (7/10).

I will fix the wrong parts and send this series again as v2.


Thanks,
SJ

[...]

Re: [PATCH 07/10] Docs/mm/damon/design: document pass/block filters behaviors

[SeongJae Park]

On Tue, 7 Jan 2025 20:04:54 -0800 SeongJae Park <sj@kernel.org> wrote:

> On Tue, 7 Jan 2025 12:17:36 -0800 SeongJae Park <sj@kernel.org> wrote:
> 
[...]
> > +The fact that the action can be applied to any memory as long as no block
> > +filter explicitly excluded it means that installing pass filters without any
> > +block filter after those is same to not installing the pass filters, in terms
> > +of the ``action`` applying.  Statistics for DAMOS filters will still be
> > +accounted, though.
> 
> The above last sentence is right in a sense, but not useful and could only
> confuse readers.  The statistics for DAMOS filters are filters passed size
> stat, which is provided as per-scheme accumulated stat and per-region instant
> information.  The stat is for any memory that be able to apply the DAMOS action
> after the filters check stage.  Hence, whether it has passed the stage due to
> existence of a pass filter that matches the memory, or the absence of any
> matching filter is distinguishable.
> 
> > It is therefore still useful for monitoring purpose.
> 
> Hence, the above sentence is completely wrong.  The case (installing pass
> filtrs without any block filter after those) is not useful even for monitoring
> purpose.
> 
> The RFC version of this patch was mentioning it correctly, but was not clearly
> describing why it is not also useless for even monitoring purpose.  I was also
> confused due to the absence of the context.  I will rewrite this part and send
> the whole series again as v2.

Another reason that I was confused is the conflict of the term.  The term
'pass' at sz_filters_passed stat means passing through the filters checking
stage.  The same term 'pass' at "DAMOS pass filter" means it will let the
memory that matches to its criteria pass the stage.  Hence, the term for
'sz_filters_passed' is a superset of it for 'DAMOS pass filter'.  This is
obviously confusing.

I will rename things that introduced by this series to be called "allow"
instead of "pass" in v2.


Thanks,
SJ

> 
> 
> Thanks,
> SJ
> 
> [...]